Building a basic model server
We have so far built a model inference pipeline that has all the code necessary to independently perform predictions from a pre-trained model. Here, we will work on building our first model server, which is essentially a machine that hosts the model inference pipeline, actively listens to any incoming input data via an interface, and outputs model predictions on any input data through the interface.
Writing a basic app using Flask
To develop our server, we will use a popular Python library – Flask [3]. Flask will enable us to build our model server in a few lines of code. A good example of how this library works is shown with the following code:
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello, World!'
if __name__ == '__main__':
app.run(host='localhost', port=8890)
Say we saved this Python script as example.py and ran it from the terminal...