‘Cortex’: An open source platform for deploying machine learning models as production web services

Image Source: https://github.com/cortexlabs/cortex

If you are looking for a tool to deploy machine learning models as production web services, then ‘Cortex’ could be a good option to try. This open-source platform is an alternative to serving models with AWS SageMaker or creating your own model deployment platform over AWS services like Elastic Container Service (ECS), Elastic Kubernetes Service (EKS), and Elastic Compute Cloud (EC2) and even open-source projects like Docker, Kubernetes, and TensorFlow.

Key Features

  • Multi framework: Cortex supports multiple frameworks, including TensorFlow, PyTorch, scikit-learn, XGBoost, and more.
  • Autoscaling: Cortex automatically does load balancing for API’s to handle production workloads.
  • Infrastructure: Cortex can run inference both on CPU or GPU infrastructure.
  • Rolling updates: Cortex updates APIs post-deployment without any downtime.

Examples of Cortex deployments


Prerequisites (Copied from https://www.cortex.dev/install)

  1. Docker
  2. AWS credentials
# install the CLI on your machine
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.11/get-cli.sh)"

# provision infrastructure on AWS and spin up a cluster
cortex cluster up

Deploy a model

# clone the Cortex repository
git clone -b 0.11 https://github.com/cortexlabs/cortex.git

# navigate to the iris classifier example
cd cortex/examples/sklearn/iris-classifier

# deploy the model to the cluster
cortex deploy

# view the status of the deployment
cortex get --watch

# get the API's endpoint
cortex get classifier

# classify a sample
curl -X POST -H "Content-Type: application/json" \
  -d '{ "sepal_length": 5.2, "sepal_width": 3.6, "petal_length": 1.4, "petal_width": 0.3 }' \
  <API endpoint>

Github: https://github.com/cortexlabs/cortex

Website: https://www.cortex.dev

Tutorial: https://www.cortex.dev/iris-classifier

Examples: https://github.com/cortexlabs/cortex/tree/0.11/examples