‘Cortex’: An open source platform for deploying machine learning models as production web services

If you are looking for a tool to deploy machine learning models as production web services, then ‘Cortex’ could be a good option to try. This open-source platform is an alternative to serving models with AWS SageMaker or creating your own model deployment platform over AWS services like Elastic Container Service (ECS), Elastic Kubernetes Service (EKS), and Elastic Compute Cloud (EC2) and even open-source projects like Docker, Kubernetes, and TensorFlow.

Key Features

  • Multi framework: Cortex supports multiple frameworks, including TensorFlow, PyTorch, scikit-learn, XGBoost, and more.
  • Autoscaling: Cortex automatically does load balancing for API’s to handle production workloads.
  • Infrastructure: Cortex can run inference both on CPU or GPU infrastructure.
  • Rolling updates: Cortex updates APIs post-deployment without any downtime.

Examples of Cortex deployments


Prerequisites (Copied from https://www.cortex.dev/install)

  1. Docker
  2. AWS credentials
# install the CLI on your machine
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.11/get-cli.sh)"

# provision infrastructure on AWS and spin up a cluster
cortex cluster up

Deploy a model

# clone the Cortex repository
git clone -b 0.11 https://github.com/cortexlabs/cortex.git

# navigate to the iris classifier example
cd cortex/examples/sklearn/iris-classifier

# deploy the model to the cluster
cortex deploy

# view the status of the deployment
cortex get --watch

# get the API's endpoint
cortex get classifier

# classify a sample
curl -X POST -H "Content-Type: application/json" \
  -d '{ "sepal_length": 5.2, "sepal_width": 3.6, "petal_length": 1.4, "petal_width": 0.3 }' \
  <API endpoint>

Github: https://github.com/cortexlabs/cortex

Website: https://www.cortex.dev

Tutorial: https://www.cortex.dev/iris-classifier

Examples: https://github.com/cortexlabs/cortex/tree/0.11/examples

Asif Razzaq is an AI Journalist and Cofounder of Marktechpost, LLC. He is a visionary, entrepreneur and engineer who aspires to use the power of Artificial Intelligence for good.

Asif's latest venture is the development of an Artificial Intelligence Media Platform (Marktechpost) that will revolutionize how people can find relevant news related to Artificial Intelligence, Data Science and Machine Learning.

Asif was featured by Onalytica in it’s ‘Who’s Who in AI? (Influential Voices & Brands)’ as one of the 'Influential Journalists in AI' (https://onalytica.com/wp-content/uploads/2021/09/Whos-Who-In-AI.pdf). His interview was also featured by Onalytica (https://onalytica.com/blog/posts/interview-with-asif-razzaq/).