‘Cortex’: An open source platform for deploying machine learning models as production web services

If you are looking for a tool to deploy machine learning models as production web services, then ‘Cortex’ could be a good option to try. This open-source platform is an alternative to serving models with AWS SageMaker or creating your own model deployment platform over AWS services like Elastic Container Service (ECS), Elastic Kubernetes Service (EKS), and Elastic Compute Cloud (EC2) and even open-source projects like Docker, Kubernetes, and TensorFlow.

Key Features

  • Multi framework: Cortex supports multiple frameworks, including TensorFlow, PyTorch, scikit-learn, XGBoost, and more.
  • Autoscaling: Cortex automatically does load balancing for API’s to handle production workloads.
  • Infrastructure: Cortex can run inference both on CPU or GPU infrastructure.
  • Rolling updates: Cortex updates APIs post-deployment without any downtime.

Examples of Cortex deployments

Install

Prerequisites (Copied from https://www.cortex.dev/install)

  1. Docker
  2. AWS credentials
# install the CLI on your machine
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.11/get-cli.sh)"

# provision infrastructure on AWS and spin up a cluster
cortex cluster up

Deploy a model

# clone the Cortex repository
git clone -b 0.11 https://github.com/cortexlabs/cortex.git

# navigate to the iris classifier example
cd cortex/examples/sklearn/iris-classifier

# deploy the model to the cluster
cortex deploy

# view the status of the deployment
cortex get --watch

# get the API's endpoint
cortex get classifier

# classify a sample
curl -X POST -H "Content-Type: application/json" \
  -d '{ "sepal_length": 5.2, "sepal_width": 3.6, "petal_length": 1.4, "petal_width": 0.3 }' \
  <API endpoint>

Github: https://github.com/cortexlabs/cortex

Website: https://www.cortex.dev

Tutorial: https://www.cortex.dev/iris-classifier

Examples: https://github.com/cortexlabs/cortex/tree/0.11/examples

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.

✅ [Featured Tool] Check out Taipy Enterprise Edition