If you are looking for a tool to deploy machine learning models as production web services, then ‘Cortex’ could be a good option to try. This open-source platform is an alternative to serving models with AWS SageMaker or creating your own model deployment platform over AWS services like Elastic Container Service (ECS), Elastic Kubernetes Service (EKS), and Elastic Compute Cloud (EC2) and even open-source projects like Docker, Kubernetes, and TensorFlow.
Key Features
- Multi framework: Cortex supports multiple frameworks, including TensorFlow, PyTorch, scikit-learn, XGBoost, and more.
- Autoscaling: Cortex automatically does load balancing for API’s to handle production workloads.
- Infrastructure: Cortex can run inference both on CPU or GPU infrastructure.
- Rolling updates: Cortex updates APIs post-deployment without any downtime.
Examples of Cortex deployments
- Sentiment analysis: Deploy a BERT model for sentiment analysis using Cortex
- Image classification: Deploy an Inception model to classify images using Cortex
Install
Prerequisites (Copied from https://www.cortex.dev/install)
# install the CLI on your machine
bash -c "$(curl -sS https://raw.githubusercontent.com/cortexlabs/cortex/0.11/get-cli.sh)"
# provision infrastructure on AWS and spin up a cluster
cortex cluster upDeploy a model
# clone the Cortex repository
git clone -b 0.11 https://github.com/cortexlabs/cortex.git
# navigate to the iris classifier example
cd cortex/examples/sklearn/iris-classifier
# deploy the model to the cluster
cortex deploy
# view the status of the deployment
cortex get --watch
# get the API's endpoint
cortex get classifier
# classify a sample
curl -X POST -H "Content-Type: application/json" \
-d '{ "sepal_length": 5.2, "sepal_width": 3.6, "petal_length": 1.4, "petal_width": 0.3 }' \
<API endpoint>Github: https://github.com/cortexlabs/cortex
Website: https://www.cortex.dev
Tutorial: https://www.cortex.dev/iris-classifier
Examples: https://github.com/cortexlabs/cortex/tree/0.11/examples
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
