Packaging ML models into containerised Apis using Chassis

BLOG

Packaging ML models into containerised Apis using Chassis

In this guide, we will transform a pre-trained YOLO (You Only Look Once) object detection model into a ChassisModel object and then build a container for it.

Introduction

The primary purpose of this blog is to guide developers, data scientists, and machine learning enthusiasts through the process of transforming a pre-trained YOLO (You Only Look Once) object detection model into a Docker container using Chassis. By providing a step-by-step walkthrough, this blog aims to demystify the containerization process, ensuring that readers can easily deploy their YOLO models across various environments such as Docker, Modzy, Kubernetes, and KServe. The tutorial covers everything from setting up the environment, writing inference code, creating a ChassisModel object, to building and running the container. By the end of this guide, readers will have a functional Docker container ready for deployment and the knowledge to run inferences using this container, significantly simplifying the deployment and scalability of their machine learning models.

How it Works

Chassis is a tool designed to encapsulate Python models (such as PyTorch, TensorFlow, scikit-learn, etc.) within Docker containers. These containers are versatile, capable of running on Docker, Modzy, Kubernetes, and KServe platforms. As a user, your responsibilities include loading a pre-trained model, specifying a prediction function, outlining pip dependencies, configuring hardware and software requirements, and optionally providing model documentation. Once these specifications are defined, Chassis proceeds to construct a Docker image. This process involves selecting a suitable base image, integrating APIs, structuring layers, and finalizing the image creation. Consequently, the resultant container, generated by Chassis, encapsulates all essential components, ranging from code and dependencies to resources required for seamless model execution. This comprehensive package encompasses data pre-processing scripts, model definitions, weight files, post-processing routines, and potentially supplementary documentation like model cards.

Step 1: Setup

First, ensure you have a Python virtual environment set up and the Chassis SDK installed. If you haven’t followed the Quickstart Guide, use the following command to install the necessary dependencies:

pip install “chassisml[quickstart]”

Step 2: Writing the Inference Code

Create a new Python file or open an existing one, then paste the following inference code. This code takes an in-memory YOLO model object, constructs a custom predict function, and uses both to create a ChassisModel object.

Import Necessary Libraries

Load Model

Load a pre-trained YOLOv8 model.

Define Predict Function

Define the function to process input data and perform predictions.

Create Chassis Model Object

Create a ChassisModel object and add the required dependencies and metadata.

Test the Model

Before building the container, test the ChassisModel object.

Step 3: Build the Container

Step 4: Run the Script

Run the script to build the Docker image:

Step 5: Run the Container

To run the container, use the following command in your terminal:

You should see the message Serving on: 45000 indicating the container is running.

Step 6: Run Inference

Create a new Python file for running inference and paste the following code. This code uses Chassis’s OMIClient to load a sample piece of data and perform inference:

Inference Code


Input image

Execute this code to perform an inference against your running container. A successful run should yield a result similar to:

Output image


Advantages of Chassis for Automatic ML Model Containerization

  • Simplicity: Chassis eliminates the need for manual Dockerfile creation, streamlining the process for data scientists and developers to deploy models.
  • Flexibility: It supports various Python ML frameworks (scikit-learn, PyTorch, TensorFlow, etc.) and target architectures (cloud, on-premise, edge devices).
  • Standardization: Leveraging the Open Model Interface (OMI) standard, Chassis ensures consistent and interoperable containerized models.
  • Efficiency: Chassis automates containerization, saving time and resources compared to manual processes.
  • Portability: The resulting OMI-compliant containers can run seamlessly across different environments.

Disadvantages of Chassis for Automatic ML Model Containerization

  • Limited Framework Support: While it supports popular Python frameworks, Chassis might not work for models built with other languages or frameworks.
  • Customization Limitations: Chassis focuses on automatic configuration, offering less control over the containerization process compared to manual Dockerfile creation.
  • Potential for Errors: Relying on automation can introduce potential issues if the model or dependencies are not properly configured.

Conclusion

In short, Chassis is your secret weapon for effortlessly containerizing ML models as prediction APIs. If you prioritize ease of use and deployment flexibility (especially for edge devices), Chassis is an excellent choice. Dive deeper into the world of Chassis, explore their documentation here, and join the community – happy containerizing!

Sudhin M

Sudhin M, Software engineer with a strong foundation in building robust, scalable web applications and APIs. Proficient in designing and implementing RESTful services using frameworks like Django and Flask. Skilled in database management with SQL and NoSQL databases such as PostgreSQL, MySQL, and MongoDB. Experienced in optimizing application performance and scalability through caching, asynchronous processing, and microservices architecture.