In today’s dynamic world of AI and ML, Machine Learning Engineers need to experiment, train, retrain, and infer ML Models rapidly. Rapid deployment of models for A/B testing, Shadow testing, and Production testing needs a streamlined process to accelerate the experimentation and production time.
As per An Introduction to Machine Learning Model Deployment, the average time to deploy an ML model in production is 30-90 days. This is a very costly affair without a proper ops mechanism in place and can hamper the pace of innovation.
ML Lifecycle
A Machine Learning lifecycle starts with defining the business goal, leading to ML problem formulation, and ends with inferencing. Not all problems can be solved using ML, and it is very important to understand the business need and its solution using ML.
Once an ML problem is identified, it leads to the actual ML development process, which starts with data collection and completes with inferencing. The process includes various stages like data gathering, data cleaning, data labeling, data pre-processing, data post-processing, Model training, Model validation, Model testing, Model deployment, Model monitoring, and inferencing for end users.
Challenges
Each stage is as important as the next stage and as crucial as the previous stage. There are various engineering and ML challenges associated with each stage. These can be divided into the legal aspect, the machine learning aspect, and the engineering aspect.
Data collection, filtering, processing, etc., can have legal challenges. These challenges can be due to various reasons, such as copyright, PII, PHI, etc., and can require an automated solution or manual intervention.
Next set of challenges are machine learning which can be building the models, training, retraining, fine tuning, validating etc. to ensure proper development of required models for the intended business case.
Finally, there are engineering challenges. These challenges are managing the model deployment, managing experimentation with models, monitoring, testing, inferencing, automation etc. This will require engineering skillset to identify the pain point and build solution around it.
The end-to-end process is tedious and requires a lot of management at each stage. Each stage adds certain overhead in the overall process, which could sum up to multiple days of delay. Another challenge in the overall process is lack of skill set. Most of the work performed during an ML Lifecycle is either by Machine Learning Engineers or Machine Learning Scientists. They are good at data and ML processes but lack engineering skills.
This leads to poor selection of tools, libraries, and methodologies in developing and deploying the models. Often, many manual processes are followed in each of the stages, which could lead to error or delay in overall model deployment.
When we discuss about Machine Learning, we cannot focus only on the Machine Learning Engineers or Machine Learning Scientists. ML Models are not only about developing a world class function to do the desired work, but also ensuring how fast it can be deployed, experimented, retrained, and tested.
MLOps
Machine Learning Operations, or MLOps, is a process of various methodologies to streamline the model lifecycle, which can include training, retraining, fine-tuning, deployment, monitoring, etc. MLOPS ensures a faster deployment time and turnaround/rollback in case of any issues detected.
This also provides flexibility for experimentation without impacting productivity. Unlike DevOps, MLOps is focused on machine learning best practices. This provides a set of tools and automation to the corresponding audience, which allows them to focus more on core model development rather than worrying about processes at various stages.
MLOps also empower Machine Learning Engineers and Scientists to implement various control mechanisms around their release processes, data, and experimentation. This ensures a faster development and deployment time of an ML model.
Stages of MLOps
As per, MLOps: Continuous delivery and automation pipelines in machine learning, there are three stages of MLOps.
- MLOps 0 (Manual): This is sufficient for the scenario where a model is rarely changed, retrained, or fine-tuned. The process is manual and controlled by scripts. However, this will become challenging as soon as model deployment becomes frequent due to changes in business requirements or data.
- MLOps 1 (ML Pipeline Automation): This allows for rapid experimentation by automating the ML pipeline. This encompasses automated data validation, feature stores, metadata management, and ML pipeline triggers.
- MLOps 2 (CI/CD Pipeline Automation): In case of a rapid development environment where training/re-training is required frequently (daily) and inference on a very large scale, this level of automation is required. This level of automation ensures end-to-end model delivery, management, control, and monitoring with various triggers in place.
Use Case
Let’s assume a chatbot scenario for a large e-commerce company. The volume of data and inquiries they get daily is huge. The data changes frequently and leads to new opportunities for the bot to learn and revert accordingly. Training a model on a snapshot of data could lead to a stale model after some days, and this required frequent training.
To cope with such a huge demand for retraining, fine-tuning, experimentation, testing, and monitoring, MLOps is required. MLOps will reduce the burden and increase the pace of innovation by a huge margin.
Disclaimer: The author is completely responsible for the content of this article. The opinions expressed are their own and do not represent IEEE’s position nor that of the Computer Society nor its Leadership.
