Why Deploying a Machine Learning Model Is Only the Beginning

It is possible that a machine learning model does really well at testing yet makes poor predictions a few months down the line after deployment. For example, a customer churn model could be very good at predicting customers who are most likely to churn. 

With changes in customer behavior, changes in pricing, or other changes in the market, the quality of the predictions made by the model starts deteriorating over time. The model keeps executing its operations, the API looks good, and all the necessary infrastructure looks fine.

This is one of the most difficult problems organizations encounter when trying to implement artificial intelligence in their systems. Model monitoring helps companies tackle this problem.

Unlike conventional software, machine learning models rely on patterns derived from past data. The patterns may or may not stay applicable with changes in customer behavior, business environment, and market situation.

Hence, using a machine learning model in production is not the end of the life cycle of such models but the start of an ongoing process of evaluation, observation, and optimization.

This post will look at the reasons why production machine learning models need monitoring, what metrics teams need to measure, how to identify whether the model drift has occurred, and how to create a good monitoring strategy.

What Is Machine Learning Model Monitoring?

Model Monitoring in Machine Learning is the process of monitoring the inputs, outputs, and performance of a model once it has been put into operation to ensure that it meets its objectives.

It is an intersection of technical considerations and business considerations. The engineer wants to know if the model responds within time boundaries that are acceptable, whereas the data scientist would want to know if the predictions made by the model still remain valid and meaningful. 

The business person would also want to know if the predictions made by the model help in decision-making.

Model Monitoring vs. Traditional Application Monitoring

Traditional application monitoring includes infrastructure and systems monitoring. This means checking server availability, application exceptions, memory usage, network latency, and API response times.

Monitoring machine learning models also includes some extra data points that do not get captured with regular infrastructure monitoring.

A fraud detection API, for example, could be processing transactions without any issues while being unable to detect an increasing number of fraud attempts. Infrastructure monitoring might show that all is well, yet model monitoring will be able to tell about declining model recall once results of fraudulent transactions are known.

This is because:

• Application monitoring tells if the software is working properly.

• Model monitoring tells if the model and predictions are behaving as expected.

• Model observability allows for understanding model behavior by means of metrics, logs, traces, data properties, and other context information.

Both of these concepts work hand in hand. Production-grade AI needs observability into both the application and model it serves.

Where Model Monitoring Fits Into the MLOps Lifecycle

Machine learning processes, referred to as MLOps, are an application of engineering practices to machine learning.

A general lifecycle involves data gathering, feature engineering, training of the model, model evaluation, deployment, monitoring, and optimization.

Monitoring is the final part that completes the feedback loop between deployment and development. The insights from the production environment can help detect any issues, understand the changes in the environment, and make the decision whether to fix or retrain the model.

Without it, companies will be at risk of using the initial evaluation results of the model as the eternal guarantee of its quality.

Why Production Models Need Continuous Oversight

Model monitoring in production is important because it is extremely unlikely that the production environment will stay the same as that in which the model was developed.

There are several reasons why continuous monitoring is needed.

1. Real-World Data Evolves Over Time

Models are trained on historical data. In case when the data becomes different, some of the assumptions about the model can become wrong.

In case of an e-commerce recommendation engine, the model is trained on customer purchasing behavior in the holiday season. As the season ends, customers' preferences and behavior can become completely different.

Demand forecasts can face changing seasons, supply problems, and other factors.

Thus, monitoring will help track the changes in incoming data and understand whether there is any impact on the predictions.

2. Model Performance Can Degrade Without Obvious Failures

Deployment of a model does not necessarily involve crashing for it to be considered unreliable.

It can go on making predictions normally but with reduced accuracy. If the system does not compare the predictions against real-world events or any other warning signs, such a situation will be unnoticeable.

Continuous monitoring adds another dimension of awareness through the monitoring of changes in data, predictions, and performance metrics.

3. Inaccurate Predictions May Lead to Business Impacts

The impact of model deterioration depends on the particular use case.

A model for fraud detection may fail to identify any suspicious activities. A model for recommendation may show less appropriate products to the users. And finally, a model for demand prediction may lead to either overproduction or lack of production.

In all these situations, the decreasing effectiveness of the model can impact business results regardless of the presence of the software itself.

4. Trust and Governance Require Ongoing Evidence

Companies utilizing AI have to provide proof that their models behave in a reasonable way throughout time.

In the case of critical applications, this can involve monitoring the performance of their models among different customer segments, versioning models, investigating strange outputs, and so forth.

Ongoing monitoring is important for responsible AI in terms of increased visibility of the model's behavior and accountability.

In general, monitoring is more than a technical control. It plays an important role in ensuring the reliability of AI solutions after they are deployed..

Understanding Model Drift and Other Production Risks

One of the main motivations for monitoring ML models is the chance of changing production data and/or relationships.

Not all data changes point at some problem with the model performance. Knowledge of types of production risks can help to choose right monitoring signals.

Data Drift

When the distribution of data coming to your model changes relative to a reference dataset or a baseline – you have data drift.

Imagine a loan acceptance model which was trained mainly on applications of a certain customer segment. When a company starts working in other markets, the applications might have different income distributions, employment, financial history etc.

These changes can be detected through monitoring of feature distributions.

Some signs of the data drift can be:

• Change in numerical feature distributions.

• An increase in missing values.

• Appearance of new or unusual categorical values.

• Shift in distribution of model inputs.

• Changes in distribution of model outputs.

Data drift doesn't necessarily imply that the accuracy of your model dropped. It just means that something should be investigated.

Concept Drift

Concept drift refers to situations whereby the association between the inputs of a model and the prediction becomes different.

Fraudsters, for instance, keep on changing their approaches in order to avoid being detected. Input factors that were once useful for detecting fraud may turn out to be unhelpful and others may arise.

The fraud detection model will become ineffective despite the fact that the distribution of input data is still the same as the training dataset.

Determining concept drift usually involves having access to the true values, including known fraud cases, as opposed to input data distribution alone.

Training-Serving Skew

Training-serving skew results when the data or transformations applied to the data for use in production are different from the data or transformations applied to the data for training or evaluation of the model.

If a model was trained on a normalized income field but receives raw income data from the production pipeline, the model may be receiving technically valid inputs but creating invalid predictions due to differences in feature representations.

Monitoring feature schemas, preprocessing procedures, and feature distributions may assist in detecting these issues.

Data Quality Issues

Other factors which may affect the effectiveness of a model include missing data, duplicate data, corrupt data inputs, wrong data types, and stale data.

These issues may come about as a result of data stored in upstream databases, from data pipelines, third-party integrations, and application changes.

An upstream system change may lead to replacing any missing numerical data with zeros. The model will still get valid numerical inputs, but the numbers may carry a different meaning than those seen during the training phase.

Changing Business and Environmental Conditions

There are circumstances where the models may face situations which were inadequately considered during their training.

These may range from economic developments, regulation changes, introduction of new products, customer behavior and even seasons. The goal of monitoring is not to stop any change at all.

It is to assess whether the model is behaving appropriately in the changing environment.

Key Metrics to Monitor in Production Machine Learning Models

A proper monitoring of a machine learning model needs to use multiple types of metrics. The appropriate choice of such metrics is based on the goals of using the model, as well as the type of data at hand, and the negative consequences of making wrong predictions.

Predictive Performance Metrics

In case the actual results are known, it is important to check how the predictions of the model match those results.

For classification models, relevant metrics are as follows:

• Precision: the proportion of positive predictions that are correct.

• Recall: the proportion of positive cases identified by the model.

• F1-score: combined metric for precision and recall.

• Accuracy: the proportion of correct predictions.

• ROC-AUC.

The decision depends on the use case. In cases of fraud detection, recall may be particularly critical due to the high costs of missing fraud. Precision is equally important as an excessively high number of false alarms will generate unnecessary investigations.

In case of regressions, common KPIs used by teams include mean absolute error (MAE), mean squared error (MSE), and root mean squared error (RMSE).

Each of these metrics evaluates prediction error differently. The decision should be made based on the problem at hand and not on choosing any metric interchangeably.

Not all outcomes become available immediately. For instance, in a demand forecasting use case, it may take some time until enough sales history becomes available to evaluate the predictions made by the model.

Data Quality and Drift Metrics

Monitoring of data quality can help identify missing value frequencies, schema inconsistencies, out-of-category values, invalid values, and freshness of features.

The approach of drift detection is comparing production data with a proper baseline distribution. Based on the nature of the data and the purpose of monitoring, approaches like Population Stability Index, Jensen-Shannon divergence, Kolmogorov-Smirnov test, and many more can be used.

There are various assumptions made by each of the above-mentioned statistical methods which must be considered when choosing a method for monitoring the data.

Most importantly, thresholds for drift detection need to be set according to the use case as sometimes a change that is statistically significant does not matter operationally.

Operational Performance Metrics

Models can be correct but still too slow and costly to work properly.

Monitoring must thus include:

• Prediction latency and throughput.

• Inference error rate and request failures.

• Usage of CPU, memory, and accelerators.

• Data freshness and pipeline latencies.

• Costs and resources required for inference.

• Model version and status.

Such metrics allow one to differentiate problems with models from problems with inference and infrastructure.

Business and Responsible AI Metrics

Technical metrics have to reflect outcomes the model aims to achieve.

Recommendation models could be measured by engagement and conversion metrics. Forecasting models can be evaluated based on inventory results and prediction errors. Also, models for decision-making can require assessments of performance for different subgroups.

Aggregated performance can mask significant differences within subgroups. Companies need to choose appropriate fairness and subgroup metrics based on intended application, harms and requirements.

The basic idea is clear – there is no metric that alone could confirm the healthy state of the model. Good monitoring consists of data quality, prediction performance, inference quality and business results.

How to Build a Continuous Model Monitoring Workflow

Step 1: Capture Baselines Prior to Deployment

Capture baselines for model evaluation metrics, distribution of relevant features, desired model behavior, and range of acceptable performance.

Employ representative training or validation sets as an appropriate baseline, depending on your monitoring goals.

Capture the version of the model, feature definitions, processing logic, and any assumptions for the baseline. Otherwise, without this information, teams will find it difficult to interpret production changes.

Step 2: Define Metrics, Thresholds, and Monitoring Windows

Select metrics that are relevant to actual risks.

In the case of forecasting, there might be a need for constant monitoring of stale data and shifting forecasting errors. In the case of fraud detection, more emphasis should be put on the recall, false positives and shifting transactions’ characteristics.

Thresholds have to take into account baseline variability, amount of data, business tolerance, and consequences of not detecting any changes.

Windows size is equally important. A short window allows detecting shifts quickly but tends to generate a lot of noise in case of small traffic. Long windows provide more data points but delay detection.

Step 3: Automate Dashboards and Alerts

The function of dashboards should be to help the team understand the present condition of the model and the changes in its behavior.

Alerts should detect material changes in the model’s behavior, for example, an increase in missing values, shift in a critical characteristic or drop in an outcome-based performance metric.

Alerts have to be integrated into the existing response process. There is no point in creating an alert that is not being handled by anybody.

Step 4: Investigate Anomalies and Find the Root Cause

An investigation must be carried out prior to any alteration of the model if the alarm generated during monitoring surpasses the threshold level.

Check if the alert is really a matter of concern, is due to some temporary fluctuation, lack of sufficient sample size, or the monitoring pipeline itself.

Comparison needs to be done between recent production data and the reference baseline.

Upstream data sources, feature transformations, model versions, and the labels available should be checked.

Step 5: Correct, Retrain, or Roll Back When Necessary

It will depend on the cause of the drift.

Broken data pipelines need repair. Mismatched pre-processing can lead to corrections of feature transformations. A degraded model may simply need re-training with better data.

At times, the best solution would be rolling back to the previous validated model or temporarily restricting automated decisions until better solutions are found.

Drift detection does not mean that retraining is automatically done without any prior investigation into the drift detected.

Retraining with bad data can do more harm than good.

Step 6: Validating the Response

Validation of the response to the problem and documentation of the result is very important after taking the corrective measures to solve the drift detected.

Ensure that the issue causing the drift is solved, but no other issues have emerged from this solution.

Machine Learning Model Monitoring Tools and Technologies

Cloud-Native Monitoring Platforms

Using managed services can make easier production data gathering, job scheduling, selected metrics evaluation, and alerting configuration.

Some examples are model monitoring in Google Cloud and monitoring functionality in Microsoft Azure Machine Learning. Signals that they have and configuration options differ per service and model type.

It may be an appealing option for teams working in corresponding clouds.

ML Monitoring and Observability Tools

There are monitoring tools which can offer such functionalities as data quality check, drift detection, model evaluation, dashboarding, and investigation process.

Such monitoring platforms can be helpful for organizations which require monitoring of multiple models or in multiple deployment environments.

Custom Monitoring Stacks

Using open-source libraries and custom monitoring pipeline gives the opportunity to customize everything according to organization needs.

A way of performing data validation, statistical analysis, application metrics, logging, and score calculation can be achieved.

How to Choose the Right Monitoring Stack

Evaluation should be based on:

• Integration into existing pipeline of data and models serving.

• Ability to work with your metrics and model types.

• Functionality regarding alerting, visualizations, and investigation.

• Scalability, pricing, and operational costs.

• Privacy and security.

A system with numerous functionalities may not necessarily be the best option. The best system is the one that will enable you to detect issues and respond to them.

Common Challenges in Production Model Monitoring

Even an effectively designed monitoring plan may face some practical challenges.

Delayed or Incomplete Ground Truth

Some results require time to manifest. Fraud is detected days after the transaction occurred, and customer churn is visible only after a while.

During the period, teams can monitor the data quality, input drift, prediction behavior, and certain proxy signals. Proxy signals should never be treated as alternatives to real predictive performance.

Alerts Overload and False Alarms

The generation of too many alerts will happen in the case of a poorly set threshold, which makes it challenging to determine the important alerts.

For the avoidance of the generation of alerts that cannot be handled, validation of the thresholds based on historical data is required.

Monitoring Costs and Big Data

Gathering all the features and predictions forever may incur huge expenses on storage and processing.

Risk based monitoring, aggregation, selective logging of features, and use of the right kind of sampling will lower costs. No matter what type of sampling strategy you select, keep in mind that it should have sufficient data to detect the errors.

Complex Pipelines and Lack of Ownership

Sometimes production models might need more than one database, transformation service, API, and infrastructure component. Lack of correct ownership makes the investigation process for incidents difficult and disorganized.

Ownership Decision Must Be Made Regarding Monitoring, Escalation, Artifacts Version Control, and Incidents Management.

Considerations of Privacy and Security

Data collected through monitoring may consist of private and corporate information.

An organization must collect only the necessary information, and there must be access control and proper retention/deletion policies for monitoring information.

The process of monitoring must be technically and operationally feasible.

Best Practices for Reliable Model Monitoring in Production

There are several principles that organizations can adopt in order to enhance their monitoring programs.

Monitor Right from the Start

Plan metrics, baseline, alerts, and incident response strategy before putting the algorithm into operation. Retrospective introduction of monitoring will definitely lead to gaps in the production log.

Monitor the Whole Pipeline

Collect metrics for data intake, transformations, inference, integration, and business results. An issue traced back to the model may turn out to be actually due to some earlier process.

Risk-Driven Approach

Rank models according to their business importance and the costs of potential failure. The model used for recommendation will have a different reaction time compared to the model used to identify possible frauds or decision-making.

Human-in-the-loop

In case of essential use cases, consider those instances when human intervention will be required and who is entitled for that.

Automated Actions

They have to be safeguarded with appropriate safeguards and rollbacks.

Integration of Monitoring Within MLOps Feedback Loop

Integrate the monitoring findings with the model registry, the retraining process, the testing process, and deployment.

The new model needs to be validated before deployment. The decision regarding retraining and deployment should be made on the basis of the findings and not on drift detection alone.

Periodically Review Monitoring Effectiveness

Check if the alerts trigger significant events, if the threshold settings are still relevant, and if it is possible to investigate any problem in an effective manner.

Monitoring also needs maintenance as well.

Real-World Applications of Continuous Model Monitoring

Each industry will have its own strategy for monitoring but the goal stays the same – to keep predictions useful and reliable once implemented.

Financial Services and Fraud Detection

Transaction pattern change monitoring, change in fraud detection rate, false positive, and verified fraud results could be monitored.

Because fraud methods constantly evolve, the monitoring process could help the team understand if their current approach to detection needs some review and optimization.

Retail and E-Commerce

Retail companies may monitor recommendations relevancy, engagement, conversion results, and changes in buying patterns.

A recommendation algorithm which works well in one season could be reviewed when conditions like products' availability, clients' preference, and buying behavior have changed.

Healthcare and Life Sciences

In healthcare applications monitoring may be required for data quality, changes in patients' population, subgroup performance, and governance requirements.

The monitoring should be complemented by proper validation and clinical oversight. Change in distribution of data itself doesn't mean that the model becomes clinically useless and dangerous.

Manufacturing and Predictive Maintenance

It will be possible for manufacturers to monitor the performance of the sensors, the state of the machinery, the accuracy of predictions, and the results of maintenance.

Monitoring could provide an indication of changes resulting from equipment deterioration, sensor changes, or altered production processes to determine if the model is still suitable for its operating conditions.

In all these scenarios, monitoring should be done according to the risks and outcomes of the particular model.

Frequently Asked Questions About Machine Learning Model Monitoring

What Is Model Monitoring for Machine Learning Models?

Model monitoring for machine learning models involves continuous analysis of a deployed machine learning model to see if there is a change in the data, predictions, performance, and operations. This is to help determine if the machine learning model is still doing what it is meant to do.

Why Is Model Monitoring Necessary in Production Environments?

Production environments keep changing over time, and the results obtained from the testing of a model cannot guarantee long-term performance. This is the reason why monitoring is important to identify any problems in data quality, data drift, operations, and deterioration of prediction quality in advance.

How Often Should Machine Learning Models Be Monitored?

The amount of data, rate of changes, risk of the model, and reaction time determine monitoring frequency. Some machine learning models need to be monitored frequently or even in real time; others can be monitored periodically at a certain interval.

Does Model Drift Necessarily Mean the Model Should Be Retrained?

No. The occurrence of model drift means that there is a difference between the current production data and the reference distribution; this does not mean that the quality of predictions has deteriorated.

Keep Production Models Reliable Through Continuous Oversight

Deploying a machine learning model is just the start of the life cycle of the model and not the end because the changes in data, customer behavior, and environment in which it operates need to be continuously monitored for the relevance and reliability of the machine learning model.

Some of the components that form part of a good machine learning model monitoring system include the following:

When an organization has invested in deploying machine learning algorithms in their production environment, monitoring is an important part of their MLOps lifecycle.

Gigmint.AI - Learn more about how we at Gigmint help organizations build reliable AI systems through production-ready AI engineering & deployment.