MLOps Maturity Model: Five Levels and How to Assess Yours
An MLOps maturity model describes how reliably an organisation can take a machine-learning model from experiment to production and keep it healthy once it is there. It grades your capability in stages, from manual, undocumented work to automated training, deployment, monitoring and retraining. For Malaysian enterprises, especially in banking, insurance and capital markets, a sound assessment also has to cover governance, security and accountability, not only automation.
This guide explains the levels, shows how to score yourself honestly, and sets out what each step up requires in practice.
TL;DR
An MLOps maturity model measures people and culture, processes and structures, and technology, not tooling alone.
Microsoft’s widely used model has five levels, from Level 0 (no MLOps) to Level 4 (fully automated operations).
Assess yourself with a structured questionnaire, then share the scores between business, data science and IT teams.
In Malaysia, add governance dimensions drawn from MOSTI’s AI guidelines, MyCERT guidance and, where relevant, Bank Negara Malaysia and Securities Commission requirements.
Aim for the next level that your use cases justify, not the highest level on the chart.
Table of Contents
What an MLOps maturity model measures
Microsoft’s Azure Architecture Center defines its maturity model as a framework for the principles and practices of building and operating machine-learning environments. It looks at technical, process and cultural capability, and it exposes the gaps in how you implement them. The assessment is qualitative. It considers three areas: people and culture, processes and structures, and objects and technology.
That three-part view matters. Many teams assume maturity means buying a platform. In practice, a model can sit on excellent infrastructure and still fail because nobody owns it after launch, or because the data science team and the operations team never agreed who responds when accuracy drops. Maturity is the combination of tooling, discipline and accountability.
The same source suggests a second use. Maturity models help you estimate the scope of work, set realistic success criteria and identify deliverables for a new engagement. In other words, the model is a planning tool, not a league table.
The five MLOps maturity levels
Microsoft describes five levels of capability. They run from Level 0, where systems are nontransparent and manual, to Level 4, where operations are automated, monitored and approach zero downtime. Automation increases at each step across training, deployment, testing, monitoring and retraining.
Level | Name | What it looks like in practice |
0 | No MLOps | Models are built in notebooks or local environments and handed over by hand. Little is documented, and nobody can easily reproduce a result. |
1 | DevOps but no MLOps | Application code has build and release discipline, but the model itself is still trained and released manually. Metrics are thin. |
2 | Automated training | Training runs as a repeatable pipeline. Experiments are tracked and models are versioned and registered. |
3 | Automated model deployment | Releases are pipeline-driven with testing gates and approvals. A/B comparison and basic production monitoring are in place. |
4 | Full MLOps automated operations | Deployed models emit centralised metrics, and drift or regression signals can trigger retraining automatically. |
The descriptions of Level 0 and Level 4, and the progression of automation between them, come from the Microsoft model. The intermediate level descriptions above are our plain-language reading of how that progression typically looks inside an enterprise.
Level 0 and Level 1: manual and fragile
At Level 0 the organisation has models, but not a system around them. A data scientist trains something, a colleague deploys it by hand, and the knowledge of how it works lives in a few heads. The risk is rarely visible until a key person leaves or an auditor asks how a decision was produced.
Level 1 is deceptive. The software side looks modern, with source control and release pipelines, so leadership assumes the AI side is equally controlled. It usually is not. The model artefact is still created outside the pipeline, and a retrain is a project rather than a routine.
Level 2 and Level 3: repeatable, then reliable
Level 2 is where reproducibility appears. Training is scripted, data and code versions are linked to each model, and results can be compared. This is the first level at which you can answer the question “which model made this prediction, and how was it built?”
Level 3 adds controlled release. Models pass automated validation before they reach production, and the path from approval to deployment is consistent. For a bank or insurer, this is often the level at which model risk and compliance teams become comfortable, because every release leaves a trail.
Level 4: operations that respond on their own
At Level 4, deployed models emit centralised metrics, and drift or regression signals trigger automatic retraining. Few organisations need this for every model. A fraud-scoring model that sees shifting transaction behaviour weekly is a strong candidate. A quarterly forecasting model probably is not.
Other ways to describe the stages
You will see maturity described with different labels. Digital Catapult’s assessment, for example, moves from Manual through Repeatable and Reproducible to Automated and Optimised. It asks whether processes are manual, repeatable, standardised and monitored, continuously deployed and retrained, or optimised end to end with established feedback loops.
The vocabulary differs, but the logic is the same: manual work becomes repeatable, repeatable work becomes standardised, and standardised work becomes automated and self-correcting. The queries people type, such as “MLOps maturity levels”, usually point to this same ladder. Pick one model, use it consistently and do not mix scores from different scales.
How to assess your current MLOps maturity
The best assessments are short, structured and shared. Three approaches from credible bodies are worth borrowing from.
Use the three-area view. Microsoft’s guidance is to assess current capabilities and gaps across people and culture, processes and structures, and objects and technology, then use the levels to plan incremental progress.
Score by domain. Fraunhofer FOKUS recommends a hands-on self-assessment using defined maturity statements across seven domains, including data gathering and preparation, model training, deployment and monitoring. Scores should be consolidated and visualised so that business, data-science and IT teams can discuss them together.
Include business alignment. Digital Catapult’s assessment scores business alignment and AI strategy, data management and pipelines, the model lifecycle, infrastructure and automation, and documentation and explainability. The results show strengths, gaps and priorities for a roadmap.
A baseline you can check this week
The MLOps Community assessment proposes a useful minimum. Documentation, traceability and reproducibility, code quality, and monitoring and support should be treated as the baseline, and every statement in those sections should be answerable “yes” before a project is considered reliably production-ready.
A practical way to apply this is to pick one production model and ask four plain questions:
Can a new engineer find the training data version, code version and parameters behind the live model within an hour?
Can you rebuild the same model from those records?
Does someone receive an alert when performance degrades?
Is there a named owner who is accountable for the model after launch?
If any answer is “no”, you are likely at Level 0 or 1 for that model, whatever the rest of your platform looks like. Score each production model separately. Maturity is often uneven: a customer-facing chatbot may be well run while a legacy credit model is not.
What it takes to move up a level
The step from manual workflows to automated production depends on a consistent set of investments. Microsoft’s guidance points to version control, CI/CD pipelines, automated model validation and centralised performance monitoring. It also describes a shift from disparate manual tasks to managed infrastructure in which data pipelines, model registries and testing gates are integrated.
Here is how we would sequence the work for an enterprise team.
From Level 0 to Level 1 or 2. Start with records, not tools. Put code, data references and configuration under version control, and document each model’s purpose, inputs and owner. Then script training so that one command reproduces a result.
From Level 2 to Level 3. Add a model registry, automated validation tests and a release pipeline with an approval step. Decide in advance what “good enough to release” means in numbers, and make the pipeline enforce it. Our guide to MLOps practices for production teams on a 6 to 12 month rollout covers the sequencing in more detail.
From Level 3 to Level 4. Instrument production. Capture input drift, prediction drift and business outcomes in one place, define thresholds, and connect them to retraining with human sign-off where the risk demands it. Automation without guardrails simply moves errors faster.
At every step, resist the urge to skip. A team that automates retraining before it can reproduce a training run will automate confusion.
Malaysia: what to add to the model
Generic maturity models are written for a global audience. In Malaysia, governance and security expectations shape what “mature” means, and they differ by sector. This section is general guidance, not legal advice. Confirm obligations with your compliance team or counsel.
National AI governance
MOSTI’s National Guidelines on AI Governance and Ethics, issued in September 2024, say governance should assign proportionate obligations to each party in the AI value chain according to its role and control over data and context. For a maturity assessment, this translates into a governance dimension: who in your organisation, and which vendors, hold which responsibilities for each model. The government has also set up the National AI Office under MyDIGITAL Corporation to drive the national AI agenda and develop regulatory frameworks, so expect expectations to keep developing. Our overview of PDPA and AI rules in Malaysia tracks how this affects businesses.
Security and accountability
MyCERT’s guidance on deploying AI systems recommends defining security boundaries, stakeholder roles and accountability, and risk tolerance. It also calls for a developer threat model, secured training data sources, and protection of model weights and system interfaces. The message for maturity is clear: it is not only automation. A Level 4 pipeline that retrains from unverified data or leaves model artefacts loosely controlled is not mature in any useful sense.
Banks and insurers
For financial institutions, Bank Negara Malaysia’s Risk Management in Technology policy requires technology-risk governance proportionate to an institution’s size, complexity, technology use and third-party exposure. That includes risk ownership, risk measurement, controls, continuous monitoring, asset criticality classification, dependency management and incident procedures. Map your models to those headings. A model registry that records criticality and owner, and monitoring that feeds an incident process, give you evidence for both your MLOps assessment and your technology-risk reviews.
Capital-market entities
The Securities Commission’s guidelines require board oversight, an approved technology-risk framework and risk appetite, implementation metrics, regular review and effective internal controls. Your maturity scorecard can serve as one of those implementation metrics, provided it is reviewed on a schedule and reported upward.
Data governance
Whatever your sector, personal data in training sets and features brings data protection obligations into the pipeline. Our guide to AI data governance mapped to ISO/IEC 42001 and Malaysia’s PDP rules shows how to link lineage and access controls to your MLOps stack.
Choosing a realistic target level
The highest level is not the right goal for every model. Use three questions to set a target.
How quickly does the world the model describes change? Fast-moving behaviour, such as fraud patterns or demand in logistics, justifies drift-triggered retraining. Stable processes, such as classifying a fixed set of document types, may be well served at Level 3.
What is the cost of a wrong prediction? Underwriting, credit and compliance decisions call for stronger testing gates, audit trails and human review, which usually means Level 3 with strict governance before Level 4 automation.
How many models do you run? A single model can be managed with modest tooling. Dozens of models across business units need shared platforms, registries and standards, otherwise every team reinvents the same controls.
Once you have a target for each priority use case, convert the gap into a roadmap: a short list of capabilities, an owner for each, and a measure of success such as time from approval to deployment, or time to detect degraded performance.
Common mistakes when applying a maturity model
The first is treating the score as a trophy. A maturity model is useful when it informs investment, so tie each gap to a business risk or cost.
The second is assessing only the data science team. Operations, security, risk and the business owner all shape whether a model survives in production. Fraunhofer’s advice to discuss results across business, data-science and IT teams exists for this reason.
The third is ignoring handoffs. Every time a model passes from research to engineering to operations, context is lost and accountability blurs. This is where many Level 1 organisations stall.
Our perspective
Most maturity assessments we see are accurate about tooling and silent about ownership. The model gets a score, but nobody can say who answers for it six months after launch. We build and run AI systems so that one senior team owns strategy, engineering and operation. That removes the handoffs where models typically lose documentation, monitoring and accountability. We engineer for production from day one, with continuity, governance and auditability designed in rather than added when an auditor arrives. Our advice to enterprise leaders is to treat maturity as a business-risk question. Start with the models that carry the most operational cost or regulatory exposure, bring them to a dependable Level 3, and automate further only where the evidence supports it. Responsible AI and measured business results are not separate from MLOps; they are what mature MLOps is for. Thomas Samuel
Where Sentient Concepts can help
If your assessment shows gaps between Level 1 and Level 3, the work is usually a mix of diagnosis, engineering and ongoing operation. We start with Readiness & Data Diligence to establish where your models, data and controls stand today. We then build the pipelines, registries and monitoring through Deployment & MLOps, and can keep them healthy with Managed AI Operations.
To discuss your own maturity assessment, explore our full range of services or get in touch through sentientconcepts.com.
FAQ
What are the levels of MLOps maturity?
Microsoft’s model has five levels, from Level 0 (No MLOps) to Level 4 (Full MLOps automated operations). Automation of training, deployment, testing, monitoring and retraining increases at each step. Other frameworks, such as Digital Catapult’s, use labels like Manual, Repeatable, Reproducible, Automated and Optimised for the same progression.
How do I know which MLOps maturity level my organisation is at?
Run a structured self-assessment across people and culture, processes and structures, and technology, then score each production model separately. Share the results with business, data science and IT teams. Overall maturity is often uneven, so expect different models to land at different levels.
Do we need to reach Level 4?
Not necessarily. Level 4 suits models whose inputs change quickly or whose errors are costly to leave uncorrected. Many stable, document-centred use cases are well served by a controlled Level 3, with strong monitoring and clear ownership.
Does Malaysia have specific requirements for MLOps?
There is no single MLOps rule, but several sources shape expectations. These include MOSTI’s AI governance guidelines, MyCERT guidance on deploying AI systems, and sector rules from Bank Negara Malaysia and the Securities Commission. Check with your compliance team which apply to you.
Sources
Recommended