MLOps Team Structure: Roles, Models and How to Staff One
A workable MLOps team combines five capabilities: data engineering, data science, machine learning engineering, MLOps (platform and automation) engineering, and business and risk oversight. Small teams cover them with three or four people who share roles. Larger organisations separate them and add a platform team. The structure matters less than one rule: every stage from data to monitoring has a named owner, and no model is passed over a wall.
This guide sets out who does what, how the main organisational models differ, how to staff at each stage of maturity, and where team designs usually break down.
TL;DR
An MLOps team spans data engineering, data science, ML engineering, MLOps engineering and business or risk oversight. In small teams one person often holds several of these roles.
Official guidance from Singapore recommends a dedicated team of 3 or 4 members as the minimum to build, deploy and maintain a machine learning system.
Organise around end-to-end project teams first. Add a specialised platform team once deployment and operations work repeats across projects.
Regulated firms, such as banks and insurers, should name model risk, audit and security roles from the start, not after go-live.
Unclear roles cause delays, quality issues and governance gaps, so write ownership down for every stage of the lifecycle.
Table of Contents
What an MLOps team is actually responsible for
MLOps applies DevOps principles to machine learning. Microsoft’s guidance describes two loops. Data scientists work in the inner loop, training the model. Machine learning engineers and IT operations work in the outer loop, where they package, validate, deploy and monitor it. When a model needs fine-tuning or retraining, the work returns to the inner loop.
That loop is the reason a team structure is needed at all. A model that performs well in a notebook is only the first step. Someone must automate its pipeline, someone must watch it in production, and someone must decide when it is retrained or retired.
Google Cloud’s architecture guidance frames MLOps as automation and monitoring at every step of building an ML system: integration, testing, releasing, deployment and infrastructure management. ml-ops.org states the objective plainly: automate the complete ML workflow without manual intervention, with triggers such as monitoring events or changes in data, training code or application code.
A team structure, then, is a decision about who owns each part of that automated chain. If a stage has no owner, it is done manually, late, or not at all.
The core roles and what each one owns
No single source defines one universal list, but the main cloud frameworks agree on the shape. AWS’s Well-Architected Machine Learning Lens names domain experts, data engineers, data scientists, ML engineers, MLOps engineers, IT auditors, model risk managers, cloud security engineers and, for generative AI, prompt engineers. Google’s guide to assembling an ML team names an ML product manager, an engineering manager, a data scientist, an ML engineer, a data engineer and a DevOps engineer.
Combined, the responsibilities look like this.
Role | What the role owns |
Domain expert or ML product manager | The business problem, the requirements document and validation of the approach against real-world needs |
Data engineer | Productionised data pipelines with monitoring and alerting, turning raw data into formats suitable for training |
Data scientist | Statistical modelling and machine learning to derive insight, plus reports and visualisations |
ML engineer | Converting prototypes into production-ready software, and delivering a deployed model with sufficient prediction quality |
MLOps or DevOps engineer | Automation pipelines for training, testing and deployment, and the process for serving, monitoring, testing and alerting on model behaviour |
Engineering manager | Team priorities and performance evaluation |
Model risk manager, IT auditor, cloud security engineer | Control requirements, access review and secure cloud configuration |
The table is a map of responsibilities, not a headcount. One person may cover two or three rows.
Where data science ends and engineering begins
The most common boundary dispute is between the data scientist and the ML engineer. A practical split follows Microsoft’s inner and outer loops. The data scientist owns experimentation and model quality. The ML engineer owns everything needed to run that model repeatedly and reliably, including packaging, CI/CD and monitoring.
The Azure ML-Ops Accelerator describes ML engineers as working across pre-processing, model development and production, with a focus on productionising models, CI/CD, monitoring and the AI infrastructure needed to scale. That breadth is why they are usually the hardest hire in the team.
What the MLOps engineer adds
The MLOps engineer builds the platform the others use. Typical duties include pipelines for continuous integration of code, data and models, automated deployment of newly trained models, scheduled or triggered retraining, and monitoring. AWS describes these four activities as continuous integration, continuous delivery, continuous training and continuous monitoring, the last covering both data and model metrics tied to business outcomes.
Monitoring is where responsibilities are most often left vague. Decide who receives the alert, who investigates, and who can roll back. Our guides to data drift detection and model retraining strategy go deeper on the operational side.
Roles that finance, insurance and manufacturing add
For banks, insurers and other regulated firms, the technical roles are only half the team. AWS lists the IT auditor, model risk manager and cloud security engineer because models in these sectors must meet internal and external control requirements. AWS’s guidance also notes that governance responsibilities are often overlooked in early ML initiatives but become critical once projects reach production.
In practice that means involving the model risk function when the use case is chosen, not when the model is ready to deploy. Reviewers should be able to challenge a model’s assumptions, including the cost and data assumptions behind it. Teams building trading or pricing models, for example, need people who can stress-test inputs such as spread modelling for backtests before anything reaches production. Our article on model risk management sets out six lifecycle controls that map onto these roles.
Manufacturers and logistics firms usually need a different addition: process owners from operations who understand the documents, sensors or workflows the model touches. They play the domain expert role, and without them document automation and predictive maintenance projects tend to solve the wrong problem.
Generative AI adds more. If you run large language models on internal data, a prompt engineer, a security lead and a data protection owner become part of the structure. See our notes on LLMOps practices and running LLMs on private data.
Three ways to organise the team
There are three common models. Most organisations move through them in order.
Embedded teams
In an embedded model, ML engineers and data scientists sit inside a business unit, such as credit, claims or the plant floor. Domain knowledge is strong and priorities are clear. The cost is duplication: each unit builds its own pipelines, tooling and monitoring, and standards drift.
Centralised platform team
A central team owns the platform, tooling and standards, and serves several business units. Governance is easier and engineering effort is reused. The risk is distance from the problem, and a queue of requests that slows delivery.
Hybrid
The hybrid model combines both. Cross-functional project teams are embedded in the business and own their models end to end, while a small platform team provides shared pipelines, deployment patterns and controls. This is the model most mature organisations settle on.
The evidence points the same way. The ACM Computing Surveys multivocal review of MLOps reports that the AWS and Azure frameworks recommend project teams that combine domain experts, data engineers, data scientists, ML engineers, MLOps engineers, auditors and security engineers, rather than building a functional silo for each capability. It also notes that, at a certain maturity, a specialised ML engineering team focused on deployment, operations and maintenance becomes helpful, because those capabilities are common across projects and independent of the specific ML problem.
Staffing by maturity stage
Team size should follow where you are, not where you hope to be.
Starting out: three or four people
Singapore’s government MLOps Playbook recommends a dedicated team of 3 or 4 members. It adds that data scientists often take on ML engineer or data engineer duties in smaller teams, and that its listed roles are the minimum required to develop, deploy and maintain a machine learning system.
A sensible starting line-up is a product owner who knows the business, a data engineer, a data scientist who is comfortable shipping code, and an ML or MLOps engineer. Borrow security and risk review from existing functions, but name the individuals.
Growing: repeatable delivery
Once you have two or three models in production, the same deployment problems recur. This is the point to carve out platform responsibilities: a shared pipeline template, a model registry, standard monitoring. The Chalmers University study of MLOps adoption describes five maturity stages, from ad hoc through DataOps, manual MLOps and automated MLOps to a continuously improving stage. It finds that companies shift towards standardised processes and regular communication with stakeholders and teams as they mature.
If you want to place your organisation on that scale, our MLOps maturity model walks through five levels and how to assess them.
Mature: platform plus product teams
At scale, expect a platform team, several product teams, and a governance function that sets control standards and reviews high-risk models. Roles become specialised: separate data engineering, ML engineering and MLOps engineering. Even here, the cross-functional project team remains the unit that owns each model.
Working together without hand-offs
The most expensive pattern in machine learning delivery is the sequential hand-off: data science builds a model, throws it to engineering, which throws it to operations. Each transfer loses context, and each team optimises for its own stage.
Three habits prevent it.
Shared ownership of the outcome. The whole team is measured on the model’s production performance and business result, not on the delivery of a prototype. A product owner is useful here. The SkeltyMLOps research paper describes the product owner as a liaison across all actors, capturing requirements, aligning them with business goals and keeping communication open.
Production standards from the first sprint. Data scientists work in the same repositories, with the same testing and versioning, as engineers. This is what makes the outer loop automatable. Our six MLOps practices for production teams describe a 6 to 12 month rollout that builds this in.
Documented roles and escalation paths. Write down who approves a model for release, who is paged when drift is detected, and who can retire a model. Governance work is the item most often left unassigned.
Common failure points
The AWS guidance is blunt about the cost of ambiguity: without clearly defined roles, ML projects often face delays, quality issues or governance challenges. Four patterns account for most of the trouble.
The lone hero. One engineer holds the pipeline knowledge. When that person leaves, retraining stops. Pair the role with a deputy and keep pipelines in code.
Data engineering as an afterthought. Teams hire data scientists first and discover that data is the bottleneck. Data pipelines need their own owner. Our guide to the data pipeline for ML lists the stages and controls.
Governance bolted on. Risk and audit join at deployment and find gaps that require rework. Involve them at use-case selection.
A platform team with no customers. A central team that builds tooling nobody asked for wastes effort. Treat it as a product team, with the project teams as its users.
Our perspective
Most organisations we meet do not have a talent problem. They have an ownership problem. A strong data science team builds a promising model, a separate engineering group is asked to industrialise it, and a third group is asked to run it. By the time the model reaches production, nobody who designed it is still accountable for how it behaves. Our position is that one senior team should own strategy, build and operation. We engineer for production from day one, with continuity, governance, auditability and measured business results built into the design, because retrofitting them is slower and more expensive. That is why our work spans readiness and data diligence, platform engineering, deployment and MLOps, and managed operations under one accountable team. For enterprise leaders in finance, manufacturing, logistics and insurance, our advice on structure is practical. Start with a small cross-functional team and name every owner. Bring model risk and security in early. Move shared deployment work to a platform capability once it repeats. Where you lack the people or the time, a managed model lets you keep accountability clear without hiring every role at once. Thomas Samuel
Getting the structure right with Sentient Concepts
If you are deciding which roles to hire, which to borrow and which to run as a managed service, we can help. Our Deployment & MLOps service covers deployment, monitoring and operation of models in production, and our readiness and data diligence work shows where your current team and data stand before you commit to a structure.
Because one senior team owns strategy, build and operation, there is no hand-off between the people who design your system and the people who keep it running. For a view of what is on offer, see our full list of services, or read how managed MLOps is scoped and bought.
FAQ
What roles are in an MLOps team?
The core roles are data engineer, data scientist, ML engineer and MLOps engineer, supported by a domain expert or product owner. Regulated organisations add a model risk manager, IT auditor and cloud security engineer, as set out in the AWS Well-Architected guidance. Generative AI projects may add a prompt engineer.
How big should an MLOps team be?
Singapore’s government MLOps playbook recommends a dedicated team of 3 or 4 members as the minimum for developing, deploying and maintaining an ML system. In small teams people hold several roles. Add specialists and a platform team as the number of models in production grows.
Should MLOps sit inside data science or IT?
Neither alone. Data science owns model quality, and IT or platform engineering owns reliable deployment and infrastructure. The better arrangement is a cross-functional project team that owns each model end to end, supported by a shared platform capability once the work repeats.
What does an MLOps engineer do day to day?
They build and maintain the pipelines for training, testing, deployment and monitoring, and the tooling that lets data scientists and ML engineers release models safely. They also handle automated retraining triggers and alerting when model or data metrics move outside agreed limits.
Sources
Microsoft Learn: Machine Learning Operations (MLOps) concepts
Google Cloud Architecture Center: MLOps continuous delivery and automation pipelines
AWS Well-Architected Machine Learning Lens: Establish ML roles and responsibilities
Azure ML-Ops Accelerator: Skills, Roles and Responsibilities
ACM Computing Surveys: A Multivocal Review of MLOps Practices
Singapore AIP MLOps Playbook: People and Knowledge Management
Recommended