Machine learning engineer after foundation models
A machine learning engineer owns a model: trains it, serves it, and keeps it correct as the world it was trained on moves. That ownership is the whole distinction from the roles that have grown up beside it. When general-purpose models became callable over an API, a large share of problems that once needed a custom model stopped needing one, and the work of building the system around a borrowed model became a separate job. What remains for this role is the part general models do not cover well: narrow tasks at high volume where cost and latency rule out a large model, tabular prediction, ranking, forecasting, and anything where the data cannot leave the building. That is a smaller field than it was and a more clearly defined one, and its central difficulty has not changed at all. It is drift.
Machine learning engineer, in short
Established role| In one sentence | Trains, tunes and serves models, and owns the pipeline that keeps them fed and fresh. |
|---|---|
| Judged on | Model quality against a held-out set, and whether the training and serving pipeline holds under real volume. |
| Fails when | The training distribution drifts away from production and nobody is watching the gap. |
| Most confused with | AI engineer, which increasingly does not train anything at all. |
Hired as a distinct position across many companies, with a recognisable scope. No compensation figures: see methodology.
What general models took, and what they left
Text classification, entity extraction, summarisation and most natural-language tasks used to justify a training project. They rarely do now. A general model with a well-written prompt and a few examples beats what a small team could train in a quarter, and it ships in an afternoon.
What they did not take is anything where the economics bite. A model scoring ten million rows a day is a different proposition from one answering a thousand questions: at that volume the difference between a small trained model and an API call is not a rounding error, it is whether the product has a margin.
They also did not take anything structured. Forecasting demand, ranking a catalogue, predicting churn from tabular data: gradient-boosted trees remain the right answer to a remarkable number of real business problems, and the fashionable technology is not competitive on them.
And they did not take anything the data cannot leave. Where regulation or contract keeps data inside an organisation, training something local is not a preference but the only available design, and this constraint appears far more often in regulated industries than public discussion of the field suggests.
Drift, which is what the job is really about
A trained model encodes the world as it was in the training data. The world moves. Nothing in the system announces this: the model keeps returning predictions, the service keeps responding, and the predictions get quietly worse.
There are two kinds and they need different instruments. Input drift is when what arrives stops resembling what was trained on, and it is detectable by watching the distribution of inputs without needing any labels. Concept drift is when the relationship itself changes, so the same input should now produce a different answer, and it is detectable only against fresh ground truth.
The practical consequence shapes the whole role: a machine learning system is not finished at deployment. It needs a labelling path that keeps producing fresh truth, monitoring that compares against it, and a retraining process somebody owns. Teams that skip this have not saved effort, they have deferred it into an incident.
The pipeline is the product
Most of the engineering in this job is not modelling. It is the path data takes from where it is generated to where the model sees it, and back out to where a prediction is used.
The failure that costs the most is training and serving skew: a feature computed one way in the training pipeline and slightly differently at serving time. The model was trained on something that does not exist in production. This produces a system that performs beautifully in evaluation and mediocrely in life, and it is invisible unless you deliberately check the same input through both paths.
The discipline that prevents it is unglamorous: compute features once, in one place, used by both paths. Teams rediscover this every few years under a new name, and the reason it keeps being rediscovered is that the shortcut is always available and always seems harmless.
Where the role is going
Two directions, and they lead to different careers. One is deeper into the systems that serve models at scale, which is an infrastructure specialisation with no shortage of hard problems and a stable demand.
The other is outward, into the applied roles where the question is no longer whether the model is good but whether anything changed because of it. An engineer with real pipeline discipline who moves toward AI engineering or toward deployment work brings something those roles are frequently missing: an instinct for what happens to a system six months after launch.
The Stanford AI Index records AI skills now requested in 2.5 % of United States job postings, up 297 % over a decade. That growth is real and it is not evenly distributed across these titles. The safe position is not a title; it is being the person who can tell whether a system is still working.
