Retrieval or fine-tuning: the question answered backwards
Teams reach for fine-tuning when the system does not know something and for retrieval when the system does not behave as they want, and both instincts are inverted. Fine-tuning is unreliable at installing knowledge: training on a corpus of documents changes how a model writes far more dependably than it changes what the model can recall, which is why systems tuned to learn a policy manual produce confident answers in the right register with the wrong content. Retrieval is what puts facts in reach. Conversely, when a system knows the right things and presents them in the wrong shape, or drifts from a required format, that is behaviour, and behaviour is what tuning adjusts well. The practical consequence is that most projects considering fine-tuning have a retrieval problem, and the cheapest fix is the one they skipped because it did not feel serious enough.
What retrieval is for
Anything the system needs to know that is specific to your organisation, changes over time, or has to be citable. Policies, product data, prior cases, current prices, the contents of a record.
It has three properties that matter beyond correctness. The knowledge can be updated without touching the model, so a policy change is a document change. The answer can cite its source, which is a requirement rather than a nicety in any regulated setting. And the system can be prevented from seeing what a particular user may not see, because retrieval runs under permissions.
When a retrieval system gives wrong answers, the cause is usually upstream of the model. Chunking that splits a table from its header, an index that ranks by the wrong signal, or a query that does not resemble how the document is written. Fixing those is unglamorous and moves quality more than any other lever available.
What fine-tuning is for
Form and behaviour. A consistent output structure the surrounding system can parse. A house style that a long prompt was failing to enforce. A classification task with a fixed label set where a small tuned model is both more accurate and far cheaper than a large general one.
There is also a cost argument that is frequently the strongest one. A prompt that has grown to several pages is paid for on every request, forever. Tuning that behaviour into a model and shortening the prompt can change the economics of a high-volume system substantially, and that is a legitimate reason to train even when quality is already acceptable.
What it does not do is make a model reliably recall specific facts. Teams testing this on a small sample often see it appear to work, because the facts were already partially known or the questions were phrased like the training examples. At scale the recall is inconsistent and, worse, confident.
The third option, which is usually the answer
Fix the prompt and the retrieval. It resolves the majority of quality problems, costs days rather than weeks, and requires no additional infrastructure to maintain afterwards.
The diagnostic that settles the argument takes an afternoon. Take twenty failing cases and, for each, look at what the model was actually given. If the right information was in the context and the answer was still wrong, that is a reasoning or prompting problem. If the right information was not there, it is retrieval, and no amount of training fixes it.
In practice this exercise attributes most failures to retrieval, and teams are consistently surprised. The reason is that a bad answer looks like a model problem, and nobody looks at the context that produced it unless the system was built to make that easy.
Using both, which is the common end state
The two are not alternatives in a mature system. Retrieval supplies the facts and a tuned model supplies the form, and each is doing what it is good at. A system that classifies incoming documents with a small tuned model and then answers questions about them from a retrieval index is using both correctly.
The sequencing matters though. Build retrieval first, get quality to the point where the remaining failures are about form rather than content, and only then consider tuning. Doing it the other way round means training a model against a context pipeline that is about to change, and the training has to be redone.
The cost of a trained model that nobody mentions
A tuned model is a thing you now own. It has a version, it drifts as the world moves, and it needs retraining when the underlying model is deprecated by its provider, which happens on someone else's schedule.
That ownership is exactly the work described on our machine learning engineer page, and it is a standing commitment rather than a project. Organisations that tune without planning for it end up with a model nobody maintains, quietly getting worse, which is the same failure as an unmaintained evaluation set and is harder to reverse.
None of which argues against tuning where it is warranted. It argues for making the decision with the maintenance cost visible, and for trying the cheap fix first, because the cheap fix is right more often than the field's enthusiasm suggests.
