Data for AI Products
Fine-Tuning vs. RAG vs. Prompting: A Data-First Framework
Prompting changes instructions. RAG changes what the model can see. Fine-tuning changes what the model is.
Here is the cleanest mental model I know, and it takes ten seconds to explain to an executive.
• Prompting is telling a very capable new hire what you want, at the moment.
• RAG (retrieval-augmented generation) is giving that new hire access to your company wiki, so they can look things up before answering.
• Fine-tuning is putting that new hire through six months of training so they instinctively work the way your company works.
Stated that way, the mistake most teams make becomes obvious. They send the new hire to a six-month training program to teach them a fact that was written on the wiki. It is slow, expensive, and it goes stale the moment the fact changes.
| Prompting | RAG | Fine-tuning | |
|---|---|---|---|
| Teaches the model | Instructions | Facts, at query time | Behaviour, style, structure |
| Data you need | A handful of examples | A clean, chunked, up-to-date document corpus | 50 to several thousand input-output pairs |
| Time to first result | Hours | 2 to 6 weeks | 3 to 8 weeks, plus data collection |
| Handles changing facts | Poorly | Very well. Update the document, done. | Badly. You have to retrain. |
| Can cite a source | No | Yes, and this is often the real reason to choose it | No |
| Ongoing cost driver | Tokens | Retrieval infra, embedding refresh, content ops | Retraining every time your product changes |
| Biggest failure mode | Brittle. Works until an edge case arrives. | Retrieval fetches the wrong chunk and the model confidently paraphrases it | You spent 8 weeks teaching a fact that changed in week 6 |
The row about citation deserves attention. In regulated products, finance, healthcare, insurance, legal, the ability to show the customer where the answer came from is frequently a hard requirement. A fine-tuned model cannot do that. It has absorbed the knowledge into its weights and cannot point at anything. This single constraint decides the architecture in a large share of B2B products, and it is a product constraint, not a technical one, which means it is yours to raise.
Fig 3.1 - The question "facts or form" resolves most architecture debates in about five minutes.
THE DISTINCTION IN ONE SENTENCE
If your model says the wrong thing, you have a retrieval problem. If your model says the right thing in the wrong way, you have a fine-tuning problem. Most teams misdiagnose this, and then spend a quarter applying the wrong medicine.
YOUR MOVE THIS WEEK
Take your last 30 AI failure reports and sort them into two piles: wrong facts, and wrong form. The bigger pile is your architecture decision, and now it has evidence behind it instead of a preference.