AI & AutomationGuide
RAG or Fine-Tuning? Which One a Small Business Actually Needs
Both promise an AI that "knows your business". They do very different jobs, cost very different amounts to keep up to date, and fail in different ways.
Author
DueClix Engineering
Published
Reading time
3 min read
When a business wants an AI assistant that answers from its own information (its prices, policies, products and procedures), two technical terms come up almost immediately: RAG and fine-tuning. They are often presented as alternatives. They are not. They solve different problems, and choosing the wrong one is an expensive way to get an assistant that still gets things wrong.
What each one does
RAG (retrieval-augmented generation) means the assistant looks things up before it answers. Your documents are indexed; when a question arrives, the system finds the few passages most likely to contain the answer and hands them to the model along with the question. The model writes its reply from those passages, and a well-built system shows which document each answer came from.
Fine-tuning means training a model further on your own examples so that it behaves differently: a particular tone, a fixed output format, or a narrow task it does the same way every time. It changes how the model writes and decides. It is a poor way to teach it facts, and facts it learned that way cannot be updated without training again.
Side by side
| RAG | Fine-tuning | |
|---|---|---|
| Best for | Answering from your documents and data | Consistent style, format or a narrow repeated task |
| When your information changes | Update the document; the next answer uses it | Retrain the model |
| Can show where an answer came from | Yes, if built to cite sources | No |
| What you need to start | Your documents, cleaned up | Hundreds of good examples of the behaviour you want |
| Typical way it fails | Finds the wrong passage, or an outdated one | Confidently repeats what it learned, even after it changed |
| Ongoing work | Keeping the documents current | Collecting new examples and retraining |
What most small businesses need
If the goal is answer customer or staff questions correctly from our information, the answer is almost always RAG. Prices change, policies change, products come and go, and an assistant that cannot be corrected by editing a document becomes wrong a little more every month.
Fine-tuning earns its place in narrower cases: sorting thousands of incoming emails into the same fifteen categories, writing product descriptions in one house style, or extracting the same fields from the same kind of form. Even then, a well-written prompt with a few examples on a current model often gets close enough that training is not worth it.
Where RAG projects actually go wrong
When a retrieval assistant gives a wrong answer, the model is rarely the cause. The usual culprits are the documents: deleted files that are still searchable, the same file indexed several times, last year's policy ranking above this year's. We wrote about this in detail in Your RAG system is not hallucinating, it is retrieving badly. A newer model will not fix it; cleaning the documents will.
A five-question checklist
- Does the assistant need to answer from information that changes? Then RAG.
- Do people need to see where an answer came from? Then RAG, built to cite its sources.
- Does the information fit in a few pages? Then try plain instructions first.
- Is the problem how it writes or classifies, not what it knows? Then try examples in the prompt; fine-tune only if that falls short.
- Do you have hundreds of good, consistent examples? If not, fine-tuning is not an option yet, whatever else is true.
For a rough sense of whether an assistant is worth building at all, the chatbot ROI calculator compares the staff time it could free against what the AI costs to run.