Fine-Tuning vs Prompting vs Retrieval: Choosing How to Customise AI

Most business owners who try a general AI assistant quickly hit the same wall. The answers are fluent, but they do not know your prices, your policies, your product names or the way your team talks to customers. The natural next question is how to make the AI "know" your business.

There are three common approaches: writing better prompts, connecting the AI to your own documents through retrieval, and fine-tuning a model on your examples. They are often confused, and vendors sometimes pitch the most expensive option first. This guide explains each one in plain language so you can pick the lightest approach that solves your actual problem.

Option 1: Better Prompts and Instructions

Prompting means giving the model clear instructions and context each time you use it. That can be as simple as a saved instruction block that tells the AI who your customers are, what tone to use and what it must never promise.

Prompting is the cheapest and fastest option. You can change it in minutes, nobody needs to write code, and it works with almost every AI tool. Many teams get most of the value they need just by improving prompts and sharing them. Our guide on building a shared prompt library shows how to make good prompts reusable.

The limit is space and memory. You can only paste so much information into a prompt, and the model will not reliably remember details you gave it last week.

Option 2: Retrieval From Your Own Documents

Retrieval, often called retrieval-augmented generation, connects the AI to a collection of your documents. When someone asks a question, the system first searches your files for relevant passages, then passes those passages to the model so it can answer using your information.

This is the usual choice when the problem is knowledge, such as:

• Answering customer questions from your FAQ, terms and product sheets
• Helping staff find steps in long SOPs or manuals
• Searching past proposals, reports or tender documents
• Keeping answers current, because you update the documents instead of retraining anything

Retrieval needs some setup: cleaning documents, choosing a tool or platform and testing answers. But it keeps your knowledge in files you control, and good systems can show which document an answer came from.

Option 3: Fine-Tuning a Model

Fine-tuning means training an existing model further on many examples, so it learns a pattern. It is best for teaching style, format or a narrow repeated task, not for teaching facts that change.

Good fine-tuning candidates include classifying support tickets into fixed categories, writing replies in a very specific house style, or turning messy notes into a strict format. Poor candidates include "learn our price list" or "know our latest policies", because the model may still mix things up and you would need to retrain every time something changes.

Fine-tuning also needs a decent set of high-quality examples, technical skills and ongoing evaluation. For most small businesses, it is the last option to try, not the first.

A Simple Way to Decide

Ask what is actually going wrong with the AI today:

1. If answers are vague or the tone is off, improve prompts first.
2. If answers are wrong because the AI lacks your information, use retrieval.
3. If the AI has the information but cannot follow a very specific format at volume, consider fine-tuning.
4. If you are unsure, run a small test with prompts and retrieval before spending on training.

Key takeaway: start with the lightest option. Prompts fix tone, retrieval fixes missing knowledge and fine-tuning fixes repeated format problems. Most SMEs never need the third.

Costs, Risks and Data Questions

Each option has different risks. Prompts can leak sensitive details if staff paste customer data into public tools. Retrieval systems must respect access rights, so a junior staff member should not be able to pull up salary files through a chatbot. Fine-tuning means sending training data to a provider or running your own infrastructure, which raises privacy and cost questions.

Before you commit, check where your data is stored, who can see it and whether the provider uses it to train their own models. Malaysia's PDPA still applies when personal data is involved, whichever approach you pick.

FAQ

Is retrieval the same as training the AI on my data?

No. Retrieval looks up your documents at the moment a question is asked and passes relevant parts to the model. The model itself is not changed, which makes updates much easier.

How many examples do I need for fine-tuning?

It depends on the task and the provider, so check their documentation. In general you need many consistent, high-quality examples, and a small messy set rarely gives good results.

Can I combine these approaches?

Yes. Many real systems use good prompts plus retrieval, and some add a fine-tuned model for one specific step. Start simple and add layers only when testing shows a clear need.

Customising AI does not have to mean an expensive training project. Write clear prompts, connect your best documents when knowledge is the gap, and keep fine-tuning in reserve for narrow, high-volume tasks where it clearly pays off.