Fine-tuning continues training a pretrained model on your own examples, so it learns a behaviour you can't get from prompting alone.
4 minute read
In short
Supervised fine-tuning trains on input and ideal output pairs, with the loss on the output only.
It teaches behaviour, format and style well, and facts poorly.
Full fine-tuning needs roughly 16 bytes per parameter; try prompting first.
In plain words
Supervised fine-tuning (SFT) trains the model on pairs of input and ideal output. The loss is the same next-token loss as pretraining, but only on the output part, and the data is small and carefully chosen.
This is how a base model becomes a chat assistant: it's fine-tuned on thousands of examples of good conversations. You can do the same to teach a format, a tone, or a narrow skill such as writing SQL for your schema.
Worked example: what an example looks like
One training example
{
"messages": [
{"role": "system", "content": "Classify support tickets."},
{"role": "user", "content": "App crashes when I upload a photo"},
{"role": "assistant", "content": "{\"category\": \"bug\", \"area\": \"uploads\"}"}
]
}
The model learns to produce the assistant message given the rest. A few hundred to a few thousand clean, consistent examples often beats tens of thousands of messy ones.
Worked example: the memory bill
1
Running a 7B model
Each parameter takes 2 bytes in bfloat16.
7 × 10⁹ × 2 bytes = 14 GB
2
Fully fine-tuning it with Adam
Each parameter needs its weight, its gradient, and Adam's running averages, roughly 16 bytes in total.
7 × 10⁹ × 16 bytes = 112 GB
3
Why it matters
Full fine-tuning needs several large GPUs. The next lesson, LoRA, cuts this to one.
When to fine-tune, and when not to
goal
better tool
Teach new facts that change often
Retrieval (RAG)
Follow a strict output format every time
Fine-tuning, or structured outputs
Match a house style or tone
Fine-tuning
Do a narrow task cheaper with a smaller model
Fine-tuning a small model
Quick improvement you can iterate on today
Better prompts and examples
Your turn
Your product's prices change weekly and the assistant must quote them correctly. Should you fine-tune them in?
What to remember
Always try prompting first: it's fast to change. Fine-tune when prompting has plateaued, you have good examples, and you can measure the improvement. Hold back a test set before you start, and watch for catastrophic forgetting, where the model gets better at your task and worse at general ones.
Common mistakes
Fine-tuning to add knowledge that changes often; retrieval is cheaper to keep current.
Skipping a strong prompt-only baseline, so you can't tell if fine-tuning helped.
Training on a few hundred messy examples and expecting consistent behaviour.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
When would you fine-tune instead of prompting?
I'd start with prompting, because it's fast, cheap and easy to change, and I'd build an evaluation set first so I can measure it. I'd consider fine-tuning when prompting plateaus on things that are about behaviour rather than knowledge: a consistent format or tone, a narrow task the model keeps getting slightly wrong, or when I want a smaller, cheaper model to match a bigger one on one job. For knowledge that changes, I'd use retrieval instead, since updating documents is easier than retraining. Fine-tuning also needs good data, usually hundreds to thousands of clean examples, and ongoing maintenance when the base model changes, so I'd only do it when the evaluation gain is clear.
More to think about
How much data do you need to fine-tune?
What is catastrophic forgetting, and how do you guard against it?