epochs

Learn · Module 07 · Training and fine-tuning · Lesson 4 of 8

Fine-tuning

Fine-tuning continues training a pretrained model on your own examples, so it learns a behaviour you can't get from prompting alone.

4 minute read

In short
  • Supervised fine-tuning trains on input and ideal output pairs, with the loss on the output only.
  • It teaches behaviour, format and style well, and facts poorly.
  • Full fine-tuning needs roughly 16 bytes per parameter; try prompting first.

This lesson is part of Pro

The first two lessons of every module are free. Pro opens every lesson and every project.

See Pro plansFrom ₹119 a month. No auto-renewal.