schema · growing
When to fine-tune
Yes for form, no for facts. The order: prompt, RAG, fine-tune, distill.
Fine-tuning is for form, not for facts. Burning knowledge that changes weekly into weights means retraining on every change.
Rendering diagram…
The two legitimate jobs
- Distillation. Move a strong model's behaviour into a small, cheap, fast one. What you gain is cost and latency.
- Locking in the residue. The tone, output schema and refusal patterns prompting cannot hold. The long tail that never reaches 100%.
Do not break the order: Prompt → RAG → Fine-tune → Distill. Reaching for a fine-tune before measuring RAG is renting GPUs without knowing the problem.