Learn · Module 12 · Inference and serving · Lesson 4 of 6
Caching
The fastest, cheapest model call is the one you don't make again. Caching works at several levels.
In short
- Caches work at several levels: KV, prompt prefix, exact response and semantic.
- Prompt caching reuses a shared prompt start and can cut input cost sharply.
- Put stable content first; semantic caches need care to avoid wrong answers.
This lesson is part of Pro
The first two lessons of every module are free. Pro opens every lesson and every project.
See Pro plansFrom ₹119 a month. No auto-renewal.