Prompt caching in practice

Previous lesson: reused exact prefixes skip prefill at ~90% off. This one is the field manual — how to actually get those hits, and how to verify them on the Bhaskara API.

The three rules

• Stable system prompt, always first. Same bytes every turn. Per-session toggles (terse mode, doc packs) are fine — they change once and stay. Per-request fiddling is cache poison.
• Append-only history. New turns go at the end, old ones are never edited, reordered, or regenerated mid-stream. Compaction (ours fires past 200K tokens) is the deliberate exception: one budgeted re-warm, then stable again.
• Volatile content goes last. Timestamps, request IDs, progress counters — anything that changes every turn belongs at the tail, never in the middle of settled context.

Reading the proof on our API

Every response's usage carries prompt_tokens_details.cached_tokens — the tokens served from cache this turn. Our theta display rates make the value visceral:

theta display rates (per 1M tokens):
  input      $0.20
  cache hit  $0.04     ← 5× cheaper
  output     $0.40

A healthy agentic session shows 70–90% of prompt tokens arriving as cache hits. If yours doesn't, the cause is almost always one of the three rules above — usually volatile content smuggled into the middle of history (timestamps in tool output are the classic offender; our context engine masks them before forwarding).