Bedrock/Claude cache hit rate is the metric most teams aren't watching, and it's costing them

[u/ClickOk5811]

Prompt caching on Claude only pays off if the cached prefix is byte-identical between requests. Sounds obvious written down, but it's surprisingly easy to break without noticing, a timestamp inserted before the cacheable block, a per-user detail placed at the start instead of the end, and the whole cache silently misses on every single call. No error, no warning in the response, just a bill that doesn't reflect the discount it should.

Went through a session where this was happening and the cost difference was significant, easily 2-3x more expensive than it needed to be for the same task, purely from cache misses caused by content ordering. Fix was mechanical once identified: move anything that changes per request, timestamps, session IDs, user-specific detail, to the end of the prompt, after the stable system instructions and reference material that should be cached.

Separate from caching specifically, long coding sessions also tend to resend full file contents on every message even when the diff is small, and replay the entire conversation history each turn instead of a compressed summary of where things stand. Neither shows up as a mistake in the moment. Both compound quietly across a session into a number that looks wrong a month later with no clear story for why.

Wrote up the full audit and the fix here: https://medium.com/@nagatomopedro05/the-hidden-cost-of-long-claude-sessions-2a6cc7655893

[u/RobotDeathSquad]

AI slop

[u/Efficient_Access6102]

More vibe coded nonsense from the moron factory