Semantic Caching for LLMs: Architecture, Thresholds, Risks
Semantic caching for LLMs explained: gateway architecture, safe similarity thresholds, tenant scoping and a tested Node.js cache, with published hit rates.
Tag Prompt caching
Bedrock Prompt Caching in Node.js: Converse Setup and Costs
Bedrock prompt caching in Node.js: add cachePoint blocks to Converse API calls, cut Claude input costs, and fix the gotchas that keep cache reads at zero.