Amazon Bedrock supports open-source weight model explicit prompt caching
The story so far
Amazon Bedrock now supports explicit prompt caching for open-source weight models, automatically reusing context prefixes via the Chat Completions API to reduce input costs and response latency. Users need to mark the end point of reusable prefixes and check the model card to understand caching mode, minimum prefix length, and validity period parameters.
AI-written from the reports · Updated 10 hours ago
Coverage timeline
Follow the coverage to see every side of the story.
- AWS Machine Learning BlogTips for using open-source weight models on Amazon Bedrock
Open-source weight models in Amazon Bedrock now support explicit prompt caching, which can automatically reuse context prefixes via the Chat Completions API to reduce input costs and response latency. Explicit caching requires marking the end point of reusable prefixes, while implicit caching automatically reuses stable prefixes. Combining both can significantly save costs. Users need to check the model card to understand caching mode, minimum prefix length, and validity period parameters.
Heat over time
- Now
- 8
- Peak
- 1017 Nov, 00:00
- 24h change
- –
Hover or tap the chart to see each hour; use the arrow keys on a keyboard.