Skip to content
Live
Trending storyDeveloping

Amazon Bedrock supports open-source weight model explicit prompt caching

1 reports1 outlets10 hours agoUpdate

The story so far

AI overview

Amazon Bedrock now supports explicit prompt caching for open-source weight models, automatically reusing context prefixes via the Chat Completions API to reduce input costs and response latency. Users need to mark the end point of reusable prefixes and check the model card to understand caching mode, minimum prefix length, and validity period parameters.

AI-written from the reports · Updated 10 hours ago

Coverage timeline

Follow the coverage to see every side of the story.

16 Nov
  1. AWS Machine Learning Blog
    Tips for using open-source weight models on Amazon Bedrock

    Open-source weight models in Amazon Bedrock now support explicit prompt caching, which can automatically reuse context prefixes via the Chat Completions API to reduce input costs and response latency. Explicit caching requires marking the end point of reusable prefixes, while implicit caching automatically reuses stable prefixes. Combining both can significantly save costs. Users need to check the model card to understand caching mode, minimum prefix length, and validity period parameters.

Heat over time

Now
8
Peak
1017 Nov, 00:00
24h change
–

Hover or tap the chart to see each hour; use the arrow keys on a keyboard.