Amazon Bedrock inference choices affect costs

Amazon introduced four aspects of Bedrock inference choices that affect costs: cache duration, service tier, inference effort, and maxTokens. These choices can affect costs without changing the model or prompt, and each choice depends on the model and workload. Amazon emphasizes that longer cache duration does not automatically mean cheaper and should keep latency-sensitive traffic at the default setting. The effects of these choices can be verified through response metadata, logs, and billing data.

AI-written from the reports · Updated 2 days ago