Airbnb Optimizes Amazon Bedrock Proxy Costs
The story so far
Airbnb selects encoding proxies on Amazon Bedrock by evaluating the cost of each task and monitoring their usage. They use custom evaluations (evals) to measure model scores and task costs, combining CloudWatch real-time metrics (such as latency, rate limiting, and cost) to adjust model defaults and inference effort limits. This approach helps Airbnb optimize performance and cost when updating models. Airbnb states that task cost is the billed cost for all tokens used by a proxy to complete a task. They use evaluation results to select each default (model and effort level, unless an engineer chooses otherwise) and each effort limit (the maximum effort level they allow the model). They compare production signals with evaluation scores and task costs to determine when to change defaults or effort limits.
AI-written from the reports · Updated 21 minutes ago
Coverage timeline
Follow the coverage to see every side of the story.
- AWS Machine Learning BlogHow Airbnb uses cost per task to operate coding agents on Amazon Bedrock
Airbnb selects coding agents on Amazon Bedrock by evaluating model task costs and monitors their usage. They use custom evaluations (evals) to measure the score of each model and the cost of each task, and combine real-time metrics (such as latency, rate limiting, and cost) in CloudWatch to adjust model defaults and inference effort limits. These methods help Airbnb optimize performance and cost when updating models.
Heat over time
Not enough data yet to draw a trend.