Databricks reduces AI coding agent unit cost by 90%
The story so far
Databricks executive Patrick stated that after fully adopting AI coding agents, costs increased exponentially with usage, once threatening efficiency gains. By establishing a control layer, unit costs were reduced by up to 90%. Currently, Databricks' per-person token expenditure is roughly stable, with total usage and value still rising. Cost reduction mainly comes from two aspects: shifting traffic to cheaper new models, contributing to over 50% reduction; Smart Router, while maintaining near the quality of the most expensive model, reduces average task costs by over 30%. Databricks also regularly sends employees detailed personal spending and savings recommendations, and handles over-spending with warnings, approvals, then downgrading to cheaper models. This internal solution was later productized as Unity Gateway, currently used by thousands of customers; Databricks has also set up a separate budget for this proxy. The above reductions and effects are Databricks' claims.
AI-written from the reports · Updated 26 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Databricks BlogFeaturedThe AI Bill is coming
Databricks executive Patrick reviews the process where the cost of AI coding agents increased exponentially after the company fully adopted them, and then reduced the unit cost by up to 90% by controlling the layer. He mentions that shifting traffic to cheaper new models contributed to more than 50% of the reduction, and Smart Router reduced the average task cost by more than 30% while maintaining near the quality of the most expensive model. This internal solution was later productized as Unity Gateway, which is now used by thousands of customers.