Skip to content
Live
Trending storyPast story

Use cheap classifiers to route Gemini models

1 reports1 outlets16 days agoUpdate

The story so far

AI overview

Google uses TypeSafe's Jev classifier to route Google Gemini models to reduce inference costs. TypeSafe's Jev classifier can route tasks to the most cost-effective Gemini models, with 50-question testing showing an accuracy of 96%, better than the 93.8% accuracy of Python regular heuristic methods. Setting the inference model MaxOutputTokens limit does not save costs but may interrupt the inference of complex tasks.

AI-written from the reports · Updated 15 days ago

Coverage timeline

Follow the coverage to see every side of the story.

3 Nov
  1. Google Developers Blog
    Cost per task: Using cheap classifiers to route across Gemini models and thinking levels

    TypeSafe's Jev classifier can route tasks to the most cost-effective Gemini models, with 50 question tests showing an accuracy of 96%, better than 93.8% of Python regex heuristic methods; setting the inference model MaxOutputTokens limit does not save costs but may interrupt complex task inference.