Epoch Capabilities Index update adopts Bayesian model methodology
The story so far
Epoch AI updated the Epoch Capabilities Index (ECI), adopting a Bayesian model methodology to provide more reliable estimates of model capability uncertainty and probabilistic comparisons of capabilities between models. The new method is based on four-parameter logistic (4PL) item response theory (IRT), using Claude 3.5 Sonnet (ECI 130) and GPT-5 (ECI 150) as anchor models, achieving cross-temporal consistency through linear transformations to fix parameter positions and scales.
AI-written from the reports · Updated 14 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Epoch AIUpdating the Epoch Capabilities Index methodology to use a Bayesian model
Epoch Capabilities Index (ECI) update adopts Bayesian model methodology, providing more reliable estimates of model capability uncertainty and the probability of comparing capabilities between models. The new method is based on four-parameter logistic (4PL) item response theory (IRT), using Claude 3.5 Sonnet (ECI 130) and GPT-5 (ECI 150) as anchor models, and achieves cross-temporal consistency through linear transformations to fix parameter location and scale.