Skip to content
Live
Trending storyPast story

Epoch Capabilities Index update adopts Bayesian model methodology

1 reports1 outlets15 days agoUpdate

The story so far

AI overview

Epoch AI updated the Epoch Capabilities Index (ECI), adopting a Bayesian model methodology to provide more reliable estimates of model capability uncertainty and probabilistic comparisons of capabilities between models. The new method is based on four-parameter logistic (4PL) item response theory (IRT), using Claude 3.5 Sonnet (ECI 130) and GPT-5 (ECI 150) as anchor models, achieving cross-temporal consistency through linear transformations to fix parameter positions and scales.

AI-written from the reports · Updated 14 days ago

Coverage timeline

Follow the coverage to see every side of the story.

4 Nov
  1. Epoch AI
    Updating the Epoch Capabilities Index methodology to use a Bayesian model

    Epoch Capabilities Index (ECI) update adopts Bayesian model methodology, providing more reliable estimates of model capability uncertainty and the probability of comparing capabilities between models. The new method is based on four-parameter logistic (4PL) item response theory (IRT), using Claude 3.5 Sonnet (ECI 130) and GPT-5 (ECI 150) as anchor models, and achieves cross-temporal consistency through linear transformations to fix parameter location and scale.