Skip to content
Live
Trending storyPast story

GPT-5 demonstrates self-awareness bias in financial backtesting

1 reports1 outlets10 days agoUpdate

The story so far

AI overview

Hugging Face has released the FinDendro benchmark, containing 2,226 U.S. financial and macroeconomic facts from 1999 to 2026, for testing LLMs' temporal self-awareness. The study found that GPT-5 correctly answered 76% of the control facts in financial backtesting but claimed to know only 38%, and 61% of the facts it claimed to not know were still answered correctly. The claimed knowledge can extend to measuring the knowledge boundary seven quarters later. Historical date instructions can eliminate most knowledge claims without changing accuracy.

AI-written from the reports · Updated 9 days ago

Coverage timeline

Follow the coverage to see every side of the story.

9 Nov
  1. Hugging Face Daily Papers
    LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting

    GPT-5 correctly answered 76% of control facts in financial backtesting but claimed to know only 38%, and still correctly answered 61% of the facts it claimed to be unaware of. The claimed knowledge can extend to seven quarters after the measurement knowledge boundary. Historical date instructions can eliminate most knowledge claims without changing accuracy.