GPT-5 demonstrates self-awareness bias in financial backtesting
The story so far
Hugging Face has released the FinDendro benchmark, containing 2,226 U.S. financial and macroeconomic facts from 1999 to 2026, for testing LLMs' temporal self-awareness. The study found that GPT-5 correctly answered 76% of the control facts in financial backtesting but claimed to know only 38%, and 61% of the facts it claimed to not know were still answered correctly. The claimed knowledge can extend to measuring the knowledge boundary seven quarters later. Historical date instructions can eliminate most knowledge claims without changing accuracy.
AI-written from the reports · Updated 9 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersLLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting
GPT-5 correctly answered 76% of control facts in financial backtesting but claimed to know only 38%, and still correctly answered 61% of the facts it claimed to be unaware of. The claimed knowledge can extend to seven quarters after the measurement knowledge boundary. Historical date instructions can eliminate most knowledge claims without changing accuracy.