LLM confidence scores are unreliable
The story so far
Using LLM for confidence scoring is a wrong idea. The scores generated by LLM are not based on information or actual calibration, but rather the continuation of text generation, which changes with the way of asking. LLM scoring lacks the ability to reflect the strength of evidence and has not been calibrated to match accuracy. jimauthors stated that LLM cannot provide reliable confidence scores.
AI-written from the reports · Updated 22 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hacker News · AI hot postsUsing LLM for confidence scoring is a bad idea
LLMs cannot provide reliable confidence scores because the scores they generate are not based on information quantity or actual calibration, but rather the continuation of text generation, which changes with the way questions are phrased. LLM scores lack the ability to reflect the strength of evidence and have not been calibrated to match accuracy.