Using LLM for confidence scoring is a wrong idea.
LLMs cannot provide reliable confidence scores because the scores they generate are not based on information quantity or actual calibration, but rather the continuation of text generation, which changes with the way questions are phrased. LLM scores lack the ability to reflect the strength of evidence and have not been calibrated to match accuracy.
















