Confidence Calibration in Large Language Models

In three linesPreregistered study shows current LLMs are overconfident: confidence exceeds accuracy on average. A hard-easy effect moderates this bias: overconfidence peaks on difficult tasks, while easy tasks show substantial underconfidence. Introduces LifeEval, a benchmark for evaluating model calibration across difficulty levels.

Summary generated by Claude — human-verified