Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks

In three linesResearchers propose the Refusal Index (RI), a metric measuring LLMs' ability to refuse questions beyond their knowledge. RI correlates refusal probability with error probability using Spearman's rank correlation. Testing across 16 models and 5 datasets shows LLMs refuse unreliably despite high factual accuracy.

Summary generated by Claude — human-verified