Can LLMs Refuse Questions They Do Not Know? Measuring Knowledge-Aware Refusal in Factual Tasks

In three linesNew metric called Refusal Index (RI) measures LLMs' ability to refuse questions beyond their knowledge. RI correlates refusal probability with error probability using Spearman's rank correlation. Testing across 16 models and 5 datasets shows LLM refusal behavior remains fragile despite high factual accuracy.

Summary generated by Claude — human-verified