On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance

In three linesarXiv study on LLM adaptation limits for annotation tasks. Toxicity detection experiments across diverse datasets show 66% of zero-shot errors resist correction via prompting (rescue rate 34.8%). Models follow misaligned definitions while maintaining confidence. Definition-Specific Familiarity (DSF) metric correlates with performance (r=+0.41), outperforming memorization metrics.

Summary generated by Claude — human-verified