When Better Codebooks Are Not Enough: Predictive Performance and Behavioral Reliability in LLM Political Event Coding

In three linesStudy on political event coding with LLMs: higher accuracy does not ensure behavioral reliability. Expert codebooks optimized with clearer definitions, examples, and context improve performance, but models fail reliability tests under controlled variations (label names, codebook order, mappings). Accuracy alone is insufficient for social-science applications.

Summary generated by Claude — human-verified