Language models fail at extended rule following

In three linesLanguage models fail to reliably apply simple rules over long sequences. Test on 126 model variants: all models cannot count above a model-dependent threshold. Failures are abrupt and persist despite increasing model size and computation. Mechanistic probing shows models use finite internal states to simulate counting, exhausting them beyond threshold.

Summary generated by Claude — human-verified