Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

In three linesMethod to identify attention-head circuits in pretrained transformers using spectral signal (time-integrated participation ratio), task-pattern filtering, and group ablation against matched-random control. Validated across 51M to 7B parameters, two architectures, four pretraining pipelines. Finding: 2-6 head induction circuit causally necessary in all models tested (94-100% drop after ablation).

Summary generated by Claude — human-verified