Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models

In three linesLLMs exhibit highly anisotropic internal representations with massive activations. Rather than treating them as artifacts, the authors identify them as interpretable functional units using a magnitude-based criterion. Steering applied to these critical dimensions outperforms conventional whole-dimension steering in domain adaptation and jailbreaking scenarios.

Summary generated by Claude — human-verified