Contribution Weights: A Geometrical Analysis of Self-Attention Transformers

In three linesNew 'Contribution Weights' metric for analyzing transformers beyond attention weights. Incorporates value vector magnitude and directional alignment. Outperforms attention-based metrics for identifying critical tokens. Reveals attention sinks play active information suppression role, stabilizing representations against semantic drift.

Summary generated by Claude — human-verified