StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

In three linesStructLens analyzes the internal organization of representations in language models through maximum spanning trees built on residual streams. The framework reveals that middle layers strongly organize nearby tokens, and that smaller local units emerge before larger units during pre-training.

Summary generated by Claude — human-verified