In-Context Learning Operates as Concept Subspace Learning

In three linesMechanistic study of in-context learning (ICL) showing structured demonstrations induce concept inference in low-dimensional subspaces. On Llama-3-8B, a 68–73-dimensional subspace of 4096 restores 78.8% of clean–corrupted accuracy gap, while the complementary subspace has zero effect. Results confirmed on Qwen2.5-7B and cross-lingual rule tasks.

Summary generated by Claude — human-verified