The Structural Attention Tax: How Retrieval Format Hijacks In-Context Learning Independent of Content

In three linesRAG systems suffer from a structural bias: knowledge graph triples capture 2-3x more attention per token than semantically equivalent natural language, compressing demonstration attention by up to 42%, regardless of relevance. Authors formalise this 'structural attention tax' and propose five mitigation strategies, validated on Mistral-7B and LLaMA-3-8B.

Summary generated by Claude — human-verified