From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data

In three linesarXiv paper analyzing LLM hallucinations as structural consequence of three architectural decisions: self-attention confuses entities via co-occurrence learning, MLE objective optimizes probability without factual constraint, autoregressive decoding cascades errors without revision. Dataset pathologies amplify but do not independently cause these failures.

Summary generated by Claude — human-verified