Gradient Dynamics of Attention: How Cross-Entropy Sculpts Bayesian Manifolds

In three linesComplete first-order analysis of gradient dynamics in transformer attention heads under cross-entropy training. Authors establish an advantage-based routing law and responsibility-weighted value updates, showing that optimization creates Bayesian manifolds implementing in-context probabilistic reasoning.

Summary generated by Claude — human-verified