Multi-layer Cross-attention is Provably Optimal for Multi-modal In-context Learning

In three linesTheoretical study proving multi-layer cross-attention is optimal for multi-modal in-context learning. Authors show single-layer linear self-attention fails to recover Bayes-optimal predictor, but linearized cross-attention mechanism achieves Bayes optimality with gradient flow.

Summary generated by Claude — human-verified