Kuramoto Attention: Synchronizing Self-Attention on the Torus

In three linesKuramoto attention introduces a self-attention layer where each hidden coordinate is an angle on the torus. Tokens are scored by gated cosine similarity and updated via attention-weighted circular mean (Kuramoto coupling term). On enwik8, the layer achieves 1.637±0.010 BPC at 1M parameters versus 1.616±0.004 for RoPE+SwiGLU, validating this constrained geometric structure.

Summary generated by Claude — human-verified