Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization

In three linesOrth-Dion improves Dion, a low-rank spectral optimizer for distributed training. By replacing column normalization with QR orthogonalization, it eliminates a √r convergence gap and achieves O(√L_r/T) rate matching exact spectral methods. Validated on large-scale language model pre-training.

Summary generated by Claude — human-verified