Worker Disagreement Reveals Sharp Directions in Local SGD

In three linesResearchers show Local SGD exposes anisotropic loss geometry through worker disagreement. Worker-average gaps provide a Hessian-free estimator of dominant spectral directions. Validated on MLPs, CNNs, and Transformers.

Summary generated by Claude — human-verified