Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway

In three linesTheoretical analysis showing discrete Gradient Descent with large step size in deep linear multi-pathway networks restores symmetry. Contrary to Gradient Flow predictions (winner-takes-all specialization), GD with Edge of Stability oscillations redistributes signals across pathways, favoring shared representations over single-pathway dominance.

Summary generated by Claude — human-verified