Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks

In three linesResearchers explain grokking (sudden generalization after prolonged overfitting) through first-order phase transitions driven by L2 regularization strength. SGD noise enables networks to escape trapped metastable states, with escape times following Arrhenius scaling. Results extend to nonlinear networks.

Summary generated by Claude — human-verified