A Link between Shock-wave Theory and Symmetry-reduced Stochastic Gradient Descent for Artificial Neural Networks

In three linesMathematical link established between shock-wave theory and symmetry-quotiented stochastic gradient descent dynamics for neural networks. After quotienting parameter symmetries and entropy coarse-graining, effective dynamics satisfy a viscous Hamilton-Jacobi equation. Applied to MLPs, CNNs, Transformers, and mean-field networks.

Summary generated by Claude — human-verified