Weight normalization: A simple reparameterization to accelerate training of deep neural networks

In three linesOpenAI introduces weight normalization, a simple reparameterization that accelerates training of deep neural networks. The technique decouples weight magnitude and direction, reducing gradient covariance and improving convergence.

Summary generated by Claude — human-verified