How AI training scales

In three linesOpenAI discovers that gradient noise scale predicts the parallelizability of neural network training across tasks. Complex tasks have noisier gradients, enabling larger batch sizes in the future and removing a potential constraint on AI system scaling.

Summary generated by Claude — human-verified