Less Data, Faster Training: repeating smaller datasets speeds up learning via sampling biases

In three linesRepeating a smaller dataset during training accelerates learning compared to using a larger dataset, via sampling biases that enable favorable layer-wise growth. Effect observed across algorithmic tasks, architectures and optimizers. Authors provide theoretical analysis and empirical interventions.

Summary generated by Claude — human-verified