GaLore: Advancing Large Model Training on Consumer-grade Hardware

In three linesGaLore reduces GPU memory requirements for large model training through gradient projection decomposition. Enables training of 7B-70B models on consumer hardware (RTX 4090) with speedups up to 65% versus standard methods.

Summary generated by Claude — human-verified