Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage
Signal
72
Hype
15
In three linesReparametrization of Shampoo-based methods (KL-Shampoo, SOAP, KL-SOAP) enabling BFloat16 storage and reducing computational cost through subspace QR decomposition. Improves memory and time efficiency without performance degradation.Read source
Your take?
Summary generated by Claude — human-verified