Unlocking Feature Learning in Gated Delta Networks at Scale

In three linesStudy of scaling rules for Gated Delta Networks via μP. Authors derive optimal parametrizations for learning-rate transfer across model widths. Experimental validation on LLM pre-training with AdamW and SGD.

Summary generated by Claude — human-verified