FastMix: Fast Data Mixture Optimization via Gradient Descent

In three linesFastMix automates data mixture optimization for model training via gradient descent. The method reformulates mixture selection as a bilevel optimization problem, jointly optimizing mixture coefficients and model parameters. A single proxy model suffices, drastically reducing search cost compared to prior approaches.

Summary generated by Claude — human-verified