GEM: Geometric Entropy Mixing for Optimal LLM Data Curation

In three linesGEM (Geometric Entropy Mixing) reformulates LLM data curation as a variational problem on the hypersphere to prevent cluster collapse. Uses provable MM algorithm and teacher-student distillation for web-scale scaling. Improves downstream accuracy by up to 1.2% on 1.1B models integrated with DoReMi and RegMix.

Summary generated by Claude — human-verified