PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning

In three linesPADD is a knowledge distillation method to convert dense teachers into MoE students without explicit routers. The 4-stage framework (initialization + training) uses teacher neuron clustering and adaptive optimization to achieve gains on mathematical reasoning benchmarks, with MoE students matching or exceeding dense teachers.

Summary generated by Claude — human-verified