TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts

In three linesTENP proposes a structured pruning framework for Mixture-of-Experts LLMs. The method identifies important experts and applies neuron-level pruning to less important experts in a trapezoidal pattern across layers. On DeepSeek with 40% routing sparsity and 63.76% activated expert parameters, accuracy drop is limited to 1 point, with +10% improvement on code generation tasks.

Summary generated by Claude — human-verified