Beyond Layer Importance in Layer-wise Sparsity: An Inter-Layer Perturbation-Absorption Perspective

In three linesStudy on layer-wise redundancy in LLMs. Authors characterize how layers absorb or amplify perturbations during pruning: early layers amplify, middle and late layers absorb. They propose absorption-aware correction using a per-layer absorption coefficient, improving OWL and AlphaPruning by 7.13% perplexity reduction and 1.02% zero-shot accuracy boost at 70% sparsity.

Summary generated by Claude — human-verified