LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation

In three linesLightTransfer converts language models (LLaMA, Mistral, QwQ-STILL) into hybrid architectures without training. The method identifies lazy layers and replaces full attention with streaming attention, reducing KV cache costs. Results: up to 2.17× throughput improvement with <1.5% loss on LongBench and 53.3% on AIME24.

Summary generated by Claude — human-verified