ART: Attention Run-time Termination for Efficient Large Language Model Decoding

In three linesART (Attention Run-time Termination) is a lightweight runtime mechanism that halts KV block accesses during decoding once their attention contribution becomes negligible. Tested on LongBench, it achieves 20% higher generation throughput on large batches while maintaining comparable accuracy.

Summary generated by Claude — human-verified