Context Memorization for Efficient Long Context Generation

In three linesTraining-free approach for long-context LLM inference: attention-state memory externalizes prefix into lightweight lookup-based memory of precomputed attention states. On LLaMA-3.1-8B, reduces attention latency by 1.36x at 8K tokens and outperforms full-attention RAG with 20% memory footprint.

Summary generated by Claude — human-verified