KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference

In three linesKVDrive is a multi-tier KV cache management system for long-context LLM inference, orchestrating cache placement across GPU/DRAM/SSD, pipeline scheduling, and cross-tier coordination. The prototype achieves 1.74x higher throughput than state-of-the-art systems while preserving accuracy.

Summary generated by Claude — human-verified