Cartridges at Scale: Training Modular KV Caches over Large Document Collections

In three linesCartridges at Scale (CAS) trains modular, reusable KV caches for large document collections, eliminating costly prefilling. The framework dynamically manages hundreds of per-document cartridges with GPU/storage rotation, scaling to over one million tokens. Performance: +10-31 points vs monolithic cartridge, 3-4x fewer tokens than conventional RAG.

Summary generated by Claude — human-verified