KV cache quant benchmarks: KVarN 6-bit matches q8_0, 4-bit matches q5_0. Massive!

In three linesKVarN KV cache quantization matches precision of one bit higher: 6-bit KVarN equals q8_0, 4-bit matches q5_0. Benchmarks on Qwen 27B 64k context show major VRAM savings. Implementation in BeeLlama v0.3.2 (llama.cpp fork). Prompt processing slower for now.

Summary generated by Claude — human-verified