I'm still surprised on how good the kv quantization has become

In three linesA r/LocalLLaMA user reports that KV (key-value) quantization has reached impressive quality: even with KV at q4_0 (including the drafter), the model accurately retrieves information within a 100k token context.

Summary generated by Claude — human-verified