Maybe KV cache offload to RAM isn't bad

In three linesllama.cpp user demonstrates KV cache offload to RAM (-nkvo flag) can be beneficial. On RTX 5060 Ti 16GB + 32GB DDR5 with Qwen 3.6 27B (IQ4_XS): offload enables 65k context in native f16 (19 tps peak) vs q4_0 quantization + 58 GPU layers (23 tps). 128k context achievable with 63 GPU layers, stable throughput. Performance trade-off acceptable for flexibility.

Summary generated by Claude — human-verified