K-Quantization and its Impact on Output Performance

In three linesEmpirical study of quantization impact (2-6 bits) on 8 LLMs evaluated on MMLU-Pro, CRUXEval, and MuSR. Results: 8-bit precision (Q8_0) optimal, aggressive quantization (Q2_K) acceptable but with variable losses across models/tasks. 7-9B models offer best efficiency/performance trade-off.

Summary generated by Claude — human-verified