[3090] Gemma4 QAT + MTP quick TPS numbers [TLDR 1.2-1.8x better]
Signal
72
Hype
35
In three linesGemma 4 with QAT and MTP achieves 70-80 tok/s on RTX 3090 (vs 40 tok/s prior), 1.2-1.8x speedup. Setup: Gemma 4 12B/31B, draft-MTP, 40960 context, Q4_K_XL + Q8_0 KV cache. Multimodal support and near-instant voice response.Read source
Your take?
Summary generated by Claude — human-verified