You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

In three linesTAQ (Task-Aware Quantization) is a training-free post-training quantization method that dynamically allocates precision to task-relevant layers using unlabeled calibration prompts. Three variants (TAQ-IS, TAQ-KL, TAQ-O) estimate layer importance from hidden representations. Significant gains in accuracy-memory ratio validated on real hardware throughput and latency.

Summary generated by Claude — human-verified