The Routing Plateau: Understanding and Breaking the Accuracy Limits of LLM Routers

In three linesStudy of 21 LLM routing methods across 5 benchmarks reveals a routing plateau: most converge to similar accuracy far below oracle performance. The bottleneck is predictability—routers learn global averaged trends rather than query-specific signals. Larger training datasets, stronger encoders, and end-to-end fine-tuning improve routing accuracy.

Summary generated by Claude — human-verified