Beyond Prediction: Tail-Aware Scheduling for LLM Inference

In three linesNew LLM inference scheduler replacing explicit length prediction with lightweight statistical signals and dynamic priority boosting. Reduces P99 TTLT by 35-50% vs SRPT with perfect length knowledge, and TTFT by 34-47% across production and open-source traces.

Summary generated by Claude — human-verified