Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption

In three linesEmpirical study of multi-model LLM scheduling challenges on heterogeneous hardware. Authors quantify layer offloading impact (non-linear throughput degradation, model-dependent sensitivity) and preemption costs (dominated by state reload). Identifies critical features for next-generation schedulers.

Summary generated by Claude — human-verified