DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training

In three linesDynaTrain is a distributed training system enabling sub-second online reconfiguration of multi-dimensional parallelism. Using a Virtual Parameter Space abstraction, it reconfigures a 70B dense model in 2s and a 235B MoE model in 4.36s, outperforming existing elastic systems by up to three orders of magnitude.

Summary generated by Claude — human-verified