AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training

In three linesAdaptiveLoad optimizes video diffusion Transformer training (DiT, MMDiT) by addressing load imbalance from quadratic attention complexity. Two components: dual-constraint adaptive load balancing and fused LayerNorm-Modulate CUDA kernel. On Wan 2.1: computational imbalance reduced from 39% to 18.9%, peak VRAM utilization +22.7%, training throughput +27.2%.

Summary generated by Claude — human-verified