From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate
Hugging Face Accelerate adds native support for PyTorch's FSDP (Fully Sharded Data Parallel), providing an alternative to DeepSpeed for distributed training. The update enables users to switch easily between DeepSpeed and FSDP based on their requirements.