3 articles
Hugging Face releases a method to reuse pre-trained language model checkpoints in encoder-decoder architectures. The technique improves training efficiency and reduces resources needed to build performant seq2seq models.
Hugging Face documents porting the WMT19 translation system from fairseq to the transformers library. Technical migration of a neural machine translation architecture to the transformers ecosystem, with reproduction of WMT19 benchmark results.
Hugging Face integrates Ray Tune for automated hyperparameter optimization in the Transformers library. This integration enables researchers and practitioners to efficiently tune training parameters (learning rate, batch size, etc.) using distributed search algorithms.