Deployment guide for GPT-J 6B model on Amazon SageMaker using Hugging Face Transformers. Covers inference setup and integration of tools to serve the model in production.
Hugging Face integrates Active Learning into AutoNLP via Prodigy. The technique reduces annotation data volume by intelligently selecting the most informative examples. Streamlined workflow for iterative labeling and model training.
OpenAI introduces simplified fine-tuning for GPT-3 via single-command interface. The method enables model customization for specific use cases without requiring deep technical expertise.
Hugging Face releases Snowball Fight, its first ML-Agents environment. This training space simulates multi-agent snowball combat to develop and test reinforcement learning strategies.
Hugging Face releases integration guide for Transformers with IPUs (Graphcore) via Optimum library. Enables accelerated inference and training of models on specialized processors.
Hugging Face releases an interactive Data Measurements Tool to analyze datasets. The tool enables exploration of dataset characteristics, quality, and potential biases for datasets hosted on the platform.
Hugging Face and Intel optimize distributed PyTorch training using Intel technologies (Xeon processors, oneAPI). Performance improvements for multi-node CPU fine-tuning of language models.
OpenAI's API is now available without a waitlist. This wider availability is enabled by safety progress.
Hugging Face releases part 2 of a series on optimizing BERT-like model inference on modern CPUs. Focus on scaling techniques and production performance gains.
Hugging Face releases a guide to train sentence embedding models with 1 billion training pairs. The method leverages contrastive learning techniques and massive datasets to improve vector representation quality.
Hugging Face publishes a guide for fine-tuning CLIP on satellite imagery and captions. The method adapts the vision-language model to remote sensing, improving recognition of geographic objects and scenes.
OpenAI explores book summarization with human feedback to improve oversight of AI systems on tasks difficult to evaluate automatically.
Hugging Face and Graphcore partner to optimize Transformers for IPU (Intelligence Processing Units). The partnership aims to improve training and inference performance of models on Graphcore's proprietary hardware architecture.
Hugging Face introduces collaborative training of language models over the Internet, enabling multiple participants to contribute to model training without centralizing data. Decentralized approach lowering barriers to LLM fine-tuning access.
spaCy, the open-source NLP library, integrates with Hugging Face Hub. spaCy models are now accessible through the platform, streamlining sharing and collaboration for natural language processing tasks.
Hugging Face and Amazon SageMaker streamline deployment of open-source models. The integration enables developers to productionize HF models directly via SageMaker without complex setup.
OpenAI publishes an evaluation methodology for large language models trained on code. The study proposes benchmarks and criteria to measure the quality and performance of code generation models.
Sentence Transformers, models specialized in encoding sentences into vectors, are now integrated into the Hugging Face Hub. This integration simplifies access, sharing, and deployment of these models for semantic search, clustering, and text similarity tasks.
Hugging Face demonstrates few-shot learning with GPT-Neo using its Accelerated Inference API. The approach enables rapid adaptation of language models to new tasks with minimal examples, without full retraining.
Former Congressman Will Hurd joins OpenAI's board of directors. OpenAI states the appointment reflects its need for public policy expertise alongside technical expertise to develop general-purpose AI.
Amazon and Hugging Face announce a strategic partnership integrating Hugging Face models into SageMaker. The collaboration streamlines deployment, training, and inference of open-source models through AWS infrastructure.
Hugging Face tutorial on fine-tuning Wav2Vec2 for English automatic speech recognition. Uses Transformers library and public datasets. Demonstrates adapting a pre-trained model to improve ASR performance.
Hugging Face and Ray integrate Retrieval Augmented Generation (RAG) into Transformers. The tutorial demonstrates combining document retrieval with language models to improve answer accuracy on external data.
Hugging Face announces optimized integration with PyTorch/XLA for Google TPUs. The collaboration improves training and inference performance for models on TPU accelerators, with native support for Hugging Face workflows.
Hugging Face releases a method to reuse pre-trained language model checkpoints in encoder-decoder architectures. The technique improves training efficiency and reduces resources needed to build performant seq2seq models.
Hugging Face introduces block-sparse matrices to reduce size and accelerate language models. This sparse structure technique improves computational efficiency without sacrificing performance.
Hugging Face introduces Reformer, a language model pushing efficiency limits. The architecture uses locality-sensitive hashing and reversible attention to process longer sequences with reduced memory, improving performance on standard benchmarks.
OpenAI co-organizes two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind. The competitions use Procgen Benchmark and MineRL to evaluate AI agents on generalization and reinforcement learning.
OpenAI standardizes on PyTorch as its official deep learning framework across all operations.
OpenAI publishes research on model-based control combining online planning with offline learning. The method improves exploration and reinforcement learning efficiency by using predictive models to guide decision-making.
OpenAI proposes iterated amplification, an AI safety technique enabling specification of complex behaviors by decomposing tasks into simpler sub-tasks, without labeled data or reward functions. Experiments remain limited to simple algorithmic domains.
OpenAI publishes variational option discovery algorithms for reinforcement learning. These methods enable agents to automatically learn reusable sub-tasks without explicit supervision, improving efficiency and generalization.
OpenAI publishes research on learning policy representations in multiagent systems. The work explores how agents can learn efficient representations to coordinate and adapt their behaviors in complex environments.
OpenAI introduces a new benchmark to evaluate generalization in reinforcement learning. The tool measures RL agents' ability to adapt to novel and varied environments beyond their training data.
OpenAI launches a transfer learning contest measuring a reinforcement learning algorithm's ability to generalize from previous experience.
OpenAI publishes a method to improve GANs using optimal transport. The technique reduces training instability and improves generated image quality by leveraging Wasserstein distances.
OpenAI publishes analysis on first-order meta-learning algorithms. The article explores theoretical and practical foundations of optimization methods that enable models to learn to learn quickly from few examples.
OpenAI launches a scholarship program for 6–10 individuals from underrepresented groups. Recipients will study deep learning full-time for 3 months and open-source a project.
OpenAI explores meta-reinforcement learning to improve agents' ability to explore efficiently. The article examines how models can learn generalizable exploration strategies rather than being pre-programmed.
OpenAI built a system for entity disambiguation using a neural network to classify words into ~100 automatically-discovered types (non-exclusive categories).