Meta releases CyberSecEval 2, a comprehensive evaluation framework measuring cybersecurity risks and capabilities of LLMs. The tool tests malicious code generation, vulnerability exploitation, and attack defense across models including Llama.
Hugging Face releases Falcon 2, an 11B parameter model trained on 5000B+ tokens across 11 languages. Available as both text and multimodal vision language model (VLM).
OpenAI and News Corp announce a multi-year global partnership. Premium journalism from News Corp will enrich OpenAI's generative AI products and platforms.
Ilya Sutskever leaves OpenAI after 9 years. Jakub Pachocki, former VP Research, becomes Chief Scientist. Sutskever co-founded OpenAI and led research until recently.
Google releases PaliGemma, an open-source vision-language model built on Gemma 2B. The model combines an image encoder and text decoder for multilingual visual understanding tasks. Weights available on Hugging Face.
Hugging Face releases Transformers Agents 2.0, a platform for building autonomous AI agents. The system enables models to make decisions, use external tools, and manage complex workflows without human intervention.
OpenAI releases a formal specification defining the expected behavior of its models. This document establishes safety, honesty, and helpfulness principles guiding OpenAI model development and evaluation.
Hugging Face releases StarCoder2-Instruct, a code generation model with full self-alignment transparency. The model employs self-alignment without proprietary data, with all process details publicly documented.
OpenAI announces general availability of GPT-4, GPT-3.5 Turbo, DALL·E and Whisper APIs. Older Completions API models will be deprecated and retired at the beginning of 2024.
Hugging Face launches a leaderboard for models using open chain-of-thought (CoT). This tool evaluates and ranks model performance on complex reasoning tasks with transparent intermediate steps.
Hugging Face launches an open-source leaderboard to evaluate LLMs in healthcare. The platform benchmarks models on medical-specific tasks, enabling comparison of performance and transparency across AI medical solutions.
Meta releases Llama 3, a new open-source large language model. The model comes in multiple sizes and shows competitive performance on standard benchmarks.
Hugging Face introduces LiveCodeBench leaderboard for code LLM evaluation. It provides holistic, contamination-free benchmarking with regularly updated test sets to prevent model overfitting on evaluation data.
Hugging Face enables privacy-preserving inferences on its endpoints through homomorphic encryption. Models run on encrypted data without server-side decryption, ensuring end-to-end confidentiality.
OpenAI opens its first office in Asia in Japan and releases a GPT-4 custom model optimized for the Japanese language.
Hugging Face integrates thousands of open-source LLMs into Google Vertex AI Model Garden. Users access Llama, Mistral, Qwen and other models through a unified interface with fine-tuning and deployment support.
Hugging Face launches serverless GPU inference for users. The service enables running AI models without infrastructure management, with pay-as-you-go billing. Available through Inference API and Spaces.
Hugging Face introduces binary and scalar embedding quantization to accelerate and reduce costs for vector retrieval. The method compresses dense representations while maintaining information retrieval quality.
GaLore reduces GPU memory requirements for large model training through gradient projection decomposition. Enables training of 7B-70B models on consumer hardware (RTX 4090) with speedups up to 65% versus standard methods.
Hugging Face introduces Cosmopedia, a method for generating large-scale synthetic data for LLM pre-training. The dataset contains 30 billion tokens generated via Mixtral 8x7B, covering mathematics, science, and programming. Models trained on this data achieve performance comparable to models pre-trained on natural data.
Hugging Face releases Quanto, a PyTorch quantization backend integrated into Optimum. The tool reduces model size and accelerates inference through quantization, compatible with popular transformer models.
Hugging Face introduces ConTextual, a benchmark to evaluate how well multimodal models jointly reason over text and images in text-rich scenes. It measures fine-grained understanding of models when handling text embedded within images.
Hugging Face launches TTS Arena, a benchmarking platform to evaluate text-to-speech models in real-world conditions. Users compare audio outputs from different TTS models through voting, creating a dynamic ranking based on human preferences.
Google releases Gemma, a new family of open-source language models based on Gemini architecture. Available in 2B and 7B versions, Gemma is optimized for efficiency and runs on consumer hardware.
Hugging Face releases a Messages API compatible with OpenAI for open-source models. The interface unifies access to Claude, Llama, Mistral and other LLMs through a standardized endpoint, reducing friction for migration from OpenAI.
Hugging Face releases NPHardEval Leaderboard, a benchmark assessing LLM reasoning abilities through NP-hard problems and dynamic updates. The leaderboard ranks models by performance on tasks of increasing complexity.
Hugging Face deploys Text Generation Inference (TGI) on AWS Inferentia2. This integration optimizes language model inference on Amazon's specialized hardware, reducing latency and costs for production deployments.
Hugging Face optimizes StarCoder using Optimum Intel on Xeon processors with Q8/Q4 quantization and speculative decoding. Techniques reduce latency and increase inference throughput for code generation models.
Hugging Face launches an open leaderboard to measure hallucinations in LLMs. The initiative provides standardized evaluation methodology and public ranking of models by their propensity to generate factually inaccurate content.
OpenAI releases new embedding models and API updates. The models deliver improved performance and reduced costs for vector search and RAG applications.
OpenAI launches ChatGPT Team, a new plan for teams of all sizes providing a secure, collaborative workspace to use ChatGPT at work.
OpenAI launches the GPT Store, a platform for creators to share and monetize custom GPTs. ChatGPT Plus, Team, and Enterprise users can access thousands of community-built GPTs, with revenue shared between OpenAI and creators.
Hugging Face and Unsloth optimize LLM fine-tuning to be 2x faster. The collaboration integrates Unsloth into TRL (Transformers Reinforcement Learning), reducing memory consumption and accelerating training without quality loss.
Hugging Face introduces aMUSEd, an efficient text-to-image generation model based on MUSE architecture. The model uses image tokenization and a decoder to generate quality images with reduced latency compared to standard diffusion models.
Hugging Face implements speculative decoding to accelerate Whisper inference by 2x. The technique uses a lightweight model to generate candidate tokens, validated by the full model in parallel, reducing latency without quality loss.
2023 marked the emergence of open-source LLMs as viable alternatives to proprietary models. Llama, Mistral and others democratized access to large language models, reducing dependence on OpenAI and Google.
OpenAI launches $10M in grants for technical research on alignment and safety of superhuman AI systems, covering weak-to-strong generalization, interpretability, and scalable oversight.
Mixtral, a state-of-the-art Mixture of Experts (MoE) model, is now available on Hugging Face. The model delivers superior performance with improved computational efficiency through its specialized expert architecture.
Hugging Face and NVIDIA release Optimum-NVIDIA, a library accelerating LLM inference with a single line of code. Native integration of NVIDIA optimizations (TensorRT-LLM, cuDNN) reduces latency and increases throughput without requiring code changes.
Hugging Face optimized LoRA inference to achieve 300% speed improvement. Optimizations target cold boot and reduce overall latency for low-rank adapters.