14 articles
Hugging Face implements speculative decoding to accelerate Whisper inference by 2x. The technique uses a lightweight model to generate candidate tokens, validated by the full model in parallel, reducing latency without quality loss.
2023 marked the emergence of open-source LLMs as viable alternatives to proprietary models. Llama, Mistral and others democratized access to large language models, reducing dependence on OpenAI and Google.
OpenAI releases governance practices for agentic AI systems, covering monitoring, operational boundaries, and control mechanisms. The document proposes frameworks for safely deploying autonomous agents in production.
OpenAI launches $10M in grants for technical research on alignment and safety of superhuman AI systems, covering weak-to-strong generalization, interpretability, and scalable oversight.
Summer Health uses OpenAI's GPT to improve accuracy of pediatric visit notes. The system automates medical documentation during pediatrician visits, reducing errors and administrative time.
OpenAI explores leveraging deep learning's generalization properties to control strong models with weak supervisors. New research direction for superalignment with promising initial results.
OpenAI and Axel Springer announce a partnership to integrate journalism into AI technologies. Axel Springer becomes the first global publishing house to sign such a deal with OpenAI.
Hugging Face explains Mixture of Experts (MoE) architecture: a mechanism where a router directs inputs to specialized experts instead of using all parameters. Reduces latency and increases model capacity without proportional computational overhead.
Mixtral, a state-of-the-art Mixture of Experts (MoE) model, is now available on Hugging Face. The model delivers superior performance with improved computational efficiency through its specialized expert architecture.
Hugging Face introduces SetFitABSA, a few-shot method for aspect-based sentiment analysis. SetFit enables efficient model training with minimal examples, without fine-tuning the base model weights.
Hugging Face and AMD announce native integration to accelerate LLMs on AMD GPUs. Models run out-of-the-box without manual optimization, supporting RDNA and CDNA architectures.
Hugging Face and NVIDIA release Optimum-NVIDIA, a library accelerating LLM inference with a single line of code. Native integration of NVIDIA optimizations (TensorRT-LLM, cuDNN) reduces latency and increases throughput without requiring code changes.
Hugging Face optimized LoRA inference to achieve 300% speed improvement. Optimizations target cold boot and reduce overall latency for low-rank adapters.
Hugging Face provides a detailed analysis of the DROP benchmark in the Open LLM Leaderboard, which evaluates reading comprehension and information extraction. The article examines model performance on this specific task and the challenges it presents.