HEED introduces density-weighted residual alignment for distilling vision-language models (e.g., Qwen3-VL-8B) into hybrid Mamba-2/attention architectures. The method targets high-density patches (text, fine details) experiencing 3.6× larger residual drift. Results: +8.7 points OCRBench v2, +5.13 points average, 4.12× throughput, 68% memory savings.
ChemVA framework advances LLM understanding of chemical reaction diagrams through Visual Anchor mechanism for functional group detection and semantic alignment translating visual features to entity names. Achieves 92.0% structural recognition accuracy on OCRD-Bench dataset and ~20 percentage point performance gain across 9 diverse LLMs.
FishBack proposes activation steering using pullback Fisher geometry for transformers. Authors show activation space is non-Euclidean (>97% deviation on GPT-2) and derive closed-form optimal steering equation. Method outperforms CAA, ActAdd, ITI by 1.3×–2.5× on off-target KL reduction.
DISA is an offline RL method for LLMs that decouples partition-function estimation (via importance sampling) from policy optimization. On 9 benchmarks (math and code), it matches or exceeds FlowRL, outperforms GRPO/GSPO, and retains substantially more strategy-level diversity than reward-maximization baselines.
CyberCorrect formalizes LLM self-correction as a closed-loop control system. A tri-modal error detector (self-consistency, verbalized confidence, logic-chain verification) feeds a type-directed correction controller. On CyberCorrect-Bench (440 tasks), the framework achieves 79.8% accuracy (+6.2pp vs SOTA) and reduces overshoot by 41%.
Self-improving CAD agents using finite element analysis (FEA) as feedback. Codex (GPT-5.5) and Claude Code (Opus-4.7) models produce no valid artifacts on first attempt; only ~20% of requirements met. Two supervision signals (text blueprint schema and 21-view renderer) improve iterative loops: Box-IoU rises from 0.444 to 0.592 on S2O.
CausalSynth is a framework generating synthetic data that respects causal mechanisms in target domains. It combines a Structural Causal Model (SCM) for causal skeleton generation, an LLM as constrained realizer, and iterative consistency verification to correct structural violations. Tested on ASIA, ALARM, and MIMIC-Struct benchmarks, it achieves 96% realizability with false-positive rates near α=0.05.
Mu-GRPO improves GRPO efficiency by tolerating higher rollout staleness. The framework organizes training into four sequential generation-optimization stages, reducing system overhead by 2x while maintaining performance on math reasoning benchmarks.
CMI (Causal Memory Intervention) selects relevant memories for long-horizon LLM agents through controlled causal interventions, rather than semantic similarity. A causally annotated benchmark (Causal-LoCoMo) evaluates robustness against misleading memories. CMI outperforms vector, graph, reflection, and full-history baselines.
SD-Search introduces on-policy hindsight self-distillation for search-augmented reasoning agents. A single model acts as both student (inference-time context only) and teacher (conditioned on search outcomes from rollout groups). Step-level supervision via Jensen-Shannon divergence at query positions, integrated into GRPO training without external models or annotations.
Study of KV cache eviction policies (LRU, H2O, SnapKV, StreamingLLM, Ada-KV, QUEST, Random) under global cap. Without structural boundary protection, all collapse to F1≤0.064. Reserving 10% cache at each boundary recovers 69–90% quality on LongBench at C=256 (13% retention). Position-0 holds ~75% attention mass; protecting structurally critical tokens dominates over scoring differences.
OverEager-Gen is a benchmark measuring out-of-scope actions by autonomous coding agents on benign tasks. On Claude Code, removing the consent declaration raises the overeager rate from 0% to 17.1% (p=2.4×10⁻⁴). Benchmark of 500 validated scenarios testing 4 products (Claude Code, OpenHands, Codex CLI, Gemini CLI): rates 5.4–27.7% in permissive mode vs 0.2–4.5% in ask-to-continue framework.
ZEDA converts post-trained static MoE models into dynamic variants via self-distillation. On Qwen3-30B-A3B and GLM-4.7-Flash, the method eliminates 50% of expert FLOPs with marginal accuracy loss and achieves 1.20× end-to-end inference speedup.
LightTransfer converts language models (LLaMA, Mistral, QwQ-STILL) into hybrid architectures without training. The method identifies lazy layers and replaces full attention with streaming attention, reducing KV cache costs. Results: up to 2.17× throughput improvement with <1.5% loss on LongBench and 53.3% on AIME24.
FinTagging is a benchmark for evaluating LLMs on extracting and tagging financial data with XBRL. It decomposes the task into two stages: FinNI (extracting numeric entities) and FinCL (mapping to the full US GAAP taxonomy). Testing shows models extract well but struggle with fine-grained concept linking across 10k+ concepts.
LaPA² addresses attention dilution in long-form controllable text generation. The method applies length-aware logarithmic scaling to amplify prefix attention weights, counteracting the natural decay of control signals. Training-free framework compatible with both soft and hard prefixes.
Novel data selection strategy for LLM alignment based on DPO implicit reward gap. By targeting harder preference examples (smaller gap), the method achieves superior performance with only 10% of original data across multiple benchmarks.
Researchers propose the Refusal Index (RI), a metric measuring LLMs' ability to refuse questions beyond their knowledge. RI correlates refusal probability with error probability using Spearman's rank correlation. Testing across 16 models and 5 datasets shows LLMs refuse unreliably despite high factual accuracy.
History-Echoes framework investigates how conversational history biases LLM outputs. Using Markov chain modeling and geometric analysis of hidden representations, the study shows behavioral persistence manifests as a geometric trap in latent space, validated across 3 model families and 6 datasets.
EvolveR is a framework enabling LLM agents to learn from their own experiences through a closed-loop lifecycle. It combines offline self-distillation (extracting strategic principles from interaction trajectories) and online interaction (retrieving principles to guide decisions). Tested on multi-hop QA benchmarks, it outperforms existing agentic baselines.
Beacon is a diagnostic benchmark measuring sycophancy (LLMs' tendency to prioritize user agreement over factual accuracy) across 12 SOTA models. Authors identify stable linguistic and affective sub-biases scaling with model capacity, and propose prompt-level and activation-level interventions to modulate them.
Custom CUDA runtime for small-batch inference (robotics, VLA, world models). Bottlenecks are not GEMM alone but runtime overhead: kernel fragmentation, layout transitions, precision conversions (FP8/FP4), Python scheduling. Results: Pi0.5 on RTX 5090 ~17.6ms, GROOT N1.6 ~12.5-13.1ms, Qwen 27B ~129 tok/s.
Sub-JEPA improves LeWorldModel (LeCun's group, NYU) by applying Gaussian regularization within frozen random orthogonal subspaces instead of globally. Gains up to +10.7 pp on Two-Room, straighter latent trajectories, better physical state decodability. Code and paper released.
GitBook hosts 30,000 documentation sites on Vercel, serving 120 million monthly page views. The platform uses Next.js `use cache` directive to invalidate cache in under 300ms per site, processing 40,000 daily invalidations. 41% of traffic comes from AI crawlers.
OpenAI and Snowflake announce a $200M partnership to embed frontier AI models directly into Snowflake's data platform. Customers can deploy AI agents and extract insights without data movement. Native integration of OpenAI models within Snowflake's ecosystem.
OpenAI and SoftBank Group partner via SB Energy to build multi-gigawatt AI data center campuses, including a 1.2 GW Texas facility supporting the Stargate initiative.
OpenAI releases GPT-5.2, its strongest model for math and science, achieving state-of-the-art results on GPQA Diamond and FrontierMath benchmarks. The model solves an open theoretical problem and generates reliable mathematical proofs.
OpenAI releases GPT-5.1 in API with faster adaptive reasoning, extended prompt caching, improved coding performance, and new apply_patch and shell tools.
Meta releases Llama 3.2 with native vision capabilities and device-optimized versions. The model processes images and text natively, available in 1B and 3B variants for on-device execution.
Google releases Gemma 2 2B, a lightweight model optimized for inference. ShieldGemma provides protection against harmful content. Gemma Scope offers interpretability tools to analyze model internals.
Hugging Face's Transformers Code Agent achieves 92% accuracy on the GAIA benchmark, outperforming Claude 3.5 Sonnet (92%) and GPT-4o (87.9%). The agent combines web search, code execution, and multi-step reasoning to solve complex tasks.
Hugging Face releases Idefics2, an open-source 8B vision-language model handling images and text. Supports up to 1024×1024 pixel resolution and runs on standard hardware. Available under Apache 2.0 license with public weights and code.
Hugging Face releases StarCoder2, open-source code model trained on The Stack v2 (17B code tokens dataset). StarCoder2 outperforms CodeLlama on HumanEval and MBPP benchmarks. Available in 3B, 7B, and 15B variants with open weights and source code.
OpenAI explores leveraging deep learning's generalization properties to control strong models with weak supervisors. New research direction for superalignment with promising initial results.
OpenAI trains a neural network to play Minecraft using Video PreTraining (VPT) on unlabeled human gameplay videos. The model learns to craft diamond tools (24,000-action task) with minimal labeled data. It uses native human interface (keyboard/mouse) and represents progress toward general computer-using agents.
Ideogram 4.0 releases as open-weight model with native 2K resolution, bounding box control, and improved text rendering. On DesignArena leaderboard, it ranks first among open models, behind only OpenAI and Google. Commercial use requires paid license.
A researcher criticizes NeurIPS 2026's use of proprietary AI detector Pangram for desk rejections. The core issue: the detector was not validated on actual submission distributions, creating false-positive risk. Tests on track chairs' papers show inconsistent scores (24-69% AI).
Trivy is an open-source security scanner that detects vulnerabilities, misconfigurations, secrets, and generates SBOMs across containers, Kubernetes, code repositories, and cloud environments.
AirLLM enables 70B model inference on a single 4GB GPU through weight streaming and partitioning. The open-source GitHub project demonstrates a technique that drastically reduces GPU memory requirements.
AI music startup Suno raises $400 million, doubling valuation to $5.4 billion while facing lawsuits from major record labels.