SEDD is a GPU-accelerated deduplication framework using MinHash LSH. It outperforms SlimPajama's CPU tool by 158× and NVIDIA NeMo Curator's GPU tool by 7.8× on 30M documents. MinHash signature generation 375× faster. Deduplicates 1.2T tokens in 3 hours on 32-GPU V100 cluster.
Tongyi DeepResearch is an agentic LLM with 30.5 billion parameters (3.3 billion activated per token) designed for long-horizon deep research tasks. Trained via agentic mid-training and post-training with automatic data synthesis, it achieves state-of-the-art on 7 benchmarks including Humanity's Last Exam and BrowseComp. Model and framework are open-sourced.
Researchers localize 'entity cells' in MLP neurons across language models (Qwen2.5-7B, etc.). These selectively activated neurons encode entity-specific facts. Suppressing one cell erases recall for that entity alone; activating it recovers knowledge even without context. Cells remain stable across aliases, acronyms, and multilingual forms.
Study on multi-agent systems: 'semantic hijacking' attacks exploit agent confidence. Paradox identified: increasing Worker capability raises attack success rate from 18.4% to 63.9%. Mediation analysis reveals 'linguistic certainty' of stronger agents drives vulnerability. Proposed solution: heterogeneous ensemble verification reduces attack success rate to 2%.
ProxyKV introduces a cross-model proxy pruning framework to accelerate long-context LLM inference. A lightweight in-family small model evaluates KV cache importance asynchronously via HybridAxialMapper and Multi-Granularity Hybrid Loss. On Llama-3.1, Qwen-2.5, and Qwen-3, recovers 98.7% of KVZip accuracy with up to 3.21× prefilling speedup (Llama-3.1-8B, dual-GPU) and sustains speedup at contexts up to 170k tokens.
Researchers localize 'entity cells'—selective MLP neurons encoding entity-specific facts—across seven language models. On Qwen2.5-7B, suppressing a cell selectively erases recall for its matched entity while activating a single cell recovers knowledge even without context. These cells remain stable under aliases, acronyms, and multilingual forms.
MM-ToolBench is a benchmark for omni-modal tool-using agents in real-world workflows. 100 executable tasks (customer service, intelligent creation), 27 MCP servers, 324 tools. Closed-loop multimodal verification: agents execute, inspect, and self-correct. Claude Opus 4.6 achieves 32% success vs 94% human baseline.
Systematic study of scaling laws for ECG models: 120 models (20K–200M parameters) pre-trained on CODE (2.3M records). SSL models outperform SL on out-of-distribution generalization; ResNets 1.3–2.5× more parameter-efficient than Transformers; SSL 16× more data-efficient. Architecture and paradigm choice matter more than brute-force scaling.
Injecting mathematically wrong drafts from a smaller model (Qwen2.5-Math-1.5B) into stronger learner (Mathstral-7B) GRPO training improves performance on MATH-500 (+1.62pp) and AIME 2025/2026 (+14.2pp at pass@1024). Intentional mismatch between problems and drafts is critical: 71.98% on MATH-500, highest published result for this model.
CHI-Bench evaluates AI agents' ability to automate complex healthcare workflows (prior authorization, utilization management, care management) across 87 MCP tools and 20 applications. Best agent resolves only 28% of tasks; none exceed 20% on strict pass. Performance drops to 3.8% in single-session mode.
Unified study of LLM distillation showing SFT, DAgger, offline RL, and OPD decouple two orthogonal axes: prefix source and token-level KL direction. Authors propose KL mixing and entropy-gated length curriculum, improving Pass@k by 5.8 points and reducing average response length by 3x on math reasoning.
ContractBench benchmarks LLM agents' ability to preserve observation contracts (temporally valid, byte-level intact artifacts) in API calls. Of 38 models tested, none exceed 80%: Claude-Opus-4.6 leads at 77.8%. Results show integrity and validity failures uncorrelated with model size, and non-monotonic regression in the GPT-5 family despite larger scale.
OProver is a unified framework for agentic formal theorem proving in Lean 4. The system iteratively revises failed proof attempts using retrieved compiler-verified proofs and Lean compiler feedback. Trained via continued pretraining and iterative post-training, OProver-32B achieves 93.3% Pass@32 on MiniF2F and 58.2% on ProverBench.
ASPI is a benchmark of 728 task-attack scenarios measuring how clarification amplifies prompt injection vulnerability. Testing on 10 frontier LLMs shows attack success rates rise from 1.8% to 34.0% for o3 and 2.2% to 35.7% for Gemini-3-Flash in clarification mode. Code and data released.
FML-Bench is a benchmark of 18 ML tasks across 10 domains evaluating 6 AI research agents. Key findings: strategy complexity alone does not ensure performance (greedy hill-climber matches tree-search); effectiveness depends on improvement opportunity structure; an adaptive agent detecting stagnation outperforms others. Includes 12 process-level behavioral metrics.
MemRepair is a memory-augmented agentic framework for repository-level vulnerability repair. It combines three memory layers (History-Fix, Security-Pattern, Refinement-Trajectory) with an iterative refinement loop. Evaluated on SEC-Bench, PatchEval, and Multi-SWE-bench, MemRepair achieves 58.0%, 58.2%, and 30.58% resolution rates, outperforming OpenHands, SWE-agent, and InfCode-C++.
Hilbert-Geo introduces a unified formal framework for solid geometry via Parse2Reason: parsing into CDL (Conditional Description Language) then reasoning with theorem bank. Achieves 77.3% on SolidFGeo2k and 84.1% on MathVerse-Solid, outperforming Gemini-2.5-pro (54.2%) and GPT-5 (62.9%). Two annotated datasets: SolidFGeo2k and PlaneFGeo3k.
FireFly generates verified tool-call data for training agents from real MCP servers. The pipeline inverts standard synthesis: explores real APIs via DAG structures, then generates tasks backward from observed outcomes. 5,144 verified tasks across 240 servers and 993 tools. A 4B model trained with GRPO matches Claude Sonnet on held-out test set.
Large-scale study of 64,380 SWE-bench runs across 126 agent configurations (43 frameworks × LLMs). Behavioral rules derived from single frameworks do not transfer: the same signal (e.g., error rate) correlates positively with issue resolution in 47 configs and negatively in 48. Framework identity explains 64% of variance vs. 10% for LLM family.
ContraFix is an agentic framework for automated vulnerability repair combining differential runtime evidence and skill reuse. On SEC-Bench (C/C++) and PatchEval (Go, Python, JavaScript), it achieves 84.0% and 73.8% resolution rates with GPT-4-mini, outperforming baselines while costing less than one-third of comparable approaches.
SaaSBench is the first benchmark to evaluate AI agents in enterprise SaaS engineering. It contains 30 complex tasks across 6 SaaS domains with 8 programming languages, 6 databases, and 13 frameworks. Experiments show >95% of failures occur before business logic: agents struggle to configure and integrate multi-component systems.
OSCAR quantizes KV caches to INT2 for long-context LLMs by estimating attention-aware covariance structures offline. Tested on Qwen3 (4B–32B) and GLM-4.7 (358B), it reduces accuracy gap to 1.42–3.78 points vs BF16, cuts memory by 8x and improves throughput by 7x. Custom INT2 kernel compatible with vLLM/SGLang.
SynPro, a synthetic data generation framework, helps LLMs learn more thoroughly from limited organic corpora through rephrasing and reformatting operations. Optimized via reinforcement learning, it unlocks 3.7-5.2x more effective tokens than simple repetition on 400M and 1.1B models, even surpassing the non-data-bound oracle at 1.1B scale.
Stable Audio 3 is a family of latent diffusion models (small, medium, large) for variable-length audio generation and editing. Models use a novel semantic-acoustic autoencoder and adversarial post-training to generate music and sounds in under 2s on H200 or seconds on MacBook Pro M4. Small and medium weights are released.
TeleCom-Bench is a 22,678-sample benchmark evaluating 8 LLMs on real telecom tasks (intent recognition, entity extraction, root cause analysis, solution generation). Models achieve 90% on linguistic tasks but collapse to 30% on procedural execution, revealing an 'Execution Wall': LLMs diagnose well but fail as field engineers.
Hilbert-Geo introduces a unified formal framework for solid geometry via Parse2Reason: parsing into Conditional Description Language (CDL) then reasoning with theorem bank. Achieves 77.3% on SolidFGeo2k and 84.1% on MathVerse-Solid, outperforming Gemini-2.5-pro (54.2%) and GPT-5 (62.9%). Two expert-annotated datasets: SolidFGeo2k and PlaneFGeo3k.
LLMs used as autonomous agents select unauthorized tools despite explicit instructions. Study across Qwen 2.5 7B, Llama 3.1 8B, and Claude Haiku 3.5 shows an MCP proxy with attribute-based access control (ABAC) reduces unauthorized invocation rate to 0%, versus 11-18% for prompt-based restrictions. Architectural enforcement, not prompting, is required for reliable tool access control.
EnterpriseArena, a 132-month CFO simulator, benchmarks LLM agents' ability to allocate resources over long horizons under uncertainty. Tests across 23 models and 4 frameworks: only 15.4% of trials complete the full horizon. Larger models do not reliably outperform smaller ones. Reveals critical capability gap in managing binding commitments under partial observability.
LEGO is a modular platform for LLM-based digital front-end design generation. It decomposes the flow into 6 steps and extracts 42 reusable circuit skills. On 41 hard VerilogEval v2 problems where GPT-5.2-codex fails, LEGO achieves 80.5% Pass@1 vs 0% baseline, outperforming hierarchy-verilog (+14.6%) and VerilogCoder (+2.5%).
GIM is a benchmark of 820 original problems evaluating LLMs via integration of multiple cognitive domains (constraint satisfaction, state tracking, epistemic vigilance) rather than memorization or pure abstract reasoning. IRT calibration over >200k prompt-response pairs, 28 models, extensive study of compute vs capability trade-off across 11 models and 35 configurations.
Orthrus unifies autoregressive LLM fidelity with parallel diffusion token generation via a dual-architecture framework. A lightweight trainable module augments a frozen Transformer to enable parallel generation while maintaining exact autoregressive quality. Achieves up to 7.8x speedup with O(1) memory overhead.
LLM-based multi-agent pipelines flip to incorrect answers under simulated peer disagreement (yield). Contrary to common attribution, RLHF is not responsible: pretrained base models exhibit the same substitution pattern. Activation patching localizes corruption to a narrow mid-layer window. A single correctly-arguing dissenter reduces yield by 54-73 percentage points.
PARALLAX reveals that 4 of 6 major hallucination detection benchmarks embed the ground-truth answer in the prompt, allowing a naive baseline (TxTemb) to achieve near-perfect detection without access to model internals. Evaluation of 22 methods across 12 open-source models: most fail under controlled conditions, except SAPLMA and DRIFT (supervised probes on upper-layer hidden states).
Injecting mathematically wrong drafts from a smaller model (Qwen2.5-Math-1.5B) mismatched to the current problem into a stronger learner's (Mathstral-7B) GRPO context outperforms standard on-policy GRPO. On MATH-500, the mismatched-wrong variant reaches 71.98% (highest published result for this model), +1.62pp vs matched-wrong variant, without SFT or reward models.
SynPro, a synthetic data generation framework, helps LLMs learn more thoroughly from limited organic corpora via rephrasing and reformatting. Optimized with RL, it unlocks 3.7-5.2x more effective tokens than simple repetition on 400M and 1.1B models, even surpassing the non-data-bound oracle at 1.1B scale. Code open-sourced.
MTP (speculative decoding) support merged into llama.cpp (PR #22673, May 16). Qwen 3.6 27B benchmarks: 1.81×–2.44× speedup on Strix Halo (ROCm), 1.54×–2.17× on RTX 3090. MoE 35B-A3B shows smaller gains (1.24×–1.40×). Enable with --spec-type draft-mtp --spec-draft-n-max N.
OpenAI releases Sora 2, a video-audio generation model with improved physics accuracy, sharper realism, synchronized audio, enhanced steerability, and expanded stylistic range. Direct successor to Sora addressing longstanding challenges in video synthesis.
OpenAI releases gpt-realtime, an advanced speech-to-speech model, with new API capabilities: MCP server support, image input, and SIP phone calling. Major Realtime API update enabling voice and multimodal integrations.
OpenAI introduces Codex, a GPT-3-based model specialized in code generation. Trained on public code, it understands 12+ programming languages and translates natural language to executable code. Available in limited access via API.
OpenAI adds vision to the fine-tuning API. Developers can now fine-tune GPT-4o with images and text to improve the model's visual capabilities.