Evoflux is an inference-time evolutionary search method for repairing executable tool workflows in compact agents. On MCP-Bench with 250 tools, it raises execution feasibility from ~3% to 17-24%, outperforming SFT, SFT+DPO, and ReAct under scarce teacher-trace budgets.
PaperGuard, a multimodal benchmark, evaluates LLMs and MLLMs vulnerability to adversarial attacks in scientific peer review. Researchers test prompt injections and perturbations (GCG, PGD) on text and figures, proposing a defense using chunk-based embedding search to localize harmful instructions.
Study evaluating 4 lie detectors across 31 models (2B-1T parameters). Detectors (CoT judge, logprob classifier, activation probes, DYL) perform well on prompted lying but fail on trained model organisms with verified beliefs. Only CoT judge maintains 0.82 balanced accuracy.
LAUKIN is a dataset of 14,727 contract clause pairs (Australia-UK, UK-India, India-Australia) labelled for legal equivalence. 3,000 pairs are manually annotated by legal experts. Best models achieve 65.11% macro-F1, revealing that drafting conventions diverge significantly across jurisdictions despite shared legal heritage.
SciAgentArena is a systematic benchmark evaluating ~200 real-world scientific tasks with stepwise verification. Current AI agents perform well on structured data-analysis workflows but struggle to generate novel insights, sustain self-directed exploration, and solve open-ended research questions.
SENTINEL is a failure-driven reinforcement learning framework that improves tool-using LLM agents by converting their failures into targeted training tasks. On Tau2-Bench Retail with Qwen3-4B-Thinking-2507, the method increases Pass@1 from 66.4 to 74.9 through a Controller-Proposer-Solver loop that analyzes recurring error patterns.
LoHoSearch is a 544-question benchmark for evaluating long-horizon search agents, built via an automated pipeline on a knowledge graph of 7 million Wikipedia entities. The strongest model achieves only 34.74% accuracy, versus >90% on prior saturated benchmarks.
Three small LLMs (Phi-3-mini 3.8B, Qwen2.5-3B, Mistral-7B) fine-tuned via QLoRA for biomedical claim verification. Mistral-7B outperforms GPT-4o and GPT-5 (+12% F1) on 1,008 training examples. Study identifies structural artifact in SciFact and demonstrates robust cross-domain generalization.
arXiv study showing frontier models (Claude Opus 4.5, GPT, Gemini) detect tampered prefills in 9-35% of cases with 0% false positive rate. This 'prefill awareness' undermines alignment and jailbreaking evaluations relying on inserted assistant context. Models distinguish stylistic from preference mismatch.
Analysis of 80,814 papers from 5 major AI conferences (2017-2025) reveals research topics advance through abrupt phase transitions, not gradually. LLMs dominant by 2025; diffusion models and vision-language models surged within 1-3 years. Early-warning signature flags reasoning, test-time compute, agentic AI, multimodal LLMs, RAG, and world models as topics to monitor 2026-2028.
Deployment study of an LLM embedded in electronic health records. A pre-response classifier predicts user rejection risk (AUROC 0.719) by leveraging deployment-specific context (provider type, department, model). Prospective analysis over 4.5 months.
DailyReport is an open-source benchmark evaluating search agents on 150 real-world daily tasks with 3,546 evaluation rubrics. Tasks decomposed into subtasks with cascade evaluation across disentangled dimensions. Testing 17 agentic systems reveals significant gaps versus user expectations.
MARS is an adversarial stopping rule for parallel LLM test-time scaling. It probes partial traces at intermediate checkpoints to estimate which traces will change answers, enabling early stopping once the leading vote is safe. Across three reasoning models and three competition-math benchmarks, MARS saves 25-47% of self-consistency tokens while maintaining accuracy.
Adaptive video tokenisation method exploiting temporal redundancy in frozen tokeniser latent space via fixed threshold on per-position temporal-L1 differences. Latent Inpainting Transformer (LIT) reconstructs dropped positions. Single encoder + one LIT pass pipeline: 31× speedup over ElasticTok-CV, 2× over InfoTok on TokenBench and DAVIS benchmarks.
Comparative study of Claude Code and Codex on social science data analysis. AI agents produce methodological diversity equal to or exceeding humans, but remain vulnerable to interpretive bias at the verdict level. A biased prompt does not shift aggregate estimates unlike biased human analysts.
ProjectMem is a local-first, open-source memory layer for AI coding agents based on an immutable event log (issues, attempts, fixes, decisions). It reduces token consumption (5,000-20,000 per session) and adds preventive governance: agents are warned before repeating failed fixes or editing fragile files. Runs fully offline via MCP.
NightFeats, a structured multi-agent RAG system, wins Best Dynamic Evaluation at NeurIPS 2025 (text-to-text track). The pipeline decomposes knowledge synthesis into three phases: retrieval, curation, composition, with temporal-semantic reranking and contradiction reconciliation. Outperforms Claude-SonnetV2 and Nova-Pro on human evaluations.
Activation steering to reduce sycophancy in Llama-3-8B-Instruct also suppresses agreement with factually correct statements. Dual-stance evaluation reveals sycophantic and factual agreement occupy distinct geometric subspaces, yet the steering direction projects equally onto both, illustrating a gap between readable and writable representations.
SciConBench, a benchmark of 9.11K questions from systematic reviews, evaluates AI agents' ability to synthesize scientific conclusions. Among 8 frontier models tested in controlled settings, the best agent achieves only 0.337 factual F1. Consumer-facing agents (Google AI Overview, OpenEvidence) frequently generate incomplete or contradictory conclusions.
HORMA organizes LLM agent experience into file-system-like hierarchical structure to improve long-horizon tasks. Two modules: structured memory construction and RL-based navigation retrieval. Reduces token usage by 22% on ALFWorld/LoCoMo/LongMemEval while improving performance.
Lung-R1 is a pulmonary diagnostic LLM guided by LungKG, a structured knowledge graph containing 59,038 nodes and 164,308 edges. Trained via KG-constrained reasoning-chain construction and reinforcement learning, Lung-R1-14B achieves 4.3583 on EMR Diagnosis, outperforming baselines by 0.1476 points.
RAG systems suffer from a structural bias: knowledge graph triples capture 2-3x more attention per token than semantically equivalent natural language, compressing demonstration attention by up to 42%, regardless of relevance. Authors formalise this 'structural attention tax' and propose five mitigation strategies, validated on Mistral-7B and LLaMA-3-8B.
MLJailDe, a multilingual jailbreak detection framework, uses back-translation data augmentation across 11 languages (2,232 benign, 1,239 jailbreak samples) and relative-distance constraints to reduce cross-lingual representation dispersion. Achieves F1=98.5% and F1=97.1% on unseen languages.
Evaluation framework for LLM adversarial robustness based on computational cost (FLOPs) rather than query count. Study of 10 models with 3 attack strategies reveals: alignment has non-monotonic effects, scaling reduces gradient attacks but not template attacks, transfer possible across models, cost varies up to 5× across harm categories.
SirenFNO combines Fourier neural operators (FNO) with sinusoidal representation networks (SIREN) to learn full-spectrum without frequency truncation. The framework reduces parameters by 4–15× on PDE benchmarks while eliminating spectral bias toward low frequencies; functional tensor decomposition variants achieve 73× parameter reduction.
Study on counterexample-guided learning to improve LLMs on regex induction tasks. Researchers propose refinement strategies (regularization, symbolic counterexample clustering) and reflection/repair loops. Results: success rates improve from 3.2% to 38.1% and 38.9% to 74.1% on hardest task groups.
FlowBank optimizes LLM-based multi-agent workflows through a reusable bank of complementary workflows. The three-stage framework (DiverseFlow for diversification, CuraFlow for compression, adaptive matching) improves over baselines by 4.26% to 14.92% across five benchmarks while remaining cost-competitive.
Researchers propose enforcing energy conservation as a hard physical constraint in modular neural network pipelines to mitigate error propagation across module boundaries. On CIFAR-10, this approach retains 77.4% accuracy at sigma=0.2 versus 35.1% for baselines. The advantage generalizes to robotic systems (Franka Panda, MuJoCo) with +18.9 pp gain.
LAPG combines conditional score-based diffusion with action-derived physical guidance to improve extrapolation in computational physics. Tested on ODE/PDE systems (free fall, springs, vortices, airfoil flows), the method reduces phase drift and preserves physical consistency outside training distribution.
SwiftCTS is a physics-informed surrogate framework for Clock Tree Synthesis. Trained in under 5 seconds on CPU with sub-millisecond inference, it uses K-shot multiplicative calibration to adapt predictions to unseen architectures without retraining. Evaluates 100,000 CTS configurations in 10 seconds with <0.5% errors on power and wirelength.
Theoretical paper proves compression-based reward (sealed-audit loss decrease) is Goodhart-resistant: if r_t = E(θ_{t-1}) - E(θ_t), cumulative reward telescopes exactly to true audit improvement. For finite panels, empirical deviation bounded by 2Δ_n(F,δ). Authors mechanize proof in Lean 4 and validate on ARC-TGI grid-transformation tasks.
GraphInfer-Bench is a benchmark of 42,000 samples across 6 real-world graphs assessing whether LLMs can perform complex graph inference (open-ended answers requiring multiple nodes). Four method families tested: graph-token alignment, frontier LLMs, Graph2Text fine-tuning, GNNs. None closes the gap; GNNs outperform LLMs on most tasks.
RAG degrades on heterogeneous collections: dense search loses discriminative power (Wyoming DOT: 75% → 40% scaling 54 to 1,128 docs). MASDR-RAG proposes domain scoping via organizational metadata, improving P@10 from 0.77 to 0.86 (p<0.05). Multi-agent orchestration creates precision-faithfulness paradox.
WebGraphMix selects pretraining data by analyzing Common Crawl web graph topology. The method computes centrality scores without model training or labeled data, then mixes central and peripheral documents. At 400M–1B parameters, 1:1 ratio achieves 41.4% average (+1.6pp vs uniform sampling), 43.8% combined with quality scores.
Context Window Lifecycle (CWL) manages long-horizon LLM agent memory via structured semantic eviction. The agent annotates its trajectory as typed, dependency-linked episodes; a deterministic policy evicts content by priority when token budget is exceeded. CWL completes 89 sequential tasks across 80M tokens with no accuracy degradation, avoiding summarization compaction limitations.
Study distinguishing what language models say from what they internally believe. Using linear truth probes on Claude, Qwen, and Llama role-playing historical personas, authors show persona adoption changes outputs more than internal truth representations. Contrasts with Emergent Misalignment where false claims shift toward true belief space.
BioDivergence is a benchmark and evaluation framework for hidden contextual contradictions in biomedical abstracts. It proposes a six-class conflict taxonomy, a 13-axis divergence ontology, and four structured outputs per claim pair. The silver benchmark contains 11,865 claim pairs across five biomedical domains. Mistral-7B-Instruct-v0.3 achieves 0.5523 accuracy and 0.3894 contextual-F1.
LLM-GNN Co-Teaching introduces a bidirectional co-teaching framework for few-shot learning on text-attributed graphs. Instead of designating one model as teacher, GNN and LLM exchange their most confident pseudo-labels and update mutually. RPL-PO mines DPO preference pairs from convergence trajectories. Achieves 7.86% 3-shot gains on Cora and 7.73% on ogbn-arxiv.
SOMA-SQL resolves multi-source ambiguity in NL-to-SQL translation via synthetic query logs and execution probing. The method constructs synthetic logs to ground schema interpretation, generates SQL candidates, then executes targeted probing queries driven by an ambiguity taxonomy. Results: +13.0% execution accuracy on average across 6 benchmarks, up to +16.7% on ambiguous questions.
APEX is a transformer foundation model specialized for wireless network telemetry. Pre-trained on 10-channel multivariate data from ~4,500 production networks (100K time series), it reduces MAE by 18% vs Toto and 38% vs SARIMA on 192-step DHCP forecasting. Two versions: APEX-Large (269M, cloud) and APEX-Edge (10.5M, edge) with F1=0.93 for anomaly detection.