ProtStructQA is an executable benchmark for protein structural question answering. 382.2K questions generated from hidden domain-specific language, evaluated on Qwen3 (0.6B–8B) and Gemma-3. Key finding: capability threshold between Qwen3-1.7B and 4B where models transition from failing to produce executable denotations to mastering chain-of-thought reasoning.
Novel shielding framework for RL agents ensuring formal safety guarantees in MDPs with unknown transition dynamics. Uses robust MDPs (RMDPs) with sets of transition probabilities and LTL formulas. Combines shielding with PAC-learning methods to construct minimally restrictive shields while guaranteeing safety.
arXiv study on LLM adaptation limits for annotation tasks. Toxicity detection experiments across diverse datasets show 66% of zero-shot errors resist correction via prompting (rescue rate 34.8%). Models follow misaligned definitions while maintaining confidence. Definition-Specific Familiarity (DSF) metric correlates with performance (r=+0.41), outperforming memorization metrics.
KG-Guard detects hallucinations in knowledge base question answering (KBQA) systems using an augmented graph and lightweight encoder. The model achieves F1 scores of 82.0–87.4 on WebQSP/ComplexWebQuestions with 305× fewer parameters than baselines, and improves downstream KBQA F1 by 13–14.5 points through iterative refinement feedback.
Theoretical and empirical study of parameter-based knowledge editing limits in LLMs. Authors prove via dimensional collapse hypothesis that localized modifications propagate global interference degrading model capabilities. Retrieval-based methods consistently outperform parameter-editing approaches.
Study of effectiveness and efficiency of tool-calling in LLM agents. Authors show evaluation pipelines are sensitive to minor choices (random seed, system prompt, multi-turn templates) affecting leaderboard reliability. They identify two sources of computational waste in RL and propose two acceleration techniques without performance degradation.
SPADER is an RL framework for tool-augmented LLM agents in Multi-Answer QA. It introduces Step-wise Peer Advantage (SPA) for fine-grained credit assignment over long trajectories, and a diversity-aware exploration reward promoting rare entity discovery. Evaluated on QAMPARI, Mintaka, WebQSP, QUEST: improves recall and F1 vs prompting and supervised RL baselines.
Interactive reasoning evaluation benchmark with 474 executable games. LLMs receive only task rules, must query a hidden environment, integrate partial observations, and decide when to submit answers. Evaluates contextual robustness, metacognitive adaptation, and interaction efficiency across frontier models.
Comparative study of a two-hop Graph-RAG architecture versus standard vector-only RAG for cross-entity financial sentiment analysis. On 100 queries (30 direct, 70 relational), Graph-RAG improves entity recall (+6.4%, p<0.001) and answer relevancy for complex queries (+11.7%), with no quality degradation, modest 22.6% latency increase but 80% variance reduction.
AbaqusAgent is a multi-agent LLM-based framework for finite element analysis (FEA) in solid mechanics. Composed of six agents (interpreter, architect, input writer, runner, reviewer, visualizer), it converts natural-language instructions into executed FEA analyses with Abaqus. Validated on 50 problems with 86% success rate.
Audit of 7 LLMs (US/China) on 2,520 responses to 60 legal-administrative prompts in English and Mandarin. Models default to the institutional framework of input language: 74.5% of English responses adopt US framework, 53.3% of Chinese responses adopt China framework. Risk of jurisdictional misselection when preferred language differs from applicable jurisdiction.
TIGER is an inference-time framework to mitigate hallucinations in multimodal generation. It independently extracts an observation graph from input and a claim graph from output, then assigns risk scores to claims based on support and conflict. The model repairs high-risk claims while keeping the backbone frozen. Convergence analysis shows geometric risk reduction to an explicit asymptotic bound.
Post-training quantization (PTQ) reduces reasoning model accuracy and increases chain-of-thought length. 52% of failures involve correct intermediate answers not output as final answers. A training-free logit penalty on overthinking markers ("wait", "but", "alternatively") reduces CoT length by 12-23% while preserving accuracy across 5 models (1.5B-32B) and 5 benchmarks.
Study on preventing catastrophic forgetting during continual pretraining of multilingual language models. Authors propose five parameter alignment strategies (layer freezing, regularization, post-hoc reversion, model merging) tested across 32 languages and four evaluation axes. Parameter alignment substantially reduces forgetting while maintaining language acquisition.
Decoder-only models hit an information-theoretic limit in deterministic state-tracking tasks beyond ~25 steps. An Attention Bottleneck Theorem bounds capacity to O(H·log(L/H)·√dh). Across 12 models and 8 domains (SWE-Bench, WebArena, SQL), tool delegation achieves 86-94% vs 24-42% for pure neural reasoning. Fine-tuning improves <5%, confirming an architectural ceiling.
SALSA adapts speech-aware LLMs via layer-wise learned steering vectors optimized with supervised objectives. Tested on children's speech, multilingual, and Mandarin-English code-switching, it achieves up to 46.8% relative improvement over zero-shot. Steering the encoder's later layers outperforms steering the LLM backbone.
New set-distance reward method for reinforcement learning of vision-language models on chest X-ray report generation. Tested on Qwen3-VL, Gemma3 with GRPO: 6.80% (BERTScore), 7.82% (RadGraph F1), 4.45% (CheXbert F1) improvements over supervised fine-tuning. Enables test-time best-of-N selection and mid-generation pruning reducing tokens by 50%.
UniKE, the first benchmark for cross-modality knowledge editing in unified multimodal models (UMMs), reveals a critical gap: text-side efficacy reaches 92% but VQA accuracy in image generation drops to 18.5%. A reasoning-augmented parameter editing method improves results by up to +18.6 percentage points.
PROBE, an optimization framework for LLM agents in drug design, resolves the conflict between binding affinity and druggability. Through controlled edit probing and pocket-specific site mapping, it guides a multi-agent loop (affinity, druggability, co-optimization) on CrossDocked2020 with SOTA results.
Theoretical study showing how transformers learn to implement tree search (DFS) via RL. A two-head transformer naturally emerges from policy gradient training on stochastic trees without expert demonstrations. The model generalizes to unseen depths and adapts its strategy based on goal distributions.
TAPS introduces a target-aware prefix selection method for diffusion-drafted speculative decoding. By converting diffusion marginals into path-conditioned acceptance estimates, TAPS selects a compact prefix-closed subtree under fixed verification budget. Results: 7.9x lossless speedup vs vanilla autoregressive decoding, 1.36x and 1.74x over DFlash and DDTree.
RAFT is a two-stage domain fine-tuning method that mitigates catastrophic forgetting. It refines data via self-conditioned rewriting and answer fusion, then applies on-policy distillation where the original model provides soft targets on student-generated trajectories. Across five domains, RAFT improves domain accuracy by 23.2% over standard SFT and recovers 18.2% of degradation on MS-Bench.
Modern LLMs systematically overestimate their competence and attempt unsolvable queries. Researchers propose Capability Self-Assessment (CSA), formulated as a policy-learning problem using reinforcement learning, to teach models to recognize their limits. RL significantly outperforms supervised fine-tuning, preserves original capabilities, and generalizes out-of-distribution.
SENSE improves retrieval-based speculative decoding by anchoring retrieval on target model hidden states for robust semantic alignment. A soft-gated evaluation module validates semantic equivalence rather than surface forms. On LLaMA and Qwen, SENSE achieves 4.09 mean acceptance length and 3.26x speedup.
arXiv paper on LLM inference batching optimization. Authors demonstrate mixed batching (MB) is suboptimal on bandwidth-constrained GPUs: exclusive batching (EB) achieves 41.9% higher throughput on RTX PRO 6000 (1.792 TB/s). They propose EB+, a hybrid scheduler that dynamically switches between EB and MB based on GPU bandwidth, model size, and workload composition, reaching 36.4% gains under non-stationary traffic.
ProactiveLLM learns when to interact with streaming inputs without external signals. Through monotonic random masking and synchronized privileged self-distillation, the model perceives semantic sufficiency from partial inputs. Reduces interaction latency while maintaining quality on text and speech streaming tasks.
Preference Delta Aggregation (PDA) aggregates weak preference signals from model pairs (e.g., Qwen3 4B vs 1.7B) via LoRA merging. Geometric Alignment Merging (GAM) aligns adapter subspaces before aggregation. On knowledge reasoning and agentic search benchmarks, PDA+GAM improves Qwen3 8B by +6.8 and +7.3 points respectively.
DLLM-JEPA pairs JEPA with masked-diffusion language models for self-supervised representation learning. Eliminates need for explicit multi-view data and reduces training FLOPs by 33% vs LLM-JEPA. Achieves +18.7pp improvement on GSM8K (LLaDA-8B) and +11.4pp (Dream-7B) while preserving base model capabilities.
Study of agreement metrics for LLM-as-Judge evaluation. Analysis of 24 recent papers shows that for binary criteria (MET/UNMET), Pearson r, Spearman ρ, Kendall τ_b, and phi are redundant. Cohen's κ alone adds information. Authors propose a reporting checklist including judgment scale, abstention handling, and confusion matrix.
KACE decouples storage from usage in context for mathematical reasoning. An epistemic tree stratified by difficulty and domain is built offline via self-reflective loop. At evaluation, tiered self-consistency dynamically classifies problems and selectively retrieves matching cards. On AIME 2025: 62.2% accuracy (+10.4 points vs Best-of-5).
Real-time multilingual ASR system routing audio between specialized monolingual models (~100M parameters each) instead of one large model. Detects language switches via SpeechBrain and re-transcribes with correct model. Achieves 13% WER on inter-utterance code-switching, outperforming cloud APIs. Open-source repo released.
Lightweight multilingual ASR routing system for local hardware using Zipformer, Silero VAD, and SpeechBrain. Routes audio between specialized monolingual models (~100M parameters) instead of one large model. Achieves 13% WER on inter-utterance code-switching, outperforming cloud APIs. Known limitation: 41% WER on intra-utterance switching. Open-source repo available.
5WBENCH, a balanced 5,000-sample benchmark across 5W categories, reveals unlearning methods fail on causal (Why) questions. MAAT, a three-phase framework operating on LoRA weights, combines gradient-projected ascent, SVD rank pruning, and KL-hidden-state repair to simultaneously achieve high forgetting and retention on causal knowledge.
TRINE is an FPGA accelerator and compiler for end-to-end multimodal inference (ViT, CNN, GNN, transformers) without reconfiguration. It unifies layers as matrix operations, switches between systolic and SIMD architectures at runtime, and applies in-stream token pruning. On Alveo U50 and ZCU104, it achieves 22.57x latency reduction vs RTX 4090 while consuming 20-21 W.
DisasterLex is a knowledge-graph-mediated text-to-SQL framework for querying geospatial disaster-analytics databases. It uses an Expert Knowledge Graph (107 concepts, 117 causal edges) to route natural-language queries across 36 heterogeneous tables. On 75 test queries, it outperforms 4 baselines (LightRAG, HippoRAG 2, ReFoRCE, CHESS) by 1.4x to 2.75x.
HypoAgent is a multi-agent framework for interactive abductive hypothesis generation over knowledge graphs. Three coordinated agents (intent recognition, hypothesis generation, root cause analysis) enable multi-turn dialogue and fine-grained diagnosis of failed hypotheses. SOTA on commonsense and biomedical KGs.
Evaluation of semantic stability in 16 LLMs (general-purpose and medical) under clinically equivalent prompt reformulations. Proposes NLI-based verification framework and three sensitivity metrics (MVS, ΔC, WCI). Finding: domain specialization does not consistently improve robustness to meaning-preserving variations.
EUDAIMONIA is a benchmark evaluating harmful social dynamics in LLMs. It contains 969 user inputs and 3,147 design-violation checks, testing 22 recent models. Claude-Opus-4.7 and GPT-5.5 violate 30.7% and 27.2% of checks respectively, revealing persistent social-alignment failures not resolved by extended thinking.
TeachObs is a human-validated multimodal benchmark for classroom video analysis. It contains 30 public lessons from 8 countries split into 5,158 15-second scenes, annotated by 7 researchers with 39 observation codes (20 visual, 19 non-visual). Evaluation of 5 vision-capable LLMs across 3 tasks: no single model consistently outperforms others.
SAGE is an adaptive gate using von Mises-Fisher density estimation to control memory evolution in agentic LLMs. It classifies candidate facts as ADD (novel), NOOP (redundant), or MERGE (uncertain), reducing expensive LLM calls. On LoCoMo, SAGE cuts API cost by 3.4× and latency by 2.5× with GPT-4o-mini.