UCSF-PDGM-VQA is a clinical VQA benchmark for brain tumor MRI interpretation. Dataset of 2,387 QA pairs from 473 glioma studies. Evaluation of 6 VLMs: all fail on multi-sequence 3D MRI, suffer modality collapse and over-reliance on language priors.
ManiSoft is a benchmark for vision-language manipulation with soft robotic arms. It includes a simulator coupling realistic soft-body dynamics with contact-rich interactions, 4 deformable control tasks, and 6,300 scenes with expert trajectories. Testing 3 policy models shows promising results in clean scenes but substantial performance drop under randomization.
ClawGym is a framework for developing agents capable of executing multi-step workflows over local files and persistent tools. Authors construct ClawGym-SynData (13.5K synthesized tasks), train ClawGym-Agents via supervised fine-tuning and RL, and propose ClawGym-Bench (200 instances) for evaluation.
Babel is a black-box jailbreak method exploiting a vulnerability in LLM safety alignment: safety relies on sparse attention heads, leaving representational space weakly monitored. Through optimized obfuscation and iterative refinement, Babel achieves 82.67% success on GPT-4o and 78.33% on Claude-3-5-haiku within ~40 queries.
RoboMME is a standardized benchmark for evaluating memory in vision-language-action (VLA) models for long-horizon robotic manipulation. 16 tasks test temporal, spatial, object, and procedural memory. 14 memory-augmented VLA variants built on π0.5 show effectiveness is highly task-dependent.
New perspective on agentic AI systems: code as central infrastructure. This research paper organizes a unified framework around three layers — harness interface (code connecting reasoning and action), mechanisms (planning, memory, feedback), and multi-agent scaling. Applications: coding assistants, GUI/OS automation, embodied agents, scientific discovery.
Study of adversarial action removal attacks in self-play reinforcement learning. An attacker selectively masks legal actions from the victim's action set. Experiments on poker (6 to 5,531 states) and two non-poker domains: learned masking causes substantially more damage than random masking, persists across Q-learning/PPO/NFSP/DQN, transfers between agents, and is amplified by self-play.
Unified framework for disease trajectory modeling in clinical AI, integrating factual forecasting, counterfactual estimation, and policy evaluation. Addresses treatment assignment bias, time-varying confounding, and observation bias to transform static predictions into treatment-sensitive dynamic estimates.
AutoLLMResearch introduces an agentic framework to automate configuration of expensive LLM experiments. The system learns from low-fidelity experiments to extrapolate toward promising high-fidelity configurations. LLMConfig-Gym provides a multi-fidelity environment with >1M GPU hours of verified experiment outcomes.
NavOne reformulates vision-language navigation (VLN) as one-step global path planning on top-down maps. The framework directly predicts dense path probabilities via single end-to-end forward pass, achieving 8x speedup over map-based methods and 80x over egocentric approaches. Introduces R2R-TopDown dataset.
Key-Gram is a conditional-memory framework separating linguistic knowledge from visual reasoning for embodied control. It decomposes instructions into task-specific key-grams, retrieves linguistic priors via O(1) hashed lookup, and injects them into hidden layers. Achieves 29.5% gains on RoboTwin2.0, 35.8% on LIBERO-Plus, 15.4% on real-world tasks.
HIBCG introduces a theoretically grounded approach to learn sparse coordination graphs in multi-agent RL. Using graph information bottleneck, it determines edge existence and message capacity with formal guarantees on learned topology and differential capacity allocation across agent groups.
UniversalRAG extends retrieval-augmented generation (RAG) to heterogeneous multi-modal corpora (text, images, videos) with variable granularities. The framework proposes modality-aware routing to avoid intra-modal bias and dynamically retrieve from the appropriate corpus. Validated on 10 multi-modal benchmarks.
STT-Arena is a benchmark of 227 interactive tasks measuring LLMs' ability to detect and adapt to spatio-temporal changes. Claude-4.6-Opus achieves under 40% accuracy. Authors identify three recurring failure modes and propose STT-Agent-4B combining iterative trajectory refinement with online RL.
TierCheck is a three-tier checkpointing system for LLM training. It maintains lightweight differential checkpoints in local/peer memory for fast recovery, asynchronously migrates base checkpoints to remote storage, and ensures global consistency without stalling training. On models up to 40B parameters, it reduces checkpointing time to under 10s.
Asynchronous RAG framework predicting when and what to retrieve using three components (retrieval predictor, context monitor, query generator). Achieves 43.5% end-to-end latency reduction and 62.4% time-to-first-token improvement while maintaining answer quality.
New Interactive Benchmarks evaluation paradigm assesses model reasoning through budgeted multi-turn interaction. Two settings: Interactive Proofs (logic, UI2Html, mathematics with objective feedback) and Interactive Games (strategic reasoning). Reveals substantial gaps in current interactive capabilities.
SkillJect automates prompt-injection attacks against skill-enabled LLM agents. The framework hides malicious payloads in auxiliary helper scripts and rewrites SKILL.md instructions using a front-loaded inducement strategy. A multi-agent loop (Attack/Victim/Evaluate) optimizes attack effectiveness across platforms and backend LLMs.
Critical study of LLM-based trading agents (FinCon, FinMem, TradingAgents, FinAgent, QuantAgent, FLAG-Trader). Reported Sharpe ratios do not constitute deployment evidence: temporal contamination, unmodeled frictions, and insufficient predictive calibration invalidate claims. Proposes P1-P6 protocol and modular architecture with LLM as audit interface.
LPG (Latent Policy Guardrail) is a safety framework for LLMs that adapts security policies at inference time without retraining. It compresses reasoning into 10 latent tokens, achieves 84.5% accuracy and 77.9% F1 on benchmarks, while running 11× faster than Qwen3-4B-Thinking.
DoublyCal, a double-calibration framework, improves LLM reliability by quantifying epistemic uncertainty in retrieved evidence and reasoning. A lightweight proxy model generates Knowledge Graph evidence with calibrated confidence, guiding a black-box LLM toward more accurate and well-calibrated predictions.
Neuro-symbolic study comparing LLMs constrained by deterministic logic scaffolds (Rulemapping) versus unconstrained prompting for hate speech moderation under German Criminal Code (§130). Rulemapping achieves precision 0.80-0.86 and recall 0.82-0.89 versus 0.34-0.49 with unconstrained prompting, eliminating conflation of moral offense with legal illegality.
LongAct is a benchmark for evaluating autonomous planning in long-horizon household tasks specified via free-form instructions. HoloMind, a VLM-driven agent with DAG-based hierarchical planner, Multimodal Spatial Memory, and Episodic Memory, achieves 59% goal completion and 16% full-task success with GPT-5 and Qwen3-VL models.
Adaptive Layerwise Perturbation (ALP) addresses off-policy issues in LLM RL by injecting learnable perturbations into hidden states across all layers. This reduces heavy-tailed importance ratios, stabilizes training, and improves performance on math and multi-turn reasoning tasks while boosting exploration.
A framework uses language models to identify 'alien' research directions—coherent with existing literature but unlikely under current researcher distribution. On 16,068 AI/NLP papers, the method explores 3.5-7× broader conceptual space than baselines while maintaining scientific coherence.
LLMs exhibit 'conversational inertia' in multi-turn agent scenarios: they over-imitate their previous responses instead of exploring. Authors identify this bias through attention analysis and propose Context Preference Learning to favor low-inertia responses. Validated across 8 agent environments.
WASIL is a dataset of 8,529 turns of Arabic voice interactions with LLMs, including audio, ASR hypotheses, responses, and user feedback (14.2% dislikes). Covers Modern Standard Arabic and 4 major dialects. Enables isolating ASR errors from intrinsic unanswerability via annotation and multi-judge LLM evaluation.
Comparative study of 6 EEG foundation models across 8 datasets beyond clean accuracy. Robustness analysis (noise, channel dropout), interpretability via Attention-Aware Layer-Wise Relevance Propagation, and expressiveness through block-wise probing. Findings: no single model dominates all failure modes; models focus on task-appropriate brain regions but decode corrupted content poorly.
StructLens analyzes the internal organization of representations in language models using maximum spanning trees built on residual streams. The framework reveals that middle layers strongly organize nearby tokens, and that smaller local units emerge before larger units during pre-training.
ESI-Bench is a benchmark for embodied spatial intelligence spanning 10 task categories on OmniGibson. Agents must combine perception, locomotion, and manipulation to actively accumulate evidence. Experiments show active exploration outperforms passive approaches, but failures stem primarily from poor action choices rather than weak perception.
Study of internal preferences in LLMs using linear probes on residual-stream activations. Researchers identify a shared preference vector in Gemma-3-27B and Qwen-3.5-122B that predicts and causally controls model choices. This vector remains stable even when the model adopts radically different personas (helpful assistant vs evil persona).
Rover is a code merge conflict resolution system combining program analysis with LLMs. It introduces Multi-layer Code Property Graph (MtCPG) to capture inter-file dependencies and uses graph connectivity algorithms to create meaningful contexts. Evaluation: Rover outperforms standalone LLMs, MergeGen, and WizardMerge at character, lexical, and semantic levels.
Systematic study of reasoning faithfulness in Vision-Language-Action (VLA) driving models. Analysis of 300 Alpamayo-R1-10B inferences across 100 PhysicalAI-AV scenarios reveals: reasoning fidelity 42.5%, 94 missed pedestrians, 97.7% trajectory fragility under visual perturbations, 48.3% reasoning-action consistency. Proposes four-component safety architecture.
A2RBench is an automated pipeline for generating formally verifiable abstract reasoning benchmarks. Using programmatic verification (cycle consistency), it eliminates hallucinations and scales task variations. Evaluations show current LLMs score 39.8% vs 68.5% for humans, and struggle with complex 3D tasks.
STRIDE-AI is a threat modeling framework for assessing generative AI system security. It bridges NIST AI RMF standards and OWASP LLM Top 10, defines a six-phase assessment lifecycle, and operationalizes through a web tool. Validation on a deployed LLM chatbot reduced attack success rate from 80% to 15%.
SWIM aligns vision-language representations for fine-grained video object understanding from text prompts alone. Uses mask supervision during training to guide cross-modal attention. Constructs NL-Refer dataset with precise natural language referring expressions. Outperforms visual-prompt-based methods on fine-grained benchmarks.
CLUES, a framework for clinical Text-to-SQL, decomposes semantic uncertainty into ambiguity and instability scores using the Schur complement of a bipartite semantic graph matrix. Tested on AmbigQA/SituatedQA and a clinical benchmark, it outperforms Kernel Language Entropy and enables efficient triage: 51% of errors in 25% of queries.
Guided Topology Diffusion (GTD) uses graph diffusion models to dynamically generate optimal communication topologies for multi-agent LLM systems. The iterative framework, guided by a proxy model predicting multi-objective rewards (accuracy, utility, cost), adapts topologies to tasks without gradient-based optimization, outperforming static approaches.
Formal study of imperfect world model exploitation in RL. Authors define exploitation as divergence between policy preferences in the model versus true environment. They prove exploitation is essentially unavoidable on large policy sets and establish theoretical bridge with reward hacking.
ShareChat is a corpus of 142,808 conversations (660,293 turns) collected from ChatGPT, Perplexity, Grok, Gemini, and Claude between April 2023 and October 2025. The dataset preserves native affordances (citations, reasoning traces, code artifacts) across 95 languages and enables analysis of cross-platform differences in user satisfaction, citation strategies, and response latency.