In AI, reasoning refers to a model's ability to solve problems through multi-step logical thinking, beyond pattern matching. OpenAI's o3 is a key example: it breaks down a problem before producing an answer.
MiniMax introduces a new attention architecture (MSA) natively supporting 1M tokens without quadratic complexity. 'KV outer gather Q' approach delivers 4× faster than Flash-Sparse-Attention, compute reduced to 1/20th, 9× prefilling speedup, 15× decoding speedup. First open-weight model combining frontier coding, 1M context, and native multimodality.
Microsoft announces MAI-Thinking-1 (35B, reasoning) and MAI-Code-1-Flash (5B, code). The former outperforms Claude Sonnet 4.6 in blind human evaluation. Both trained on commercially licensed data without third-party distillation.
Wavelet-based technique to improve LLM contextual understanding of code. Multi-scale analysis approach for code structures to optimize processing by language models.
Pull request for llama.cpp adding a thinking mode toggle with reasoning effort levels and improvements to the chat form action UI. Feature demonstrated in video.
Comparative study of learning rules (backprop, feedback alignment, predictive coding, STDP) via RSA alignment with human V1 fMRI. Backprop destroys 90% of V1 alignment after 1 epoch (r: 0.102→0.011), while PC and STDP lose only 25-31%. At epoch 40: PC/STDP >> BP/FA. Suggests fundamental trade-off between global error signals (higher layers) and early-layer alignment.
2-week trial: replaced Claude with Qwen3.6-27B (Q6_K, 22GB) in multi-agent orchestrator on RTX 3090. Wins: plan generation (95% valid), memory extraction, auto-review. Failures: tool-call reliability 12% error rate vs 0.5% Claude, context drift >14k tokens, weak cascade-failure handling.
Mathematical primer on generative AI foundations by Tianhua Chen. Covers VAE, diffusion models, normalizing flows, autoregressive factorizations, GANs, Wasserstein GANs, and energy-based models through derivation-oriented approach.
ExpGraph, an arXiv preprint from May 29, 2026, proposes a self-evolving graph memory architecture for LLM agents. The system accumulates experience through dynamic nodes and edges, enabling agents to improve performance on complex tasks without retraining.
Asymptotic theory for SGLD-Gibbs combining stochastic gradient Langevin dynamics with Gibbs updates for Bayesian inference in latent variable models. Authors derive jump-diffusion limits and propose hyperparameter tuning guidance ensuring statistically meaningful uncertainty quantification.
InfoAtlas is a foundation model that estimates mutual information between high-dimensional variables in a single forward pass, eliminating costly iterative optimization. Pretrained on synthetic data with diverse dependence patterns, it achieves 100× speedup over neural MI estimators while matching accuracy and generalizes to complex real-world scenarios.
Theoretical study showing how transformers learn to implement tree search (DFS) via RL. A two-head transformer naturally emerges from policy gradient training on stochastic trees without expert demonstrations. The model generalizes to unseen depths and adapts its strategy based on goal distributions.
PROBE, an optimization framework for LLM agents in drug design, resolves the conflict between binding affinity and druggability. Through controlled edit probing and pocket-specific site mapping, it guides a multi-agent loop (affinity, druggability, co-optimization) on CrossDocked2020 with SOTA results.
Study of LLM agent faithfulness in a Texas Poker simulator. Researchers measure the gap between stated reasoning and actual actions by decomposing the problem into two steps: reasoning-conclusion and conclusion-action. The two steps exhibit opposite behaviors.
Post-training quantization (PTQ) reduces reasoning model accuracy and increases chain-of-thought length. 52% of failures involve correct intermediate answers not output as final answers. A training-free logit penalty on overthinking markers ("wait", "but", "alternatively") reduces CoT length by 12-23% while preserving accuracy across 5 models (1.5B-32B) and 5 benchmarks.
ART (Attention Run-time Termination) is a lightweight runtime mechanism that halts KV block accesses during decoding once their attention contribution becomes negligible. Tested on LongBench, it achieves 20% higher generation throughput on large batches while maintaining comparable accuracy.
CSRP, a three-stage framework for Chinese grammatical error correction, combines continual pre-training (5.9M samples), Chain-of-Thought fine-tuning, and policy optimization with efficiency-aware rewards. Achieves 50.99 F₀.₅ on NACGEC and outperforms GPT-4 on spelling correction (59.61 F1).
SENSE improves retrieval-based speculative decoding by anchoring retrieval on target model hidden states for robust semantic alignment. A soft-gated evaluation module validates semantic equivalence rather than surface forms. On LLaMA and Qwen, SENSE achieves 4.09 mean acceptance length and 3.26x speedup.
ProactiveLLM learns when to interact with streaming inputs without external signals. Through monotonic random masking and synchronized privileged self-distillation, the model perceives semantic sufficiency from partial inputs. Reduces interaction latency while maintaining quality on text and speech streaming tasks.
DLLM-JEPA pairs JEPA with masked-diffusion language models for self-supervised representation learning. Eliminates need for explicit multi-view data and reduces training FLOPs by 33% vs LLM-JEPA. Achieves +18.7pp improvement on GSM8K (LLaDA-8B) and +11.4pp (Dream-7B) while preserving base model capabilities.
DeSQ decomposes complex questions into atomic constraints to generate SPARQL queries. The framework operates in three stages: decomposition, generation of SPARQL fragments with placeholders, and final assembly. Outperforms state-of-the-art on 4/5 major benchmarks with improved robustness to lexical variation.
TOPD (Trajectory-aware On-Policy Distillation) improves LLM reasoning by using near-future trajectory information to identify truly divergent states. On AIME24/25, TOPD reaches 63.3%/53.3% vs 60.0%/46.7% in standard OPD, showing 30% of high-loss tokens are false positives.
ProtStructQA is an executable benchmark for protein structural question answering. 382.2K questions generated from hidden domain-specific language, evaluated on Qwen3 (0.6B–8B) and Gemma-3. Key finding: capability threshold between Qwen3-1.7B and 4B where models transition from failing to produce executable denotations to mastering chain-of-thought reasoning.
LaSR proposes a training paradigm for Speech LLMs featuring latent reasoning aligned to acoustic feature regions of target words. Without explicit intermediate tokens, the method improves specialized vocabulary recognition on Fun-Audio-Chat. A new Spoken Darwin-Science corpus for academic terminology is introduced.
SelSkill, a dual-granularity preference-learning framework, optimizes selective skill invocation in agentic tasks. On ALFWorld with Qwen3-8B: +10.9 pp task success, +29.1 pp execution precision. On BFCL: +5.7 pp task success, +29.5 pp execution precision. Zero-shot transfer to Tau-bench and PopQA.
Theoretical and empirical study of parameter-based knowledge editing limits in LLMs. Authors prove via dimensional collapse hypothesis that localized modifications propagate global interference degrading model capabilities. Retrieval-based methods consistently outperform parameter-editing approaches.
SPADER is an RL framework for tool-augmented LLM agents in Multi-Answer QA. It introduces Step-wise Peer Advantage (SPA) for fine-grained credit assignment over long trajectories, and a diversity-aware exploration reward promoting rare entity discovery. Evaluated on QAMPARI, Mintaka, WebQSP, QUEST: improves recall and F1 vs prompting and supervised RL baselines.
arXiv paper introduces Universal Quantum Transformer (UQT), a quantum-native architecture leveraging multi-qubit properties for exact mathematical reasoning. On 5 qubits, UQT perfectly learns modular arithmetic (Z₁₁) and non-Abelian algebra (S₄) with deterministic generalization, no stochastic instability. Deployed on IBM Quantum hardware, it bypasses classical attention's quadratic bottleneck.
Interactive reasoning evaluation benchmark with 474 executable games. LLMs receive only task rules, must query a hidden environment, integrate partial observations, and decide when to submit answers. Evaluates contextual robustness, metacognitive adaptation, and interaction efficiency across frontier models.
TIGER is an inference-time framework to mitigate hallucinations in multimodal generation. It independently extracts an observation graph from input and a claim graph from output, then assigns risk scores to claims based on support and conflict. The model repairs high-risk claims while keeping the backbone frozen. Convergence analysis shows geometric risk reduction to an explicit asymptotic bound.
Geodesic Flow Matching adapts Riemannian transport to denoise Spatial Semantic Pointers (SSPs) on toroidal manifolds. Tested on a Spiking Neural SLAM system, the method achieves 72% reduction in tracking error and 40% increase in neural efficiency versus baselines.
Survey paper proposing Intelligent Computing Architecture Model (ICAM), a six-layer framework for model-native computing. Maps classical computer architecture concepts to LLM systems (cache management, context, agents). Introduces three design laws: Semantic Locality Law, Context Budget Law, Agent Speedup Law. Distinguishes probabilistic execution plane from deterministic control plane.
Hybrid framework coupling LLMs with physics-based simulation for inorganic material synthesis planning. Case study on niobium-oxygen system: LLM-generated synthesis routes outperform classical path-planning algorithms by leveraging implicit priors.
Decoder-only models hit an information-theoretic limit in deterministic state-tracking tasks beyond ~25 steps. An Attention Bottleneck Theorem bounds capacity to O(H·log(L/H)·√dh). Across 12 models and 8 domains (SWE-Bench, WebArena, SQL), tool delegation achieves 86-94% vs 24-42% for pure neural reasoning. Fine-tuning improves <5%, confirming an architectural ceiling.
VESTA is a statistical agent framework equipping VLMs with a dynamically growing exploration toolkit for quantitative model refinement. Evaluated on DAWN (benchmark for distribution fitting and time series modeling), VESTA outperforms prior agentic systems, especially on complex astronomy tasks (initial mass functions, gravitational-wave chirp signals).
TAPS introduces a target-aware prefix selection method for diffusion-drafted speculative decoding. By converting diffusion marginals into path-conditioned acceptance estimates, TAPS selects a compact prefix-closed subtree under fixed verification budget. Results: 7.9x lossless speedup vs vanilla autoregressive decoding, 1.36x and 1.74x over DFlash and DDTree.
KACE decouples storage from usage in context for mathematical reasoning. An epistemic tree stratified by difficulty and domain is built offline via self-reflective loop. At evaluation, tiered self-consistency dynamically classifies problems and selectively retrieves matching cards. On AIME 2025: 62.2% accuracy (+10.4 points vs Best-of-5).
TRACE, a trajectory compression method, detects safety risks in long-horizon LLM agents. A Compressor encodes the full trajectory into a supervised latent evidence state, while a Reader judges safety using this reference. Achieves up to 12.6 percentage point improvements on ASSEBench, Pre-Ex-Bench, and R-Judge.
Researchers demonstrate that hidden reasoning traces in LLMs can be extracted via Reasoning Exposure Prompting (REP), a lightweight prompting method using shadow-model-generated demonstrations in auxiliary code-like formats. REP exposes internal traces even when deployed systems intentionally hide them, while preserving useful reasoning signals for distillation.
BudgetDraft trains a sparse drafter for speculative decoding in long-context inference (4K-16K tokens). The method exposes the model to multiple KV budgets during training and aligns each sparse view with a shared full-cache teacher target. Results: 6.55x, 4.46x, 2.10x speedup vs autoregressive decoding at 4K, 8K, 16K tokens.
arXiv paper proposing principled approaches for designing practical LLM-based agentic systems. Framework with "pseudo-tools" recursively calling LLMs on restricted context. Hand-engineered fixed workflows are cheaper and more accurate than dynamically-planned workflows. Learning methods for pseudo-tools and fixed workflows outperform hand-engineered agents.