OpenDeepThink uses Bradley-Terry aggregation to select best solutions from multiple parallel candidates. The system randomly compares answer pairs, aggregates votes, and preserves top candidates for mutation. On Codeforces, Gemini 3.1 Pro gains +405 Elo points across 8 LLM calls (~27 min). Authors release CF-73, 73 expert-annotated problems.
SD-Search introduces on-policy hindsight self-distillation for search-augmented reasoning agents. A single model acts as both student (inference-time context only) and teacher (conditioned on search outcomes from rollout groups). Step-level supervision via Jensen-Shannon divergence at query positions, integrated into GRPO training without external models or annotations.
Researchers formalize activation steering (LLM control without retraining) as budget-constrained optimization over intervention layer and coefficient. They introduce concept granularity to measure directional heterogeneity and present GRACE, a framework using activation geometry to diagnose steering difficulties and reduce required evaluations by 39.8% on average.
Study showing diverse ensembles of monitors detect misaligned AI agent actions better than homogeneous ensembles. 12 GPT-4.1-Mini monitors (prompting + fine-tuning) evaluated on coding tasks: best 3-monitor ensemble achieves 2.4x greater detection gain versus identical-monitor ensemble, with strong generalization on independent data.
HEED introduces density-weighted residual alignment for distilling vision-language models (e.g., Qwen3-VL-8B) into hybrid Mamba-2/attention architectures. The method targets high-density patches (text, fine details) experiencing 3.6× larger residual drift. Results: +8.7 points OCRBench v2, +5.13 points average, 4.12× throughput, 68% memory savings.
DexWild introduces a low-cost system for collecting dexterous manipulation data through natural human hand gestures. The framework co-trains on human and robot demonstrations, achieving 68.5% success rate in unseen environments (4× better than robot-only) and 5.8× improved cross-embodiment generalization.
Sign-Muon is a 1-bit distributed optimizer combining majority-vote sign aggregation (signSGD) with Muon's polar-step framework. It reduces communication by 32× in float32 and achieves 92.15% accuracy on CIFAR-10/ResNet-50, with 37% less training time on 4 GPUs.
GeoSym127K is a dataset of 127K geometric questions with exact symbolic ground truths, generated by an automated neuro-symbolic engine. Fine-tuning on Qwen3-VL-8B: +22.21% on MathVerse Vision-Only, 61.52% on WeMath. RLVR via GRPO further improves performance.
SkillMOO optimizes skill bundles for coding agents via multi-objective search (NSGA-II) on pass rate and inference cost. Across 16 SkillsBench SE tasks, the framework achieves top pass rate rank on 11/12 non-zero-pass tasks, reducing costs up to 31.7% and gaining up to 21 percentage points in pass rate.
Researchers formalize activation steering (LLM control without retraining) as budget-constrained optimization. They introduce "concept granularity" to explain why some concepts are expensive to steer, and propose GRACE, a framework using activation geometry to diagnose difficulties and reduce evaluations needed by 39.8% on average.
EPIC (Efficient Preference-aligned Index Construction) optimizes on-device RAG by storing user preferences instead of raw data. Across 4 benchmarks, reduces indexing memory by 2404×, improves preference-following accuracy by 20.17 percentage points, and lowers retrieval latency by 33.33×. Memory footprint under 1 MB with 29.35 ms/query.
Critical study of state-of-the-art claims in AI/ML. Analysis of 10 public benchmarks reveals that over 50% of top-model comparisons fail to support implicit superiority properties (meaningful effect size, cross-task consistency, robustness). Aggregate gains often driven by outlier datasets. Proposes more honest claim language without additional experiments.
LARGER is a context retrieval framework for repository-level coding agents combining lexical search with structural graph exploration (imports, call chains, type hierarchies) without external databases. On LocBench, it improves file-level Acc@5 by +13.9 points (or +11.8 with fixed hyperparameters) and shows consistent gains on test generation and codebase QA benchmarks.
LoopQ introduces a post-training quantization (PTQ) method for recursive language models (LoopLMs) that reuse Transformer blocks. It addresses three challenges: distribution shift across roles, state reuse between loops, and recursive error accumulation. Results: +68.8% downstream accuracy and -87.7% perplexity reduction in W4A4 vs strongest static PTQ baseline.
RTPurbo transforms LLMs into sparse models in ~100 training steps. The approach exploits three observations: only certain heads require full attention, long-range retrieval uses a 16D subspace, and dynamic top-p selection outperforms fixed top-k. Results: 9.36× prefill speedup at 1M tokens, 2.01× decode speedup, accuracy preserved.
Comparative study of fine-tuning vs. in-context learning on LLMs using formal language tasks. Fine-tuning outperforms ICL on in-distribution generalization, but both perform equally on out-of-distribution. Inductive biases diverge at higher proficiency levels. ICL shows sensitivity to vocabulary and model size.
Med-V1 is a family of 3-billion-parameter language models trained on synthetic data for biomedical evidence attribution. It outperforms base models by +27% to +71% on five benchmarks and matches frontier LLMs like GPT-5, while detecting hallucinations and misattributions in clinical guidelines.
SomaliWeb v1: Somali corpus of 819,322 documents (~303M tokens) with BPE-16K tokenizer and language-identification benchmark. Reveals critical defects in HPLT v2 (17.3% exact duplicates, 56.1% mojibake, 10.7% near-duplicates). Tokenizer 40.2% more efficient than cl100k_base on FLORES-200.
Masked language models (MLMs) define local conditional distributions incompatible with any consistent global joint distribution. Authors model iterative resampling as Glauber dynamics Markov chain, proving O(n log n) mixing time under bounded cross-token influence, but showing exponential metastability at low temperature with persistent semantic basins.
DataClawBench is a benchmark for exploratory real-world financial data analysis agents, containing 2.06 million raw records and 492 cross-domain tasks. Evaluation of 8 advanced LLMs shows that increased exploration does not reliably translate to task progress or correct answers.
Study on how representation geometry organization in language models depends on scale. Subspace PGA metric tests alignment of intermediate geometry with unembedding matrix readout. Small models (≤1024) progressively lose organization at late layers during training, while large models (≥2048) preserve it throughout. Scale determines how geometry organizes for prediction.
GCPO (Group Cooperative Policy Optimization) replaces competitive rollout optimization with team-level credit assignment. Rollouts are rewarded by contribution to valid solution coverage (determinant volume over semantic embeddings), not individual accuracy. Results: improved reasoning accuracy and solution diversity across benchmarks.
UA-StatuteRetrieval: 20-year benchmark on 396M Ukrainian court citations. Co-citation predictability declines 33-47% (Adamic-Adar MRR 0.43→0.29). Non-uniform decay: criminal law stable (~0.40), civil law collapses (0.35→0.15) post-2017 reform. Mid-frequency articles (1K-10K citations) lose 50% predictability. E5-large detects 4.3% semantic drift.
RaBiT introduces residual-aware binarization training for 2-bit LLM quantization. It solves inter-path feature co-adaptation by sequentially deriving each binary path from a shared full-precision weight, ensuring error correction across layers. Results: SOTA performance, 4.49× inference speedup on RTX 4090.
Mosaic, a probabilistic weather forecasting model, addresses three spectral degradation failures in ML-based prediction: spectral damping, high-frequency aliasing, and residual leakage. With 214M parameters at 1.5° resolution, it matches models trained 6× finer and generates well-calibrated ensembles in 12s for 10-day forecasts on H100.
COAST (Contrastive Conceptor Activation Steering) improves Vision-Language-Action models by identifying and steering latent representations toward success-critical subspaces. Across three distinct architectures, COAST increases task success rates by +20% in simulation and +40% on real robots, without additional training.
LiTS is a modular Python framework for LLM reasoning via tree search. It decomposes search into three reusable components (Policy, Transition, RewardModel) compatible with MCTS and BFS. Tests on MATH500, Crosswords, and MapEval demonstrate component-algorithm orthogonality. Key finding: LLM policy diversity, not reward quality, is the bottleneck in infinite action spaces.
CompactAttention optimizes chunked prefill attention for long-context LLMs using Block-Union KV Selection. The method converts block-sparse masks into GQA-aware per-group KV block tables, eliminating explicit KV compaction. On LLaMA-3.1-8B, it achieves 2.72× speedup at 128K tokens while maintaining accuracy near dense attention (RULER benchmark).
A bank develops a conversational AI agent to triage fraud, scam, and dispute cases. The agent uses LLMs to ask targeted questions and route customers to appropriate specialist teams. Evaluation via synthetic digital twins simulating realistic dialogues. Result: +30.6% improvement in classification accuracy with compliance guardrails.
Discrete diffusion models generate code through iterative refinement. CDC (Constrained Diffusion for Code) integrates constraints directly into the denoising process without additional training, combining mathematical optimization and program analysis to improve functional correctness, security, and syntax.
Scales++ proposes benchmark subset selection based on intrinsic task properties rather than model-specific failure patterns. Using 0.25% of data on Open LLM Leaderboard, it predicts full scores with 3.2% mean absolute error, reducing selection cost by 18x.
R-AIRL (Reasoning Adversarial Inverse Reinforcement Learning) infers process-level reward functions from expert Chain-of-Thoughts without explicit reward definitions. Tested on GSM8K, MMLU-Pro, and MedReason: improves pass@1 by 17.4 points via inference-time reranking, outperforms SFT in post-training, localizes reasoning failures with 86.1% accuracy.
Systematic study of JEPA world models for physical planning. Authors characterize technical choices (architecture, training objective, planning algorithm) and propose a model outperforming DINO-WM and V-JEPA-2-AC on navigation and manipulation tasks. Code, data and checkpoints released.
SocialMemBench evaluates AI memory systems in multi-party social group settings. The benchmark includes 430 personas, 7,355 conversation turns, and 1,031 QA pairs across 5 social archetypes. Gemini 2.5 Flash reaches 0.721 on small networks; open-source frameworks (Mem0, LangMem, Graphiti, Cognee) plateau at 0.12-0.18, revealing a significant capability gap.
SkillGenBench is a benchmark for evaluating skill generation pipelines for LLM agents. It covers two regimes: task-conditioned generation and task-agnostic generation, with procedural sources grounded in repositories or documents. Experiments reveal substantial performance variation and distinct failure modes between software repositories and long-form documents.
BPO, a three-stage framework (bootstrapping, extrapolation, refinement), creates a self-improving data flywheel to train robust reasoning models for long-horizon sparse-reward planning. Uses planning quaternions, long-short chain-of-thought fusion, and complexity-stratified curriculum learning. SOTA on ALFWorld, ScienceWorld, WebShop with significant token efficiency.
Study of KV cache eviction policies (LRU, H2O, SnapKV, StreamingLLM, Ada-KV, QUEST, Random) under global cap. Without structural boundary protection, all collapse to F1≤0.064. Reserving 10% cache at each boundary recovers 69–90% quality on LongBench at C=256 (13% retention). Position-0 holds ~75% attention mass; protecting structurally critical tokens dominates over scoring differences.
AgentArk distills multi-agent intelligence into a single LLM through three hierarchical strategies: reasoning-enhanced fine-tuning, trajectory-based augmentation, and process-aware distillation. The resulting model preserves single-agent computational efficiency while retaining multi-agent reasoning and self-correction capabilities.
KVDrive is a multi-tier KV cache management system for long-context LLM inference, orchestrating cache placement across GPU/DRAM/SSD, pipeline scheduling, and cross-tier coordination. The prototype achieves 1.74x higher throughput than state-of-the-art systems while preserving accuracy.
Tabular foundation models (TFMs) for healthcare are distilled into lightweight models using context-leakage-aware distillation. Across 19 medical datasets and 6 TFMs, student models retain ≥90% of teacher AUC while running 26× faster on CPU, preserving calibration and fairness critical for health applications.