Systematic study comparing state space models (SSM) for time series classification. S4D outperforms Mamba variants in accuracy and efficiency. Authors introduce MS4 and MS4N, lightweight S4D variants with linear input projection and channel-mixing. Evaluation on 59 datasets (MONSTER, UEA): MS4N matches models 10× larger in parameters.
Theoretical paper on sample complexity of models predicting their own latent representations (data2vec, JEPA). Proves latent prediction reduces sample complexity from exponential in L (depth) to constant, versus token prediction. Validated on probabilistic grammars and neural networks.
HQMQ, a calibration-free KV cache compression method for LLMs, quantizes each 4-element chunk as a Hurwitz quaternion. Tested on Mistral-7B, Llama-3-8B, Qwen2.5/3-8B, and gpt-oss-20b: matches fp16 quality at ~5 bits, achieves up to 5.05× compression (Llama-3-70B: 43 GB → 8.5 GB), outperforms naive int4 by 3–1900×.
Laguna M.1 (225.8B parameters, 23.4B activated) and Laguna XS.2 (33.4B total, 3B activated) are two MoE foundation models trained end-to-end for agentic coding. Competitive on SWE-bench Verified, SWE-bench Multilingual, SWE-Bench Pro, and Terminal-Bench 2.0. XS.2 released under Apache 2.0.
EnterpriseMem-Bench, a multi-turn Text-to-SQL benchmark with 1,400 turns across 300 sessions, evaluates GPT-5 mini, GPT-5.2, Claude Sonnet 4.5/4.6, and Opus 4.6. Key findings: without memory, accuracy collapses by Turn 3; working memory dominates complex architectures; Sonnet 4.6 regresses 17-33pp on SEC EDGAR vs Sonnet 4.5.
Qwen3 improves reasoning via Self-Verified Distillation, a post-training algorithm requiring no external data. The model generates solutions, filters them through self-verification (cycle-consistency, factuality, correctness), then trains on self-curated data. Gains: +16.7 points math (AIME26/HMMT), +11.1 science (GPQA), +8.3 coding for Qwen3-4B.
SPEAR is an agentic prompt optimizer integrating a Python sandbox for structural error analysis (confusion matrices, clustering). Evaluated on 13 industrial LLM-as-judge tasks and BBH-7, it outperforms GEPA and TextGrad (κ 0.857 vs 0.359 on tool-selection; F1-macro 0.815 vs 0.763). Python tool contributes +0.79κ on complex judge tasks.
ScientistOne, an autonomous research system, introduces Chain-of-Evidence (CoE) to trace every claim to its source. Evaluation across 75 papers: baseline systems show 21% hallucinated references, 42% score verification pass rate. ScientistOne achieves 0 hallucinations, perfect verification, and matches or exceeds human expert performance on five tasks.
InfoQuant proposes a training-free post-training quantization (PTQ) method for LLMs. It uses Peak Suppression Orthogonal Transformation (PSOT) to reshape activations into quantization-friendly distributions. On LLaMA-2 13B under W4A4KV4, it preserves 97% floating-point accuracy and reduces the performance gap by 42% over prior state-of-the-art.
Study on cross-lingual retrieval asymmetry in 5 multilingual models (Gemini, Mistral, OpenAI, Qwen). Analysis of 6,518 idiomatic expressions in English, Bengali, Hindi, Arabic. Finding: hubness (vector concentration) is the dominant causal driver (49.5% dominance share), far exceeding anisotropy. CSLS correction closes 63.5% of reciprocity gap.
AgingBench, a longitudinal reliability benchmark, measures how deployed AI agents degrade over time. Study across 14 models and ~400 runs shows reliability depends on four mechanisms: compression, interference, revision, and maintenance aging. Agents lose factual precision even when behavioral tests remain clean.
ChaosBench-Logic v2 is a 40,886-question benchmark evaluating logical reasoning of 14 LLMs on 165 dynamical systems. The CARE protocol reveals critical failures: regime-transition reasoning remains near-random (MCC=0.05), while FOL deduction reaches MCC=0.52. Qwen 2.5-32B outperforms proprietary models on indicator diagnostics.
EvoCode-Bench evaluates 13 coding agents on 26 tasks with 5-15 iterative rounds. Agents must maintain a working codebase as specifications change. Results: 22-40 point gap between single-round (SR) and multi-turn (MT@4) performance, <50% success on multi-turn metrics, and progressive degradation (pass rate halved by round 5).
LLM-AutoSciLab proposes a closed-loop scientific discovery framework coupling hypothesis generation, hypothesis-conditioned experiment selection, and mechanism refinement. Evaluated on ActiveSciBench (57 enzyme-kinetics tasks, 45 gene-regulatory-network tasks), the system achieves 67.6% symbolic accuracy and 2-5x better sample efficiency than competing baselines.
Study identifies 106 dedicated neural circuits in a sparse 8-layer transformer trained on Python code. Circuits organize by computational principles (atomicity, lexical ambiguity) rather than semantics. Up to 62.5% of loudest-firing neurons at mid-to-late layers are concept-specific for AST constructs.
Study quantifying reasoning redundancy in LLMs: 61-93% of thinking steps can be truncated without affecting correct answers. Analysis across 4 frontier models and 2 math benchmarks (MATH-500). Redundancy is structural, stemming from length-agnostic outcome rewards, not model-specific artifact.
QUEST is a family of open-source models (2B to 35B) trained as deep research agents via data synthesis pipeline and RL. With only 8K synthetic tasks, QUEST matches or exceeds proprietary systems across 8 research benchmarks, excels at citation grounding and report synthesis. Models, data, and training scripts released.
BODHI, a domain knowledge prompting method, improves automated OS kernel specification generation via LLMs. Tested on 9 models (Anthropic, Mistral, Amazon, DeepSeek, Meta, Alibaba), it reaches 96.73% Pass@1 with Claude Opus 4.6 versus 55.10% baseline, by structuring C-to-Python translation across pattern categories.
Raon-Speech is a 9B multilingual speech language model (English/Korean) that understands and generates speech while preserving text capabilities. Trained on 1.38M hours of curated data, it outperforms 8 comparable audio models (Qwen2.5-Omni, Fun-Audio-Chat) across 42 benchmarks. Raon-SpeechChat extends it with real-time full-duplex conversation trained on 119K hours of dialogue.
Delta Attention Residuals improves residual connections by routing over layer deltas (vᵢ = hᵢ₊₁ − hᵢ) instead of cumulative hidden states. Results: −8.2% PPL at 7.6B, 1.8× sharper cross-layer routing (max weight 0.2→0.6), <0.01% parameter overhead. Code and paper released.
Study showing model collapse (progressive degradation of LLMs trained on their own outputs) follows cultural evolution laws. Tests on LLaMA-2-7B and Mistral-7B over 10 generations in English, German, and Turkish reveal compositionality follows non-monotonic trajectory (rise then fall). Task-grounded filtering, not random filtering, sustains quality.
FuRA introduces full-rank parameter-efficient fine-tuning via spectral preconditioning through SVD decomposition. By freezing pretrained singular bases and optimizing only compact cores via block tensor-train factorization, FuRA outperforms full fine-tuning and LoRA on LLaMA-3-8B (+1.37 commonsense reasoning) and VLMs while maintaining LoRA-comparable efficiency.
Sparse autoencoders decompose GPT-2 XL and Llama-3.1-8B into 16K-32K interpretable features per layer. Semantic features alone recover 94% of peak encoding performance (r=0.285) and align with known cortical semantic organization (ρ=0.72, p<0.001). Results generalize across English, Chinese, and French.
Brain-LLM alignment depends on training language dominance, not inherent English properties. Test on 112 participants (English, Chinese, French) with 7 LLMs: a Chinese-dominant model (Baichuan2-7B) reverses alignment gradient. Typological distance and tokenization fertility explain remaining variation.
ImProver 2 is a neurosymbolic framework for automated proof optimization in Lean 4. A 7B-parameter model trained outperforms orders-of-magnitude larger models and is competitive with mid-tier frontier models. The scaffold exposes formal structure alongside lightweight informal abstractions.
IDS (Inductive Deductive Synthesis) is a multi-agent LLM system jointly synthesizing implementation and formal proof for distributed systems. On 7 key-value store specifications, IDS achieves 7/7 in 6.8h/$106, versus 2/7 for GPT-5.4 and Claude Opus 4.6. Result is 200x faster than expert effort, 17% cheaper than SOTA agents.
hipEngine is an open source (AGPLv3) LLM inference engine optimized for RDNA3 (RX 7900 XTX, W7900). Written in Python with HIP/C++ kernels, it runs Qwen 3.6 MoE faster than llama.cpp on prefill (2718 tok/s at 512 tokens vs 2436 for GGUF Q4_K_S). Near-lossless INT8 KVCache enables 256K context in <24GB.
BitCPM-CANN demonstrates native ternary (1.58-bit) quantization-aware training on Huawei Ascend NPU. Four models (0.5B–8B) retain 95.7–97.2% of full-precision performance across 11 benchmarks (reasoning, GSM8K, BBH). Training overhead: 4.5%. Weight memory reduction: 8×, 6× end-to-end. First 1.58-bit training system scaled to 8B on domestic NPU.
Qwen3.6 35B-A3B MTP reaches 249 t/s on RTX 5090M (24GB), 3.4× faster than dense 27B variant. MoE architecture (128 experts, ~3B active params per token) combined with MTP (86.6% draft acceptance) explains the speedup. Context scaling up to 262K tokens with minimal degradation.
BeeLlama v0.2.0 delivers major performance gains with DFlash optimization. On RTX 3090: Qwen 3.6 27B reaches 164 tps (4.40x speedup), Gemma 4 31B 177.8 tps (4.93x). Full Gemma 4 31B support, reduced DFlash overhead, improved prefill handling, stricter draft/target validation.
Numind releases NuExtract3, a 4B open-weight VLM based on Qwen3.5-4B under Apache-2.0 license. The model extracts structured data from complex documents (PDFs, forms, tables, invoices) to Markdown or JSON. Trained for 3 days on 8xH100, it supports multiple quantizations (GPTQ, W8A8, FP8, Q4, Q6) and runs on 4GB VRAM minimum.
Researchers train language models to forecast empirical success of research ideas before experimentation. On 11,488 idea pairs from PapersWithCode, an 8B model reaches 77.1% accuracy via SFT, outperforming GPT-5 (61.1%). RLVR approach generates interpretable justifications with 71.35% accuracy.
AI text detectors amplify a pretrained typicality axis rather than construct an AI-vs-human boundary. On RoBERTa-base, raw projection onto centroid(AI)-centroid(HC3) achieves AUROC 0.806-0.944, matching or exceeding fine-tuning. A closed-form Jacobian predictor transfers to 16/16 third-party detectors with oracle-equivalence, reducing FPR by 57% on the OpenAI detector.
AgForce, an encoder-decoder architecture with GNN, addresses three failure modes in antibody design: antigen blindness, vocabulary collapse, and inability to generate antigen-specific sequences. Uses framework dropout, gated bottlenecks, hyperbolic attention, and Mixture Density Network. Improves amino acid recovery by 8% on CHIMERA-Bench.
Chronicle is a 324M-parameter multimodal foundation model trained from scratch on natural language and time series in a unified architecture. Both modalities share the same transformer blocks and attention mechanisms. It matches Gemma-3-270M on 19 NLU tasks, sets new benchmarks on 24 UCR/UEA datasets, and outperforms supervised fusion baselines on Time-MMD.
Study across 11 generations of self-training on 5 models (GPT-2, Pythia, OPT). Contrary to uniform 'flattening', language restructures: surface markers (connectives, em-dashes) rise while deep syntactic structures (questions, passives, subjunctives) collapse. Structural Depth Hypothesis predicts this decay (ρ=0.540, p<10⁻⁶).
Two-phase RAG system for corporate credit analysis: phase 1 combines lexical and dense multilingual retrieval; phase 2 applies adaptive controller and LLM-as-Judge scoring based on analytical utility rather than semantic similarity. On-premise deployment on proprietary multilingual corpus. Production: document review time reduced from hours to 3 minutes across 800+ analysts.
Introspective Training (IXT) uses a thinking reward model to annotate data with natural language feedback from pre-training onward. On 7.5-12B LLMs trained up to 18T tokens, the method improves compute efficiency by 2.8x and achieves performance levels unattainable otherwise in math and code domains.
CANTANTE solves credit assignment in LLM-based multi-agent systems by decomposing global rewards into per-agent optimization signals. Evaluated on MBPP, GSM8K, and HotpotQA, it outperforms GEPA and MIPROv2 (+18.9 pts MBPP, +12.5 pts GSM8K) with no inference overhead.
UCCI is an LLM cascade router using uncertainty calibration to reduce inference costs. Via isotonic regression, it maps token-level margin uncertainty to per-query error probability, then selects escalation threshold via cost minimization. On 75,000 NER queries with 4B/12B models, UCCI cuts costs by 31% while reducing calibration error from 0.12 to 0.03.