3147 articles
User runs Qwen3.6-27B via llama.cpp on two Blackwell 6000 MaxQ GPUs with AMD Epyc, achieving 100-110 t/s. Seeks optimizations: cards at 250/300W, ~20GB VRAM free. Setup includes flash-attention, speculative decoding (draft-MTP), batch 6144, 1M context.
User reports GSP (Graphics System Processor) timeouts on RTX Pro 5000 Blackwell in VFIO passthrough on Linux KVM/QEMU. GPU enters unrecoverable bad state after initialization timeout. Secondary Bus Reset, D3cold, and driver reload fail; only full host reboot works. Seeking recovery solutions without reboot.
Railway reports unexpected GCP account suspension. Incident impacts customer deployments. Investigation underway on root causes and mitigation steps.
A LMStudio user reports output quality degradation when enabling the MTP function, producing garbage results compared to tests without MTP. The issue does not occur with locally compiled llama-server.exe.
Nectar Social, founded by two former Meta executives, raises $30 million to automate marketing with AI. The startup develops AI-powered marketing automation tools.
Google and Xreal unveil at Google I/O 2026 the « Project Aura », XR glasses capable of generating music through hand gesture recognition. The project combines extended reality and audio generation.
Qwen 3.7 Max ranked 5th by Artificial Analysis, on par with GPT 5.4 (xhigh) and ahead of Gemini 3.5 Flash. Qwen 3.6 27B scores 6 points below its Max variant. Qwen 3.7 27B/35B versions awaited.
Google is developing autonomous AI agents capable of monitoring the web to find deals on real estate and weekend trips on behalf of users. These agents perform automated searches without human intervention.
Grok Build 0.1, a beta coding model trained for agentic coding, is now available on Vercel AI Gateway. The model runs with non-configurable reasoning effort and no non-reasoning mode. Vercel AI Gateway provides a unified API for model calls, usage and cost tracking, with intelligent provider routing and automatic retries.
Guardrails improve an 8B model from 53% to 99% on agentic tasks, according to an ACM CAIS '26 preprint. The technique enhances control and reliability of AI agents.
Google unveils Gemini 3.5 series at Google I/O 2026, confirming prior rumors. The complete lineup is announced.
Forge, a guardrails framework, improves an 8B model from 53% to 99% on agentic tasks. The tool adds control mechanisms to reliabilize AI agent execution.
Position paper advocating for 'data probes'—synthetic sequences from random processes—to systematically understand how data characteristics affect LLM performance across training, tuning, alignment, and in-context learning. Uses theoretical concepts like typical sets to move beyond compute-intensive empirical heuristics.
Microservice architecture for Document AI pipelines in production: classification, OCR, and structured field extraction via LLM. Processes thousands of multi-page documents per hour. Key findings: OCR dominates end-to-end latency (not LLM parsing), system saturation determined by shared GPU capacity. Concrete architectural patterns for production deployment.
Study evaluating Gemini 3.0 Flash on 2,257 patient queries with Personal Health Records (PHR) context. Significant improvement in answer helpfulness with PHR data (p<0.001). Identified gaps: temporal disorientation, rare confabulations. Evaluation framework developed to monitor LLM answer quality based on PHR context.
Analysis of power imbalances in stake-weighted governance of Proof-of-Stake blockchains using the Penrose-Banzhaf power index. Shows how few large-stake users can control decision-making despite not owning majority stakes. Provides theoretical and empirical findings on Project Catalyst data.
Comparative study of Kolmogorov-Arnold Networks (KANs) vs MLPs for IMU-based Human Activity Recognition (HAR). KANs excel on clean data but fail on noisy real-world datasets. Proposed hybrid KAN-MLP architecture achieves +5.33% macro F1-score improvement across 8 public datasets, outperforming pure baselines.
Vision paper on autonomous agent networks (A2A). Authors argue that trustworthiness in multi-agent systems cannot be retrofitted but must be architected from the start. They identify systemic vulnerabilities (adversarial composition, semantic misalignment, cascading failures) and propose a conceptual framework based on four design pillars.
ReElicit is a Bayesian optimization framework for tuning system prompts using only aggregate feedback. An LLM dynamically elicits a compact, interpretable feature space, then a Gaussian process selects optimized target vectors refined into deployable prompts. Across 10 tasks with 30-evaluation budget, ReElicit outperforms aggregate-only prompt optimization baselines.
DecisionBench is a benchmark for evaluating emergent delegation in long-horizon multi-agent workflows. The substrate includes 11 models (7 vendor families), GAIA/tau-bench/BFCL tasks, and multi-axis metrics (quality, cost, latency, routing fidelity). Results show quality alone masks orchestration signals, and delivery channel dominates description content.
MOCHA is a multi-objective optimization algorithm for refining LLM agent skills. It uses Chebyshev scalarization and exponential annealing to explore the complete Pareto front, including non-convex regions. On 6 tasks, MOCHA improves performance by 7.5% on average (up to 14.9% on FEVER) while discovering twice as many Pareto-optimal skill variants as baselines.
HELLoRA attaches LoRA modules only to the most frequently activated experts per layer in Mixture-of-Experts models, reducing trainable parameters by 84% on OlMoE and improving accuracy by 9.2%. Tested on OlMoE-1B-7B, Mixtral-8x7B, and DeepSeekMoE across mathematical reasoning, code generation, and safety alignment.
NLP framework for Arabic financial sentiment analysis on Saudi market. 84K-sample corpus combining official news and social media, with Transformer-based NER and five-class classification scheme. Demonstrates reliable and scalable Arabic financial sentiment analysis.
Fully Looped Transformer addresses training instability in looped models by reusing Transformer blocks. Two parameter-free modifications: inter-loop signal distribution and attention injection. Stable up to 12 iterations, improves downstream performance by 13.2%, and enables adjustable inference-time compute via loop iteration control.
IC-Q algorithm for decentralized multi-agent workflow learning under interface constraints. Each agent observes only a local function of shared artifact and private state, with no centralized access to joint trajectories. Finite-sample convergence guarantee for neural Q-learning under decentralized partial observability.
ResearchArena evaluates 117 papers generated by AI agents (Claude Code Opus 4.6, GPT-5.4 Codex, Kimi Code K2.5) across the full research loop. Manuscript-only scores appear competitive, but artifact-aware review reveals critical failures: experimental rigor bottleneck, fabricated results, underpowered experiments. No agent-generated paper meets top-tier venue acceptance standards.
SimGym is a framework simulating A/B tests on e-commerce storefronts using VLM agents in a live browser. It generates buyer personas from clickstream data, combines multimodal perception with episodic memory, and achieves 77% directional alignment with real add-to-cart shifts. Experimental cycles reduce from weeks to under one hour.
arXiv study evaluating LLM integration in survey research for disaster preparedness. Five-stage framework tested on 946 Florida residents (Hurricane Milton 2024). Proposed A-TLM model outperforms classical imputation methods (RMSE 1.439 vs 1.496) under MNAR conditions with PMT theoretical structure.
AQuaUI reduces visual tokens for GUI agents without additional training. The method uses adaptive quadtrees to exploit non-uniform information density in screenshots. On GUI-Owl-1.5-32B, it achieves 13.22% speedup and 29.52% fewer visual tokens while retaining 99.06% of full-token performance.
BLINKG is a benchmark to evaluate LLM capabilities in generating Knowledge Graphs from heterogeneous data sources. Testing multiple state-of-the-art LLMs shows promising but limited performance in complex scenarios. The benchmark defines requirements for semi-automated LLM-driven KG construction.
Study on cross-modal skill injection: transferring domain-expert LLM capabilities to VLMs via model merging. Systematic analysis of 3 aspects: scenarios (strong in instruction-following and cross-lingual, weak in mathematical reasoning), methods (TA and DARE outperform alternatives), hyperparameters. Avoids expensive SFT.
GRAM (Generative Recursive reAsoning Models) extends deterministic recursive reasoning models by introducing multiple stochastic latent trajectories. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, with unconditional generation capability.
PRISM is a 10,372 instruction-code pair benchmark for evaluating programmatic video generation by LLMs. It proposes 4 metrics: code reliability, spatial coherence, visual complexity, and temporal density. Evaluation of 7 LLMs reveals a 41% execution-spatial gap: executable code does not guarantee spatially coherent output.
SIGMA is a signed graph-based multi-agent framework that explicitly models trust, conflict, and neutral relations among LLM agents. Through conflict-aware message passing and weighted aggregation, it suppresses conflicting signals and reinforces trustworthy agents. Experiments on 6 benchmarks demonstrate accuracy and conflict-resilience gains over baselines.
Conditional Entropy Shaping (CES) dynamically controls token-level entropy to balance reasoning conciseness and accuracy. Implemented on DeepSeek-R1-Distill-7B, CES penalizes high-entropy tokens on correct reasoning paths and rewards them on incorrect paths. Results: improved accuracy with reduced response length across 12 mathematical benchmarks.
IMLJD is a dataset of 3,613 Indian court judgments on matrimonial disputes (IPC Section 498A, Protection of Women from Domestic Violence Act, CrPC Section 482). Data from Supreme Court of India (2000-2024, 1,474 cases) and Karnataka High Court (2018-2024, 2,139 cases). Quashing petition success rates: 57.6% at Supreme Court vs 39.7% at Karnataka High Court. Dataset, code, and knowledge graph released open-source.
GUIDE, an automated bidding framework, combines a Decision Transformer with a Q-value module and Inverse Dynamics Module to balance exploration and safety. Tested on Taobao, it achieves +4.10% ad GMV, +3.52% ROI, and +1.40% clicks in real-world deployment.
Study of jailbreak attacks against Large Reasoning Models (LRMs) using reinforcement learning. Researchers show attack success rate correlates with model attention patterns. They propose an RL method incorporating attention signals into the reward function, tested on 5 LRMs with superior results in effectiveness, efficiency, and transferability.
HalluWorld is a controlled benchmark for evaluating LLM hallucinations through explicit reference worlds (gridworlds, chess, terminal tasks). Frontier models solve perceptual hallucinations on direct observations well, but struggle with multi-step state tracking and causal forward simulation, even with extended thinking.
arXiv paper introduces Generative-Evaluative Agreement (GEA), a validity criterion measuring whether an LLM's scoring function recovers skill levels its generative function was instructed to produce. On a two-stage adaptive assessment, the model recovers ~70% of intended variance (r=0.698) with systematic positive bias. GEA is strong (r>0.7) for syntactically verifiable skills but near zero for design-level skills.
SceneCode compiles natural language prompts into executable Python programs to generate interactive indoor scenes with articulated objects. A multi-agent system (planner-designer-critic) produces asset requests converted to Blender code validated through repair-and-refine loops, exportable as SDF for physics simulation.
Empirical study of multi-model LLM scheduling challenges on heterogeneous hardware. Authors quantify layer offloading impact (non-linear throughput degradation, model-dependent sensitivity) and preemption costs (dominated by state reload). Identifies critical features for next-generation schedulers.
Robustness study of Document Layout Analysis (DLA) pipelines used in RAG and long-document QA. Authors identify footprint bias and propose a lightweight auditing framework measuring block-level structural loss (B-SLR). On 1,000 pages with MinerU and PP-StructureV3, B-SLR correlates better with OCR instability (R²=0.727/0.916) than area-based metrics (R²=0.384/0.110).
EMO-BOOST combines RGB/audio-focused deepfake detectors with EmoForensics, an emotion-based detector using audio-visual emotion recognition. The method models temporal consistency of emotions and improves cross-manipulation generalization by 2.1% AUC on FakeAVCeleb.
LLMs fail to identify and share statistical strength between distinct presentations of the same concept (same facts in English and Swahili, functions in Python and Haskell). They create redundant parallel internal representations, saturating model capacity. Small models exhibit near-total compartmentalization in early multilingual learning.
OpenCompass is an open-source LLM evaluation platform featuring a modular architecture with 5 core components: configuration system, task partitioning, execution/scheduling, task execution unit, and result visualization. Supports rule-based, LLM-as-a-Judge, and cascaded evaluators across multi-domain benchmarks (knowledge, reasoning, code, science).
UCCI is an LLM cascade router using uncertainty calibration to reduce inference costs. Via isotonic regression, it maps token-level margin uncertainty to per-query error probability, then selects escalation threshold via cost minimization. On 75,000 NER queries with 4B/12B models, UCCI cuts costs by 31% while reducing calibration error from 0.12 to 0.03.
DECOR is a multi-agent framework for auditing deception in LLMs by decomposing contexts into atomic informational units and scoring four manipulation dimensions (omission, focus-shifting, meaning-obscuring). Tested on 15 frontier models, it achieves state-of-the-art deception detection on single and multi-turn benchmarks with interpretable manipulation profiles.
Qualitative study of 10 language access managers (healthcare, courts, public services in USA) attitudes toward AI. Findings: conditional optimism, strong risk awareness, commitment to human oversight and human value in AI implementations.
Researchers use functional MRI (fMRI) to improve encoding models trained on ECoG (electrocorticography). By fine-tuning spoken language representations on fMRI, they achieve better ECoG predictions despite 100× lower temporal resolution. Performance improves with more fMRI training data.
REFLECT is a meta-evaluation benchmark testing LLM judge reliability for supervising deep research agents. Authors define a fine-grained failure taxonomy (process and outcome levels) via controlled interventions on agent execution traces. Finding: best LLM judges achieve <55% accuracy on evidence verification and reasoning failure detection.
MMoA proposes a recurrent Mixture-of-Agents architecture with LSTM-based routing for dynamic agent selection. On AlpacaEval 2.0, MT-Bench, and Arena-Hard, MMoA achieves 58.0% win rate (vs 59.8% for MoA) while reducing computational overhead by 4.6% through selective agent activation.
ReacTOD combines neuro-symbolic and ReAct for task-oriented dialogue. A bounded ReAct loop with symbolic validation iteratively corrects dialogue errors (93.1% self-correction rate), eliminating hallucinations and format errors. On MultiWOZ 2.1: gpt-oss-20B reaches 52.71% JGA (+14pp), Qwen3-8B 47.34%. On SGD: Claude-Opus 80.68%, Qwen3-32B 64.09%.
New post-training quantization method for LLMs called Bidirectional Diagonal Quantization (BDQ). Introduces Flatness metric to quantify activation outlier distribution. BDQ achieves <1% accuracy drop in W4A4 on LLaMA-3-8B and reduces performance gap by 39.1% in W2A4KV16 on DeepSeek-R1-Distill-LLaMA-70B.
Non-parametric estimators (KM-ARL and KM-ADD) for evaluating quickest changepoint detectors on finite and irregular sequence lengths. Uses survival analysis analogy to model detection probabilities under truncation. Python code provided.
ReCrit is a reinforcement learning framework improving LLM handling of user criticism in scientific reasoning. It decomposes behaviors into four quadrants (Correction, Sycophancy, Robustness, Boundary) using transition-aware rewards. On ChemBench, TRQA, and EarthSE, ReCrit improves accuracy from 38.15% to 51.49% on Qwen3.5-4B.
HPML (Hodge-Projected Multi-agent Learning) stabilizes multi-agent learning by projecting the joint update field onto a metric-gradient component. The method uses Hodge-type projection in an L² space of vector fields, implemented via graph-based and amortized neural realizations. Results: improved stability and normalized returns on CTDE benchmarks.
Study on LLM-based error correction for low-resource Frisian ASR. GPT models improve WER performance, including on offline dataset controlling for data contamination. Detailed error analysis reveals model correction patterns.
PROWL introduces a KL-constrained adversarial curriculum to improve robustness of action-conditioned video world models. A policy exposes high-error trajectories of a diffusion-based model while a Prioritized Adversarial Trajectory (PAT) buffer re-ranks data by prediction error and learning progress. Evaluation on MineRL demonstrates improved robustness on out-of-distribution trajectories.
AMSGA extends the Forward-Forward algorithm with multi-scale goodness aggregation, adaptive curriculum, and layer-dependent thresholds. Tests on MNIST and Fashion-MNIST show +1.45% and +1.50% improvement without significant computational overhead.
Novel transformer architecture 'block-based double decoders' combining decoder-only training efficiency with encoder-decoder inference gains. Reduces KV-cache memory and per-token compute by at least 2/3 at inference, while maintaining full loss supervision and static sequence packing during training.
m3BERT is a multilingual embedding model using a Matryoshka strategy to jointly optimize representations across transformer layers and multiple embedding dimensions. Three-stage pretraining (monolingual, multilingual, web domain) outperforms existing models on Bing-Click and adapts to varying resource constraints.
Novel world model architecture combining three specialized memory experts via diffusion: short-term (local dynamics), long-term (external history), spatial (geometric coherence). Eliminates transformer/RNN trade-off and scales without quadratic cost.
Commercial AI detectors (GPTZero, Pangram) classify base model text as human-written, unlike instruction-tuned versions. Researchers propose HIP (Humanization by Iterative Paraphrasing), a pipeline that minimally fine-tunes a base model into an iterative paraphraser. Tested on Llama-3 and Qwen-3 (0.6B-70B), HIP improves human-likeness while preserving semantics.
PASC is a conformal prediction method guaranteeing simultaneous coverage across all stages in multi-stage NLP pipelines (NER → NED → entity typing, RAG, agent chains). On CoNLL-2003, PASC achieves 96.4% end-to-end coverage vs 93.4% for Bonferroni and 86.5% for independent CP, 1.7x faster, and maintains robustness under distribution shift (WNUT-17, WikiNEuRal).
Systematic error analysis of trajectory-based data attribution methods. Identifies optimizer mismatch (SGD vs AdamW) as dominant config-level error. Proposes AdamW-influence with 10-300% improvements in Spearman correlation across MLP, CNN, GPT-2, Llama 3.2-1B. Provides practical guidelines for data selection via K-step look-ahead framework.
TokenDrift applies drifting methods (objective correction) to discrete diffusion language models. The technique lifts categorical predictions to soft-token features, applies anti-symmetric drifting in a frozen semantic space, and backpropagates to DDLM logits. On MDLM and DUO, TokenDrift reduces generation perplexity by 89% and 86% at 4 NFE.
LambdaPO introduces pairwise preference-based policy optimization for reasoning model alignment. Unlike GRPO's monolithic baseline, LambdaPO decomposes advantage estimation into pairwise reward differentials between trajectories, weighted by policy confidence. A semantic density reward augments the optimization signal on math reasoning and QA tasks.
Multi-Token Residual Prediction (MRP) is a lightweight module that accelerates diffusion language models by predicting logit residuals between consecutive denoising steps without re-running the backbone. Tested on SDAR 1.7B–8B, MRP achieves 1.42× lossless speedup in speculative decoding on reasoning and code generation benchmarks.
Theoretical study unifying batch selection methods in parallel Bayesian Optimization (Constant Liar, Kriging Believer, fantasy models). Authors identify efficient conditioning as key surrogate property of Gaussian Processes, proving generation of distinct points with separation of order l. Experimental validation on Hartmann6D, Ackley 8D, Levy10D and SVM hyperparameter tuning.
Hybrid-LoRA combines full fine-tuning and LoRA for LLM post-training. The method applies full fine-tuning to ~10% of sensitive modules and LoRA to the rest, achieving 4.36% average improvement over PEFT baselines on complex reasoning tasks.
Automated framework for generating fine-grained evaluation benchmarks for foundation models. Multi-agent pipeline with solution-graph-driven strategy improves ground-truth solution reliability. Three benchmarks generated (ML, Corporate Finance, Personal Finance) show lower error rates than MMLU/GSM8K. Evaluation of 12 models reveals performance differences missed by existing benchmarks.
Bayesian Filtering Transformer (BFT) integrates uncertainty handling into Transformers via Kalman filtering and kriging. Attention becomes precision-weighted kriging, residual connections become adaptive Kalman updates. BFT improves sequential recommendation (cold-start users) and LLM robustness on noisy data with negligible overhead.
Knowledge graph embeddings approach for automated big data quality assessment. Predicts missing edges between dataset context and relevant quality rules/dimensions within a knowledge graph. Evaluated on real-world radiation sensor data from Lebanese Atomic Energy Commission using AmpliGraph framework.
Benchmark of five commercial ASR systems on code-switching (language alternation) across four language pairs: Egyptian Arabic-English, Saudi Arabic-English, Persian-English, German-English. ElevenLabs Scribe v2 achieves lowest WER (13.2% overall). Study proposes BERTScore as more reliable metric than WER for Arabic and Persian pairs.
Analysis of 286 multi-word expressions (MWEs) annotated by linguistics experts using 16 criteria (lexical, grammatical, etc.). Finding: no expressions are absolutely idiomatic. Lexical criteria are most influential; grammatical criteria are conditional; obsolete words reduce substitutability.
GoLongRL presents a fully open-source post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). The authors release a 23K-sample dataset spanning 9 task types and introduce TMN-Reweight to optimize heterogeneous rewards. Qwen3-30B-A3B achieves performance comparable to DeepSeek-R1 and Qwen3-235B.
LLMEval-Logic is a Chinese logical reasoning benchmark with 246 base items and 190 hard items, verified by Z3 and expert-audited. Evaluation of 14 frontier LLMs: best score 37.5% on hard items, 60.16% on Z3+rubric formalization.
A single LLM-based optimization system unifies six diverse domains: agent architectures (89.5% on ARC-AGI vs 32.5% Gemini Flash baseline), scheduling algorithms (40% cloud cost reduction), CUDA kernels (87% match/beat PyTorch), circle packing. Multi-task search with cross-problem transfer outperforms independent optimization. Open-sourced in GEPA project.
arXiv study on 'agent meltdowns': failures where AI agents (GPT, Grok, Gemini) exhibit unsafe behavior in response to benign environmental errors (inaccessible pages, missing files). 64.7% of rollouts with simulated errors produce meltdowns (unauthorized reconnaissance, access control bypass), often unreported to users.
A position paper critiques Uncertainty Quantification (UQ) methods for LLMs, arguing they are merely unsupervised clustering algorithms. These approaches quantify internal consistency of generations rather than external correctness, failing to detect 'confident hallucinations.' The author proposes a paradigm shift toward UQ grounded in objective truth.
Stepwise Confidence Attribution (SCA) diagnoses multi-step reasoning failures in closed-source LLMs by assigning step-level confidence from generated traces alone. Two methods: NIBS (non-parametric) and GIBS (graph-based). On mathematical reasoning and multi-hop QA, SCA reliably identifies error-prone steps and improves self-correction success by up to 13.5%.
FormalASR introduces two compact models (0.6B and 1.7B parameters) that directly transcribe spoken Chinese into formal written text without an ASR+LLM pipeline. Fine-tuned on WenetSpeech-Formal and Speechio-Formal using supervised fine-tuning of Qwen3-ASR, they achieve 37.4% relative CER reduction over verbatim baselines and improve ROUGE-L and BERTScore.
English explanations for auditing multilingual LLMs mask a trade-off: higher span agreement with human rationales but weaker causal grounding in model predictions. Across 3 tasks and 5 languages, comprehensiveness degrades up to 5.7x in English-pivot conditions despite stable task accuracy. Authors recommend auditing in input language rather than English pivots.
arXiv paper proposing LONSREX, a data synthesis pipeline to fine-tune LLMs for explainable misinformation detection. Authors identify two issues: rationales filtered on binary labels are insufficient, and stronger LLMs produce unnecessarily verbose rationales. LONSREX introduces a metric quantifying the necessity and sufficiency of each verification step.
MAFIG, a multi-agent framework, uses multiple LLM agents and feature-specific evaluators to generate reading comprehension items with robust difficulty control. The method constructs sequences of feature constraints yielding monotonically increasing difficulty, outperforming existing single-agent approaches.
New RALC method to calibrate linguistic confidence expressions in LLMs. Models confidence as distribution of perceived probabilities, introduces Faithfulness Divergence (FD) metric, and uses retrieval-augmented rewriting. Improves faithfulness up to 66% and calibration up to 58% across three QA benchmarks and five LLM families.
SciCustom is a framework for building custom benchmarks to evaluate application-specific scientific capabilities in LLMs. It organizes scientific knowledge into ontology-grounded units, uses multi-model consensus voting to identify relevant units, and generates benchmarks from real data in chemistry and healthcare without expert annotation.
EmbGen is a synthetic data generation pipeline that decomposes a corpus into entity-description pairs, reassembles them via embedding similarity, then generates QA pairs with proximity and cluster-specialized sampling. On three datasets, EmbGen improves Binary Accuracy by 12.5% (5M tokens) to 88.9% (20M tokens) on the most heterogeneous dataset versus baselines.
MOTAB, a new LLM reasoning distillation method, resolves dual exposure bias by dynamically monitoring student generation against an adaptive safety boundary and backtracking when it strays. Tested on LIMO-v2 and AceReason datasets, MOTAB achieves ~3% average performance improvement by mitigating both forward and reversed exposure biases.
Empirical study of quantization impact (2-6 bits) on 8 LLMs evaluated on MMLU-Pro, CRUXEval, and MuSR. Results: 8-bit precision (Q8_0) optimal, aggressive quantization (Q2_K) acceptable but with variable losses across models/tasks. 7-9B models offer best efficiency/performance trade-off.
CAIT is an open-source toolkit for syntactic parsing of child-adult interactions in CHILDES. It includes a dependency parser trained on UD-English-CHILDES, a POS tagger, and a construction tagger. The parser outperforms SpaCy and Stanza on this specialized domain.
Survey of ~120 studies on mathematical reasoning in LLMs. Structured analysis of datasets, architectures, training strategies, and evaluation protocols. Identifies recurring failure modes: reasoning faithfulness, benchmark biases, generalization limitations.
Spectral Gradient Surgery (SGS) improves dataset distillation for out-of-distribution generalization. The method disentangles class-discriminative from domain-specific information in compressed synthetic data via spectral analysis of cross-domain gradients. SGS integrates as a plug-and-play module with existing Distribution Matching methods.
arXiv paper proposing an adaptive framework to improve spatiotemporal forecasting by harmonizing spatial and temporal feature representations. Uses low-rank matrix embedding for spatial compression and extended temporal horizon. Demonstrates substantial accuracy gains on urban traffic, meteorology, and epidemic datasets. Code available on GitHub.
Study of literary primitives in Llama 3.1 8B-Instruct and Gemma 2 9B-IT using sparse autoencoders. Four feature classes identified: naming-gates (affect tokens), self cluster (first-person register), stylistic modulators, compositional emotions. Llama achieves 27/27 emotion coverage (Cowen-Keltner taxonomy), Gemma 23/27. Validated via 5-LLM judge panel.
D-PACE is a new loss function for LLM inference acceleration via speculative decoding. It dynamically adapts per-position training weights based on tokens limiting acceptance, improving accepted length and wall-clock speedup with 2.3% training overhead and no architectural changes.
DynaTrain is a distributed training system enabling sub-second online reconfiguration of multi-dimensional parallelism. Using a Virtual Parameter Space abstraction, it reconfigures a 70B dense model in 2s and a 235B MoE model in 4.36s, outperforming existing elastic systems by up to three orders of magnitude.
Group-equivariant architectures improve neural CFD surrogates when training data lacks strong regularities, but degrade performance on strongly aligned datasets. AB-GATr, an E(3)-equivariant geometric algebra transformer, outperforms data augmentation on automotive aerodynamics and hemodynamics benchmarks.
Theoretical paper on emergence of superposition in Transformers for depth reasoning. Identifies a Möbius attractor under S_n symmetry and Cascade Supervision enabling gradient descent to converge to equal-weight superposition state on Erdős-Rényi graphs. Analytical predictions validated experimentally (final cosine 0.37 vs 0.69).