Comparative study of generic vs domain-specific embeddings for multilingual clinical search (ICD-10-CM). A bi-encoder fine-tuned on Gemini-generated synthetic data (6 languages) outperforms BioBERT-ST: R@5=0.822 vs 0.790, with major gains in Portuguese (+0.115). Open recipe for LLM-based medical retrievers.
LLM-FACETS is an open-source framework for evaluating LLM factuality, epistemic calibration, and reproducibility. Web interface, plugin architecture, deterministic metrics (BLEU, ROUGE, BERTScore) run locally, log-probability visualization, multi-judge consensus, RAG Triad metrics. Designed for technical experts, domain experts, and compliance officers per EU AI Act and NIST standards.
SubsurfaceGen is a GPU-accelerated generator for 3D velocity models and seismic data at field scale. Authors release a dataset of 4,276 2D slices covering 6 geological settings (10 km × 10 km × 6.19 km at 10 m resolution). Evaluation of neural operators on wavefield prediction and end-to-end velocity inversion with out-of-distribution testing.
EHRBench is an automated and reliable benchmark for evaluating LLMs on clinical decision-making tasks. Built via an EHR-LLM-KB pipeline, it generates ~960k QA items covering diagnosis, treatment, and prognosis. 30+ LLMs benchmarked reveal persistent gaps toward clinical reliability.
Activation steering study across four multilingual LLMs (5 figurative categories, 6 languages). Directions learned in one language transfer effectively to others, particularly German. Composite cross-lingual directions match or exceed native directions, providing direct evidence of reusable but target-dependent figurative signals across languages.
CSRM (Configurable Safety Reward Model) jointly optimizes calibrated safety compliance and reward modeling to adapt LLMs to heterogeneous and evolving safety requirements. Achieves 94.6% F1 on CoSApien and 75.8% F1 on DynaBench without additional human annotation.
Researchers train a small encoder-decoder transformer on the zeta map, a classical bijection in q,t-Catalan combinatorics. Mechanistic interpretability tools (cross-attention analysis, linear probing, causal intervention) reveal a level-based mechanism. Translation into an explicit peak-centered traversal algorithm (scaffolding map) proven equivalent to the zeta map.
ImmigrationQA: source-grounded QA dataset of 17,058 pairs across 13 U.S. immigration law subdomains. Fine-tuned Llama 3.2 3B with LoRA on corpus of 10,056 validated documents. Fine-tuned model: 1.08/3.0 (16.8% fully correct) vs Llama 3 8B base: 0.85/3.0 (4% fully correct), 27% relative improvement. Cost: ~$29. Dataset, model, and code publicly released.
A researcher built Mamba-Titan-1.4B-Reasoning (2.54B params MoE) on RTX 3060 by freezing a 1.4B Mamba-1 backbone and adding 8 trainable experts. Trained on DeepSeek CoT traces, the model developed a 'vault door' mechanism: the </think> token isolates at the smallest norm (1.991 vs 4.742 mean) to control latent reasoning termination.
Systematic comparison of 13 abliterated Gemma 4 E2B variants across 44 GPU hours. coder3101 achieves 96% ASR (refusals) with full capability preservation and outperforms base model on math. Surgical approaches preserve performance better than aggressive methods, with some losing up to 6.9 points on GSM8K.
Parallax is a parameterized Local Linear Attention mechanism for LLMs derived from statistical regression. It replaces softmax's local constant estimate with a linear estimate, yielding better bias-variance tradeoffs. Pretrained at 0.6B and 1.7B scales, Parallax shows consistent perplexity improvements and matches or outperforms FlashAttention 2/3 in decoding.
vLLM merged a PR adding native HIP W4A16 kernel for ROCm. Benchmarks show significant gains: 270.2 tk/s in fp16 (max-num-seqs=8) and 445.7 tk/s (max-num-seqs=32), outperforming previous Triton implementations.
GTA is a framework for automatically generating complex web tasks with executable trajectories. It combines crawling, retrieval, in-context generation, and quality control across 50+ websites (e-commerce, government, forums, news). The benchmark reveals a significant performance gap between humans and AI agents.
Investigation of Platonic Representation Hypothesis through tripartite framework: signal (universal linear object-attribute relationships), bias (architecture differences, mitigated by centering/normalization), noise (word frequency correlation with alignment). Sparse autoencoders show stronger cross-modal alignment than dense representations.
PrismFlow introduces a new Flow Matching method for time-series generation using Koopman-inspired dynamical experts that learn residual corrections in latent space with a confidence-aware Winner-Take-All objective. Results: +15.6% Context-FID gain and +38.6% Discriminative Score improvement across benchmarks.
Preference optimization method guided by hallucination detection to improve clinical summarization reliability. On Llama-3.1-8B-Instruct, reduces hallucinations by 24% at inference and 48% after fine-tuning, preserving fluency. Evaluated on MIMIC-IV.
GenesisFunc is an automated multi-agent pipeline for generating function-calling training data. Starting from reliable tools in public benchmarks, the system produces diverse conversations with multi-stage quality control. An 8B model fine-tuned on this synthetic data outperforms similarly-sized open-source models in in-domain performance and out-of-domain generalization.
Study of 17 models (410M-100B+ parameters) showing instruction-tuning causes linguistic entropy collapse (amplification: 1,949-16,853%), independent of RLHF. Strong control (lambda=5.0) reduces this effect by 40.5% and outperforms frontier models by 96.7-98.2% despite 200-1000x scale disadvantage.
arXiv paper proposing strategic robustness for simulator learning in MBRL. Formulates objective as minimax game between model and adversarial policy player. Proves convergence with sublinear regret bounds and Error-MDP duality. Experiments show 1.5–2.2× reduction in prediction error and simulation-trained policies matching near-optimal real-world performance.
PRO-CUA introduces a process-reward optimization framework for training computer use agents (CUAs). The method decouples live environment interaction from policy optimization through iterative step-level reinforcement learning, using a process reward model (PRM) to provide dense feedback signals without relying on expert trajectories or golden answers.
Conf-Gen adapts conformal prediction (CP) and conformal risk control (CRC) to generative models (LLMs, image generators, AI agents). The framework provides formal uncertainty guarantees for unsupervised tasks, extending conformal methodology to new domains.
OpenClawBench is a dataset of 31,264 annotated trajectories to detect process-side anomalies in agent execution beyond task success. Among 31,135 passing executions, 2,904 contain anomalies (unresolved ambiguity, unsafe writes, ignored errors). A fine-tuned Gemma 3 12B detector reaches F1=0.729.
Reasoning models maintain factually correct chain-of-thought traces but flip their final answer under sustained adversarial pressure in multi-turn dialogue. This unfaithful capitulation affects ~50% of cases in think mode and 11-15% without reasoning. The effect correlates with reasoning architecture (high in Qwen3-32B and GPT-OSS-20B, low in inline-CoT Gemma-4-31B-it).
Comparative study of RL vs SFT on Qwen2.5-3B-Instruct: reinforcement learning better preserves internal circuits of the base model than supervised fine-tuning (SFT), which adapts faster but destroys more prior capabilities. Proposed metric: differential circuit vulnerability at attention head level.
LLM agents (Claude and GPT) automatically annotate biological phenotypes by linking free-text descriptions to ontology terms. Tested on Dahrul et al. (2018) Gold Standard benchmark, all agents fall within inter-curator human variability, substantially outperforming the Semantic CharaParser NLP tool on all four metrics.
Orthogonal Concept Erasure (OCE) proposes an editing method to remove undesired concepts from diffusion models using multiplicative orthogonal transformations. Unlike existing additive approaches, OCE preserves neuron magnitude and angular geometry while precisely erasing concepts. The approach erases up to 100 concepts in 4.3 seconds.
Cross-Model Entropy (CME) proposes a label-free reward signal for LLM post-training RL. CME uses mean log-likelihood of responses under an independent verifier model, avoiding self-consistency and reward hacking. Integrated into GRPO, CME achieves 52.5–71.4% tie-adjusted win rates on UltraFeedback/AlpacaEval 2.0 across Qwen, Llama, Gemma, OLMo.
ReasonOps is an unsupervised method for analyzing chain-of-thought traces from LLMs. It identifies 7 recurring reasoning operators (backtracking, inferring, hypothesizing) from 44,662 traces across 12 models on 8 benchmarks. These operators enable source model identification at 70-76% accuracy and predict answer correctness before trace completion.
CoHyDE iteratively co-trains a dense encoder and LLM rewriter to improve tool retrieval over large API catalogs. On ToolBench (~10k tools), three rounds gain +2.5 pp NDCG@5 on standard queries and +6.3 pp on vague queries, outperforming single-component baselines.
A2X is a service discovery system for LLM agents that automatically organizes services into hierarchical taxonomies. It solves context scarcity by walking the hierarchy layer-by-layer, reducing token consumption by 89% while gaining 6.2 Hit Rate points over full-context dumping and +20 points over embedding baselines.
OISD introduces on-policy internal self-distillation to improve language model reasoning. The final layer acts as a detached teacher for intermediate layers via logit alignment (reasoning behaviors) and attention alignment (attention patterns), without external privileged information. Positive results across four mathematical reasoning tasks.
Study of log-alignment ratio (LAR), a parameter-activation alignment metric. LAR predicts memorization-to-generalization transition in grokking (effective dimension k ≈ n^(2(1-LAR))) and 3B-parameter language model pre-training. Computable without validation data, negligible overhead.
Study on source-dependence in multi-source medical RAG systems. Authors demonstrate that the same system can produce different answers depending on retrieved source, revealing a missing evaluation axis in NLP. They introduce TransplantQA (benchmark), HERO-QA (hierarchical retrieval strategy), and a structured judge to audit inter-source relationships using a validated 5-label taxonomy.
UA-Legal-Bench evaluates 11 LLMs (3B–675B) on 5 Ukrainian legal reasoning tasks from 99.5M court decisions. Results show task-dependent few-shot effects: +38.6 pp improvement for judgment form classification, but mixed effects on outcome prediction. Accuracy is misleading on imbalanced tasks: highest accuracy model (62%) is a majority-class predictor (macro-F1: 23%).
COLAGUARD, a guardrail model, transfers multi-step safety reasoning into continuous latent space via stage-wise training curriculum. Evaluated on 10 moderation tasks across 8 safety benchmarks, it improves macro-F1 by 8.24 points over Llama Guard 3, matches GuardReasoner performance while delivering 12.9X speedup and 22.4X token reduction.
Study of chain-of-thought (CoT) transfer across models using a provider-receiver framework. Full traces often transfer successfully, but mechanisms vary: answer extraction (AIME), receiver competence (MMLU-Pro), or partial structured information (ZebraLogic). In free-generation mode, partial CoTs improve performance, suggesting guidance for continued reasoning.
Model merging formulated as convex quadratic programme over residual updates. Subsumes existing methods (task arithmetic, model soups, TIES, DARE) and provides closed-form diagnostic predicting merge quality via fraction of residual energy captured. Consistent gains across language and vision benchmarks.
Domain-specific NER model for identifying clinical entities in immunology and infectious disease contexts. 371 manually annotated case reports by clinical specialists. Transformer-based model with clinical embeddings achieves F1=0.89, outperforming BERT and zero-shot approaches. Supports case report analysis and clinical decision support.
Anthropic releases Claude Opus 4.8, outperforming GPT-5.5 and Gemini 3.1 Pro on most benchmarks. The model catches its own coding errors 4× better than its predecessor. Anthropic also rolls out dynamic workflows enabling hundreds of parallel sub-agents for codebase-wide migrations.
Wall-OSS-0.5 is a 4B VLA from X Square Robot with open training code. Zero-shot evaluation on 17 real-robot tasks: 4 tasks >80% progress, including Rope Tightening (82%). Post fine-tuning: 60.5% average task progress (+17.5pp vs pi0.5). Mixture-of-Transformers architecture with vision-aligned RVQ tokenizer and distributed DMuon optimizer.