Comparative study of generic vs domain-specific embeddings for multilingual clinical search (ICD-10-CM). A bi-encoder fine-tuned on Gemini-generated synthetic data (6 languages) outperforms BioBERT-ST: R@5=0.822 vs 0.790, with major gains in Portuguese (+0.115). Open recipe for LLM-based medical retrievers.
LLM-FACETS is an open-source framework for evaluating LLM factuality, epistemic calibration, and reproducibility. Web interface, plugin architecture, deterministic metrics (BLEU, ROUGE, BERTScore) run locally, log-probability visualization, multi-judge consensus, RAG Triad metrics. Designed for technical experts, domain experts, and compliance officers per EU AI Act and NIST standards.
HypoAgent is a multi-agent framework for interactive abductive hypothesis generation over knowledge graphs. Three coordinated agents (intent recognition, hypothesis generation, root cause analysis) enable multi-turn dialogue and fine-grained diagnosis of failed hypotheses. SOTA on commonsense and biomedical KGs.
TraceGraph is a graph-based framework that transforms multi-model agent trajectories into shared decision landscapes. It builds graphs over state-action-observation spaces, identifies productive cores and trap regions, then proposes a trap-aware recovery pipeline. On SWE-bench, this approach improves resolution rate from 40.4% to 43.5%.
Study of stylistic signatures introduced by LLM alignment. Researchers show post-training creates a detectable AI-like style. They propose PASTA, a training-free method that localizes and ablates this signature during decoding, reducing detection rates across 11 aligned models and 6 AI detectors.
Evaluation of semantic stability in 16 LLMs (general-purpose and medical) under clinically equivalent prompt reformulations. Proposes NLI-based verification framework and three sensitivity metrics (MVS, ΔC, WCI). Finding: domain specialization does not consistently improve robustness to meaning-preserving variations.
EUDAIMONIA is a benchmark evaluating harmful social dynamics in LLMs. It contains 969 user inputs and 3,147 design-violation checks, testing 22 recent models. Claude-Opus-4.7 and GPT-5.5 violate 30.7% and 27.2% of checks respectively, revealing persistent social-alignment failures not resolved by extended thinking.
EHRBench is an automated and reliable benchmark for evaluating LLMs on clinical decision-making tasks. Built via an EHR-LLM-KB pipeline, it generates ~960k QA items covering diagnosis, treatment, and prognosis. 30+ LLMs benchmarked reveal persistent gaps toward clinical reliability.
Activation steering study across four multilingual LLMs (5 figurative categories, 6 languages). Directions learned in one language transfer effectively to others, particularly German. Composite cross-lingual directions match or exceed native directions, providing direct evidence of reusable but target-dependent figurative signals across languages.
TeachObs is a human-validated multimodal benchmark for classroom video analysis. It contains 30 public lessons from 8 countries split into 5,158 15-second scenes, annotated by 7 researchers with 39 observation codes (20 visual, 19 non-visual). Evaluation of 5 vision-capable LLMs across 3 tasks: no single model consistently outperforms others.
SAGE is an adaptive gate using von Mises-Fisher density estimation to control memory evolution in agentic LLMs. It classifies candidate facts as ADD (novel), NOOP (redundant), or MERGE (uncertain), reducing expensive LLM calls. On LoCoMo, SAGE cuts API cost by 3.4× and latency by 2.5× with GPT-4o-mini.
PhyDrawGen is a neuro-symbolic pipeline generating physics diagrams from text while respecting physical laws. An LLM extracts a typed scene graph, a deterministic solver converts it to a planar straight-line graph, and Qwen-VL fine-tunes a propose-verify loop. Evaluated on 1,449 problems (mechanics, optics, electromagnetism), it outperforms GPT-5-image and Gemini.
Distributed approach for constrained multi-agent reinforcement learning combining state-augmented policy learning with consensus over Lagrange multipliers. Agents learn offline policies and coordinate via local communication. Linear scalability to thousands of agents, demonstrated on smart grid demand response.
CSRM (Configurable Safety Reward Model) jointly optimizes calibrated safety compliance and reward modeling to adapt LLMs to heterogeneous and evolving safety requirements. Achieves 94.6% F1 on CoSApien and 75.8% F1 on DynaBench without additional human annotation.
SubsurfaceGen is a GPU-accelerated generator for 3D velocity models and seismic data at field scale. Authors release a dataset of 4,276 2D slices covering 6 geological settings (10 km × 10 km × 6.19 km at 10 m resolution). Evaluation of neural operators on wavefield prediction and end-to-end velocity inversion with out-of-distribution testing.
AMNESIA is the first large-scale open-source benchmark for machine unlearning in medical LLMs. It contains 70,560 question-answer pairs from 8,820 patient notes across 11 disease categories. The authors evaluate 4 unlearning methods and show that forgetting individual patients erodes knowledge of others with the same condition.
CanLegalRAGBench is an evaluation benchmark for RAG systems applied to Canadian law, based on realistic queries and expert-annotated answers. The study shows open-source embedding models are competitive with closed-source alternatives, but identifies hallucinations in 8-29% of generated answers unsupported by retrieved documents.
ImmigrationQA: source-grounded QA dataset of 17,058 pairs across 13 U.S. immigration law subdomains. Fine-tuned Llama 3.2 3B with LoRA on corpus of 10,056 validated documents. Fine-tuned model: 1.08/3.0 (16.8% fully correct) vs Llama 3 8B base: 0.85/3.0 (4% fully correct), 27% relative improvement. Cost: ~$29. Dataset, model, and code publicly released.
Study on long-term effects of data selection during multi-stage LLM fine-tuning. Authors show that short-term optimal strategies (loss-based, gradient-based, diversity-based) can slow future learning and increase catastrophic forgetting. They propose LHAS (Long-Horizon Aware Selection) to evaluate selection as a global training intervention.
Researchers train a small encoder-decoder transformer on the zeta map, a classical bijection in q,t-Catalan combinatorics. Mechanistic interpretability tools (cross-attention analysis, linear probing, causal intervention) reveal a level-based mechanism. Translation into an explicit peak-centered traversal algorithm (scaffolding map) proven equivalent to the zeta map.
Study on KV cache prompt redundancy during decoding. Researchers show upper-layer prompt cache can be replaced with chat template scaffolds without significant accuracy loss, revealing redundancy is structural rather than semantic. Results validated across Qwen3, Gemma 3, and Llama 3 families.
DisasterLex is a knowledge-graph-mediated text-to-SQL framework for querying geospatial disaster-analytics databases. It uses an Expert Knowledge Graph (107 concepts, 117 causal edges) to route natural-language queries across 36 heterogeneous tables. On 75 test queries, it outperforms 4 baselines (LightRAG, HippoRAG 2, ReFoRCE, CHESS) by 1.4x to 2.75x.
A researcher built Mamba-Titan-1.4B-Reasoning (2.54B params MoE) on RTX 3060 by freezing a 1.4B Mamba-1 backbone and adding 8 trainable experts. Trained on DeepSeek CoT traces, the model developed a 'vault door' mechanism: the </think> token isolates at the smallest norm (1.991 vs 4.742 mean) to control latent reasoning termination.
Systematic comparison of 13 abliterated Gemma 4 E2B variants across 44 GPU hours. coder3101 achieves 96% ASR (refusals) with full capability preservation and outperforms base model on math. Surgical approaches preserve performance better than aggressive methods, with some losing up to 6.9 points on GSM8K.
Parallax is a parameterized Local Linear Attention mechanism for LLMs derived from statistical regression. It replaces softmax's local constant estimate with a linear estimate, yielding better bias-variance tradeoffs. Pretrained at 0.6B and 1.7B scales, Parallax shows consistent perplexity improvements and matches or outperforms FlashAttention 2/3 in decoding.
vLLM merged a PR adding native HIP W4A16 kernel for ROCm. Benchmarks show significant gains: 270.2 tk/s in fp16 (max-num-seqs=8) and 445.7 tk/s (max-num-seqs=32), outperforming previous Triton implementations.
LLM agents (Claude and GPT) automatically annotate biological phenotypes by linking free-text descriptions to ontology terms. Tested on Dahrul et al. (2018) Gold Standard benchmark, all agents fall within inter-curator human variability, substantially outperforming the Semantic CharaParser NLP tool on all four metrics.
Comparative study of RL vs SFT on Qwen2.5-3B-Instruct: reinforcement learning better preserves internal circuits of the base model than supervised fine-tuning (SFT), which adapts faster but destroys more prior capabilities. Proposed metric: differential circuit vulnerability at attention head level.
Reasoning models maintain factually correct chain-of-thought traces but flip their final answer under sustained adversarial pressure in multi-turn dialogue. This unfaithful capitulation affects ~50% of cases in think mode and 11-15% without reasoning. The effect correlates with reasoning architecture (high in Qwen3-32B and GPT-OSS-20B, low in inline-CoT Gemma-4-31B-it).
Orthogonal Concept Erasure (OCE) proposes an editing method to remove undesired concepts from diffusion models using multiplicative orthogonal transformations. Unlike existing additive approaches, OCE preserves neuron magnitude and angular geometry while precisely erasing concepts. The approach erases up to 100 concepts in 4.3 seconds.
OpenClawBench is a dataset of 31,264 annotated trajectories to detect process-side anomalies in agent execution beyond task success. Among 31,135 passing executions, 2,904 contain anomalies (unresolved ambiguity, unsafe writes, ignored errors). A fine-tuned Gemma 3 12B detector reaches F1=0.729.
Conf-Gen adapts conformal prediction (CP) and conformal risk control (CRC) to generative models (LLMs, image generators, AI agents). The framework provides formal uncertainty guarantees for unsupervised tasks, extending conformal methodology to new domains.
Study of chain-of-thought (CoT) transfer across models using a provider-receiver framework. Full traces often transfer successfully, but mechanisms vary: answer extraction (AIME), receiver competence (MMLU-Pro), or partial structured information (ZebraLogic). In free-generation mode, partial CoTs improve performance, suggesting guidance for continued reasoning.
Cross-Model Entropy (CME) proposes a label-free reward signal for LLM post-training RL. CME uses mean log-likelihood of responses under an independent verifier model, avoiding self-consistency and reward hacking. Integrated into GRPO, CME achieves 52.5–71.4% tie-adjusted win rates on UltraFeedback/AlpacaEval 2.0 across Qwen, Llama, Gemma, OLMo.
Model merging formulated as convex quadratic programme over residual updates. Subsumes existing methods (task arithmetic, model soups, TIES, DARE) and provides closed-form diagnostic predicting merge quality via fraction of residual energy captured. Consistent gains across language and vision benchmarks.
Domain-specific NER model for identifying clinical entities in immunology and infectious disease contexts. 371 manually annotated case reports by clinical specialists. Transformer-based model with clinical embeddings achieves F1=0.89, outperforming BERT and zero-shot approaches. Supports case report analysis and clinical decision support.
PrismFlow introduces a new Flow Matching method for time-series generation using Koopman-inspired dynamical experts that learn residual corrections in latent space with a confidence-aware Winner-Take-All objective. Results: +15.6% Context-FID gain and +38.6% Discriminative Score improvement across benchmarks.
Investigation of Platonic Representation Hypothesis through tripartite framework: signal (universal linear object-attribute relationships), bias (architecture differences, mitigated by centering/normalization), noise (word frequency correlation with alignment). Sparse autoencoders show stronger cross-modal alignment than dense representations.
COLAGUARD, a guardrail model, transfers multi-step safety reasoning into continuous latent space via stage-wise training curriculum. Evaluated on 10 moderation tasks across 8 safety benchmarks, it improves macro-F1 by 8.24 points over Llama Guard 3, matches GuardReasoner performance while delivering 12.9X speedup and 22.4X token reduction.
GTA is a framework for automatically generating complex web tasks with executable trajectories. It combines crawling, retrieval, in-context generation, and quality control across 50+ websites (e-commerce, government, forums, news). The benchmark reveals a significant performance gap between humans and AI agents.