Embeddings are numerical vector representations of text, images, or audio that capture their semantic meaning. For example, OpenAI's text-embedding-3-small model converts sentences into vectors used for search or similarity tasks.
Developer seeks to build a free offline AI tutor grounded in a university textbook. Proposed architecture: RAG as core component (chunking, embedding, retrieval with page/section citations) + optional LoRA for pedagogical style. Questions on model selection (Qwen, Gemma), handling complex structures (figures, equations), and packaging for non-technical users.
Study of automatic assessment for massive multilingual bitext: decomposed into parallelism evaluation via multilingual embeddings and reference-free quality estimation. Benchmarks 4 embedding models and 9 evaluators on FLORES-200 covering 6,654 language-pair directions. Key finding: no single model is universally reliable; direction-aware routing and calibration required.
InfoNCE, the standard contrastive learning objective, relies on a statistical assumption misaligned with modern normalized embeddings. Authors propose WEINCE, a simple modification using extreme value theory and anchor-wise online batch statistics to correct softmax logits without additional parameters. Consistent improvements across 5 vision benchmarks.
Study combining cognitive-linguistic features (first-person pronouns, absolutist words, negative emotions) with DistilBERT embeddings and Holographic Reduced Representation to detect depression in Reddit posts. Hybrid model achieves F1=0.94 vs 0.80 TF-IDF baseline, AUC=0.981.
Comparative study of generic vs domain-specific embeddings for multilingual clinical search (ICD-10-CM). A bi-encoder fine-tuned on Gemini-generated synthetic data (6 languages) outperforms BioBERT-ST: R@5=0.822 vs 0.790, with major gains in Portuguese (+0.115). Open recipe for LLM-based medical retrievers.
Method to establish correspondences between embedding vectors from different black-box encoders. Exploits local geometric consistency of independently trained contrastive encoders: short-range distances preserved up to scale factor. Uses iterative reference-based geometric embedding hashing with paired anchors to recover vector links. Code released.
CanLegalRAGBench is an evaluation benchmark for RAG systems applied to Canadian law, based on realistic queries and expert-annotated answers. The study shows open-source embedding models are competitive with closed-source alternatives, but identifies hallucinations in 8-29% of generated answers unsupported by retrieved documents.
CoHyDE iteratively co-trains a dense encoder and LLM rewriter to improve tool retrieval over large API catalogs. On ToolBench (~10k tools), three rounds gain +2.5 pp NDCG@5 on standard queries and +6.3 pp on vague queries, outperforming single-component baselines.
Neuro-symbolic framework for ontology-grounded knowledge graph construction combining open-domain extraction, embedding-based canonicalization, and targeted LLM-based correction of ontology violations. Defers corrections to post-extraction stage to reduce token usage, improve KG consistency, and preserve QA quality for multi-hop reasoning and symbolic operations.
Method to create linear probes detecting concepts in LLM embeddings. Authors define a process: concept delineation via contrastive datasets, layer-wise probe training, tracking across large contexts. Tested on 4 concepts and 3 different LLMs. Goal: scalable monitoring of new models.
Investigation of Platonic Representation Hypothesis through tripartite framework: signal (universal linear object-attribute relationships), bias (architecture differences, mitigated by centering/normalization), noise (word frequency correlation with alignment). Sparse autoencoders show stronger cross-modal alignment than dense representations.
Comparative study of 7 transformer models (MiniLM to LLaMA-2, 22M to 13B parameters) for topic modeling via BERTopic. Finding: model size has negligible impact on topic quality measured by coherence and divergence metrics. Smaller models achieve comparable performance to larger ones.
Writ is an enforcement layer for AI coding agents using a local Neo4j knowledge graph and hybrid RAG. A 5-stage retrieval pipeline (BM25, HNSW vector similarity, graph traversal, reciprocal rank fusion) surfaces only relevant rules per task. 30 bash hook scripts enforce execution: no code without approved plan, mandatory tests, static analysis required.
User has been fine-tuning Jina-v5 on Slovak legal corpus for a month without success. Model fails to capture Slovak syntactic nuances, especially on ambiguous cases ("krádež" vs "prepadnutie"). Tested multiple approaches: LLM-generated queries, similar chunk injection, logit mining with Qwen 3.5-397B, but fine-tunes consistently underperform base model.
Tomesphere: Chrome extension + web layer enriching arxiv with LLM-curated TLDRs, OpenReview reviews, GitHub/HuggingFace links, citation graphs, SPECTER2 semantic neighbors. 3M papers indexed, free, no signup.
MONET, an Apache 2.0 dataset of 104.9M high-quality images with captions and metadata, released on Hugging Face. Built from 2.9B images and refined. Includes paper, UMAP visualization, text/image retrieval tool, and codebase for training T2I models.
BioELX is a two-stage cross-lingual biomedical entity linking system requiring no annotated training data. It enriches SapBERT with Wikidata-derived multilingual aliases and uses an LLM for context-aware disambiguation. On five benchmarks, it achieves +19.2 Recall@1 on XL-BEL, with major gains for low-resource languages (Turkish +21.6, Korean +22.1, Thai +30.8).
Systematic survey on Mixture-of-Experts (MoE) applied to multimodal learning. Analyzes MoE from three perspectives: efficient engine (scalability, redundancy reduction), representation learner (multi-expert alignment), modular adapter (modality imbalance, missing data). Identifies gaps: interpretable routing, expert communication, modality integration, lifelong learning.
MERIT, a multimodal pretraining framework, combines masked ECG modeling with ECG–text contrastive alignment to learn cardiac representations. On PTB-XL: +3% F1 (All) and +5% F1 (SubClass), +2.66% AUC zero-shot. Also improves clinical text generation with LLMs.
Meilisearch is a lightning-fast search engine API providing AI-powered hybrid search for websites and applications.
Study on cross-lingual retrieval asymmetry in 5 multilingual models (Gemini, Mistral, OpenAI, Qwen). Analysis of 6,518 idiomatic expressions in English, Bengali, Hindi, Arabic. Finding: hubness (vector concentration) is the dominant causal driver (49.5% dominance share), far exceeding anisotropy. CSLS correction closes 63.5% of reciprocity gap.
GEM (Geometric Entropy Mixing) reformulates LLM data curation as a variational problem on the hypersphere to prevent cluster collapse. Uses provable MM algorithm and teacher-student distillation for web-scale scaling. Improves downstream accuracy by up to 1.2% on 1.1B models integrated with DoReMi and RegMix.
FAST-GOAL enhances CLIP to handle lengthy text descriptions through global-local semantic alignment. The method combines efficient local region extraction (FLISM) and token similarity-based learning (TSL). A new GLIT100k dataset with global image-caption pairs and derived local pairs validates the approach on DOCCI, DCI, MSCOCO, Flickr30k.
Tomesphere indexes 3 million arxiv/OpenAlex papers with Gemini TLDRs, OpenReview peer reviews, GitHub repos, citation graph (250M edges), and SPECTER2 semantic graph (768D pgvector). Four ranking modes: Influential, Recent, Hidden gems, Nearest. Chrome extension for arxiv. Free, no signup.
ScholarScout v1.5.3 adds a Chrome Dino-style game to the pipeline wait screen (2-3 min). A pixel owl runs through a parallax forest; each spawned paper dot maps to a real SSE backend event (600ms intervals). Colors indicate source (arXiv white, PubMed green, Crossref purple). New features: k-means clustering on embeddings, per-cluster synthesis, paper freshness management with least-used prioritization.
GitRAG lets users ask questions about any public GitHub repo and get answers grounded in source code with exact file paths and line numbers. System combines AST-aware parsing, dense embeddings, BM25 index, RRF fusion, and Cohere reranking before generation via llama-3.3-70b on Groq. Supports 15+ languages.
Automated framework to detect lexical gaps (words absent in certain languages) using embeddings from multilingual LLMs. On Korean-English translation pairs, 4000 embedding spaces show gap words have weaker cross-lingual semantic alignment. Logistic classifiers achieve AUC 0.81–0.76 and retrieve 18/19 and 26/27 gap words.
Researchers train neural networks on WikiText-103 (103M tokens) using Successor Representations from RL to predict future word distributions. Without explicit linguistic supervision, grammatical categories (nouns, verbs, adjectives) spontaneously emerge and become separable via unsupervised clustering, organized by predictive horizon.
Unveil is a visual-textual embedding framework for multi-modal document retrieval. It integrates textual and visual features through knowledge distillation, transferring semantic capabilities from a visual-textual model to a purely visual model. Results: improved retrieval accuracy and efficiency without parsing.
BoxLitE is a knowledge base embedding model for DL-Lite^H using convex optimization. It maps concepts to convex regions in vector space to represent ontological hierarchies. For any satisfiable DL-Lite^H KB, BoxLitE produces a weakly faithful embedding.
Comparison of Structural Topic Models (STM) and BERTopic for analyzing short open-ended survey responses. BERTopic produces higher topic coherence, strengthened by contextual augmentation (strategy introduced to enrich very short responses). STM offers better support for inferential covariate analysis, BERTopic for interpretability.
Study showing task-relevant information is distributed non-monotonically across layers in foundational models. Introduces LOES (Layer-wise Optimal Embedding Selection), a spectral method identifying task-discriminative subspaces, and GeoReg, geometric regularization enforcing simplicial structure on class manifolds. Consistent improvements across architectures and modalities.
RADAR is a geometric metric for estimating cross-domain transferability in foundation models. It analyzes layer-wise representation evolution by measuring angular alignments and distance changes. Evaluated on cross-lingual sentiment classification and cross-domain image classification, RADAR achieves competitive performance against existing transferability metrics.
User computed embeddings for NVIDIA's Nemotron-Personas dataset (millions of synthetic personas) using Qwen 0.6B. Precomputed vectors enable semantic search and persona clustering. Precomputed embeddings and web demo available on Hugging Face.
Qdrant is a high-performance vector database designed for large-scale AI applications. Available as open-source and cloud service.
Empirical RAG study on 3 production websites (Intercom, HubSpot, KPMG) with tiered chunking and embeddings. Results: 31% HIGH/MEDIUM chunks for Intercom, 32% HubSpot, 8% KPMG. Tier weighting (HIGH ×1.20) reranks top-k. Proposed metric: 'yield score' predicts corpus quality before generation.
Developer asks whether building a custom image encoder is better than CLIP/SigLIP/DINO for video frame classification. Pipeline: 15 frames/30s → embeddings → Transformer 1.5-9M params. Constraints: speed (CLIP-S0: 10 img/s on 4 vCPUs) and CPU-only deployment. Considers custom encoder trained on proprietary dataset (millions of images, 4-5 labels).
Temporal Contrastive Transformer (TCT): self-supervised representation learning framework for financial fraud detection via transaction sequence embeddings. AUC 0.8644 standalone, 0.9245 combined with engineered features. Captures temporal structure but no additive gain over baseline.
Comparative study of RAG systems for Khmer. BGE-M3 outperforms Jina-Embeddings-v3 and Qwen3-Embedding in dense retrieval (Hit Rate@3: 0.285). Evaluation of 5 generators (Qwen3, Qwen3.5, Sailor2, SeaLLMs-v3, Llama-SEA-LION-v2) on 200 QA pairs using 6 RAGAS metrics. No single model dominates all criteria; retriever selection remains the bottleneck.
Comparative study of four chunking strategies (Recursive, Khmer-Aware, Sentence-Based, LLM-Based) for RAG on Khmer agricultural documents. Recursive chunking with 300 characters achieves best performance: L2 distance 0.4295, Answer Relevance 0.8663, Khmer IoU 0.6441. Statistically significant improvement over Sentence-Based (p=0.0121).