Computer vision is the field of AI that enables machines to analyze and interpret images or videos. GPT-4o, for instance, can describe the content of a photo, read printed text, or identify objects within a scene.
ToolGate is a lightweight controller that decides whether to execute or skip tool calls proposed by vision-language agents. Across five benchmarks with Qwen3-VL, it reduces token cost to 64-69% of the ReAct baseline while preserving accuracy, and further improves accuracy by 1.65 points with matched-domain training.
MiniMax introduces a new attention architecture (MSA) natively supporting 1M tokens without quadratic complexity. 'KV outer gather Q' approach delivers 4× faster than Flash-Sparse-Attention, compute reduced to 1/20th, 9× prefilling speedup, 15× decoding speedup. First open-weight model combining frontier coding, 1M context, and native multimodality.
Gemma 4 E4B in Google's LiteRT format achieves 157.2 tok/s text generation, 2.4× faster than Q4 GGUF (66.3 tok/s) via multi-token prediction (MTP). Image captioning shows only 1.1× speedup as vision encoder is the bottleneck. Tested on RTX 4060 Ti 16GB.
Starbucks withdraws NomadGo, its computer vision tool deployed across 11,000 stores after nine months. The system, validated at 99% in controlled environment, failed at production scale, highlighting the gap between sandbox testing and real-world deployment.
Nvidia unveils Cosmos 3, an AI model presented at Taipei GTC alongside humanoid robot Isaac GROOT. The model aims to enhance understanding of real-world scenes.
DraDDP is the first public multimodal dataset for discourse parsing in multi-party dialogues. Built from American TV dramas, it contains 495 dialogue segments (6,374 utterances, 9.1 hours of video). Benchmarks demonstrate the value of multimodal information for identifying dependency structures and relation types between utterances.
TIGER is an inference-time framework to mitigate hallucinations in multimodal generation. It independently extracts an observation graph from input and a claim graph from output, then assigns risk scores to claims based on support and conflict. The model repairs high-risk claims while keeping the backbone frozen. Convergence analysis shows geometric risk reduction to an explicit asymptotic bound.
Hoeffding Concept Bottleneck Models (HCBM) replace linear concept aggregation with non-linear sparse decomposition based on gradient-boosted trees. Robust to inter-concept leakage, they improve explainability of computer vision predictions, particularly on overhead imagery.
Demo2Reward optimizes VLM reward model language instructions at test-time using 3-10 expert demonstrations to reduce false positives in robotics. No additional training required. Validated on simulated tasks and real-world transfer.
CTRL-STEER introduces a closed-loop control framework for Vision-Language-Action (VLA) models. Instead of fixed steering coefficients, it adaptively adjusts intervention strength over time using PID or reinforcement learning controllers. Experiments on OpenVLA with LIBERO task suites demonstrate improved concept regulation stability and better steering-task success trade-offs without retraining the base model.
VESTA is a statistical agent framework equipping VLMs with a dynamically growing exploration toolkit for quantitative model refinement. Evaluated on DAWN (benchmark for distribution fitting and time series modeling), VESTA outperforms prior agentic systems, especially on complex astronomy tasks (initial mass functions, gravitational-wave chirp signals).
New set-distance reward method for reinforcement learning of vision-language models on chest X-ray report generation. Tested on Qwen3-VL, Gemma3 with GRPO: 6.80% (BERTScore), 7.82% (RadGraph F1), 4.45% (CheXbert F1) improvements over supervised fine-tuning. Enables test-time best-of-N selection and mid-generation pruning reducing tokens by 50%.
Novel approach for Major Depressive Disorder detection from EEG without data augmentation. SGC (Score-Guided Classification) uses an unsupervised generative network to model pathological anomalies as prior, fused with deep feature representations. Cross-Channel Spatial Adaptation module handles multi-center channel heterogeneity. Validated on Mumtaz2016 and MODMA datasets.
UniKE, the first benchmark for cross-modality knowledge editing in unified multimodal models (UMMs), reveals a critical gap: text-side efficacy reaches 92% but VQA accuracy in image generation drops to 18.5%. A reasoning-augmented parameter editing method improves results by up to +18.6 percentage points.
Developer trains a local model to detect mosquitoes and eliminate them with a laser. Applied robotics project combining vision and automated control.
Qwen3.7-Plus introduces multimodal capabilities for AI agents. The model processes text, image, and video in a unified architecture designed for autonomous agent tasks.
Google rolls out an update to Google Drive's scanner on Android with three new AI-powered features for document scanning. Older Android devices may lack compatibility with these new capabilities.
MiniMax releases M3, an open-weight model featuring a one-million-token context window, top-tier coding performance, and native multimodality. First open-source model combining these three capabilities.
Qwen 3.7 Plus from Alibaba is now available on Vercel AI Gateway. This multimodal model unifies vision and language for AI agents, covering GUI/CLI, coding, and visual tasks. Access via SDK with transparent pricing and zero platform fees.
NVIDIA releases Cosmos 3, an open omni-model for physical AI that reasons and acts. The model processes video, text, and images to understand real-world physics and generate robotic actions.
CoSee, an auditing framework, analyzes failure modes of modular visual reasoning systems using shared working memory. On 4B–8B models, two dominant failure modes emerge: Noise Reinforcement (reusing ungrounded notes) and Policy Collapse (under-specified answers). The study shows naive shared workspaces amplify hallucinations without explicit verification.
FAM-Bench is a 2500-instance multimodal benchmark verified by nutrition experts, evaluating Food-as-Medicine reasoning across 13 health conditions. Two tasks: assess dish suitability for a condition (image + ingredients) and rank 4 dishes by clinical relevance. Tests integration of nutritional constraints, visual cues, and ingredient evidence.
Bias evaluation of multimodal speech recognition models (audio-visual). Researchers create videos pairing different faces with identical audio and measure transcription accuracy variations. Findings: quality-of-service gaps up to 4.05 word error rate points across gender, ethnicity, and intersections on Whisper-Flamingo and Gemini.
DSFM (Dual-Spectral Flow Matching) generates synthetic fMRI time series by combining discrete wavelet transform (DWT) and discrete cosine transform (DCT) with spectral flow matching. The model captures non-stationarity and spatiotemporal dynamics of BOLD signals to improve brain network classification.
PhyDrawGen is a neuro-symbolic pipeline generating physics diagrams from text while respecting physical laws. An LLM extracts a typed scene graph, a deterministic solver converts it to a planar straight-line graph, and Qwen-VL fine-tunes a propose-verify loop. Evaluated on 1,449 problems (mechanics, optics, electromagnetism), it outperforms GPT-5-image and Gemini.
SCALE is a self-improving framework for web agents using MLLMs. It employs three adversarial roles (Selector, Predictor, Judger) to autonomously explore agent limitations and expand cognitive boundaries. SCALE-Hop optimizes global planning via graph exploration. A SCALE-20k dataset from 19 real websites with 20k structured demonstrations validates the approach across multiple MLLMs.
TRINE is an FPGA accelerator and compiler for end-to-end multimodal inference (ViT, CNN, GNN, transformers) without reconfiguration. It unifies layers as matrix operations, switches between systolic and SIMD architectures at runtime, and applies in-stream token pruning. On Alveo U50 and ZCU104, it achieves 22.57x latency reduction vs RTX 4090 while consuming 20-21 W.
Method to align speech and co-speech gestures using semantic motion anchors: discretizes 3D gestures into motion primitives, verbalizes them into structured descriptions, and provides contrastive supervision. 8.2% R@1 improvement on BEAT2; retrieved gestures are semantically meaningful rather than generic motion patterns.
TeachObs is a human-validated multimodal benchmark for classroom video analysis. It contains 30 public lessons from 8 countries split into 5,158 15-second scenes, annotated by 7 researchers with 39 observation codes (20 visual, 19 non-visual). Evaluation of 5 vision-capable LLMs across 3 tasks: no single model consistently outperforms others.
MiniMax M3 announced with 1M token context window, advanced coding and agentic capabilities, multimodal support. Positioned at the frontier of high-performance open-source models.
Stepfun 3.7 Flash delivers quality close to GLM 5.1 with 80% 3D world understanding while using 75% fewer parameters and featuring built-in vision. Recommended for RAM-constrained setups.
MiniMax M3, MiniMax's first model with 1M-token context window and native multimodality, is now available on Vercel AI Gateway. M3 excels at software engineering, terminal-based tool use, and agentic web browsing, optimized for multi-turn collaboration.
One million ancient Greek text fragments will be translated using AI. The project leverages vision and language models to decipher damaged manuscripts and generate automated translations.
Llama.cpp B9406 fixes GGML_ASSERT crash in get_rows/mtmd_helper_decode_image_chunk when using MTP + MoE model + vision with Qwen 3.6-35B-A3B.
PaddleOCR is a lightweight, multilingual OCR toolkit (100+ languages) designed to convert PDF and image documents into structured data for LLM consumption.
VFEAgent is a multimodal multi-agent system automating Finite Element Analysis (FEA) from images and text descriptions. The framework combines a vision-language pipeline with ReAct reasoning and verification-first code synthesis to generate physically valid simulations, outperforming existing LLM-based approaches.
New BMLR method balances multimodal learning by reshaping label space to equalize mapping difficulty across modalities. Addresses modality imbalance where faster-converging modalities dominate optimization. Novel label-side approach rather than gradient adjustment.
Paper introduces Program-of-Thoughts prompting for chart summarization: VLMs generate Python programs to derive valid summary statistics instead of direct text. Proposes chart-to-dictionary auxiliary task. Results match existing methods on semantic and factual metrics.
Study of persona effects on explanations generated by multimodal LLM agents in urban perception. Analysis of 59,808 annotations from 1,200 persona-conditioned agents: captions show strong convergence, justifications display systematic variation tied to socioeconomic and political attributes, perception tags show no significant persona-related differences.
Defect grading framework for power transmission equipment using MLLM. In-context learning on commercial models, chain-of-thought Q&A generation to reduce manual annotation, then fine-tuning Qwen3-VL-8B via LoRA. SOTA on three grading tasks.