LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning

In three linesLatentOmni proposes an audio-visual reasoning framework using unified latent space instead of explicit text chain-of-thought. The model interleaves textual reasoning with audio-visual latent states, introduces Omni-Sync Position Embedding (OSPE) for temporal consistency, and leverages LatentOmni-Instruct-35K (35K annotated trajectories). Outperforms text-based baselines on audio-visual benchmarks.

Summary generated by Claude — human-verified