Topic

#Voice

In AI, voice refers to speech synthesis and recognition technologies that enable machines to speak or understand human speech. ElevenLabs, for instance, generates realistic synthetic voices from text.

40Articles

8Sources

65Avg. signal

GitHub Trending·16h ago

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> Open-LLM-VTuber /</span> Open-LLM-VTuber

Open-LLM-VTuber enables hands-free voice interaction with any LLM, featuring voice interruption and locally-running Live2D animated avatars across platforms.

Voice Open source Tools

SIG

HYP

GitHub Trending·16h ago

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> Open-LLM-VTuber /</span> Open-LLM-VTuber

Open-LLM-VTuber enables hands-free voice interaction with any LLM, featuring voice interruption and Live2D facial animation running locally across platforms.

Voice Open source Tools

SIG

HYP

arXiv cs.CL·1d ago

LaSR: Context-Aware Speech Recognition via Latent Reasoning

LaSR proposes a training paradigm for Speech LLMs featuring latent reasoning aligned to acoustic feature regions of target words. Without explicit intermediate tokens, the method improves specialized vocabulary recognition on Fun-Audio-Chat. A new Spoken Darwin-Science corpus for academic terminology is introduced.

Voice Reasoning RAG

SIG

HYP

arXiv cs.CL·1d ago

SALSA: Speech Aware LLM Adaptation via Learned Steering Activation Vectors

SALSA adapts speech-aware LLMs via layer-wise learned steering vectors optimized with supervised objectives. Tested on children's speech, multilingual, and Mandarin-English code-switching, it achieves up to 46.8% relative improvement over zero-shot. Steering the encoder's later layers outperforms steering the LLM backbone.

Voice Fine-tuning Reasoning

SIG

HYP

Reddit r/MachineLearning·1d ago

Full duplex vs half duplex - the spectrum of AI voice models [D]

Discussion on AI voice model architecture: half-duplex (strict turn-taking, current standard) vs full-duplex (simultaneous two-way channels, like human conversation). Half-duplex systems cannot handle overlaps, backchannels ("mhm", "yeah"), or barge-in interruptions, explaining their robotic feel.

Voice AI Agents

SIG

HYP

Reddit r/MachineLearning·2d ago

Real-time multilingual ASR using rolling buffers and monolingual models [P]

Real-time multilingual ASR system routing audio between specialized monolingual models (~100M parameters each) instead of one large model. Detects language switches via SpeechBrain and re-transcribes with correct model. Achieves 13% WER on inter-utterance code-switching, outperforming cloud APIs. Open-source repo released.

Voice Code generation Open source

SIG

HYP

Reddit r/LocalLLaMA·2d ago

A lightweight, real-time multilingual ASR router that runs on local hardware

Lightweight multilingual ASR routing system for local hardware using Zipformer, Silero VAD, and SpeechBrain. Routes audio between specialized monolingual models (~100M parameters) instead of one large model. Achieves 13% WER on inter-utterance code-switching, outperforming cloud APIs. Known limitation: 41% WER on intra-utterance switching. Open-source repo available.

Voice Open source Tools

SIG

HYP

arXiv cs.CL·2d ago

Your Multimodal Speech Model Says I Have a Face for Radio

Bias evaluation of multimodal speech recognition models (audio-visual). Researchers create videos pairing different faces with identical audio and measure transcription accuracy variations. Findings: quality-of-service gaps up to 4.05 word error rate points across gender, ethnicity, and intersections on Whisper-Flamingo and Gemini.

Vision Voice Benchmarks

SIG

HYP

Reddit r/LocalLLaMA·2d ago

I ported NVIDIA Parakeet (speech-to-text) to ggml: same output as NeMo, faster, GGUF-quantized, no Python

NVIDIA Parakeet speech-to-text ported to C++/ggml without Python or PyTorch. Byte-for-byte identical output to NeMo, up to 5x faster on GPU for larger models, 600x realtime on audio clips. Quantized GGUFs (f16, q8_0, q6_k, q5_k, q4_k), flat C API, integrated in LocalAI with OpenAI-compatible endpoint.

Voice Open source Tools

SIG

HYP

Reddit r/LocalLLaMA·4d ago

made a local voice AI for windows you can talk to in any language. open source, bring your own key

Shadow AI is an open-source (AGPL-3.0) local voice assistant for Windows. Natural multilingual conversations, local web search via SearXNG, persistent memory, optional Google integrations (Gmail, Calendar, Drive). Uses user's free Gemini API key, zero remote servers.

Voice Gemini Open source

SIG

HYP

Reddit r/LocalLLaMA·4d ago

this new Moss tts 1.5 is damn good with voice cloning

MOSS-TTS v1.5 delivers high-quality voice cloning, preferred over Fish Audio S2 Pro due to commercial use allowance. Long Cat DiT 3.5 noted as another strong model.

Voice Open source Tools

SIG

HYP

arXiv cs.CL·5d ago

Transcribing Children's Speech: ASR Performance and Obtaining Reliable Orthographic Transcriptions

Comparative study of 9 ASR models (Whisper, Parakeet, Wav2Vec2) on child speech in Dutch. Fine-tuned Whisper-medium achieves 5.54% WER on JASMIN and 70.37% on DART. An utterance-level selection method identifies 42% (JASMIN) and 18.1% (DART) of utterances as correctly pronounced with ≥98.3% precision, reducing manual verification needs.

Benchmarks Voice Evals

SIG

HYP

Reddit r/LocalLLaMA·6d ago

Reachy Mini goes fully local!

Hugging Face has developed a fully local experience for Reachy Mini, a conversational robot. A blog post details setup and customization for various use cases, including building voice agents without cloud dependency.

Voice AI Agents Open source

SIG

HYP

Le Big Data·6d ago

YouTube : Vous pouvez enfin dicter à l’IA ce que vous voulez regarder

YouTube is testing an AI-powered feature that lets users create a fully personalized video feed by dictating their preferences. The system generates recommendations based on user voice or text instructions.

Voice RAG

SIG

HYP

GitHub Trending·6d ago

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> OpenMOSS /</span> MOSS-TTS

MOSS-TTS is an open-source speech and sound generation model family from MOSI.AI and OpenMOSS. It covers stable long-form speech, multi-speaker dialogue, voice design, sound effects, and real-time streaming TTS.

Voice Open source Tools

SIG

HYP

GitHub Trending·6d ago

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> OpenMOSS /</span> MOSS-TTS

MOSS-TTS is an open-source speech and sound generation model family from MOSI.AI and OpenMOSS team. It covers stable long-form speech, multi-speaker dialogue, voice/character design, environmental sound effects, and real-time streaming TTS.

Open source Voice Tools

SIG

HYP

arXiv cs.CL·6d ago

Syllabic-Structure Decoder for Automatic Speech Recognition in Vietnamese

New ASR approach for Vietnamese using syllabic-structure phoneme-based decoding. Model captures phonological composition of syllables instead of orthographic units, reducing vocabulary size. Outperforms PhoWhisper and Wav2Vec2 on LSVSC and UIT-ViMD benchmarks.

Voice Benchmarks Papers

SIG

HYP

arXiv cs.CL·6d ago

Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models

Fine-grained style control technique for prompt-based TTS models. Inter-utterance interpolation using direction vectors in embedding space (99-100% gender conversion success, 36 Hz pitch variation). Intra-utterance transitions via KV-cache swapping and sliding-window attention masking (speaker similarity 0.81-0.91).

Voice Prompt engineering Papers

SIG

HYP

arXiv cs.CL·6d ago

Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models

Spoken Language Models (SLMs) for speech synthesis in low-resource languages face a trade-off: synthetic data improves phonetic accuracy but suppresses prosodic variability (Synthetic Erosion). Authors propose two self-alignment frameworks (DGSA and TDSC) to recover expressivity, outperforming ElevenLabs and Gemini Pro, enabling zero-shot voice cloning for Lao.

Voice Papers Reasoning

SIG

HYP

Reddit r/MachineLearning·May 27

noisekit - CLI for generating realistic degraded speech datasets for ASR benchmarking [P]

noisekit is an open-source CLI to generate annotated degraded speech datasets for realistic STT benchmarking (telecom G.711, ambient noise, reverb). Solves the gap: public datasets (FLEURS, CommonVoice) are too clean to evaluate production performance. HuggingFace AudioFolder compatible, includes PESQ/SNR/NISQA metrics.

Voice Evals Benchmarks

SIG

HYP

GitHub Trending·May 27

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> cjpais /</span> Handy

Handy is a free, open-source speech-to-text application that operates entirely offline without cloud dependencies.

Voice Open source Tools

SIG

HYP

arXiv cs.AI·May 27

A Dataset of Robot-Patient and Doctor-Patient Medical Dialogues for Spoken Language Processing Tasks

MeDial-Speech: dataset of 111+ hours of spoken medical dialogues (robot-patient and doctor-patient) covering 4 health conditions. Benchmark of 3 LLMs (GPT-4 mini, DeepSeek-V3, Claude Sonnet 4) via sentence selection: Claude Sonnet 4 achieves 71.1% accuracy. Reveals systematic overconfidence in model predictions.

Benchmarks Claude DeepSeek

SIG

HYP

Hugging Face Blog·May 27

Reachy Mini goes fully local

Reachy Mini, Pollen Robotics' humanoid robot, now runs fully locally without cloud dependency. Integrates open-source models (Llama, Whisper) for vision, speech, and motor control. Deployed on embedded hardware.

Robotics Open source Llama

SIG

HYP

Hacker News (AI)·May 26

Bay Area mom out thousands after scammers use AI to mimic daughter's voice

A Bay Area mother lost thousands of dollars after scammers used AI to mimic her daughter's voice and request emergency money. The incident highlights growing risks of voice deepfakes in targeted fraud schemes.

AI safety Voice

SIG

HYP

Le Big Data·May 26

Spotify ajoute des articles narrés à son app, vos lectures passent maintenant par l’audio

Spotify adds narrated articles to its app (May 26, 2026). The platform converts text content to audio to expand beyond music and podcasts.

Voice

SIG

HYP

Reddit r/LocalLLaMA·May 26

OpenMOSS-Team/MOSS-TTS-v1.5 · Hugging Face

MOSS-TTS-v1.5 improves multilingual speech synthesis (31 languages), zero-shot voice cloning, and stability. New features: explicit pause control, better long-reference short-text cloning, more stable punctuation-driven prosody. Open-source model on Hugging Face.

Voice Open source Code generation

SIG

HYP

GitHub Trending·May 26

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> modelscope /</span> FunASR

FunASR is an industrial-grade speech recognition toolkit supporting 170x realtime, 50+ languages, speaker diarization, emotion detection, streaming, and OpenAI-compatible API.

Voice Open source Tools

SIG

HYP

GitHub Trending·May 26

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> dograh-hq /</span> dograh

Dograh is an open-source self-hosted voice AI platform, alternative to Vapi and Retell. Supports Speech-to-Speech, LLM/STT/TTS, visual workflow builder, native MCP and telephony.

Voice Open source MCP

SIG

HYP

Reddit r/LocalLLaMA·May 26

I finally put my NPU (Intel Arrow Lake) to use doing ASR for my smart home

User optimized ASR (automatic speech recognition) on Intel Arrow Lake NPU via OpenVINO. Results: 4.8× faster and 10.7× less energy than CPU INT8 on 10s audio. NPU (13 TOPS) frees CPU and VRAM for other ML tasks, outperforming RTX 3060 eGPU in latency.

Code generation Voice Infrastructure

SIG

HYP

arXiv cs.CL·May 26

Raon-Speech Technical Report

Raon-Speech is a 9B multilingual speech language model (English/Korean) that understands and generates speech while preserving text capabilities. Trained on 1.38M hours of curated data, it outperforms 8 comparable audio models (Qwen2.5-Omni, Fun-Audio-Chat) across 42 benchmarks. Raon-SpeechChat extends it with real-time full-duplex conversation trained on 119K hours of dialogue.

Voice Benchmarks Open source

SIG

HYP

arXiv cs.CL·May 26

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

Researchers apply Direct Preference Optimization (DPO) to improve English-Mandarin code-switching transcription in Audio LLMs. Three failure modes identified: language omission, translation-instead-of-transcription, hallucination. Training on 100K pairs (570 hours) reduces MER up to 89.6% (in-distribution) and 20.0% (out-of-distribution).

Reinforcement learning Alignment Voice

SIG

HYP

GitHub Trending·May 25

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> moeru-ai /</span> airi

Airi is a self-hosted, open-source AI companion capable of real-time voice chat, Minecraft and Factorio gameplay. Supports Web, macOS, and Windows. Inspired by Neuro-sama.

AI Agents Voice Open source

SIG

HYP

GitHub Trending·May 25

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> Zackriya-Solutions /</span> meetily

Meetily is an open-source, self-hosted meeting assistant built on Rust. 4x faster transcription than Whisper/Parakeet, speaker diarization, Ollama-based summarization. 100% local processing, no cloud required.

Open source Voice Tools

SIG

HYP

GitHub Trending·May 25

<svg aria-hidden="true" data-component="Octicon" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted"> <path d="M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z"></path> </svg> <span data-view-component="true" class="text-normal"> moeru-ai /</span> airi

Airi is a self-hosted AI companion supporting real-time voice chat, Minecraft and Factorio gameplay. Web, macOS and Windows support. Open-source project inspired by Grok and Neuro-sama.

Open source Voice AI Agents

SIG

HYP

Reddit r/LocalLLaMA·May 25

I shipped a windows desktop app for running local LLMs with a button that turns your "no thats wrong" into actual LoRA training data

SEELS, a Windows desktop app for local LLMs, lets users correct model replies via a « Teach » button that accumulates corrections into a JSONL corpus, then triggers PEFT LoRA fine-tuning without terminal access. Includes local STT/TTS (Whisper/Piper), hardware dashboard, 0.6B model pre-trained on 110 examples. Free stable version; pro tier (image/video gen, MCP) and max tier (workflows, multi-GPU) in roadmap.

Fine-tuning Open source Tools

SIG

HYP

Reddit r/LocalLLaMA·May 23

meituan-longcat/LongCat-Video-Avatar-1.5 · Hugging Face

Meituan releases LongCat-Video-Avatar 1.5, an open-source framework for audio-driven human avatar video generation. Upgrades audio encoder from Wav2Vec2 to Whisper-Large, supports Audio-Text-to-Video and Video Continuation with 8-step inference. Human evaluation on 508 image-audio pairs across 6 scenarios and 2 languages.

Video generation Open source Benchmarks

SIG

HYP

Hacker News (AI)·May 22

NTSB pulls docket after AI recreates dead pilots' voices

NTSB withdraws docket after AI recreates voices of deceased pilots. The incident raises ethical concerns about voice synthesis and the use of sensitive data in aviation accident investigations.

Voice AI safety Regulation

SIG

HYP

Reddit r/LocalLLaMA·May 22

I fine-tuned Cohere Transcribe to support diarization and timestamps

Developer fine-tuned Cohere Transcribe to add diarization (speaker identification) and timestamps. Model outputs parsable format with average temporal precision of ±0.097s. Supports up to 4 speakers per 30s, extensible to 32 with diarize_long.py script. Available free on Hugging Face.

Open source Fine-tuning Voice

SIG

HYP

Reddit r/LocalLLaMA·May 22

Open source: cloned Rocky's voice from Project Hail Mary in two days, full pipeline + 2:10 of training audio + trained RVC v2 model

Rocky's voice (Project Hail Mary) cloned in two days via open-source pipeline. Audio extraction (ffmpeg + demucs), transcription (Whisper), diarization (pyannote), then RVC v2 training on 2:10 min audio. Trained .pth model (55MB) and code public. Tested XTTS v2 / YourTTS / RVC v2 / OpenVoice v2.

Voice Open source Code generation

SIG

HYP

arXiv cs.CL·May 22

Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?

Study of factual recall mechanisms in multimodal language models (text + speech). Using causal mediation analysis on SpiritLM, researchers show that knowledge storage and retrieval mechanisms only partially transfer from text to speech modality.

Papers Reasoning Voice

SIG

HYP

Voice — AI news · Signal IA