Local-first week: voice, heterodox GPU builds, and TTS — edge inference keeps maturing

The dominant signal today is the acceleration of the full local stack, no remote server required. Shadow AI (AGPL-3.0) assembles in a single Windows project what most local demos leave as disconnected pieces: multilingual ASR, persistent memory, web search via SearXNG, optional Google integrations — all driven by the user's own Gemini key. This isn't a proof-of-concept: it's a usable product surface, and the choice of Gemini as backend suggests that high-quota free keys (Gemini 2.0 Flash, 1,500 req/day) are now the real adoption lever for local AI. Meanwhile, MOSS-TTS v1.5 (OpenMOSS-Team) is being benchmarked as superior to Fish Audio S2 Pro on voice cloning with a commercial license — if that holds up on listening tests, it's a direct drop-in replacement for proprietary TTS pipelines.

On the infrastructure side, the Blackwell/R730 project looks anecdotal on the surface but is instructive in practice: running an RTX Pro 6000 (96 GB VRAM, Blackwell architecture) in a 2016 Dell PowerEdge R730 via PCIe and firmware workarounds enables 650k token context on fully depreciated hardware. The opportunity cost of a used R730 is incomparable to a new HGX server. This kind of low-cost memory-density hacking will multiply as long-context models become the operational norm.

VT Code (Rust, open-source) and the CPU-cache spiking neuron library remain weak signals: the former is yet another terminal coding agent, but the Rust implementation signals serious attention to latency and portability; the latter, benchmarked against PyTorch on Wikipedia and developed with Gemini Flash 3.5, illustrates how LLMs are now being used to write specialized low-level code — a use case still sparsely documented but growing.