AI infrastructure refers to the hardware and software layers that enable training and deploying machine learning models at scale. This includes GPU clusters such as AWS EC2 P5 instances or serving frameworks like Ray Serve.
RelGT-AC extends RelGT for autocomplete tasks on relational databases. The model combines column masking, a unified task head (classification/regression), and a TF-IDF encoder for free text. On 7 RelBench v2 tasks, it outperforms GraphSAGE on 3 regression tasks and gains +10 AUROC on text-heavy eligibility tasks.
Paper proposes modular reference architecture for Embedded Agent Systems on microcontrollers. Decouples On-Device Agents (compressed neural networks, rule-based logic) from Cloud-Augmented Agents (SLMs for reasoning). Includes cross-cutting Governance Layer for observability and safety across distributed device fleets.
AURA-Mem introduces a constant-size recurrent memory (4,224 bytes) for robot policies, with a learned gate that writes only when observations would change the next action. On LIBERO-Long with OpenVLA-OFT 7B, it matches baseline policy (0.233 success) while reducing memory writes by 7× and VRAM by 6,061× versus KV-cache.
MiniMax introduces a new attention architecture (MSA) natively supporting 1M tokens without quadratic complexity. 'KV outer gather Q' approach delivers 4× faster than Flash-Sparse-Attention, compute reduced to 1/20th, 9× prefilling speedup, 15× decoding speedup. First open-weight model combining frontier coding, 1M context, and native multimodality.
A user installed a V100 datacenter GPU in their gaming PC for £200 and documented their experience running local models. The blog post generated significant interest beyond Reddit.
Microsoft launches Project Solara, a hardware platform dedicated to AI applications. The project aims to provide optimized infrastructure for running AI models in production.
StepFun 3.5 MTP integrated into llama.cpp before Gemma MTP. Pull request #23274 adds support for StepFun model with MTP (Multi-Token Prediction) architecture.
With no clear regulatory framework for AI, Americans are opposing data center construction as a tangible target in the AI debate. This strategy sidesteps the political difficulty of directly regulating AI.
The Linux Foundation unveils DNS-AID (Agent IA Discovery), an open-source project using DNS as a discovery layer for AI agents. This decentralized approach challenges proprietary registries from hyperscalers (OpenAI, Google, Anthropic).
Wasmer is a WebAssembly runtime for running fast, secure, lightweight containers. The project gains traction on GitHub Trending, positioning WebAssembly as an alternative to traditional containerization.
mistral.rs is an optimized LLM inference framework prioritizing speed and flexibility. Open-source project enabling efficient execution of language models.
OpenShell is a secure, private runtime for autonomous AI agents developed by NVIDIA. The project is available on GitHub and aims to provide controlled execution infrastructure for multi-agent systems.
Headroom compresses tool outputs, logs, files, and RAG chunks before sending to LLM. Reduces token consumption by 60-95% without degrading answers. Available as library, proxy, and MCP server.
Alphabet plans to raise up to $80 billion to strengthen its AI infrastructure amid global competition.
Comprehensive benchmark of 8 tiny LLMs (135M–1B) on Jetson Orin Nano Super 8GB with llama.cpp CUDA across 4 power modes (7W–MAXN). 25W mode optimal: SmolLM2-135M achieves 165 tok/s and 22.6 tok/J; LFM2.5-1.2B best in ~1B class (54.1 tok/s). 384 benchmark cells, raw datasets published.
Snowflake and Anthropic strengthen their partnership to deploy Claude models directly within enterprise environments with integrated governance. The integration aims to accelerate governed AI adoption in enterprises.
STMicroelectronics raises its data center revenue guidance to $1 billion for 2026, despite an 89% drop in net income. The Franco-Italian chipmaker adjusts forecasts upward for the data center segment.
Warren Buffett's Berkshire Hathaway invests $10 billion in Alphabet's AI infrastructure buildout. Alphabet raises $80 billion to scale AI capacity, with capital expenditures expected to reach $190 billion in 2026.
Qwen 3.6-35B-A3B achieves 977 tokens/s prompt processing and 262k context window on Intel Arc B70 Pro via llama.cpp with SYCL backend. User reports stable, usable local inference for complex tasks including game generation.
OpenAI makes GPT-5.5, GPT-5.4, and Codex available through Amazon Bedrock at identical pricing to OpenAI's platform. Models run in commercial and government AWS regions, currently limited to the US. Usage counts toward existing AWS contracts.
A user built a large-scale scraping pipeline aggregating 2M+ active job postings from 100,000+ company career sites. Dataset in Parquet format, daily-refreshed, freely accessible with standard fields (title, company, description, location, URL).
llama.cpp benchmarks on Intel Arc Pro B70 posted. Performance measured: 63 tokens/sec with Qwen via SYCL. Test of Intel discrete GPU for local inference.
Vercel now enables configuring Git settings for all projects in a monorepo from a single location, eliminating the need to configure each project individually. Settings include commit status and repository_dispatch events.
ART (Attention Run-time Termination) is a lightweight runtime mechanism that halts KV block accesses during decoding once their attention contribution becomes negligible. Tested on LongBench, it achieves 20% higher generation throughput on large batches while maintaining comparable accuracy.
Survey paper proposing Intelligent Computing Architecture Model (ICAM), a six-layer framework for model-native computing. Maps classical computer architecture concepts to LLM systems (cache management, context, agents). Introduces three design laws: Semantic Locality Law, Context Budget Law, Agent Speedup Law. Distinguishes probabilistic execution plane from deterministic control plane.
arXiv paper on LLM inference batching optimization. Authors demonstrate mixed batching (MB) is suboptimal on bandwidth-constrained GPUs: exclusive batching (EB) achieves 41.9% higher throughput on RTX PRO 6000 (1.792 TB/s). They propose EB+, a hybrid scheduler that dynamically switches between EB and MB based on GPU bandwidth, model size, and workload composition, reaching 36.4% gains under non-stationary traffic.
BudgetDraft trains a sparse drafter for speculative decoding in long-context inference (4K-16K tokens). The method exposes the model to multiple KV budgets during training and aligns each sparse view with a shared full-cache teacher target. Results: 6.55x, 4.46x, 2.10x speedup vs autoregressive decoding at 4K, 8K, 16K tokens.
Release of micropython-wasm 0.1a0: an experimental package bundling a WASM build of MicroPython with a wasmtime wrapper to execute Python code in a sandboxed environment.
NVIDIA announces Cosmos 3 (video model), Nemotron 3 Ultra (optimized LLM), and RTX Spark. Jensen Huang claims a major win for the company.
Vercel Blob now offers time-bound signed URLs to upload, download, inspect, or delete objects without full store access. Each URL is scoped to a single operation, pathname, and expiry up to 7 days. Multipart uploads let browsers stream directly to Blob storage without server round-trips.
Linux ROCm now supports WSL2 with stable functionality, though bugs remain. Build instructions included for users.
A researcher who spent months inside verl (ByteDance's RL post-training framework) documents its internals: RLHF loop orchestration, single-controller pattern, data structures (DataProto), and a NCCL bug discovered. Abandoned fork but knowledge shared with the community.
OpenAI makes frontier models and Codex available on AWS. Users can now deploy GPT-4, GPT-4 Turbo, and Codex directly on AWS infrastructure without routing through OpenAI's API.
Correction: The 600 GB/s claimed for RTX Spark refers to NVLink bandwidth, not GPU bandwidth. Media outlets confused the two specs based on Computex slides.
Alphabet announces an $80 billion equity capital raise to expand AI infrastructure and compute capacity. This massive injection aims to support AI services growth and strengthen the group's competitive position against other tech giants.
llama.cpp b9455 merges a major fix for KV cache quantization in tensor mode on multi-GPU. The solution extends the meta backend to properly handle tensor flattening without losing shape information, avoiding changes to compute graphs.
Ohio pauses datacenter tax breaks that have been draining state coffers. The decision follows years of substantial subsidies to AI infrastructure with insufficient fiscal returns.
Intel launches Crescent Island at Computex 2026, a professional GPU based on Arc Xe 3P architecture with up to 480GB VRAM using LPDDR5X. 350W TDP, supports FP4/MXFP4 to FP64 formats for next-generation AI workloads.
Vercel rolls out automatic memory monitoring on elastic build machines. The system dynamically adjusts resources to prevent OOM failures: automatic upgrade if memory approaches threshold, no downgrade for fast but memory-intensive builds.
NVIDIA RTX Spark will feature 600GB/s memory bandwidth, double that of DGX Spark (273GB/s). Unified memory architecture with 128GB LPDDR5X.