World Wire

Buoyant, the company behind Linkerd, has released Kinference , an open source project for building inference platforms on Kubernetes. Kinference is an inference control plane for Kubernetes and vLLM. It sits in front of a fleet of vLLM workers and decides where each request goes. It applies routing policies, token rate limits, KV-cache-aware load balancing and per-tenant metrics in one layer. For dev teams serving many open-weight models to many internal clients, it targets the layer above the model server. The project is Apache 2.0 licensed on GitHub . The current release is v0.1.0-alpha.7 . TL;DR Size: v0.1.0-alpha.7, written in Rust, Apache 2.0, configured through 5 Kubernetes CRDs. Runs on: any GPU-enabled Kubernetes cluster running vLLM. Buoyant’s benchmark used 3 L40S GPUs per pool. Performance: large gains on prefix-heavy traffic versus plain Kubernetes Service routing (self-reported). Best: median TTFT fell from 6.88 s to 0.31 s (95%) on a shared...

Zyphra has released ZYFA (Zyphra FlashAttention on AMD) , a set of FP8 and BF16 prefill attention kernels for AMD Instinct MI355X GPUs. ZYFA is Zyphra’s FP8 attention engine for long-context prefill, and the code is open source under Apache 2.0 . On real model activations, its FP8 kernels run 1.16-1.76x faster than AITER , AMD’s default kernel library. At head dimension 128, ZYFA’s FP8 error is 62-78% lower than AITER’s. TL;DR Framework: It is a kernel library. Causal forward (prefill) attention in FP8 e4m3 and BF16, head dim 128 and MLA shape (192/128). Runs on: AMD Instinct MI355X only (CDNA4, gfx950), ROCm 7.2.3, PyTorch 2.12. Drop-in flash_attn_func interface. Performance: 1.16-1.76x AITER’s FP8 throughput on real activations; BF16 roughly ties (1.01-1.02x). Best: 2,672 TFLOP/s FP8 at 32K tokens, 53% of the FP8 datasheet peak (AITER: 39%). Worst: Only 1.16x on Mistral-7B activations; FP8 clock drops to 1,628 MHz vs AITER...

Thinking Machines Lab has released Inkling 2 Small , the first model of its second Inkling generation. The Inkling 2 Small model is a sparse Mixture-of-Experts (MoE) with 594B total and 17B active parameters. It accepts text and images, and supports variable reasoning effort. On the Artificial Analysis Intelligence Index , it scores 31. Its predecessor, Inkling Small , scores 13 on the same index. The model is live on Tinker for fine-tuning and inference. TL;DR Size: 594B total, 17B active MoE; 1M-token context. Runs on: Tinker API only for now. Self-hosting hardware is not disclosed. Weights support MXFP8 and NVFP4. Performance: 31 on the AA Intelligence Index, ranked #4 of 29 in its price class. Best: 86.3% on SWE-Bench Pro v2 (Hard), second only to Kimi K3 in the lab’s table. Worst: 13.1% on Terminal-Bench 4.0, versus 32.8% for GLM-5.3-Flash. What is Inkling 2 Small? Inkling 2 Small is a multimodal MoE model from Thinking Machines Lab fo...

Google has released Gemini Omni 1.2 Flash , the newest version of its video generation and editing model. Gemini Omni 1.2 Flash generates and edits clips up to 15 seconds natively. It also cuts the price by 30% compared with Omni 1.1 Flash. The model is generally available on Google Cloud and the Gemini API . For developers, the takeaways are longer clips, fewer re-renders and a lower bill per second. TL;DR Runs on: Google-hosted only (Gemini API, Google Cloud). No open weights, no local deployment. Performance: Google reports better reference adherence and less drift. No 1.2 benchmark scores are published. Best: Native 15-second clips at $0.07 per second of 720p video, about $1.05 per 15-second clip. Worst: Google says fast sports action, rapid script changes and complex physics still need work. What is Gemini Omni 1.2 Flash? Gemini Omni 1.2 Flash is a multimodal video model from Google DeepMind. It takes text, images, short video clips and audio as...