Posts

Lowest-Latency Inference APIs for Voice and Realtime Agents: A Time to First Token TTFT-First Benchmark

Time to first token (TTFT) is the metric teams use to pick an inference API for voice. It is also the metric that misleads them. TTFT marks when generation starts; a text-to-speech model cannot speak until a full clause arrives. Between those two points sits the difference between an agent that feels conversational and one that gets interrupted. This piece benchmarks every layer of the voice stack including LLM, speech-to-text, text-to-speech, and speech-to-speech. Why TTFT Is the Right Entry Point and the Wrong Finish Line A voice agent is a latency budget with a language model inside it. Every stage spends milliseconds the user can hear. Time to first token (TTFT) is the interval between sending an inference request and receiving the first token back. IBM’s definition frames it as the moment a system transitions from idle to visibly active. For chat, TTFT is close to the whole story. For voice, it is one term in a sum. The reason is mechanical. A text-to-sp...

Google AI Introduces EnvHarness: A Programmable Layer That Turns Static Agent Environments Into Adaptive Training Worlds

Image
A team of researchers from Google Cloud AI Research, Washington University in St. Louis and UNC Chapel Hill has released EnvHarness , a programmable layer that turns a static agent benchmark into one that adapts to the policy training on it. LLM agents now learn less from curated text and more from interactive environments, but those environments are hand-built and frozen: they behave identically no matter which agent is acting or how much it has improved. The usual fix is to generate new environments, which pins you to domain-specific pipelines and LLM-written verifiers that have to be over-generated and filtered. EnvHarness inverts the move. It wraps an existing environment in plug-in components that operate strictly through the standard reset() / step() interface, changing where an episode starts, what the agent may do, and what it sees, while the underlying simulator, tasks, and human-built verifier stay untouched. An LLM designer called EnvRigger writes those wrappers automatic...

Anthropic Opens a Research Preview of the Model Hardware Standard (MHS): A Shared Specification for AI Agents to Safely Operate Physical Devices

Anthropic has opened a research preview of the Model Hardware Standard (MHS) , a shared specification that lets AI agents discover and operate physical devices. The problem it targets is plumbing. A lab bench or factory cell is assembled from vendors that never planned to interoperate, so every instrument ships its own interface and specialists hand-write bespoke translators between each pair. According to Anthropic team, the setup normally takes weeks to months, and that MHS cuts it to hours or minutes. The Integration Tax Each instrument ships its own programming interface, so specialists hand-write bespoke translators between every pair. Even once wired together, there is no common way for devices to hand state to an agent or be operated safely by one. Anthropic says setup typically runs weeks to months; MHS reduces it to hours or minutes. How it works MHS standardizes the driver — the layer between an OS and a device. It exposes a small primitive set: read (get ...