Lowest-Latency Inference APIs for Voice and Realtime Agents: A Time to First Token TTFT-First Benchmark
Time to first token (TTFT) is the metric teams use to pick an inference API for voice. It is also the metric that misleads them. TTFT marks when generation starts; a text-to-speech model cannot speak until a full clause arrives. Between those two points sits the difference between an agent that feels conversational and one that gets interrupted. This piece benchmarks every layer of the voice stack including LLM, speech-to-text, text-to-speech, and speech-to-speech. Why TTFT Is the Right Entry Point and the Wrong Finish Line A voice agent is a latency budget with a language model inside it. Every stage spends milliseconds the user can hear. Time to first token (TTFT) is the interval between sending an inference request and receiving the first token back. IBM’s definition frames it as the moment a system transitions from idle to visibly active. For chat, TTFT is close to the whole story. For voice, it is one term in a sum. The reason is mechanical. A text-to-sp...
