Voice Agent Latency: What Every Vendor Actually Publishes

Every voice vendor publishes a latency number and almost none of them measure the same thing. One is model inference with the network explicitly subtracted. One is the gap between a word ending in the audio and that word showing up in a partial transcript. One is time to the first audio byte. One is a whole-turn figure from a platform's own dashboard, which starts and stops inside that platform's stack. Stack them and you get a total that corresponds to nothing a caller would experience.

This page is a compilation, not a measurement. Every figure below comes from a published, linkable source, and each row states what that source was actually measuring and when it was published. I have no vendor credentials for most of this list and did not run a single timing test for it. The second half is the part you can act on: a method for producing your own end-to-end number, and the controls that make it comparable between two stacks.

The four things called "latency"

Before the tables, the vocabulary. These are distinct measurements and the naming across vendors is inconsistent enough that you have to read the definition every time.

Transcript lag (STT). How far behind the transcription is from the audio being sent. Deepgram's documentation defines it exactly that way and gives the arithmetic: subtract the transcript cursor from the audio cursor, where the transcript cursor is the amount of audio processed so far according to interim results. It is a measure of a stream falling behind, not of a response arriving.

Word emission latency (STT). AssemblyAI's definition, quoted from their Universal-Streaming announcement: "for each correctly transcribed word, measure the time from when the word ends in the audio stream to when that same word first appears in the partial transcript received by the client." Per-word, client-side, and measured on a stream that is already flowing.

End-of-turn detection delay. The time from a caller stopping speaking to the system deciding the turn is over. Deepgram's docs call this EOT latency and put the expected range at 100 to 500 ms. LiveKit calls it end_of_utterance_delay and defines it as time from the end of speech as detected by VAD to the point when the user's turn is considered complete. This is the stage where most of the perceived delay in a slow agent actually lives, and it is the stage vendors are least likely to include in a headline figure.

Time to first audio (TTS), and voice-to-voice. For a TTS model, the elapsed time from the request to the first playable audio. For a whole agent, the elapsed time from the caller's last syllable to the first syllable of the reply. Only the second one describes what a caller experiences, and no vendor in this compilation publishes it as a measured distribution.

LiveKit publishes the cleanest decomposition of the three machine stages, as a formula in its agent metrics documentation: total_latency = eou.end_of_utterance_delay + llm.ttft + tts.ttfb, where TTFT is the time for the LLM to generate the first token of the completion and TTFB is the time for the TTS model to generate the first byte of its audio output. Worth noting what that sum leaves out: network transit, the telephony leg, jitter buffering and the audio player's own buffer. It is a diagnostic breakdown, not an end-to-end number.

Speech-to-text

Two of these deserve a caveat in bold type. The AssemblyAI figures are a vendor benchmarking a named competitor, which is exactly the situation where you read the method before the number. To their credit the method is stated precisely, which is more than most such comparisons offer. And the Deepgram Flux numbers describe a different quantity from the Nova-3 numbers on the row above: Flux merges transcription and turn detection into one model, so its p90 and p95 cover the decision that a turn has ended, not the lag of a transcript stream.

Text-to-speech

The ElevenLabs rows are the most honest set of numbers in this entire compilation, and worth reading together rather than separately. A 75 ms model with a 20 to 200 ms network round trip and a 500 ms player buffer in front of it does not deliver audio to a caller in 75 ms. The vendor says so on its own documentation page. Anyone quoting the 75 ms as an agent's response time is quoting a number the vendor did not make that claim about.

Full-stack platforms

Retell's table is the only published head-to-head of full-stack platforms I could find from a primary source, and it is a vendor scoring itself against three competitors with no disclosed sample size and no percentiles. That does not make the numbers wrong. It makes them a starting hypothesis rather than evidence. If you are choosing between platforms on latency, this table is the thing you should be replicating, not citing.

LiveKit's refusal is the more useful position for a reader. Publishing a formula that stays true while publishing no numbers that will rot is a defensible call, and the formula is the one I use when a pipeline is slow and I need to know which stage to blame.

The language axis

This is the shortest section in the compilation, because the answer is nearly uniform: no vendor in this list publishes a latency figure broken out by language.

Why this matters more than it looks: language changes the work at almost every stage. Code-mixed Hindi-English speech is harder to segment, which moves end-of-turn detection. Devanagari text expands differently from Latin text for the same spoken content, which moves TTS first-byte time. A model serving 23 languages from one endpoint has different routing behaviour from a model serving one. None of that is captured by a single number published once, in English, in a launch post.

If you are building for Indian callers, the practical consequence is that nothing in the published record substitutes for measuring your own language mix. That is doubly true for anything touching Indian government service delivery, where the language list runs well past the ones any commercial vendor benchmarks and where Indic-language public service agents have to work in languages that have never appeared in a vendor latency chart.

What the independent evaluations measure, and what they don't

There are two widely used independent evaluations of speech models. Neither measures latency.

Artificial Analysis reports, for speech-to-text, a word error rate and a "Speed Factor" defined as input audio seconds transcribed per second, measured as a median across trials on ten-minute audio files. For text-to-speech it reports a quality Elo from its Speech Arena, price, and throughput in characters per second of generation time. Throughput on a ten-minute file and time-to-first-token on a live stream are different quantities, and only the second one predicts whether a conversation feels responsive.

That is the state of it. Every latency figure in circulation is a vendor figure. There is no neutral, reproducible, published end-to-end benchmark of voice agents that I was able to verify from a primary source, which is a strange gap in a category this large.

How to measure it yourself

What follows is a method, not a result. It is what I would hand to someone comparing two stacks who needs an answer they can defend.

Measure voice-to-voice, from the recording

The only number that describes a caller's experience is the elapsed time from the caller's last syllable to the first audible syllable of the reply, measured at the caller's end. Get it from the recorded audio, not from a platform dashboard. Platform timestamps start and stop inside the platform, which means they systematically exclude the telephony leg, the jitter buffer and the player buffer. Those are not rounding errors; ElevenLabs' own documentation describes a 500 ms player buffer as common.

Detect the speech boundaries with a voice activity detector run over the recording, then refine with an energy threshold and eyeball a random sample to confirm the detector is finding what you think it is. A VAD that clips trailing breath or counts line noise as speech will shift every number in the same direction, and you will not notice from the aggregate.

Control these, or the comparison means nothing

• Region and point of presence. Same region for both stacks. A model served from Virginia and one served from Mumbai are not being compared on model speed.
• Transport and codec. PSTN with 8 kHz μ-law and WebRTC with 48 kHz Opus are different measurements. Pick one and state it.
• Network path. Same connection, ideally the connection your callers use. A test from an office fibre line does not predict a mobile call in a market town.
• Prompt and context length. LLM time-to-first-token moves with context size. Two stacks with different system prompts are not being compared on infrastructure.
• Turn detection settings. Endpointing thresholds, turn timeouts and any eagerness setting. These change latency more than the model choice does, and they are usually left at default in a comparison where one default happens to be aggressive.
• Barge-in. On or off, stated. Interruption handling changes both the measurement and what counts as a turn boundary.
• Warm versus cold. First request after an idle period behaves differently. Discard or report the cold starts separately; do not average them in silently.
• Utterance script. Same set of caller utterances through both stacks, covering short confirmations and long questions. A "yes" and a forty-word question do not exercise turn detection the same way.

Sample size and what to report

Collect enough turns that p95 stops moving when you add more, which in practice means several hundred, spread across times of day. Then report p50 and p95 together with n, and never the mean. Latency distributions have a long tail, the mean hides it, and the tail is the part callers remember, because a caller does not average their turns.

Alongside the end-to-end figure, log the decomposition from LiveKit's formula if your stack exposes it: end-of-utterance delay, LLM time-to-first-token, TTS time-to-first-byte. The sum will be lower than your measured end-to-end. The difference is your network, telephony and buffering, and knowing its size tells you whether swapping a model is worth doing at all.

One reference point that does not apply

Telephony has a long-standing standard for delay, ITU-T Recommendation G.114, on one-way transmission time. It is worth knowing it exists and worth knowing it does not answer this question: it governs transmission delay on a call path, not the time an agent takes to decide a turn has ended and start speaking. There is no equivalent standard for conversational turn latency, which is a large part of why every vendor gets to define the term for themselves.

What I take from the compilation

Three things. Vendor latency figures are mostly honest and mostly useless for comparison, because each one measures the stage its publisher controls. The stage that dominates perceived delay, end-of-turn detection, is the one least often included in a headline number. And for Indic languages, the published record is close to empty, so anyone building for Indian callers is measuring for themselves or guessing.

For the cost side of the same decision, the pricing breakdown across the three commercial models covers what these stacks bill for and where the per-minute rates land. And if you want the operational view rather than the timing one, how voice agents actually handle support calls goes through the stages above from the caller's side. I build on several of these stacks in production through Voxdonna, which is how I came to care about the difference between a 75 ms number and a 900 ms call.

Suyash Raj Founder, Voxdonna AI and AiSewak. Writes here about n8n, AI agents, and voice automation.