Measure a voice companion’s pauses before calling it natural

“Real time” is often a mood rather than a measurement. In voice companionship, a delay can feel thoughtful, broken or interruptive depending on where it occurs. A field test should therefore measure the conversation’s visible stages instead of collapsing everything into one impressive number.

Define four timestamps

Record when the user stops speaking, when the interface shows input complete, when the first response audio begins, and when the answer ends. From those points calculate end-detection delay, response-start delay and playback duration. If the app exposes a transcript, also mark when first text appears. Do not call any of these “model latency” unless the product provides internal telemetry; the observed interval includes device, network and service work.

Control what you can

Use the same device, headset, connection and ten short prompts. Disable unrelated downloads. Run one set on stable Wi-Fi and another on cellular, then report them separately. Repeat at two times of day. The W3C WebRTC statistics specification defines current round-trip time for a selected candidate pair, but an app may not expose that statistic and network round-trip time is not the full conversational delay.

Count interruptions and repairs

Timing alone misses turn-taking. Mark false end detection, cases where the companion talks over a pause, failed barge-in, duplicated audio and requests the user must repeat. The ITU’s G.114 recommendation concerns one-way transmission time in telecommunication networks; it is a useful reminder that transport delay matters, not a threshold that certifies an AI conversation as natural.

Work a hypothetical example

Across ten prompts, suppose first audio starts in 0.9 to 2.8 seconds after speech ends, with a median of 1.4. Two long prompts trigger false endings and one interruption fails. Report the distribution and three repair events. Do not publish only 0.9 seconds, and do not infer why the slow turn occurred. A screen recording can support later coding, provided it contains no private conversation or account data.

Compare versions honestly

Keep the prompt set and measurement rule unchanged. A later version is better only if the chosen measures improve without more recognition or interruption errors. Record sample size and conditions next to every chart. Ten prompts reveal usability clues, not population performance.

Choose a threshold from the task

A storytelling pause and an emergency handoff do not tolerate the same delay. Before testing, define a product target for each interaction: casual reply, interruption, correction and safety referral. If median response improves while failed barge-in doubles, do not call the release faster overall. Keep raw observations so a later team can recalculate. The decision consequence should be explicit: ship, limit rollout, investigate a network region, or retain the previous voice path.

Latency becomes useful when it explains a specific friction rather than crowning a winner. Pair the results with the accessibility audit and the Moshi technical record to distinguish a reader test from a developer’s architecture claim.