A voice agent latency budget I can actually debug

A useful voice-agent latency number is not a single average. The user experiences a path: audio ingress, VAD endpointing, STT, prompt and memory assembly, LLM first token, tool interception, TTS first audio, transport readiness, and playout.

Each stage needs a stable owner, a bounded queue or explicit pressure policy, and a telemetry name that survives internal refactors. Otherwise a fast-looking dashboard can hide slow endpointing, provider stalls, queue age, or an agent that keeps speaking after the user interrupts.

The operational view I want is per-turn and correlated: stage percentiles, queue pressure, provider fallback, interruption detection, audio cutoff, and the degraded path when voice output is unhealthy. Averages alone are not enough to debug a rude or hesitant conversation.

This public note deliberately omits customer context, provider credentials, private topology, and unsupported production metrics. The publishable artifact is the architecture method: assign the latency budget, instrument each boundary, preserve cancellation, and make text-first recovery a first-class mode instead of an exception.