Async cancellation is a protocol, not an exception

The exception is the least interesting part of cancellation. In a real-time turn, the work is already distributed across a topology: an ingress task receives audio, endpointing decides that speech ended, STT produces partial and final text, context assembly reads memory, the model streams tokens, a tool may cross a side-effect boundary, TTS converts an unstable text prefix into audio, and the transport schedules playout. Cancelling only the coroutine currently awaiting the model does not cancel that topology. It creates a stale generation whose outputs can race the user who already moved on.

Cancellation is modeled as a protocol carried by a stable turn identity plus a monotonically increasing generation. A cancellation request names the generation that lost authority. Each stage observes that request, stops accepting new work for the generation, drains or discards queued work according to an explicit policy, and reports a terminal state. The supervisor can then distinguish cancellation requested, cancellation observed, cleanup running, and cancellation committed. That distinction matters when a provider ignores client disconnects, a tool call is already in flight, or the audio device still contains buffered frames.

Backpressure and cancellation are the same control problem viewed at different timescales. Backpressure answers whether a stage may accept more work while the generation is valid. Cancellation answers what happens to already accepted work after the generation becomes invalid. An unbounded queue evades both questions: it turns overload into latency, keeps obsolete generations resident, and makes shutdown time proportional to invisible backlog.

Every queue needs a capacity, an age budget, a saturation policy, and a cancellation policy. Audio ingress may drop the oldest frames only when the product can tolerate discontinuity; control events usually require ordered delivery; speculative TTS should prefer discarding stale chunks over preserving completeness; persistence queues should reject duplicates but retain committed events. The policy belongs next to the queue declaration and telemetry, not in tribal knowledge.

A user interruption can revoke conversational authority, but it cannot make a remote API forget a request it accepted. That boundary needs a different mechanism. An idempotency key names the logical operation while a generation fence identifies the authority that requested it. Before starting a side effect, the tool adapter verifies that the generation is still current. When the provider supports idempotency, retries reuse the same key. When it does not, the runtime records an ambiguity state instead of pretending the operation failed safely.

The important state is not success versus exception. It is not-started, accepted-locally, accepted-remotely, committed, rejected-by-fence, and ambiguous-after-timeout. Those states let the UI and recovery path tell the truth. A timed-out payment-like operation, message send, or infrastructure mutation must not be automatically retried as if no effect occurred. Cancellation turns into a safety property only when the effect boundary can answer whether the revoked generation still owns the operation.

1async def run_effect(turn, generation, request):2    await turn.assert_current(generation)3    operation_id = stable_operation_id(turn.id, request.logical_key)4 5    outcome = await provider.execute(6        request.payload,7        idempotency_key=operation_id,8    )9 10    await turn.commit_effect(11        generation=generation,12        operation_id=operation_id,13        provider_receipt=outcome.receipt,14    )

A runtime that handles barge-in but leaks tasks during shutdown does not have a cancellation model; it has a happy-path interruption feature. Shutdown revokes multiple scopes at once: active turns, provider streams, retry loops, heartbeat tasks, background flushers, and transport readers. The supervisor must close admission first, then request cancellation, allow bounded cleanup, flush authoritative state, and finally prove that no owned task remains.

TaskGroup helps because child failure and parent lifetime become visible, but structure alone does not define semantics. Cleanup code must preserve cancellation instead of swallowing it under a broad exception handler. A timeout around cleanup must report what was abandoned. Shielding is reserved for narrow critical sections such as committing a final idempotent record; shielding an entire provider call merely hides the leak from the caller.

1. Close admission so no new turn or tool operation enters the runtime.
2. Advance the generation fence and broadcast cancellation to every active turn.
3. Stop producers before consumers, then apply each queue’s explicit drain or discard policy.
4. Run bounded cleanup for provider sessions, transport buffers, and idempotent persistence.
5. Assert that the supervisor owns zero live tasks; report every timeout or ambiguous effect as state.

A cancellation metric named duration is not enough. A per-turn trace should mark request time, first stage observation, last audible frame from the revoked generation, provider disconnect acknowledgement, queue drain outcome, persistence completion, and supervisor terminal state. The user-facing measurement is often audio cutoff latency; the engineering measurements explain why the cutoff moved.

The failure counters matter more than a flattering percentile: stale chunks rejected by generation fence, side effects blocked before start, ambiguous remote outcomes, cleanup timeouts, orphan tasks at session close, and queue items discarded by policy. These are protocol violations or deliberate safety decisions. When they are named explicitly, a team can change providers or refactor the task graph without losing the operational contract.