中文 English

You Said “Stop”—So Why Is the AI Still Running? I Broke the Turn Manager’s Cancellation Chain on Purpose

Published: 2026-08-23 · 阅读量 --
AI Agent AI 架构 平台工程 可观测性 分布式 WebRTC 工程实践

TL;DR

A disabled “Stop generating” button does not prove that the AI stopped. The model stream may still consume tokens, retrieval may still hold a connection, the TTS queue may keep playing an obsolete answer, and a tool may already have sent an irreversible commit to an external system. Real cancellation is neither a UI style nor a forcibly closed HTTP connection. It is a state transition owned by a Turn Manager: identify every child operation belonging to the turn, propagate a cancellation signal through that tree, clear audio that has not played, fence late events with a generation number, prevent new side effects when the deadline budget is exhausted, and honestly record whether the outcome is cancelled, completed, or requires reconciliation. I built a zero-network, standard-library Python lab that deliberately creates cancellation races, voice barge-in, deadline exhaustion, an uncancellable commit, and disconnect/reconnect. All eight scenarios and all 24 assertions pass.

Original cover: stopping must be a platform event, not a button style.

Figure 1: Original cover. One stop signal must govern the model stream, retrieval, tool work, and audio output. Missing any branch creates the experience of “I said stop, but the AI kept working.”

1. The Background: The Dangerous Part Is Not a Slow AI, but One That Pretends to Have Stopped

The simplest chat UI implements Stop like this: when the user clicks the button, the browser aborts the current request and stops appending text. That can appear sufficient for a demo that only generates a paragraph. A production AI Platform, however, may have all of these operations behind one visible turn:

Cancelling only in the browser is like turning off a restaurant’s pickup display without telling the kitchen to cancel the order. The customer can no longer see it, but the cook is still preparing it and the cashier may still capture payment. The customer assumes the order is gone and places another one; the business may produce two effects while showing only one.

The symptoms are familiar. Text stops while token usage continues. A user interrupts a voice answer, yet half a sentence comes out of the speaker later. Reconnecting repeats a paragraph. Three clicks on Stop launch three cleanup operations. A task marked “cancelled” later sends an email, opens a ticket, or commits a change. These are not unrelated SDK bugs. They share one root cause: the system treats a transport connection as the work itself, and no component truly owns the lifecycle of the whole turn.

2. A Turn Is Not an HTTP Request

An HTTP request is a vehicle. A Turn is the unit of work. A Turn means: “a user supplied an intention; the system generated, retrieved, invoked, played, or persisted work for that intention until one authoritative terminal state was reached.” It may begin with one HTTP request, send incremental events over a WebSocket, execute tools through a queue, and finally replay its terminal event through a later reconnect request.

Think of a teacher asking a child to answer one problem. The question starts the Turn. Reading notes, thinking, raising a hand, and speaking are child operations. The classroom loudspeaker is the transport. If the loudspeaker fails, the problem does not automatically cease to exist. The teacher saying “Stop, the question has changed” is the business cancellation. A disconnect is an observed fact; cancellation is a policy decision. The two must not be silently equated.

Original diagram: HTTP transports events; the Turn Manager owns the lifecycle.

Figure 2: After the browser disconnects, who stops the model, tool, and audio? If the answer is “nobody knows,” the system is missing a Turn Manager. It can begin as a well-defined module in a modular monolith; the name does not require an immediate microservice split.

A production Turn envelope should carry at least:

One invariant belongs in the state machine and, where possible, the database constraint: a Turn can acquire only one terminal state. A cancel arriving after COMPLETED receives “already completed.” A model-completed event arriving after CANCELLED is rejected. If an irreversible external commit succeeds after cancellation was requested, the platform must not relabel reality as CANCELLED merely to make the UI look tidy.

3. Establish the Happy-Path Baseline First: Fast TTFT Is Not Turn Completion

Before injecting cancellation, the local lab establishes a normal synthetic stream. The Turn is accepted; retrieval completes; the model starts streaming; the first token becomes visible at 61ms; and exactly one COMPLETED state is recorded at 110ms. This virtual clock is not a performance benchmark. It creates a reproducible timeline against which races can be asserted.

Real local lab capture: the baseline stream finishes within its deadline and writes one terminal state.

Figure 3: Real output from the local script. A socket guard blocks networking and every event is synthetic: provider=synthetic, real_models=0. TTFT and E2E belong to the same Turn, but “started answering,” “finished answering,” and “completed the business action” remain three different moments.

Why bother with a baseline? Many dashboards call an HTTP 200 or the first token “success.” For a tool-using Agent, that proves only that it started talking. If the request is “cancel my reservation,” the first token may arrive in 300ms while the real cancellation takes eight seconds. When the user says stop at second two, the platform needs to know whether it stopped text, planning, the tool, or all of them.

4. The Cancellation Tree: Do Not Broadcast One Boolean and Hope Everyone Notices

When a Turn Manager accepts a stop request, it should create an auditable state transition and then propagate the signal to every child operation derived from that Turn. A useful model is structured concurrency: the parent owns its children, and a child cannot quietly escape into an orphan task. Model adapters, retrievers, tool runners, and audio buffers register cancellation callbacks or cooperative checkpoints; the parent waits for them to converge on explainable outcomes.

Original cancellation tree: one parent token reaches four child branches, while a delivery gate rejects late events.

Figure 4: Propagating the signal is only the first half. The other half is the delivery gate. Even if a provider produces another delta after accepting cancellation, an obsolete generation must never re-enter the UI.

A cancellation tree is not a global cancelled = true variable. Its signal needs a reason, time, deadline, generation, and request identity. A robust sequence looks like this:

  1. transactionally compare and move the Turn from RUNNING to CANCEL_REQUESTED;
  2. immediately close the old generation’s delivery gate to prevent ghost output;
  3. notify retrieval, model, tool, and TTS children to stop or clean up;
  4. wait for a bounded grace period and collect which branches stopped and which are committing;
  5. write CANCELLED, FAILED, or RECONCILE_REQUIRED from facts rather than always displaying “cancel succeeded.”

The lab injects turn.cancel at 75ms. Retrieval, model, and TTS observe it. A model delta arriving at 82ms is discarded by the delivery gate. The Turn enters CANCELLED at 83ms, an 8ms propagation interval in the virtual timeline.

Real local lab capture: cancellation drains three children and the generation gate drops a late delta.

Figure 5: Real lab output. The 8ms value is a deterministic virtual-clock assertion, not a claim about any model provider. Production acceptance should measure p50, p95, and p99 across the real chain.

5. A Deadline Is Not a Separate Timeout at Every Layer

timeout=30s sounds simple. If retrieval, model, and tool each independently receive 30 seconds, however, the user may wait 90 seconds. Worse, a parent with only 200ms remaining can start a write tool likely to take 20 seconds; the page times out, but the side effect continues.

A deadline is the absolute end of the Turn. A timeout limits one stage. The Turn Manager should allocate the remaining time like a finite allowance: 25ms for retrieval, 55ms for the model, and 10ms held back for shutdown. Every child receives the smaller of its own maximum and the parent’s remaining budget. If a child has ten dollars, spends four on a pencil, and then behaves as if it still owns ten for a notebook, its arithmetic is wrong. Layered timeouts make the same mistake.

Real local lab capture: after the total budget expires, the model is cancelled and no tool starts.

Figure 6: Real lab output. At the 90ms deadline, the model observes deadline_exceeded; the planner records tool.not_started reason=no_budget; and the terminal result is the explicit FAILED/DEADLINE_EXCEEDED, not a vague network error.

Production systems should distinguish user cancellation, the ingress deadline, time to first token, stream idle timeout, tool timeout, and platform shutdown drainage. Their retry policies differ. A user stop normally must not auto-retry. A first-token timeout might permit bounded fallback. A tool whose acknowledgement timed out after commit must be queried by idempotency key, never blindly executed again.

6. The Hard Part of Streaming: Events Already in Flight Do Not Obey You

After sending cancel to a model, data may remain in the provider, SDK, reverse proxy, kernel, or local buffer. Closing the current socket has three problems: the connection may be pooled, close may not cancel remote computation, and a reconnect may replay old events from storage.

A safer internal event has turn_id, monotonic seq, generation, event type, and time. A Channel Adapter can normalize provider-specific SSE, WebSocket, or WebRTC events, but it must preserve sequence and generation. The delivery gate accepts only the active generation. Obsolete deltas may be retained as audit evidence; they cannot be displayed or played.

Return to the classroom analogy. When the teacher changes the question, an old answer already written on paper does not evaporate. The board now says “answer version 8.” A child arriving with version 7 can file it in the evidence folder but cannot read it aloud. Propagation asks upstream work to stop; generation fencing prevents contamination even when upstream stops late. Both are required.

7. Voice Barge-in: Interrupting Requires More Than Muting the Speaker

Voice makes cancellation visible. Text generation, speech synthesis, audio encoding, jitter buffering, and physical playback all have queues. If an old answer plays for half a second after the user starts speaking, the system sounds like a rude person talking over someone. If it only mutes playback, the model may continue generating an answer nobody will hear.

Barge-in is not one wire event. It is a tightly coordinated chain:

  1. VAD or push-to-talk detects new user speech;
  2. the Turn Manager accepts interruption and advances the generation;
  3. queued output audio is cleared immediately;
  4. the old model/speech response is cancelled and no new synthesis is scheduled;
  5. the delivery gate drops late audio from the obsolete generation;
  6. a new Turn is created for transcription and response.

Original timeline: speech started, clear audio, cancel the old response, fence the late packet, and create a new Turn.

Figure 7: User-perceived interruption requires at least three outcomes: immediate silence, old work asked to stop, and old packets unable to revive. Calling response.cancel without audio.clear still plays what is already buffered.

In the lab, generation 7 plays two synthetic audio frames before speech_started is injected at 70ms. Barge-in is accepted at 71ms, four buffered frames are cleared at 72ms, the old response is cancelled at 73ms, and a late generation-7 packet is dropped at 79ms. Generation 8 then completes the new answer.

Real local lab capture: voice barge-in clears audio, cancels the old generation, and lets the new Turn continue.

Figure 8: Real lab output. No microphone or speech provider is used; the events test the minimum protocol semantics. A real implementation must additionally handle echo cancellation, VAD false positives, half-duplex hardware, and mobile audio focus.

VAD must not cancel every time it sees noise. Loudspeaker echo, keyboard clicks, and coughing can look like speech. Practical policy combines acoustic confidence, duration, push-to-talk state, whether the current response is interruptible, user preferences, and device echo state. A cough must not push a high-risk tool into an inconsistent state. Voice UX optimizes for speed; the state machine optimizes for truth. The Turn Manager reconciles the two.

8. Stop Must Be Idempotent: Three Clicks May Apply the Brake Only Once

Mobile networks retransmit. Frontends retry when acknowledgements vanish. Users click repeatedly. If every cancel launches a new cleanup task, the system may revoke twice, send duplicate notifications, or accidentally kill a newer Turn. A cancel carries a stable request ID, and the server compares state transactionally: only the first request can move RUNNING to CANCEL_REQUESTED. Later requests return the same terminal result without producing another cancellation effect.

Real local lab capture: three stop requests create only one cancellation effect.

Figure 9: Real lab output. The first stop writes one transition. A retry with the same request ID and a later stop with a different ID both receive already_terminal; cancel_effects remains 1. Idempotency does not mean ignoring requests—it means repeating one request yields one consistent result.

The API response should be explicit. accepted means cancellation has begun. already_terminal means the Turn previously ended. reconcile_required means an external commit remains unresolved. Hiding all branches behind HTTP 200 and “operation successful” destroys the information the caller needs.

9. Tool Cancellation Boundaries: Cooperative Work Can Stop; Committed Work Needs Honesty

Operating systems and runtimes generally let us request cancellation; they cannot safely cut arbitrary code at an arbitrary instruction. Killing a thread can strand a lock, leave a half-written file, or make transaction state unknowable. A cancellable tool therefore exposes checkpoints: after reading, after producing temporary output, and before publishing. On cancellation it cleans up temporary work and keeps committed side effects at zero.

Original tool boundary: cooperative work rolls back temporary state; an external commit in flight requires reconciliation.

Figure 10: Cancellation is not time travel. Before commit, work can stop. After a commit has been sent, the platform can stop further delivery, preserve the idempotency key, and query the outcome. Labeling an unknown commit “cancelled” creates more danger than an honest timeout.

The cooperative lab tool reads its synthetic input and writes temporary output. The user cancels at 65ms. At 70ms, the tool observes the token before publish, removes temporary work, and finishes with zero committed side effects.

Real local lab capture: a cooperative tool observes cancellation before publish and cleans temporary work.

Figure 11: Real lab output. A production adapter should declare its cancellation capability, checkpoints, and cleanup result. Writing “supports cancellation” in a tool description is useless unless the Worker can prove where it stopped.

The second scenario injects cancellation into the worst window: an external commit carrying an idempotency key has been sent, but the acknowledgement has not arrived. The Turn Manager can stop new text, TTS, and subsequent tools; it cannot promise that the commit did not happen. A synthetic acknowledgement arrives at 105ms with one side effect. The correct state is therefore RECONCILE_REQUIRED, not CANCELLED.

Real local lab capture: cancellation follows an external commit, so the Turn honestly enters RECONCILE_REQUIRED.

Figure 12: Real lab output. This is not a failure to understand cancellation; it is an accurate physical boundary. The next step is to query by idempotency key, compensate, or ask a human—not to replay unconditionally.

Classifying tools by cancellation behavior helps:

This is also why Tool Calling and MCP solve capability discovery and invocation protocol, not cancellation, authorization, and transactions automatically. A protocol may carry the intent to cancel; the platform still defines the tool’s physical boundary.

10. Disconnect and Reconnect: Resume Display; Do Not Regenerate the Turn

Mobile network changes, browser sleep, and WebRTC renegotiation are normal. If reconnect asks the model the question again, it wastes cost and may repeat tools. The Turn Manager should persist normalized events briefly before sending them. The client acknowledges the highest sequence it has processed and later reconnects with a non-guessable resume token plus after_seq.

Original reconnect flow: replay after the last acknowledged sequence and acknowledge duplicates without rendering them twice.

Figure 13: Sequence controls order, the resume token controls authorization, and TTL controls lifecycle. A sequence without authorization risks cross-user disclosure; a token without sequence creates duplicate rendering.

The lab disconnects after the client has acknowledged sequence 3. The server persists sequences 4 and 5 plus terminal event 6. The client resumes with after_seq=3, receives 4..6, and ignores a duplicate sequence 5. The rendered text is exactly one The service is healthy.

Real local lab capture: cursor replay resumes after disconnect and does not render the duplicate event.

Figure 14: Real lab output. The scenario tests event semantics, not real network quality. Production also needs resume TTL, user/device binding, event compaction, and an explicit “cannot resume” error if storage is unavailable rather than silently re-executing.

If the Turn is cancelled while the client is disconnected, reconnect must receive the authoritative terminal state before deciding how much earlier text to replay. A CANCELLED Turn cannot be resurrected by an old cursor. Generation and terminal-state checks apply on every replay, not only the initial connection.

11. Observability: “The Cancel Endpoint Returned 200” Is Not a Useful SLO

A trustworthy dashboard separates:

Original acceptance dashboard: TTFT, cancellation, late events, resume, and side effects must be read together.

Figure 15: The article’s numbers use a virtual clock for a reproducible gate. Production thresholds must vary by text, voice, and tool risk. An external effect confirmed after cancel is not automatically a bug; failing to label and reconcile it is.

Use the Turn span as the trace root, with transport, retrieval, model stream, policy, tool, TTS, cancellation propagation, and replay as children. Cancellation reason, generation, deadline, last acknowledged sequence, and tool idempotency key can be controlled attributes. Prompts, audio, and tool output still require data classification and redaction; observability must not become a new secret warehouse. If the telemetry stack itself is missing, the MySQL + Prometheus + Loki + Grafana Agent lab offers a starting point, but dashboards cannot replace a state machine.

12. The Reproducible Lab: Eight Scenarios Proving Stop Is Not a UI Illusion

The downloadable standard-library Python lab performs no network request, invokes no real model or tool, and reads no credential. While scenarios execute, it replaces socket connection and DNS entry points with fail-closed guards. Every identity, tool, and event is synthetic, and a virtual clock keeps event order and captures repeatable.

The eight scenarios cover baseline streaming, a cancellation tree, voice barge-in, deadline budgeting, a cancellable tool, a commit that cannot be recalled, reconnect replay, and idempotent stop. Evidence includes a log per scenario, events.jsonl, metrics.json, report.json, and the aggregate release gate.

Real local lab capture: all eight scenarios and all 24 assertions pass.

Figure 16: Real local acceptance output. 24/24 PASS proves the control semantics defined by this lab are internally consistent. It is not a production security certification and does not promise that every provider stops promptly. External cost is zero.

One-Command Manual Run

Download and extract the lab bundle into a new directory. Python 3 is the only runtime prerequisite. The scripts install and download nothing.

Windows 11:

powershell -ExecutionPolicy Bypass -File .\run-windows11.ps1

Ubuntu 26.04:

bash ./run-ubuntu-2604.sh

macOS 26:

bash ./run-macos-26.sh

All three wrappers invoke the same Python core and exit nonzero unless lab-output/report.json says PASS. Individual downloads are also available for Windows 11, Ubuntu 26.04, macOS 26, and the full README.

Agent-Driven Configuration and Acceptance

Give agent-task.json to a local Agent and restrict it to the extracted directory. The task declares: deny network, forbid real models and tools, forbid secrets, and limit writes to lab-output/. Success requires exit code 0, report.status=PASS, and 24/24 assertions together.

This prompt is intentionally portable:

Read agent-task.json in the current directory, identify the operating system,
and run its matching command. Do not access the network, install dependencies,
read environment credentials, or expand the write scope. Then parse
lab-output/report.json. Report success only if exit code is 0, status is PASS,
and assertions are 24/24. On failure, preserve all evidence, name the scenario
and assertion, do not retry forever, and do not edit the lab to make it pass.

Agent mode adds --agent to the shell wrappers (-Agent in PowerShell) and emits compact JSON for automation. “Agent-driven configuration” here means safely wiring an Agent into the acceptance task. It does not authorize production Turn Manager changes; those still require review, staged deployment, and rollback.

13. Pre-Release Fault Injection: What “Passed” Should Mean

Add these races to a release gate instead of manually clicking Stop once:

  1. cancel at random token boundaries and prove the obsolete generation is no longer delivered;
  2. cancel both slow-retrieval/fast-model and fast-retrieval/slow-model orders, leaving no orphan;
  3. barge in with 100ms, 300ms, and 800ms TTS buffers and measure audible tail;
  4. race cancel against COMPLETED in the same scheduling window and permit one terminal state;
  5. cancel before, during, and after commit and reconcile state with side effects;
  6. drop the cancel acknowledgement and retry three times, producing one cancellation effect;
  7. complete, cancel, or fail while disconnected, then resume with an old cursor and prioritize authority;
  8. leave almost no deadline budget and verify that a new write tool cannot start;
  9. make the provider ignore cancel and prove the delivery gate still blocks late content while measuring wasted tokens;
  10. restart a Turn Manager instance and prove state, generation, and replay cursor were not memory-only.

The release verdict must remain precise. “All cooperative work stopped within SLO,” “the irreversible operation entered reconciliation,” and “the user received no obsolete output” are separate statements. None substitutes for another.

14. A Practical Migration Path: Install the Delivery Gate Before Rewiring Everything

If today’s system owns only a Stop button, it does not need a one-night rewrite. Move inward by risk:

First, add Turn ID, sequence, and generation to events. Build the delivery gate at the edge so obsolete output cannot reach users. Upstream work may temporarily waste compute, but it cannot contaminate a new Turn.

Second, persist authority and deadline. Use a transaction for one terminal state, allowing reconnect and multiple instances without relying on one process’s memory. Distinguish CANCEL_REQUESTED from CANCELLED.

Third, wire cooperative cancellation into models, retrieval, and TTS. Each adapter reports acceptance time, stop time, and remaining buffer. A provider that cannot cancel is still isolated by the generation fence.

Fourth, define cancellation capability for every tool. Alongside schema, document cancellability, commit point, idempotency, compensation, and reconciliation. Block high-risk tools when remaining deadline is insufficient.

Fifth, implement replay and fault injection. Once persistence, ACK/sequence, resume token, TTL, and authorization binding are present, add the races to continuous release testing.

This path follows the layers in the production AI Platform reference architecture: the Turn Manager sits in the experience plane, but its cancellation signal crosses control and execution, and its evidence lands in the data and observability plane. Use the AI Platform architecture reading map to continue by responsibility.

15. Q&A

Q1: Is AbortController.abort() in the frontend enough?

It is an excellent entry point, but it proves only that a local consumer received a signal. It does not prove that a remote model, queue Worker, tool, or TTS stopped. Map the abort to an authenticated server event with Turn ID, then propagate it through the tree.

Q2: Should closing the browser automatically cancel a Turn?

It depends. Pure chat may cancel; a long report may continue in the background; a write operation must not use disconnect as its decision. Record transport_disconnected, then apply explicit product policy to continue, pause, or cancel.

Q3: Is killing the Worker not the fastest solution?

It is an isolation last resort, not a protocol. The Worker may hold a lock, have half-written a file, or already have sent an external request. Prefer cooperative cancellation and deadlines. After a grace period, terminate the process if needed and hand unknown effects to recovery and reconciliation.

Q4: Why keep CANCEL_REQUESTED instead of writing CANCELLED immediately?

“The user asked to stop” and “all work converged safely” are different moments. The intermediate state lets UI show stopping, allows the backend to await tool facts, and prevents an irreversible commit from being falsely reported as recalled.

Q5: Must voice barge-in wait for transcription to finish?

Usually not. VAD or push-to-talk can clear audio and advance generation first; transcription then supplies the new Turn’s content. Echo and noise policy is still necessary so the system does not interpret its own speaker as user interruption.

Q6: What if the model provider does not support cancel?

Stop delivery of the old generation immediately, stop downstream TTS and tool planning, and record the tokens and time the provider continues consuming. Close a dedicated connection or use a provider API if available, but do not claim remote compute definitely stopped.

Q7: What happens to a side effect that completes after cancellation?

Query the authoritative result by idempotency key. Compensate if possible or request human confirmation if not. Audit when cancel was requested, when commit was sent, and when the outcome became known. Never blindly retry.

Q8: Which transport is best: SSE, WebSocket, or WebRTC?

They serve different needs. SSE is simple for one-way text, WebSocket for bidirectional events, and WebRTC for low-latency media and data channels. None eliminates the need for Turn ID, generation, sequence, deadline, and authoritative state.

Q9: What should the cancellation-propagation SLO be?

There is no universal number. Text UI can tolerate tens to a few hundred milliseconds; people notice voice audible tail more sharply; an external tool may need seconds to clean up. Define separate SLOs for stopping delivery, stopping cooperative work, and confirming external outcomes.

Q10: Does the Turn Manager become a single point of failure?

It does if authority lives only in memory. Keep API instances stateless or recoverable, persist Turn state, terminal version, generation, and replay events, and coordinate instances through optimistic locking, leases, or a single-writer rule rather than letting each announce its own outcome.

16. References and the Final Boundary

A Turn Manager is not a larger controller and not a cancel flag added to every request. It is the explicit answer to four questions: who owns this turn, when does it end, what happens to late events, and how are irreversible actions accounted for?

The final boundary is simple: after a user says stop, the platform may honestly say “stopping” or “the commit was already sent; confirming its outcome.” It must never display “cancelled” while obsolete text, old audio, and hidden side effects continue behind the screen.

本文阅读量 --