Stop Testing Only the Answer: I Put Rate Limits, Injection, Crashes, and Duplicate Effects into the AI Platform Release Gate
TL;DR
Answering a test set well does not make an AI Platform ready for production. Model evaluation measures probabilistic answer quality. A release must also prove that identity survives every handoff, 401 never triggers a misleading fallback, retries after 429 are bounded, dangerous tools wait for approval, duplicate delivery produces one side effect, stale memory never enters the prompt, a correct policy denial is not counted as an outage, and every decision can be reconstructed from traces, audit, and cost. I turned those requirements into a zero-network, zero-credential lab with thirty executable gates across identity, gateway, policy, execution, memory, and evidence. Only 30/30 produces
RELEASE=PASS. A failed gate blocks the release; deleting the test to make the dashboard green is not remediation.
Figure 1: Original diagram. A candidate contains the model, prompt, tools, policy, and task state. Traffic grows only after identity, routing, permission, execution, memory, and evidence pass together.
1. Background: Demo Acceptance and Production Acceptance Are Different Exams
Many AI projects prepare a few dozen questions, run a new model or prompt, and ask reviewers whether the answers look better. This is valuable, but it covers only one slice of the system.
A model may answer accurately while another tenant’s memory enters context. It may write beautifully while queue redelivery performs two payments. Its happy path may be green while a 401 causes three provider switches, triple cost, and the same final failure. None of those versions should ship.
Imagine a school cafeteria testing a new meal. A tasting panel can prove that the dish tastes good. It cannot prove that ingredients are traceable, refrigeration works, allergens are labeled, a card is charged once, or food from a power outage is handled safely. Answer quality is the taste. Platform controls are the food-safety system around the kitchen. Production requires both.
2. Symptoms: Seven Surprises After a Green Pre-Release Dashboard
- Offline scores rise while real task completion does not.
- A model update calls one extra tool, doubling cost and side effects.
- Provider throttling triggers unbounded retries, exploding latency and spend together.
- Ten prompt-injection examples are refused; the eleventh still changes a tool call.
- A Worker crashes, the queue redelivers, and one service restart becomes two.
- A dangerous request is correctly denied but counted as a reliability failure.
- After an incident, only the final prose remains; identity, routing, approval, and actual effects cannot be proven.
The root cause is not simply an undersized evaluation set. Different classes of evidence were compressed into one score.
3. Root Cause: Deterministic Control, Probabilistic Quality, and Business Outcome Share One Number
Production acceptance needs at least three layers.
Figure 2: Original diagram. Transactions and authorization must be right every time. Model quality needs distribution and variance analysis. The team must still verify whether useful work was completed.
- Deterministic control: identity, tenant, argument schema, state transitions, idempotency, and audit must always pass. One failure in one hundred cannot hide behind an average.
- Probabilistic quality: task success, grounding, safety, language quality, latency, and cost vary. Fixed datasets, repeated sampling, and statistical comparison are appropriate.
- Human and business outcome: did the user solve the problem, was approval understandable, could a human take over, and did rollback work?
Model evaluation cannot replace platform testing. Platform testing cannot prove that an answer is useful. A mature release gate keeps both forms of evidence instead of blending everything into “82 points.”
4. Thirty Gates Organized by Failure Boundary
The lab contains six categories with five checks each.
Figure 3: Original diagram. The categories follow failure boundaries: who acts, where execution may run, whether it may happen, whether it happens once, whether context remains true, and whether the result can be proven.
Identity: Never Drop the Passport at a Handoff
The identity category verifies that the envelope contains tenant, user, role, and request ID; a missing tenant is rejected; cross-tenant records remain hidden; expired delegation fails; and one request identity crosses entry, gateway, Worker, and audit.
Figure 4: Real lab-output capture. Putting a username in a prompt is not identity propagation. Policy needs verified structured claims that the model cannot rewrite.
5. Gateway: 429 May Fall Back; 401 Must Stop
The gateway category does more than check whether a backup model can answer:
- synthetic 429 is a transient capacity error eligible for bounded fallback;
- 401 means credential or authentication failure and must stop;
- the retry budget permits at most one fallback;
- an estimated request above remaining budget is blocked before invocation;
- candidates outside the allowed data region are filtered before routing.
Figure 5: Real lab-output capture. Fallback is not “try another model after any failure.” Error class, capability, data policy, budget, and attempt count all constrain the decision.
6. Policy: The Model May Propose; It Cannot Approve Itself
The policy checks allow a read-only tool, hold a high-risk write for approval, stop prompt injection before model and tool execution, reject an extra shell argument outside the tool schema, and bind approval to normalized arguments so a changed target invalidates the old approval.
Figure 6: Real lab-output capture. “The user agreed” in model prose is not approval evidence. Approval should bind tool, target, normalized arguments, versions, requester, approver, expiry, and a one-time nonce.
Prompt-injection evaluation should not stop at ten hostile phrases and ten model refusals. The meaningful questions are whether hostile content changed tool selection, arguments, permission, secret access, audit fields, or side effects. Deterministic platform policy must remain independent of model cooperation.
7. Execution: The Queue May Deliver Twice; the Business Effect May Not
Execution checks validate state transitions, lease-expiry redelivery, an idempotency uniqueness constraint, fencing of a stale Worker, and cancellation before a new effect starts.
Figure 7: Real lab-output capture. SQLite receives the same delivery twice under one unique key and stores one effect. This validates the control idea; it does not pretend that end-to-end exactly-once is trivial.
Separate “messages delivered” from “business effects executed.” A courier may scan one parcel twice after a network problem; the warehouse may store both scan events but must ship one box. Production metrics should expose redelivery count and duplicate-side-effect count separately. The latter must remain zero.
8. Memory: An Old Fact Can Be More Dangerous Than No Fact
Memory checks ensure that expired records, another tenant’s records, untrusted retrieved instructions, superseded versions, and deleted records remain outside context.
Figure 8: Real lab-output capture. Vector similarity is not one of these five gates because it only orders candidates. Deterministic lifecycle policy decides whether a candidate may be used.
For the full lifecycle model behind this test, read the companion article on structured AI memory.
9. Evidence: Final Prose Is Not an Incident Record
The evidence category requires model and tool spans to share one trace, audit records to contain who acted, which tool was considered, what decision was made, and why; estimated and measured cost reconcile; a correct denial is not classified as outage; and synthetic recovery completes inside its SLO.
Figure 9: Original diagram. Logs in every service do not equal observability. If request, trace, task, and audit identifiers cannot be joined, an incident remains a jigsaw puzzle.
Figure 10: Real lab-output capture. A policy denial is classified as correct-control, not outage. Otherwise, teams are rewarded for weakening safety to improve a success percentage.
10. Why “Success Rate” Is So Easy to Misuse
Suppose a day contains ten requests: seven complete, one dangerous request is correctly denied, one waits for human approval, and one suffers an actual provider failure. A crude success rate says 70%, mixing three healthy or unhealthy states together.
Figure 11: Original diagram. Denial proves the brake works. Waiting approval proves the task is durably held without an effect. Failure means the intended business outcome was unexpectedly lost.
A better dashboard separates business completion, correct and incorrect policy denial, approval wait time, technical outage, recovery time, duplicate effects, and cost per completed task. That prevents a team from sacrificing safety to make one percentage green.
11. Release Flow: Grow Blast Radius Only After Each Stage Passes
A production release should not jump from a local prompt edit to 100% traffic:
- statically inspect schemas, secrets, dependencies, and configuration;
- run a fixed golden task set repeatedly for probabilistic quality;
- inject throttling, timeout, hostile content, queue redelivery, Worker crash, and stale memory;
- shadow production traffic without executing effects;
- canary a small percentage with automatic rollback thresholds;
- monitor business, cost, safety, and recovery SLOs after full release.
Figure 12: Original diagram. Any blocked gate means stop, explain, fix, and rerun. Deleting a failing test is not a release strategy.
The gate itself needs versions. A change to the model, prompt, tool schema, policy, retrieval corpus, route, or evaluation set must answer which evidence belongs to that exact combination. Yesterday’s PASS does not certify tomorrow’s bundle.
12. Reproducible Lab: 30/30 Required for RELEASE=PASS
The downloadable release-gate lab uses a fixed clock, synthetic identities, and local SQLite. It never calls a model, tool, or network. The code executes thirty boolean assertions; it does not hard-code a page of PASS text.
Figure 13: Real lab-output capture. Thirty of thirty means every teaching assertion passed. A real release still needs model quality, business outcome, capacity, security, compliance, and rollback evidence.
One-click run on Windows 11
Download the lab and Windows launcher into one directory:
powershell -ExecutionPolicy Bypass -File .\run_windows.ps1
One-click run on Ubuntu 26.04
Download the Ubuntu launcher beside the lab:
chmod +x ./run_ubuntu.sh
./run_ubuntu.sh
One-click run on macOS 26
Download the macOS launcher beside the lab:
chmod +x ./run_macos.sh
./run_macos.sh
For manual verification, run python3 release_gate_lab.py --clean, then inspect six category reports and the final JSON instead of reading only the last terminal line. For agent-assisted acceptance, provide the downloadable agent task contract. It forbids networking, package installation, and weakened gates; requires total=30, passed_count=30, and release=PASS; and stops on the first failure with the exact category and evidence.
13. Connecting the Gates to Real CI/CD
The lab is a single-machine teaching model. A production integration should follow four principles:
- Hard-block deterministic failures. Cross-tenant exposure, duplicate effects, unapproved high-risk actions, and missing audit cannot be waived to make a deadline.
- Compare probabilistic distributions. Repeat the same version and compare task success, quality, latency, and cost rather than trusting one lucky sample.
- Give critical paths independent thresholds. Average chat quality cannot compensate for a failure in payment, deletion, or production control.
- Bind evidence to versions. Record model, prompt, policy, tool schema, dataset, code commit, and environment summary.
CI can run fast standard-library gates while scheduled jobs perform more expensive model evaluation. The release system consumes both and adds real traffic during shadow and canary stages. Production SLO violations pause rollout or trigger rollback before users become the fault-injection system.
14. Q&A
Are these thirty checks an industry standard?
No. They are an executable teaching baseline across six boundaries. Healthcare, finance, IoT, and code execution need additional domain-specific gates.
After a model upgrade, may I rerun only answer evaluation?
No. A new model can change tool choice, argument shape, token use, latency, and refusal behavior. Policy, routing, tools, and cost gates need another run.
Why is a policy denial not a failure?
It achieved the safety objective. Track correct denial and false denial separately. Counting all denial as failure incentivizes unsafe authorization.
Can shadow traffic validate tools?
It can validate plans and policy, but real side effects should be blocked or replaced with sandbox/read-only adapters. Testing must not duplicate production writes.
Does a golden set cause overfitting?
It can. Keep hidden tests, add production failures, use adversarial and metamorphic variants, and combine offline evidence with canary outcomes. A golden set is a health check, not an immunity certificate.
Does 30/30 guarantee that I may ship?
No. It proves those thirty deterministic assertions only. A release also needs model quality, capacity, privacy, legal, dependency, rollback, and business-owner evidence. The value of a gate is stating what has and has not been proven.
15. Closing
The most dangerous AI Platform release is one that treats a handful of beautiful answers as proof that the entire system is healthy. Models vary, providers throttle, hostile content appears, Workers crash, queues redeliver, and memory expires. Release engineering should make those failures happen deliberately before users do.
This is the acceptance companion to the AI Platform production architecture. The architecture reading map connects tool calling, gateway, observability, and security, while the other deep dives cover durable Workers, Turn Manager, AI Gateway, and structured memory.
References: