AI Agents / DevOps / Production

AI Agent Production Readiness Checklist: A Definition of Done With Evidence

A process can start, a port can listen, CI can pass, a push can succeed, and a URL can return 200 while the user still cannot complete the intended task. This checklist turns production readiness into eight evidence gates, five explicit states, and one verdict that refuses to guess.

August 13, 2026 · About 9 minutes · Free Markdown template

Five green lights that do not prove production

This is not pedantry. Google SRE describes implicit errors such as an HTTP 200 response containing the wrong content; end-to-end checks may be the only way to detect them. On this site, a sitemap once returned 200 while serving the React homepage instead of XML.

The eight production evidence gates

1. Artifact identity: prove that the local revision, remote SHA, deployment revision or image digest, and running artifact identify the same release.
2. Control plane: verify that CI, cloud deployment, environment settings, schedulers, and queues point to this release in the intended environment.
3. Runtime readiness: confirm process survival, listener ownership, initialization, required dependencies, and readiness to accept real work.
4. Public HTTP and network: check DNS, TLS, redirects, status, content type, a release-specific body marker, and authorization from outside the local machine.
5. User journey and UI: use a real browser to finish the core task. Check mobile layout, console errors, network failures, and fallback pages.
6. Data and side effects: verify migrations, retries, webhooks, queues, notifications, and writes without duplicates, loss, or cross-environment leakage.
7. Observability: compare a baseline and observation window for latency, traffic, errors, saturation, and the actual business outcome.
8. Rollback and recovery: identify the known-good artifact, trigger conditions, data compatibility, and the checks that prove recovery after rollback.

Liveness is not readiness

Kubernetes separates startup, liveness, and readiness probes. A startup probe asks whether initialization has completed. A liveness probe asks whether the container should be restarted. A readiness probe asks whether it should receive traffic. A failed readiness probe removes the Pod from matching Service endpoints while the container can continue running.

The model remains useful outside Kubernetes: alive, started, and able to serve are different questions. If an application needs a database to complete its job, an open TCP port alone does not prove readiness.

Use evidence states that preserve uncertainty

WAITING and NOT_MEASURED prevent false certainty. A search platform processing a sitemap is not a failure. Missing analytics access is not zero traffic, and it is not evidence of growth. The final report must state every unmeasured surface.

How to use the checklist with an AI coding agent

Download the template into a repository's docs or operations directory, or paste it into the task context. Define the release contract first. Then require the agent to attach commands, URLs, SHAs, timestamps, screenshots, logs, or readbacks for every passing gate.

Validate this release against the Production Definition of Done.

Rules:
- A process, port, HTTP 200, green CI, or successful push is never enough alone.
- Every PASS must include evidence from this run.
- Missing access is NOT_MEASURED, never inferred.
- External mutations must stay inside Authorized Mutations.
- End with PASS, FAIL, PARTIAL, or WAITING and list residual risk.

The full template covers release identity, control plane, runtime, HTTP, UI, side effects, observability, rollback, and the final verdict. Use the AI Agent Production Readiness Checklist Generator or download the Markdown file. Neither requires an account or email address.

What evidence changes by product type?

Static site or single-page application

Match the remote SHA to the deployment revision. Verify public content type, release marker, canonical, robots, sitemap, desktop rendering, and mobile rendering. A server must not return the application shell for a missing XML or content URL and call it success; see the Cloudflare Workers React SPA SEO case study for a production example.

API or long-running service

Verify that the process survives and the expected process owns the listener. Readiness should cover required dependencies. Test positive and negative API paths, authorization, payment boundaries, and data isolation. Localhost alone is not public evidence.

Scheduler, AI agent, or media pipeline

An enabled scheduler proves only that the schedule is enabled. Inspect the latest run, logs, output artifact, checksum, and destination readback. The observation window must cover a meaningful work unit: a thirty-second check cannot validate a twenty-minute rendering job.

Monitoring is more than having a dashboard

Google SRE recommends focusing on latency, traffic, errors, and saturation. For a canary release, compare the canary with a control separately; a small cohort with a high error rate can disappear inside a healthy site-wide average.

This checklist does not force every static site to build an SRE platform. A low-risk page can use public HTTP checks, a real browser, console inspection, and a traffic baseline. Payments, personal data, migrations, or high-volume background jobs require stronger observability and a rehearsed rollback.

Authority is part of the Definition of Done

A production checklist does not grant an agent unlimited permission. Put allowed actions and forbidden actions in the release contract. Stop when the next step requires new authority.

• Do not create charges, refunds, purchases, or paid resources without explicit approval.
• Do not delete production data, storage, databases, or run irreversible migrations without explicit approval.
• Do not change IAM, OAuth scopes, account roles, or public access without explicit approval.
• Do not publish posts, send messages, submit sitemaps, or request indexing without explicit approval.

A compact release contract

Before execution, record the intended environment, exact artifact, core user outcome, observation window, authorized mutations, forbidden mutations, rollback target, and the person who makes the final go or no-go decision. This makes the evidence comparable and prevents the scope from changing after a failure appears.