Two security gates separate an AI demo from controlled production

An AI demo proves that a model can produce a useful result under selected conditions. Controlled production requires evidence for the data path, users, actions, failures, and operating owner of one exact release.

Compsia uses two security gates. Gate A must pass before real customer data enters a new system path. Gate B must pass before that path can run repeatedly and cause consequential actions. The distinction prevents a polished prototype from quietly inheriting production authority it was never designed to hold.

This AI production security checklist supplies evidence for named people to examine before they choose to release, narrow, pause, or stop.

A demonstration answers the smallest question

I like prototypes. They expose whether a model understands the material, whether an interface makes sense, and whether the proposed work could be useful. Trouble begins when a prototype is asked to prove something it never tested.

A demonstration normally uses a controlled sample, a cooperative user, and a happy path. The team can repair mistakes by hand. Credentials may belong to the builder. Nobody has yet tested a revoked permission, a missing source, a duplicate event, a model timeout, or an approver who no longer owns the project.

Those omissions define the prototype's stage.

The NIST Generative AI Profile places strong emphasis on pre-deployment testing, governance, provenance, and incident disclosure. The useful translation for a delivery team is simple: capability evidence and production evidence answer different questions.

Gate A authorizes one real data path

Gate A asks whether the system may process real customer data through the proposed path. It applies before a connector, upload, retrieval task, model request, log, or approval view receives that data.

The release owner should be able to produce evidence for each of these areas:

1. The data categories, purpose, authority, and permitted users are written down.
2. The complete path names sources, destinations, model providers, infrastructure providers, regions, and subprocessors.
3. Appropriate agreements and provider records exist for the path.
4. Test accounts prove the access boundary, including denied access.
5. Retention, export, return, and deletion behavior are defined and tested where required.
6. No unapproved direct write can occur through the path.
7. An incident owner and usable response procedure exist.

Before Gate A, use fictional or genuinely de-identified material. Removing a person's name while leaving client, venue, date, commercial detail, and contact history may still leave the record identifiable. De-identification is a decision about the resulting information, not a find-and-replace operation.

Gate A is system-specific. Evidence for a meeting-transcript summary covers that path alone; a new CRM write path or second model provider returns to review.

Gate B authorizes recurring action

Gate B begins after Gate A. It applies when the system runs on a schedule or event and can affect another system, person, commitment, or record.

The action policy should begin with denial. Each permitted action is then added deliberately, with a named identity and destination. Material actions that require review must stop at an authenticated approval bound to the exact payload. A generic “approve” button is weak evidence if the recipient, amount, project, or changed fields are hidden.

Gate B also requires behavior that ordinary happy-path testing tends to miss:

The system also needs monitoring, rollback, a manual fallback, and production-like tests for the exact release. Human approval governs a proposed action. The agent's upstream access requires separate authorization.

Apply both gates to an event project-change brief

Consider a hypothetical event company that wants a daily brief of changes affecting live client programs. The brief reads the project system, approved meeting records, and selected client communication. It highlights changed dates, supplier dependencies, and unresolved decisions. A production manager may approve a supplier follow-up draft.

The demo question is whether the system can make a useful brief from sample records.

Gate A asks which live projects it may read, how a user is associated with those projects, which transcript fields reach the model, where traces are stored, and how a project is removed from the index. It also tests whether a manager from Project A can retrieve Project B material.

Gate B begins when the brief runs every morning and can prepare an external message. The team tests duplicate calendar events, a cancelled project, conflicting dates, a missing connector, a stale approver, and a repeated approval. The supplier email remains a draft until the correct project owner sees the recipient and body and approves that exact action.

No customer result is implied by this example. It illustrates the evidence needed to release the path.

A failed gate should narrow the release

Security reviews are often treated as a binary verdict on the whole initiative. That creates pressure to explain away gaps because “no” appears to kill the project.

A narrower release is usually available.

If deletion cannot yet be verified across derived stores, keep real customer data out and continue with fictional material. If retrieval permissions are incomplete, restrict the source to a dedicated test folder. If authenticated approval is not ready, produce an internal draft with no sending tool. If replay behavior is uncertain, run the path manually while the team collects evidence.

The question becomes “Which smaller perimeter can we support now?” That keeps progress honest and gives the next engineering decision a clear target.

Keep the evidence attached to the version

At Skybridge, development and production are separate environments. Development deploys automatically after tests. Production promotion remains a deliberate action. Releases map to exact code revisions, and rollback is part of the deployment path.

That separation matters because Gate A and Gate B evidence is bound to a version and configuration. A new connector, provider, permission model, or action class can change the accepted boundary.

The release record should therefore name the system result, data path, users, providers, actions, tests, fallback, approvers, and remaining limits. NIST's Secure Software Development Framework offers a common vocabulary for integrating security practices into development and supplier conversations. AI-specific release evidence belongs beside that software discipline.

Compsia designs each production system around a bounded result and keeps the security statement attached to that path. If you are choosing a first event operation, the production perimeter worksheet is the right place to begin.

Production-readiness questions

Can a prototype use real customer data?

Only after the real data path has its own authorization, agreements, access tests, retention rules, and incident ownership. Prototype status does not create an exception to privacy or security duties.

Does every AI action need human approval?

No. The policy should reflect consequence and evidence. Low-risk, reversible actions inside a narrow allowlist may be permitted. Consequential or uncertain actions often require authenticated approval. Some actions should remain prohibited.

Who approves an AI system for production?

The exact roles depend on the company and use case. At minimum, a business owner and technical owner need clear accountability. Security, privacy, legal, data, and operational owners participate where the path requires their authority.