Evaluation and release gates
Turn acceptance criteria into repeatable promotion decisions with recorded evidence and rollback conditions.
A release gate is a decision system
An AI release gate turns requirements into a repeatable allow, reject, hold, or exception decision. It applies to models, prompts, retrieval pipelines, policies, runtimes, tools, infrastructure, and complete workload releases.
A gate is more than a test suite. It defines the candidate, acceptance criteria, evidence, decision authority, exception route, promotion destination, and rollback conditions.
Build the gate contract
For each gate, specify:
- Candidate identity: the immutable versions and configuration being evaluated.
- Approved purpose: workload, users, data classes, deployment boundary, and prohibited uses.
- Evaluation set: representative, boundary, adversarial, permission, failure, and regression cases.
- Measures: quality, safety, security, latency, capacity, reliability, cost, and policy outcomes that matter.
- Thresholds: pass, fail, warning, and manual-review criteria with justified tolerances.
- Decision authority: who can approve, reject, or accept an exception.
- Evidence: inputs, environment, results, logs, reviewer notes, decision, and integrity information.
- Aftercare: monitoring signals, rollback trigger, expiry, and required reassessment events.
Avoid metric theatre
A high aggregate score can hide an unacceptable failure for a specific user group, permission case, language, content type, or high-impact task. Keep disaggregated results and inspect failure severity as well as frequency.
Do not reuse a pass outside its evaluated context. A new prompt, knowledge corpus, quantization, runtime, policy, tool, hardware profile, or deployment boundary can change behavior and may require partial or complete reassessment.
Exceptions
An exception should identify the failed criterion, rationale, compensating controls, owner, affected scope, expiry date, monitoring, and closure evidence. Exceptions must not silently redefine the acceptance threshold.
Release evidence
The final record should allow an independent reviewer to reproduce the decision logic: what was tested, under which conditions, what failed, who decided, what was promoted, and how it can be rolled back.