Model lifecycle workshop

Scenario

Evaluate two candidate open-weight models for a synthetic internal summarization workload. No candidate is assumed acceptable.

Tasks

1. Define intended purpose, prohibited uses, workload distribution, quality, safety, privacy, latency, capacity, and portability criteria.
2. Create representative, boundary, adversarial, and failure cases before reviewing results.
3. Record candidate provenance, license evidence, model and tokenizer identities, runtime, hardware, and test conditions.
4. Compare results and document severe failures separately from averages.
5. Select, reject, or hold each candidate with rationale and limitations.
6. Build the approved package and registry record.
7. Design the promotion gate, monitoring signals, rollback triggers, update comparison, and retirement process.

Deliverables

Submit the requirements, evaluation plan and results, license/provenance record, model decision record, package manifest, registry entry, approval, and lifecycle plan.

The review should be reproducible: another reviewer must be able to identify exactly what was evaluated and why the decision applied only to the stated context.