Model monitoring, updates, and retirement

Monitor fitness, not personality

Monitor the properties connected to the approved purpose: workload success, supportedness, refusal and escalation, policy outcomes, latency and capacity, security findings, license or supplier changes, and user-reported harm. Protect sensitive prompts and responses while retaining enough structured evidence to investigate.

Update triggers

Reassess after new weights, tokenizer, quantization, prompt, runtime, hardware, retrieval, policy, deployment boundary, user population, data class, security finding, incident, or material drift. Use the controlled-change model to determine evaluation depth.

Compare the candidate with the currently approved version using the same baseline plus new regression cases. Record gains, regressions, migration needs, exception decisions, and rollback conditions.

Retirement

Define an end-of-use date, replacement or shutdown plan, owner communication, deployment discovery, approval revocation, credential changes, artifact retention, evidence retention, and deletion obligations. Verify that routing aliases, caches, batch jobs, and hidden endpoints cannot still invoke the retired model.

A model may remain retained for investigation or reproducibility while being prohibited from active use. Make those states distinct.