GPU, model, and runtime operations

Manage the compatibility set

Accelerator hardware, firmware, host driver, device plugin or operator, container runtime, libraries, inference runtime, model format, quantization, and serving configuration form a tested compatibility set. Change one component only with impact analysis and validation.

Scheduling and isolation

Define resource discovery, node labels and taints, quotas, priorities, sharing mode, topology, memory behavior, health checks, drain, maintenance, and failure replacement. Consider cross-workload leakage, denial of service, noisy neighbors, and residual accelerator memory.

Model operation

Load models only from approved internal stores. Verify identity before use, constrain download and cache paths, record which version is resident, and prevent unauthorized alias changes. Plan load time, storage bandwidth, warm capacity, eviction, and corrupted-cache recovery.

Health and evidence

Monitor hardware and driver errors, memory pressure, throttling, runtime load failures, queueing, model identity, request behavior, and operator changes. A device being allocatable is not proof that the approved model service works correctly.

Test node loss, accelerator fault, model corruption, runtime rollback, and re-scheduling without uncontrolled external dependency.