SAI-220 knowledge check

How to use this assessment

Complete the questions without referring to the chapter text. Then review the guidance and update your runtime decision where your answer exposes a gap.

This public knowledge check supports learning. It does not certify a runtime, accelerator, or benchmark result.

Scenario questions

1. File versus service

A team says their model is "deployed" once the weight file is copied onto a server. Explain what is missing before that is actually a model service.

2. Stable endpoint contract

A team plans to switch inference runtimes and expects every client to keep working unchanged. Which endpoint details need to stay stable for that to hold?

3. Context-scoped routing approval

A router destination was approved for one workload. Explain why that approval does not automatically extend to a different context.

4. Throughput without conditions

A vendor reports "500 tokens per second" with no other detail. Explain why this number cannot be used for capacity planning as given.

5. Healthy process, unhealthy service

A load balancer reports every replica as healthy, yet users are receiving degraded or incorrect responses. Explain how both can be true.

6. Change-triggered re-evaluation

A team upgrades only the GPU driver and firmware, with no changes to the model or application code. Explain why this alone can still require re-evaluation.

7. Headroom beyond average

A system is sized to handle exactly the average observed demand. Explain why this sizing is risky.

8. External fallback risk

When local capacity is exhausted, a system automatically and silently routes requests to a public external endpoint. Explain the sovereignty problem with this behavior.

Answer guidance

1. A model file is only weights; a service also requires runtime loading, accelerator scheduling, API translation, authentication and authorization, rate and quota enforcement, routing, policy inspection, and telemetry — responsibilities that must exist even when one product combines them.
2. Accepted request shapes, size and context limits, streaming behavior, timeout and cancellation semantics, response structure, error and refusal codes, and version negotiation should stay stable, so a runtime change doesn't silently break every client.
3. A router may need to deny or restrict a destination based on workload approval, data classification, region, tenant, or health for a specific context — approval for one context does not establish approval for another.
4. Throughput depends on the declared model, tokenizer, precision, runtime, hardware, context distribution, concurrency, batching, and quality settings it was measured under; the same hardware can produce very different numbers under different conditions.
5. A process can be alive and responding while the wrong model is loaded, requests are unauthenticated or unauthorized, or output quality has degraded — liveness alone does not confirm the intended model is loaded, responsive, authorized, and producing acceptable behavior.
6. Runtime, driver, firmware, quantization, model, routing, configuration, and hardware changes can all alter served behavior or capacity, so each should trigger evaluation under representative load before being promoted.
7. Sizing to average demand leaves no margin for failures, maintenance windows, model loading time, or demand variance — the conditions that actually cause outages occur above the average, not at it.
8. Silently falling back to a less-controlled or external endpoint moves data and requests outside the approved boundary without the review, evaluation, and authorization the primary path received — a fallback destination needs its own independent approval, not an automatic bypass.

Completion rubric

Mark the runtime decision complete only when:

• Workload assumptions and endpoint behavior are explicit and stable across runtimes.
• Responsibilities such as authentication, routing, policy, and telemetry are named even where a product combines them.
• Routes and destinations are approved per context, not assumed globally.
• Capacity measurements state their exact model, hardware, and load conditions.
• Hardware constraints, isolation, and degradation behavior are documented.
• Recovery, observability, release gates, and rollback are demonstrated, not assumed.

Completion outcome

SAI-220 is complete when the learner can explain why a live process is not proof of a healthy model service, and can produce the runtime decision package from the Model-serving design workshop.