Interactive guide / Architecture & procurement
Choose an AI model you can operate, evaluate, and afford. Build a shortlist around your data boundary, task, runtime, and cost per accepted outcome, then validate it on your own workload.
Reviewed · Vendor-neutral guidance
Your deployment choice is a constraint. Missing hardware prompts a feasibility review; it never silently changes an on-premises requirement to cloud hosting. Inputs stay in this page and are not submitted.
Candidate path; validate feasibility
Compare a compact instruction model with a stronger candidate using the same authorized retrieval and citation tests.
Benchmark a compatible vLLM or equivalent serving release on the specified GPUs; confirm exact model, quantization, and feature support.
A curated starting set, not a benchmark ranking or an exhaustive catalog. Sources reviewed 2026-10-29. Names below identify the actual repository or service model. Hardware fit, licensing approval and quality remain pilot decisions.
Context planning assumption: 8,192 tokens including prompt, retrieved content, tool history and generated output. Change the context need above for longer tasks.
Self-hosted candidate
Qwen/Qwen3-8B
General instruction and tool-use baseline for RAG, summarization and multilingual evaluation.
8.2B parameters. Budget thinking tokens and test the runtime's tool parser. Native context only; extensions are not assumed.
Context limit used: 32,768 tokens. Advertised capacity does not establish recall quality or local memory fit.
Self-hosted candidate
Qwen/Qwen3.5-9B
Text-and-image candidate for document understanding, long-input evaluation and visual question answering.
Hybrid architecture: measure recurrent state, attention cache and image-encoder memory in a supported runtime. It is not a general sensor classifier or an automatic memory fit.
Context limit used: 262,144 tokens. Advertised capacity does not establish recall quality or local memory fit.
Baseline to beat: Compare with your current workflow using the same holdout tasks, quality gates and accepted-outcome cost.
For RAG, select and evaluate the embedding model and optional reranker separately. For agents, measure the entire tool workflow. The generator shortlist alone does not specify the complete system.
Browse additional model families01 / Decision framework
A high benchmark score cannot compensate for an unacceptable data path or an unsupported production runtime. Treat these as pass/fail gates before comparing quality.
Map inference, embeddings, retrieval, telemetry, backups, support access, and updates. On-premises hosting alone does not establish sovereignty; document who can access each component and how that access is enforced.
Record the exact model repository and revision, tokenizer, chat template, weight format, license, and acceptable-use terms. Check commercial use and redistribution for your intended deployment. Open weights do not automatically mean unrestricted use.
Check the selected architecture, quantization, accelerator, tool-call parser, and multimodal support against the pinned serving release. A compatible API shape does not guarantee identical behavior across providers or runtimes.
Name the owner for patching, access review, capacity, incidents, model replacement, and rollback. Include support and evaluation effort in the comparison with a managed service.
02 / Decision framework
Compare a simple baseline, a cost-conscious candidate, and a quality-focused candidate within the approved boundary. Keep the dataset and acceptance rules identical.
| Workload | Starting shortlist | Evidence to collect |
|---|---|---|
| Document RAG | Retriever + compact instruction model; larger model only if failures justify it. | Authorized retrieval, claim support, citation coverage, abstention, and cost per accepted answer. |
| Extraction / classification | Rules or a trained classifier alongside a small language model. | Per-field accuracy, schema validity, rare-class recall, and human correction effort. |
| Code assistance | Code-capable model plus permission-aware repository retrieval. | Held-out executable tests, insecure changes, license constraints, and sandbox behavior. |
| Agents and tools | Tool-capable model behind explicit tool permissions and approval rules. | Completed authorized tasks, denied actions, loop limits, recovery, and total task cost. |
| Vision / sensor | Task-specific vision or signal model; multimodal model where its flexibility is needed. | Device-level latency, false alarms, missed events, energy, and field robustness. |
Current example: Qwen3.5-9B combines vision and language with hybrid attention. It is a candidate to test, not a universal recommendation. Its memory behavior differs from a conventional full-attention 9B model; use its model card and the serving runtime's measurements.
03 / Decision framework
Agree the decision rules before running the pilot so that a successful demo does not become the only acceptance test.
Freeze a representative holdout set, including languages, long inputs, missing evidence, and unauthorized requests. Keep development examples separate.
Set quality and security gates first. Measure p95 time to first token, end-to-end latency, failure rate, and completed work under realistic simultaneous demand.
Test the exact quantized artifact and decoding settings you will deploy. Include reasoning output, retries, tool calls, and evaluator costs in the budget.
Record a primary candidate, an approved fallback, rejected alternatives, and the conditions that trigger reassessment. Re-run the suite after model, prompt, or runtime changes.
Advertised context length is a technical limit, not evidence of reliable recall across that entire length. Test the locations and document lengths that matter to your users.
Working materials
Download the editable materials, adapt them to your environment, and keep the assumptions alongside the results.
Work with SovAIHub
Scope a model selection assessment around one priority workload and its real deployment constraints.
Share the use case, permitted data locations, sample task shapes, languages, expected demand, available hardware, and the person who accepts the result.
Scope, acceptance criteria, delivery schedule, and fees are agreed before implementation.
Request a model selection reviewConnect the decision to the next engineering or operating step.
Official documentation checked for this revision. Confirm the exact model, runtime, and license versions when implementing.