Hardware-to-Model Matchmaker
Pick your GPU — or enter your VRAM — and estimate which open-weight models could fit, at which quantization, with estimated headroom. These are self-hostable candidates. Sovereignty requires deployment evidence for identity, network access, telemetry, artifacts and recovery.
Hardware and workload
Models that fit your hardware
Best (least lossy) quantization that fits is shown. Larger models are listed first.
| Model | License | Context | Quantization | Needs | Headroom | Runtimes |
|---|---|---|---|---|---|---|
Gemma 2 27B Legacy comparison candidate; recheck source card | Gemma Terms of Use | 8K | Q4 | 22 GB | 8% | Ollama · vLLM |
StarCoder2 15B Legacy comparison candidate; recheck source card | BigCode OpenRAIL-M | 16K | INT8 | 22 GB | 9% | Ollama · vLLM |
Code Llama 13B Legacy comparison candidate; recheck source card | Llama 2 Community License | 16K | INT8 | 20 GB | 19% | Ollama · vLLM |
Mistral Nemo 12B Legacy comparison candidate; recheck source card Long-context document intelligence at mid size. | Apache-2.0 | 128K | INT8 | 19 GB | 24% | Ollama · vLLM |
Gemma 2 9B Legacy comparison candidate; recheck source card Strong reasoning; short 8K context. | Gemma Terms of Use | 8K | INT8 | 15 GB | 39% | Ollama · vLLM |
Qwen3-8B Model card: reviewed 2026-10-20Native 32K context; extended context is not assumed. Quantized artifact and runtime support require validation. | Apache-2.0 | 32K | FP16 | 24 GB | 4% | Compatible local runtime; verify release |
Llama 3.1 8B Legacy comparison candidate; recheck source card Established comparison baseline; benchmark against newer candidates on the same workload. | Llama 3.1 Community License | 128K | FP16 | 23 GB | 6% | Ollama · vLLM · OpenShift AI |
Mistral 7B Legacy comparison candidate; recheck source card Efficient permissively-licensed baseline. | Apache-2.0 | 32K | FP16 | 21 GB | 15% | Ollama · vLLM · Docker |
Qwen 2.5 7B Legacy comparison candidate; recheck source card | Apache-2.0 | 128K | FP16 | 21 GB | 15% | Ollama · vLLM |
Phi-3.5 Mini (3.8B) Legacy comparison candidate; recheck source card Runs on laptops and edge devices; excellent quality-to-size ratio. | MIT | 128K | FP16 | 13 GB | 46% | Ollama · Edge · Docker |
Needs more memory
Minimum requirement shown at the smallest common quantization with your context and concurrency.
Qwen3-Coder-30B-A3B-Instruct
Needs ≈ 25 GB at Q4
DeepSeek R1 Distill 32B
Needs ≈ 26 GB at Q4
Mixtral 8×7B
Needs ≈ 36 GB at Q4
Llama 3.1 70B
Needs ≈ 51 GB at Q4
Qwen 2.5 72B
Needs ≈ 52 GB at Q4
Llama 3.1 405B
Needs ≈ 274 GB at Q4
DeepSeek R1 (671B MoE)
Needs ≈ 451 GB at Q4
Don't own the hardware yet? Rent it by the hour first.
Before buying GPUs, validate your model choice on rented hardware. Run your actual workload for a few dollars, measure tokens/sec, then size your on-prem purchase with real numbers.
Email me these results + sovereign AI deployment notes
Get your hardware match summary and occasional practical notes on self-hosting LLMs, private RAG, and air-gapped deployment. No spam.
Turn a fit estimate into a deployment project
Save your own GPU, model and workload settings. Review the architecture-aware capacity estimate, then download a versioned vLLM pilot package for a supported model.
Open deployment workspaceHardware Sizing Guide
Work the estimate the other way: start from a model class and size the hardware you need.
Model Registry
Sovereignty tiers, deployment runtimes, and a decision guide for private AI model selection.
Planning a real deployment?
Get a fixed-scope architecture review: model selection, hardware sizing, runtime topology, and a written findings document for your private RAG or air-gapped AI platform.
Estimates are approximate. Actual memory depends on runtime, batching, quantization format, KV cache implementation, and model architecture. Always benchmark before purchasing hardware.