Quick Summary (TL;DR):
For a useful local Ollama home server, choose the model and context length first, then buy hardware. GPU VRAM is usually the limiting factor for responsive inference: 8GB can be a sensible entry point for small quantised models; 12–16GB provides more room for an 8B model and some 14B workloads; and 24GB makes larger quantised models substantially more practical. These are planning tiers, not guaranteed model-fit or speed benchmarks. A typical single-user machine can start with 32GB system RAM and a 500GB–1TB NVMe SSD, while larger models, CPU offload and concurrent users justify more. Measure whole-system idle and loaded power before estimating 24/7 electricity cost. If you only need occasional experiments, test Ollama on hardware you already own before buying a high-power GPU. All model sizes and support notes below refer to manufacturer/project information checked in October 2026, not hands-on testing.
The Short Answer: Match the Build to the Model, Not the Other Way Round
A home AI server is not the same as a general-purpose home lab. Plex needs media decoding, Home Assistant needs predictable low-power uptime, and a local language model needs enough fast memory to hold its weights and working context. An Intel N100PriceAliExpress price or N150 mini PCPriceAliExpress price can run small models on its CPU, but that does not turn it into a substitute for a workstation with a supported discrete GPU. Conversely, a powerful gaming PC that idles all day may be an expensive way to serve two prompts every evening.
Ollama supports local models and provides a local API, but model selection changes the hardware requirement dramatically. For a quick trial, start with a small model such as qwen3:4b. For sustained personal chat or code assistance, consider an 8B-class model and a GPU with adequate VRAM. For longer context, multiple users or 30B-class models, design the complete machine around memory capacity, cooling and power rather than buying the cheapest advertised “AI PC”.
| Your main workload | Sensible starting hardware direction | Key limitation to check |
|---|---|---|
| Occasional experiments, short prompts | Existing laptop/mini PCPriceAliExpress price; 16–32GB RAM; CPU or supported integrated GPU | Latency can be high; not a speed recommendation |
| One user, 4B–8B quantised chat | Supported 8–12GB GPU; 32GB system RAM; NVMe SSD | Long context and GPU overhead can exceed the card |
| One user, 8B–14B or document workflows | 12–16GB GPU; 32–64GB RAM; SSD | Model quantisation and KV cache determine fit |
| 30B/32B-class quantised models | 24GB GPU or larger/multiple supported GPUs; 64GB RAM often sensible | Not every 30B variant fits or performs equally |
| Several users, agents, large context | Plan from measured simultaneous workloads; consider 24GB+ VRAM | Parallel requests multiply memory demand |
The recommendations above are deliberately conservative categories rather than invented tokens-per-second figures. CPU, memory bandwidth, model architecture, quantisation, context and software backend all affect performance. A single GPU that fits the model is usually simpler than spreading it across multiple cards.
What Actually Uses Memory in Ollama?
There are four separate memory consumers: model weights, the key/value (KV) cache for context, runtime/compute buffers, and other services or loaded models. A model download of 5.2GB is not a promise that a 6GB graphics card will run it entirely in VRAM. File size and live memory consumption are different quantities. Larger prompts and more simultaneous sessions raise memory use even when the model file does not change.
- Model weights: primarily determined by parameter count and quantisation (for example Q4 versus Q8).
- KV cache: grows with context length, architecture, precision and number of active sequences.
- Runtime buffers: depend on backend, batch settings and model implementation.
- Other GPU users: a desktop compositor, video workload or second AI model reduces available VRAM.
- CPU offload: allows some models to run with less GPU memory, but transfers and system RAM can limit responsiveness.
Ollama’s current context-length guidance states that its defaults are based on available VRAM and that increasing context requires more memory. Always inspect the context actually allocated rather than assuming a model’s maximum advertised window is automatically in use. A model claiming 128K or 256K context is not a guarantee that your home server can afford to run it at that length.
Real Ollama Model Sizes: What the Downloads Tell You
The following figures are the published Ollama library download sizes for specific Qwen3 tags checked on 11 October 2026. They are useful for estimating SSD space and relative weight size, not minimum GPU VRAM. Tags can be updated, so verify the tag and quantisation before purchasing hardware.
| Ollama tag | Library download size | Practical starting interpretation |
|---|---|---|
qwen3:1.7b | 1.4GB | Lightweight trial or constrained CPU-only system |
qwen3:4b | 2.5GB | Useful first local model; test latency and answer quality |
qwen3:8b | 5.2GB | Good 8GB/12GB GPU candidate at modest context, verify live fit |
qwen3:14b | 9.3GB | 12–16GB GPU candidate depending on context and overhead |
qwen3:30b | 19GB | 24GB class merits testing; context may push beyond VRAM |
qwen3:32b | 20GB | Large single-GPU candidate with limited headroom on 24GB |
The qwen3:30b and qwen3:32b entries should not be treated as interchangeable. The former is a mixture-of-experts model while the latter is dense; active compute, model behaviour and memory access patterns differ. Similarly, changing a tag from its default quantisation to an FP16 variant can multiply its storage and memory needs. Use the official Qwen3 model listing and its per-tag details, not a generic “8B requires 8GB” rule.
GPU VRAM: The Most Important Purchase Decision
For a responsive single-user LLM, first try to keep the complete active model and working context on the GPU. Ollama’s ollama ps reports whether a loaded model is 100% GPU, 100% CPU or split. A split is not automatically a failure, but it changes the performance expectation. Memory bandwidth also matters: two cards with the same VRAM may produce very different throughput. That is why a card’s advertised AI TOPS or gaming frame rate is not enough to choose it.
| VRAM class | Typical use | What can go wrong |
|---|---|---|
| 4–6GB | Small quantised 1B–4B models; experimentation | Even modest contexts and other GPU loads can be restrictive |
| 8GB | 4B models and selected 8B Q4 models | An 8B download around 5GB still needs runtime/context headroom |
| 12GB | Comfortable small-model headroom; selected 14B Q4 attempts | 14B fit depends heavily on context, KV cache and backend |
| 16GB | 8B/14B with more breathing room | Long context and concurrency still consume VRAM |
| 24GB | Selected 30B/32B quantised models; bigger 14B context | 19–20GB weights leave finite space for KV cache and buffers |
| 32GB+ | Larger models, longer context, heavier multi-service use | Power, cost, support and thermal constraints rise |
Do not buy a nominally 16GB card without checking the exact SKU: NVIDIA’s RTX 4060 Ti, for example, was sold in 8GB and 16GB variants. The difference matters more to a memory-bound AI workload than the product name alone. The RTX 3060 12GB and RTX 3090 24GB are also useful reference points, but their age, second-hand condition and electricity use may outweigh their capacity advantages in a new 24/7 build.
Which GPU Should You Consider? Verified Hardware Examples
This is a specification-based comparison, not a ranking from our own inference benchmarks. Availability, new-versus-used price and board-specific cooling vary by region and retailer. NVIDIA lists 12GB on the RTX 3060, 8GB or 16GB on the RTX 4060 Ti, and 24GB on the RTX 3090. These capacities can be attractive at different budgets, but confirm that the actual card is supported by the current Ollama backend and driver.
| GPU example | Manufacturer VRAM specification | Manufacturer board-power reference | Buying caution |
|---|---|---|---|
| GeForce RTX 3060 (12GB variant) | 12GB GDDR6 | 170W graphics card power | Check that listing is 12GB, not another similarly named product |
| GeForce RTX 4060 Ti (16GB variant) | 16GB GDDR6 | 165W or 160W total graphics power | 8GB variant exists; check exact SKU |
| GeForce RTX 3090 | 24GB GDDR6X | 350W graphics card power | Large cooler, power connectors and used-card condition matter |
These wattages are manufacturer board-power ratings, not measured idle draw, actual Ollama consumption or total server power. The RTX 3090 reference specification recommends a 750W system PSU in NVIDIA’s reference configuration; that does not mean the server draws 750W continuously. A quality power supply must still accommodate GPU transient demand, CPU, drives, fans and the chosen card’s connector requirements.
Newer NVIDIA 50-series models, supported AMD ROCm devices, Apple silicon and Vulkan-compatible GPUs are valid options, but support is not uniform across operating systems or exact devices. Ollama’s official hardware support list is the source of truth. As checked in October 2026, the documentation specifies NVIDIA compute capability 5.0+ and driver version 550+ (570+ for capability 5.0–6.2), ROCm v7 requirements for supported AMD configurations, Metal on Apple devices, and additional Vulkan support. Verify driver and backend before ordering.
NVIDIA, AMD, Intel and Apple Silicon: Which Platform Is Easiest?
| Platform | Main advantage | Important limitation |
|---|---|---|
| NVIDIA CUDA | Widely documented Ollama GPU path; mature container tooling | VRAM pricing, power, card clearance; driver version |
| AMD ROCm | Potentially attractive VRAM and platform choice | Supported GPU/OS matrix and ROCm version must match |
| AMD/Intel via Vulkan | Broader GPU coverage in current Ollama releases | Backend/driver differences; test your exact hardware |
| Apple silicon / Metal | Unified-memory machines and compact, quiet hardware | Memory is shared with OS; upgrade options limited |
| CPU only | No GPU purchase; easy proof of concept | Token generation can be slow for larger models |
Ollama announced in June 2026 that its GGUF-based improvements enabled Vulkan by default for broader GPU support. That makes the old claim “only NVIDIA works” inaccurate, but it does not remove compatibility checks. A platform advertised as having an NPU is not necessarily accelerated by Ollama through that NPU. Check the actual backend used by ollama ps, not a laptop’s marketing label.
How Much System RAM Does a Local AI Server Need?
System RAM remains important even when the GPU has enough VRAM. The operating system, model loading, page cache, retrieval/indexing services, Docker containers, embeddings and any CPU-offloaded layers all consume memory. The server may start with 16GB, but 32GB is a more flexible single-user starting point when building a dedicated machine. If you expect 30B-class models, large document indexing, VMs or multiple users, budget for 64GB or more if the platform supports it.
| Host RAM | Reasonable scenario | When to move up |
|---|---|---|
| 16GB | Lightweight single-user experiments, smaller models | Swap pressure, containers, offload or large files |
| 32GB | Balanced dedicated server for 4B–14B GPU-focused use | Multiple services, long context, larger models |
| 64GB | Workstation-class 24GB GPU, document pipelines and some CPU offload | Heavy concurrency or multiple large resident models |
| 96–128GB+ | Specialised CPU-heavy or large shared-memory use | Validate CPU/memory bandwidth before spending |
These are workload-oriented planning ranges rather than Ollama hard minimums. Check the motherboard’s memory slots, supported DIMM density, BIOS, ECC compatibility and maximum validated capacity. A cheap mini PC with soldered 16GB RAM can be the wrong purchase if your next model needs 64GB. Also distinguish Apple unified memory from discrete GPU VRAM: the operating system and model share one physical pool, so “36GB unified memory” does not mean 36GB is freely available for model weights.
Quantisation: Why Q4 Can Change the Entire Build
Quantisation stores model weights with fewer bits. A Q4-family model generally uses less storage and memory than an 8-bit or 16-bit version of the same architecture, but may alter accuracy. There is no single universally best quantisation: use a tag with a clearly documented format, compare responses on your own representative prompts, and record the tag so the configuration remains reproducible.
- Q4-class: common starting point for balancing capacity and quality on home GPUs.
- Q8-class: more memory-intensive, potentially useful where output quality differences justify it.
- FP16: substantially more storage and RAM/VRAM than Q4 for the same parameter count.
- KV-cache precision: a separate setting from model-weight quantisation; Ollama documents optional KV cache quantisation with quality/memory trade-offs.
Do not equate a model’s parameter count with its exact disk size. Tokeniser, tensor formats, quantisation metadata and architecture matter. Nor should you assume a MoE model only needs memory for its active parameters: all resident expert weights still have to be stored or offloaded.
Context Length: The Hidden VRAM Upgrade
A short home assistant prompt and a 60,000-token coding session are fundamentally different memory workloads. Context stores previous tokens and model state so the next token can attend to relevant history. Increasing the requested context increases KV-cache allocation, and running parallel sessions can multiply that cost. Ollama currently documents VRAM-dependent context defaults: below 24GiB, 4K; 24–48GiB, 32K; at least 48GiB, 256K. These are defaults, not promises that every model/hardware combination can sustain them.
For a new build, define two target profiles: interactive chat at 4K–8K and your actual long-document or coding workload. Test both on candidate hardware where possible. If the 8B model fits at 4K but offloads at 32K, decide whether the larger context is worth a GPU upgrade, KV-cache quantisation or lower concurrency. Do not simply raise OLLAMA_CONTEXT_LENGTH to the model’s theoretical maximum.
ollama pull qwen3:8b
ollama run qwen3:8b
# In another terminal, while a request is active:
ollama ps
# Inspect model details and actual loaded location:
ollama show qwen3:8b
Look at the PROCESSOR and CONTEXT columns in ollama ps. A displayed “100% GPU” indicates that the active model is fully GPU-resident in that run; a CPU/GPU split indicates offload. Record the model tag, context, concurrency, backend and driver version when comparing machines. These checks are more actionable than a single unrepeatable benchmark screenshot.
Storage: NVMe SSD for Models, NAS for Backups
Ollama models are large immutable-ish files, and a library of variants accumulates quickly. A 500GB SSD can hold many small and medium models, but a 1TB NVMe drive is a more comfortable default for an enthusiast who downloads several 14B–32B tags, runs other containers and retains snapshots. An NVMe SSD primarily improves downloads, startup and model loading; it does not replace missing VRAM or guarantee higher token throughput once a model is resident in GPU memory.
| Storage layout | Suggested purpose | Why |
|---|---|---|
| 256–500GB SSD | Small single-purpose Ollama installation | Sufficient for a modest model collection and OS, subject to free space |
| 1TB NVMe SSD | Flexible personal AI server | Room for model variants, container images and growth |
| 2TB+ SSD | Many large models, datasets, RAG indexes or multiple services | Less frequent pruning and space pressure |
| NAS / second disk | Versioned backups of configuration, prompts and irreplaceable indexes | Do not confuse model re-download with data recovery |
The standard Linux installation stores models under /usr/share/ollama/.ollama/models; the OLLAMA_MODELS environment variable can relocate them. Ensure the service account has access. If the server is virtualised, prefer storage that gives predictable local read performance for active model files; use a NAS for backups and shared datasets where appropriate. A 1GbE NAS link may make loading large models noticeably slower, but it will not change the GPU’s memory capacity.
Keep a record of the model tags and checksums where reproducibility matters. Back up any custom Modelfiles, prompts, scripts, embeddings, vector databases and application configuration. You can normally re-pull public models, whereas a carefully prepared local dataset or private index may not be replaceable.
CPU, Motherboard and PCIe: Avoid Buying an Unbalanced System
Once the GPU holds the active model, a flagship desktop CPU is often a poor first upgrade for a single interactive chat user. Spend on adequate VRAM, stable RAM, a suitable SSD, cooling and PSU before paying for extra CPU cores you may not use. More CPU threads matter for simultaneous services, preprocessing, embeddings, document parsing, CPU offload and virtual machines. Memory bandwidth matters especially when model weights sit in system RAM.
- CPU: choose enough cores for operating system, ingestion and supporting services; avoid assuming CPU generation alone determines LLM speed.
- Motherboard: verify a physical PCIe slot and lanes for the card, firmware compatibility, RAM expansion and suitable M.2 slots.
- Chassis: check GPU length, thickness, power-connector bend radius, intake and exhaust paths.
- Networking: 1GbE is usually sufficient for text prompts; larger file ingestion and remote model storage benefit from faster links.
- Virtualisation: GPU passthrough adds IOMMU, driver and isolation complexity. Prefer bare metal for the first benchmark baseline.
A tiny fanless firewall appliance or N100 mini PC with no discrete-GPU expansion is excellent for some home services but is rarely the most flexible Ollama platform. If you want one box to run Proxmox, Home Assistant, Frigate and Ollama, reserve CPU, RAM and storage for each workload and test whether the GPU can be passed through reliably. Otherwise separate low-power always-on services from the high-power AI workstation.
Power Supplies, Cooling and Noise: Plan for Real Peak Load
GPU power specifications are not the same as wall power. A server includes CPU, motherboard, memory, fans, storage, network interfaces and PSU losses. A reference RTX 3090 is specified at 350W graphics-card power; a build using it needs a PSU sized for the entire system, not merely 350W. NVIDIA’s reference RTX 3090 guidance lists 750W required system power for its stated reference configuration. Treat the PSU recommendation as a compatibility starting point and check the exact card manufacturer’s requirements.
Prefer a reputable PSU with the necessary native connectors, appropriate protections and good low-load efficiency. Do not use unapproved connector adapters or daisy-chained cables to feed a demanding GPU. Ensure that the card can exhaust heat without recirculating into NAS drives. A server installed in a cupboard may be quieter in the room but hotter inside the cupboard; monitor temperatures under sustained inference and provide ventilation.
| Component / consideration | Practical check | Common mistake |
|---|---|---|
| GPU connector | Exact board manufacturer requirement | Assuming all cards in a GPU family use the same connector |
| PSU capacity | Whole-system peak, transient margin, efficiency | Treating GPU TGP as complete system draw |
| Cooling | Sustained load temperature and airflow | Buying a short case that blocks GPU fans |
| Acoustics | Idle fans, PSU fan curve, GPU under load | Assuming a “silent” gaming card stays quiet under inference |
| UPS | Output watts, waveform, supported shutdown | Choosing UPS by VA alone without load/runtime check |
Electricity Costs: Calculate Idle and Active Time Separately
For a machine that remains on 24/7, idle power is often more important than peak GPU power. A 350W board rating does not imply 350W continuous use. Conversely, a system that idles at 80W can cost more over a year than an efficient server doing brief 300W inference sessions. Measure at the wall with a suitable power meter, after allowing background tasks and the GPU to settle. Do not derive home-server power from gaming review charts.
Annual electricity (kWh) = wall watts × hours per day × days / 1000
Annual cost = kWh × your electricity unit rate
Illustrative idle-only calculation (NOT a measured server):
35 W × 24 h × 365 / 1000 = 306.6 kWh/year
At £0.25/kWh: 306.6 × £0.25 = £76.65/year
Illustrative extra active load (NOT a benchmark):
+250 W × 2 h/day × 365 / 1000 = 182.5 kWh/year
At £0.25/kWh: +£45.63/year
The examples above are arithmetic scenarios, not real hardware measurements or a claim about current UK tariffs. Substitute your actual measured watts, usage pattern and full marginal electricity rate. Include cooling or ventilation overhead where material. If the AI server is only used in the evenings, consider scheduled shutdown or wake-on-LAN rather than keeping a discrete GPU host running all day. For a system that also hosts essential home automation, separating the AI box avoids powering a large GPU platform 24 hours a day.
| Illustrative 24/7 wall idle | Annual energy | Annual cost at illustrative £0.25/kWh |
|---|---|---|
| 20W | 175.2kWh | £43.80 |
| 35W | 306.6kWh | £76.65 |
| 60W | 525.6kWh | £131.40 |
| 90W | 788.4kWh | £197.10 |
This table excludes active inference above idle and is not a comparison of particular products. It shows why 20–30W of avoidable idle consumption can matter over several years. For full ownership cost add the case, motherboard, CPU, RAM, SSD, GPU, PSU, network accessories, backup storage and UPS—not just the graphics card.
A Sensible First Ollama Installation on Linux
On a supported Linux machine, follow Ollama’s official installation instructions rather than copying unreviewed third-party scripts. Confirm the GPU driver is installed first (nvidia-smi for NVIDIA). The command below is Ollama’s documented installer; review the script and your organisation’s security policy before executing it on a production server.
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
ollama pull qwen3:4b
ollama run qwen3:4b
# In a separate terminal while inference is running:
ollama ps
# NVIDIA hosts: confirm driver and live utilisation:
nvidia-smi
Begin with qwen3:4b, then try qwen3:8b only after confirming stable operation. Compare representative prompts, not a synthetic “hello” alone. If you change context, record that change with the results. On Linux the standard Ollama service is managed by systemd; use journalctl -u ollama and systemctl status ollama when troubleshooting startup or driver issues.
Docker and GPU Passthrough: Use the Official Runtime
Ollama also publishes a container image. For NVIDIA on Linux, install and configure the NVIDIA Container Toolkit before starting a GPU-enabled container. The official documentation shows --gpus=all and a persistent volume for model files. For a home server, bind the published port to localhost unless you have deliberately designed a secure authenticated access layer. The following adaptation preserves the documented GPU and volume options but avoids publishing the unauthenticated API to the whole LAN.
docker run -d \
--name ollama \
--restart unless-stopped \
--gpus=all \
-v ollama:/root/.ollama \
-p 127.0.0.1:11434:11434 \
ollama/ollama
docker exec -it ollama ollama pull qwen3:4b
docker exec -it ollama ollama run qwen3:4b
On AMD, use the ROCm-specific image and documented device mappings only when the exact GPU and driver combination is supported. On Apple computers, Ollama’s own FAQ says Docker Desktop on macOS does not provide the GPU acceleration path used by native Metal Ollama. Do not assume a container magically enables hardware support that the host lacks. For virtualised deployments, GPU passthrough must be verified separately.
Keep the Local API Local: Security and Privacy
Ollama listens on 127.0.0.1:11434 by default. Changing OLLAMA_HOST to 0.0.0.0:11434 makes the service reachable from other interfaces and may expose it to other users or devices. A local API should not be directly port-forwarded to the public internet. Use a VPN, authenticated reverse proxy, firewall rules and appropriate client authentication when remote access is genuinely needed.
Local inference can keep prompts and outputs on the machine, but modern Ollama also offers cloud features. If your requirement is strictly local operation, Ollama documents OLLAMA_NO_CLOUD=1 or a disable_ollama_cloud setting in ~/.ollama/server.json. Confirm the actual mode before sending sensitive material. Also consider telemetry and cloud integrations in any separate chat UI, retrieval service or browser extension connected to Ollama.
- Restrict access to trusted clients and keep the OS, GPU driver, Ollama and front-end applications updated.
- Do not publish port 11434 on your router or expose it through an unauthenticated tunnel.
- Use a non-admin service account where possible; restrict filesystem access to private datasets.
- Back up custom models and application data, and avoid giving an LLM process unrestricted access to host Docker sockets or secrets.
- Review model licences and permitted uses individually; “available for download” does not imply unrestricted commercial use.
How to Test the Server Without Inventing Benchmarks
The most useful benchmark is a reproducible test of your workload. Select two model tags and three representative prompts: a short factual question, a long context task and a code or document question. Record the context setting, CPU/GPU split, wall power, model-load time, response quality and latency. Run each more than once after the system has warmed up. A single output token rate is not enough to assess an interactive service.
| Record this | Why it matters | How to obtain it |
|---|---|---|
| Exact model and quantisation | Different tags are different workloads | ollama list and ollama show |
| Context and CPU/GPU split | Explains memory pressure and offload | ollama ps during a request |
| GPU memory and utilisation | Shows whether the card is constrained | nvidia-smi or vendor tools |
| Host RAM and swap | Finds CPU offload and paging problems | free -h, OS metrics |
| Idle and active wall watts | Calculates actual electricity cost | External power meter |
| User-perceived latency and quality | Decides whether hardware is useful | Repeat your real prompts; keep notes |
Ollama’s API can return timings such as load duration and generation duration, but compare like with like and be aware that prompt evaluation, model loading and output generation are different phases. Record software versions because inference backends evolve. A vendor GPU benchmark measured with a different model, quantisation or context cannot be transplanted into your build as a guaranteed result.
Common Problems and How to Diagnose Them
| Symptom | Likely cause | First check |
|---|---|---|
| Model runs but is unexpectedly slow | CPU offload, CPU-only backend or high context | ollama ps, GPU driver, context |
| GPU not detected | Unsupported card/driver, permissions, container runtime | Official GPU matrix, nvidia-smi, logs |
| Out-of-memory on a model that “fits” | KV cache, runtime buffers, other GPU workloads | Lower context; unload other models |
| Long first response | Model download or cold loading from disk | Repeat prompt; check disk and model residency |
| Several users cause slowdowns | Parallel context/memory and queueing | Reduce concurrency or use smaller model |
| Fans loud / throttling | Cooling and sustained power limit | Monitor temperature, airflow, fan curve |
| API works locally but not remotely | Localhost bind or firewall | Keep local; configure secure access intentionally |
| Model files fill system drive | Multiple quantisations or unused tags | ollama list; prune intentionally |
If you use OLLAMA_NUM_PARALLEL, remember Ollama’s FAQ states that parallel requests increase context memory requirements. The scheduler may queue work rather than loading more models when memory is exhausted. For one home user, it is often better to run one reliable model than to keep three large models resident in VRAM.
Three Complete Build Directions
Quiet, low-cost experiment: use the hardware you already have
Start with a laptop or existing mini PC, 16–32GB RAM and an SSD. Try qwen3:1.7b or qwen3:4b and measure latency before buying anything. This is the right choice for occasional learning, scripting and API experiments where response time is not critical. Avoid a 24/7 GPU workstation merely because the software is labelled “AI”.
Balanced single-user GPU server: 12–16GB VRAM
Choose a supported NVIDIA or AMD GPU with 12–16GB, 32GB system RAM, a 1TB NVMe SSD, an adequately ventilated case and a quality PSU. Begin with an 8B quantised model, then test 14B and the context length you actually need. The exact card, power and noise depend on local stock and your chassis. This is a practical balance for one user who wants interactive local chat, code help or light document workflows.
Larger-model workstation: 24GB VRAM and room to grow
A supported 24GB card, 64GB host RAM and 1–2TB NVMe storage is a more credible starting point for selected 30B/32B quantised models than a small fanless PC. But 24GB is not unlimited: a roughly 20GB model file leaves limited VRAM for context and runtime buffers. Check the specific model and be prepared to shorten context or accept some offload. Power, card size, cooling, UPS and replacement cost become major parts of the decision.
Should You Combine Ollama with a NAS and Home Assistant?
For most home labs, separate the high-power AI workload from always-on automation and storage. Run Home Assistant, the router and NAS on low-power equipment that stays online; wake the GPU machine when local inference is required. Store irreplaceable datasets and configuration backups on the NAS. If a shared AI server genuinely needs to run all day, measure its idle power and include the extra electricity in the purchasing decision.
For related hardware decisions, see Home Assistant Local Voice and AI: Green vs N150/N305 Mini PC, N100 vs N150 vs Core i5 vs Ryzen for a Home Server, and Home-Server Power Supplies: ATX vs SFX vs External DC. These guides cover different workloads and should not be read as a substitute for checking model memory requirements.
Local Ollama Home Server: Final Recommendation
Buy VRAM for the model and context you actually use, not the biggest GPU your budget can stretch to. A 4B model on existing hardware is the best first experiment. For a responsive dedicated single-user server, a supported 12–16GB GPU, 32GB RAM and 1TB SSD is a sensible middle ground. Move towards 24GB VRAM and 64GB host RAM only when your own 14B–32B workloads justify it. Keep the API private, document model tags, check GPU offload with ollama ps and measure wall power before committing to 24/7 operation. This yields a server designed around actual use rather than unverified AI performance claims.
Related Home-Server Guides
- Home Assistant Local Voice and AI: Green vs N150/N305 Mini PC
- N100 vs N150 vs Core i5 vs Ryzen for a Home Server
- Best Mini PCs for Proxmox and Multiple Virtual Machines
- Home-Server Power Supplies: ATX vs SFX vs External DC
- NAS and Home-Server Electricity Costs
- Best Small Home Servers for Plex, Jellyfin and Home Assistant Together
Datasheets & External Resources
- Ollama — Hardware support, NVIDIA, AMD, Metal and Vulkan
- Ollama — Context length and GPU residency
- Ollama — FAQ, parallelism, model storage and local/cloud settings
- Ollama — Official Linux installation and service instructions
- Ollama — Docker and GPU runtime instructions
- Ollama model library — Qwen3 tags and published download sizes
- Ollama — June 2026 GGUF and Vulkan support update
- NVIDIA — RTX 4060 Ti 8GB/16GB variants and power specifications
- NVIDIA — RTX 3090 24GB and board-power specifications
- NVIDIA — RTX 30 series specifications including RTX 3060