Local AI Home Server: GPU VRAM, RAM, Storage and Power for Ollama

Build a local Ollama AI server with realistic GPU VRAM, RAM, SSD, PSU and electricity planning. Includes model-size examples, configurations and checks.

Quick Summary (TL;DR):
For a useful local Ollama home server, choose the model and context length first, then buy hardware. GPU VRAM is usually the limiting factor for responsive inference: 8GB can be a sensible entry point for small quantised models; 12–16GB provides more room for an 8B model and some 14B workloads; and 24GB makes larger quantised models substantially more practical. These are planning tiers, not guaranteed model-fit or speed benchmarks. A typical single-user machine can start with 32GB system RAM and a 500GB–1TB NVMe SSD, while larger models, CPU offload and concurrent users justify more. Measure whole-system idle and loaded power before estimating 24/7 electricity cost. If you only need occasional experiments, test Ollama on hardware you already own before buying a high-power GPU. All model sizes and support notes below refer to manufacturer/project information checked in October 2026, not hands-on testing.

The Short Answer: Match the Build to the Model, Not the Other Way Round

A home AI server is not the same as a general-purpose home lab. Plex needs media decoding, Home Assistant needs predictable low-power uptime, and a local language model needs enough fast memory to hold its weights and working context. An Intel N100PriceAliExpress price or N150 mini PCPriceAliExpress price can run small models on its CPU, but that does not turn it into a substitute for a workstation with a supported discrete GPU. Conversely, a powerful gaming PC that idles all day may be an expensive way to serve two prompts every evening.

Ollama supports local models and provides a local API, but model selection changes the hardware requirement dramatically. For a quick trial, start with a small model such as qwen3:4b. For sustained personal chat or code assistance, consider an 8B-class model and a GPU with adequate VRAM. For longer context, multiple users or 30B-class models, design the complete machine around memory capacity, cooling and power rather than buying the cheapest advertised “AI PC”.

Your main workloadSensible starting hardware directionKey limitation to check
Occasional experiments, short promptsExisting laptop/mini PCPriceAliExpress price; 16–32GB RAM; CPU or supported integrated GPULatency can be high; not a speed recommendation
One user, 4B–8B quantised chatSupported 8–12GB GPU; 32GB system RAM; NVMe SSDLong context and GPU overhead can exceed the card
One user, 8B–14B or document workflows12–16GB GPU; 32–64GB RAM; SSDModel quantisation and KV cache determine fit
30B/32B-class quantised models24GB GPU or larger/multiple supported GPUs; 64GB RAM often sensibleNot every 30B variant fits or performs equally
Several users, agents, large contextPlan from measured simultaneous workloads; consider 24GB+ VRAMParallel requests multiply memory demand

The recommendations above are deliberately conservative categories rather than invented tokens-per-second figures. CPU, memory bandwidth, model architecture, quantisation, context and software backend all affect performance. A single GPU that fits the model is usually simpler than spreading it across multiple cards.

What Actually Uses Memory in Ollama?

There are four separate memory consumers: model weights, the key/value (KV) cache for context, runtime/compute buffers, and other services or loaded models. A model download of 5.2GB is not a promise that a 6GB graphics card will run it entirely in VRAM. File size and live memory consumption are different quantities. Larger prompts and more simultaneous sessions raise memory use even when the model file does not change.

  • Model weights: primarily determined by parameter count and quantisation (for example Q4 versus Q8).
  • KV cache: grows with context length, architecture, precision and number of active sequences.
  • Runtime buffers: depend on backend, batch settings and model implementation.
  • Other GPU users: a desktop compositor, video workload or second AI model reduces available VRAM.
  • CPU offload: allows some models to run with less GPU memory, but transfers and system RAM can limit responsiveness.

Ollama’s current context-length guidance states that its defaults are based on available VRAM and that increasing context requires more memory. Always inspect the context actually allocated rather than assuming a model’s maximum advertised window is automatically in use. A model claiming 128K or 256K context is not a guarantee that your home server can afford to run it at that length.

Real Ollama Model Sizes: What the Downloads Tell You

The following figures are the published Ollama library download sizes for specific Qwen3 tags checked on 11 October 2026. They are useful for estimating SSD space and relative weight size, not minimum GPU VRAM. Tags can be updated, so verify the tag and quantisation before purchasing hardware.

Ollama tagLibrary download sizePractical starting interpretation
qwen3:1.7b1.4GBLightweight trial or constrained CPU-only system
qwen3:4b2.5GBUseful first local model; test latency and answer quality
qwen3:8b5.2GBGood 8GB/12GB GPU candidate at modest context, verify live fit
qwen3:14b9.3GB12–16GB GPU candidate depending on context and overhead
qwen3:30b19GB24GB class merits testing; context may push beyond VRAM
qwen3:32b20GBLarge single-GPU candidate with limited headroom on 24GB

The qwen3:30b and qwen3:32b entries should not be treated as interchangeable. The former is a mixture-of-experts model while the latter is dense; active compute, model behaviour and memory access patterns differ. Similarly, changing a tag from its default quantisation to an FP16 variant can multiply its storage and memory needs. Use the official Qwen3 model listing and its per-tag details, not a generic “8B requires 8GB” rule.

GPU VRAM: The Most Important Purchase Decision

For a responsive single-user LLM, first try to keep the complete active model and working context on the GPU. Ollama’s ollama ps reports whether a loaded model is 100% GPU, 100% CPU or split. A split is not automatically a failure, but it changes the performance expectation. Memory bandwidth also matters: two cards with the same VRAM may produce very different throughput. That is why a card’s advertised AI TOPS or gaming frame rate is not enough to choose it.

VRAM classTypical useWhat can go wrong
4–6GBSmall quantised 1B–4B models; experimentationEven modest contexts and other GPU loads can be restrictive
8GB4B models and selected 8B Q4 modelsAn 8B download around 5GB still needs runtime/context headroom
12GBComfortable small-model headroom; selected 14B Q4 attempts14B fit depends heavily on context, KV cache and backend
16GB8B/14B with more breathing roomLong context and concurrency still consume VRAM
24GBSelected 30B/32B quantised models; bigger 14B context19–20GB weights leave finite space for KV cache and buffers
32GB+Larger models, longer context, heavier multi-service usePower, cost, support and thermal constraints rise

Do not buy a nominally 16GB card without checking the exact SKU: NVIDIA’s RTX 4060 Ti, for example, was sold in 8GB and 16GB variants. The difference matters more to a memory-bound AI workload than the product name alone. The RTX 3060 12GB and RTX 3090 24GB are also useful reference points, but their age, second-hand condition and electricity use may outweigh their capacity advantages in a new 24/7 build.

Which GPU Should You Consider? Verified Hardware Examples

This is a specification-based comparison, not a ranking from our own inference benchmarks. Availability, new-versus-used price and board-specific cooling vary by region and retailer. NVIDIA lists 12GB on the RTX 3060, 8GB or 16GB on the RTX 4060 Ti, and 24GB on the RTX 3090. These capacities can be attractive at different budgets, but confirm that the actual card is supported by the current Ollama backend and driver.

GPU exampleManufacturer VRAM specificationManufacturer board-power referenceBuying caution
GeForce RTX 3060 (12GB variant)12GB GDDR6170W graphics card powerCheck that listing is 12GB, not another similarly named product
GeForce RTX 4060 Ti (16GB variant)16GB GDDR6165W or 160W total graphics power8GB variant exists; check exact SKU
GeForce RTX 309024GB GDDR6X350W graphics card powerLarge cooler, power connectors and used-card condition matter

These wattages are manufacturer board-power ratings, not measured idle draw, actual Ollama consumption or total server power. The RTX 3090 reference specification recommends a 750W system PSU in NVIDIA’s reference configuration; that does not mean the server draws 750W continuously. A quality power supply must still accommodate GPU transient demand, CPU, drives, fans and the chosen card’s connector requirements.

Newer NVIDIA 50-series models, supported AMD ROCm devices, Apple silicon and Vulkan-compatible GPUs are valid options, but support is not uniform across operating systems or exact devices. Ollama’s official hardware support list is the source of truth. As checked in October 2026, the documentation specifies NVIDIA compute capability 5.0+ and driver version 550+ (570+ for capability 5.0–6.2), ROCm v7 requirements for supported AMD configurations, Metal on Apple devices, and additional Vulkan support. Verify driver and backend before ordering.

NVIDIA, AMD, Intel and Apple Silicon: Which Platform Is Easiest?

PlatformMain advantageImportant limitation
NVIDIA CUDAWidely documented Ollama GPU path; mature container toolingVRAM pricing, power, card clearance; driver version
AMD ROCmPotentially attractive VRAM and platform choiceSupported GPU/OS matrix and ROCm version must match
AMD/Intel via VulkanBroader GPU coverage in current Ollama releasesBackend/driver differences; test your exact hardware
Apple silicon / MetalUnified-memory machines and compact, quiet hardwareMemory is shared with OS; upgrade options limited
CPU onlyNo GPU purchase; easy proof of conceptToken generation can be slow for larger models

Ollama announced in June 2026 that its GGUF-based improvements enabled Vulkan by default for broader GPU support. That makes the old claim “only NVIDIA works” inaccurate, but it does not remove compatibility checks. A platform advertised as having an NPU is not necessarily accelerated by Ollama through that NPU. Check the actual backend used by ollama ps, not a laptop’s marketing label.

How Much System RAM Does a Local AI Server Need?

System RAM remains important even when the GPU has enough VRAM. The operating system, model loading, page cache, retrieval/indexing services, Docker containers, embeddings and any CPU-offloaded layers all consume memory. The server may start with 16GB, but 32GB is a more flexible single-user starting point when building a dedicated machine. If you expect 30B-class models, large document indexing, VMs or multiple users, budget for 64GB or more if the platform supports it.

Host RAMReasonable scenarioWhen to move up
16GBLightweight single-user experiments, smaller modelsSwap pressure, containers, offload or large files
32GBBalanced dedicated server for 4B–14B GPU-focused useMultiple services, long context, larger models
64GBWorkstation-class 24GB GPU, document pipelines and some CPU offloadHeavy concurrency or multiple large resident models
96–128GB+Specialised CPU-heavy or large shared-memory useValidate CPU/memory bandwidth before spending

These are workload-oriented planning ranges rather than Ollama hard minimums. Check the motherboard’s memory slots, supported DIMM density, BIOS, ECC compatibility and maximum validated capacity. A cheap mini PC with soldered 16GB RAM can be the wrong purchase if your next model needs 64GB. Also distinguish Apple unified memory from discrete GPU VRAM: the operating system and model share one physical pool, so “36GB unified memory” does not mean 36GB is freely available for model weights.

Quantisation: Why Q4 Can Change the Entire Build

Quantisation stores model weights with fewer bits. A Q4-family model generally uses less storage and memory than an 8-bit or 16-bit version of the same architecture, but may alter accuracy. There is no single universally best quantisation: use a tag with a clearly documented format, compare responses on your own representative prompts, and record the tag so the configuration remains reproducible.

  • Q4-class: common starting point for balancing capacity and quality on home GPUs.
  • Q8-class: more memory-intensive, potentially useful where output quality differences justify it.
  • FP16: substantially more storage and RAM/VRAM than Q4 for the same parameter count.
  • KV-cache precision: a separate setting from model-weight quantisation; Ollama documents optional KV cache quantisation with quality/memory trade-offs.

Do not equate a model’s parameter count with its exact disk size. Tokeniser, tensor formats, quantisation metadata and architecture matter. Nor should you assume a MoE model only needs memory for its active parameters: all resident expert weights still have to be stored or offloaded.

Context Length: The Hidden VRAM Upgrade

A short home assistant prompt and a 60,000-token coding session are fundamentally different memory workloads. Context stores previous tokens and model state so the next token can attend to relevant history. Increasing the requested context increases KV-cache allocation, and running parallel sessions can multiply that cost. Ollama currently documents VRAM-dependent context defaults: below 24GiB, 4K; 24–48GiB, 32K; at least 48GiB, 256K. These are defaults, not promises that every model/hardware combination can sustain them.

For a new build, define two target profiles: interactive chat at 4K–8K and your actual long-document or coding workload. Test both on candidate hardware where possible. If the 8B model fits at 4K but offloads at 32K, decide whether the larger context is worth a GPU upgrade, KV-cache quantisation or lower concurrency. Do not simply raise OLLAMA_CONTEXT_LENGTH to the model’s theoretical maximum.

ollama pull qwen3:8b
ollama run qwen3:8b
# In another terminal, while a request is active:
ollama ps

# Inspect model details and actual loaded location:
ollama show qwen3:8b

Look at the PROCESSOR and CONTEXT columns in ollama ps. A displayed “100% GPU” indicates that the active model is fully GPU-resident in that run; a CPU/GPU split indicates offload. Record the model tag, context, concurrency, backend and driver version when comparing machines. These checks are more actionable than a single unrepeatable benchmark screenshot.

Storage: NVMe SSD for Models, NAS for Backups

Ollama models are large immutable-ish files, and a library of variants accumulates quickly. A 500GB SSD can hold many small and medium models, but a 1TB NVMe drive is a more comfortable default for an enthusiast who downloads several 14B–32B tags, runs other containers and retains snapshots. An NVMe SSD primarily improves downloads, startup and model loading; it does not replace missing VRAM or guarantee higher token throughput once a model is resident in GPU memory.

Storage layoutSuggested purposeWhy
256–500GB SSDSmall single-purpose Ollama installationSufficient for a modest model collection and OS, subject to free space
1TB NVMe SSDFlexible personal AI serverRoom for model variants, container images and growth
2TB+ SSDMany large models, datasets, RAG indexes or multiple servicesLess frequent pruning and space pressure
NAS / second diskVersioned backups of configuration, prompts and irreplaceable indexesDo not confuse model re-download with data recovery

The standard Linux installation stores models under /usr/share/ollama/.ollama/models; the OLLAMA_MODELS environment variable can relocate them. Ensure the service account has access. If the server is virtualised, prefer storage that gives predictable local read performance for active model files; use a NAS for backups and shared datasets where appropriate. A 1GbE NAS link may make loading large models noticeably slower, but it will not change the GPU’s memory capacity.

Keep a record of the model tags and checksums where reproducibility matters. Back up any custom Modelfiles, prompts, scripts, embeddings, vector databases and application configuration. You can normally re-pull public models, whereas a carefully prepared local dataset or private index may not be replaceable.

CPU, Motherboard and PCIe: Avoid Buying an Unbalanced System

Once the GPU holds the active model, a flagship desktop CPU is often a poor first upgrade for a single interactive chat user. Spend on adequate VRAM, stable RAM, a suitable SSD, cooling and PSU before paying for extra CPU cores you may not use. More CPU threads matter for simultaneous services, preprocessing, embeddings, document parsing, CPU offload and virtual machines. Memory bandwidth matters especially when model weights sit in system RAM.

  • CPU: choose enough cores for operating system, ingestion and supporting services; avoid assuming CPU generation alone determines LLM speed.
  • Motherboard: verify a physical PCIe slot and lanes for the card, firmware compatibility, RAM expansion and suitable M.2 slots.
  • Chassis: check GPU length, thickness, power-connector bend radius, intake and exhaust paths.
  • Networking: 1GbE is usually sufficient for text prompts; larger file ingestion and remote model storage benefit from faster links.
  • Virtualisation: GPU passthrough adds IOMMU, driver and isolation complexity. Prefer bare metal for the first benchmark baseline.

A tiny fanless firewall appliance or N100 mini PC with no discrete-GPU expansion is excellent for some home services but is rarely the most flexible Ollama platform. If you want one box to run Proxmox, Home Assistant, Frigate and Ollama, reserve CPU, RAM and storage for each workload and test whether the GPU can be passed through reliably. Otherwise separate low-power always-on services from the high-power AI workstation.

Power Supplies, Cooling and Noise: Plan for Real Peak Load

GPU power specifications are not the same as wall power. A server includes CPU, motherboard, memory, fans, storage, network interfaces and PSU losses. A reference RTX 3090 is specified at 350W graphics-card power; a build using it needs a PSU sized for the entire system, not merely 350W. NVIDIA’s reference RTX 3090 guidance lists 750W required system power for its stated reference configuration. Treat the PSU recommendation as a compatibility starting point and check the exact card manufacturer’s requirements.

Prefer a reputable PSU with the necessary native connectors, appropriate protections and good low-load efficiency. Do not use unapproved connector adapters or daisy-chained cables to feed a demanding GPU. Ensure that the card can exhaust heat without recirculating into NAS drives. A server installed in a cupboard may be quieter in the room but hotter inside the cupboard; monitor temperatures under sustained inference and provide ventilation.

Component / considerationPractical checkCommon mistake
GPU connectorExact board manufacturer requirementAssuming all cards in a GPU family use the same connector
PSU capacityWhole-system peak, transient margin, efficiencyTreating GPU TGP as complete system draw
CoolingSustained load temperature and airflowBuying a short case that blocks GPU fans
AcousticsIdle fans, PSU fan curve, GPU under loadAssuming a “silent” gaming card stays quiet under inference
UPSOutput watts, waveform, supported shutdownChoosing UPS by VA alone without load/runtime check

Electricity Costs: Calculate Idle and Active Time Separately

For a machine that remains on 24/7, idle power is often more important than peak GPU power. A 350W board rating does not imply 350W continuous use. Conversely, a system that idles at 80W can cost more over a year than an efficient server doing brief 300W inference sessions. Measure at the wall with a suitable power meter, after allowing background tasks and the GPU to settle. Do not derive home-server power from gaming review charts.

Annual electricity (kWh) = wall watts × hours per day × days / 1000
Annual cost = kWh × your electricity unit rate

Illustrative idle-only calculation (NOT a measured server):
35 W × 24 h × 365 / 1000 = 306.6 kWh/year
At £0.25/kWh: 306.6 × £0.25 = £76.65/year

Illustrative extra active load (NOT a benchmark):
+250 W × 2 h/day × 365 / 1000 = 182.5 kWh/year
At £0.25/kWh: +£45.63/year

The examples above are arithmetic scenarios, not real hardware measurements or a claim about current UK tariffs. Substitute your actual measured watts, usage pattern and full marginal electricity rate. Include cooling or ventilation overhead where material. If the AI server is only used in the evenings, consider scheduled shutdown or wake-on-LAN rather than keeping a discrete GPU host running all day. For a system that also hosts essential home automation, separating the AI box avoids powering a large GPU platform 24 hours a day.

Illustrative 24/7 wall idleAnnual energyAnnual cost at illustrative £0.25/kWh
20W175.2kWh£43.80
35W306.6kWh£76.65
60W525.6kWh£131.40
90W788.4kWh£197.10

This table excludes active inference above idle and is not a comparison of particular products. It shows why 20–30W of avoidable idle consumption can matter over several years. For full ownership cost add the case, motherboard, CPU, RAM, SSD, GPU, PSU, network accessories, backup storage and UPS—not just the graphics card.

A Sensible First Ollama Installation on Linux

On a supported Linux machine, follow Ollama’s official installation instructions rather than copying unreviewed third-party scripts. Confirm the GPU driver is installed first (nvidia-smi for NVIDIA). The command below is Ollama’s documented installer; review the script and your organisation’s security policy before executing it on a production server.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
ollama pull qwen3:4b
ollama run qwen3:4b

# In a separate terminal while inference is running:
ollama ps

# NVIDIA hosts: confirm driver and live utilisation:
nvidia-smi

Begin with qwen3:4b, then try qwen3:8b only after confirming stable operation. Compare representative prompts, not a synthetic “hello” alone. If you change context, record that change with the results. On Linux the standard Ollama service is managed by systemd; use journalctl -u ollama and systemctl status ollama when troubleshooting startup or driver issues.

Docker and GPU Passthrough: Use the Official Runtime

Ollama also publishes a container image. For NVIDIA on Linux, install and configure the NVIDIA Container Toolkit before starting a GPU-enabled container. The official documentation shows --gpus=all and a persistent volume for model files. For a home server, bind the published port to localhost unless you have deliberately designed a secure authenticated access layer. The following adaptation preserves the documented GPU and volume options but avoids publishing the unauthenticated API to the whole LAN.

docker run -d \
  --name ollama \
  --restart unless-stopped \
  --gpus=all \
  -v ollama:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

docker exec -it ollama ollama pull qwen3:4b
docker exec -it ollama ollama run qwen3:4b

On AMD, use the ROCm-specific image and documented device mappings only when the exact GPU and driver combination is supported. On Apple computers, Ollama’s own FAQ says Docker Desktop on macOS does not provide the GPU acceleration path used by native Metal Ollama. Do not assume a container magically enables hardware support that the host lacks. For virtualised deployments, GPU passthrough must be verified separately.

Keep the Local API Local: Security and Privacy

Ollama listens on 127.0.0.1:11434 by default. Changing OLLAMA_HOST to 0.0.0.0:11434 makes the service reachable from other interfaces and may expose it to other users or devices. A local API should not be directly port-forwarded to the public internet. Use a VPN, authenticated reverse proxy, firewall rules and appropriate client authentication when remote access is genuinely needed.

Local inference can keep prompts and outputs on the machine, but modern Ollama also offers cloud features. If your requirement is strictly local operation, Ollama documents OLLAMA_NO_CLOUD=1 or a disable_ollama_cloud setting in ~/.ollama/server.json. Confirm the actual mode before sending sensitive material. Also consider telemetry and cloud integrations in any separate chat UI, retrieval service or browser extension connected to Ollama.

  • Restrict access to trusted clients and keep the OS, GPU driver, Ollama and front-end applications updated.
  • Do not publish port 11434 on your router or expose it through an unauthenticated tunnel.
  • Use a non-admin service account where possible; restrict filesystem access to private datasets.
  • Back up custom models and application data, and avoid giving an LLM process unrestricted access to host Docker sockets or secrets.
  • Review model licences and permitted uses individually; “available for download” does not imply unrestricted commercial use.

How to Test the Server Without Inventing Benchmarks

The most useful benchmark is a reproducible test of your workload. Select two model tags and three representative prompts: a short factual question, a long context task and a code or document question. Record the context setting, CPU/GPU split, wall power, model-load time, response quality and latency. Run each more than once after the system has warmed up. A single output token rate is not enough to assess an interactive service.

Record thisWhy it mattersHow to obtain it
Exact model and quantisationDifferent tags are different workloadsollama list and ollama show
Context and CPU/GPU splitExplains memory pressure and offloadollama ps during a request
GPU memory and utilisationShows whether the card is constrainednvidia-smi or vendor tools
Host RAM and swapFinds CPU offload and paging problemsfree -h, OS metrics
Idle and active wall wattsCalculates actual electricity costExternal power meter
User-perceived latency and qualityDecides whether hardware is usefulRepeat your real prompts; keep notes

Ollama’s API can return timings such as load duration and generation duration, but compare like with like and be aware that prompt evaluation, model loading and output generation are different phases. Record software versions because inference backends evolve. A vendor GPU benchmark measured with a different model, quantisation or context cannot be transplanted into your build as a guaranteed result.

Common Problems and How to Diagnose Them

SymptomLikely causeFirst check
Model runs but is unexpectedly slowCPU offload, CPU-only backend or high contextollama ps, GPU driver, context
GPU not detectedUnsupported card/driver, permissions, container runtimeOfficial GPU matrix, nvidia-smi, logs
Out-of-memory on a model that “fits”KV cache, runtime buffers, other GPU workloadsLower context; unload other models
Long first responseModel download or cold loading from diskRepeat prompt; check disk and model residency
Several users cause slowdownsParallel context/memory and queueingReduce concurrency or use smaller model
Fans loud / throttlingCooling and sustained power limitMonitor temperature, airflow, fan curve
API works locally but not remotelyLocalhost bind or firewallKeep local; configure secure access intentionally
Model files fill system driveMultiple quantisations or unused tagsollama list; prune intentionally

If you use OLLAMA_NUM_PARALLEL, remember Ollama’s FAQ states that parallel requests increase context memory requirements. The scheduler may queue work rather than loading more models when memory is exhausted. For one home user, it is often better to run one reliable model than to keep three large models resident in VRAM.

Three Complete Build Directions

Quiet, low-cost experiment: use the hardware you already have

Start with a laptop or existing mini PC, 16–32GB RAM and an SSD. Try qwen3:1.7b or qwen3:4b and measure latency before buying anything. This is the right choice for occasional learning, scripting and API experiments where response time is not critical. Avoid a 24/7 GPU workstation merely because the software is labelled “AI”.

Balanced single-user GPU server: 12–16GB VRAM

Choose a supported NVIDIA or AMD GPU with 12–16GB, 32GB system RAM, a 1TB NVMe SSD, an adequately ventilated case and a quality PSU. Begin with an 8B quantised model, then test 14B and the context length you actually need. The exact card, power and noise depend on local stock and your chassis. This is a practical balance for one user who wants interactive local chat, code help or light document workflows.

Larger-model workstation: 24GB VRAM and room to grow

A supported 24GB card, 64GB host RAM and 1–2TB NVMe storage is a more credible starting point for selected 30B/32B quantised models than a small fanless PC. But 24GB is not unlimited: a roughly 20GB model file leaves limited VRAM for context and runtime buffers. Check the specific model and be prepared to shorten context or accept some offload. Power, card size, cooling, UPS and replacement cost become major parts of the decision.

Should You Combine Ollama with a NAS and Home Assistant?

For most home labs, separate the high-power AI workload from always-on automation and storage. Run Home Assistant, the router and NAS on low-power equipment that stays online; wake the GPU machine when local inference is required. Store irreplaceable datasets and configuration backups on the NAS. If a shared AI server genuinely needs to run all day, measure its idle power and include the extra electricity in the purchasing decision.

For related hardware decisions, see Home Assistant Local Voice and AI: Green vs N150/N305 Mini PC, N100 vs N150 vs Core i5 vs Ryzen for a Home Server, and Home-Server Power Supplies: ATX vs SFX vs External DC. These guides cover different workloads and should not be read as a substitute for checking model memory requirements.

Local Ollama Home Server: Final Recommendation

Buy VRAM for the model and context you actually use, not the biggest GPU your budget can stretch to. A 4B model on existing hardware is the best first experiment. For a responsive dedicated single-user server, a supported 12–16GB GPU, 32GB RAM and 1TB SSD is a sensible middle ground. Move towards 24GB VRAM and 64GB host RAM only when your own 14B–32B workloads justify it. Keep the API private, document model tags, check GPU offload with ollama ps and measure wall power before committing to 24/7 operation. This yields a server designed around actual use rather than unverified AI performance claims.

Related Home-Server Guides

Datasheets & External Resources

CONTINUE EXPLORING / NAS & HOME SERVERS

Read next

Practical guides related to this article.

Explore the topic →
HARDWARE DECISION GUIDEPlanning your next NAS upgrade?

Compare hardware, storage and software before buying.

Compare options →