Whether the Ollama performance of NAS deployment is insufficient: how to judge the CPU, memory and graphics card

NAS can deploy Ollama, but “can be installed” and “quick enough to use” are two different things.

If the goal is to make occasional summaries, file Q&A, home automation and low-frequency API calls, x86 NAS can use CPU to run small models; if you want to have continuous conversations like chat software, run 7B/8B code models, or make multiple service calls, the key becomes video memory, memory and heat dissipation, not how many TB the hard drive has.

Let’s give the conclusion first:

**NAS without available GPU is suitable for small models and low-frequency tasks; only NVIDIA GPUs correctly recognized by Ollama are more suitable for using 7B/8B models as daily services. **

First confirm which category your NAS belongs to

Don’t just look at “AI chip”, “NPU” or “4K transcoding” on the product promotion page. Whether Ollama can accelerate depends on whether the system, driver, container permissions and hardware are recognized in the actual running environment.

The table is a starting point for deployment judgment, not a performance promise. CPU model, memory channels, model format, context length, number of concurrencies, NAS system limitations, and thermal dissipation will vary actual results.

Let’s first look at “storage” and “reasoning” separately.

The large hard disk of NAS mainly solves the problem of model file storage: you can save many GGUF, Ollama model layers and document libraries. The inference speed is mainly determined by the following items:

• CPU core performance and memory bandwidth;
• Whether the system memory is sufficient to install the model and runtime;
• Whether the GPU is available and whether the video memory can accommodate the model and context;
• Whether PCIe lanes, drivers and containers are transparently transmitted correctly;
• Context length, model quantization, concurrency and output length.

So the answer to “Can a NAS run a 32B model if it has a 40TB hard drive?” is usually: files can certainly be placed, but they may not be able to be inferred at an acceptable speed.

CPU-only NAS: When is it worth deploying?

It is possible to run Ollama without a GPU. The official Docker image supports pure CPU boot:

1
2
3
4
5

docker run -d \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama

CPU-only suitable for:

• Nightly batch summarization, document classification, tag extraction;
• low-frequency text tasks in home automation;
• Validate API calls, RAG pipelines, and permissions logic;
• Personal tools that are insensitive to initial token latency and generation speed.

Not very suitable for:

• Multi-person chat service;
• Long time code Agent;
• Large models are generated continuously;
• Real-time voice assistant or high-frequency web Q&A.

The correct strategy for CPU-only is to use a small model first, a short context, and limit concurrency. Don’t download the largest model first and then expect to get a cloud-like experience by “relying on the NAS and calculating slowly”.

How much memory should be reserved at least?

System memory is not only used for model weights, but also for the NAS itself, file cache, Docker, download tasks, photo index, containers and KV Cache.

A practical principle: **Don’t let Ollama eat up the NAS memory until it starts swapping a lot. ** Once swap intervenes frequently, the response time will significantly deteriorate and file sharing and other services will also be affected.

Model file size is only a lower limit. Model loading, context, and runtime will continue to take up memory, so don’t rely on “file 8GB, machine 8GB” to judge whether it can run.

The biggest difference in experience is seen with NVIDIA GPUs

If the NAS is x86 Linux and has a supported NVIDIA graphics card, Ollama can use GPU acceleration. Ollama officially lists the support range of NVIDIA compute capability 5.0+ and driver 531+, among which the RTX 30 series is also included in the support list.

The Docker solution requires the host to first configure the NVIDIA Container Toolkit, and then use the GPU parameters to start the container:

1
2
3
4
5
6

docker run -d \
  --gpus=all \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  --name ollama \
  ollama/ollama

After the container is started, verify the GPU transparent transmission itself instead of directly downloading the large model:

1

docker run --gpus all ubuntu nvidia-smi

When this command does not display the graphics card properly, the Ollama container cannot use the GPU. At this time, you should first check whether the driver, Container Toolkit, Docker runtime, and NAS system support PCIe GPU transparent transmission.

Can the core graphics, NPU and transcoding capabilities of NAS be used directly?

This cannot be assumed by default.

The core graphics of many NAS are mainly used for video decoding, transcoding or graphics output; the NPU advertised by some devices may only be open to the manufacturer’s own applications. Whether they can be used by Ollama depends on the backend, operating system, driver and device permissions supported by Ollama.

Actual evidence should be seen before deployment:

• Whether the host can see the GPU device;
• Whether nvidia-smi or the corresponding manufacturer’s detection tool can be run in the container;
• Do Ollama logs clearly show GPU usage;
• Under the same prompt, is the speed difference between GPU and CPU reasonable?

When you don’t see this evidence, plan according to CPU-only expectations. Don’t purchase a model based on discrete graphics performance just because the device has a “transcoding engine”.

How to choose a model so as not to bring down the NAS

The core of NAS deployment is not to pursue the largest model, but to let common tasks be stably completed first.

Once the model is too large, the common symptoms of NAS are not immediate failure, but long loading, slow first token generation, low generation speed, and continuous increase in system memory, which ultimately affects the stability of file services and containers.

Recommended deployment structure

For most families or small teams, it is recommended to use NAS as a “data and light reasoning node”:

1
2
3
4
5
6
7
8

NAS
├─ Ollama：小模型、Embedding、低频 API
├─ 文档与向量库：私有文件、备份、RAG 数据
├─ 反向代理：局域网访问控制
└─ 监控：CPU、内存、GPU、容器日志

高显存主机或云端
└─ 大模型、长上下文、复杂 Agent、高并发推理

This division of labor is more stable than forcing all inference into the NAS: sensitive files remain local, and heavy tasks are routed to more appropriate devices.

Five-minute test before deployment

Don’t download dozens of GB models first. First judge according to this process:

1. Confirm NAS architecture, Docker support, and remaining memory.
2. Start the Ollama container and confirm that http://NAS地址:11434 is accessible.
3. Pull up a small model and ask 5 to 10 fixed questions in a row.
4. Observe CPU, memory, swap, disk IO and GPU usage.
5. Then switch to a model that is close to real requirements, and gradually increase context and concurrency.

You can use the following command to view container resources:

1

docker stats ollama

If you have an NVIDIA GPU, observe at the same time:

1

nvidia-smi

As long as file sharing, photo serving, or backup tasks are noticeably slower when running a model, the current model or concurrency is not suitable for NAS, and the model size should be reduced or the inference should be moved to other machines.

Don’t ignore network and security

Ollama’s default local service port is 11434. If you want other devices on the LAN to call it, first implement access control at the NAS firewall or reverse proxy layer, and do not directly expose the unauthenticated inference port to the public network.

Especially when there are family photos, backups, documents and private files in the NAS, the model service, file service and management backend should use different permission boundaries. For calls that can be completed on the LAN, there is no need to open the public network port.

Summarize

Whether Ollama is sufficient for NAS deployment can be judged as follows:

1
2
3
4
5
6
7
8

只做摘要、分类、Embedding
-> CPU-only NAS 可以尝试

想要 7B/8B 模型的日常交互体验
-> 需要可用 GPU，优先看显存

想做长上下文、复杂 Agent 或多人并发
-> 更适合高显存主机或云端推理

First use a small model to verify resources and stability, and then decide whether to add a graphics card, expand memory, or split inference nodes. NAS is great for hosting private data and lightweight model services, but it is not a natural large model server.

refer to:

• Ollama Hardware Support
• Ollama Docker Deployment
• Ollama GPU Troubleshooting
• Ollama multi-model switching tutorial