How to configure Ollama multi-model switching: resident, video memory and Modelfile tutorials

Sat, 11 Jul 2026 09:32:06 +0800

After Ollama has installed several models, the first question many people have is: How to quickly switch between chat, code, translation and Embedding models? The second question is often more practical: Why does the previous one disappear from the video memory just after switching to another model?

Let me start with the conclusion: Ollama does not need to serve each model separately. Use ollama run <模型名> for daily switching; use ollama ps to see which models are currently in memory; use ollama stop <模型名> to release unnecessary models. Whether multiple models can be resident at the same time depends on whether they can fit into the available video memory or memory, rather than how many models are downloaded to the local disk.

Most commonly used switching commands

First list the models that have been downloaded by this machine:

`1`	`ollama ls`

Start a chat or code model:

`1`	`ollama run qwen3:8b`

When you need to change to another model, run another name directly:

`1`	`ollama run qwen3:4b`

Or switch to the Embedding model for testing:

`1`	`ollama run embeddinggemma "测试一段文本"`

Model files remain local and will not be re-downloaded when switching. The weights need to be put into video memory or system memory when loading the model for the first time. If the model is still retained in memory, it will be faster to call it again.

`ollama ps`: First see who occupies the video memory

When the switching is not smooth, first execute:

`1`	`ollama ps`

It lists models that are running or still resident in memory. The most interesting things to look at here are the model name, footprint, processor location, and expiration time.

If you only have a medium-sized graphics card, and you launch two large models one after another, Ollama may uninstall the first model to make room for the second one. This is normal resource scheduling, not that the model is lost. The model is still on disk and will be reloaded on the next call.

If you don’t want to wait for it to expire naturally, you can stop it proactively:

`1`	`ollama stop qwen3:8b`

Then use ollama ps to confirm that the video memory has been released.

Default dwell time: 5 minutes

Ollama defaults to retaining a model for approximately 5 minutes after it was last used. This design is suitable for continuous questions: the first loading is slightly slower, and subsequent requests do not need to repeatedly move the same model back to the video memory.

If you use multiple models in turn on a machine with small video memory, 5 minutes may actually cause the feeling of “just finished running one, and the video memory has not been returned yet”. There are three control methods at this time.

Method 1: Uninstall immediately after one call

Set keep_alive to 0 via API:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "用一句话解释 KV cache",
  "keep_alive": 0
}'

This is suitable for situations where the model is large, video memory is tight, and each task is independent.

Method 2: Keep the model resident for a long time

If you are using the same code model all day, you can set keep_alive to a negative number:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "keep_alive": -1
}'

This way the model will remain in memory until manually ollama stop or the service is restarted. Don’t set this up for multiple large models when you don’t have enough video memory.

Method 3: Globally modify the default resident time

OLLAMA_KEEP_ALIVE can be set for Ollama services. For example, if you want all models to be retained for 30 seconds by default:

`1`	`OLLAMA_KEEP_ALIVE=30s`

Under Windows, Ollama inherits user or system environment variables. Once setup is complete, you need to exit Ollama from the tray and restart from the Start menu. If Ollama is managed by systemd on Linux, set the service environment variable and restart the service.

keep_alive in the API request will override the global OLLAMA_KEEP_ALIVE, so it is more suitable to set different policies for different tasks.

The key to multi-model persistence: `OLLAMA_MAX_LOADED_MODELS`

OLLAMA_MAX_LOADED_MODELS is used to limit the number of models that can be loaded simultaneously. For example, you only want the service to retain at most one model:

`1`	`OLLAMA_MAX_LOADED_MODELS=1`

The purpose of this setting is to avoid filling up the video memory for a long time when the model is rotated, but it is not a method of “forcing a large model to fit into the graphics card”. During GPU inference, a new model can reside concurrently with other models only if it can be completely fit into the available video memory. Otherwise Ollama will unload the old model, or put the model to a slower memory path.

For a single 8GB, 12GB or 16GB graphics card, the more stable strategy is usually:

scene	suggestion
Only use one chat model every day	`OLLAMA_MAX_LOADED_MODELS=1`, keep for 5 minutes or less
Small model chat + Embedding	First check the actual occupancy of both. If you can put them down at the same time, then increase the quantity.
Code model and general model are used alternately	Do not insist on dual resident, switch according to tasks and take the initiative `ollama stop`
Server multi-user call	Combine the model size, video memory, and request volume, and then set the concurrency and queue

Don’t ignore concurrency: contexts eat memory too

Multi-model problems are not just about model weights. Each parallel request increases the resource consumption of the context and KV cache.

Ollama also provides two related environment variables:

1
2

OLLAMA_NUM_PARALLEL=1
OLLAMA_MAX_QUEUE=512

OLLAMA_NUM_PARALLEL controls the number of requests that can be processed in parallel by the same model. When the number of concurrency increases, the required resources will increase with the context length. When using a single card locally, it is often easier to troubleshoot by leaving the default or explicitly setting it to 1; do not load multiple models while increasing concurrency.

OLLAMA_MAX_QUEUE is the number of requests that can be queued when busy. It only solves queuing and does not increase video memory.

Use Modelfile to create fixed aliases for different purposes

If you always set system prompt words, temperature, or context policies repeatedly for the same base model, you can use the Modelfile to create multiple local aliases. Instead of duplicating an entire set of weights, they define different configurations based on a model.

For example, create a code-biased configuration file Modelfile.code:

1
2
3

FROM qwen3:8b
SYSTEM 你是一个中文编程助手。先说明修改思路，再给出可运行的最小代码。
PARAMETER temperature 0.2

Create a model alias:

`1`	`ollama create qwen3-code -f Modelfile.code`

Then run directly:

`1`	`ollama run qwen3-code`

You can also make a writing-oriented version:

1
2
3

FROM qwen3:8b
SYSTEM 你是中文写作助手，回答前先给结论，再给必要的结构化说明。
PARAMETER temperature 0.7

1
2

ollama create qwen3-write -f Modelfile.write
ollama run qwen3-write

Note: qwen3-code and qwen3-write, although sharing the same base orientation, are still different model configurations at runtime. When video memory is tight, don’t assume they can be resident indefinitely at the same time.

Give scripts or APIs to switch models by task

In the API, the model name itself is the routing field. The script does not need to restart the service, just pass in different model according to the task:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3-code",
  "messages": [
    {"role": "user", "content": "解释这段 Python 的异常处理逻辑"}
  ],
  "stream": false,
  "keep_alive": "10m"
}'

A common division of labor is:

Small model: classification, rewriting, summary, simple question and answer;
Code model: interpret the warehouse, generate scripts, and fix errors;
Embedding model: vector retrieval, not responsible for chatting;
Larger models: complex problems, loaded on demand, released after use.

Writing “which model to choose” in the task routing of the code makes it easier to control speed and memory than cramming all the work on one large model.

Common pitfalls under Windows

The environment variable was changed but it didn’t take effect

After modifying OLLAMA_KEEP_ALIVE, OLLAMA_MAX_LOADED_MODELS, or OLLAMA_MODELS on Windows, you must exit the running Ollama tray program and restart. Simply reopening PowerShell is usually not enough.

Model files crowd the system disk

You can set OLLAMA_MODELS to move the model directory to another disk, for example:

`1`	`OLLAMA_MODELS=D:\OllamaModels`

Save and restart Ollama. Before migrating existing models, confirm the disk space and directory permissions. Do not delete the old directory just to change the volume.

I thought the model was deleted

After switching, you can’t see the old model with ollama ps, which just means that it has been unloaded from the memory; you can still see the downloaded model with ollama ls. Local model files will be deleted only by executing the following command:

`1`	`ollama rm qwen3:8b`

A set of default strategies suitable for a single card

If you only have one consumer-grade graphics card and frequently switch between multiple models, you can start with this strategy:

1
2
3

OLLAMA_MAX_LOADED_MODELS=1
OLLAMA_KEEP_ALIVE=2m
OLLAMA_NUM_PARALLEL=1

Then call the qwen3-code, qwen3-write, or Embedding model on a per-task basis. When you need to run a large model, execute ollama stop first to stop unnecessary models; when encountering long contexts or large files, lower the number of concurrency and resident numbers.

The goal of this configuration is not to allow the graphics card to load as many models as possible at the same time, but to allow each switch to be predictable and the graphics memory will not be filled up with models that will be forgotten for a long time.

Summarize

There are only four core commands for Ollama multi-model switching:

ollama ls
ollama run <模型名>
ollama ps
ollama stop <模型名>

If you want to better manage models for different purposes, use Modelfile to create aliases; if you want to control loading and release, use keep_alive, OLLAMA_KEEP_ALIVE and OLLAMA_MAX_LOADED_MODELS. First determine the number of simultaneous residents according to the video memory, and then consider multi-model routing and concurrency. The configuration will be much more stable.

refer to:

Model Management on KnightLi Blog