DeepSeek V4 Local Deployment: Pro vs. Flash Memory, Hardware, and API Choice

DeepSeek V4 cannot be understood in the same way as ordinary 7B and 32B models. The two officially announced versions are both MoE models:

• DeepSeek-V4-Pro: About 1.6T total parameters, each token activates about 49B parameters;
• DeepSeek-V4-Flash: About 284B total parameters, each token activates about 13B parameters.

“Few activation parameters” mainly affects the amount of single-step calculations, and it does not mean that you only need to prepare VRAM for activation parameters. If the complete weights are involved in routing, all expert weights still need to be stored. For a single consumer graphics card, the official API is often much more realistic than a full on-premises deployment.

Separate official facts from estimates

The official release notes confirm the model name, overall parameters, activation parameters, context, and API changes. The low bit volume below is a mathematical estimate, not an actual measured value of an official GGUF file, nor does it mean that the community has provided a quantization package that can be directly run.

The most basic weight volume formula is:

1

权重体积（GiB）≈ 参数量 × 每参数位数 ÷ 8 ÷ 1024³

quantization will also increase grouping scale, metadata and alignment overhead; inference also requires KV cache, runtime buffer and communication space, so the actual memory must be higher than the bare weight.

Theoretical weight size

These numbers only answer “what is the weight at least?” For example, the 4-bit weight of Flash is theoretically about 132 GiB. In actual deployment, space must be reserved for quantified metadata, KV cache, and back-end buffering. Even if Pro is 4-bit, it is already in the range of multi-machine or high-end multi-card servers.

Why context consumes additional memory

KV cache is related to the following factors:

• Number of context tokens;
• Number of concurrent requests and batch size;
• KV cache data type;
• Number of model layers, attention structure and backend implementation;
• Whether to enable prefix caching or other optimizations.

Therefore, “supporting 1M context” does not mean that the local computer must be started with 1M, nor does it mean that the VRAM remains unchanged. Deployment evaluation should start with a 4K or 8K context, single concurrency, and increase and record peaks.

Which hardware is worth testing?

“Can be loaded” and “can be used interactively” are two different things. A large amount of CPU offload may make the first token and generation speed so slow that it is of no practical value.

Minimal validation using official API

DeepSeek official description V4 provides OpenAI Chat Completions compatible calls. First save the key in the environment variable, and then send the minimum request:

1
2
3
4
5

$env:DEEPSEEK_API_KEY = "你的密钥"
curl.exe https://api.deepseek.com/chat/completions `
  -H "Authorization: Bearer $env:DEEPSEEK_API_KEY" `
  -H "Content-Type: application/json" `
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"只回复 OK"}]}'

should return JSON on success and see the helper message in choices in the response. If 401 is returned, check the key first; if the returned model does not exist, check the official model list and do not use the name in the old article and try again.

The official announcement pointed out that the old deepseek-chat and deepseek-reasoner have entered the migration process after V4 went online. Model names, prices, and retirement dates are subject to change, and production configurations must be subject to current API documentation.

Checklist for local quantization package

If V4 quantified weight appears in the community in the future, at least check:

1. Whether the upstream points to the DeepSeek official weight warehouse;
2. Whether the file covers all shards and experts;
3. Whether the quantification algorithm, calibration data and inference backend are made public;
4. Does the backend clearly support the MoE architecture and not just be able to read the file header;
5. Does the test result indicate GPU, CPU, RAM, context, concurrency and tokens/s;
6. Whether the hash, license and download source are verifiable.

When there are no real files and back-end tests, you can only publish a “capacity estimate” and cannot write “A certain graphics card can run according to actual tests.”

Confirm what you actually obtained

Before discussing local V4 deployment, classify the download into four categories:

1. Official complete weight;
2. Official API model name;
3. Third party quantification or conversion of weights;
4. Only unofficial derived models with similar names.

The API name cannot be used to prove that the weights have been exposed; nor can a Hugging Face repository prove that it contains the full V4 based on the title alone. If the file is only a few GB, it is more likely an adapter, configuration, tokenizer, index, or incomplete shard.

Before downloading hundreds of gigabytes of files, check the repository file list and total size, and confirm that all shard numbers are consecutive.

Derive a server baseline from raw weights

Capacity planning cannot use “the total VRAM is exactly equal to the weight” as a solution. Taking the theoretical 4-bit value of V4 Flash as approximately 132 GiB as an example, you must also reserve:

• Quantification scale, grouping information and metadata;
• KV cache;
• CUDA/ROCm context;
• Activation and temporary calculation buffer;
• The reasoning framework is occupied by itself;
• Multi-card communication and fault tolerance margin.

Engineering often requires leaving a sizable margin rather than taking every card to 100%. The specific ratio must be confirmed by the real logs of the target framework.

Aggregate VRAM is only the first requirement

Assuming that the total VRAM of multiple GPUs can accommodate Flash, you still need to confirm:

• Whether the framework supports V4’s expert routing;
• Whether the weights can be reasonably divided by experts or tensors;
• Whether to use NVLink, PCIe or cross-machine network between cards;
• Whether there is a card responsible for additional embedding, output layers or buffering;
• Whether the slowest link drags down the entire request;
• Whether the single-machine power supply, cooling and motherboard channels are sufficient.

The addition of VRAM can only show that the capacity is possible, but it cannot prove that reasoning can be started, let alone that the speed is available.

Validate a server configuration in small steps

If you really want to study local V4, it is recommended to divide the acceptance into the following levels:

Level 1: Identify configuration

Only read the configuration and weight index and confirm that the model type, number of shards, dtype and expert parameters can be recognized by the current framework. Do not allocate all GPUs at this time.

Level 2: Load short context

Start with a single request, the shortest available context. Record the memory usage of each GPU and host to confirm that there is no silent fallback to CPU or disk mapping.

Level 3: Generate fixed short answers

Test 32–64 output tokens with deterministic prompts, observing first token latency, generation speed, error logging, and cross-card communication.

Level 4: Gradually increase context

Grow by 4K, 8K, 16K, don’t test extreme contexts directly. Restart each time and record the peak value to exclude the impact of the previous cache.

Level 5: Increase concurrency

Only test concurrency after a single request is stable. Concurrency changes caching, scheduling, and throughput, potentially making configurations available for single requests instantly OOM.

Evidence required for a credible multi-GPU benchmark

At least publicly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

模型仓库与 commit：
权重格式与量化方法：
推理框架与 commit：
GPU 型号、数量、单卡显存：
CPU、系统内存、NUMA：
卡间连接与网络：
上下文、输入/输出 token：
并发与批大小：
每卡峰值显存：
首 token 延迟、tokens/s、总吞吐：
是否使用 CPU offload 或磁盘映射：

When the model file and framework version are missing, the speed numbers cannot be reproduced; when the context and concurrency are missing, the memory numbers are meaningless for comparison.

Production acceptance tests for the official API

After the minimum request is successful, latency, usage, error handling, and model switching must also be verified.

PowerShell can record the total duration of a request:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

$headers = @{
  Authorization = "Bearer $env:DEEPSEEK_API_KEY"
  'Content-Type' = 'application/json'
}

$payload = @{
  model = 'deepseek-v4-flash'
  messages = @(
    @{ role = 'user'; content = '将这段日志归纳为三条排错建议。' }
  )
  temperature = 0
} | ConvertTo-Json -Depth 5

$elapsed = Measure-Command {
  $response = Invoke-RestMethod `
    -Uri 'https://api.deepseek.com/chat/completions' `
    -Method Post `
    -Headers $headers `
    -Body $payload
}

$elapsed.TotalSeconds
$response.usage
$response.choices[0].message.content

record usage can help check input, output and cache billing; specific fields and prices are subject to the current official API documentation.

Migrate from old model names

Don’t just replace strings in configuration files. For a complete migration check:

• Whether the new model name is available in the current account and region;
• Whether system prompt and sampling parameters are still compatible;
• Whether the streaming response event changes;
• Whether the tool call structure is correctly parsed by the client;
• Whether the maximum output, context and timeout need to be adjusted;
• Whether the old model rollback entrance is still retained.

First copy a production request for shadow testing, compare answer quality, delay, token usage and failure rate, and then gradually switch traffic.

Handle API failures by category

Key and data boundaries

The API key should be placed in an environment variable, system credentials store, or team secret manager. Don’t write it in Markdown, Git configuration, front-end JavaScript, or share screenshots.

Before sending company data to the cloud API, you must also clarify:

• Which fields contain personal information or trade secrets;
• Whether desensitization is needed;
• Will the log retain the request body or only the metrics;
• Whether team members share keys;
• Rotation and revocation process after key leakage.

If your data policy requires going completely offline, you should choose a smaller open-weight model that can be deployed stably on existing hardware, rather than pretending to be “on-premises” with an opaque third-party intermediary.

Recognize exaggerated single-GPU claims

The following statements require additional evidence:

• “49B active, so only 49B of VRAM is needed”;
• “4-bit is equal to the total parameter divided directly by two”;
• “1M context does not increase VRAM”;
• “Single card is loaded successfully, so it can run smoothly”;
• “It uses a model with the same name, so it must be an official V4”;
• “The screenshot shows 20GB, and the usage is the complete model.”

Trusted reporting must explain whether the weights are complete, how many layers are on the CPU, whether a remote API is used, and how the speed is measured.

How to choose between API and local model

The selection should be based on tasks, data boundaries, latency, and total cost, rather than just comparing parameter quantities.

Practical recommendations

• Individuals and general development teams: give priority to using the official API;
• Data cannot leave the intranet: evaluate smaller open weight models first instead of forcefully loading V4;
• Do research on reasoning framework: start with Flash, short context, single concurrency, and completely record the software and hardware;
• To see the precise single card speed or memory table: first check whether the quantitative files, backend logs and test configurations are public.

Official DeepSeek references

• DeepSeek V4 official release notes
• DeepSeek API Documentation