Local speech recognition Whisper NAS deployment: faster-whisper transliteration tutorial

The most suitable scenario for deploying Whisper to NAS is not to pursue “real-time conference subtitles”, but to put home videos, interviews, course recordings, podcasts, and conference files into a private directory to generate transcripts and SRT subtitles on demand or in batches.

For most NAS, it is recommended to start with faster-whisper. It uses the CTranslate2 inference engine, can use CPU or CUDA GPU, and supports quantization, VAD silence filtering, and word-level timestamping. First use a small model to run through the process, and then decide whether to upgrade the model based on accuracy and waiting time.

Let’s talk about the conclusion first

The safest route to NAS local speech recognition is:

1
2
3
4
5
6

准备输入 / 输出目录
-> 安装 ffmpeg 与 faster-whisper
-> 用短音频跑 small 或 base
-> 检查 TXT 和 SRT
-> 开启 VAD 过滤
-> 再做批量任务、GPU 加速或 Docker 封装

NAS without discrete graphics can also transcribe, but it should be treated as an offline task. If you want to frequently transcribe long videos, record multiple people, or pursue faster response, the role of the GPU is usually more obvious than simply increasing the hard drive capacity.

How to choose a model

Whisper comes in different size models. Larger models are generally better at handling accents, noise, and complex context, but have slower inference and higher memory requirements.

Don’t judge the effect just by the model name. Recording clarity, speaker overlap, background music, microphone distance, and proper nouns often affect the final transcript more than moving up from small to a larger model.

Directory planning: separate input, output and model cache

Take Linux NAS as an example:

1
2
3

mkdir -p ~/asr/inbox
mkdir -p ~/asr/out
mkdir -p ~/asr/models

Suggested agreement:

1
2
3

~/asr/inbox/   放 MP3、M4A、WAV、MP4 等输入文件
~/asr/out/     放 TXT、SRT、JSON 等结果
~/asr/models/  放下载的模型缓存

Batch tasks are most afraid of mixing input, output and model cache in the same directory. After separation, backup, cleanup and permission control are easier.

Installation environment

In a Debian/Ubuntu-like NAS or Linux container, first install the basic tools:

1
2

sudo apt update
sudo apt install -y python3 python3-venv ffmpeg

Create an independent virtual environment:

1
2
3
4

python3 -m venv ~/asr/.venv
source ~/asr/.venv/bin/activate
pip install --upgrade pip
pip install faster-whisper

ffmpeg is used to read common audio and video formats. Without it, MP4, M4A, or some encoding formats may not be transcoded or read properly.

Minimal transliteration script

Write the following code in ~/asr/transcribe.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

from pathlib import Path
from faster_whisper import WhisperModel

input_file = Path("/home/USER/asr/inbox/sample.mp3")
output_dir = Path("/home/USER/asr/out")
output_dir.mkdir(parents=True, exist_ok=True)

model = WhisperModel(
    "small",
    device="cpu",
    compute_type="int8",
    download_root="/home/USER/asr/models",
)

segments, info = model.transcribe(
    str(input_file),
    language="zh",
    vad_filter=True,
    word_timestamps=True,
)

segments = list(segments)

lines = []
for segment in segments:
    text = segment.text.strip()
    lines.append(text)
    print(f"[{segment.start:.2f} -> {segment.end:.2f}] {text}")

(output_dir / f"{input_file.stem}.txt").write_text(
    "\n".join(lines), encoding="utf-8"
)

Change /home/USER in the code to your actual Linux user directory, then run:

1
2

source ~/asr/.venv/bin/activate
python ~/asr/transcribe.py

When running for the first time, faster-whisper will download the corresponding model. After the model is successfully downloaded, it will be saved in the download_root you specified, and there is no need to download it again in the future.

CPU NAS recommended parameters

When there is no GPU, priority is given to controlling the model size and calculation accuracy:

1
2
3
4
5
6

model = WhisperModel(
    "small",
    device="cpu",
    compute_type="int8",
    download_root="/home/USER/asr/models",
)

int8 is often suitable as a starting point for CPU inference. If the NAS also provides file synchronization, photo indexing, downloading or media services, it is recommended that:

• Start with base or small;
• Avoid running multiple transcription tasks at the same time;
• Long videos are scheduled during off-peak hours;
• Observe system memory, CPU temperature and swap;
• Do not allow the NAS to perform large amounts of transcoding, verification, and transcoding at the same time.

The goal of a CPU NAS should be “stable completion” rather than catching up with audio playback progress in real time.

How to configure when there is NVIDIA GPU

When your NAS or Linux host has an NVIDIA GPU available, try:

1
2
3
4
5
6

model = WhisperModel(
    "small",
    device="cuda",
    compute_type="float16",
    download_root="/home/USER/asr/models",
)

When memory is tight, you can try:

1
2
3
4
5
6

model = WhisperModel(
    "small",
    device="cuda",
    compute_type="int8_float16",
    download_root="/home/USER/asr/models",
)

The actual compute_type available depends on CTranslate2, CUDA, driver and GPU. Verify with short audio first, then process long files. If you put it into Docker, you need to first confirm that the container can see the GPU; when nvidia-smi is not available in the container, the transcriber will not receive CUDA acceleration.

Why is it recommended to turn on VAD filtering

VAD (Voice Activity Detection) skips segments that are obviously speechless. For conference recordings, long pauses, background music, or files with a lot of blank space at the beginning and end of the recording, there are usually two advantages:

• Reduce meaningless reasoning time;
• Reduce the probability of silent segments being recognized as irrelevant text.

Enabled in faster-whisper:

1
2
3
4

segments, info = model.transcribe(
    "input.mp3",
    vad_filter=True,
)

But VAD is not a “never fail” switch. When the voice is very soft, music and human voices are mixed, or the phone recording is intermittent, you should randomly check whether the VAD has accidentally cut off the valid content.

Generate SRT subtitles

Each segment of Whisper has a start and end time. A simple SRT can be output based on the above script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

def srt_time(seconds: float) -> str:
    milliseconds = round(seconds * 1000)
    hours, milliseconds = divmod(milliseconds, 3_600_000)
    minutes, milliseconds = divmod(milliseconds, 60_000)
    seconds, milliseconds = divmod(milliseconds, 1000)
    return f"{hours:02}:{minutes:02}:{seconds:02},{milliseconds:03}"

blocks = []
for index, segment in enumerate(segments, start=1):
    blocks.append(
        f"{index}\n"
        f"{srt_time(segment.start)} --> {srt_time(segment.end)}\n"
        f"{segment.text.strip()}\n"
    )

(output_dir / f"{input_file.stem}.srt").write_text(
    "\n".join(blocks), encoding="utf-8"
)

If you want to do word-by-word highlighting or finer editing, you can read the word-level timestamp after word_timestamps=True. However, word-level results also require manual spot checks, especially mixed Chinese and English, names of people, places and abbreviations.

Do not run full batch transfers in one go

It is recommended to process batch files serially or with low concurrency by queue:

1
2
3

for file in ~/asr/inbox/*; do
  python ~/asr/transcribe.py "$file"
done

Logic for extension filtering, success/failure logging, and skipping completed files should be added to the actual script. The most important thing is: process 5 samples first, confirm that the output directory, language, model and subtitle timeline are correct before starting the entire batch of tasks.

Blindly concurrently running multiple large-v3 transfer tasks on the NAS can easily cause memory, GPU memory, or heat dissipation to become bottlenecks. If the transcription service also shares resources with photo albums, backups, and downloaders, the queue should be limited.

How to use Docker stably

Many NAS are better suited to managing services with Container Manager or Docker. It is recommended to run the Python version on the host or an ordinary Linux container first, and then package the container. When containerizing, at least:

• Mount the input directory read-only;
• Mount the output directory separately and writably;
• Mount the model cache directory persistently;
• Fixed faster-whisper, CTranslate2 and Python versions;
• If using a GPU, first verify container runtime and device passthrough.

Do not mount the entire NAS shared directory into the transcoding container in read-write mode. The speech recognition task only requires reading audio and video and writing text results. The smaller the permissions, the smaller the impact of misoperation.

Accuracy and privacy boundaries

The advantage of local deployment is that the audio does not have to be uploaded to a third-party service, but this does not mean that the output is naturally accurate. Whisper may mishear names, jargon, numbers, and silent segments, as well as unreliable text when exposed to noise or long whitespace.

The following content should be reviewed manually:

• medical, legal, financial and security records;
• Subtitles released to the public;
• Interview quotes and meeting resolutions;
• Name, amount, date, phone number and product model.

Keeping the original audio and treating the transcript as a first draft is a safer process than “delete the original file immediately after transcribing.”

A set of default configurations suitable for NAS

If you don’t have a discrete graphics card:

1
2
3
4
5

model: base 或 small
device: cpu
compute_type: int8
vad_filter: true
并发: 1

If you have an NVIDIA GPU available:

1
2
3
4
5

model: small 起步，按效果再试 medium
device: cuda
compute_type: float16 或 int8_float16
vad_filter: true
并发: 从 1 开始压测

First use the same 10 to 30-minute real recording to compare the time-consuming, omitted words, typos and subtitles timeline before deciding whether to upgrade the model. Don’t just rely on a few seconds of clear samples to judge whether a NAS is suitable for long-term transcription.

Summarize

Whisper NAS deployment can start with faster-whisper: small model, low concurrency, VAD filtering, input and output directory separation. CPU NAS can handle low-frequency offline transcription; GPU NAS with a correct CUDA environment is more suitable for a large number of videos and larger models.

What really determines the experience is not “Whisper can be installed”, but your audio quality, model size, actual transcoding queue and how many other services the NAS is responsible for. Run through a piece of real audio first, and then scale it up.

refer to:

• faster-whisper official warehouse
• OpenAI Whisper official repository
• NAS deployment Ollama performance is not enough