Incrementally Sync Two Large Directories with rsync: Resume and Verification Guide

When using rsync to copy two large directories, the safest way is not to write the two source paths into one command, but to first clarify which destination directory each source directory corresponds to, and then execute it group by group:

1
2

/mnt/disk1/photos/   → /mnt/disk2/photos/
/mnt/disk1/archive/  → /mnt/disk2/archive/

Preview each directory pair with --dry-run first. During the actual transfer, rsync retains partial files and writes logs. After copying finishes, run a read-only comparison.

If there is no output from the standard comparison, it means that rsync does not find any differences based on file size, modification time and related attributes; if you want to confirm that the file contents are consistent, add --checksum for full read verification.

A safe rsync setup for two directory pairs

The following assumes that both the source drive and the destination disk are mounted correctly:

1
2
3
4

SRC1=/mnt/disk1/photos
DST1=/mnt/disk2/photos
SRC2=/mnt/disk1/archive
DST2=/mnt/disk2/archive

First create the destination directory:

1

sudo mkdir -p "$DST1" "$DST2"

Preview the synchronization of the two groups separately:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

sudo rsync -aHAXn \
  --numeric-ids \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

sudo rsync -aHAXn \
  --numeric-ids \
  --itemize-changes \
  "$SRC2/" \
  "$DST2/"

After confirming the source paths, destination paths, and change lists, remove -n and run the two actual synchronization jobs sequentially:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

sudo rsync -aHAX \
  --numeric-ids \
  --partial \
  --partial-dir=.rsync-partial \
  --info=progress2,stats2 \
  --log-file=/var/log/rsync-photos.log \
  "$SRC1/" \
  "$DST1/"

sudo rsync -aHAX \
  --numeric-ids \
  --partial \
  --partial-dir=.rsync-partial \
  --info=progress2,stats2 \
  --log-file=/var/log/rsync-archive.log \
  "$SRC2/" \
  "$DST2/"

It is recommended not to add --delete during the first migration.

This way, even if there are extra files in the destination directory, rsync will not delete them. After the directory structure and verification results are confirmed, decide whether you need to make the target a complete mirror of the source.

Why each directory pair should run separately

The final positional argument to rsync is the destination; every preceding path is treated as a source.

This way of writing:

1

rsync -a /source/a/ /source/b/ /target/

It means that both sources are imported into the same /target/, but it does not mean that /source/a/ corresponds to /target/a/ and /source/b/ corresponds to /target/b/.

When two directories need to fall into two target paths respectively, rsync should be executed twice, or an explicit serial script should be written.

There are several practical benefits to running separate operations:

• Each directory has independent logs and exit status;
• When one group fails, the results of the other group are not obscured;
• Differences can be located group by group during verification;
• It allows the mechanical hard disk to read and write sequentially to avoid two tasks competing for the magnetic head;
• Different exclusion rules can be set for different directories.

If two sources really want to import the same target, first confirm whether they contain relative paths with the same name.

For example, two sources have 2026/report.pdf, and the source executed later may overwrite the file with the same name executed first. It is safer to keep separate subdirectories for them at this time.

Verify mount points before copying

One of the most dangerous mistakes in large directory migration is that the destination disk is not mounted successfully, but the target path still exists.

At this time, rsync will write the data into the ordinary directory in the root file system until the system disk is full.

Check block devices, file systems, and mount points before starting:

1
2
3
4
5
6

lsblk -o NAME,SIZE,FSTYPE,UUID,MOUNTPOINTS
findmnt -T /mnt/disk1/photos
findmnt -T /mnt/disk2/photos
findmnt -T /mnt/disk1/archive
findmnt -T /mnt/disk2/archive
df -hT /mnt/disk1 /mnt/disk2

Key points to confirm:

1. /mnt/disk1 and /mnt/disk2 correspond to the expected devices;
2. The destination disk has enough free space;
3. The target mount is not in a read-only state;
4. The two paths do not accidentally point to the same file system location;
5. The destination path is not a subdirectory within the source path.

You can use the following command to view the parsed real path:

1
2
3
4

readlink -f "$SRC1"
readlink -f "$DST1"
readlink -f "$SRC2"
readlink -f "$DST2"

Do another small write test:

1
2
3
4
5

sudo touch "$DST1/.rsync-write-test"
sudo rm "$DST1/.rsync-write-test"

sudo touch "$DST2/.rsync-write-test"
sudo rm "$DST2/.rsync-write-test"

Here only delete the test files you just created explicitly, do not use wildcards to clean the destination directory.

How trailing slashes change the directory layout

The results of these two commands are different:

1
2

rsync -a /source/ /target/
rsync -a /source  /target/

The first copy of the content in /source/:

1
2

/target/file1
/target/subdir/file2

The second copy copies the source directory itself:

1
2

/target/source/file1
/target/source/subdir/file2

When synchronizing two corresponding directories that have been established, / is usually written after the source path and the target path:

1

rsync -a "$SRC1/" "$DST1/"

Don’t just guess the result based on the directory name. Before executing for the first time, use --dry-run --itemize-changes to check the relative path that will be created.

What each recommended option does

The basic option set is:

1

-aHAX

in:

If you are copying ordinary documents, photos, and videos, and the source directory does not have hard links, ACLs, or extended attributes, you can omit unneeded options as needed.

Don’t mechanically think that the more options, the safer it is.

For example, when the target is FAT, exFAT, or some network file systems, permissions, owners, ACLs, and xattr may not be saved completely, and -A, -X, or owner settings may report errors.

First check the file system types at both ends:

1
2

findmnt -no FSTYPE -T "$SRC1"
findmnt -no FSTYPE -T "$DST1"

If the capabilities of the two ends are different, you should first clarify whether the file content or the complete Linux metadata needs to be retained.

What rsync copies during an incremental run

By default, rsync mainly determines whether a file needs to be updated based on the file size and modification time.

When running the same command again:

• New files will be copied;
• Files that change size or modification time will be updated;
• Files that have not changed will be skipped;
• Target-unique files are retained;
• Interrupted files are rechecked according to their current status and processing continues.

This is why incremental synchronization is suitable for large directory migrations.

But it is not a file system snapshot.

If the source directory continues to change during the copy process, the final destination may contain file status at different points in time. Ordinary photo libraries or archive directories can run an incremental complement after the first copy; databases, virtual machine images, and application data being written should first stop services, freeze writing, or use file system snapshots.

How to read the dry-run change list

The recommended preview command is:

1
2
3
4
5

sudo rsync -aHAXn \
  --numeric-ids \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

-n is the short form of --dry-run. It only calculates the plan and does not write it to the file.

--itemize-changes will display each change item and the reason. Common output forms include:

1
2
3
4

>f+++++++++ new-file.iso
>f..t...... changed-time.log
.d...p..... existing-directory/
*deleting   old-file.tmp

It can be roughly understood like this:

• >f+++++++++: The file does not exist in the target and will be created;
• Starting with >f and followed by an attribute tag: the file content or attributes will be updated;
• .d: The properties of the directory itself need to be adjusted;
• *deleting: When combined with --delete, the target’s unique content will be deleted.

If an unexpected top-level directory name appears in the preview, it is usually because / is written incorrectly at the end of the source path.

If almost all large files appear to be retransmitted, you should check the time on both ends, file system time accuracy, mount options, and whether the file is being continuously rewritten by the application.

A serial script for both directory pairs

When you need to run it repeatedly, you can save it as /usr/local/sbin/sync-two-dirs.sh:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/usr/bin/env bash
set -Eeuo pipefail

sync_one() {
  local name=$1
  local source=$2
  local target=$3
  local log_file="/var/log/rsync-${name}.log"

  printf '开始同步 %s: %s -> %s\n' "$name" "$source" "$target"

  rsync -aHAX \
    --numeric-ids \
    --partial \
    --partial-dir=.rsync-partial \
    --info=progress2,stats2 \
    --log-file="$log_file" \
    "$source/" \
    "$target/"

  printf '完成同步 %s\n' "$name"
}

sync_one photos /mnt/disk1/photos /mnt/disk2/photos
sync_one archive /mnt/disk1/archive /mnt/disk2/archive

Give execute permissions and run:

1
2

sudo chmod 750 /usr/local/sbin/sync-two-dirs.sh
sudo /usr/local/sbin/sync-two-dirs.sh

The script uses set -Eeuo pipefail.

If the first set of rsync returns a non-zero status, the script will stop and the overall task will not be falsely reported as successful. Just rerun after handling the error. Already consistent files will be skipped.

Run long transfers inside tmux

This can take hours or even days for several terabytes of data. It is recommended to execute in tmux instead of relying on an SSH window that is always open:

1

tmux new -s rsync-copy

After entering the session run:

1

sudo /usr/local/sbin/sync-two-dirs.sh

Press Ctrl+b, then d to continue the session in the background.

Reconnect:

1

tmux attach -t rsync-copy

At the same time, you can observe the logs from another terminal:

1
2

sudo tail -f /var/log/rsync-photos.log
sudo tail -f /var/log/rsync-archive.log

Check the progress:

1

pgrep -af rsync

Do not start multiple identical scripts at the same time. When two tasks write to the same mechanical hard drive, parallel copying usually only increases random seek and wait times.

Check the rsync exit status and logs

After the command ends, first check the exit status:

1

echo $?

0 means that no errors were detected in this run, but it cannot independently prove that the contents of each file have been completely verified.

Common non-zero states include:

Look at the end of the log:

1
2

sudo tail -n 100 /var/log/rsync-photos.log
sudo tail -n 100 /var/log/rsync-archive.log

Don’t just look at the “Total Bytes” that appears at the end of the terminal. If IO error, Permission denied, failed or vanished file exists in the log, it still needs to be determined whether it affects verification.

Verification level 1: repeat the dry run

After actual transfer, run the following two sets of directories:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

sudo rsync -aHAXn \
  --numeric-ids \
  --delete \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

sudo rsync -aHAXn \
  --numeric-ids \
  --delete \
  --itemize-changes \
  "$SRC2/" \
  "$DST2/"

-n is also used here, so --delete will only report excess content on the target side and will not actually delete it.

No output usually means:

• rsync found no files that need to be added or updated;
• There are no additional paths on the destination that do not exist on the source;
• There are no differences in the common attributes required to be retained this time.

If the output is only .rsync-partial/, first confirm that there are no unfinished tasks. After all synchronization and verification are completed, decide whether to clean up the temporary directory.

If the output contains *deleting, there are files in the destination that are not in the source directory. Check first whether these files should be retained and do not remove -n immediately.

Verification level 2: compare file contents with checksums

Ordinary dry-run mainly relies on size and modification time, and cannot exclude the rare situation of “the same size and time but different content”.

Strict verification can be used:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

sudo rsync -aHAXnc \
  --numeric-ids \
  --delete \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

sudo rsync -aHAXnc \
  --numeric-ids \
  --delete \
  --itemize-changes \
  "$SRC2/" \
  "$DST2/"

Where -c is equivalent to --checksum.

It reads the file contents from both sides and calculates a checksum instead of just comparing the size and modification time. Because it also contains -n, when differences are found, they will only be reported, not corrected, and the files will not be deleted.

Executing -c on terabytes of data can take a long time and create a full read load on the source and destination disks.

Scenarios suitable for strict verification include:

• An over-read error occurred on the original disk or the disk was dropped;
• Migrating through unstable networks;
• The file content is very important and there is no other verification mechanism;
• Final verification needs to be completed before the source drive is taken offline;
• The destination file system or memory is suspected to be causing silent corruption.

If the disk health is not good, check the SMART and kernel logs first, and do not read the failing disk repeatedly for verification purposes.

Verification level 3: compare file counts and logical bytes

Count the number of ordinary files:

1
2
3
4
5

find "$SRC1" -xdev -type f -printf '.' | wc -c
find "$DST1" -xdev -type f -printf '.' | wc -c

find "$SRC2" -xdev -type f -printf '.' | wc -c
find "$DST2" -xdev -type f -printf '.' | wc -c

Statistics of the total logical size of ordinary files:

1
2
3
4
5

find "$SRC1" -xdev -type f -printf '%s\n' | awk '{sum += $1} END {printf "%.0f\n", sum}'
find "$DST1" -xdev -type f -printf '%s\n' | awk '{sum += $1} END {printf "%.0f\n", sum}'

find "$SRC2" -xdev -type f -printf '%s\n' | awk '{sum += $1} END {printf "%.0f\n", sum}'
find "$DST2" -xdev -type f -printf '%s\n' | awk '{sum += $1} END {printf "%.0f\n", sum}'

-xdev means not to enter other file systems mounted in the directory.

If the source directory originally contains nested mount points, and these contents should also be copied, you cannot use -xdev directly, and you must confirm that the mount point processing policy of rsync meets expectations.

The equal number of files and the number of logical bytes can only be used as auxiliary evidence, but cannot prove that the path and content are consistent one by one.

du -sh cannot be used as the only basis for verification, because sparse files, compression, deduplication, block size, hard links and file system metadata will all affect the actual space occupied.

Generate SHA-256 manifests for independent verification

If you want to keep an auditable list of content, you can calculate SHA-256 on relative paths inside each directory:

1
2
3
4
5
6
7

cd "$SRC1"
find . -type f -print0 | sort -z | xargs -0 -r sha256sum > /tmp/photos-source.sha256

cd "$DST1"
find . -type f -print0 | sort -z | xargs -0 -r sha256sum > /tmp/photos-target.sha256

diff -u /tmp/photos-source.sha256 /tmp/photos-target.sha256

The second set of directories is processed similarly:

1
2
3
4
5
6
7

cd "$SRC2"
find . -type f -print0 | sort -z | xargs -0 -r sha256sum > /tmp/archive-source.sha256

cd "$DST2"
find . -type f -print0 | sort -z | xargs -0 -r sha256sum > /tmp/archive-target.sha256

diff -u /tmp/archive-source.sha256 /tmp/archive-target.sha256

diff has no output and the exit status is 0, indicating that the two lists are consistent.

This still only verifies normal file content and relative paths, not directory permissions, ACLs, owners, hard link relationships, or extended attributes. A complete Linux data migration should be viewed in conjunction with the results of rsync -aHAXnc.

Verify permissions, ACLs, extended attributes, and hard links

Randomly check the basic attributes of key files:

1
2

stat "$SRC1/关键文件"
stat "$DST1/关键文件"

Check ACL:

1
2

getfacl -p "$SRC1/关键目录"
getfacl -p "$DST1/关键目录"

Check extended attributes:

1
2

getfattr -d -m- "$SRC1/关键文件"
getfattr -d -m- "$DST1/关键文件"

Check the inode and link number of hard-linked files:

1
2

ls -li "$SRC1/路径/文件1" "$SRC1/路径/文件2"
ls -li "$DST1/路径/文件1" "$DST1/路径/文件2"

The inode numbers on the source and destination do not need to be the same, but files within the same end that originally belonged to a set of hard links should share inodes and retain the correct number of links.

When to enable –delete

--delete allows the target to eventually maintain a mirroring relationship with the source: paths that exist in the target but not in the source will be deleted.

Do a read-only preview first:

1
2
3
4
5
6

sudo rsync -aHAXn \
  --numeric-ids \
  --delete-delay \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

Note: --delete-delay also enables delete semantics, but defers the actual delete until the end of the transfer.

Before official mirroring, at least confirm:

1. The source and target are not reversed;
2. The target mount point is indeed online;
3. Every *deleting in the dry-run works as expected;
4. There are no files on the target that need to be kept independently;
5. There is another restoreable backup.

If you want to still be able to review the deletion, you can use the backup directory first:

1
2
3
4
5
6
7
8

sudo rsync -aHAX \
  --numeric-ids \
  --delete-delay \
  --backup \
  --backup-dir="/mnt/disk2/rsync-deleted-$(date +%F)" \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

The backup directory must be located in a location with sufficient space and must not fit into the source directory tree that will be synchronized again.

Exclude recycle bins and temporary directories

There are some common cache directories in NAS or desktop disks that do not need to be migrated. Only add exclusion rules after confirming that they are indeed discardable:

1
2
3
4
5
6
7
8
9

sudo rsync -aHAX \
  --numeric-ids \
  --partial \
  --info=progress2,stats2 \
  --exclude='#recycle/' \
  --exclude='@eaDir/' \
  --exclude='.Trash-*/' \
  "$SRC1/" \
  "$DST1/"

'#recycle/' must be enclosed in quotation marks, otherwise the Shell may regard the content after # as a comment.

The actual command and the verification command must use the same exclusion rules, otherwise the intentionally ignored content will be reported as missing during verification.

When there are many exclusion rules, save them to a file:

1
2
3

#recycle/
@eaDir/
.Trash-*/

Then use:

1
2
3
4
5

rsync -aHAXn \
  --exclude-from=/etc/rsync-large-dirs.exclude \
  --itemize-changes \
  "$SRC1/" \
  "$DST1/"

Control disk and network load

When migrating large directories on a production server, you can reduce the CPU and I/O priority of the process:

1
2
3
4
5
6
7

sudo nice -n 10 ionice -c2 -n7 \
  rsync -aHAX \
  --numeric-ids \
  --partial \
  --info=progress2,stats2 \
  "$SRC1/" \
  "$DST1/"

Cross-network transmissions can also be speed-limited. It is recommended that the remote target write the absolute path directly to avoid confusing local variables and remote paths:

1
2
3
4
5
6

rsync -aHAX \
  --partial \
  --info=progress2,stats2 \
  --bwlimit=80M \
  "$SRC1/" \
  user@backup-host:/mnt/disk2/photos/

Local disk to local disk generally does not require -z compression. Photos, videos, and compressed packages are also difficult to compress further, and turning on -z will only increase the CPU load.

Common failures and fixes

Permission denied

First, confirm in the log whether it failed to read the source file, failed to create the destination file, or failed to set the owner, ACL, or xattr.

1

sudo grep -Ei 'permission denied|operation not permitted|failed' /var/log/rsync-photos.log

Do not directly execute a broad chmod -R 777 on the entire target. The mount options, destination directory owner, and destination file system capabilities should be checked.

No space left on device

Check both capacity and inode:

1
2

df -hT "$DST1"
df -i "$DST1"

A large number of small files may exhaust the inodes first, even though df -h still shows free capacity.

Input/output error

View kernel and disk status now:

1
2

dmesg -T | tail -n 100
sudo smartctl -a /dev/sdX

Replace /dev/sdX with the actual device. When persistent read errors occur, you should first protect the data that can still be read, and do not repeatedly perform full disk verification to increase the pressure on the failed disk.

file has vanished

Status code 24 usually indicates that the source file was moved or deleted after scanning but before copying.

It’s not uncommon to see this happen occasionally in caches, logs, and download directories. For business directories that require consistency, you should pause writing or copying from a snapshot, then rerun and confirm that missing files no longer appear.

Every file is transferred again

Common reasons include:

• The destination file system cannot preserve the source time precision;
• The application rewrites the file after copying it;
• Abnormal source and destination clocks;
• Use a mounting method that changes file attributes;
• The files are re-encoded or repackaged through an intermediary program.

First use --dry-run --itemize-changes to observe whether content, size, time or permissions trigger the update.

Final verification checklist

After completing the migration of two large directories, retain at least the following evidence:

• Both actual rsync commands return 0;
• There are no outstanding I/O, permissions, or space errors in both logs;
• There were no unexpected outputs from both sets of normal dry-runs;
• If mirroring is required, dry-run with --delete has no accidental deletions;
• File counts and total logical bytes match between source and destination;
• Important data completes rsync -nc or independent SHA-256 verification;
• Key files can actually be opened from the target;
• Permissions, ACLs, xattr, and hard links are retained as per operational requirements;
• The source drive has not been formatted or taken offline before verification is complete;
• If possible, complete a restore test from the destination.

The most practical confirmation command is still:

1
2
3
4
5
6

sudo rsync -aHAXnc \
  --numeric-ids \
  --delete \
  --itemize-changes \
  /源目录/ \
  /目标目录/

When it has no output, it means that rsync found no content, path or related attribute differences within the current exclusion rules and options.

For terabytes of data, it is normal for strict verification to be slow. Do not delete the source data immediately just because the copy command displays 100%; copy completion, verification passing and restore verification are three different stages.

FAQ

Do I need to add a special “resume” command after an interruption?

Usually not required. Just rerun the same rsync command. --partial --partial-dir=.rsync-partial will retain incomplete data, and files that are already consistent will be skipped.

Can two large directories be copied in parallel?

Technically yes, but parallel tasks tend to reduce throughput if they are on the same source drive or the same target HDD. Default serial makes logging, retrying, and verification easier.

What is the difference between --progress and --info=progress2?

--progress prefers to display the current file progress, while --info=progress2 displays the overall progress of the entire transfer task. Very large directories are usually easier to view.

Why does du -sh differ between the two sides after copying?

File system block size, sparse files, compression, hard links, metadata, and reserved blocks can all cause occupancy differences. Prioritize rsync comparison, file list and content verification, don’t just look at du.

Will rsync -nc modify destination files?

Won’t. -n is a dry-run, and -c only requires the difference to be determined based on the file content checksum. The command does a bulk read from the disk but does not perform a copy or delete.

Why include --delete during verification?

Because only the files that exist in the source are compared, the extra paths on the target side cannot be found. When used together with -n, --delete only reports these extra contents and will not actually delete them.

When can the source directory be deleted?

At least wait until the two sets of synchronization exit status are normal, the logs have no unresolved errors, strict verification passes, and key files are successfully opened or restored from the target end. Another independent backup should be kept of important data.