Skip to content

Usage

zfs-encrypt-migrate [--version] COMMAND [OPTIONS] ARGS

Run every command as root, with the host's zfs and zpool on PATH. Bare zfs-encrypt-migrate only shows help.

Command What it does
seed DATASET Copy a dataset tree to an encrypted staged tree, then unload its key.
catchup DATASET Send what changed since the seed to the staged tree.
cutover DATASET Swap the staged tree into place; the original stays read-only under STAGE/old.
tree-seed DATASET Copy only the parent to an encrypted staged dataset; its children move one by one.
tree-move DATASET CHILD Copy one child subtree under the staged parent; the source stays until you destroy it.
tree-finish DATASET Send the parent's last changes and swap the staged tree into place once no children remain.
audit [--exact NAME]... [--subtree NAME]... POOL... List unencrypted datasets outside the allowlists; exit 1 if there are any.

Shared Options

The six migration and tree commands take the same options. audit takes none of them.

Option Environment variable Default Description
--stage DATASET ZFS_ENCRYPT_MIGRATE_STAGE required Staging dataset, such as tank/stage. Never guessed.
--key-file PATH ZFS_ENCRYPT_MIGRATE_KEY_FILE /run/zfs-encrypt-migrate.key Key file for the staged tree, as an absolute path; must have mode 600.
--min-free SIZE ZFS_ENCRYPT_MIGRATE_MIN_FREE 1T Free space to keep on the stage pool after the copy.
--keyformat FORMAT ZFS_ENCRYPT_MIGRATE_KEYFORMAT passphrase Key format of the key file: passphrase, hex or raw.
--final-keylocation LOC ZFS_ENCRYPT_MIGRATE_FINAL_KEYLOCATION prompt keylocation set on the dataset after the swap.

Encryption is always aes-256-gcm.

--stage. The stage holds the staged copy at <stage>/<leaf> and the original, after the swap, at <stage>/old/<leaf>, where <leaf> is the last component of the dataset name. seed and tree-seed create <stage> and <stage>/old with canmount=off when they're missing. cutover and the tree commands rename between the dataset and the stage, so both must be on the same pool. The stage and the dataset may not overlap: every migration and tree command refuses, before any zfs command, a dataset that is the stage, contains it or lies inside it. Because staged names use only the leaf, tank/a/data and tank/b/data both stage as tank/stage/data: migrate one, destroy its old copy, then the other.

--key-file. It must be a regular file with mode 600; a symlink is followed, and its target must be that file. Only seed and tree-seed read it: they record it on the staged copy as keylocation=file://<path>, and later commands load the key from that keylocation. Keep the file in place until the swap. It must be an absolute path; a relative one is a usage error, because zfs would reject the relative file:// keylocation only after seed had taken its snapshots.

--min-free. Plain bytes, or a number with a K, M, G, T or P suffix in powers of 1024, optionally followed by B or iB, in any case: 1099511627776, 512M, 1T, 2GiB.

--final-keylocation. prompt, file:///absolute/path, or an http:// or https:// URL. It's checked up front, because zfs would only reject it at the very end of the swap.

Dataset names. DATASET, --stage and the audit's names and pools may contain letters, digits and _ . : -, separated by /, and may not start with -. CHILD is a single component without /. Anything else, such as a snapshot name with @ or a name with spaces, is a usage error before any zfs command runs.

Migration Commands

seed

zfs-encrypt-migrate seed tank/data

Copies the dataset and everything under it to an encrypted staged tree, while the dataset stays mounted and in use:

  1. zfs snapshot -r DATASET@enc-seed
  2. zfs send -R DATASET@enc-seed | zfs recv -u -o encryption=aes-256-gcm -o keyformat=FORMAT -o keylocation=file://KEY STAGE/LEAF
  3. zfs unload-key -r STAGE/LEAF

seed refuses, before it changes anything, with:

Message Fix
key file PATH not found Create the key file.
key file PATH must have mode 600 chmod 600 PATH
DATASET overlaps the stage STAGE; neither may be or contain the other Use a stage outside the dataset, and don't migrate the stage or anything under it.
dataset DATASET does not exist Check the name.
DATASET is already encrypted Nothing to do.
a migration is already in flight under STAGE Finish the migration whose staged copy is under the stage, or clean up one whose seed or tree-seed was interrupted. Once a tree-move ran, never destroy the staged parent: it holds the moved children, whose sources are gone.
STAGE/LEAF already exists A staged copy with that leaf is in flight.
STAGE/old/LEAF already exists Destroy the old copy of an earlier dataset with the same leaf first.
free space on POOL (AVAIL) minus size of DATASET (USED) is below floor MIN_FREE Free space, or lower --min-free. Sizes are in bytes; USED is the dataset's used, which includes its children and snapshots.

seed also works on a pool root, with a stage on another pool; cutover then refuses, but rebuilding a pool encrypted uses exactly that.

catchup

zfs-encrypt-migrate catchup tank/data

Sends what changed since the seed, without unmounting the dataset:

  1. zfs load-key STAGE/LEAF
  2. zfs snapshot -r DATASET@enc-final
  3. zfs send -R -I @enc-seed DATASET@enc-final | zfs recv -u -x encryption STAGE/LEAF
  4. Checks that every dataset has @enc-final with the same GUID on both sides, and that every dataset under STAGE/LEAF has STAGE/LEAF as its encryptionroot.
  5. zfs unload-key -r STAGE/LEAF

-x encryption makes a child created after the seed inherit encryption instead of arriving in plaintext.

catchup is optional: cutover runs the same catch-up itself, unless @enc-final already matches on both sides. Running it separately lets you check the incremental step before the downtime. If anything is written to the dataset after catchup, cutover refuses and prints two zfs rename -r commands that move both @enc-final snapshots aside; run them and rerun cutover, which then catches up again. See stale snapshots for why they're renamed rather than destroyed.

Message Fix
STAGE/LEAF does not exist; run seed first Run seed.
DATASET@enc-seed does not exist; run seed first Run seed.
NAME@enc-final already exists; rename it aside with: zfs rename -r NAME@enc-final @enc-final-N One line per side, with the same free suffix N; run the commands shown, then rerun.
enc-final snapshots of DATASET and STAGE/LEAF differ The receive didn't produce matching snapshots; investigate before retrying.
NAME is not encrypted under STAGE/LEAF A dataset arrived outside the staged encryption root; investigate before retrying.

cutover

zfs-encrypt-migrate cutover tank/data

Swaps the staged tree into the dataset's place. Stop everything that writes to the dataset first.

  1. zfs load-key STAGE/LEAF
  2. Unmounts the dataset and its descendants, children first, never forcing.
  3. Catches up as catchup does, unless @enc-final already matches on both sides.
  4. zpool sync POOL, then refuses if written@enc-final is non-zero anywhere in the tree.
  5. Sets canmount=noauto on every filesystem in the tree. Volumes have no canmount; they just move with the renames.
  6. zfs rename -u DATASET STAGE/old/LEAF, then readonly=on on it.
  7. zfs rename -u STAGE/LEAF DATASET
  8. Sets keylocation to --final-keylocation.
  9. Mounts, parents first, every filesystem with canmount=on whose mountpoint isn't none or legacy.

On success it prints:

verify the key with: zfs load-key -n DATASET
cutover done; once verified, free the old copy with: zfs destroy -r STAGE/old/LEAF

zfs load-key -n asks for the passphrase and checks it without changing anything. Run it before you destroy the old copy: after that, the encrypted dataset is the only copy.

Message Fix
DATASET is a pool root; cutover is not supported See rebuilding a pool encrypted.
DATASET is on pool POOL but the stage is on OTHER; cutover cannot rename across pools Use a stage on the dataset's pool.
STAGE/LEAF does not exist; run seed first Run seed.
DATASET is a volume; cutover swaps filesystems only, with any volumes under them Migrate the volume with its parent filesystem (cutover of the parent, or tree-move); one directly under the pool root needs the pool rebuild.
cannot unmount NAME; a writer is probably still running Stop the writer and rerun.
Any catchup message As for catchup.
written since catchup: followed by the datasets Run the two zfs rename -r ...@enc-final @enc-final-N commands it prints, then rerun cutover.

Tree Commands

For a parent with many large children, the tree commands move one child at a time, so you only need room for the parent's own data plus one child. See the tree guide for a walkthrough.

tree-seed

zfs-encrypt-migrate tree-seed tank/backups

Copies only the parent:

  1. zfs snapshot DATASET@enc-tree (not recursive)
  2. zfs send -p DATASET@enc-tree | zfs recv -u -o encryption=aes-256-gcm -o keyformat=FORMAT -o keylocation=file://KEY STAGE/LEAF
  3. zfs unload-key -r STAGE/LEAF

It refuses with the same messages as seed, plus DATASET is a pool root; the tree move is not supported and ... tree-seed cannot rename across pools. Its free-space check counts only the parent's own data and snapshots (usedbydataset plus usedbysnapshots).

tree-move

zfs-encrypt-migrate tree-move tank/backups host1

Copies one direct child, with everything under it, beneath the staged parent:

  1. zfs load-key STAGE/LEAF
  2. Unmounts DATASET/CHILD and its descendants, never forcing.
  3. zfs snapshot -r DATASET/CHILD@enc-tree
  4. zfs send -R DATASET/CHILD@enc-tree | zfs recv -u -x encryption STAGE/LEAF/CHILD
  5. Checks @enc-tree GUIDs on both sides, and that everything under STAGE/LEAF/CHILD has STAGE/LEAF as its encryptionroot.
  6. zpool sync POOL, then refuses if written@enc-tree is non-zero anywhere under the child.
  7. zfs unload-key -r STAGE/LEAF

On success it prints:

verify the key with: zfs load-key -n -L prompt STAGE/LEAF
moved DATASET/CHILD; once verified, free it with: zfs destroy -r DATASET/CHILD

Destroy the source child only after a run that printed moved .... Once it's destroyed, the staged copy is the only one, while its key still comes from the key file. -L prompt makes zfs load-key -n ask for the passphrase instead of reading that file, so it checks the passphrase you stored; run it before every destroy.

The source child stays where it is, unmounted, until you destroy it. It keeps its canmount, so zfs mount -a or a reboot mounts it again; destroy it soon after you've checked the copy.

When the child is already staged, a rerun of tree-move checks the staged copy instead of copying:

  • If DATASET/CHILD has no @enc-tree, it was created after an earlier move whose source was destroyed, so the staged child may be the only copy of the original data. It refuses without any cleanup commands; resolve it by hand, for example by renaming the new DATASET/CHILD aside.
  • If any dataset under DATASET/CHILD lacks an @enc-tree with the same GUID under STAGE/LEAF/CHILD, or anything there isn't encrypted under STAGE/LEAF, the staged copy is partial, left by a run that was interrupted or failed. It refuses with the commands that remove it.
  • Otherwise it checks the source as a move does: it unmounts it, runs zpool sync and refuses if written@enc-tree is non-zero anywhere under it, with the same two commands. Only if nothing was written was the child already moved: it prints DATASET/CHILD was already moved to STAGE/LEAF/CHILD, then the two lines above, and exits 0.
Message Fix
DATASET is on pool POOL but the stage is on OTHER; tree-move cannot rename across pools Use a stage on the dataset's pool.
STAGE/LEAF@enc-tree does not exist; run tree-seed first Run tree-seed.
dataset DATASET/CHILD does not exist Check the name.
STAGE/LEAF/CHILD already exists but DATASET/CHILD has no @enc-tree; DATASET/CHILD looks recreated Keep STAGE/LEAF/CHILD: it may be the only copy of the original data. Resolve it by hand, for example by renaming the new DATASET/CHILD out of DATASET with zfs rename.
STAGE/LEAF/CHILD already exists but is not a complete copy of DATASET/CHILD; a tree-move was interrupted or failed Only a run that printed moved ... finished the child; don't destroy the source. Run the two commands it prints, zfs destroy -r DATASET/CHILD@enc-tree and zfs destroy -r STAGE/LEAF/CHILD, then rerun.
free space on POOL (AVAIL) minus size of DATASET/CHILD (USED) is below floor MIN_FREE Free space, or lower --min-free.
cannot unmount NAME; a writer is probably still running Stop the writer and rerun.
enc-tree snapshots of DATASET/CHILD and STAGE/LEAF/CHILD differ, or NAME is not encrypted under STAGE/LEAF The received copy doesn't match; investigate, then run the two zfs destroy -r commands it prints and rerun.
written since the tree snapshot: followed by the datasets Stop the writer, run the two zfs destroy -r commands it prints (the stale snapshot and the copy without those writes), then rerun.

tree-finish

zfs-encrypt-migrate tree-finish tank/backups

Once every child has been moved and destroyed, sends the parent's last changes and swaps the staged tree, with the moved children, into place:

  1. zfs load-key STAGE/LEAF
  2. Unmounts the parent, never forcing.
  3. zfs snapshot DATASET@enc-tree-final (not recursive)
  4. zfs send -p -I @enc-tree DATASET@enc-tree-final | zfs recv -u STAGE/LEAF
  5. Checks @enc-tree-final GUIDs on both sides.
  6. zpool sync POOL, then refuses if written@enc-tree-final is non-zero.
  7. Swaps exactly as cutover does.

On success it prints:

verify the key with: zfs load-key -n DATASET
tree-finish done; once verified, free the old copy with: zfs destroy -r STAGE/old/LEAF
Message Fix
DATASET is a pool root; the tree move is not supported See rebuilding a pool encrypted.
DATASET is on pool POOL but the stage is on OTHER; tree-finish cannot rename across pools Use a stage on the dataset's pool.
STAGE/LEAF@enc-tree does not exist; run tree-seed first Run tree-seed.
DATASET@enc-tree does not exist; run tree-seed first Run tree-seed.
children remain under DATASET; move each with tree-move and destroy it first Finish moving the children.
NAME@enc-tree-final already exists; rename it aside with: zfs rename NAME@enc-tree-final @enc-tree-final-N One line per side, with the same free suffix N; run the commands shown, then rerun.
cannot unmount NAME; a writer is probably still running Stop the writer and rerun.
enc-tree-final snapshots of DATASET and STAGE/LEAF differ The received copy doesn't match; investigate before retrying.
written since the final snapshot: followed by the dataset Run the two zfs rename ...@enc-tree-final @enc-tree-final-N commands it prints, then rerun.

Audit

zfs-encrypt-migrate audit --exact tank --subtree tank/stage tank

Lists every filesystem and volume of the named pools whose encryption is off, except the allowlisted ones, one per line on stdout.

  • --exact NAME allows exactly that dataset.
  • --subtree NAME allows that dataset and its descendants, by path component: --subtree tank/media covers tank/media/photos but not tank/media2.
  • Both are repeatable.

It exits 1 if it lists anything. It fails closed: if zfs list fails for any pool, for example one that isn't imported, it prints the error and exits 1 rather than reporting a clean result. Naming no pool is a usage error.

A typical allowlist is the pool root (--exact tank), unless you created the pool encrypted, and the stage (--subtree tank/stage), whose old copies are unencrypted on purpose.

Output and Exit Codes

Errors go to stderr as error: <message>, followed by any recovery hints. Success lines go to stdout. When a zfs or zpool command fails, the message is the command followed by failed: and its error output; this includes zfs missing from PATH or not running as root.

Exit status Meaning
0 Success; for audit, nothing unencrypted outside the allowlists.
1 A refusal or a failed zfs command; for audit, an unencrypted dataset or a pool it couldn't list.
2 Usage error: a missing --stage, an invalid name, size or keylocation, or no pool for audit.
130 A migration or tree command interrupted by Ctrl-C, SIGTERM or SIGHUP. audit has nothing to clean up: Ctrl-C ends it with 1, and SIGTERM or SIGHUP end it like any other process.

Recovery Hints

If a command fails or is interrupted after it loaded the staged key, the key is unloaded again before it exits. If that unload fails too, it prints:

could not unload key; run: zfs unload-key -r STAGE/LEAF

Once cutover, tree-move or tree-finish has started on the source dataset, the error is followed by <command> aborted. and the commands for the step it reached. DATASET is the dataset being swapped, or DATASET/CHILD for tree-move, which never gets past the first step:

Step reached Hint
Before the swap DATASET may be unmounted; remount with: zfs mount -a
canmount set to noauto DATASET is unmounted and canmount may be noauto; restore with:, a zfs set canmount=on NAME line for each dataset that had it, and zfs mount -a
Original renamed aside the original is at STAGE/old/LEAF (canmount=noauto, readonly=on); put it back with: zfs rename -u STAGE/old/LEAF DATASET, zfs inherit readonly DATASET, the canmount=on lines, and zfs mount -a
Staged copy renamed into place the swap completed; only the final steps failed. Finish with: zfs set keylocation=LOC DATASET and zfs mount -a

Once tree-move has unmounted the source and its @enc-tree snapshots exist, a failure or interrupt also prints STAGE/LEAF/CHILD may hold a partial copy; remove it and the tree snapshots, then rerun tree-move:, followed by zfs destroy -r DATASET/CHILD@enc-tree and zfs destroy -r STAGE/LEAF/CHILD. Until both have run, a rerun refuses.

Stale snapshots

When a retry is blocked by an @enc-final or @enc-tree-final left over from an earlier attempt, the tool prints commands that rename it aside, on both sides, to the first -N suffix that neither side uses yet:

zfs rename -r tank/data@enc-final @enc-final-1
zfs rename -r tank/stage/data@enc-final @enc-final-1

Renaming, not destroying, is what makes the retry work. The staged copy still matches its newest snapshot, so the next incremental receive can build on it; destroy that snapshot and zfs recv refuses the staged copy as modified, which leaves only a fresh seed. A renamed snapshot keeps its GUID, so the retry's receive skips it instead of sending it again. @enc-tree-final is renamed without -r, because tree-finish snapshots only the parent. Once the migration is done, the renamed snapshots are ordinary snapshots you can destroy.

Interrupts and hangups

Ctrl-C (SIGINT), SIGTERM and SIGHUP all take the same path: a running zfs send and zfs recv are stopped, the staged key is unloaded, error: interrupted and the hints are printed, and the exit status is 130. While it unloads the key, further signals are ignored, so a second Ctrl-C can't skip the unload.

A signal that was already ignored when the tool started stays ignored. For a long seed over SSH, nohup zfs-encrypt-migrate seed ... therefore survives a dropped connection instead of aborting.

Safety Model

  • It never destroys anything. It never runs zfs destroy. Old copies and moved children are left for you, with the exact command printed; stale snapshots get a printed rename instead.
  • Staged keys stay unloaded. The staged copy keeps every mountpoint set locally in the original tree, so its key is loaded only while the tool works on it. seed, catchup, tree-seed and tree-move unload it when they finish, and any failure or interrupt part way through unloads it too.
  • One migration in flight. seed and tree-seed refuse while anything other than <stage>, <stage>/old and the old copies under it exists under the stage.
  • A free-space floor. seed, tree-seed and tree-move refuse unless the stage pool's free space minus the size of what they copy stays at least --min-free.
  • No lost writes. cutover, tree-move and tree-finish unmount the source first and refuse if it's busy. After the final snapshot they run zpool sync and refuse if written@<snapshot> is non-zero anywhere in the tree.
  • Nothing lands unencrypted. Catch-up and tree-move receives use -x encryption. After each catch-up and each tree-move, every dataset in the received tree is checked to have the staged copy as its encryptionroot, and snapshots are compared by GUID.
  • Recovery hints that match the failure. See recovery hints.
  • Signals are cleanups, not crashes. See interrupts and hangups.

And its limits:

  • Old plaintext blocks stay on disk until they're overwritten. Destroying the old copy frees its blocks; it doesn't wipe them.
  • You need free space for one dataset at a time (or the parent plus one child), until you destroy the old copy.
  • Plan for writer downtime during the cutover. The dataset is unmounted for the final catch-up and the swap.
  • NFS clients see stale file handles after a swap. The path is now served by a different dataset; remount it on the clients.
  • Children moved with tree-move get no old copy to fall back on. Once you destroy a moved source child, the encrypted copy is the only one; only the parent lands in <stage>/old. Check the passphrase you stored first, with the zfs load-key -n -L prompt STAGE/LEAF line that tree-move prints.
  • A dataset whose last component is old can't be migrated. Its staged copy would be <stage>/old, where the old copies live.
  • On NixOS, keep the stage out of boot-time key loading. With the default boot.zfs.requestEncryptionCredentials = true, a reboot while a staged copy exists makes the pool import fail with Key load error. Before the first seed, set it to false or to a list without the stage, and keep it that way until the stage is empty; see NixOS boot prompts.
  • Volumes move with their parent filesystem. cutover refuses a volume on its own (a volume directly under the pool root needs the pool rebuild), and since a volume can't be unmounted, the tool can't tell that it's in use: stop whatever uses it first.
  • It needs zfs rename -u. It is tested on OpenZFS 2.4.

NixOS Module Options

services.zfs-encrypt-migrate = {
  enable = true;
  audit = {
    enable = true;
    pools = [ "tank" ];
    exact = [ "tank" ];
    subtrees = [ "tank/stage" ];
  };
};
Option Default Description
enable false Install the CLI into the system profile.
package flake package Package providing zfs-encrypt-migrate. Set by nixosModules.default; set it yourself when importing nix/nixos-module.nix directly.
audit.enable false Run the audit on a timer. Works without enable.
audit.pools [ ] Pools to audit. Must be non-empty when the audit is enabled.
audit.exact [ ] Datasets allowed to stay unencrypted, matched by whole name (--exact).
audit.subtrees [ ] Datasets allowed to stay unencrypted together with their descendants (--subtree).
audit.onCalendar "daily" systemd calendar expression for the timer.
audit.onFailure [ ] Units to start when the audit fails.

The audit runs as the oneshot zfs-encrypt-migrate-audit.service, ordered after zfs-import.target, from the Persistent zfs-encrypt-migrate-audit.timer. It uses the host's boot.zfs.package, runs as root (it needs /dev/zfs) and is otherwise sandboxed, for example with ProtectSystem=strict and ProtectHome=true.