nixbackup

Three failure modes that hide behind a green dashboard

A mounted destination develops a phantom write

Mount a receive destination’s root and its children’s own mounts occlude the parent’s placeholder directories — a few bytes of directory-entry metadata churn that zfs receive’s incremental-safety check cannot tell apart from real data changing. The next incremental refuses with “destination has been modified since most recent snapshot”, even though every child shows zero bytes written. It reads exactly like data loss. It is not one. It still costs a manual re-baseline, and it recurs on every root a reboot or a rebuild remounts.

“Auto-create the destination” doesn’t create anything

Across more than one popular replication tool, the auto-create flag turns out not to actually create. A newly added child dataset under an already-replicating tree gets no destination at all — the tool logs “destination does not exist, will be rechecked every run” forever and never attempts the create, no matter how long you wait.

An exit code is not a signal

A replication job can exit 0 every single night while silently replicating nothing. A truncated btrfs receive can leave behind a directory with a perfectly fresh timestamp and no actual snapshot subvolume inside it. Both look identical to “healthy” from anything that isn’t reading the artifact’s structural ground truth.

Three modules, one per failure mode

destinations.nix

Every backup receive-destination dataset stays canmount=noauto + readonly=on, both set locally (a locally-set property always wins over one carried in by a received stream), plus actually unmounted. A boot and timer oneshot re-asserts all three, so nothing — not a stray zfs set, not a host rebuild that remounts the pool — can silently reopen the phantom-write divergence.

autobootstrap.nix

A timer that enumerates replication plans at runtime from ZFS user-properties — never a hardcoded list, because a plan absent from a hand-list is structurally invisible to anything built around that list — and seeds any destination child that does not exist yet with one zfs send -w | zfs receive -u. Everything it creates is immediately set canmount=noauto + readonly=on, so it starts life already compliant with the destinations invariant, whether or not that module also runs on the same host.

monitor.nix

The ground-truth freshness evaluator: it stats the artifact a job was supposed to produce, never the job’s exit code. Freshness reduces by per-leaf MIN, never a recursive MAX that lets one fresh child hide a dead subtree. Source and destination are diffed structurally, so a newly-missing child is its own failure rather than silence. Every result is published to a configurable push-style monitoring endpoint.

Husk detection

A numbered btrfs snapshot directory that exists with no snapshot subvolume ever having landed inside it reads as fresh by mtime and is actually empty. The monitor calls that a husk and fails it, because the name and the timestamp were never proof of a complete transfer.

Cadence-aware staleness

Deadlines follow the schedule the job actually runs on. A weekend evaluation still compares against the last weekday the job was expected to run, rather than an invented deadline that turns every Saturday into a false alarm.

Journal error budgets

A configurable per-unit journal check, because a unit can exit 0 while its own log already recorded a real per-item send or receive failure. Patterns, the since-window, and the destinations to exclude are all options.

How the three fit together

  SOURCE POOL                                DESTINATION POOL
  ───────────                                ────────────────
  tank/data/a  ─┐                          ┌─ tank/backups/a
  tank/data/b  ─┤  your replication tool   ├─ tank/backups/b
  tank/data/c  ─┘  (znapzend, syncoid, …)  └─ tank/backups/c   ← new child
       │                                            ▲
       │ org.nixbackup:enabled=on                   │ seeded by
       │ org.nixbackup:destination=…                │ send -w | receive -u
       │                                            │
       ├───────────────► autobootstrap ─────────────┘
       │                 (reads properties at runtime)
       │
       │                 destinations ──► canmount=noauto + readonly=on
       │                                  + unmounted, re-asserted on a timer
       │
       └───────────────► monitor ───────► stat the artifact, MIN-reduce,
                                          structural diff, push the verdict

Properties are the runtime contract

Your replication tool’s own config (or a small companion zfs set) stamps org.nixbackup:enabled=on and org.nixbackup:destination=<dest dataset> on each plan-root source dataset. That pair of properties is the entire contract the modules read — both property names are configurable options.

Independently toggleable

The modules share one namespace (nixbackup.*) and one convention, but no module depends on another being enabled. Take one, take all three; a typical host enables all three.

Deliberately out of scope

Which datasets to back up, and how often

This project has no opinion on your retention policy or your pool layout. It owns the invariants that make whatever you already run stay correct, not the policy decisions about what to run.

The replication daemon itself

nixbackup does not run or configure znapzend, syncoid, or anything else that schedules and executes zfs send/receive on your behalf. It assumes one of those — or an equivalent you wrote yourself — already exists and does the routine incremental work. nixbackup covers the three gaps those tools tend to share.

Status: pre-alpha, all three modules real

Extracted from a working setup

Each rule above corresponds to an incident that was actually hit, root-caused, and fixed on a live setup before being pulled out here, with every site-specific value replaced by a generic parameter.

Verified at module-evaluation level

Verified so far by NixOS module evaluation — nix eval on each nixosModules.* attribute, plus a toy nixosSystem importing all three together. Not yet run against a second, independent real ZFS pool.

Nothing claimed beyond that

Nothing advertised here is invented or missing, and nothing is claimed as battle-tested beyond that evaluation level until it has actually run elsewhere. Everything under Roadmap is future work, clearly marked as such.

Install

Add the flake input

inputs.nixbackup.url =
  "github:julian-corbet/nixbackup-corbet-ch";

Three flake outputs: nixosModules.destinations, nixosModules.autobootstrap, and nixosModules.monitor.

Enable what you need

modules = [
  nixbackup.nixosModules.destinations
  {
    nixbackup.destinations = {
      enable = true;
      datasets = [
        "tank/backups/dbs"
        "tank/backups/office"
      ];
    };
  }
];

Stamp the runtime contract

zfs set org.nixbackup:enabled=on tank/data
zfs set org.nixbackup:destination=tank/backups/data \
    tank/data

Only autobootstrap and the monitor’s zfs-dynamic targets read these. See examples/configuration.nix for a complete flake wiring all three modules on one host.

Monitor target kinds

`zfs-leaves`	Per-leaf ZFS snapshot freshness over an explicit list of datasets, MIN-reduced so one fresh leaf cannot mask a dead one.
`zfs-dynamic`	Replication plans discovered at runtime under a `scanRoot`, with a structural source-vs-destination diff. Supports a `cadence` block.
`btrfs-received`	A flat tree of received btrfs snapshot subvolumes, husk-aware: a numbered directory with no subvolume inside it fails.
`btrfs-mtime`	Plain mtime freshness for a btrfs path, when there is no subvolume structure to verify.
`stampfile`	Freshness from a file a job writes on real completion — the escape hatch for anything the other kinds cannot see.

Roadmap

Receive-destination invariant	Done — `modules/destinations.nix`.
Runtime-discovered auto-bootstrap	Done — `modules/autobootstrap.nix`.
Ground-truth freshness monitor	Done — `modules/monitor.nix`.
Second independent ZFS pool	Run against a real pool that isn’t the one these were extracted from, beyond module evaluation. Planned.
`lib` property helper	A helper for stamping the `org.nixbackup:*` properties, so a consumer doesn’t hand-write the `zfs set` calls. Planned.
Cadence documentation	Full documentation of the cadence and `journalChecks` interaction beyond the inline option docs. Planned.