Reinstalls the server from scratch, for example onto a new disk, and brings it back as the same host with all of its data. The backup is restored into the freshly partitioned disk before the first install, so the host keys, the sops age key and the UID/GID map are in place when the system first activates.
flowchart LR
A[freeze services] --> B[final backup]
B --> C[power off, swap disk]
C --> D[live ISO]
D --> E[disko]
E --> F[restore into /mnt]
F --> G[bootstrap install]
G --> H[unlock over SSH]
H --> I[rebuild push]
I --> J[post-install]
The restore source is either the B2 repository or a local replica on a workstation reached over the LAN. A local replica restores about ten times faster (about 12 minutes for 120 GiB at 1 Gb/s) and costs nothing, while a B2 restore is billed per GiB downloaded past the daily free allowance. Keep the old disk untouched until Post-Install passes: it is the fallback.
Stop everything that writes state, so the final backup holds the last write of every service. Incoming mail waits in the senders’ queues, which retry for days
Pause the server device in Syncthing on every peer as well: until the restore is verified, the server must not push an empty or partial folder to them. The backup carries no Syncthing index, so on resume the server pulls anything the restore lacks from the peers rather than reporting it deleted.
The backup unit rotates the read-only @restic-persist
snapshot first, so every path comes from the same instant
Record the snapshot ID it prints.
The replica is a second restic repository on a workstation’s disk,
written over SFTP with the same password and the same include and
exclude lists as the B2 job. Check that both ends negotiated gigabit
first: a NIC that reports downshifted in the kernel log has
a bad wire pair and falls back to 100 Mb/s, which makes the transfer
take ten times as long
Record the snapshot ID it prints.
Check the repository and spot-check the paths a reinstall cannot do
without. restic ls does not descend into subdirectories, so
pass --recursive when looking for nested files
Boot the minimal NixOS image. The graphical image suspends itself when idle and freezes the session during long steps such as the restore
The disk is addressed by model and serial, so read the path and set
it as device in hosts/server/nixos/disko.nix,
then commit it
Paths are stored absolute, so /persist/... lands on
/mnt/persist and /mnt/atlas on
/mnt/mnt/atlas, where disko mounted each subvolume. The
cache goes to the new disk because the installer’s tmpfs cannot hold it.
Run the restore as a transient unit so a dropped SSH session does not
kill it.
The B2 key must be allowed to read files: a key created with write access only answers downloads with 403
The installer reads the replica over SFTP with a throwaway key. On
the workstation the key is forced into a read-only
sftp-server -R and limited to the target’s address, so it
can neither open a shell nor modify the repository;
--no-lock lets restic read without writing a lock file
An interrupted restore can be started again with the same command: files that already match the snapshot are skipped.
The summary must add up to the snapshot’s size (restored plus
skipped). Then confirm the keys exist and that state directories are
owned by the UIDs the restored map assigns; the setup guide’s
ssh-keygen and sops key steps are skipped because both keys
came back
BOOTSTRAP=1 installs the minimal feature list, reference
values/bootstrap.nix. The first
activation then reads the restored UID/GID map, so every service user
gets back the UID that owns its files
The restored initrd host key is unchanged, so
known_hosts still matches. The passphrase is sent without a
trailing newline
Health first: systemctl --failed must be empty, and
Keycloak login, Forgejo, mail send and receive, OpenCloud and Trilium
must answer. Then rebuild what the backup deliberately leaves out.
open-terminal — its Nix store lives in a podman
volume. The backup holds it as an exported tar, because the global
excludes (**/build, **/cache and others) would
corrupt a file-level copy
podman volume import open-terminal-nix /persist/var/lib/open-terminal/nix-volume.tar
rm /persist/var/lib/open-terminal/nix-volume.tarCeph — a fresh cluster with the same fsid, its OSD
on the raw pool/ceph logical volume
(ceph-volume lvm create --data pool/ceph). Reference incus-cluster.md Phase 1, then copy the
admin keyring to the workstations again.
Incus — remove the stale member from another cluster member, then rejoin the server with a fresh token as in incus-cluster.md Phase 3
incus cluster remove server --forceKubernetes — recreate the Talos VMs from
talos-config/. Reference bootstrap.md.
Finish with a first backup from the new host
(systemctl start restic-backups-b2.service), resume the
server device in Syncthing on every peer, and only then reuse the old
disk. Before wiping it, mount it read-only and list what it holds that
the new host lacks
(rsync -n -a --ignore-existing <old>/ <new>/):
anything besides deliberate deletions is outside the backup paths or
matched an exclude pattern.