Backblaze B2 Backups

Automated encrypted backups to Backblaze B2 using Restic. Runs daily at 3AM with GFS retention.

What is covered, grouped by what it costs to lose:

Group Paths
Recovery keys /persist/var/keys, /persist/etc/ssh, /persist/var/dkim, /persist/var/lib/sbctl
Host identity /persist/etc/secrets (initrd SSH key), /persist/etc/machine-id, .ssh and .config/syncthing under /persist/home/user
Host bookkeeping /persist/var/lib/nixos (UID/GID map), /persist/etc/{opencloud,NetworkManager,containers}
Service state forgejo, postgresql (Keycloak, Miniflux, Invidious), trilium, opencloud, n8n, navidrome, anki-sync-server, slskd, memos, open-webui, open-terminal, openbao, acme under /persist/var/lib
Mail /persist/var/vmail, postfix and dovecot under /persist/var/lib
Personal data Documents, Pictures, Music, Videos, Projects, Software, agent data in .claude and .pi, and the nixos, dotfiles, nvim repos under /persist/home/user
Application config .app-state under /persist/home/user — Krita, Blender, Okular and opencode rewrite their own config, so no repository carries it
Bulk /mnt/atlas, OpenCloud storage under /mnt/aether/network/opencloud

Service users get no fixed UID, so /persist/var/lib/nixos is what keeps a restored state directory owned by the same user after a reinstall: restore it before the host first activates.

Deliberately excluded because they are reproducible: container images and volumes, Incus storage pools, Ceph monitor and OSD data, Talos nodes (declarative in talos-config/), and logs.

Syncthing’s index (~/.config/syncthing/index-*) is excluded on purpose. It lists every file the host held, so restoring it next to any file the backup lacks makes Syncthing report that file deleted to every peer. A host restored without it rescans and pulls what it lacks from the peers.

Exclude patterns match by name anywhere in a path (build, tmp, cache, *.log, LaTeX by-products and others in modules/features/restic-backup.nix), so data kept under such a name is never backed up.

Retention: 3 daily, 2 weekly, 3 monthly, 2 yearly snapshots

Snapshot Consistency

Everything under /persist is read from a read-only btrfs snapshot rather than the live subvolume. A backup that copies files straight off a running system reads each one at a different moment, and can catch a SQLite or PostgreSQL file mid-transaction — a defect that surfaces only at restore time.

flowchart LR
    P["@persist (live)"] -->|"btrfs subvolume snapshot -r"| S["@restic-persist"]
    S -->|"mounted at /run/restic-persist"| B["bind over /persist"]
    B -->|"read"| R["restic"]
    R --> X["Backblaze B2"]

restic-persist-snapshot.service rotates the snapshot and mounts it, then the backup unit binds it over /persist inside its own mount namespace. Every path in a given snapshot therefore comes from the same instant, which both database engines recover from by design since it is indistinguishable from a power cut.

Paths under /persist are spelled with that prefix rather than their runtime bind-mount target: impermanence binds such as /var/lib/forgejo resolve to the live subvolume, so only the /persist spelling is covered by the snapshot.

The previous snapshot is dropped at the start of the next run rather than after the backup, because unmounting it from inside the backup’s namespace is not possible. One read-only snapshot therefore survives between runs, pinning a day of overwritten extents and doubling as a local rollback point.

Note: All commands in this guide run on the server. SSH to [email protected] first.

Restic Credentials

Most restic commands require credentials. Export these before running commands:

export $(cat /run/secrets/restic/b2-credentials | xargs)
export RESTIC_PASSWORD_FILE=/run/secrets/restic/password
export RESTIC_REPOSITORY=b2:yhkze-infra-backups:nixos

Initial Setup

Create B2 Bucket

  1. Log into Backblaze B2 dashboard
  2. Buckets → Create a Bucket
  3. Settings:
  4. Create Bucket

Generate Application Key

  1. App Keys → Add a New Application Key
  2. Settings:
  3. Create New Key
  4. Copy immediately (shown only once):

Add Secrets

Generate restic password:

openssl rand -base64 32

Edit hosts/server/secrets/secrets.yaml:

restic:
  password: <output-from-openssl>
  b2-credentials: |
    B2_ACCOUNT_ID=<keyID>
    B2_ACCOUNT_KEY=<applicationKey>

Encrypt:

sops -e -i hosts/server/secrets/secrets.yaml

Update Repository Path

Edit modules/features/restic-backup.nix:

repository = "b2:<bucket-name>:<path>";

Example: b2:yhkze-infra-backups:nixos

Deploy

rebuild 10.100.0.1

First backup runs automatically after deployment.

Daily Operations

Check Status

systemctl status restic-backups-b2.service

Check Schedule

systemctl list-timers | grep restic

View Logs

journalctl -u restic-backups-b2.service -n 50

Manual Backup

systemctl start restic-backups-b2.service
journalctl -fu restic-backups-b2.service  # Follow logs

Browsing Backups

Load credentials first (reference top of document).

List Snapshots

restic snapshots

Browse Files

restic ls latest              # Latest snapshot
restic ls <snapshot-id>       # Specific snapshot

Find Files

restic find "*.jpg"           # All JPG images
restic find "myfile.txt"      # Specific file
restic find "/home/user/Documents"  # Directory contents

Compare Snapshots

restic diff <old-id> <new-id>

Repository Stats

restic stats                  # Overall stats
restic stats <snapshot-id>    # Specific snapshot
restic stats --mode raw-data  # Before deduplication

Mount as Filesystem

Browse backups with Nemo or any file manager via FUSE mount.

Create mount:

mkdir -p /tmp/restic-mount
restic mount /tmp/restic-mount
# Leave terminal open, mount stays active

In another terminal:

nemo /tmp/restic-mount

Mount structure:

/tmp/restic-mount/
├── snapshots/
│   ├── 2026-04-12T23:21:38-05:00/  # Dated snapshot directories
│   │   ├── mnt/atlas/
│   │   └── persist/
│   ├── 2026-04-13T03:00:00-05:00/
│   └── latest → <most-recent>
├── hosts/
│   └── server/
└── tags/

Browse to the file you need, then copy it:

cp /tmp/restic-mount/snapshots/latest/persist/home/user/Documents/file.pdf ~/

Unmount: Ctrl+C in the terminal running restic mount

Restoring Data

Load credentials first (reference top of document).

Full Restore

restic restore latest --target /tmp/full-restore

Restore Directory

restic restore latest --target /tmp/restore --include /persist/home/user/Documents

Restore File

restic restore latest --target /tmp/restore --include /persist/home/user/Documents/file.pdf

File appears at: /tmp/restore/persist/home/user/Documents/file.pdf

Point-in-Time Restore

restic snapshots              # Find snapshot ID
restic restore <snapshot-id> --target /tmp/restore

Pattern Restore

restic restore latest --target /tmp/restore --include "*.jpg"
restic restore latest --target /tmp/restore --include "/persist/home/user/nixos/**"

Disaster Recovery

A full restore onto a new disk happens from the installer, before the first activation, from B2 or a local replica. Reference Server Reinstall.

Maintenance

Load credentials first (reference top of document).

Check Integrity

restic check              # Quick check (metadata)
restic check --read-data  # Full check (downloads data)

Run monthly.

View Retention

restic forget --dry-run --keep-daily 3 --keep-weekly 2 --keep-monthly 3 --keep-yearly 2

Unlock Repository

If backup interrupted:

restic unlock

Manual Prune

restic prune

Note: Auto-pruning runs after each backup.

Troubleshooting

Service Failed

journalctl -u restic-backups-b2.service -n 100

Common issues:

Repository Not Found

restic init

Slow Backups

Limit upload speed in modules/features/restic-backup.nix:

extraBackupArgs = [
  "--limit-upload 10000"  # KB/s
];

Verify Files

restic mount /tmp/restic-mount
# In another terminal:
ls -lah /tmp/restic-mount/snapshots/latest/persist/home/user/

Check exclusion patterns if files missing.

Security

Encryption:

Access control:

Backup the backup — store restic password and B2 credentials securely:

Cost Estimation

B2 pricing (2026):

Item Cost
Storage $6/TB/month
Download $10/TB (1GB/day free)
API calls 2,500/day free

100GB backup: ~$0.60/month + free API calls

Deduplication means incremental backups only upload changes.

References