Backing up

• Note that the first time you use a save directory (repository), you'll need --init-savedir.
• Back up your root filesystem (absent ZFS, which breaks find's -xdev), with file-count-based progress, creating the repository if it does not yet exist:
  • find / -xdev -print0 | backshift --save-directory /where/ever/save-directory --backup -subset slash --init-savedir --progress-mode minimal
• To back up / with a more accurate progress report. This one is based on the lengths of files; the above two are just based on file counts:
  • find / -xdev -print0 | backshift --save-directory /where/ever/save-directory --backup -subset slash --init-savedir --progress-report full+poststat
• To back up / with a more accurate progress report (assumes your find supports -printf). This one also is based on the lengths of files, however it's a little faster than the previous example, especially on large collections of many small files:
  • find / -xdev -printf '%s %p\0' | backshift --save-directory /where/ever/save-directory --backup -subset slash --init-savedir --progress-report full+prestat
• To back up / with a minimal progress report - this one does not do a big inhale of filenames at the beginning:
  • find / -xdev -print0 | backshift --save-directory /where/ever/save-directory --backup -subset slash --init-savedir --progress-report minimal
• To back up / with no progress report - this one is usually best in cron jobs:
  • find / -xdev -print0 | backshift --save-directory /where/ever/save-directory --backup -subset slash --init-savedir --progress-report none
• This one backs up /movie with a progress report, keeping the progress report pretty accurate despite a previous incomplete backup by using randomize --preserve-directories. Note that this example splits one logical line into multiple physical lines in the manner of POSIX shells and *csh, by using backslashes on all but the last line:
  • find /movie -xdev -printf '%s %p\0' | \
  •    ~/src/home-svn/backshift/trunk/randomize -0 -v --preserve-directories --skip-size | \
  •    /usr/local/pypy-1.4.1/bin/pypy ~/src/home-svn/backshift/tags/0.94/backshift \
  •      --backup \
  •      --save-directory /mnt/backshift-incremental-test/save-directory \
  •      --subset movie \
  •      --progress-report full-prestat
• To pull from an sshfs, which flattens filesystems into a single filesystem, to a local filesystem (but writing to a remote filesystem is faster than reading from one), not creating the savedir (note that backing up a ZFS could be done analogously):
  • cd /ssh/fs/base
  • find . -xdev \( \( \
  •    -path ./sys -o \
  •    -path ./dev -o \
  •    -path ./var/run -o \
  •    -path ./var/lock -o \
  •    -name .gvfs \) -prune -o -print0 \) | \
  •    backshift --save-directory /where/ever/save-directory --backup --subset fullsave

Restoring

• Overview of process
  • First, locate what backups are available to restore from, using --list-backups, and select the best one, for some definition of "best" ^_^
  • Second, locate the files within that backup you require using --list-backup --backup-id
  • Third, use "--produce-tar --starting-directory | tar xvfp -" to extract the files
  • Strictly speaking, you can use --produce-tar with a pipe to "tar tvf -" in the second step too, but it's much slower.
• Example restore:
  • First we list all backups that finished (the last column is None for an unfinished backup). For the sake of discussion, assume the green backup id is the "best" one:
    • # ~/src/home-svn/backshift/trunk/backshift --save-directory /mnt/backshift-incremental-test/save-directory --list-backups | awk ' { if ($4 != "None") print }' | sort
    • 1305581966.39_openindiana_export_mon-may-16-14-39-26-2011_6244d94b726da6c6 Mon-May-16-14:39:26-2011 2 Mon-May-16-14:39:26-2011
    • 1305583872.56_openindiana_export_mon-may-16-15-11-12-2011_8cfbd6e4f5d87142 Mon-May-16-15:11:12-2011 2 Mon-May-16-15:11:13-2011
    • 1305609181.37_openindiana_slash_mon-may-16-22-13-01-2011_04be24c2e608ec32 Mon-May-16-22:13:01-2011 160326 Tue-May-17-13:18:34-2011
    • 1305609205.38_openindiana_export_mon-may-16-22-13-25-2011_20abd67bf8d07db3 Mon-May-16-22:13:25-2011 17177 Tue-May-17-04:12:41-2011
  • Next we identify what file we need (the red path is its directory):
    • # ~/src/home-svn/backshift/trunk/backshift --save-directory /mnt/backshift-incremental-test/save-directory --list-backup --backup-id 1305609205.38_openindiana_export_mon-may-16-22-13-25-2011_20abd67bf8d07db3 2>&1 | egrep -i 'xz.*local-script'
    • -rw-r--r-- strombrg/staff 249 2011-05-16 10:32 export/home/strombrg/src/xz/local-script
  • Note that in the preceding step, if we already knew the directory, but not the filename, we could've used the following much more rapidly:
    • # ~/src/home-svn/backshift/trunk/backshift --save-directory /mnt/backshift-incremental-test/save-directory --list-backup --backup-id 1305609205.38_openindiana_export_mon-may-16-22-13-25-2011_20abd67bf8d07db3 --starting-directory /export/home/strombrg/src/xz 2>&1
    • -rw-r--r-- strombrg/staff 216 2011-05-16 10:07 export/home/strombrg/src/xz/Notes
    • -rw-r--r-- strombrg/staff 626 2011-05-16 10:41 export/home/strombrg/src/xz/last-archives
    • -rw-r--r-- strombrg/staff 249 2011-05-16 10:32 export/home/strombrg/src/xz/local-script
    • drwxr-xr-x strombrg/staff 0 2011-05-16 10:11 export/home/strombrg/src/xz/old/
    • -rw-r--r-- strombrg/staff 1023720 2011-04-01 03:11 export/home/strombrg/src/xz/xz-5.0.2.tar.bz2
    • -rw-r--r-- strombrg/staff 1270541 2011-04-12 03:49 export/home/strombrg/src/xz/old/xz-5.1.1alpha.tar.gz
  • Finally, we extract the file we want:
    • # ~/src/home-svn/backshift/trunk/backshift --save-directory /mnt/backshift-incremental-test/save-directory --backup-id 1305609205.38_openindiana_export_mon-may-16-22-13-25-2011_20abd67bf8d07db3 --starting-directory /export/home/strombrg/src/xz --produce-tar | tar xvf - export/home/strombrg/src/xz/local-script
    • export/home/strombrg/src/xz/local-script
  • Note that during the restore, backshift didn't write to your filesystem; tar did (except on Windows, where some temp files are written briefly).

Confirming to an extent that the software is working correctly - here we do a restore to a pipe, and without writing to disk, use GNU tar's --diff option to compare what's coming out of backshift with what's on the system:

• ~/src/home-svn/backshift/trunk/backshift \
•    --save-directory $(pwd) \
•    --produce-tar \
•    --backup-id 1307122264.51_benchbox_test_fri-jun--3-10-31-04-2011_8bb8b332b1b34ea4 | \
•    (cd / && tar --diff)