A suitable find command:
- with -printf (GNU find has it)
- or with -print0 (BSD and GNU find's have it)
- or a traditional (SysV, early BSD) find with -print combined with a "tr" command that can translate newlines to nulls. This breaks on filenames
containing newlines of course.
Compression
- Backshift will prefer to compress chunks from greatest preference to least:
- The CPython 3.3 "lzma" module. This gives xz-compatible compression.
- liblzma (likely in a -dev package) via ctypes. This gives xz-compatible compression.
- An xz binary via the subprocess module.
- The Python bz2 (bzip2) module.
- xz/lzma give a bit better compression ratios than bzip2.
- Note that the Debian python-lzma package not the same as the new CPython 3.3 lzma module; if you have python-lzma installed, you'll probably
need to uninstall it.
Filesystems - for remote saves
- sshfs
- CIFS
- NFS
- ...but see the backshift notes for your OS, if any.
- Others are probably possible, but have not been tested. Perhaps Lustre, GFS, PVFS2, OCFS2...?
For building and/or testing:
- Some form of "make"
- You may want SVN (Subversion)
- You want an m4 binary with which to create the pure python and/or cython rolling checksum modules