A high performance snapshot based backup system for Linux and UNIX like operating systems.
Designed with system administrators in mind.
Utilises native BTRFS / ZFS filesystem features for seamless compression, deduplication and snapshoting of the backup pool.
Existing Rsync backup approaches (eg Rsnapshot / BackupPC) don't scale, are hard to monitor and maintain when used with dozens or hundreds of hosts.
They also don't elegantly handle ad hoc backups, nor do they facilitate quickly adding and removing hosts.
Our solution is centralised and agentless, so there is nothing to consume resources on your hosts, and all configuration is managed on the backup server.
Utilises BTRFS / ZFS native filesystem snapshots, and per host filesystems for better performance, scalability and ease of management.
Backups can be run at anytime, with custom expiry and a short message so you know why the backup was taken.
- High performance and scalability. (Benchmarked faster than other Rsync backup solutions. eg BackupPC, Rsnapshot etc.)
- Simplicity: Simple text based configuration. And simple files-on-disk backup format.
- Aims to be a paranoid system administrator's best friend.
- Agentless
- Utilises BTRFS / ZFS filesystem features, eg snapshot, dedup and compression
- Uses Rsync and SSH for transport
- Integration with monitoring tools such as Nagios or Icinga using NSCA passive checks.
- Centralised configuration and management - all configuration and scheduling is done on the backup server
- Ad hoc annotated backups - allows for ad hoc backups with an explanation as to when or why the backup was taken and per backup retention periods
- Per host backup, retention and quota policies
- Per host configuration and logs stored with the snapshot
- Utilise LVM snapshots for performing atomic backups of Linux systems. See atomic.sh (Work in progress).
An operating system with BTRFS or ZFS support is required (eg FreeBSD or ZFS on Linux) and a dedicated storage pool.
Check out the source code into the root of your backup pool and review
./conf/backup.conf. Modify as necessary to set your pool and filesystem
options.
Create backup zpool with dedup and compression.
zpool create -f backup vdb
zfs set dedup=on backup
zfs set compression=gzip backup
Install Adlibre Backup into root of backup zpool.
yum -y install git
cd /backup && git clone git://github.com/adlibre/adlibre-backup.git .
Install NSCA Client (optional) for Nagios / Icinga integration
yum -y install nsca-client
Generate SSH Key, this is used for authentication.
ssh-keygen -t dsa -N "" -f ~root/.ssh/id_dsa
Add server.example.com host config and copy the SSH Key to host example.com
cd /backup && ./bin/add-host.sh example.com
Now run the backup
./bin/backup-runner.sh --all
The output
[root@zbackup backup]# ./bin/backup-runner.sh --all
Info: Begin backup run of hosts example.com
Info: Begining backup of example.com
Running: rsync -a --numeric-ids --hard-links --compress --delete-after --delete-excluded --fuzzy --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/tmp --exclude=/var/tmp --exclude=/var/run --exclude=/selinux --exclude=/cgroups --exclude=lost+found [email protected]:'/' /backup/hosts/example.com/d/
Warning: NSCA Plugin not found.
Backup Successful. Runtime 1757 seconds.
Warning: NSCA Plugin not found.
Snapshot example.com@2013-06-14-15:12:39-1371186759 Created
Info: Completed backup of example.com
That's it.
Now if you want to schedule daily backups Add the following to your root crontab:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
@daily /backup/bin/backup-runner.sh --all --comment "Backup Daily" && /backup/bin/prune.sh --all
./bin/add-host.sh <hostname>
Then customise the per host config in ./hosts/<hostname>/c/backup.conf and
ssh options in ~/.ssh/config if required.
To immediately purge the host configuration and all backup data:
zfs umount zfs-pool-name/hosts/<hostname> && zfs destroy zfs-pool-name/hosts/<hostname>
To disable future backups and allow existing backups to expire in line with the
retention policy set DISABLED=true in ./hosts/<hostname>/c/backup.conf.
This is the preferred method for host removal as it allows the old backups to
naturally expire.
./bin/backup.sh <hostname> <annotation> <expiry-in-days>
./bin/backup-runner.sh --all
or multiple hosts
./bin/backup-runner.sh <hostname> <hostname>...
All backups are stored on disk in plain sight. To restore all you need to do is copy (or rsync) the files from the backup pool to your host.
To find a particular snapshot:
./bin/list-backups.sh <hostname>
eg:
backup-host# ./bin/list-backups.sh example.com
example.com 2012-10-25-23:35:19-1351168519 1352377190 successful "first backup"
example.com 2012-11-04-15:40:49-1352004049 1354418267 successful "before acme software upgrade"
The files are stored plainly within the ZFS snapshot:
backup-host# ls -lah /backup/hosts/example.com/.zfs/snapshot
total 3
dr-xr-xr-x 4 root wheel 4B Nov 17 15:14 .
dr-xr-xr-x 4 root wheel 4B Oct 16 20:18 ..
drwxr-xr-x 5 root wheel 5B Oct 16 20:18 2012-10-25-23:35:19-1351168519
drwxr-xr-x 5 root wheel 5B Oct 16 20:18 2012-11-04-15:40:49-1352004049
Just dive in and copy the files out of the snapshot:
cd /backup/hosts/example.com/.zfs/snapshot/ && \
rsync -aH --numeric-ids 2012-11-04-15:40:49-1352004049/d/ example.com:/restore-point/
Each backup has a TTL defined, after which the prune script will delete the backup's snapshot. You can configure the retention/expiry of backups through various configuration properties. Each of them defines the TTL by means of "fully days after backup creation". A value of 0 defines "keep forever".
Once a backup is created you can't change the TTL anymore, as it is stored within the read-only snapshot.
A specific expiry TTL given as last parameter to the backup script overrules any of the other TTL configuration.
The default TTL configuration is set in the required property EXPIRY. It gets
applied when none of the other option matches.
All other options apply until to the first complete/successful backup at a given day:
EXPIRY_DAY: TTL for daily backupsEXPIRY_WEEK: TTL for the backup on a MondayEXPIRY_MONTH: TTL for backup on the first day of a monthEXPIRY_QUARTER: TTL for backup on the first day of a quarterEXPIRY_YEAR: TTL: for backup on the first day of a year
If a given day matches multiple conditions, the expiry configuration for the longest
period that has a non-empty value gets selected. E.g when on January 1st EXPIRY_YEAR
is not set, but EXPIRY_QUARTER and EXPIRY_DAY are, then
EXPRIY_QUARTER gets selected.
When a backup doesn't succeed completely (partial backups), then the configured TTL gets also applied to the next attempt to do a backup at the same day. If all backups at a given day (partially) fail, then you won't get a backup with the special TTL. You then would have to trigger a manual backup with a TTL set explicitly.
This has been in production use for many years now and is stable.
See TODO and ISSUES for outstanding issues. And NOTES for development information.