README

# Bup cron wrapper #

This is a simple wrapper around [bup][] to make it easier to run nightly
backup jobs. While it's designed to run under cron, it can also be
called directly. `bup-cron` supports:

 * remote backups
 * LVM and VSS snapshotting
 * parity blocks
 * usage statistics stored as `git notes`
 * exclude lists
 * logfile logging (automatically rotated)
 * syslog logging

Quick introduction
==================

To run a simple backup of your `Documents` directory to a `bup`
repository, run:

    ./bup-cron -d bup Documents

The repository will be created it if doesn't exist. You can also use
the `$BUP_DIR` environment variable the same way you would do with the
regular `bup` commands. The above will simply run `bup index` and `bup
save` with the right arguments.

There is a detailed usage available under `./bup-cron --help` and also
explained below.

Installation
============

`bup-cron` can be ran directly from the source directory, but you can
also hook it into the regular `bup` commands set by deploying it to
(e.g.) `/usr/lib/bup/cmd`. The rest of this manual assumes you have
done so, but you can also simply put `bup-cron` anywhere in your
`$PATH` and run it as is.

`bup-cron` is also available at [PyPI] and can be installed with:

    pip install bup_cron

 [PyPI]: https://pypi.python.org/pypi/bup_cron

Configuration
=============

Since `bup-cron` is designed to run automatically, it is capable of
reading a configuration file. The config file is searched in
`/etc/bup-cron.conf`, `~/.bup-cron.conf` or `~/.config/bup-cron.conf`,
in that order. All configuration files are read and the last config
file will append its configuration to the previous ones. You can also
pass an arbitrary configuration file on the commandline by passing it
as an argument, prefixed with `@`. For example, if you have different
backup jobs you want to run, you could have two cron jobs:

    bup cron @/etc/bup-cron-main.conf

and:

    bup cron @/etc/bup-cron-srv.conf

... with distinct configurations, with common configuration in
`/etc/bup-cron.conf`.

The content of the configuration file is one argument per line,
without `--`. For example, this:

    # paths to backup
    path=/
    path=/boot
    path=/usr
    path=/var
    path=/home
    
    # where to backup to
    repository=/media/anarcat/calyx/bup
    
    # exclude patterns
    exclude=/\.Trash-
    exclude=/\.cache/
    exclude=/[Cc]ache/
    exclude=/\.local/share/Trash/
    exclude=/\.thumbnails/
    exclude=/\.bitcoin/blocks/
    exclude=/tmp/
    exclude=/build-area/
    exclude=/var/log/
    
    # snapshot and add par2 parity
    snapshot
    parity
    stats
    
    # logging options
    syslog=DEBUG

Is equivalent to:

    bup cron --path / --path /boot --path /usr --path /var \
        --repository=/media/anarcat/calyx/bup --exclude=/\.Trash- \
        --exclude=/\.cache/ --exclude=/[Cc]ache/ \
        --exclude=/\.local/share/Trash/ 
        --exclude=/\.thumbnails/ \
        --exclude=/\.bitcoin/blocks/ \
        --exclude=/tmp/ \
        --exclude=/build-area/ \
        --exclude=/var/log/ \
        --snapshot \
        --parity \
        --stats \
        --syslog=DEBUG

This, in turn, is roughly equivalent to:

    export BUP_DIR=/media/anarcat/calyx/bup
    bup init
    for path in / /boot /usr /var; do
        lvcreate -s ... # remember how to make a snapshot? i don't!
        bup index --exclude [...] $path
        bup save $path
        git note ... # create a note with useful stats
    done
    bup fsck --par2 --repair

Except that you don't need to remember all that, that it's logged
through syslog, handles locking, etc. (Notice also how I forgot to
create a mountpoint for the LVM snapshot, to mount it and to remove it
and the snapshot. `bup-cron` makes you not have to think about all
those pesky things.)

`bup-cron` does not do any sort of scheduling, that task is left to
`cron(8)` or the equivalent daemon on your system.

Branch naming
-------------

By default, `bup-cron` will store the backups in a branch named
`host-path` where `host` is the hostname of the machine (as returned
by the `hostname(1)` command) and `path` is the path to be backed
up. So for example, in the first example above:

    ./bup-cron -d bup Documents

The backups will be in the `example-Documents` branch (assuming the
hostname is `example`).

Snapshots
---------

If the `--snapshot` argument is provided, `bup-cron` will attempt to
make a snapshot of the current filesystem by guessing which `LVM`
device the target path is associated with. The snapshot is then
mounted on `/media/bup/vg-lv`, where `vg` is the Volume Group name and
`lv` is the Logical Volume name. That mountpoint path is configurable
with the `--mountpoint` option. The snapshot size is by default `1GB`
and can be tuned with the `--size` option.

A failure to create the snapshot will not abort the backup but will
spawn a warning.

Parity checks
-------------

If `--parity` is used, `bup-cron` will run `bup fsck -g` after the
backup, which in turn will call `par2(1)` to make parity blocks for
the backups.

Statistics
----------

If `--stats` is used, some basic statistics about disk usage before
and after the backup will be saved as a git note associated with the
backup. Example:

    $ bup cron --stat -d backup Documents
    $ git --git-dir backup show example-Documents
    commit af47078b8a787fff8f5cd42d067eb2fd92001c88
    Author: anarcat <anarcat@example>
    Date:   Thu Nov 6 04:02:20 2014 +0000
    
        bup save
        
        Generated by command:
        ['/usr/lib/bup/cmd/bup-save', '--quiet', '--name', 'marcos-foo', '--strip-path', 'foo', 'foo']
    
    Notes:
        Repository size
        
        * Before: 14.7KiB (15027 bytes)
        * After: 16.0KiB (16378 bytes)
        * Diff: 1.3KiB (1351 bytes)
        
        Local versions
        
        *    bup: debian/0.25-1
        *    git: 2.1.1
        * python: 2.7.8

The format of those notes shouldn't be relied upon and may change in
the future.

Also note that this will fail if git cannot be run. If you see the
following error:

    fatal: empty ident name (for <you@example.com>) not allowed

... it's because git isn't configured properly. In that case, you
should follow the instructions given by git and configure your
identity correctly, both on the local and remote servers, using:

    git config --global user.email "you@example.com"
    git config --global user.name "Your Name"

See `git-config(1)` for detailed information about git configuration.

Logging
-------

By default, `bup-cron` tries to be silent, so it can be run through a
cron job and leverage the typical *if there is output we send an
email* adhoc policy. In other words, if everything goes well,
`bup-cron` will produce no output. You can use the `-v, --verbose`
argument to print more information on the console. A single `-v` will
explain what `bup-cron` is doing, `-vv` will also show the
actual commands called and `-vvv` will also pass `-v` to those
commands. Example:

    $ bup cron -d backup foo
    Indexing: 1, done.
    bloom: adding 1 file (1 object).
    $ bup cron -vv -d backup foo
    configured stdout level 10
    locked pidfile backup/.bup-cron.pid
    indexing foo
    calling command `bup index --one-file-system foo`
    Indexing: 1, done.
    saving foo
    calling command `bup save --name marcos-foo --strip-path foo --tree --commit foo`
    Reading index: 1, done.
    Saving: 100.00% (0/4k, 1/1 files), done.
    bloom: adding 1 file (1 object).
    b316cd132d45aa9de3ca66d58a054fb819c70043
    3288df3ba7d515181fdf7d65f6bff836e4d9f042
    removed pidfile backup/.bup-cron.pid
    elasped: 0:00:00.650106 (user 0.06 system 0.01 chlduser 0.25 chldsystem 0.14)

However, `bup-cron` can also use `syslog(3)` to send logs to the
system log. Using syslog, all messages are logged and are sorted by
the syslog daemon according to their priority. By default, `--syslog`
will send messages up to the `INFO` level (equivalent of on
`--verbose` argument), an explicit level can be passed to `--syslog`
to send more information. For example this will send all message to
syslog, including `DEBUG`:

    bup cron --syslog DEBUG

Remote backups
--------------

Remote backups are done with the `--remote HOST` command, where `HOST`
is in the `user@example.com:path` format. In this case, only the index
is stored in the `--repository` and the files are stored remotely.

Remote backup support isn't well tested so feedback would be welcome
on its use.

Other options
-------------

More minor options are available and should be self-explanatory in the
`--help` output.

Caveats
=======

`bup-cron` is a fairly new project and has seen limited testing. It
may need a little hand holding at first, especially the logging setup
and configuration. You should test this system as you would any new
backup system. Bug reports are welcome by email (`anarcat@debian.org`)
or on the [Github issue queue][].

Limitations
-----------

`bup-cron` has been written on a Debian GNU/Linux (8.x Jessie) system,
and has seen little testing on other platforms. Any system with proper
POSIX semantics should be fine, and it has successfully be ran on
Windows.

The test suite is incomplete.

As shown in the examples above, when `bup-cron` calls `bup-index`, it
produces output because `bup index` doesn't have a `--quiet` flag
which makes backups unnecssarily noisy on the terminal. However,
through `cron(8)` it will stay silent, you can test this with
`nohup(1)`.)

Missings features
-----------------

Those are features that could possibly be implemented:

 * --test to run compare-tree after backup
 * support for snapshots on `BTRFS` and `ZFS`

More information
================

The [bup][] project has its own documentation which you will need to
perform restore and inspection of the backups.

See the `--copyright` option for legalese, a copy of the license given
to you is in the [LICENSE](LICENSE) file.

Project maintenance information is in the [HACKING](HACKING.mdwn)
file. A history of changes is in the [CHANGELOG](CHANGELOG.mdwn).

 [bup]: https://github.com/bup/bup
 [Github issue queue]: https://github.com/anarcat/bup-cron/issues