Syncers

Syncers

Currently supported syncers:

• RSync

The difference between Storages and Syncers

Storages are part of the main Backup procedure. The main backup procedure is the one where the following actions take place:

• The copying files/dumping databases to the ~/Backup/.tmp directory
• The packaging (tar'ing) of the copied/organized files
• The (optionally) compressing of the packaged file
• The (optionally) encrypting of the packaged file
• The storing of the packaged file

The last step is what storages do, store the final result of a backup file to the specified destination.

Syncers completely bypass this whole procedure. They are meant to instantly transfer folders of data from the production server to the backup server. This is extremely useful if you have lots of gigabytes of data you need to transfer, for example "user-uploaded-content" that over time built up to 50GB worth of images, music, videos or other heavy file formats. With a Syncer you would basically just say: "Keep a mirror of this folder (/var/apps/my_app/public/music/) on my backup server in (/var/backups/my_app/music)". Then every time you run Backup on this Syncer, it won't copy over 50GB of data, then tar it, and then transfer it. This'll transfer the actual user@server1:/var/apps/my_app/public/music/ folder to user@server2:/var/backups/my_app/music. This way no additional disk storage on your production box will be used to store the temporary files (copy + tar) before the transfer, which can be very CPU intensive, slow and expensive.

Examples

Below are examples you can copy/paste in to your "Backup" configuration file. These blocks should be placed between Backup::Model.new(:my_backup, 'My Backup') do and end.

RSync

sync_with RSync do |rsync|
  rsync.ip       = "123.45.678.90"
  rsync.port     = 22
  rsync.username = "my_username"
  rsync.password = "my_password"
  rsync.path     = "~/backups/"
  rsync.mirror   = true
  rsync.compress = true

  rsync.folders do |folder|
    folder.add "/var/apps/my_app/public/uploads"
    folder.add "/var/apps/my_app/logs"
  end
end

Additional Notes

The rsync.mirror option, when set to true will tell RSync to keep an exact mirror of the files, on your production box, on the backup server. This means that when files get removed from the /var/apps/my_app/public/uploads folder it'll also remove these files from the backup server during the next sync. When set to false, it'll ignore removed files and just keep them on the backup server.

The rsync.compress option, when set to true will tell RSync to compress the data that'll be transferred. The compression is only meant for transferring data, this improves transfer speed and lowers bandwidth usage. Turning this option on will resort in more CPU usage for the compression. Once the changes have been transferred, it'll automatically uncompress back to it's original state.

The folder.add method allows you to add the folders you want to sync from the production server to your backup server. When a path ends with a '/' (forward slash) it'll only sync over the contents (and sub-folders) of that folder. If the provided path does not end with a '/', it'll create that folder on the backup server (thus, syncing the whole folder, including it's contents and sub-folders).

Default Configuration for the RSync Syncer

If you want to create multiple RSync syncer objects then you might want to configure some default settings to reduce redundancy. For example, if you always want to enable mirroring and compression for RSync syncers, and you always want to use the same server ip, port, username and password, then you can do the following:

Backup::Configuration::Syncer::RSync.defaults do |rsync|
  rsync.ip       = "123.45.678.90"
  rsync.port     = 22
  rsync.username = "my_username"
  rsync.password = "my_password"
  rsync.mirror   = true
  rsync.compress = true
end

With this in place, whenever you want to use the above default configuration you can just omit it from your backup models, like so:

sync_with RSync do |rsync|
  rsync.path     = "~/backups/for_my_other_app"

  rsync.folders do |folder|
    folder.add "/var/apps/my_other_app/public/uploads"
    folder.add "/var/apps/my_other_app/logs"
  end
end

Since we didn't specify the ip, port, username, password, mirror and compress options, it'll default to the values we specified in the configuration block.