Skip to content

Latest commit

 

History

History
125 lines (88 loc) · 7.29 KB

README.md

File metadata and controls

125 lines (88 loc) · 7.29 KB
Raw

CRATE (Container Recovery and Archival ToolsEt)

Introduction

CRATE is a utility I wrote in my final days at CERN to facilitate the backup and restore of local container data (Docker, podman etc). This is a majorly re-factored fork to remove any CERN branding or propriatery libraries, but also add some new features.

CRATE has 2 modes:

  • backup-local (Runs on localhost, auto/silent with cron/sysd, backups up data to a local path and optionally a local or sftp offsite path)
  • recover-local (Runs on the local host, restores a backup archive to the data path - interactive only) CRATE used to support backup remote and recover remote (multi host single instance via SFTP), but this was deprecated due to design shortfalls and reliability issues.

For EL distibutions, there is a draft spec file included (in example dir) if you wish to build and distribute as RPM. For any other distro, you will need to manually git clone to /usr/share and configure cron (example also included). I may consider the inclusion of CRATE as a container in an upcomming release.

CRATE requires container data to be stored in a local directory, NOT within a docker volume. This is for 2 reasons - Firstly, this tool was initally designed to be used primarily with a really old Podman version where volumes weren't mature, Secondly, I much prefer a flat file system approach for my production containers as I like to know where my critical data actually lives. If docker volume support is heavily demanded, I may implement this in the future...

   

How to use

  1. Git Clone this repo to /usr/share
  2. Install python3 and any additional dependencies (a requirements.txt is provided but you may need the python3-pkgname packages depending on distro)
    • Colorama
    • ast
    • configparser
    • datetime
    • pysftp
  3. Edit conf/config.ini to configure CRATE for your environment
  4. Run CRATE with syntax: python3 src/crate.py -m $mode $args (or let cron take care of it...)

Arguments:

Short Arg Long Arg Action Applies to
-h --help Display help info All cases
-m $mode --mode $mode Select which mode the script runs in All cases
-z $zip_path --zip $zip_path Provide zip to recover from Recovery mode (mandatory)
-srm --skip-rm Skips removing source dir before unzipping when recovering Recovery mode (optional)
-fp --fix-perms Chowns and chmods recovered data to values in config.ini Recovery mode (optional)

Example commands:

backup-local (bl)

  • Run command: Short example: python3 src/crate.py -m bl Long example: python3 src/crate.py --mode backup-local

recover-local (rl)

  • Run command: Short example (skip remove source): python3 src/crate.py -m rl -z /tmp/2023-08-23---16-25-25.zip -srm Short example (fix perms): python3 src/crate.py -m rl -z /tmp/2023-08-23---16-25-25.zip -fp Long example (skip remove source): python3 src/crate.py --mode recover-local --zip /tmp/2023-08-23---16-25-25.zip --skip-rm Long example (fix perms): python3 src/crate.py --mode recover-local --zip /tmp/2023-08-23---16-25-25.zip --fix-perms

   

Config file breakdown

This section provides an explanation for each configurable within the config file.

Main

Configurable Description
SOURCE_DIR The directory that will be backed up in backup mode, or written to in recovery mode
DEST_DIR The directory that will be written to in backup mode, contains the zips to restore from

Permissions

Configurable Description
CONTAINER_USER Recovery mode with fix permissions enabled will chown the restored data to this user
PERMISSION Recovery mode with fix permissions enabled will chmod the restored data to this chmod value

Offsite

Configurable Description
OFFSITE_BACKUP_ENABLED "True" or "False" Enables/Disables creating a 2nd offsite copy
OFFSITE_BACKUP_MODE "local" (copies to a local file path) or "sftp" - Uploads to SFTP server
LOCAL_OFFSITE_BACKUP_DIR Path for offsite backup in local mode
SFTP_USER Username for SFTP Server
SFTP_PASS Password for SFTP Server
SFTP_HOST SFTP Server address/ip
SFTP_PATH SFTP Server backup directory path

Retention

Configurable Description
LOCAL_RETENTION_ENABLED "True" or "False" Enables/Disables retention cleanup on the local copy
OFFSITE_RETENTION_ENABLED "True" or "False" Enables/Disables retention cleanup on the offsite copy
LOCAL_RETENTION_PERIOD_DAYS Store backups for X days. Older backups will be deleted.
OFFSITE_RETENTION_PERIOD_DAYS Store backups for X days. Older backups will be deleted.

NTFY

Configurable Description
NTFY_ENABLED "True" or "False" Enables/Disables Ntfy notifications
NTFY_HOST Ntfy server URL
NTFY_TOPIC Ntfy topic to subscribe to

   

Future expectations

Im a solo SysAdmin/Homelabber who wrote CRATE to automatically backup my own container data. This is not a full time job and I am not a software developer by trade. I anticipate updates (especially feature adds) will be slow at best. All the time it functions to a good enough degree in my homelab environment, thats that. That said, im open to taking feature suggestions and im open to anyone else who wishes to contribute/fork/maintain. This should be considered FOSS.

Screenshot

screenshot