Autoreplicate May Consume All Local Storage Space

[ifadams]

Describe the bug
Autoreplicate creates a local tarball of the the database data and metadata at a set time interval.

However, there are no checks for how often this runs, nor any infrastructure for rotating/removing old database backups. Because of this, a misconfiguration (or just a very long running server with a large database) can potentially consume all the available space in the target location silently.

To Reproduce

At configuration time, set a small number for the autoreplicate interval, (e.g. 5) with the units in seconds.

for example:

{
    "port": 55555,
    "autoreplicate_interval":5, // it should be > 0
    "unit":"s",
    "max_simultaneous_clients": 100,
    // "backup_path":"backups_test", // set this if you want different path to store the back up file
    "db_root_path": "db",
    "backup_flag" : "false",
    "storage_type": "local", //local, aws, etc
    "bucket_name": "minio-bucket",
    "more-info": "github.com/IntelLabs/vdms"
}

This will generate a new tarball every 5 seconds, until the server is killed.

Expected behavior

Ideally we should have some guardrails to prevent over-aggressive runs of the autoreplication thread (e.g. a loud warning or a graceful crash if we have an interval of a few seconds) as well as some defaults number of maximum snapshots/backups to be maintained and automatically removed.