Warning: forgetting snapshots without instance ID

[ovizii]

Running backrest 1.1.0 via docker. I have 3 plans and I have been happily using backrest since it was first announced on the self-hosted reddit.

I now have my first warning inside the logs:

backrest  | 2024-06-14T04:01:56.829+0200        INFO    task finished   {"task": "index snapshots for repo \"OneDrive1TB\"", "duration": "3.847400696s"}
backrest  | 2024-06-14T04:01:56.829+0200        INFO    running task    {"task": "forget for plan \"OneDrive1TB\" in repo \"Pictures\""}
backrest  | 2024-06-14T04:01:56.850+0200        DEBUG   auto-unlocking repo     {"repo": "OneDrive1TB"}
backrest  | 2024-06-14T04:01:59.045+0200        WARN    forget for plan "OneDrive1TB" in repo "Pictures"        found legacy snapshots without instance ID, recommending legacy forget behavior.
backrest  | 2024-06-14T04:01:59.048+0200        WARN    forget for plan "OneDrive1TB" in repo "Pictures"        forgetting snapshots without instance ID, using legacy behavior (e.g. --tags not including instance ID)
backrest  | 2024-06-14T04:01:59.048+0200        WARN    forget for plan "OneDrive1TB" in repo "Pictures"        to avoid this warning, tag all snapshots with the instance ID e.g. by running:
backrest  | restic tag --set 'plan:Pictures' --set 'created-by:pve02' --tag 'plan:Pictures'
backrest  | 2024-06-14T04:02:02.317+0200        DEBUG   forget snapshots        {"plan": "Pictures", "count": 1, "policy": "policy_time_bucketed:{hourly:24 daily:7 weekly:4 monthly:6 yearly:3}"}
backrest  | 2024-06-14T04:02:02.317+0200        DEBUG   forget for plan "OneDrive1TB" in repo "Pictures"        found 3 snapshots were forgotten, marking this in oplog

I can follow the advice, but I was wondering if this was a new feature? I know that "instance ID" was introduced by when it was introduced, I used the same name which I had previously given my docker container using hostname, so there was no change.

[garethgeorge]

Hey, good question & this is (imo) probably one of the more interesting changes in 1.0.0 --

tl;dr this warning is benign and does not affect the correctness of your forget operations if you're not sharing restic repos with multiple backrest installations. That said, there's no downside (and some small upsides) to proactively migrating your snapshots with the suggested command! In a future version (probably ~6mo - 1year from now) backrest may execute that migration automatically on lingering snapshots and deprecate the legacy compatibility fully at that point.

more details The addition of this warning and some changes to how snapshots are grouped when running forget was one of the bigger changes in 1.0.0 (which in addition to the new feature work and general stability improvements justified the major version bump!). In 1.0.0 backrest will group snapshots by both the plan tag and the created-by (instance ID) tags rather than just the plan. This addresses a pre-1.0.0 potential data loss bug if using multiple backrest installations with a single repository. If two installs have conflicting plan IDs (e.g. two hosts both defining a 'documents' plan) it's possible for them to forget eachother's snapshots if only grouping on plan. Fixing this is was a pre-requisite for multi-host backup management and much of the recent refactoring work I've done is with the goal of improved stability and paving the way for that on the roadmap.

[ovizii]

If two installations have conflicting plan IDs (e.g. two hosts both defining a 'documents' plan) it's possible for them to forget each other's snapshots if only grouping on plan.

I understood your explanation but does this mean backrest is relying on grouping snapshots by the plan and created-by tags and ignore the value of Host? I'm not questioning this, just making sure I fully grasped the idea. If I got this right, Host is completely ignored, even though its value might be the same as the created-by tag, right?

[garethgeorge]

That's right -- the host value is left as the actual hostname of the computer creating the backup and, as you say, it's ignored for grouping. Backrest only cares about the plan and created-by tags.

Adding better documentation of this is definitely something that should be on my todo list!