Support for APFS Snapshots

[philipkozeny]

As discussed in #829, there are some filesystems that support snapshots, like BTRFS, ZFS and APFS. It would be convenient to create an automatic APFS snapshot, create the backup on the snapshot and destroy it after the backup has finished / failed.

For APFS it would work like this (simplified)

$ tmutil snapshot /
$ mount -t apfs -o -s=com.apple.TimeMachine.%snapshot-date-time-from-output% / /tmp/snapshot
# run borg create on /tmp/snapshot/%PATH%
$ unmount /tmp/snapshot
$ tmutil deletelocalsnapshots %snapshot-date-time-from-output%

[philipkozeny]

@m3nu I am thinking about working on a PR for that specific feature. I am not sure though where to add the functionality. Would this be something vorta or borg should implement (if there is interest at all). What do you think?

[m3nu]

I'm not sure Vorta is a right place to add this, since APFS is not platform-independent. Unlike Borg and Qt.

Sounds more like a new project to be honest. Also since the concepts (archives, extract, compression...) will be different from Borg.

Would of course be nice to have a GUI for APFS snapshots. Or ZFS. Great for local backups.

[philipkozeny]

I think we are talking about two different things here.

First of all I am thinking of just creating a snapshot of the filesystem and using borg / Vorta to use the snapshot as a backup source. That way we could guarantee that the filesystem is in a consistent state while doing a backup and files are not changing while the backup is running.

So before running 'borg create' create a snapshot just in time and use the mounted snapshot as a data source and throw it away after the backup. This is also how other backup tools (restic, duplicity, Arq, ...) do that.

Second this should be an interface to support different file systems (APFS, ZFS, BTRFS, ...) but just utilizing the snapshot feature to create a backup.

[m3nu]

Ah. Then I misunderstood. Using it like this makes sense in the context of this project I think. We already have several OS-dependent packages. Like for networking, autostart, keyring.

To clarify:

• Root permissions needed to take a snapshot? Vorta generally runs without root.
• How to sources relate to the snapshot? Is there one snapshot per source folder?
• Need to make sure snapshots are removed after the backup. Even if Vorta or Borg crashes. Maybe naming them in a special way and then finding orphaned snapshots during startup?

[sten0]

I suspect there may be overlap with the Linux equivalent of this feature. In Linux one would traditionally make an LVM snapshot (requires root for snapshot creation and deletion); however, btrfs snapshots are finally a well-supported alternative. Btrfs snapshots can be made by a normal user, but deletion requires root. Unfortunately, user creation of snapshots introduces the potential for a local denial of service attack, which is one of the reasons that I suspect that MacOS has probably made this an admin-only capability. If I remember correctly, MacOS's root is now a restricted account that does not posses full admin capabilities. This is the opposite of Linux, where permissions less that super-user can be granted as a subset of CAP_SYS_ADMIN.

It seems to me that it will be necessary to add either an OS-dependent abstraction layer or some kind of case structure when Vorta takes the next step towards ensuring consistent backups, because backing up live databases is always potentially unsafe.

The most simple, and potentially naïve way that I can think of doing this is to introduce a configurable path for a hook script that takes a minimum of two arguments like pre and post or create and cleanup. The OS-specific method, along with the method for getting admin permissions is then controlled by the user. The user can then also define the preferred post or cleanup behaviour; not everyone wants the snapshot to disappear if the backup fails. Example scripts could be provided in the conventional way: a contrib subdir of the project.

There is, however, one major problem with this proposal: If the borg process doesn't run with admin privileges then backups can only be made of files accessible by the user session/seat that is running. Also, this method won't work with Flatpak which has its own set of of complicated permission issues (this is one of the many reasons I don't believe Flatpak "is the future"). @m3nu, this may be a question for for mid to late 2022 or 2023, but are you still necessarily against allowing the called borg process to run with admin privileges? From a user perspective a checkbox with "run with admin privileges" would then check for the correct OS-specific method of getting these capabilities, and error early if this fails. If a hook script is also necessary for snapshot creation/deletion, then this checkbox could have a tooltip like "Enables admin capabilities for the (optional) hook script and borg process". This also allows restored backups to have the correct user permissions, because the user that calls "borg extract" may not be the user who called "borg create". Please let me know when you'd like me to contact my Debian colleagues about the best way to do this in PyQt ;-)