Does rustic verify blobs before upload?

[duracell]

Hey, I'm using restic and rustic together, depending on different hosts.
I use the max compression level which seems to corrupt data sometimes (see Restic Forum). Beside the fix itself there is also a new function that verify blobs before uploading (Github).
My Question would be, does rustic also do such a verification to catch problems and if not, would it be safer and be planned to do this?
I tried to search in the Issues, Discussion and Code but couldn't find an answer.

Thank you!

[aawsome]

Hi @duracell and thanks for the question!

Currently there is no verification of blobs before upload. As with restic, such problems would be detected by check --read-data.
If there is the wish for an additional verification, we can add this possibility, but I would favor to have this as optional feature which can switched on using CLI options or the config file. If you wish this to be implemented, please open a feature request issue - best in https://github.com/rustic-rs/rustic_core, where this needs to be implemented. I think the implementation would be not too hard, but might slow down backup runs a bit...

Note that rustic uses the original C implementation of the zstd algorithm by facebook (https://github.com/rustic-rs/rustic_core) and provides the compression parameters defined there (e.g. up to level 22), like e.g. the zstd command.
I remember there was once a similar bug for the highest possible compression rates but that AFAIK has been fixed and did not appear again, so far.
By using the identical codebase, rustic is as vulnerable to such things like many tools which are widely used and which do not directly check the compressed content by decompressing and checksumming it.

[aawsome]

I wanted to add that I can totally understand the restic maintainers' decision to always run such a verification. They use a Golang-native implementation (for good reason, they wanted to circumvent dealing with cgo) which now produces this kind of critical bug a second time. Don't want to blame anyone, I think the go zstd author does a incredible good job - these things can and do happen! But the fact that it seems to be detected by restic users does indicate that the userbase of the library is not as large and especially there are not that many fuzzing tests running against it. This makes the decision a very reasonable one.

For rustic using a very widely used library (which I think is much more heavily fuzzy tested) the risk judgement is IMO a bit different. But you are right, better safe than sorry, so I strongly do support allowing users to self decide how to deal with that risk.

[duracell]

Thanks for this very fast and detailed feedback!
I created an issue for this: rustic-rs/rustic_core#153