[Feature] V0 full encryption + support v1 + better arg handling + better logging + extra bip39 validation

[crass]

This is a major upgrade and refactoring that adds support for full V0 backup encryption, V1 backup decryption, better logging, and argument parsing. Included are the changes from #10. This should be backwards compatible with existing decrypted backups. Major refactoring is done such that there are now a class for the decryptor and encryptors. The method naming scheme was left as is, but that leaves much to be desired. This fixes #14.

[khimaros]

this is exciting to see! i'll be testing this out and will report my experience here.

[grote]

Awesome contribution, thanks a lot! I just browsed through a bit and wondered if it would make sense to split the v0/v1 parts into separate files instead of mixing both into huge files.

[crass]

Awesome contribution, thanks a lot! I just browsed through a bit and wondered if it would make sense to split the v0/v1 parts into separate files instead of mixing both into huge files.

Yeah, was I thinking there might be utility in having it as one file, like just download the file and use it. Currently that doesn't work because of the protocol buffer modules, which I was thinking of adapting to be put inline in the file. But thinking more about it, even then there's still the non-builtin dependencies, makes it not as simple as downloading or transferring a single file. So yeah, it makes more sense to break it out. I'm not sure I'll get to this soon though.

[khimaros]

tested this out, unfortunately it aborts when running show

i've verified that golang seedvault-extractor extracts this backup correctly, though it does skip a number of packages in unsupported states.

here's the output of the failure:

  org.connectbot
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/tink/core/_tink_error.py", line 34, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/tink/streaming_aead/_decrypting_stream.py", line 67
, in _read_from_input_stream_adapter
    return self._input_stream_adapter.read(size)
tink.cc.pybind.tink_bindings.PythonTinkException: INTERNAL: Authentication failed: 30594816:error:
1e000065:Cipher functions:OPENSSL_internal:BAD_DECRYPT:external/boringssl/src/crypto/fipsmodule/ci
pher/e_aes.c:1078:


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/tink/streaming_aead/_streaming_aead_wrapper.py", li
ne 96, in read
    data = self._attempting_stream.read(size)
  File "/usr/local/lib/python3.9/dist-packages/tink/streaming_aead/_decrypting_stream.py", line 98
, in read
    data = self._read_from_input_stream_adapter(size)
  File "/usr/local/lib/python3.9/dist-packages/tink/core/_tink_error.py", line 36, in wrapper
    raise TinkError(e)
tink.core._tink_error.TinkError: INTERNAL: Authentication failed: 30594816:error:1e000065:Cipher f
unctions:OPENSSL_internal:BAD_DECRYPT:external/boringssl/src/crypto/fipsmodule/cipher/e_aes.c:1078
:


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/seedvault_backup_parser/parse.py", line 1126, in <module>
    main()
  File "/srv/seedvault_backup_parser/parse.py", line 1114, in main
    kv_parsed = SeedVaultBackupDecryptor(args.backupfolder, password=args.password).decrypt()
  File "/srv/seedvault_backup_parser/parse.py", line 581, in decrypt
    return self.parse_backup(self.backupdir, self.targetdir)
  File "/srv/seedvault_backup_parser/parse.py", line 894, in parse_backup
    self.parse_apk_backup(pkg_name, pkg_metadata, key, salt)
  File "/srv/seedvault_backup_parser/parse.py", line 736, in parse_apk_backup
    self.parse_apk_data_backup(pkg_name, pkg_metadata, key, salt)
  File "/srv/seedvault_backup_parser/parse.py", line 692, in parse_apk_data_backup
    bytes_read = dec_stream.read()
  File "/usr/local/lib/python3.9/dist-packages/tink/streaming_aead/_streaming_aead_wrapper.py", li
ne 109, in read
    raise core.TinkError(
tink.core._tink_error.TinkError: No matching key found for the ciphertext in the stream

seems to fail while attempting to extract org.connectbot. perhaps a hint is that seedvault-extractor skips this package with the reason:

skipping "org.connectbot" (unsupported state "WAS_STOPPED")

[crass]

seems to fail while attempting to extract org.connectbot. perhaps a hint is that seedvault-extractor skips this package with the reason:

skipping "org.connectbot" (unsupported state "WAS_STOPPED")

This is odd. I thought I'd read that STOPPED apps do not get backed up. So there shouldn't be a backup data file for this app. Yet the code path taken indicated that there is such a file. I understand you to be saying that seedvault-extractor just skips this backup. It could be that there should be a backup, but that there has been some corruption of the encrypted file. I think that would produce a similar error.

I've added a change that will catch the exception you're hitting, output an error message, and continue on. The error message will be of the form Error: Failure to decrypt base64string: ..., where base64string should be the name of a file in the encrypted backup. Can you tell me the size of that file?

Also please tell me what versions you are running for python and the various modules. Here's mine:

absl-py      2.1.0
protobuf      4.25.3
pybip39       0.1.0
pycryptodome  3.20.0
tink          1.10.0

And I'm running Python 3.10.12 on an ubuntu derivative, and have tested on Python 3.11.2. tink version 1.9.0 is working for me as well.

If you versions do not match, please update them to those versions (perhaps in a virtual env if needed) and retest. Also, note that multiple -v options can be given to increase the verbosity level (up to two currently).

[crass]

@khimaros Looking at this again, I have many apps in the WAS_STOPPED state that have backup data that is successfully decrypted. Its unclear why seedvault-extractor skips all apps with a state that is not the empty string, but that seems unnecessary in my experience.

[grote]

Could it be that an app was backed up, then later entered the stopped state, so it is in WAS_STOPPED, but still has historic backup data available? Would be nice to figure this one out as it may point to a bug in Seedvault.

[Moini]

Mmmh, looks like this MR is stalled? Could this be made its own fork, as the repo owner doesn't seem to react?

(although I find seedvault horrible to use - no 'manual only backups', no 'backup all user data with the directory structure that is on the device', no 'I don't want encryption' choice, no 'backup to computer via mtp or whatever' (but you have to find a usb c adapter, and a usb stick, or need to waste tons of energy and bandwidth and space to backup into nextcloud, or have to own a device that has an sd card slot, or need to waste space on the actual drive, where it's not safe anyway - I have a laptop, and want to use it!). It's the worst piece of backup software I've ever encountered, but I don't have / want root, so this is what I'm stuck with, as apparently, adb backup doesn't work anymore, according to old deprecation messages. If you have any tips on how to backup the system to a (linux) pc directly, I'll take that.)