[FEATURE] Allow a HO user to securely submit an encrypted database to the HO developers

[tychobrailleur]

Is your feature request related to a problem? Please describe.
Sometimes when a user encounters a problem with HO, developers need to be able to see their database to be able to debug the problem. In the past, users have been sharing their db by uploading it to GitHub, which allows anyone to access their team details.

Describe the solution you'd like
HO should provide a feature that automatically encrypts the database and uploads it to GitHub, with only the HO developers able to decrypt, so the data is fully protected.

Additional context
N/A

[tychobrailleur]

I have started work on this, and my current thinking on this one is as follows:

• First create a zip file containing the content of the db (and logs?) of an HO instance (facilities already exist in core.db.backup)
• Generate a random 16-byte secret used to encrypt the zip file (using AES-256, CBC mode)
• Encrypt secret with a pre-defined public key (EC, with curve P-256)
• Create a new package with encrypted file + encrypted secret
• Submit that package as part of a new issue to Github using REST API (I think this requires a new Github App with OAuth)

We would then have to have a small utility to decrypt the DB on our side, and agree on how to manage the secret key to decrypt. Thoughts?

[wsbrenk]

@tychobrailleur What would you find if we used pgp for example and built a public key into HO from each developer, which the user can deselect individually when creating the backup.
I don't know if it is possible, but a nice feature would be, if the encryption keys will expire after a defined timespan.

[tychobrailleur]

@tychobrailleur What would you find if we used pgp for example and built a public key into HO from each developer, which the user can deselect individually when creating the backup. I don't know if it is possible, but a nice feature would be, if the encryption keys will expire after a defined timespan.

The algorithm I described is pretty much pgp, except the public key is not a signed cert, rather a key we would embed in HO. I however like the idea of a cert, because we could retire it if we need to -- it would have to be self-signed though because I can't afford a proper cert. Also I believe pgp is RSA-based rather than ECC.

I was also thinking one key for all (therefore devs would share the same private key), one key per developer might complicate things further, although there are probably 1-out-of-n schemes that are possible... I can research further. I don't want to roll out my own crypto, though.

Thanks for your comments.

[tychobrailleur]

Also I believe pgp is RSA-based rather than ECC.

Interestingly ECC is not supported yet in plain JDK 17, so I may need to revert to RSA. I knew that, having had that issue in the past, but had forgotten – and the solution would be to use BouncyCastle, but if we can avoid adding that dependency, all the better. Current POC encrypts the zip with a random secret with AES, then encrypts that secret with a hard-coded RSA public key.

[tychobrailleur]

Current POC encrypts the zip with a random secret with AES, then encrypts that secret with a hard-coded RSA public key.

POC now also decrypts key, and successfully decrypts the zip to create a valid HO DB backup. Now onto making this production code. :)

[tychobrailleur]

Some good progress here albeit slow as I have been busy at work: started to clean up the code, and removed the “decryption” code (probably a separate app in the future?), and started looking into Github Apps. Shaping up nicely, getting excited by this!

[tychobrailleur]

I have implemented a successful “on-behalf” authentication for the Github App. The next step is to store the access token for the github user in the database, make the UI for the OAuth flow, and create the GH issue along with the encrypted DB.

[tychobrailleur]

I managed to automatically create a Github issue using my new HOReport Github app:
https://api.github.com/repos/tychobrailleur/NotHO/issues/1

I haven't been able to create one under ho-dev/HattrickOrganizer ("Resource not accessible by integration") despite permissions being set correctly. I think it's either due to where the Github app is created, or the owner of the token.