Add web3.eth.encrypt method for RPC & web3

[danfinlay]

Update: Parity Implemented

This EIP now recommends that the Parity methods encryptMessage and decryptMessage be added to the personal namespace.

https://github.com/paritytech/parity/wiki/JSONRPC-parity-module#parity_encryptmessage

Below is the original post, which is less refined. Below that is the full discussion.

Specification

Option 1: Add a new method, encrypt(account, data, [cb]), and decrypt(account, data, [cb]) or something else, that allows a user to encrypt & decrypt arbitrary lengths of data with an account's private key, as well as clarify the documentation of eth.sign.

Option 2: Clarify to current implementations that eth.sign should work for arbitrary lengths of data, not only with sha3 hashes, as well as add an encrypt method.

Rationale

Currently there is a web3.eth.sign method that is described as a general signature method in the wiki, but in practice, implementations have made it a de-facto signHash method.

There is still a valuable place for a method to sign and encrypt arbitrary data with an account's private key (or an arbitrary public key). For example, using an account's private key can be used to encrypt private data that is stored in public, like personal data. A recent casual example would be saving wallet nicknames in a secure way, but obviously more serious examples abound, including potentially medical data.

To reconcile the current lack of a general purpose data signing method, I recommend we either endorse a new method or change the old. Since applications exist and already rely on the current eth.sign, that's probably an inconsiderate option, so I personally think a new method is in order.

[aakilfernandes]

@FlySwatter actually just deleted the comment bc i realized you need a json rpc method, and that is eip fitting

[coder5876]

It's not really clear to me how encrypt(account, data, [cb]) should be defined. From the input it looks like we should be treating the private key corresponding to account as a symmetric encryption key and doing symmetric encryption with that key (i.e. no recipient public keys in the input parameters etc). If this is the case then we need to specify the encryption algorithm (AES? Which mode? XSalsa20? something else?) and it seems that we would be moving away quite a bit from what we would expect the RPC spec to support.

If we are doing asymmetric (public key) encryption/decryption (through something like ECIES), then the inputs to the encrypt function should be a public key or a list of public keys that we want to have as targets of the encryption, and we need to define a spec for the structure of the encrypted blob including nonces or IVs etc. Again it seems to be a lot of work that fall outside the realm of expected supported functionality of the RPC spec.

[danfinlay]

I think the method I’m really looking for is better defined as decrypt(account_number, data, algorithm, [cb]).

That’s because I think there’s a valuable place in letting a user decrypt data of an arbitrary size using their private key, unlike the current eth.sign method which only supports signing hashes.

The reason I think this is suitable for the RPC spec is because web3 currently provides a variety of methods which assume the user has a private key. Since web3 is already responsible for exposing an interface to a key pair, and we’re already doing the work of getting users to manage a key pair, it seems very strange that we limit that key to a single function.

To demonstrate the usefulness of this feature, try to answer this question: If a Ðapp wanted to use ether accounts to allow encrypted communication, what would the flow be?

Encrypting to a public key can actually be done fine outside the scope of web3, so this proposal probably needs some revision, but once a message is received, the user needs a method to decrypt an arbitrary blob with an arbitrary algorithm.

Currently, there’s no method to formally request this via the web3 interface, so the Ðapp would have to provide some copy-paste text, and trust the user to extract their private key manually, and feed it into an encryption algorithm themselves.

That’s a lot of work to utilize an encryption key that is held in every wallet. Since web3 has already assumed the job of integrating user-controlled encryption into the browser, I can’t see a better place to finish the feature.

I do understand that these operations aren’t explicitly required for the bare Ethereum protocol itself, but I don’t think web3’s job is to merely support Ethereum, it should aspire to support the next wave of distributed applications, and that includes public key cryptography.

[coder5876]

Yes, a decrypt function would be doable I think. But you would still need to define a corresponding encrypt function outside of web3, and make sure you can extract the public key in order for people to encrypt to you.

It's a tricky problem design wise for sure. Currently I'm leaning towards letting various dapps and/or whisper-like protocols handle the key management themselves and using your main Ethereum keys to "provision" these encryption keys (by signing the public encryption keys) and associate them to your identity through a registry or similar. But I do agree that there is value in using your keystore for encryption keys as well.

[wighawag]

Any update on this ?

the ability to encrypt and decrypt data for the user is a must.

The ability to sign arbitrary data with personal_sign has already be proven useful but whenever we want to share private data across device we would require encryption/decryption.

Note that parity is already implementing something along these line : https://github.com/paritytech/parity/wiki/JSONRPC-parity-module#parity_encryptmessage

it require the ability to get a public key out of an address though:
openethereum/parity-ethereum#5869

It would be great to have it standardised

[danfinlay]

I'd be happy to implement the feature as specified by Parity, but as eth_encryptMessage and eth_decryptMessage instead of using the parity prefix. I don't understand why Parity made this method platform-specific. Maybe @gavofyork would like to comment.

[danfinlay]

One reason might be that the decrypt method requires using accounts, so this means it requires a signer, which may not be the appropriate responsibility of the eth_ prefix. Maybe then this belongs on the personal_ API?

Also, since you don't need access to private keys to encrypt, it might make just as much sense to keep it on a separate utility.

Really what you need is a personal_decrypt function, and maybe nothing else, so you could rely on utility libraries like eth-sig-util for encrypting to public keys.

[wighawag]

encrypting would still require to get the public key out of the ethereum address

maybe a personal_getPublicKey ?

[danfinlay]

That's not a bad idea. like, personal_getPublicKey(address).

Right now you can also get the public key from any signature, so like personal_sign.

[wighawag]

except personal_sign require confirmation while personal_getPublicKey should not require any

[danfinlay]

Maybe. My understanding is that the whole "Ethereum address as hash of public key" thing is a security measure to protect the public key. If that's the case, automatically returning the public key would be a security issue. I'm not clear what that concern was, though. Would appreciate if someone could clarify it.

[aakilfernandes]

@FlySwatter i believe the main issue is that quantum computers can more easily (not sure how practically feasible it is) turn public keys into private keys. However, the public key is (I believe) made public with each transaction. In theory a node could track all the public keys of all known transactions and add them to a database.

[danfinlay]

Yes, the public key is made public with each transaction, so maybe we can accept that web3-browsers are "hot" enough that their public keys are assumed public, and the address-protection is more a resilience for cold wallets that have never been used.

[wighawag]

I agree, especially since once it is out it is out. It would be hard to explain to the user without scaring him that once it accept to release its public key it would be given forever

[wighawag]

I'd like to propose a different signature to make encrypt non vulnerable to a certain type of malicious dapp

Instead of

decrypt(account_number, data, algorithm, [cb])

it would be

decrypt(account_number, origin, data, algorithm, [cb])

The extra parameter origin purpose is to ensure a malicious dapp cannot request to decrypt data encrypted by another.

Let me explain by giving an example:

Let say a user is navigating to "onedapp.eth", The user trust that url and the javascript served from it. This dapp is not a malicious one and it deal with confidential data.

The javascript call encrypt and store that data in a deterministic location for later retriveal. Let say that such location can be known by anone having the user's address.

The user is then navigating to a new url "anotherdapp.eth".
The user is trusting its signer to protect them from malicious dapps.
The javascript file from "anotherdapp.eth" download the data from the deterministic location and request the signer to decrypt the confidential data encrypted by the trusted dapp above.

If the version of encrypt without origin was used, the signer could not know whether that javascript is allowed to request decryption or not.

And from the user point of view, it is impossible to know if "anotherdapp.eth" javascript is requesting to decrypt data that it has encrypted in an earlier visit to "anotherdapp.eth" and might accept it because it trusts its signer.

On the other hand, if origin was provided and stored along with the encrypted data, the signer can simply refuse the "anotherdapp.eth" javascript to request decryption from the user by checking the origin stored and the domain from which that javascript comes from.

It can work this way:

the dapp who want to encrypt data set origin to be the domain, swarm hash ... (or a multitude of them) allowed to decrypt it later

The origin is prefixed to the data, the data is then encrypted with the user public key. The dapp then store it somewhere

When a dapp request decryption, the signer decrypt it but check the prefix before sending the decrypted data to the dapp and interpret it as the allowed domains.

If they match the domain,swarm hash... where the javascript come from the signer could even skip asking the user, allowing dapp to provide a seamless experience to sync data across devices.

If they do not match, the signer refuse directly by giving an "not authorized" error

If no origin are specified, we could fallback to the behavior without origin but signer should present a warning message both for encryption and decryption.

I suspect the use of empty origin would only make sense when the caller is a human. When a javascript file is involved the trust involved means decryption should be prefixed at least with the origin from which the javascript file originate.

This brings another possibility for when origin is empty : the signer set automatically it to be the domain from where the javascript file requesting encryption come from

Note that, if it is manually provided (non empty), all domain provided should be considered. Alowing dapps to provide multiple domain or a combination of swarm hash, domain...

Note

While we could use a convention so the data to be encrypted require to have a prefix interepreted as origin, I think the origin parameter is more explicit and force dapps to provide it explicitely.

[catageek]

@wighawag Instead of twisting an encrypt/decrypt scheme with the 'origin' data, I would incline to the usage of an authenticated signature stored with the encrypted data, from the eth.sign() function for example, or a HMAC. I prefer a bare encrypt/decrypt with more security around.

I wonder if it is possible to have access to more operations on the key. for example, to build a linkable ring signature, we must perform modular operation with the private key. Today there is no way to do this operation, or entirely out-of-band. This need is in line with #208 where users will be able to adopt other schemes than ECDSA.

[wighawag]

What do you mean by twisting? it is basically just an extra field that can be used for dapp to indicate they were the one requesting encryption (or more accurately, who is allowed to decrypt ).

The signer is responsible to ensure that other dapps which do not match the "origin" field are not allowed to decrypt the encrypted data.

Note that the use of origin on encryption does not require confirmation from the dapp's user since it only requires its public key (assuming the signer give access to such public key from the address). So the api flow stay the same.

On the other hand the use of eth.sign would require confirmation of the user but I am not sure to follow where eth.sign compete with "origin".

The idea behind the "origin" field is to allow dapp to encrypt data without the risk of another dapp decrypting it ( and thus exposing potential confidential data).

By providing "origin" it would also allow the dapp that fit the "origin" field to request decryption without confirmation (as "origin" indicates that it was that same dapp who requested encryption in the first place ( = it had access to the decrypted data and thus there would be no point to ask confirmation for decryption, providing a nice user experience for dapp that require saving encrypted data regularly on behalf of the user) ).

eth.sign would add a confirmation flow in encryption and would not protect another dapp for requesting decryption unless there is a field (like origin) indicating who/which dapp requested encryption in the first place and signer take this field in consideration before giving out the decrypted data.

But maybe I missed something?

I am not sure how linkable ring signature relates to the "origin" field and its added safety. Could you explain a bit more?

[catageek]

Hi @wighawag

Forget the point about linkable ring signature, it has nothing to do with encrypt/decrypt. It is just a thought I had when writing.

It is a good idea to have an 'origin' or something like that to define who has the right to get the plain data. There are one major issue that I see with the implementation of origin that you detail: the decryption comes before the verification of the access right. This is not a good practice and there is no way to prevent any web site to request an access to the data, with the user having to enter his password to validate the (possibly fake) request. It will be very intrusive.

So, instead of storing:
ECIES(origin||data)

we need to store origin in plain text to validate the request:
ECIES(data), origin

But now, origin is not protected against unwanted modification.

So I thought a lot about a better solution. Here it is:

When a dapp wants to encrypt some data:

1. A pair of key (Pk,pk) of secp256k is generated or provided by the dapp
2. We compute E = ECIES(data) using Pk
3. We compute S = ECDSA(E||origin) using pk
4. We compute Ek = ECIES(pk) using the public key of the user
5. We store (E, S, origin, Ek) in the user storage

When a dapp wants to decrypt the data:

1. We check the signature S using E||origin
2. We check origin against the dapp request
3. If origin is accepted, we decrypt Ek to get pk (needs user password)
4. We check that pk is the private key that signed S
5. We decrypt E with pk to get data

So, to encrypt we do not need the user's password, and to decrypt we ask the password after the origin check. It is up to the dapp to decide if it keeps Pk and pk.

We can define (Pk,pk) as an external account protected by the user's password: It would be wise that clients manage these kind of accounts that would be potentially shared by the dapps.

We could also rename origin to acl, for access list because it is what it defines, and because we have ECDSA(E||acl) we can manage the access control in chain with a transaction tx = E||acl signed by Pk and received by a smart contract that will manage the access, like parity did with secret store. When the called contract will be able to pay for the transaction, it will possible without the intervention of the user.

So:

• We need to define the semantics of origin aka acl
• We need to define and implement external accounts, ie account for which the private key is shared with the dapp
• We need to have more controls on secp256k curve to get some keys and sign messages without the need of a personal account involved in the process, but external accounts will be able to use it.

[wighawag]

@catageek Thanks for a more details answer but I am not sure I get your proposal.

The key purpose of the origin field (which I agree to rename to access_list or something similar) is that it is checked by the signers, not the dapps. In your proposal the signer seems absent and as such I do not see how it will prevent other dapp from requesting the data from the user. I am not sure what is "We" in your proposal?

There are one major issue that I see with the implementation of origin that you detail: the decryption comes before the verification of the access right. This is not a good practice and there is no way to prevent any web site to request an access to the data, with the user having to enter his password to validate the (possibly fake) request. It will be very intrusive.

You did not seem to understand here the involvement of the signer. The signer's role is to prevent unauthorized access to the data. The signer would decrypt first but check the access_list field before sending out the decrypted data to the dapp. In other words, while decryption happen before, it is not given to the dapp until the access_list field is checked. By decrypting first there is no need for a signature when encrypting, allowing dapps to encrypt without user intervention. Furthermore there would be no need for user intervention in decrypting too since access_list would have been decided by the dapp that encrypted the data in the first place, which is pretty nice.

[catageek]

@wighawag

The key purpose of the origin field (which I agree to rename to access_list or something similar) is that it is checked by the signers, not the dapps. In your proposal the signer seems absent and as such I do not see how it will prevent other dapp from requesting the data from the user. I am not sure what is "We" in your proposal?

I deliberately used 'We' because, in my opinion, the operations are to be distributed to different actors: the dapp, the client( Geth, Parity, etc.) and the user. it is not defined who is 'We' in each operation.

If another dapp wants the data, it will have to match the access list policy (defined by the original dapp), and this first check is made before the user intervention, and only after the user has to decrypt the external account key. The user can check the acl at this step, like:

dapp at othedapp.eth wants to open external acount originaldapp.eth. This operation is authorized by the (local|contract) access list [other details here]. Do you want to proceed ?''

Such a message will not pop up if the access list did not match early in the process, protecting the user from annoying pop ups.

Furthermore there would be no need for user intervention in decrypting too since access_list would have been decided by the dapp that encrypted the data in the first place, which is pretty nice.

If I try to rephrase, you want the dapp to provide a secret at encryption and decryption, and you want the user to check this secret before giving the data. This implies that the dapp stores this secret on her side, in a (centralized) database. This implies also that the user will check if she trusts otherdapp.eth if it provides the good secret, without knowing if the access is legitimate or fraudulent (it may be wanted by original dapp, but original dapp could have been hacked, or leaked the secret).

What I propose is that the original dapp provides an access list policy, that can be local or on chain. The dapp does not keep anything on her side, and this simplifies a lot the system and is in line with its essence of 'decentralized application'. The access list may contain any address that the original dapp provides, and it is upgradeable. It allows a dapp to share some data with another dapp, or to share some data with its upgraded version. The user has only to check that the access list is legitimate, and does not have to decide if otherdapp is legitimate or not: if original dapp has recorded otherdapp in the access list, then it is legitimate.

Basically, the dapps says implicitly 'here is some data and the access list policy associated with it, please protect the data and respect the access policy when giving access to the data'.

[wighawag]

Such a message will not pop up if the access list did not match early in the process, protecting the user from annoying pop ups.

This is also achieved using the simple access_list field since the signer will only show a "non authorized" popup when a malicious dapp will request to decrypt the data. There would be no popup otherwise

If I try to rephrase, you want the dapp to provide a secret at encryption and decryption, and you want the user to check this secret before giving the data. This implies that the dapp stores this secret on her side, in a (centralized) database. This implies also that the user will check if she trusts otherdapp.eth if it provides the good secret, without knowing if the access is legitimate or fraudulent (it may be wanted by original dapp, but original dapp could have been hacked, or leaked the secret).

No that is not how it works, there is no extra "secret" involved and dapps do not require to store any extra info :

I ll just describe the flow :

imagine a Dapp :

1. its javascript file from swarm hash "0xeeeee" generate confidential data
2. this same javascript request the signer to encrypt data (with access_list set to be "0xeeeee")
3. signer simply encrypt using the user public key (or alternatively the javascript have access to that public key and encrypt the data itself)
4. the dapp save that encrypted data somewhere

the user leave the page
and come back to it later

5. the javascript file from swarm hash "0xeeeee" get the encrypted data back
6. it then call decrypt
7. the signer decrypt the data (keeping it to himself at this stage)
8. the signer check if the access_list field exists and match the origin of the javascript file requesting decryption (in that case this is "0xeeeee")
9. since it matches, the signer give the decrypted data back without requesting permission from the user
   In case the request was coming from a different swarm hash, the signer would reject and not give back the decrypted data (could show a "non authorized popup" alerting the user of what that dapp tried to do)

Hope it makes sense. The important bit is that the signer is capable of checking from where the javascript file come from, similar to web browsers.

Basically, the dapps says implicitly 'here is some data and the access list policy associated with it, please protect the data and respect the access policy when giving access to the data'.

This is what my proposal aims to do too by sending that message ('here is some data and the access list policy associated with it, please protect the data and respect the access policy when giving access to the data') to the signer. I did not get yet how yours do it though.

[catageek]

Ok I get your point: you wish same origin policy for the encrypted data, and my proposal looks more like CORS because I think that there will be some needs to have cross-dapp data.

Maybe the CORS/SOP topic should be separated from the encrypt/decrypt topic.

[wighawag]

you wish same origin policy for the encrypted data

That's correct, that's where I picked the word "origin" but the main difference here is that the location of data is not relevant, hence it is not protecting "access" as such, but "decryption". Hence this scheme allows data to be stored anywhere.

Maybe the CORS/SOP topic should be separated from the encrypt/decrypt topic.

I won't comment about your proposal but for access_list proposal I strongly disagree: It is important for that field to be part of the encryption api since it protects from malicious dapp to request decryption of data encrypted by third party dapps. It is also adding potential user experience benefit that should not be taken lightly.

I thus strongly recommend adding that field to the encryption api and make it a mandatory standard for signers to consider it when dealing with encrypted data.

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[github-actions]

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.