Ethereum wallet & transaction support

[ilblackdragon]

Proposal to support using Ethereum wallets & transactions on NEAR.
This opens up an existing broader Ethereum user base to interact with NEAR apps in a familiar way.
Reduces some of the integration work on adding NEAR support for infrastructure partners (at least for Ethereum formatted addresses).

User journey

The desired user journey, is a user coming to a NEAR app like https://app.ref.finance or https://mintbase.io, connect with their Metamask or other Ethereum wallet. The wallet would pop up to switch networks to "NEAR" and allow user to transact by signing with their Ethereum wallet.

Users are familiar with finding a bridge to move assets from Ethereum Mainnet or other existing network to NEAR if they can connect with Metamask. Additionally, if user doesn't have any assets, a Bridge UI can be shown to suggest to bridge (or deposit directly for something like dex.woo.org) assets automatically.

Technical details

near-api-js should support to add NEAR network with a RPC that supports Ethereum endpoints that Ethereum Wallet needs.

Special RPC endpoints are needed to be added that respond with required information: chainId, blockNumber and account information.

Transaction signing in this case happens by packaging NEAR transaction information into an Ethereum transaction format. The NEAR information will go into calldata field. from - signer address, gasLimit and value are used for respective values. The rest are ignored.

Addresses

New implicit account format will need to be added that fits Ethereum address format "0x[\w\d]32". These addresses can be accesses by a public key that matches keccak(public_key)[:20]. These will still be a normal NEAR accounts, as it's a valid account ID under current rules.

One caveat to keep in mind is that Ethereum is using capitalization as a way to do check sums, and all tooling will need to be able to handle that.

Handling legacy accounts

There are a number of legacy 0x[\w\d]32 accounts that are not matching to user's key. From user experience perspective, it's important to block recieval of assets to these accounts.

[almal]

Having Metamask users will be a huge help in user acquisition.
It could also help in B2B adoption as well - it seems like corps have got used to Metamask as standard realoble secure web3 wallet.
Wondering if subaccounts can be somehow involved - like MM user comes to Mintbase and magically but meaningfully affordable gets 0x12344567.mintbase.near account.

[ilblackdragon]

To compare this with Snaps: this offers not just Metamask support but all possible Ethereum wallets and existing tooling that can work with Ethereum transactions.

Specifying more and splitting into actionable items:

NEP 1: Ethereum address implicit accounts

Ethereum addresses of 0x[\w\d]32 are extremely popular way people have accounts on-chain.

This proposal is to add support for such addresses as extension of existing implicit accounts.

Implicit accounts are accounts that have inferable key from the address.

In the case of Ethereum address, the key can be verified (addresss == keccak(pub_key)[:20]) but not explicitly inferred which creates additional challenge on NEAR.

Special rule will be added that if account matches Ethereum address, it can be accessed by the respective key even if it's not explicitly added on-chain into KEYS.

Flow:

• When someone sends $NEAR funds to 0x address that doesn't exist on-chain. On receival, if address matches 0x address => new account is created without any explicit keys.
• User signs a transaction with ECDSA key matching a 0x... address. At processing of receipt, if sender's address is 0x... and the provided key is not in KEYS already, the verification of public key first happens by matching keccak(pub_key)[:20] to accountId and at this time key is added to KEYS with nonce to prevent replaying going forward.

NEP 2 Ethereum transaction format for Meta Transaction

Ethereum transaction format is a very popular format supported by many wallets. It's also a very generic format, that supports a way to introduce any arbitrary data.

The proposal is to add support for Meta Transactions to include signed Ethereum transaction messages as an alternative to DelegateAction (see Meta Transaction details).

When meta transaction is being processed, if the internal format matches Ethereum transaction - a different processing to unpack is applied.

The Ethereum transaction format (encoded in RLP) that will be acceptable:

- nonce: u256 // used as nonce
- gasPrice: u256 // ignored
- gasLimit: u256 // used as gas amount
- to: bytes[20] // ignored
- value: u256 // used as attached NEAR
- data: contains next fields Borsh serialized:
       - sender: string // this allows to support non 0x addresses while still with ECDSA keys
       - receiver: string // this allows to support non 0x addresses
       - actions: Actions[] // list of actions same as standard meta transaction format
- (v, r, s): bytes[65] // signature that is used to infer pub_key

Ethereum JSON RPC Relayer

Relayer provides Ethereum endpoints that wrap NEAR's RPC for NEAR state and smart contracts.

Not all RPC end points are going to be supported, only ones required for work of Ethereum wallets.

Relayer also receives Ethereum formatted transactions and sends them on-chain as meta transactions.

List of supported endpoints:

eth_chainId -- returns NEAR's chain ID
eth_gasPrice -- returns NEAR's gas price
eth_blockNumber -- returns NEAR's block number
eth_getBalance -- returns balance of NEAR for given account
eth_sendTransaction -- sends given transaction as a meta transaction to NEAR

The idea that NEAR fees initially will be paid by relayer, but to make this sustainable, there should be a way to pay back from relaid account. For example user can be paying with stablecoin or in some other way to reduce need to have $NEAR token initially.

To implement that Wallet Selector must attach respective Action to transfer to relayer account a fee amount in desired currency. Relayer would then on eth_sendTransaction verify that such action is included and processes payment.

Initially, this can be omitted, but design should include this to ensure anyone can run a relayer.

Wallet Selector extension

Wallet Selector adding support for Ethereum wallets and provide a way to inject the Relayer API to relay transaction.

Work similar to near/wallet-selector#773 must be implemented to add directly Metamask and other popular wallets or WalletConnect.

Implementing WalletBehaviourFactory to use signTransactions via Metamask and then send received signed Ethereum transaction object via JSON RPC Relayer.

[bowenwang1996]

The Ethereum transaction format that will be acceptable:

Hmm how does the conversion between two formats actually work? Ethereum transactions are serialized in RLP and on chain we need to check that the signature is valid, so some RLP encoding/decoding will need to happen when we process those transactions.

[ilblackdragon]

The Ethereum transaction format that will be acceptable:

Hmm how does the conversion between two formats actually work? Ethereum transactions are serialized in RLP and on chain we need to check that the signature is valid, so some RLP encoding/decoding will need to happen when we process those transactions.

The Ethereum transaction is encoded in RLP and stored into DelegateAction extension. Data field inside Ethereum transaction is encoded with Borsh to fit our specific fields. This field has no prescribed encoding and just bytes from perspective of wallets.

[birchmd]

Thanks for starting this discussion @ilblackdragon . Enabling Metamask to be used directly with Near apps is an appealing way to onboard existing web3 users from EVM-compatible chains.

The Protocol Working Group met today and we discussed this issue. We are thinking about ways we can have the desired user journey while minimizing the protocol changes needed. One idea is to use a contract to handle the RLP encoded transaction instead of making the protocol aware of RLP. The deployment of this contract to an Ethereum address-like Near account could be done automatically by the Relayer, and this could even be free by combining zero balance accounts with the shared contracts feature (the latter is not yet implemented though).

Another issue raised concerned the design of using the data field of a normal Ethereum transaction to store the Near function call arguments (Borsh-encoded). This design could make it difficult for Metamask users to verify the contents of the transaction they are signing, which may create opportunities for phishing attacks. The trouble is that Borsh is not an Ethereum ecosystem standard so wallets will not know how to display the transaction data in a human-readable way. An alternative design is to leverage EIP-712 and provide a schema for Near transactions so that Metamask could display the transaction data clearly; similar to how Near wallets would display the transaction details.

I think that using EIP-712 is technically different from submitting a normal Ethereum transaction, so the user flow might not be exactly as described in the initial post for this issue. But it should be possible to create a reasonably seamless UX by filling in the gaps on the tooling side (which was already a part of the proposal here with the Wallet Selector extension).

What do you think about these ideas @ilblackdragon ?

[ewiner]

I love this proposal, regardless of how we implement it. @birchmd I think EIP-712 has some benefits and some drawbacks vs the Ethereum TX signing flow, given the complications I'll point out below. The main downsides are:

1. The general expectation is that you aren't moving on-chain assets when you're just signing data, though that can indeed already happen in the Ethereum ecosystem (via a previous on-chain "approval" TX and using a relayer or counterparty to broadcast the requested action).
2. Once you sign the TX, there's no followup on whether the TX was valid or mined. Though we could built that into the Wallet Selector JS functionality.

Some points to figure out if we use the TX signing flow instead of the Typed Data flow:

Ethereum JSON-RPC Compatibility

We'll have to audit MetaMask and maybe a few other popular wallets to see what JSON-RPC methods they call during the transaction preview, sign, send, and monitoring steps. I think we'll have to add a few to the Ethereum JSON-RPC shim, such as:

• eth_getTransactionCount: used to determine the next nonce for the pubkey
• eth_estimateGas: more on this below
• eth_feeHistory: I'm not sure if this is in use – it's meant for determining EIP-1559 maxPriorityFeePerGas values
• eth_sendRawTransaction: many wallets, IIRC including Metamask, do their own RLP encoding of the transaction then use this hex API. It's easy enough to decode it then continue with eth_sendTransaction's logic, though.
• eth_getBlockByNumber: we can likely fake or omit most of the returned object, but the gasLimit property is used to ensure that a proposed transaction could fit under the next block's gas limit (i.e. eth_estimateGas < block.gasLimit).

In addition, we'll have to add sufficient RPC calls to ensure the main Metamask UI shows reasonable values and doesn't crash during the course of normal operation. For instance, we'll probably need an implementation of eth_getCode that returns 0x0 for all inputs, so it can't detect any ERC-20 tokens.

Post-broadcast monitoring might be a challenge. I'm not sure if Metamask pings eth_getTransactionByHash in a loop or if it uses the "filter" (block or tx subscription) features of Ethereum JSON-RPC. Either way, we'll have to inform it if/when the transaction has been accepted, similar to how an EVM chain would. We could do that when it's accepted by the relayer, but ideally we wait until the underlying transaction is in a NEAR block.

Gas

Using eth_gasLimit as the input for the NEAR gas limit seems like a problem for two reasons:

1. Ethereum sets gas at the transaction level, while NEAR sets gas per functionCall action. (Please correct me if I'm wrong, I'm still a NEAR noob!)
2. Metamask and other wallets will check if value + (gasLimit * gasPrice) > eth_getBalance and if so, it'll refuse to sign & submit the transaction, assuming it would get stuck in the mempool anyway due to insufficient funds in the source account.

So I think those values should reflect what the fees that the relayer is going to charge the user, which is initially 0. In other words eth_gasPrice should return 0 and eth_estimateGas should return either 0 or whatever is acceptable to Metamask (maybe 21000, the minimum gas for an EVM tx). Then Metamask should calculate that the tx will cost 0 NEAR and allow it to be signed and broadcast.

If that fails some validation within Metamask (b/c 0-fee transactions are unusual), we could fake it out by having a very small eth_gasPrice value and also returning a small, though inaccurate, result from eth_getBalance that's enough to cover that small gas price. That would be pretty hacky though!

[bowenwang1996]

@ewiner Re. gas: while it is true that gas is set on different levels, it is possible to calculate on the transaction level how much gas a NEAR transaction attaches overall. In addition, it is fairly uncommon for a user to have multiple function calls in one transaction since they have to call the same smart contract. As far as I know, that feature today is mostly used by large application like SweatCoin to batch many user transactions together.

The balance check you mentioned is also similar to how a transaction is checked on NEAR, so I think it could work fine if gas limit and gas price are set to the NEAR transaction gas limit and NEAR gas price. I am not sure how value is calculated exactly, so there could potentially be some issue there.

[birchmd]

Thanks for the comment @ewiner ! I agree EIP-712 has pros and cons.

Another option to make it easier for users to decode the data in the transactions they sign would be to make a tool that converts Near contract ABIs into Solidity ABIs and have the front-ends encode the data in the common Ethereum way. Then this Solidity ABI encoded data could be automatically decoded into JSON (or Borsh) data when the Ethereum transaction is converted into a Near contract call (this could be done as part of the same contract that handles the RLP decoding). The downside to this approach would be a requirement on contract developers to publish an ABI for their contracts if they wanted it to work with Ethereum-compatible wallets.

@bowenwang1996 the value field is where the user defines how much base currency to attach to the transaction (equivalent to deposit on Near). This reminds me that there is another complexity in translating Ethereum transactions to Near transactions: the difference in the number of decimals for the base currency. $NEAR has 24 decimals, but $ETH has 18. I think the number of decimals is hard-coded into Metamask, so to display balances properly there would need to be some rounding in the eth_getGetBalance RPC endpoint. That rounding is not a big deal, but for the value field to display properly there would need to be extra zeros padded when the transaction is converted to a Near contract call.

This padding is an issue because it would mean 1M yoctoNEAR is the smallest quantity that can be attached to such calls, but there are standards in the Near ecosystem that require exactly 1 yoctoNEAR attached (e.g. NEP-141 fungible tokens). One work around for this issue would be to assume that 1 WEI maps to 1 yoctoNear and values greater than 1 have the zeros padded, but that feels pretty hacky. I'm open to other suggestions about how to address this issue!

[wacban]

I have a small suggestion to for the "NEP 1: Ethereum address implicit accounts". Let's use the address.eth format.
a) It's more human readable which is in line with near values
b) There is a chance that there aren't as many legacy accounts in that format. If by any chance there aren't any existing accounts in that format, it gets rid of the problem of handling those legacy accounts.

cc @staffik

[ilblackdragon]

The challenge with using address.eth is that when people are going to send funds from exchanges / wallets - they will not know they need to put .eth address and so they will send it to address which is a valid account name on NEAR and will loose funds.

[birchmd]

In the interest of pushing this project forward, I am proposing a more detailed specification below. It is still not 100% complete, but does fill in more details than previous posts on this issue. This spec is based on the suggestions of @ilblackdragon above, but with a few key modifications.

1. The call data of the Ethereum transaction is encoded using the Solidity ABI instead of Borsh. The reason for this change is to make it easier for Ethereum tooling to display the call data in a human-readable way. Users being able to understand the transactions they are making is important to prevent phishing.
2. Instead of building on the meta-transaction spec to submit the transaction, define a new kind of action which can carry RLP-encoded transactions as a payload. The reason for this change is to make the gas payment align more closely with the experience on existing EVM-compatible chains. In the meta-transaction standard it is the relayer account which covers the gas fee, with the expectation that it will receive compensation from one of the actions instead (e.g. via a fungible token transfer). However, it is typical on EVM chains for the user to cover the gas cost themselves using the base token of the chain. The easiest way to replicate this experience is to have a separate action to handle Ethereum transactions instead of always including a $NEAR transfer to the relayer as part of the meta-transaction. Note that the meta-transaction standard allows delegating any protocol action (except the delegation action itself), therefore this new action kind could still be used with meta-transactions if a different gas payment experience is desired for specific applications; this proposal simply makes the standard EVM protocol for gas payment the default.

Specification

Summary of protocol changes

20-byte addresses as implicit accounts on Near

Account Ids matching the regex ^0x[0-9a-f]{40}$ will be considered as implicit accounts on Near. This means $NEAR transfers where the recipient account matches that regex will always succeed (if the source account has enough $NEAR to make the transfer) and the target account will be created if it did not exist previously. No keys will be added to the account at the time it is created.

If there is a Near transaction where the sender is of the form ^0x[0-9a-f]{40}$ and there are no access keys yet added to the account then whether the public key used to sign the transaction is valid for that account or not is checked by (python-like pseudocode)

'0x' + keccak256(pub_key)[12:32].hex() == account_id

If this check passes then pub_key is added to the account automatically as a full access key with nonce equal to the one on the sent transaction, then the transaction is converted to a receipt as normal.

This means that future transactions for that account will be able to be checked following the usual Near procedure of checking the public key is added to the account.

RLP-encoded transactions as actions on Near

A new variant is added to the Action enum for Near transactions. This variant is called RlpTransaction and it is defined by the following struct

struct RlpTransactionAction {
    payload: Vec<u8>,
}

The gas cost of converting a transaction including an RlpTransaction action into a receipt must include the cost of checking the receipt validity (more on what this entails below). This ensures these costs are paid regardless of the validity of the payload (because the signing account will be charged this before the computation is performed). In the case that the payload is valid and the implicit account the payload is signed by has sufficient funds then these costs will be refunded back to the signing account. This is so the relayer does not have to pay anything to send valid transactions to the network, but I'm getting ahead of myself.

This kind of action has the following validity rules:

1. The receiver_id must be an address-based implicit account (account id matches ^0x[0-9a-f]{40}$).
2. The action payload must be a valid RLP-encoded signed Ethereum transaction. This could be an Ethereum legacy transaction, or on of the newer formats (EIP-1559, EIP-2930).
3. The signature part of the Ethereum transaction must be valid. I.e. the public key recovered from the signature must hash to the address corresponding to the account id.
4. The Ethereum transaction must include a chain id and it must be equal to NEAR_CHAIN_ID (TODO: decide on this value and claim it on https://chainlist.org/)
5. The Ethereum transaction must have a non-empty to field.
6. If the public key recovered from the Ethereum transaction signature is not an access key for receiver_id then the nonce value of the Ethereum transaction must be 0, otherwise the Ethereum transaction's nonce must be greater than or equal to the nonce value of that access key.
7. The effective gas price of the Ethereum transaction must be equal to the Near gas price for that block.
8. The data part of the Ethereum transaction must be a 4-byte function selector followed by the Solidity ABI encoded bytes corresponding to that selector. The supported selectors are derived from the following types:

FunctionCall(to: string, method: string, args: bytes, yoctoNear: uint32)
Transfer(to: string, yoctoNear: uint32)
Stake(public_key: bytes, yoctoNear: uint32)
AddKey(public_key: bytes, nonce: uint64, is_full_access: bool, allowance: uint256, receiver_id: string, method_names: string[])
... // TODO other Near actions here

9. The yotoNear value in the data (if present) must be less than 1 million.
10. The to field in the data (if present) must either equal (the hex-encoded) to address of the Ethereum transaction, or the last 20 bytes of the keccak hash of the to filed in the data must equal the to address of the Ethereum transaction.

When a receipt containing valid RlpTransaction actions is executed it produces at least two new receipts. The first is a refund of the amount of $NEAR predecessor_id would have had to spend to create the receipt (this ensures relayers are free to operate). The second is a receipt containing the actions defined by the Ethereum transaction call data in each of the RlpTransaction actions. The fields of these new actions are populated from a combination of the data present in the Ethereum transaction call data and the other fields in the Ethereum transaction. For example, a FunctionCall action will have the method_name, and args come directly from the call data; the balance will be computed as transaction.value * 1_000_000 + transaction.data.yoctoNear (the reason for this is because Ethereum wallets will assume the base token has 18 decimal places instead of 24); and the gas will come from the Ethereum transaction's gas_limit.

To prevent replaying transactions, the nonce of the access key corresponding to the public key which hashes to the address for the account will also be updated to the largest nonce present in any of the Ethereum transactions in the RlpTransaction actions plus 1.

Note: unless a new host function is added to the Wasm runtime, it will not be possible to create RlpTransaction actions from a Near smart contract. Since that functionality is not needed to enable Ethereum wallets being used on Near we do not propose adding such a host function at this time. If there is a use-case for smart contracts creating RlpTransaction actions in the future then this functionality can be added as a separate NEP.

Note: The RlpTransaction action translates into a single action. To create a batch of actions would require a batch of RlpTransaction actions, and therefore multiple signatures from the user. This should not be a problem since typical Near apps do not use batches of actions from an individual user to interact with a smart contract.

Ethereum RPC relayer service

This is separate from the Near protocol, but is needed for Metamask to have something to connect to. This is a service exposing (a minimal subset of) the Ethereum JSON RPC interface at some URL. This URL is included by users in a custom network added to Metamask. The service requires a Near account to operate because it will be submitting transactions to the network on behalf of Metamask users. What follows is a brief description of how to implement the core RPC methods.

• eth_sendRawTransaction: extract the signing address and construct a Near transaction to the corresponding implicit account on Near using the RlpTransaction action (described above) with the Ethereum transaction bytes as the payload. Async broadcast the transaction and return the Near transaction hash as the transaction hash.
• eth_getBalance: return the yoctoNEAR balance of the account on Near divided by 1M (to account for the decimals mismatch between $NEAR and $ETH).
• eth_getTransactionCount: get the access keys for the account on Near, find the one with the public key which hashes to the address and report the nonce corresponding to that access key. If no such key is found return 0.
• eth_estimateGas: always return 2100 (the minimum amount for Ethereum). This should not matter because gas limits should be set by app front-ends, not Metamask.
• eth_call: always return an empty success result. There is not a reliable way to simulate the transactions on Near, so we won't worry about it.
• eth_getTransactionByHash / eth_getTransactionReceipt: since transaction hashes returned by eth_sendRawTransaction are simply the Near transaction hashes, we can assume the transactions passed to these endpoints will also be Near hashes and look up the results by forwarding the request to Near RPC. The relayer will need to properly format the Near response into the expected Ethereum JSON RPC response.

Note: A drawback of using the Near transaction hash as the hash returned by eth_sendRawTransaction is that it will be different from the Ethereum result (Ethereum would hash the RLP-encoded transaction). But I think Metamask will accept the returned hash no matter what.

[bowenwang1996]

the first is a refund of the amount of $NEAR predecessor_id would have had to spend to create the receipt (this ensures relayers are free to operate).

@birchmd I have a couple of concerns regarding this change:

• This means an additional special logic we need to encode into the protocol (vs. just using existing rules about gas and refund)
• It seems that this implies that users will pay for their own transactions, which is fine except that when they first get onboarded, they don't have any NEAR. Ideally a user should be able to select "NEAR" in metamask as the network and then be able to start using apps right away.

[birchmd]

vs. just using existing rules about gas and refund

It's a little extra complexity, but I don't think it's too much. It's a transfer from the user account to the relayer account (as opposed to a system receipt), and the amount should be easy to compute (the only variable is the size of the payload, so it will simply be something like gas_price * (RLP_BASE + payload.len() * RLP_BYTE)).

a user should be able to select "NEAR" in metamask as the network and then be able to start using apps right away

I agree a smooth on-boarding experience is important. This can be handled on the relayer side. For example relayer operators could give a number of free transactions to a new address by including the RlpTransaction action inside a Delegate action (i.e. send it using the existing meta-transaction functionality) at first and then later transition to users paying by relaying without Delegate. I think it is easier to have the default situation be the user paying for gas because it allows this composition with existing meta-transactions to offer flexible gas payment.

[staffik]

Nonce

Currently, a 64-length hex (NEAR-implicit) account is created upon a transfer to that address, and an access key is added with its nonce set to (block_height - 1) * 1e6. This approach aims to prevent scenarios where we might delete an implicit account and then recreate it with an access key nonce reset to 0. Refer to Issue with access key nonce and Set access key nonce on implicit account creation.

For addresses that start with ’0x’ and are followed by a 40-length hex (ETH-implicit), an account will be created when there's a transfer to this address. No access key will be added at this point. When a transaction originates from this account, an access key will be added (if it doesn't exist yet) with its nonce set to match the transaction’s nonce.

Should a transaction be sent from an ETH-implicit account, resulting in the addition of an access key to this account, should near-api-js then set the nonce to (block_height - 1) * 1e6 to safeguard against transaction replays?

To prevent replaying transactions, the nonce of the access key corresponding to the public key which hashes to the address for the account will also be updated to the largest nonce present in any of the Ethereum transactions in the RlpTransaction actions plus 1.

That approach appears to solve the problem for RlpTransaction.

eth_getTransactionCount: get the access keys for the account on Near, find the one with the public key which hashes to the address and report the nonce corresponding to that access key. If no such key is found return 0.

Won't this result in a problem related to replaying transactions for transactions other than RlpTransaction?

[birchmd]

Thanks for bringing this up @staffik . We could make the default value (for when the key is not yet added) equal to (block_height - 1) * 1e6 to be consistent with the treatment of existing implicit accounts on Near.

[walnut-the-cat]

Hello, Protocol NEP moderator here. Do we plan to replace this NEP with #518? If so, shall we close this one? cc. @alexauroradev , @birchmd , @bowenwang1996

[birchmd]

I think it is reasonable to close this issue in favor of #518 . The new issue links to this one for those that want to come back to it for context.