Add Secure Multisig Signing Process #122

engn33r · 2025-03-05T16:03:47Z

This PR suggests recommended process steps for multisig signers to take in order to be informed about and have trust in the transaction they are signing (as opposed to blindly signing). This helps mitigate the risk of the Safe UI becoming a single point of failure (see the recent ByBit incident).

Comments are very welcome since this is a critically important topic.

…ecurity-alliance#117) * Added prebuild.sh to vercel_build and updated Summaries. * Updating prebuild to work in vercel for dynamic summaries. * Updating wordlist as well for summaries * Updating wordlist

* Added prebuild.sh to vercel_build and updated Summaries. * Updating prebuild to work in vercel for dynamic summaries. * Updating wordlist as well for summaries * Updating wordlist * Updating prebuild config.

vercel · 2025-03-05T16:03:55Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
frameworks	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 18, 2025 3:13pm

pcaversaccio · 2025-03-05T16:54:02Z

One thing people need to understand is that offline is not safer.

Let me elaborate:

If the API is compromised and they inject a malicious tx you would see the malicious calldata locally on your device and the hashes would mismatch as well.
let's say you push a malicious tx not into the API but find other ways to distribute to the signers and you instead keep a benign tx in the API for the same nonce. The hashes will not match. You can only work around it if you modify locally the script itself.

If you have a RATed device and you have simple clipboard hijack there, you will be faster "convinced" since they could let you copy the correct data while in my case you need to modify directly the script. The API approach is harder to really break it.

Also, fyi, my tool is actively maintained and will add further features soon, e.g.

So, if you use forks, make sure they merge these features upstream or implement them differently.

sambacha · 2025-03-05T17:51:48Z

This PR suggests recommended process steps for multisig signers to take in order to be informed about and have trust in the transaction they are signing (as opposed to blindly signing). This helps mitigate the risk of the Safe UI becoming a single point of failure (see the recent ByBit incident).

Comments are very welcome since this is a critically important topic.

This does not go far back enough, it is only concerned with the actual ceramony which is not well defined.

Multisig Best Practices

Contract Source Link:

Action Items
> Local Machine conformance enforcement
> Dry run practices
> Hardware safeguards

This appendix summarizes the best practices for the use of a 2-of-3
multi-signature wallet where the authority to execute a transaction requires a consensus of
two individuals in possession of two of the wallet’s three private keys.

The private keys must be stored or held separately, and each must be respectively access-limited to separate individuals.
Multiple keys should not be stored with the same custodian. For example, if the keys are physically held in the custody of a third party (e.g., a bank), then no more than one key can be custodied in that bank. Doing so would violate best practice 1.
The second signatory (a.k.a. the co-signer) must refer to a policy established beforehand specifying the conditions for approving the transaction before signing it with their key.
The co-signer should verify that the half-signed transaction was generated willfully by the intended holder of the first signature’s key.

Policy for Transaction Validation

Best practice 3 prevents the co-signer from becoming merely a “deputy” acting on behalf
of the first (forfeiting the decision responsibility back to the first signer, and defeating the security model). If the co-signer refuses to approve the transaction for any reason, then the due-diligence conditions for approval may be unclear. That is why a policy for validating the transaction is needed. Example verification policy rules may include:

● A predetermined protocol for being asked to cosign (e.g., a half-signed transaction
will be accepted only via an approved channel).

● An allowlist of specific actions that can be performed by the wallet.

● A threshold for the number of transactions that can be executed on a given day,
week, etc.

Best practice 4 mitigates the risk of a single stolen key. In a hypothetical example, an
attacker somehow acquires the unlocked Ledger Nano of one of the signatories.

Duress / Safeword

A voice call from the co-signer to the initiating signatory to confirm the transaction will reveal that the key has been stolen and that the transaction should not be co-signed. If under an active threat of violence, a “duress code” (code word, phrase, or other system agreed upon in
advance) can be used as a covert way for one signatory to alert the other signers of an issue.

For example, a phrase can be used as a challenge when confirming over voice/video.

e.g. "Pineapple Fried Rice".
e.g. "Pork Fried Rice" as a duress phrase.

Warrant Cannary

Usage of Warrant Canary is advised for escliated policy identification for duress levels.

a private key with some native token attached to it, stored in plaintext for the purposes of a honeypot for detection compromise.

Operational Safeguards

Note that each multi-signature wallet used by the team should be treated independently of
the others. Thus, each wallet should have its own policies for transaction validity and
storage.

https://hackmd.io/@manifoldx/multisig-best-practices

Additionally, a defined Incident Response plan in the case of compromise.

● Determine when the team will seek assistance from external parties (e.g., auditors, affected users, other protocol developers, etc.) and how it will onboard them.

○ Effective remediation of certain issues may require collaboration with external parties

Defining Wallet Hygiene schedules, e.g. periodic revoking of all permissions

smartcontracts · 2025-03-05T18:43:38Z

src/operational-security/secure-multisig-signing-process.md

+Before the transaction proposer gives the first signature for the transaction, they should verify the signature using the [Cyfrin safe-tx-hashes tool](https://github.com/Cyfrin/safe-tx-hashes), which has a feature for verifying the first tx that [pcaversaccio's safe-tx-hashes-util tool](https://github.com/pcaversaccio/safe-tx-hashes-util) lacks. The feature generates a signature for transactions that have not been initialized, which is the case for this first transaction. More information exists in [the safe-tx-hashes documentation](https://github.com/Cyfrin/safe-tx-hashes?tab=readme-ov-file#not-initialized-transactions). Additional information about using the Cyfrin tool is found in [this Cyfrin Updraft tutorial video](https://updraft.cyfrin.io/courses/wallets/wallets/verify-multi-sig-signatures?lesson_format=video). An example command for using the tool to verify the first signature is with a command like:
+
+```bash
+./safe_hashes.sh --network base --address 0x86D46EcD553d25da0E3b96A9a1B442ac72fa9e9F --nonce 7 --untrusted


Worth noting that there's a dependency on forge here which makes forge a massive target. It'd be valuable if safe_hashes had an entirely independent implementation of the verification script and fuzzed the two against each other.

audityourcontracts · 2025-03-05T22:27:00Z

Some new ideas in relation to the human process if it helps or you can apply;

Splitting Proposer and Signer responsibilities is an important mitigation.

A number of Proposer keys should be added and solely sourced for proposals. It's preferable these "keys" are multiple hardware wallets but they could also be separate keys on a single hardware wallet. The keys can be held by a number of individuals or a single one. It's preferable the nature of this setup is held closely within the organisation. For the paranoid the proposal key could be an MPC key linked to it's own quorum and policy.

Splitting proposers and signers has a number of benefits;

Off-chain monitoring and alerting can detect whether a proposal is from a proposer or signer key. Proposals from signers should alert.
The adversary now needs to compromise a proposer along with the quorum of signers. Proposers can round robin or randomise the use of proposal key and this is a worthy deception. Treasury ops knows the order but the adversary does not.
I realise proposers are off-chain but as stated earlier significant treasuries should have onboarded Safe API data into a security operations platform and have detections and playbooks that monitor/detect.
Proposers allow you to create "organisational ambiguity". Signers in large organisations are reasonably easy to predict but proposers can be hidden in an org reasonably easy.
The proposal can be though of round 1, all checks and policies, calldata inspection must be green before round 2 human broadcasting to signers. This needs to go beyond function selector matching as you suggest.
It allows the proposer to be technical and organisationally close to security experts while the signers can be less technical. Signers still need to inspect transactions in case the proposer is compromised (see MPC key quorum as a mitigation).

Safe Wallet Hacks need to be deep-dived and detections and mitigating controls adjusted

ByBit and WazirX were different on the Web2 side but near identical on the Web3 side. As a quick summary;

From ZachXBT you learnt of a number of past attacks involving spear-phishing and browser extension side loading (man-in-the-browser). From these past attacks you knew the device, browser and browser extensions were vulnerable and that Javascript manipulation is likely in the endpoint or website supply chains.
From WazirX you learnt about the delegatecall (operation=1) I documented that it was a novel attack. Furthermore WazirX showed the lengths the adversary would go to paying an insider, trading on GALA to ensure the cold wallet needed to transfer funds and finding the vulnerability/bypass in Liminal. WazirX also had a predictable set of signers. The adversary had a very good idea of what keys they needed and likely who had them. Increasing entropy across proposers and signers elevates the cost to the adversary.
Radiant. INLETDRIFT and Browser extensions return they quickly removed traces of their second-stage backdoor and related browser extensions (source).
ByBit Javascript and supply chains. Blind signing and no change to hardware keys after WazirX. Work laptops. Low signing quorum etc.

Deep diving each of these attacks meant you discontinued the use of blind signing devices, implemented an enterprise browser with specific controls, knew that you needed to independently generate and confirm Safe transactions and have a separate machine/iOS device to sign transactions.

engn33r · 2025-03-06T11:45:18Z

I'll try to respond to all the comments briefly.

@pcaversaccio maybe you're right, I haven't thought as deeply about the different attack path for relying on the Safe API vs. offline mode. If you want to discuss on X instead of here, I added some comments today (but we can continue here if you prefer). No doubt what is written will change once the Safe API is back (perhaps in a modified form, only time can tell).

@sambacha Thanks for sharing your best practices doc. I limited this PR to just the signing process. I have another list (still draft) of multisig best practices, but that will be a separate PR. tbh I haven't seen anyone using an allowlist of actions for their multisig, the closest I've seen is a description in plain English describing what each tx does, because I imagine updating the allowlist would be another separate process that must be done securely.

@audityourcontracts I think these ideas would also mostly be a better fit into multisig best practices rather than adding to this secure signing process. I'll tag you when I make the other PR for best practices.

I'm seeing more about warnings about using Safe operation 1 AKA delegatecalls (like the PR from @pcaversaccio) so I expect to add more details here soon to clarify what to do for deeper inspection of the delegatecall case (because using delegatecall to batch safe transactions is currently quite common).

pcaversaccio · 2025-03-06T13:03:18Z

@pcaversaccio maybe you're right, I haven't thought as deeply about the different attack path for relying on the Safe API vs. offline mode. If you want to discuss on X instead of here, I added some comments today (but we can continue here if you prefer). No doubt what is written will change once the Safe API is back (perhaps in a modified form, only time can tell).

Cross-posting my reply: Let me rephrase: offline mode is helpful but doesn't mean it's safer by design. So what's the ultimate goal we want to achieve here? We want to detect if the transaction data sent to the Ledger itself is the one you intended to sign based on the information you see on the screen. One important part is the calldata as you say: let's say Safe is fully compromised, i.e. UI & API. You copy (via their copy button) the calldata from the Safe UI and want to verify it. In that step the copied data could switch as the UI is compromised remember, so you verify not the correct calldata. So you can be tricked into believing you verified the correct calldata. Since the Safe API is also compromised, you pull the data locally (let's assume the device is not compromised here). The data you will get is the malicious calldata BUT the benign decoded data to try tricking people into believing you see the correct decoded data. The calldata enters the local calculation of the hashes, so in order to match the message hash you see on the Ledger it must be the malicious one. But you need individually verify the decoded calldata always, and it doesn't matter via offline mode or Safe API mode. Since you got the calldata pulled via the Safe API (which must be malicious in this scenario), you can try to locally decode it yourself. If you use the version without the Safe API, in most of the times the people will use the UI to get the data from, which can be the wrong one. What I try to say is, offline is not bad at all, but how you get to the parameters you need for the offline mode can be risky.

I'm seeing more about warnings about using Safe operation 1 AKA delegatecalls (like pcaversaccio/safe-tx-hashes-util#14 from @pcaversaccio) so I expect to add more details here soon to clarify what to do for deeper inspection of the delegatecall case (because using delegatecall to batch safe transactions is currently quite common).

It's important to say untrusted delegatecalls. You can find a list of all trusted delegatecallable contracts in my PR: pcaversaccio/safe-tx-hashes-util#14.

bsamuels453 · 2025-03-07T01:12:19Z

I think it's easy to spend an inordinate amount of time theorycrafting the correct guidance for multisigs, especially if the end goal is to provide concrete, step-by-step guidance for a specific multi-sig implementation.

I recommend trying to refactor this guidance to be as high level as possible. Talk about the different kinds of security controls that users need to think of, threat modeling, procedural considerations, monitoring. Ultimately there are going to be drastically different solutions in place to secure a $1B multisig vs. a $1m one, and each of those have different threat models and security budgets.

If you really want to provide concrete guidance, make it clear that the guidance is specific for a certain user profile and that based on the amount stored, the system's requirements and controls should vary. There's still a lot of benefit in providing concrete guidance, but we also have to be honest about the limitations of one-size-fits-all guidance so high-tvl users know they will probably have to do more than the bare minimum.

GianfrancoBazzani · 2025-03-07T09:19:05Z

src/operational-security/secure-multisig-signing-process.md

+
+## Step 3: Transaction proposer prepares the first signature
+
+Before the transaction proposer gives the first signature for the transaction, they should verify the signature using the [Cyfrin safe-tx-hashes tool](https://github.com/Cyfrin/safe-tx-hashes), which has a feature for verifying the first tx that [pcaversaccio's safe-tx-hashes-util tool](https://github.com/pcaversaccio/safe-tx-hashes-util) lacks. The feature generates a signature for transactions that have not been initialized, which is the case for this first transaction. More information exists in [the safe-tx-hashes documentation](https://github.com/Cyfrin/safe-tx-hashes?tab=readme-ov-file#not-initialized-transactions). Additional information about using the Cyfrin tool is found in [this Cyfrin Updraft tutorial video](https://updraft.cyfrin.io/courses/wallets/wallets/verify-multi-sig-signatures?lesson_format=video). An example command for using the tool to verify the first signature is with a command like:


OpenZeppelin has also an ui for Safe transaction hash calculation for verification that could be worth to mention. It can be used through OZ hosted version https://safeutils.openzeppelin.com/ or running it locally https://github.com/openzeppelin/safe-utils

the safe ui is using this instead of simulations via tenderly

GianfrancoBazzani · 2025-03-07T12:04:53Z

src/operational-security/secure-multisig-signing-process.md

+
+## Step 1: Connecting a hardware wallet
+
+Hardware wallets offer better security than most software wallets, so it's recommended that all multisig signer addresses are from hardware wallets. Multisig signing should ideally involve a direct connection of your hardware wallet with your browser. Avoid Metamask, Rabby, Frame, or other browser extension wallets as a middle man, as this increases the attack surface. However, this feature appears to have been removed recently after the ByBit hack so that a browser extension is required in this process.


I don't think that using the HW wallet through an injected wallet is inherently bad per se, it adds an extra layer of verification and setup diversity. However, its verification should be considered a confirmation of a valid transaction but NEVER a dismissal of a potentially malicious transaction

sambacha · 2025-03-07T19:01:26Z

https://crypto-wallet-designed.github.io/crypto-key-calculator

this computes the success rate of a wallet based on fault probabilities.

engn33r · 2025-03-17T13:23:19Z

I overhauled this draft and added a new markdown file outlining multisig best practices that aren't directly related to the multisig signing process. So now any suggestions or thoughts related to multisig security can be added to this PR, it just needs to be put into the proper place.

@pcaversaccio I've switched the docs over to using your tool since I expect it will be better maintained going forward and I like the recent feature upgrades. I gave readers 2 options for verifying the first signature (which the transaction proposer should do). If you think it's better to give only 1 option, let me know if you think one is better to suggest.

@audityourcontracts the proposer and signer focus on different steps in the signing process as documented (proposers do step 3, later signers do step 5). Do you think further info about this split should be written somewhere?

@GianfrancoBazzani acknowledged, I removed the suggestion to use only a hardware wallet. I also added a link to the OZ Safe Utils website, but with a warning about the risks of trusting a 3rd party UI.

@sambacha your link to https://crypto-wallet-designed.github.io/ is broken, is there a new link?

@bsamuels453 If we assume the user profile is an average DeFi project, how would you alter the current docs for such users?

mattaereal · 2025-03-17T17:21:28Z

I've been lurking, waiting for me to find a moment to intervene and help you with this whenever it is ready to be merged. Pay attention that I have been and will be pushing new updates and restructuring many things.

For example, some modifications you suggested are to no longer existing pages or heavily modified ones (been updating many things these past days). This week, I'll start with Operational Security, so I suggest you continue as you are doing right now, focusing on a single page, the one we'll be able to add as a subsection to the Opsec framework.

If you want to modify non-content-related code, like prebuild.sh scripts, create a separate PR to address that so I can keep track of it on a separate branch.

And of course, thank you very much for taking an interest on frameworks, particularly in something like this.

@fredriksvantes have you seen this?

fredriksvantes · 2025-03-18T08:19:54Z

I have created an initial version of what a multisig SOP could look like. feel free to have a look at it and see if any of it could be of value to these documents: https://notes.ethereum.org/@fredrik/multisig-sop

engn33r · 2025-03-18T15:17:25Z

Thanks @fredriksvantes I pushed some ideas borrowed from the link you shared

@mattaereal welcome to the thread, and thanks for leading this initiative 👍 This is my first PR here, feel free to lmk if I'm straying from the process

audityourcontracts · 2025-03-19T07:28:24Z

@engn33r I think this is fine. I was specifically talking about separating proposers from signers using https://safe.mirror.xyz/GHmhFYhPS8gO3kManlQ8JrVUumr8gqfNkfhZifi5pUk I think we can publish a separate guidance for a more advanced / secure setup in the future.

engn33r · 2025-03-24T15:42:41Z

@mattaereal I don't have much more to add on this topic at the moment and I see less active discussion now. One option is to leave this open for other contributions or the other option would be to merge it as it is if you think it's ready.

mattaereal · 2025-05-08T19:21:04Z

I was OOO and will be looking at this these next weeks. Thanks for all your work :)

Btw howtomultisig.com

GianfrancoBazzani · 2025-05-09T04:52:44Z

Dropping this here for context:

https://medium.com/@bazzanigianfranco/fundamentals-for-high-value-safe-smart-accounts-313fa6e16632

https://medium.com/@bazzanigianfranco/safe-secops-guidelines-for-high-value-safe-smart-accounts-a5641f6ee725

mattaereal · 2025-05-19T18:44:45Z

@engn33r can you merge develop into this, resolve any conflicts, if any, and then push changes? So I can actually compare and try to understand how to complement this with other pending upgrades I have on queue?

mattaereal and others added 3 commits February 27, 2025 11:10

Updating prebuild.sh to work with vercel's config and fixing grammar (s…

d620581

…ecurity-alliance#117) * Added prebuild.sh to vercel_build and updated Summaries. * Updating prebuild to work in vercel for dynamic summaries. * Updating wordlist as well for summaries * Updating wordlist

Merge prebuild-summary into develop (security-alliance#118)

245eaf8

* Added prebuild.sh to vercel_build and updated Summaries. * Updating prebuild to work in vercel for dynamic summaries. * Updating wordlist as well for summaries * Updating wordlist * Updating prebuild config.

Add secure multisig signing process

c863b33

vercel bot deployed to Preview March 5, 2025 16:05 View deployment

smartcontracts reviewed Mar 5, 2025

View reviewed changes

GianfrancoBazzani reviewed Mar 7, 2025

View reviewed changes

mattaereal force-pushed the develop branch 2 times, most recently from 40978af to 443a2e9 Compare March 13, 2025 19:47

Add multisig best practices page, update msig signing process

b0f90bc

vercel bot deployed to Preview March 17, 2025 13:21 View deployment

Add additional best practices from EF doc

2760132

vercel bot deployed to Preview March 18, 2025 15:13 View deployment


		## Step 3: Transaction proposer prepares the first signature

		Before the transaction proposer gives the first signature for the transaction, they should verify the signature using the [Cyfrin safe-tx-hashes tool](https://github.com/Cyfrin/safe-tx-hashes), which has a feature for verifying the first tx that [pcaversaccio's safe-tx-hashes-util tool](https://github.com/pcaversaccio/safe-tx-hashes-util) lacks. The feature generates a signature for transactions that have not been initialized, which is the case for this first transaction. More information exists in [the safe-tx-hashes documentation](https://github.com/Cyfrin/safe-tx-hashes?tab=readme-ov-file#not-initialized-transactions). Additional information about using the Cyfrin tool is found in [this Cyfrin Updraft tutorial video](https://updraft.cyfrin.io/courses/wallets/wallets/verify-multi-sig-signatures?lesson_format=video). An example command for using the tool to verify the first signature is with a command like:


		## Step 1: Connecting a hardware wallet

		Hardware wallets offer better security than most software wallets, so it's recommended that all multisig signer addresses are from hardware wallets. Multisig signing should ideally involve a direct connection of your hardware wallet with your browser. Avoid Metamask, Rabby, Frame, or other browser extension wallets as a middle man, as this increases the attack surface. However, this feature appears to have been removed recently after the ByBit hack so that a browser extension is required in this process.

Add Secure Multisig Signing Process #122

Are you sure you want to change the base?

Add Secure Multisig Signing Process #122

Uh oh!

Conversation

engn33r commented Mar 5, 2025

Uh oh!

vercel bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pcaversaccio commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sambacha commented Mar 5, 2025

Multisig Best Practices

Policy for Transaction Validation

Duress / Safeword

Warrant Cannary

Operational Safeguards

Uh oh!

smartcontracts Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

audityourcontracts commented Mar 5, 2025

Splitting Proposer and Signer responsibilities is an important mitigation.

Safe Wallet Hacks need to be deep-dived and detections and mitigating controls adjusted

Uh oh!

engn33r commented Mar 6, 2025

Uh oh!

pcaversaccio commented Mar 6, 2025

Uh oh!

bsamuels453 commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GianfrancoBazzani Mar 7, 2025

Choose a reason for hiding this comment

Uh oh!

sambacha Mar 7, 2025

Choose a reason for hiding this comment

Uh oh!

sambacha Mar 7, 2025

Choose a reason for hiding this comment

Uh oh!

GianfrancoBazzani Mar 7, 2025

Choose a reason for hiding this comment

Uh oh!

sambacha commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

engn33r commented Mar 17, 2025

Uh oh!

mattaereal commented Mar 17, 2025

Uh oh!

fredriksvantes commented Mar 18, 2025

Uh oh!

engn33r commented Mar 18, 2025

Uh oh!

audityourcontracts commented Mar 19, 2025

Uh oh!

engn33r commented Mar 24, 2025

Uh oh!

mattaereal commented May 8, 2025

Uh oh!

GianfrancoBazzani commented May 9, 2025

Uh oh!

mattaereal commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vercel bot commented Mar 5, 2025 •

edited

Loading

pcaversaccio commented Mar 5, 2025 •

edited

Loading

bsamuels453 commented Mar 7, 2025 •

edited

Loading

sambacha commented Mar 7, 2025 •

edited

Loading

mattaereal commented May 19, 2025 •

edited

Loading