Prefetch JUMPDESTs through RPC #427

einar-polygon · 2024-07-23T13:59:50Z

resolves #290

evm_arithmetization/Cargo.toml

zero_bin/rpc/Cargo.toml

zero_bin/rpc/src/native/txn.rs

zero_bin/rpc/src/main.rs

zero_bin/rpc/src/native/mod.rs

Nashtare

Thanks Einar! Aside from the inline comments, a miss in this PR is the application of JD pre-fetching for jerigon as well, not only the native tracer.

I haven't tested it yet as I write this, but I think your current branch would crash on any jerigon payload (from the ZeroTracer) because the RPC will provide a Vec<ZeroTxResult> which contains TxnInfo, i.e. with the jumpdest table field already processed, as opposed to the TxnMeta from which you're starting the processing.

I think we're also missing fallback mechanisms, wherever the procedure fails for some reason, then we should default to the current behavior, i.e. no provided JDT and let the prover do the work.

evm_arithmetization/src/cpu/kernel/interpreter.rs

evm_arithmetization/src/cpu/kernel/tests/add11.rs

Cargo.toml

evm_arithmetization/src/generation/prover_input.rs

zero_bin/rpc/src/native/txn.rs

Nashtare

You also need to address the format change on all hardcoded witnesses, either by re-generating them, or adding default empty JDTs to each of them, and leaving the prover handle them internally as before.

einar-polygon · 2024-08-26T10:13:15Z

You also need to address the format change on all hardcoded witnesses, either by re-generating them, or adding default empty JDTs to each of them, and leaving the prover handle them internally as before.

Indeed. Which solution do you prefer and who could I talk to about this CI setup?

Nashtare · 2024-08-26T12:12:56Z

You also need to address the format change on all hardcoded witnesses, either by re-generating them, or adding default empty JDTs to each of them, and leaving the prover handle them internally as before.

Indeed. Which solution do you prefer and who could I talk to about this CI setup?

I'm not sure I understand the relation between the broken tests and the CI. We don't have to update them all (especially because some dev blocks are from an outdated devchain IIRC), but at least have them have proper format so that they can run correctly. If your question was how to test this logic of JDT pre-fetching in the CI, we could do this with the Jerigon Integration related jobs, we're testing both native and jerigon tracers there.

Nashtare · 2024-08-26T14:28:08Z

@einar-polygon Has it been tested against some endpoint? Should I run it against a few dozen blocks to see how it behaves? (ideally after conflicts are addressed)

einar-polygon · 2024-08-26T15:00:47Z

@einar-polygon Has it been tested against some endpoint? Should I run it against a few dozen blocks to see how it behaves? (ideally after conflicts are addressed)

I checked that the JUMPDEST_tables were identical for the small Calcun block and a few others against the paid Quicknode endpoint. But it would be good to have a set of interesting blocks with exceptions, reverts and what not.

Here is a start:

txn type 0-3
20548415
20240058
19665756

I will add a bash script to iterate over them while we test.

Nashtare · 2024-08-26T15:22:34Z

I'll launch a run against the devchain once it's rebased (it has a conflicting Erigon version that requires some latest PRs here)

Nashtare · 2024-08-29T13:19:01Z

I got a panic with your endpoint on block 0x13adb68
at line let lower_bytes = U160::from(callee_raw); in zk_evm/zero_bin/rpc/src/native/txn.rs:

Uint conversion error: Value is too large for Uint<160>

Nashtare · 2024-08-29T14:08:15Z

So some more data with your endpoint. I tried 140 consecutive blocks independently.

140/140 failed
36 valid witnesses, failing when batching transactions (tested with -b 2)
32 failures due to unreachable code, all from failed transactions -> cf discussion on slack
72 failures due to Uint conversion error: Value is too large for Uint<160> as shared above

trace_decoder/src/decoding.rs

evm_arithmetization/src/generation/jumpdest.rs

evm_arithmetization/src/generation/prover_input.rs

zero_bin/rpc/src/jumpdest.rs

zero_bin/rpc/src/jerigon.rs

zero_bin/rpc/src/main.rs

zero_bin/rpc/src/jumpdest.rs

zero/src/prover/cli.rs

zero/src/rpc/jerigon.rs

atanmarko · 2024-09-23T10:07:11Z

zero/src/rpc/jerigon.rs

            code_db: Default::default(),
        },
        other_data,
    })
 }
+
+/// Processes the transactions in the given block and updates the code db.
+pub async fn process_transactions<ProviderT, TransportT>(


I don't think having two very similar public process_transactions (one in native one in jerigon) doing same work and returning different result is a good idea. Could we unify the logic for both ?

If we end up moving the fetching logic to Jerigon, this would become moot.

The idea was to reuse the same overall structure from Native but fetch exactly the data needed for Jerigon instead. The alternative, if I understand you correctly, would be that each function includes a dispatching mechanism based on a tag and then return a tagged value. I don't have a preference to either, but I would like some more inputs on this before rewriting it, @Nashtare @muursh.

I am thinking if we could refactor and unify the logic somehow in the common module both jerigon and native would use to retrieve the transaction data. At least, you should rename this function not to introduce confusion with the native tracer process_transactions.

atanmarko · 2024-09-23T10:11:52Z

zero/src/rpc/jerigon.rs

+
+/// Processes the transaction with the given transaction hash and updates the
+/// accounts state.
+pub async fn process_transaction<ProviderT, TransportT>(


Same question about duplicated code. Could we unify the logic (e.g. if tx_trace is passed use if, if not go fetch it).

Valid point, let's keep the discussion in comment above.

Nashtare · 2024-09-23T13:03:22Z

.gitignore

@@ -23,3 +23,4 @@
 #################
 /proofs
 /**/*.zkproof
+/witnesses


what's this?

Nashtare · 2024-09-23T13:08:36Z

evm_arithmetization/src/cpu/kernel/interpreter.rs

+            trace!(
+                "code: {:?}, code_addr: {:?} <============",
+                &code,
+                &code_addr
+            );
+            trace!("code_map: {:?}", &code_map);


Do we want to keep this?

I want to keep all tracing until we are happy with the computed jumpdest tables. It doesn't takes longer to re-add it than to remove it.

Nashtare · 2024-09-23T13:10:27Z

evm_arithmetization/src/generation/jumpdest.rs

+#[derive(PartialEq, Eq, Debug, Clone, Serialize, Deserialize, Default)]
+pub struct ContextJumpDests(pub HashMap<usize, BTreeSet<usize>>);
+
+/// The result after proving a `JumpDestTableWitness`.
+#[derive(PartialEq, Eq, Debug, Clone, Serialize, Deserialize, Default)]
+pub(crate) struct JumpDestTableProcessed(HashMap<usize, Vec<usize>>);
+
+/// Map `CodeAddress -> (Context -> [JumpDests])`
+#[derive(PartialEq, Eq, Debug, Clone, Serialize, Deserialize, Default)]
+pub struct JumpDestTableWitness(HashMap<H256, ContextJumpDests>);


I had already asked but I think my comment got resolved, why do we need actual wrappers here instead of type aliases? Most of the implementation below would come for free instead of having to reimplement everything. Or if it's more convenient for merging tables we could just expose the inner map like what ContextJumpDests is doing?

here was my reponse:

When I was diving into this, it made it more clear what the purpose of both types was. But otherwise it provides a bit of encapsulation and type safety.

I have a mild preference for this New Type pattern over type aliases, but it is not the hill I am going to die on.

Ok, let's keep it.
On another note, why do we need to expose the inner map in ContextJumpDests? Also it is public but only used in this crate I think?

Nashtare · 2024-09-23T13:12:50Z

evm_arithmetization/src/generation/prover_input.rs

+        info!("Generating JUMPDEST tables");
+        // dbg!(&self.inputs.jumpdest_table);
+        // w for witness
+        // let txn_idx = self.next_txn_index - 1;
+        // let rpcw = self.inputs.jumpdest_tables[txn_idx].as_ref();contract_code


nit: to remove

Nashtare · 2024-09-23T13:14:58Z

evm_arithmetization/src/generation/prover_input.rs

+        let rpc: Option<JumpDestTableProcessed> = rpcw
+            .as_ref()
+            .map(|jdt| set_jumpdest_analysis_inputs_rpc(jdt, &self.inputs.contract_code));
+
+        let sims = simulate_cpu_and_get_user_jumps("terminate_common", self);
+
+        let (sim, simw): (Option<JumpDestTableProcessed>, Option<JumpDestTableWitness>) =
+            sims.map_or_else(|| (None, None), |(sim, simw)| (Some(sim), Some(simw)));
+
+        if let (Some(rw), Some(sw)) = (rpcw, simw)
+            && rw != &sw
+        {
+            info!("SIMW {}", sw);
+            info!("RPCW {}", rw);
+            assert_eq!(rw, &sw);
+        }
+
+        info!("SIM {:#?}", sim);
+        info!("RPC {:#?}", rpc);
+
+        self.jumpdest_table = if rpc.is_some() { rpc } else { sim };


would need to change, as there is no simulation if the table exists

This will indeed be removed. I just need to keep it around until we are happy with the results, so I can use the simulation output as ground truth.

Nashtare · 2024-09-23T13:27:53Z

zero/src/rpc/jumpdest.rs

+                ensure!(entry.stack.as_ref().is_some(), "No evm stack found.");
+                // We reverse the stack, so the order matches our assembly code.
+                let evm_stack: Vec<_> = entry.stack.as_ref().unwrap().iter().rev().collect();
+                let operands_used = 2; // actually 6 or 7.


I don't understand this line

Elaborated.

Nashtare · 2024-09-23T13:28:07Z

zero/src/rpc/jumpdest.rs

+
+                if evm_stack.len() < operands_used {
+                    trace!( "Opcode {op} expected {operands_used} operands at the EVM stack, but only {} were found.", evm_stack.len());
+                    // maybe increment ctx here


is it an outdated comment?

asked on slack.

zero/src/rpc/jumpdest.rs

Nashtare · 2024-09-23T13:43:43Z

zero/src/rpc/jumpdest.rs

+                // We reverse the stack, so the order matches our assembly code.
+                let evm_stack: Vec<_> = entry.stack.as_ref().unwrap().iter().rev().collect();


This is called 4 times, I think we could do it just once ahead for clarity before the match statement

It is only called once per StructLog entry. The reason I don't just do it once, and for all ops, is that for most ops, we do not need to do anything, so reversing the stack would be wasteful.

zero/src/rpc/jumpdest.rs

Nashtare · 2024-09-23T13:59:27Z

zero/src/rpc/jumpdest.rs

+/// Provides a way to check in constant time if an address points to a
+/// precompile.
+fn precompiles() -> &'static HashSet<Address> {
+    static PRECOMPILES: OnceLock<HashSet<Address>> = OnceLock::new();
+    PRECOMPILES.get_or_init(|| {
+        HashSet::<Address>::from_iter((1..=0xa).map(|x| Address::from(U160::from(x))))
+    })
+}


This is network specific, i.e. see trace_decoder/src/core.rs related code

rg "precompile" -i trace_decoder trace_decoder/src/core.rs 499: const PRECOMPILE_ADDRESSES: Range<alloy::primitives::Address> = 503: if receipt.status || !PRECOMPILE_ADDRESSES.contains(&addr.compat()) {

I don't see anything network specific here, but we could at least share the code.

Done sharing.

atanmarko · 2024-09-23T14:13:00Z

@Nashtare @einar-polygon Just to summarize our previous discussion (#427 (comment)), we will remove jerigon changes and keep JUMPDEST generation only with native tracer?

Nashtare · 2024-09-23T14:13:13Z

@einar-polygon I briefly looked at some Eth mainnet blocks, it seems there are a few contract deployments that return a valid, but empty JDT, which would then cause the prover to fail as missing proofs.
I'm running the test chain now with the actual witness payloads generated to see if they are consistent.

Nashtare · 2024-09-23T14:14:02Z

@Nashtare @einar-polygon Just to summarize our previous discussion (#427 (comment)), we will remove jerigon changes and keep JUMPDEST generation only with native tracer?

Moving this to slack

einar-polygon · 2024-09-23T14:30:26Z

it seems there are a few contract deployments that return a valid, but empty JDT, which would then cause the prover to fail as missing proofs.

Please share block numbers.

Nashtare · 2024-09-23T14:47:36Z

it seems there are a few contract deployments that return a valid, but empty JDT, which would then cause the prover to fail as missing proofs.

Please share block numbers.

Tried from 20727640 to 20727650. It's visible with block 20727641 for instance:

            "byte_code": "0x02f91c52018202228402faf080850188927c70831f9e388080b91bf660808060405234601557611bdb908161001b8239f35b600080fdfe6080604052600436101561001257600080fd5b60003560e01c63d0c86e3e1461002757600080fd5b60c0366003190112610724576001600160401b03600435116107245760c0600435360360031901126107245760ff6044351660443503610724576001600160401b0360a4351161072457604060a435360360031901126107245761009036606460043501611220565b6040818101513360009081527ff99f5a59dae478b454da96b28b027339f1f555cd7d66f9ae4c949b5abb157d3f6020529190912054811115611158575060208101518042116111385750516001600160a01b03163303611120576101406101486100fe600480350180611265565b610115602460049493943501600435600401611265565b949061013861012e604460043501600435600401611297565b96909436916112cc565b9536916112cc565b923691611347565b610152828461168d565b90156110fe5781519384156110ec5761016a906114b9565b9081548110156110d65760029160005260036020600020910201015490610190836114fe565b549360009460005b8181106110a557508582106110945791906000926000925b828410610ea35750505050818103610e6e57836040516101cf81611178565b600435600401356001600160401b038111610724576101f5906004369181350101611329565b8152602460043501356001600160401b0381116107245761021d906004369181350101611329565b6020820152604460043501356001600160401b0381116107245760043501366023820112156107245761025a903690602460048201359101611347565b604082015261026e36606460043501611220565b806060830152604080519160018060a01b03815116602084015260208101518284015201516060820152606081526102a581611178565b6040820151906040519182604081019160208083015280518093526060820192602060608260051b8501019201936000905b828210610e0f575050506102f69250601f1982820301825203836111ff565b60208351818151910120930151602081519101209160208151910120906020815191012091604051937f9ced5b44a2b3af08ce74be3ad81d6ad5addaf8756d8e783d649054a5c4a8ed04602086015260408501526060840152608083015260a082015260a08152610366816111ae565b7446756e6472616973696e673a4d696c6573746f6e6560581b602060405161038d816111c9565b601581520152640323a333a360dc1b60206040516103aa816111c9565b6005815201526020815191012060405160208101907f8b73c3c69bb8fe3d512ecc4cf759cc79239f7b179b0ffacaa9a75d522b39400f82527f8b7bf121a4925c079594e9c05e9a8a973e9640c48e4905971b36414abc2cb56760408201527f979ef4274de621f4ded3b8de128f7dfd54c9f43ae0d540a16b524de5175e362060608201524660808201523060a082015260a08152610447816111ae565b5190209060405190602082019261190160f01b8452602283015260428201526042815261047381611178565b5190206024358103610def575061049f6104976084356064356044356024356119fc565b919091611a8b565b6001600160a01b031660008181527ff99dac35d4c04cf62531c589d43ecfd87ca6ae92fd3fae21943930c2156dcee8602052604090205460ff1615610dd7575061051a6104f6602460043501600435600401611265565b610140610510604460049594953501600435600401611297565b94909236916112cc565b90815192610527826114fe565b54908181810311610dc157808203610c65575b5050610545906114fe565b54808311610bac575b50503360009081527ff99f5a59dae478b454da96b28b027339f1f555cd7d66f9ae4c949b5abb157d3f6020526040902060043560a4013591505560016105ba60206105a76105a0600480350180611265565b36916112cc565b816040519382858094519384920161145d565b7f9d245c8905a41e40b5d7d8d225eabcef70dc5100963a988bf7a9d7b6a0cd4a74818301528101030190200154468103610729575b50610642610601600480350180611265565b90610650610619602460043501600435600401611265565b61062d604460043501600435600401611297565b9690946040519660608852606088019161140b565b91858303602087015261140b565b928284036040840152808452602084019160208260051b8601019481936000915b8483106106a0577f65cebc1b121235e152b12f6f8be0a2c605c91bcbd76e6abab932c553ffc3837987890388a1005b909192939496601f19828203018352873590607e198636030182121561072457602080918760019401906060806107046106eb6106dd868061142c565b60808752608087019161140b565b6106f78787018761142c565b908683038988015261140b565b936040810135604085015201359101529901930193019194939290610671565b600080fd5b60a43560040135600381101561072457600103610b9a57610754602460a4350160a435600401611265565b60006040809493945161076681611193565b60608152606060208201520152820191602081840312610724578035906001600160401b03821161072457019160608382031261072457604051926107aa84611193565b80356001600160401b03811161072457826107c6918301611329565b845260208101356001600160401b038111610724576040926107e9918301611329565b60208501520135604083015281515115610b8857806000527ff0eb99ac3554f9674e53e01a2c3f3f46c41664575cc6216a02c3bba3b58e2a9e60205263ffffffff6040600020541615610b7057815180516020909101516001600160e01b03198116919060048210610b50575b5050816000527ff6d13f7412e547870332b41e12245afcb7d558e215b48ce696ab1c680af6a77460205260406000209063ffffffff60e01b16908160005260205260ff6040600020541615610b325750604082015115610b20576040820151803410610b0257506000527ff0eb99ac3554f9674e53e01a2c3f3f46c41664575cc6216a02c3bba3b58e2a9e60205263ffffffff6040600020541660018060a01b037fe76a1a8cc2a93ef95736d523edd260dabc1e17bd1543fba2a8f0c977d6dc855954167ff6d13f7412e547870332b41e12245afcb7d558e215b48ce696ab1c680af6a77360205260018060a01b03604060002054169060018060a01b037ff0eb99ac3554f9674e53e01a2c3f3f46c41664575cc6216a02c3bba3b58e2a9d541692845194602081015190604051908160a08101106001600160401b0360a084011117610ad95760809663ffffffff9460a084016040528352602083019687526040830198895260608301938452610a2460408985019360008552015194610a116040519b8c9a8b998a986302637a4560e41b8a52604060048b0152511660448901525160648801525160a0608488015260e4870190611480565b90518582036043190160a4870152611480565b9151151560c4840152602483015203925af18015610af657610a48575b50806105ef565b8060803d608011610aef575b610a5e81836111ff565b81010390608082126107245760405191610a7783611193565b815183526020820151906001600160401b0382168203610724576040916020850152603f1901126107245760405191604083018381106001600160401b03821117610ad957604092606091845283810151855201516020840152015280610a41565b634e487b7160e01b600052604160045260246000fd5b503d610a54565b6040513d6000823e3d90fd5b60449060405190637166d3ed60e01b82523460048301526024820152fd5b604051636142d24160e01b8152600490fd5b6044925060405191634f1e4a6d60e01b835260048301526024820152fd5b6001600160e01b031960049290920360031b82901b161690508380610856565b602490604051906309e90d5d60e41b82526004820152fd5b604051630b8fc7cd60e21b8152600490fd5b604051637f4d001d60e01b8152600490fd5b8281101561054e57610bbe81836114a5565b5190610bcd60208301516114fe565b91805192805468010000000000000000811015610ad957610bf39160018201815561178b565b610c4f57600193610c03916117ba565b60036060610c1e610c176020850151611543565b8451611588565b92610c2a8151856117ba565b610c3a60208201518786016117ba565b60408101516002850155015191015501610bac565b634e487b7160e01b600052600060045260246000fd5b60005b8183038110610c77575061053a565b610c80846114fe565b5490610c8b856114fe565b6000199283810191908211610dc157610cad91610ca79161178b565b506115e8565b610cb6866114fe565b8054938415610dab57610ccb8186018361178b565b959095610c4f5785610cdf600197546115ae565b9081610d6e575b505001905560038660405192610cfb84611178565b604051610d07816111e4565b600081528452610d5e604051610d1c816111e4565b6000815260208601908152610d55610d4d604088019460008652610d4760608a019760008952611543565b90611588565b9651876117ba565b518786016117ba565b5160028401555191015501610c68565b600091601f8082118a14610d87575050555b8b80610ce6565b91610da484929382845260208420940160051c84018b85016117a3565b5555610d80565b634e487b7160e01b600052603160045260246000fd5b634e487b7160e01b600052601160045260246000fd5b602490604051906333ffff9b60e01b82526004820152fd5b60449060405190633bae936760e21b825260048201526024356024820152fd5b91935091602080600192605f198a8203018552875190606080610e4e610e3e6080865190808752860190611480565b8686015185820388870152611480565b9360408101516040850152015191015296019201920186939194926102d7565b604051632207376360e21b81526060600482015293508392610e94906064850190611480565b91602484015260448301520390fd5b90919293610eb185836114a5565b51838610610ed3575b604001518101809111610dc157936001019291906101b0565b610edd86896118bf565b602082018051604051610f0d60208281610f00818301968781519384920161145d565b81010380845201826111ff565b51902060208301908151604051610f3460208281610f00818301968781519384920161145d565b5190200361103d5750504260608201511115610f51575b50610eba565b8151604051610f7060208281610f00818301968781519384920161145d565b5190208151604051610f9260208281610f00818301968781519384920161145d565b51902003610ffc57604081015160408301511490811591610fe9575b50610fb95738610f4b565b610fdf9051604051918291633090512760e01b8352604060048401526044830190611480565b4260248301520390fd5b6060915001516060820151141538610fae565b611039611027925191516040519384936338f5a4a360e21b8552604060048601526044850190611480565b83810360031901602485015290611480565b0390fd5b9061103961107092611082865193519151916040519586956301a7356160e71b8752606060048801526064870190611480565b85810360031901602487015290611480565b83810360031901604485015290611480565b60405162f3e6cf60e71b8152600490fd5b4260606110b283896118bf565b015111156110c3575b600101610198565b956000198114610dc157600101956110bb565b634e487b7160e01b600052603260045260246000fd5b60405163cc393bbb60e01b8152600490fd5b60405163134d7e2360e11b815260206004820152806110396024820186611480565b604051633ed482bf60e11b8152336004820152602490fd5b60405163451445c960e11b81523360048201526024810191909152604490fd5b604051632b6069a960e01b81523360048201526024810191909152604490fd5b608081019081106001600160401b03821117610ad957604052565b606081019081106001600160401b03821117610ad957604052565b60c081019081106001600160401b03821117610ad957604052565b604081019081106001600160401b03821117610ad957604052565b602081019081106001600160401b03821117610ad957604052565b90601f801991011681019081106001600160401b03821117610ad957604052565b91908260609103126107245760405161123881611193565b91829080356001600160a01b03811681036107245760409182918452602081013560208501520135910152565b903590601e198136030182121561072457018035906001600160401b0382116107245760200191813603831361072457565b903590601e198136030182121561072457018035906001600160401b03821161072457602001918160051b3603831361072457565b9291926001600160401b038211610ad957604051916112f5601f8201601f1916602001846111ff565b829481845281830111610724578281602093846000960137010152565b6001600160401b038111610ad95760051b60200190565b9080601f8301121561072457816020611344933591016112cc565b90565b92919061135381611312565b9160409161136460405194856111ff565b839581855260208095019160051b8101938385116107245781925b85841061138f5750505050505050565b6001600160401b038435818111610724578401916080838803126107245783516113b881611178565b833583811161072457886113cd918601611329565b81528984013592831161072457836113ea898c9695879601611329565b8483015285810135868301526060809101359082015281520193019261137f565b908060209392818452848401376000828201840152601f01601f1916010190565b9035601e19823603018112156107245701602081359101916001600160401b03821161072457813603831361072457565b60005b8381106114705750506000910152565b8181015183820152602001611460565b906020916114998151809281855285808601910161145d565b601f01601f1916010190565b80518210156110d65760209160051b010190565b60206114d291816040519382858094519384920161145d565b81017f863e98553d473a330f13570b828238eab81d26cbf5ccc0856a5fcff7a8742b5681520301902090565b602061151791816040519382858094519384920161145d565b81017f67c705175a85517628ec6756607a9a1ba0ec7dfd34fa758c0a03f49e7663a81081520301902090565b602061155c91816040519382858094519384920161145d565b81017f67c705175a85517628ec6756607a9a1ba0ec7dfd34fa758c0a03f49e7663a81181520301902090565b6020906115a292826040519483868095519384920161145d565b82019081520301902090565b90600182811c921680156115de575b60208310146115c857565b634e487b7160e01b600052602260045260246000fd5b91607f16916115bd565b906040519182600082546115fb816115ae565b9081845260209460019160018116908160001461166b575060011461162c575b50505061162a925003836111ff565b565b600090815285812095935091905b81831061165357505061162a935082010138808061161b565b8554888401850152948501948794509183019161163a565b9250505061162a94925060ff191682840152151560051b82010138808061161b565b611699909291926114b9565b918254906116a682611312565b916040926116b760405191826111ff565b81815260209182820160009788528388208489925b84841061174e575050505050855b8151811015611745576116ed81836114a5565b5151855161170a858281610f00818301968781519384920161145d565b519020855184810190611729868281610f008b8781519384920161145d565b51902014611739576001016116da565b95505050505060019190565b50505050508190565b6001916003918a5161175f81611193565b611768866115e8565b8152848601548382015260028601548c82015281520192019201919085906116cc565b80548210156110d65760005260206000200190600090565b8181106117ae575050565b600081556001016117a3565b91909182516001600160401b038111610ad9576117d782546115ae565b601f8111611882575b50602080601f831160011461181d575081929394600092611812575b50508160011b916000199060031b1c1916179055565b0151905038806117fc565b90601f19831695846000526020600020926000905b88821061186a57505083600195969710611851575b505050811b019055565b015160001960f88460031b161c19169055388080611847565b80600185968294968601518155019501930190611832565b6118af90836000526020600020601f840160051c810191602085106118b5575b601f0160051c01906117a3565b386117e0565b90915081906118a2565b9190604060606119398251936118d485611178565b828552602094838682015260009381858781940152015261190b856118f889611543565b988187519382858094519384920161145d565b81017f67c705175a85517628ec6756607a9a1ba0ec7dfd34fa758c0a03f49e7663a81081520301902061178b565b509082519181815461194a816115ae565b926001918083169081156119e057506001146119ae575b5050505092818482969760039652030190209080519361198085611178565b611989836115e8565b8552611997600184016115e8565b908501526002820154908401520154606082015290565b909180939450528582205b8383106119cf5750505081018186600386611961565b8054858401529186019181016119b9565b60ff191687525050505080151502820190508186600386611961565b9291907f7fffffffffffffffffffffffffffffff5d576e7357a4501ddfe92f46681b20a08311611a7f5791608094939160ff602094604051948552168484015260408301526060820152600093849182805260015afa15611a725781516001600160a01b03811615611a6c579190565b50600190565b50604051903d90823e3d90fd5b50505050600090600390565b6005811015611b8f5780611a9c5750565b60018103611ae95760405162461bcd60e51b815260206004820152601860248201527f45434453413a20696e76616c6964207369676e617475726500000000000000006044820152606490fd5b60028103611b365760405162461bcd60e51b815260206004820152601f60248201527f45434453413a20696e76616c6964207369676e6174757265206c656e677468006044820152606490fd5b600314611b3f57565b60405162461bcd60e51b815260206004820152602260248201527f45434453413a20696e76616c6964207369676e6174757265202773272076616c604482015261756560f01b6064820152608490fd5b634e487b7160e01b600052602160045260246000fdfea2646970667358221220d899e6564031f203144a6663e3c889b8d82779eb69f72444bafb39620a2106ea64736f6c63430008190033c001a09ecce3330dbb40744ca4e89d59ac9352d5096f3dc101591e16a6a9c01242e5bba01ea0be6d0ffd21dde9bd8f7a01a216552ee38148503846e9f113644eb49499a6",
            "new_receipt_trie_node_byte": "0xb9010e02f9010a0184016381eeb9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c0",
            "gas_used": 1594387,
            "jumpdest_table": {}

einar-polygon · 2024-09-24T10:09:07Z

From the discussion on Slack: This PR should add a CLI parameter allowing switching between RPC and Simulation (i.e. setting Jumpdest_table: None). Default should possibly depend on type:

Jerigon: Simulation
Native: RPC

Nashtare · 2024-09-24T17:02:26Z

zero/src/rpc/mod.rs

+pub enum JumpdestSrc {
+    Simulation,
+    Zero,
+    Jerigon,
+}


This is kind of confusing. Zero and Jerigon sound like they are the same source (or what else do you mean by Zero?) also we default jerigon nodes to Simulation so why do we have a Jerigon option?

I agree. How about:

pub enum JumpdestSrc { ProverSimulation, ClientFetchedStructlogs, ServerFetchedStructlogs, // later Serverside, // later }

@atanmarko What do you think about these names?

Not sure if they are convenient to type into cli, but OK

zero/src/bin/rpc.rs

zero/src/rpc/jerigon.rs

atanmarko · 2024-09-25T10:44:51Z

zero/src/rpc/jerigon.rs

+{
+    let tx_traces = tx_trace
+        .iter()
+        .map(|(h, t)| (Address::from(h.to_fixed_bytes()), t.clone()))


Cloning TxnTrace is not a good idea. Lot of overhead.
There could be 2 solutions:

Define generate_jumpdest_table with:

tx_traces: impl Iterator<Item = (&'a Address, &'a TxnTrace)>,

Convert tx_trace H160 to Address when you initially create it.

Confusing here is that we use both type Address = H160 and alloy_primitives::Address. But anyway, conversion should be done so that there is no unnecessary cloning of data.

…nsaction_jumps/pr

einar-polygon added performance Performance improvement related changes crate: evm_arithmetization Anything related to the evm_arithmetization crate. crate: zero_bin Anything related to the zero-bin subcrates. labels Jul 23, 2024

einar-polygon self-assigned this Jul 23, 2024

github-actions bot added the crate: trace_decoder Anything related to the trace_decoder crate. label Jul 23, 2024

Nashtare added this to the Performance Tuning milestone Jul 24, 2024

einar-polygon force-pushed the einar/prefetch_transaction_jumps/pr branch 2 times, most recently from 7ad89de to 18d5da4 Compare July 30, 2024 12:36

einar-polygon force-pushed the einar/prefetch_transaction_jumps/pr branch 2 times, most recently from 6255f00 to 6613802 Compare August 7, 2024 15:25

Nashtare reviewed Aug 7, 2024

View reviewed changes

evm_arithmetization/Cargo.toml Outdated Show resolved Hide resolved

evm_arithmetization/Cargo.toml Outdated Show resolved Hide resolved

zero_bin/rpc/Cargo.toml Outdated Show resolved Hide resolved

zero_bin/rpc/Cargo.toml Outdated Show resolved Hide resolved

Nashtare requested a review from 4l0n50 August 8, 2024 13:26

Nashtare reviewed Aug 17, 2024

View reviewed changes

zero_bin/rpc/src/native/txn.rs Outdated Show resolved Hide resolved

zero_bin/rpc/src/native/txn.rs Outdated Show resolved Hide resolved

einar-polygon force-pushed the einar/prefetch_transaction_jumps/pr branch from bdcf935 to 270256f Compare August 18, 2024 23:50

einar-polygon commented Aug 19, 2024

View reviewed changes

zero_bin/rpc/src/main.rs Outdated Show resolved Hide resolved

einar-polygon commented Aug 19, 2024

View reviewed changes

zero_bin/rpc/src/native/mod.rs Outdated Show resolved Hide resolved

Nashtare requested changes Aug 19, 2024

View reviewed changes

Nashtare reviewed Aug 22, 2024

View reviewed changes

einar-polygon force-pushed the einar/prefetch_transaction_jumps/pr branch 2 times, most recently from a0f0438 to 05e065a Compare August 28, 2024 13:58

Nashtare reviewed Aug 30, 2024

View reviewed changes

trace_decoder/src/decoding.rs Outdated Show resolved Hide resolved

trace_decoder/src/decoding.rs Outdated Show resolved Hide resolved

trace_decoder/src/decoding.rs Outdated Show resolved Hide resolved

einar-polygon force-pushed the einar/prefetch_transaction_jumps/pr branch from a9b13a3 to e351ff9 Compare September 2, 2024 17:54

Nashtare requested changes Sep 3, 2024

View reviewed changes

einar-polygon requested review from atanmarko and LindaGuiga as code owners September 20, 2024 22:33

small adjustments

62c7053

atanmarko reviewed Sep 23, 2024

View reviewed changes

fmt

74b86fd

einar-polygon requested a review from Nashtare September 23, 2024 12:25

feedback

c8be888

Nashtare reviewed Sep 23, 2024

View reviewed changes

feedback

d00439f

einar-polygon mentioned this pull request Sep 24, 2024

Simulate to get jumpdests on a per-transaction basis #653

Open

einar-polygon added 3 commits September 24, 2024 16:31

Add JumpdestSrc parameter

6876c07

Refactor

60efef9

Add jmp src to native

b07752d

Nashtare reviewed Sep 24, 2024

View reviewed changes

einar-polygon added 2 commits September 24, 2024 20:06

Feedback

66ea811

fixup! Feedback

f230b84

atanmarko reviewed Sep 25, 2024

View reviewed changes

einar-polygon added 2 commits September 25, 2024 23:35

feedback

90722a3

fix missing code for CREATE

a0e0879

einar-polygon requested a review from wborgeaud as a code owner September 25, 2024 21:42

einar-polygon added 3 commits September 26, 2024 17:26

fix

6bff4e4

Merge remote-tracking branch 'origin/develop' into einar/prefetch_tra…

5e4162d

…nsaction_jumps/pr

fix arguments

c26f475

		// We reverse the stack, so the order matches our assembly code.
		let evm_stack: Vec<_> = entry.stack.as_ref().unwrap().iter().rev().collect();

Prefetch JUMPDESTs through RPC #427

Are you sure you want to change the base?

Prefetch JUMPDESTs through RPC #427

Conversation

einar-polygon commented Jul 23, 2024 • edited Loading

Nashtare left a comment

Choose a reason for hiding this comment

Nashtare left a comment

Choose a reason for hiding this comment

einar-polygon commented Aug 26, 2024

Nashtare commented Aug 26, 2024

Nashtare commented Aug 26, 2024

einar-polygon commented Aug 26, 2024

Nashtare commented Aug 26, 2024

Nashtare commented Aug 29, 2024

Nashtare commented Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atanmarko Sep 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atanmarko commented Sep 23, 2024

Nashtare commented Sep 23, 2024

Nashtare commented Sep 23, 2024

einar-polygon commented Sep 23, 2024

Nashtare commented Sep 23, 2024

einar-polygon commented Sep 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atanmarko Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

einar-polygon commented Jul 23, 2024 •

edited

Loading

Nashtare commented Aug 29, 2024 •

edited

Loading

atanmarko Sep 23, 2024 •

edited

Loading

atanmarko Sep 25, 2024 •

edited

Loading