Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: "Aggregator failed to respond to task" when running with local DevNet #61

Open
wawrzek opened this issue May 14, 2024 · 4 comments

Comments

@wawrzek
Copy link

wawrzek commented May 14, 2024

I use a local devnet (https://github.com/ivy-net/iv1) to run EigenLayer and this AVS locally. Smart contracts are deployed properly, both aggregator and operator starts after only adjusting the Makefile (or passing 2 variables). However, aggregator fails with the following error message (more context from logs below).

 Error submitting SubmitTaskResponse tx while calling respondToTask {"err":"execution reverted: BLSSignatureChecker.checkSignatures: invalid reference block"}

I tried to track where the reference block set was, and how it became invalid. I cannot find anything obvious in the code, especially the anvil deployment works fine. On Discord, it was suggested that there is a problem with the AVS code, and it would be fixed.

I see that the anvil deployment is pushed by 100 block forward. Is that related?

[2024-05-14 13:41:28.763 BST] INFO (aggregator/aggregator.go:199) Aggregator sending new task {"numberToSquare":"9"}
[2024-05-14 13:41:30.774 BST] INFO (txmgr/txmgr.go:143) Transaction not yet mined {"txID":"0x86a0ecfab3410f48821ad6e8360cff4c02691865d855d706d48b755e7c459d9f"}
[2024-05-14 13:41:31.603 BST] INFO (aggregator/rpc_server.go:51) Received signed task response: &aggregator.SignedTaskResponse{TaskResponse:contractIncredibleSquaringTaskManager.IIncredibleSquaringTaskManagerTaskResponse{ReferenceTaskIndex:0x9, NumberSquared:81}, BlsSignature:bls.Signature{G1Point:(*bls.G1Point)(0x140000d41d8)}, OperatorId:types.Bytes32{0xc4, 0xc2, 0x10, 0x30, 0xe, 0x28, 0xab, 0x4b, 0xa7, 0xb, 0x7f, 0xbb, 0xe, 0xfa, 0x55, 0x7d, 0x2a, 0x2a, 0x5f, 0x1f, 0xbf, 0xa6, 0xf8, 0x56, 0xe4, 0xcf, 0x3e, 0x9d, 0x76, 0x6a, 0x21, 0xdc}}
[2024-05-14 13:41:33.605 BST] INFO (aggregator/rpc_server.go:51) Received signed task response: &aggregator.SignedTaskResponse{TaskResponse:contractIncredibleSquaringTaskManager.IIncredibleSquaringTaskManagerTaskResponse{ReferenceTaskIndex:0x9, NumberSquared:81}, BlsSignature:bls.Signature{G1Point:(*bls.G1Point)(0x140000fa068)}, OperatorId:types.Bytes32{0xc4, 0xc2, 0x10, 0x30, 0xe, 0x28, 0xab, 0x4b, 0xa7, 0xb, 0x7f, 0xbb, 0xe, 0xfa, 0x55, 0x7d, 0x2a, 0x2a, 0x5f, 0x1f, 0xbf, 0xa6, 0xf8, 0x56, 0xe4, 0xcf, 0x3e, 0x9d, 0x76, 0x6a, 0x21, 0xdc}}
[2024-05-14 13:41:33.610 BST] INFO (aggregator/aggregator.go:142) Received response from blsAggregationService {"blsAggServiceResp":{"Err":null,"NonSignerQuorumBitmapIndices":[],"NonSignerStakeIndices":[[]],"NonSignersPubkeysG1":[],"QuorumApkIndices":[1],"QuorumApksG1":[{"X":"643552363890320897587044283125191574906281609959531590546948318138132520777","Y":"7028377728703212953187883551402495866059211864756496641401904395458852281995"}],"SignersAggSigG1":{"g1_point":{"X":"8443873543902734941423897488132867553403723007728710173648445437781070293444","Y":"21607580942168206894016089909611195006504832862680716687827944912459106791693"}},"SignersApkG2":{"X":{"A0":"15669747281918965782125375489377843702338327900115142954223823046525120542933","A1":"10049360286681290772545787829932277430329130488480401390150843123809685996135"},"Y":{"A0":"14982008408420160629923179444218881558075572058100484023255790835506797851583","A1":"4979648979879607838890666154119282514313691814432950078096789133613246212107"}},"TaskIndex":9,"TaskResponseDigest":[158,82,41,43,207,191,248,182,122,229,107,175,3,37,244,192,174,97,210,14,175,116,199,54,192,157,239,85,1,22,173,103],"TotalStakeIndices":[1]}}
[2024-05-14 13:41:33.610 BST] INFO (aggregator/aggregator.go:181) Threshold reached. Sending aggregated response onchain. {"taskIndex":9}
[2024-05-14 13:41:33.614 BST] ERROR (chainio/avs_writer.go:115) Error submitting SubmitTaskResponse tx while calling respondToTask {"err":"execution reverted: BLSSignatureChecker.checkSignatures: invalid reference block"}
Stacktrace
    github.com/Layr-Labs/incredible-squaring-avs/core/chainio.(*AvsWriter).SendAggregatedResponse
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/core/chainio/avs_writer.go:115
    github.com/Layr-Labs/incredible-squaring-avs/aggregator.(*Aggregator).sendAggregatedResponseToContract
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/aggregator.go:190
    github.com/Layr-Labs/incredible-squaring-avs/aggregator.(*Aggregator).Start
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/aggregator.go:143
    main.aggregatorMain
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/cmd/main.go:57
    github.com/urfave/cli.HandleAction
    	/Users/wawrzek/go/pkg/mod/github.com/urfave/[email protected]/app.go:524
    github.com/urfave/cli.(*App).Run
    	/Users/wawrzek/go/pkg/mod/github.com/urfave/[email protected]/app.go:286
    main.main
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/cmd/main.go:33
    runtime.main
    	/opt/homebrew/Cellar/go/1.22.2/libexec/src/runtime/proc.go:271
[2024-05-14 13:41:33.614 BST] ERROR (aggregator/aggregator.go:192) Aggregator failed to respond to task {"err":"execution reverted: BLSSignatureChecker.checkSignatures: invalid reference block"}
Stacktrace
    github.com/Layr-Labs/incredible-squaring-avs/aggregator.(*Aggregator).sendAggregatedResponseToContract
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/aggregator.go:192
    github.com/Layr-Labs/incredible-squaring-avs/aggregator.(*Aggregator).Start
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/aggregator.go:143
    main.aggregatorMain
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/cmd/main.go:57
    github.com/urfave/cli.HandleAction
    	/Users/wawrzek/go/pkg/mod/github.com/urfave/[email protected]/app.go:524
    github.com/urfave/cli.(*App).Run
    	/Users/wawrzek/go/pkg/mod/github.com/urfave/[email protected]/app.go:286
    main.main
    	/Users/wawrzek/Ethereum/EigenLayer/incredible-squaring-avs/aggregator/cmd/main.go:33
    runtime.main
    	/opt/homebrew/Cellar/go/1.22.2/libexec/src/runtime/proc.go:271

Makefile diff

diff --git a/Makefile b/Makefile
index a2f5164..435d00e 100644
--- a/Makefile
+++ b/Makefile
@@ -6,8 +6,8 @@ help:

 AGGREGATOR_ECDSA_PRIV_KEY=0x2a871d0798f97d79848a013d4936a73bf4cc922c825d33c1cf7073dff6d409c6
 CHALLENGER_ECDSA_PRIV_KEY=0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a
-DEPLOYER_PRIVATE_KEY=0x2a871d0798f97d79848a013d4936a73bf4cc922c825d33c1cf7073dff6d409c6
-CHAINID=31337
+DEPLOYER_PRIVATE_KEY=0x2e0834786285daccd064ca17f1654f67b4aef298acbb82cef9ec422fb4975622
+CHAINID=32382
 # Make sure to update this if the strategy address changes
 # check in contracts/script/output/${CHAINID}/credible_squaring_avs_deployment_output.json
 STRATEGY_ADDRESS=0x7a2088a1bFc9d81c55368AE168C2C02570cB814F
@samlaf
Copy link
Collaborator

samlaf commented May 15, 2024

That's very strange. I don't think this is related to advancing the chain by 100 blocks upon startup, that's for separate reasons.

You're hitting this require: https://github.com/Layr-Labs/eigenlayer-middleware/blob/a23de118e7d16081d350c7f83c24261d1421b0ba/src/BLSSignatureChecker.sol#L115

We recently updated the check to be an inequality < instead of <=, so now you can't use a RBN that is the current block.
But when submitting a tx the block is advanced (new block is mined when tx is submitted), which seems to indicate that a RBN of currentBlockNumber + 1 would be set..? That seems very strange.

Let me know if this helps... hopefully this will be enough for you to debug the issue.

@wawrzek
Copy link
Author

wawrzek commented May 15, 2024

@samlaf looks like you are right. I applied the following change and error went away:

diff --git a/src/BLSSignatureChecker.sol b/src/BLSSignatureChecker.sol
index c79a225..84a13c7 100644
--- a/src/BLSSignatureChecker.sol
+++ b/src/BLSSignatureChecker.sol
@@ -112,7 +112,7 @@ contract BLSSignatureChecker is IBLSSignatureChecker {
             "BLSSignatureChecker.checkSignatures: input nonsigner length mismatch"
         );

-        require(referenceBlockNumber < uint32(block.number), "BLSSignatureChecker.checkSignatures: invalid reference block");
+        require(referenceBlockNumber <= uint32(block.number), "BLSSignatureChecker.checkSignatures: invalid reference block");

         // This method needs to calculate the aggregate pubkey for all signing operators across
         // all signing quorums. To do that, we can query the aggregate pubkey for each quorum

Of course, it's not something I cannot keep long term, or suggest to adjust. I just wanted to have a quick test.

We recently updated the check to be an inequality < instead of <=, so now you can't use a RBN that is the current block. But when submitting a tx the block is advanced (new block is mined when tx is submitted)

Is it true for a POS chain? My understanding is that for a POS Ethereum network, a block is validated every 12 seconds (on average). It would mean that for that this AVS is too quick for local network.

@wawrzek
Copy link
Author

wawrzek commented May 15, 2024

I'm not sure if's pure timing.
a) I adjusted the block time creation in Anvil (using the --block-time 10). AVS works fine.
b) I looked at the POS network, and block ticking much faster than 12 seconds. My knowledge is limited, so I don't know how it's controlled, but maybe that's the problem somewhere in the settings.

diff --git a/tests/anvil/utils.sh b/tests/anvil/utils.sh
index 84184e3..f6f9ded 100644
--- a/tests/anvil/utils.sh
+++ b/tests/anvil/utils.sh
@@ -35,6 +35,7 @@ start_anvil_docker() {
     docker run --rm -d --name anvil -p 8545:8545 $LOAD_STATE_VOLUME_DOCKER_ARG $DUMP_STATE_VOLUME_DOCKER_ARG \
         --entrypoint anvil \
         $FOUNDRY_IMAGE \
-        $LOAD_STATE_ANVIL_ARG $DUMP_STATE_ANVIL_ARG --host 0.0.0.0
-    sleep 2
+        $LOAD_STATE_ANVIL_ARG $DUMP_STATE_ANVIL_ARG --host 0.0.0.0 \
+        --block-time 10
+    sleep 12
 }

@wawrzek
Copy link
Author

wawrzek commented May 16, 2024

After further investigation and playing with 2 parameters in the prysm (included below), I noticed that the problem become intermediate. Sometimes no errors at all, sometimes all the time, and I had runs when the error appeared from time to time.

# Time parameters
SECONDS_PER_SLOT: 2
SLOTS_PER_EPOCH: 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants