-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: avalanche blockchain addValidator
is failed for a PoS L1 deployed to Fuji
#2526
Comments
Thanks for the issue. Please use the command |
Hi @felipemadero I have recognized the 5 latter nodes in the list is exactly the 5 validators that I had added beforehand. But I got confused with the first node, the Because there is an error that shown in the screenshots:
I worry this error will lead to a more significant problem for L1 operation. Moreover, when I check the logs of the local bootstrap validator node, which is created during deployment of the L1 to Fuji testnet and resides in the folder
How to fix the error? |
The local bootstrap validator node created during deploy was indeed a 6th validator on your setup. The issue is that |
In order to recover the L1, you should add more balance to the validator that got its balance to 0. |
Yeah I agree with @felipemadero's suggestion above. For some additional context, when a validator runs out of AVAX balance for the pay-as-you-go fee on the P-Chain, it becomes "inactive", meaning that it nodes are unable to connect to it, and that it is unable to participate in ICM signatures. In this case, it looks like the validator that became inactive due to running out of balance represented 50% of the weight of the L1, so other nodes are unable to progress at all while it is inactive, and ICM messages (which require 67% of stake) can't be constructed. |
Thanks for valuable advice and suggestions. I have increased the node balance using I rechecked the validator set using command The validator list is also updated on the Subnets site properly. I checked the logs of local-bootstrap validator node, it has passed the stake-amount health check. The L1 resumes to work such that it completes the tx and continue building the new blocks. However, there is a minor issue in the Subnet site that I think might be an UI error, the number of validators is zero as shown in the screenshots. Moreover, a bigger problem that I have found during resolving this issue is how do we figure out a proper validator set configuration for our L1. Let's discuss about it. The problem lies in our limit computing resources, such that we are only able to allocate up to 5 validator nodes for L1 testnet, including the bootstrap node. Thus with the weight of 20 for each non-bootstrap node, and the weight of 100 for the bootstrap node, we have total weight of 180 (or 200 in case we add 5 non-bootstrap nodes) for all validators of the L1. In my opinion, this weight distribution is bias towards the bootstrap node quite heavily. If the bootstrap node is stopped of working due to hardware problems or out of AVAX balance in our case, the whole L1 is halted. Is it possible to adjust the maximum weight that we can set to the non-bootstrap validator node upon adding it to L1, for example, weight of 50? With this possibility, I think we can operate the L1 with 5 physical nodes more safety and better fault tolerance. |
Hei! Taking note on the validators set to 0 on subnet site. There is no way to circumvent atm the total weight 20% churn restriction for the validators |
Other possibility is to set to use say 5 bootstrap validators on |
I told about the first option (1 bootstrap, and then add validators) because you already have an L1. It is needed to add 9 validators in this case, not 20, as per |
Also, considering the option of adding 9 validators, you can opt to add some of them locally on the same machine where the boostrap validator is executing. |
I have tried to go with the first option, i.e. launching the L1 with 5 local bootstrap validator nodes. The achieved result is good so far, thus I want to wrap the things up and share the result here in hoping that it will provide valuable insight to other teams troubling with this PoS issues.
Waiting for the L1 is deployed to Fuji testnet and bootstrapped with 5 local validator nodes, all has weight equal to 100. The initial validator set can be confirmed using the CLI command
You can notice that the balance of all validators is low as 0.1 AVAX only. The L1 should be deloyed to Fuji successfully and be shown in the Subnet site .
The weight is 100 and the AVAX balance to deposit is 1 AVAX (but you can choose to deposit with greater balance). The logs of command should be like following:
The local bootstrap nodeID can be referenced using
The new validator set is also updated in the Subnet detail page.
We are satisfied with this result and get confidence to go to the next step. Wish all other team achieve this good result too. |
Thanks so much on this detailed feedback for other teams! Also happy you unblocked. |
Describe the bug
Our team at derachain are trying to leverage ACP-77 to create our PoS L1 blockchain. We have successfully created the blockchain using avalanche-cli, then deployed it to Fuji testnet. Once we tried to add the new validators to L1 on Fuji, it seems does not work as expected.
To Reproduce
Expected behavior
The validator set is added successfully.
Screenshots
Deployment to Fuji Testnet is succeeded and it is tracked on the Subnet-networks site https://subnets-test.avax.network/subnets/2L6CmVuASKTG99FKhkqbRpitvgP4p4aXh52bFPwhc1FQFNnF6t
The validator set is added using CLI command
avalanche blockchain addValidator
, we have added totally 5 validators, with weight equal 20 for each node. All the nodes is configured to track and fully bootstrapped the L1.NODE1
Validator weight: 20
ValidationID: p3msdN2oiSPeiTYii8Mjv1DzkoVp798uHjFnmvc91G3z155i
RegisterL1ValidatorTx fee: 0.000099502 AVAX
RegisterL1ValidatorTx ID: 21eoE6tHNaMma5J7S39ayihVW4QU5pHo5zjMkEVeGvFH6Hqw3d
NODE2
Validator weight: 20
ValidationID: wpLhcSPJMTDBFtyb8PiFUE8CHcVnwD3M6gu1eVjk5aBvWfK9Y
RegisterL1ValidatorTx fee: 0.000099502 AVAX
RegisterL1ValidatorTx ID: 2HxztTo2xP85fadfM1JPmgyc9xXY8WGUbwHdvUuLKjjqDaoqAc
NODE3
Validator weight: 20
ValidationID: uE21rPFMqFbZy1TnbnWaGoMuPiUzt87iYrKXFp2WuTWav4HVu
RegisterL1ValidatorTx fee: 0.000093580 AVAX
RegisterL1ValidatorTx ID: 8KJzUpjSWakC7BbpjP8N86yyiFjhwFbuxMZPSTdmc4YktnBd7
NODE4
Validator weight: 20
ValidationID: 2aJu2G5Gbtdgu5s2Hh3Vtqeq9aD86miDC5CjhxqiW2c9uTpr44
RegisterL1ValidatorTx fee: 0.000093620 AVAX
RegisterL1ValidatorTx ID: TVtGp69Za3tmpmE9bwcUavwRhy3YGfNe5ZPhWTYs3SN188Thn
NODE5
Validator weight: 20
ValidationID: 67gYBXqmVTqeXweYzTnaAX9MWwq1hfMTYLyo3hWvSxQA2tyPz
RegisterL1ValidatorTx fee: 0.000093660 AVAX
RegisterL1ValidatorTx ID: jcmzk2UoRaHxF3RuB4NBew1rFDqhvgiJxdAwPi42GdQsQa6FL
avalanche blockchain validators
returns emptyLogs
If applicable, please include the relevant logs that indicate a problem.
HLySu3YcULhhhW19VSeeLwKkrbujGBJ2tzmiNB7y5mRcQRKiV.zip
Operating System
Ubuntu 22.04
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: