DHT bootstraping success check

[stormshield-pj50]

Hi,

I stumbled on a weird behavior recently and I would like to know if it's normal or if it hides an error.

I was doing some DHT tests on the bootstrap when the bootstrapping node cannot be contacted (due to timeout for instance).

To know that the bootstrapping has finished and that it's ok to work with the DHT, I watch for BootstrapOk event with num_remaining set to 0.

During my tests, the bootstrapping actually fails since no node is reachable. But to my surprise, I still got a
BootstrapOk event with the num_remaining to 0, which let me think that everything is ready when it's not.

After a bit of research, I found that I could look into the stats of the KademliaEvent::OutboundQueryProgressed to check if at least one request is succesful, and therefore if the bootstrap did actually succeed.

However it feels a bit weird, moreover since there's a BootstrapTimeout, and therefore I'd like to know if this is the intended way of checking the bootstraping success.

[mxinden]

Thanks for the report.

After a bit of research, I found that I could look into the stats of the KademliaEvent::OutboundQueryProgressed to check if at least one request is succesful, and therefore if the bootstrap did actually succeed.

This is not the intended way of checking for failure.

I would determine failure/success based on whether all KademliaEvent::OutboundQueryProgressed for the given query ID failed, or whether one of them succeeded. Is that an option for you? Can you confirm that that works?

[stormshield-pj50]

Hi Max,

Thanks for your answer, I am going to check whether this works.

[stormshield-pj50]

Hi Max,
Sorry for the late response !

The solution is definitively an option for us.

I was just wondering if you could be interested about making these checks into some kind of a native event ?
Doing theses checks manually each time we want to make sure we are successfully connected to the DHT is not very convenient.

Also, I have noticed that during the bootstrap, if the first phase fails (nobody answer) we still proceed to phase two (random peerid request to fill the kbuckets).
That seem kinda weird since a failure during phase 1 mean that there is a very low probability of the phase 2 to work out.
And if the bootnode were to suddenly answer during phase 2, wouldn't we end up in a weird half booted status since phase 1 failed to correctly init the kbuckets ?

[mxinden]

And if the bootnode were to suddenly answer during phase 2, wouldn't we end up in a weird half booted status since phase 1 failed to correctly init the kbuckets ?

Do you have concerns from a functionality perspective or a security perspective?

[stormshield-pj50]

No, neither functionality or security concerns, as far as I know.
By the way would you be interested if we propose a patch in libp2p in order to make those checks (determine failure/success based on whether all KademliaEvent::OutboundQueryProgressed for the given query ID failed, or whether one of them succeeded) ? This could be interesting for libp2p users.

[mxinden]

Wouldn't that just be a method taking a bunch of KademliaEvents and returning a logic OR on the success/failure state?

[stormshield-pj50]

Well, not exactly.

Let me explain what I had in mind:
We launch the bootstrap to join the p2p network, and then we want to wait to be connected before proceeding to our tasks.

As such, we would expect something like launching the bootstrap with the .bootstrap() method, and then later, receiving an event such as KademliaEvent::BootstrapFinished(failed/success) to know how it went.
Obviously, we could implement something using KademliaEvent::OutboundQueryProgressed to find out if the bootstrap failed or succeeded, but it feels very much like a boilerplate.

Also, I would be quite surprised if we were the only ones needing to know the status of the bootstrap, which is why I think it could be interesting to integrate it in libp2p.

[thomaseizinger]

@stormshield-pj50 Is this still an issue?

We launch the bootstrap to join the p2p network, and then we want to wait to be connected before proceeding to our tasks.

Allow me to propose a slightly different way of thinking: Instead of "waiting to be connected", you should tie your logic to being connected to a number of peers or specific peers, depending on the concrete usecase.

Bootstrapping is also something that you should do regularly and not just once. Some of the nodes you discovered might have gone offline in the meantime for example.