Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to bench custom network topologies #92

Open
fyrchik opened this issue Sep 4, 2021 · 3 comments
Open

Allow to bench custom network topologies #92

fyrchik opened this issue Sep 4, 2021 · 3 comments
Labels
feature Completely new functionality I1 High impact S2 Regular significance U4 Nothing urgent

Comments

@fyrchik
Copy link
Contributor

fyrchik commented Sep 4, 2021

In real networks topology is far from full mesh. It would be nice if we could bench such topologies too.
The easiest way to achieve this is to restrict MaxPeers to 2, but we also need to be sure that each node has it's way to RPC node.
This can be achieved either with some custom docker network configuration or by explicitly blacklisting IPs inside the container.

@roman-khimov
Copy link
Member

I don't think it makes much sense for a network with 4 CNs, but I'd really like to see 7 CNs plus 2 RPC nodes setup with a limit of up to 3-4 connections per node, that'd be interesting. Also, we probably want to have some well-known set of topologies to compare different results against each other.

@fyrchik
Copy link
Contributor Author

fyrchik commented Sep 21, 2021

@roman-khimov Actually I already have some interesting results even for 4 CNs with line/star topologies, where one of the nodes tries to initiate change view and/or receives new blocks before the commit messages for them ("invalid signature" error).

The problem with just restricting MaxPeers is less control over the actual topology and thus less deterministic results (e.g. for 7 nodes we can have a single bridge between 2 almost full-mesh clusters or a more balanced variant. Also, the load depends on which node will be connected with the RPC, we can even fail to run bench if the first one is not).

I think the best solution is:

  1. Generate subnet for each connected set of nodes (e.g. to connect 4 nodes in line we have neo_go_12, neo_go_23, neo_go_34 networks with each node except the last belonging to 2 networks + the first one uses default to connect with RPC node).
  2. Have single docker-compose template (this also helps to unify go/mixed/sharp as well as 7+2 setups).
  3. The actual files can be written by hand (or with some generation utility) or generated on the fly based on simple descriptions (star.4, mesh.7 etc.).

(1) also allows to perform traffic shaping on a per-subnet basis (subnets should correspond to different interfaces in containers, though I haven't checked this).
We can do (2) separately in #95 .

@roman-khimov
Copy link
Member

We can have a hell lot of different connection patterns even for seven nodes. Some of them are unrealistic (like "line" mentioned above), maybe it's interesting to see how things go in this case, but at least initially I'd be more interested in something resembling real network. That's why I'm trying to create some scheme that doesn't need a lot of configuration tweaks, because subnets will require you to provide different seeds for every machine and be careful with connection limits as the node will try to reach those other nodes it'll receive via getaddr. Subnets make the scheme more fragile, I'd say. At the same time, reproducibility is important, so the more control we have the better. Maybe we need to try different approaches and see how they work, but let's start with simpler ones.

@roman-khimov roman-khimov added U4 Nothing urgent S2 Regular significance I1 High impact feature Completely new functionality labels Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Completely new functionality I1 High impact S2 Regular significance U4 Nothing urgent
Projects
None yet
Development

No branches or pull requests

2 participants