gossamer with Zombienet

[edwardmack]

TL;DR

In order for Zombienet to work with gossamer nodes both systems need to understand details about each other to start and test networks. There are differences with CLI attributes, parsing logs, and prometheus metrics. This discussion is to detail these issues and determine steps necessary to make both systems work together.

Spawning

• Zombienet details: https://github.com/paritytech/zombienet
• Zombienet builds a CLI command to spawn nodes, the CLI attributes that it uses are different that many of the CLI attributes gossamer uses. See comments in issue Investigate ZombieNet Integration Requirements #2843 for details. For testing I've created a fork of Zombienet that modifies the start command to conform to gossamer's CLI attributes. See this PR for details of changes made (note, these are just for testing, not necessarily the correct approach for solving the issues) Modifications to make work with gossamer node edwardmack/zombienet#1
• Zombienet parses log output to determine if a node has started, the logging that gossamer generates does not conform to what Zombienet expects. Details outlined in issue Updates logs to report when node has started to conform with ZombieNet's start check #2998.
• Zombienet uses polkadot CLI attribute node-key to define the libp2p key, which defines the nodes addresses, gossamer does not have that CLI argument. I've implemented this argument in for testing in a Draft PR feature (dot/network): add CLI flag --node-key, hex encoded libp2p node key #3043. (Currently draft until we determine direction based on this discussion)
• Gossamer (bug) requires config file to start as authority node, this should be able to be done without config file using CLI attributes, however there seems to be an issue with this. See issue Unable to start node as Authority Role without toml config #3020 for details.

Testing

• Zombienet testing with RPC calls seem to be working, further testing will determine if there are any issues.
• Zombienet uses prometheus metrics for some tests, some of these metrics are not in gossamer including node_roles and sub_libp2p_is_major_syncing

Next Steps?

• There are different ways to get these systems working together, modifying the gossamer node, or modifying the Zombinet client, or both. This discussion is to explore possible solutions.

[timwu20]

Zombienet uses polkadot CLI attribute node-key to define the libp2p key, which defines the nodes addresses, gossamer does not have that CLI argument. I've implemented this argument in for testing in branch https://github.com/ChainSafe/gossamer/tree/ed/zombienet_testing

Can you create a PR for this and link that?

[edwardmack]

Yes, I've created draft PR #3043 for this.

[timwu20]

Zombienet parses log output to determine if a node has started, the logging that gossamer generates does not conform to what Zombienet expects. Details outlined in issue #2998.

Have we looked into the log parser in zombienet? Can we modify it to support our logs?

[timwu20]

Gossamer (bug) requires config file to start as authority node, this should be able to be done without config file using CLI attributes, however there seems to be an issue with this. See issue #3020 for details.

We should prioritise this then in the next grooming.

Zombienet uses prometheus metrics for some tests, some of these metrics are not in gossamer including node_roles and sub_libp2p_is_major_syncing

Do we have issues for this?

[edwardmack]

Update, I've created PR #3091, and PR #3098 that address suggested updates to gossamer node so that it works with zombienet. These PRs include updates that enable gossamer to run without a toml config file, and updates to CLI interface to include adding aliases to CLI arguments to conform with zombienet/polkadot default CLI arguments. And implementing new CLI argument --node-key and --listen-addr

I've also forked zombienet from paritytech/zombienet repo to ChainSafe/zombienet repo and created PR #1 which includes suggested updates to zombienet that enable zombienet to spawn gossamer nodes.

Let me know your thought regarding these (if acceptable we'll approve and merge these) or suggestions/comments.

[qdm12]

I've also forked zombienet from paritytech/zombienet repo to ChainSafe/zombienet repo and created https://github.com/ChainSafe/zombienet/pull/1which includes suggested updates to zombienet that enable zombienet to spawn gossamer nodes.

Why not PR directly to parity's zombienet repo?

[dimartiro]

@edwardmack I think we can close this discussion since we have zombienet running in our CI now