Support monitoring USDT probes from multiple ovs-vswitchd processes

[igsilya]

In some setups, multiple OVS instances can run on the same host. Main examples would be test deployments, but it might be useful in some production configurations as well, in theory.

Two types of test deployments where we would like to be able to use retis today are:

• OVN system tests. These can start multiple instance of OVS and OVN in different sandboxes (network namespaces) in order to test cross-"node" traffic.
• ovn-fake-multinode Similar setup, but designed for small test deployments or used by the ovn-heater to mimic large scale deployments using limited hardware resources. In this case the fake node is a docker/podman container that runs a separate OVS+OVN stack. All containers are connected by another instance of OVS on the host that also connects to similar deployments on different physical hosts.

As we can see in this case multiple instances of ovs-vswitchd are managing the datapath in their own namespaces, so will have different handler threads and different OpenFlow rules. Would be great to be able to track the packet from one sandbox to another including upcalls handled by different ovs-vswitchd processes, i.e. see USDT probes generated by all the instances.

Should be useful for debugging production OVN issues when someone managed to reproduce it with a system test or within a fake-multinode setup.

[amorenoz]

Thanks @igsilya for the suggestion. Indeed I have been wanting to add this for a while. In fact I also wanted to include kind deployments.

A possible inconvenient that would difficult user-friendliness is the fact that retis needs to access the ovs-vswitchd binary (and possibly the shared libraries) in order to extract USDT addresses so in the case of containers, we need to bind-mount them.