Automated cherry pick of #5532: Improvements to FlowExporter benchmarks #5592: Remove redundant log in fillPodInfo/fillServiceInfo #4252: e2e framework for Antrea's native secondary network #5731: Enhance Records Filtering: Utilizing Labels In The E2E Test #5770: Improve flow-visibility e2e test #5882

Also add BenchmarkConnStore, meant to measure memory usage of the connection store (and all the connections). Signed-off-by: Antonin Bas <[email protected]>

The net/netip package was introduced in Go 1.18. Compared to the existing net.IP type, the netip.Addr type takes less memory (especially for IPv6 addresses), is immutable, and is comparable so it supports == and can be used as a map key. In our case, this means that the flowexporter.Connection type takes a little less memory, and that the flowexporter.Tuple can be used directly as the flowexporter.ConnectionKey (no need to generate a string for use as the map key). Overall, this leads to higher performance (decreased CPU and memory usage) for the FlowExporter. There is potential for further improvement, as several key functions take as parameters both the flowexporter.Connection and flowexporter.ConnectionKey objects, which is no longer really required. See antrea-io#5271 Signed-off-by: Antonin Bas <[email protected]>

This release uses netip.Addr instead of net.IP to represent connections. Signed-off-by: Antonin Bas <[email protected]>

Signed-off-by: Antonin Bas <[email protected]>

Signed-off-by: Yun-Tang Hsu <[email protected]>

Signed-off-by: Arunkumar Velayutham <[email protected]>

In this commit, we: 1. Set a label to Perftest Pods before initiating traffic in each subtest to filter records in both the IPFIX collector Pod and the ClickHouse. 2. Remove the --since-time flag used during log retrieval from the IPFIX collector Pod in the e2e test. 3. Cease reliance on timestamps for record filtering due to potential time discrepancies between the testbed and Kubernetes nodes, which might hinder the retrieval of desired logs. Signed-off-by: Yun-Tang Hsu <[email protected]>

In this commit, we do: 1. Changed the order where we append expired records before exporting them from our exporter. For inter-node traffic with egress/ingress np with action drop, we will receive records from PacketIn and the conntrack table. For the ingress case, there is no issue as we will receive the records from both nodes and these two records are both correlationRequired. For the egress case, the record from the conntrack table is correlationRequired and the record from PacketIn is not correlationRequired. If the record from the conntrack table arrive at the FA first, then the record will need to do correlation at FA. The ReadyToSend will be true forever as it keeps waiting to do correlation. 2. Add check to verify if Flow Exporters can successfully resolve the Flow Aggregator Service address before sending traffic. 3. Add check to verify if Flow Aggregator can successfully connect to the ClickHouse before sending traffic. 4. Add labels to External subtest to filter useless logs from the IPFIX collector Pod. 5. Confirm the correct addition of a label to a specific Pod after updating the Pod. 6. Remove the octetDeltaCount check and, instead, filter out all records with octetDeltaCount=0 when retrieving records from the IPFIX collector Pod and ClickHouse. 7. Use new image from go-ipfix. We improve the IPFIX collector by: a. Disable printing records whenever we receive it. Instead, we store records in a string array. b. Add http listener and handler to receive request to return or reset records. In this way, we can reduce the retrieving log time from ~4s to ~80ms when we have ~1900 records inside it. Signed-off-by: Yun-Tang Hsu <[email protected]>

Signed-off-by: Yun-Tang Hsu <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Commits on Jan 16, 2024