(cherry-pick) NFR Test Results for NGF version 1.6.0 (#3013)

tests/results/dp-perf/1.6.0/1.6.0-oss.md

@@ -0,0 +1,92 @@
+# Results
+
+## Test environment
+
+NGINX Plus: false
+
+NGINX Gateway Fabric:
+
+- Commit: b61c61d3f9ca29c6eb93ce9b44e652c9a521b3a4
+- Date: 2025-01-13T16:47:24Z
+- Dirty: false
+
+GKE Cluster:
+
+- Node count: 12
+- k8s version: v1.30.6-gke.1596000
+- vCPUs per node: 16
+- RAM per node: 65853984Ki
+- Max pods per node: 110
+- Zone: us-west1-b
+- Instance Type: n2d-standard-16
+
+## Summary:
+
+- Performance stayed consistent with 1.5.0 results. Average latency slightly increased across all routing methods.
+- Errors that occurred are consistent with errors that occurred in the previous results.
+
+## Test1: Running latte path based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.04, 998.38
+Duration      [total, attack, wait]             30s, 29.999s, 844.157µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  309.6µs, 718.534µs, 681.308µs, 786.633µs, 827.114µs, 971.115µs, 18.386ms
+Bytes In      [total, mean]                     4741568, 158.05
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           99.84%
+Status Codes  [code:count]                      200:29951  503:49
+Error Set:
+503 Service Temporarily Unavailable
+```
+
+## Test2: Running coffee header based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.01, 999.98
+Duration      [total, attack, wait]             30.001s, 30s, 708.18µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  519.443µs, 728.205µs, 716.283µs, 820.709µs, 859.918µs, 962.843µs, 6.974ms
+Bytes In      [total, mean]                     4770000, 159.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test3: Running coffee query based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.02, 1000.00
+Duration      [total, attack, wait]             30s, 29.999s, 746.102µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  533.22µs, 735.075µs, 722.549µs, 830.432µs, 871.714µs, 973.911µs, 6.9ms
+Bytes In      [total, mean]                     5010000, 167.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test4: Running tea GET method based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.02, 1000.00
+Duration      [total, attack, wait]             30s, 29.999s, 737.741µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  528.445µs, 724.715µs, 711.435µs, 816.76µs, 859.214µs, 967.474µs, 11.985ms
+Bytes In      [total, mean]                     4680000, 156.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test5: Running tea POST method based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.03, 1000.01
+Duration      [total, attack, wait]             30s, 29.999s, 643.191µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  538.368µs, 728.96µs, 714.974µs, 818.991µs, 860.142µs, 971.866µs, 11.543ms
+Bytes In      [total, mean]                     4680000, 156.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```

tests/results/dp-perf/1.6.0/1.6.0-plus.md

@@ -0,0 +1,90 @@
+# Results
+
+## Test environment
+
+NGINX Plus: true
+
+NGINX Gateway Fabric:
+
+- Commit: b61c61d3f9ca29c6eb93ce9b44e652c9a521b3a4
+- Date: 2025-01-13T16:47:24Z
+- Dirty: false
+
+GKE Cluster:
+
+- Node count: 12
+- k8s version: v1.30.6-gke.1596000
+- vCPUs per node: 16
+- RAM per node: 65853984Ki
+- Max pods per node: 110
+- Zone: us-west1-b
+- Instance Type: n2d-standard-16
+
+## Summary:
+
+- Performance stayed consistent with 1.5.0 results. Average latency slightly increased across all routing methods.
+
+## Test1: Running latte path based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.03, 1000.00
+Duration      [total, attack, wait]             30s, 29.999s, 744.047µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  535.49µs, 722.768µs, 702.708µs, 807.78µs, 849.575µs, 981.854µs, 21.041ms
+Bytes In      [total, mean]                     4770000, 159.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test2: Running coffee header based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.01, 999.98
+Duration      [total, attack, wait]             30s, 30s, 718.788µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  558.587µs, 766.304µs, 750.921µs, 866.313µs, 907.422µs, 1.022ms, 10.872ms
+Bytes In      [total, mean]                     4800000, 160.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test3: Running coffee query based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.02, 999.99
+Duration      [total, attack, wait]             30s, 30s, 733.649µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  572.624µs, 771.492µs, 758.449µs, 867.491µs, 907.997µs, 1.032ms, 11.906ms
+Bytes In      [total, mean]                     5040000, 168.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test4: Running tea GET method based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.03, 1000.00
+Duration      [total, attack, wait]             30s, 29.999s, 712.155µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  549.224µs, 760.423µs, 746.75µs, 853.877µs, 894.554µs, 1.008ms, 8.12ms
+Bytes In      [total, mean]                     4710000, 157.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```
+
+## Test5: Running tea POST method based routing
+
+```text
+Requests      [total, rate, throughput]         30000, 1000.00, 999.98
+Duration      [total, attack, wait]             30.001s, 30s, 778.666µs
+Latencies     [min, mean, 50, 90, 95, 99, max]  544.486µs, 762.077µs, 748.375µs, 852.722µs, 893.014µs, 1.009ms, 9.632ms
+Bytes In      [total, mean]                     4710000, 157.00
+Bytes Out     [total, mean]                     0, 0.00
+Success       [ratio]                           100.00%
+Status Codes  [code:count]                      200:30000
+Error Set:
+```

tests/results/longevity/1.6.0/oss.md

@@ -0,0 +1,97 @@
+# Results
+
+## Test environment
+
+NGINX Plus: false
+
+NGINX Gateway Fabric:
+
+- Commit: 8be03e1fc5161a2b1bc0962fb0d8732114a9093d
+- Date: 2025-01-14T18:57:38Z
+- Dirty: true
+
+GKE Cluster:
+
+- Node count: 3
+- k8s version: v1.30.6-gke.1596000
+- vCPUs per node: 2
+- RAM per node: 4018128Ki
+- Max pods per node: 110
+- Zone: us-central1-c
+- Instance Type: e2-medium
+
+## Traffic
+
+HTTP:
+
+```text
+Running 5760m test @ http://cafe.example.com/coffee
+  2 threads and 100 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency   189.49ms  147.10ms   2.00s    78.44%
+    Req/Sec   293.54    193.84     1.95k    66.59%
+  198532845 requests in 5760.00m, 67.91GB read
+  Socket errors: connect 0, read 309899, write 63, timeout 2396
+Requests/sec:    574.46
+Transfer/sec:    206.05KB
+```
+
+HTTPS:
+
+```text
+Running 5760m test @ https://cafe.example.com/tea
+  2 threads and 100 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency   179.59ms  121.50ms   1.99s    67.56%
+    Req/Sec   292.54    193.88     2.39k    66.47%
+  197890521 requests in 5760.00m, 66.57GB read
+  Socket errors: connect 176, read 303560, write 0, timeout 7
+Requests/sec:    572.60
+Transfer/sec:    201.98KB
+```
+
+### Logs
+
+No error logs in nginx-gateway.
+
+No error logs in nginx.
+
+
+### Key Metrics
+
+#### Containers memory
+
+![oss-memory.png](oss-memory.png)
+
+#### NGF Container Memory
+
+![oss-ngf-memory.png](oss-ngf-memory.png)
+
+### Containers CPU
+
+![oss-cpu.png](oss-cpu.png)
+
+### NGINX metrics
+
+![oss-stub-status.png](oss-stub-status.png)
+
+### Reloads
+
+Rate of reloads - successful and errors:
+
+![oss-reloads.png](oss-reloads.png)
+
+Reload spikes correspond to 1 hour periods of backend re-rollouts.
+
+No reloads finished with an error.
+
+Reload time distribution - counts:
+
+![oss-reload-time.png](oss-reload-time.png)
+
+
+## Comparison with previous results
+
+Graphs look similar to 1.5.0 results. There is a color change swap in a few graphs which is a little confusing.
+NGINX container memory decreased dramatically. NGINX Stub Status graph is confusing to interpret, which can make it seem
+quite different to the 1.5.0 results, but it is similar, only with an increase in requests.