Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Case test_pfc_pause_extra_lossless_active of script test_tunnel_qos_remap.py could not run pass on mellanox platform due to lack of background traffic #17252

Open
echuawu opened this issue Feb 28, 2025 · 0 comments
Labels

Comments

@echuawu
Copy link
Contributor

echuawu commented Feb 28, 2025

Issue Type

Test Gap

Description

Case test_pfc_pause_extra_lossless_active of script test_tunnel_qos_remap.py failed on mellanox platform due to lack of background traffic, no pfc storm detected. In the end, after send packet in function pfc_pause_test, the queue counter value would increased by 1, then case failed:

Failed: The queue 3 for port Ethernet200 counter increased unexpectedly: Received packet that we expected not to receive on device 0, port 50.
========== RECEIVED ==========
0000  C2 A2 6A 9A B0 32 00 AA BB CC DD EE 08 00 45 0D  ..j..2........E.
0010  00 56 00 01 00 00 3E 06 B9 C6 01 01 01 01 C0 A8  .V....>.........
0020  00 24 04 D2 00 50 00 00 00 00 00 00 00 00 50 02  .$...P........P.
0030  20 00 94 66 00 00 74 65 73 74 73 2E 71 6F 73 2E   ..f..tests.qos.
0040  74 65 73 74 5F 74 75 6E 6E 65 6C 5F 71 6F 73 5F  test_tunnel_qos_
0050  72 65 6D 61 70 20 74 65 73 74 73 2E 71 6F 73 2E  remap tests.qos.
0060  74 65 73 74                                      test
==============================

Issue Details

The mellanox platform pfc storm detection need a background traffic, it's defined in tests/common/helpers/pfcwd_helper.py called send_background_traffic.
Case test_pfc_pause_extra_lossless_active of script test_tunnel_qos_remap.py failed on mellanox platform due to lack of background traffic, no pfc storm detected. In function pfc_pause_test, the queue counter after sent packet would be increased by 1:

def pfc_pause_test(storm_handler, peer_info, prio, ptfadapter, dut, port, queue, pkt, src_port, exp_pkt, dst_ports):
    try:
        # Start PFC storm from leaf fanout switch
        start_pfc_storm(storm_handler, peer_info, prio)
        ptfadapter.dataplane.flush()
        # Record the queue counter before sending test packet
        base_queue_count = get_queue_counter(dut, port, queue, True)
        # Send testing packet again
        testutils.send_packet(ptfadapter, src_port, pkt, 1)
        # The packet should be paused
        testutils.verify_no_packet_any(ptfadapter, exp_pkt, dst_ports)
        # Check the queue counter didn't increase
        queue_count = get_queue_counter(dut, port, queue, False)
        assert base_queue_count == queue_count
        return True
    finally:
        stop_pfc_storm(storm_handler)
show queue counters Ethernet200
Last cached time was 2025-02-28T08:28:12.037131
Ethernet200 Last cached time was 2025-02-28T08:28:12.037131
       Port    TxQ    Counter/pkts    Counter/bytes    Drop/pkts    Drop/bytes
-----------  -----  --------------  ---------------  -----------  ------------
Ethernet200    UC0               0                0            0           N/A
Ethernet200    UC1               0                0            0           N/A
Ethernet200    UC2               0                0            0           N/A
Ethernet200    UC3               1              104            0           N/A
Ethernet200    UC4               0                0            0           N/A
Ethernet200    UC5               0                0            0           N/A
Ethernet200    UC6               0                0            0           N/A
Ethernet200    UC7           1,358          118,410            0           N/A
Ethernet200    MC8             N/A              N/A          N/A           N/A
Ethernet200    MC9             N/A              N/A          N/A           N/A
Ethernet200   MC10             N/A              N/A          N/A           N/A
Ethernet200   MC11             N/A              N/A          N/A           N/A
Ethernet200   MC12             N/A              N/A          N/A           N/A
Ethernet200   MC13             N/A              N/A          N/A           N/A
Ethernet200   MC14             N/A              N/A          N/A           N/A
Ethernet200   MC15             N/A              N/A          N/A           N/A

The test should be enhanced by sending a background traffic to trigger a pfc storm detection, then the queue counter in this scenario would not be increased.

Impact or Proposed Behavior

Case test_pfc_pause_extra_lossless_active of script test_tunnel_qos_remap.py could not pass

Importance or Severity

Critical

Is it platform specific

mellanox

Relevant log output

Output of show version

202411 branch

Attach files (if any)

No response

echuawu added a commit to echuawu/sonic-mgmt that referenced this issue Feb 28, 2025
…net#17212 sonic-net#17214 #20414 sonic-net#17252

Skip ipv6 everflow test on dualtor setup due to git hub issue sonic-net#17209
Skip pfcwd cli test on dualtor aa testbed due to github issue sonic-net#17218
Skip L2 configuration test on dualtor setup due to git hub issue sonic-net#17212
Skip nvgre and vxlan hash case in fib test on dualtor aa testbed due to sonic-net#17214
Skip test_po_cleanup_after_reload test on dualtor setup due to github issue #20414
Skip test_pfc_pause_extra_lossless_active test on mellanox platform due to github issue sonic-net#17252
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants