Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky VRF tests due to missing IPv6 route #1103

Closed
LionelJouin opened this issue Oct 3, 2024 · 1 comment · Fixed by #1104
Closed

Flaky VRF tests due to missing IPv6 route #1103

LionelJouin opened this issue Oct 3, 2024 · 1 comment · Fixed by #1104

Comments

@LionelJouin
Copy link
Member

In the tests:

  • adds the interface and custom routing to new VRF - Setting custom routing first
  • filters the correct routes to import to new VRF - Setting custom routing for IF0Name
  • filters the correct routes to import to new VRF - Setting custom routing for IF1Name

IPv6 are added to an interface, and routes using that IPv6 is created. In order to create the route, we should wait for some IPv6 route to be created (created automatically by the kernel when adding the IPv6 to the interface). Without that route, the RouteAdd call in the tests will fail, for example:

• [FAILED] [0.011 seconds]
vrf plugin [It] filters the correct routes to import to new VRF
/home/runner/work/plugins/plugins/plugins/meta/vrf/vrf_test.go:278

  Timeline >>
  STEP: Setting custom routing for IF0Name @ 09/20/24 07:14:34.461
  STEP: Setting custom routing for IF1Name @ 09/20/24 07:14:34.462
  STEP: Adding if1 to the VRF @ 09/20/24 07:14:34.464
  STEP: Checking routes are moved correctly to VRF @ 09/20/24 07:14:34.467
  [FAILED] in [It] - /home/runner/work/plugins/plugins/plugins/meta/vrf/vrf_test.go:392 @ 09/20/24 07:14:34.469
  << Timeline

  [FAILED] Unexpected error:
      <syscall.Errno>: 
      invalid argument
      0x16
  occurred
  In [It] at: /home/runner/work/plugins/plugins/plugins/meta/vrf/vrf_test.go:392 @ 09/20/24 07:14:34.469

Here is some job that have failed.
https://github.com/containernetworking/plugins/actions/runs/10954784634/job/30417500257?pr=1092
https://github.com/containernetworking/plugins/actions/runs/11109122762/job/30863918356?pr=1082

@LionelJouin
Copy link
Member Author

Here is the second issue related to the VRF plugin:

Running tests
without coverage profile generation...
github.com/containernetworking/plugins/plugins/meta/vrf
=== RUN   TestVRF
Running Suite: plugins/meta/vrf - containernetworking/plugins/plugins/meta/vrf
=======================================================================================================
Random Seed: 1728311452

Will run 1 of 1 specs
------------------------------
• [FAILED] [0.630 seconds]
vrf plugin [It] adds the interface and custom routing to new VRF
containernetworking/plugins/plugins/meta/vrf/vrf_test.go:183

  Timeline >>
  STEP: Setting custom routing first @ 10/07/24 16:30:52.644
  [FAILED] in [It] - containernetworking/plugins/plugins/meta/vrf/vrf_test.go:269 @ 10/07/24 16:30:53.263
  << Timeline

  [FAILED] Unexpected error:
      <*errors.errorString | 0xc000228970>: 
      cmdAdd failed: could not add route '{Ifindex: 2 Dst: 1111:dddd::/80 Src: abcd:1234:ffff::cdde Gw: abcd:1234:ffff::1 Flags: [] Table: 1 Realm: 0}': invalid argument
      {
          s: "cmdAdd failed: could not add route '{Ifindex: 2 Dst: 1111:dddd::/80 Src: abcd:1234:ffff::cdde Gw: abcd:1234:ffff::1 Flags: [] Table: 1 Realm: 0}': invalid argument",
      }
  occurred
  In [It] at: containernetworking/plugins/plugins/meta/vrf/vrf_test.go:269 @ 10/07/24 16:30:53.263
------------------------------

Summarizing 1 Failure:
  [FAIL] vrf plugin [It] adds the interface and custom routing to new VRF
  containernetworking/plugins/plugins/meta/vrf/vrf_test.go:269

Ran 1 of 1 Specs in 0.633 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestVRF (0.63s)
FAIL
FAIL	github.com/containernetworking/plugins/plugins/meta/vrf	0.649s
FAIL

I am still investigating the issue. Adding a 100ms sleep time after this (plugins/meta/vrf/vrf.go#L130-L144) solves the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant