Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running tests in parallel has unexpected results #4625

Open
Su3h7aM opened this issue Dec 24, 2024 · 0 comments
Open

Running tests in parallel has unexpected results #4625

Su3h7aM opened this issue Dec 24, 2024 · 0 comments

Comments

@Su3h7aM
Copy link
Contributor

Su3h7aM commented Dec 24, 2024

Context

I am creating a neotest adapter to run Odin tests in neovim.
Using neotest it is possible to run individual tests, by file or by directory, when I run the tests by file I cannot use the -file flag because some tests may depend on other files, so I am using the -define:ODIN_TEST_NAMES=main.test_part01 flag, I also cannot use -define:ODIN_TEST_NAMES=main.test_part01,main.test_part02 because the tests need to run separately.
At this point I encountered a problem, because neotest runs the tests in parallel, and the odin test command does not behave as it should.

To reproduce

#!/bin/bash

odin test 2024/odin/day02 \
    -define:ODIN_TEST_LOG_LEVEL=warning \
    -define:ODIN_TEST_TRACK_MEMORY=false \
    -define:ODIN_TEST_FANCY=false \
    -define:ODIN_TEST_NAMES=main.test_part01 &

odin test 2024/odin/day02 \
    -define:ODIN_TEST_LOG_LEVEL=warning \
    -define:ODIN_TEST_TRACK_MEMORY=false \
    -define:ODIN_TEST_FANCY=false \
    -define:ODIN_TEST_NAMES=main.test_part02 &

wait

Expected Behavior

Only one of the tests must pass

Finished 1 test in 160.542µs. The test was successful.

[ERROR] --- [2024-12-24 18:57:29] [main_test.odin:34:test_part02()] Part Two: 4

Finished 1 test in 186.031µs. The test failed.
 - main.test_part02     Part Two: 4

Current Behavior

Only one of the tests runs in both cases, sometimes both pass and sometimes both fail.

Finished 1 test in 160.542µs. The test was successful.

Finished 1 test in 183.505µs. The test was successful.
[ERROR] --- [2024-12-24 18:57:29] [main_test.odin:34:test_part02()] Part Two: 4

Finished 1 test in 202.932µs. The test failed.
 - main.test_part02     Part Two: 4

[ERROR] --- [2024-12-24 18:57:29] [main_test.odin:34:test_part02()] Part Two: 4

Finished 1 test in 186.031µs. The test failed.
 - main.test_part02     Part Two: 4
sh: line 1: /home/su3h7am/Projects/advent-of-code/day02: Text file busy

Operating System & Odin Version:

    Odin:    dev-2024-12:7be003557
    OS:      CachyOS, Linux 6.12.6-2-cachyos
    CPU:     AMD Ryzen 7 5700X 8-Core Processor
    RAM:     32017 MiB
    Backend: LLVM 18.1.8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant