Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Continuous integration sometimes generates false "test failed" notifications #484

Open
hdwalters opened this issue Sep 16, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@hdwalters
Copy link
Contributor

I recently received a random "Cargo & Clippy Test" email on my glob function branch, directing me to https://github.com/amber-lang/amber/actions/runs/10872830349/job/30168293449. This complains about a failed unit test, which I haven't touched:

---- tests::stdlib::test_stdlib_src_tests_stdlib_abs_ab stdout ----
thread 'tests::stdlib::test_stdlib_src_tests_stdlib_abs_ab' panicked at src/tests/mod.rs:18:28:
assertion `left == right` failed
  left: ""
 right: "1\n1"

I do not see how this unit test could possibly fail, as it doesn't access the file system or anything else external:

$ cat src/tests/stdlib/abs.ab 
import * from "std/math"

// Output 
// 1
// 1

main {
    echo abs(1);
    echo abs(-1);
}
@hdwalters hdwalters added the bug Something isn't working label Sep 16, 2024
@b1ek
Copy link
Member

b1ek commented Sep 17, 2024

probably related to bash version

@hdwalters
Copy link
Contributor Author

probably related to bash version

If that were the problem, surely it would succeed or fail consistently? I would expect the CI server to do something like spinning up a Docker instance, to run tests in a consistent environment.

@Mte90
Copy link
Member

Mte90 commented Sep 23, 2024

So it is a bug that happens sometimes.
Re run the jobs fixes it, I think that #381 could fix it but that PR is blocked because there is an issue in escaping with bash generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants