Skip to content

Latest commit

 

History

History
23 lines (13 loc) · 1.76 KB

DEBUGGING.md

File metadata and controls

23 lines (13 loc) · 1.76 KB
Raw

Debugging test262 errors in CI

The babel mono runs test262 on CircleCI. This documents aims to serve as a guide for babel maintainers to debug CI errors arising due to babel-test262-runner this repo.

Configuration

To see what the job is doing it's probably best to take a look at config.yml.

We run test262 tests using babel-test262-runner on babel monorepo. It is run on each PR (on manual trigger by babel maintainers) and also on master.

The tests are run on the master branch or PR. They are compared with the previous master branch test results. Only the changed results are rendered in the logs and CircleCI test results output.

Debugging a failing test

  1. When looking at a failing test, it's probably a good idea to keep in mind that certain tests in test262 could be intermittent failures.
  2. You can run the failing test locally by running babel-test262-runner:
    1. BABEL_PATH=/path/to/babel/repo node lib/run-tests <test-name>.

Known issues/Caveats

  • The test262 job compares against the previous master job. While this should not cause any issue, if someone were to manually trigger the master job for a relatively older commit, it will compare againt that. The way to resolve it would be to either land a new commit on master or rerun the master branch job for the most recent commit on master.

  • If there is an intermittent test, add it to intermittent-tests.txt. When you change this file, the next job will fail because the comparision check will fail. Best way to address this issue is to run the master job again to refresh the master artifact. Following jobs should pass.