Skip to content

Commit

Permalink
Do not report error for test failures
Browse files Browse the repository at this point in the history
Test failures are not an exceptional event. By reporting them as though
they are exceptional, test262-harness forces consumers to ignore the
status code. This makes it difficult to detect true operational errors,
e.g. when the interpreter under test crashes, since they are
indistinguishable from test failures.

Consumers who wish to react to test failures should do so by parsing the
tool's output.
  • Loading branch information
jugglinmike authored and rwaldron committed May 25, 2021
1 parent f30b6b2 commit ae3a3f0
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 6 deletions.
3 changes: 0 additions & 3 deletions bin/run.js
Original file line number Diff line number Diff line change
Expand Up @@ -178,9 +178,6 @@ const results = zip(pool, tests).pipe(
);

const emitter = new ResultsEmitter(results);
emitter.on('fail', function () {
process.exitCode = 1;
});
reporter(emitter, reporterOpts);

function printVersion() {
Expand Down
4 changes: 1 addition & 3 deletions test/test.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ const tests = [
'--includesDir', './test/test-includes',
'./test/collateral/test/**/*.js',
],
{ exitCode: 1 },
],
[
[
Expand All @@ -48,7 +47,7 @@ const tests = [
'--includesDir', './test-includes',
'collateral/test/**/*.js',
],
{ cwd: 'test', exitCode: 1 },
{ cwd: 'test' },
],
[
[
Expand Down Expand Up @@ -117,7 +116,6 @@ const tests = [
'--reporter-keys', 'attrs,result,rawResult',
'./test/collateral-preprocessor/test/autofail.js',
],
{ exitCode: 1 },
],
].reduce((accum, a) => {
let b = a.slice();
Expand Down

0 comments on commit ae3a3f0

Please sign in to comment.