CT: Utterance: alertMaximumDelay tests failed

[jessegreenberg]

This is intermittent on CT, occurs somewhat rarely.

Ill see if I can pin it down.

[jessegreenberg]

It seems to mostly fail when the phet-io query parameter is used.

[jessegreenberg]

Running the tests in isolation, the UtteranceQueue is not initialized. I don't think that is the problem on CT because only the last test is failing from "lertMaximumDelay tests". But I am not sure.

  await timeout( 150 );
  assert.ok( utteranceQueue[ 'queue' ].length === 0, 'not stable, but past max' );

[jessegreenberg]

The UtteranceQueue tests are hitting the announceImmediately, which is necessary for browser compatibility on the first utterance. But is is messing with the expectations of the UtteranceQueue timing tests. Ill disable that for these tests and see if that fixes it on CT.

The failures are intermittent.

If we can get ~10 columns passing Ill close this.

[jessegreenberg]

I am pretty sure that this is fixed and allowed #116 to come through. Closing.

[jessegreenberg]

Unfortunately I still see this one at SHA fec40dc on CT.

[pixelzoom]

This is still occurring in CT on 12/21/23 and 12/22/23. I would not call this intermittent - it occurs in 3 of the 10 more recent cycles.

100% of the failures that I currently see are in top-level-unit-tests, for example see below. This also makes me wonder how useful these unit tests are if they are not reproducible on every run.

utterance-queue : top-level-unit-tests : unbuilt?ea
http://128.138.93.172/continuous-testing/ct-snapshots/1703258473895/utterance-queue/utterance-queue-tests.html?ea
7 out of 8 tests passed. 1 failed.
Utterance: announceImmediately failed:
myUtterance announced immediately when Announcer was ready["This is my utterance text"]
at Object.<anonymous> (http://128.138.93.172/continuous-testing/ct-snapshots/1703258473895/chipper/dist/js/utterance-queue/js/UtteranceTests.js:176:10)

[URL] http://128.138.93.172/continuous-testing/aqua/html/qunit-test.html?url=..%2F..%2Fct-snapshots%2F1703258473895%2Futterance-queue%2Futterance-queue-tests.html%3Fea&testInfo=%7B%22test%22%3A%5B%22utterance-queue%22%2C%22top-level-unit-tests%22%2C%22unbuilt%3Fea%22%5D%2C%22snapshotName%22%3A%22snapshot-1703258473895%22%2C%22timestamp%22%3A1703258947951%7D
[NAVIGATED] http://128.138.93.172/continuous-testing/aqua/html/qunit-test.html?url=..%2F..%2Fct-snapshots%2F1703258473895%2Futterance-queue%2Futterance-queue-tests.html%3Fea&testInfo=%7B%22test%22%3A%5B%22utterance-queue%22%2C%22top-level-unit-tests%22%2C%22unbuilt%3Fea%22%5D%2C%22snapshotName%22%3A%22snapshot-1703258473895%22%2C%22timestamp%22%3A1703258947951%7D
[NAVIGATED] about:blank
[NAVIGATED] http://128.138.93.172/continuous-testing/ct-snapshots/1703258473895/utterance-queue/utterance-queue-tests.html?ea
[CONSOLE] Failed to load resource: the server responded with a status of 404 (Not Found): http://128.138.93.172/continuous-testing/ct-snapshots/1703258473895/utterance-queue/utterance-queue-strings_en.json
[CONSOLE] enabling assert

id: "Sparky Node Puppeteer"
Snapshot from 12/22/2023, 8:21:13 AM

[jessegreenberg]

Unfortunately, they only fail on CT/puppeteer and I have not been able to figure out why. One more shot in the above commit that is always passing locally but if its still no good on CT Ill probably remove these timing tests.