get TAV tests passing again #1860
Comments
This tests just the latest in each minor version. Refs: open-telemetry#1860
|
More details on that instrumentations' tests failed in the run linked above:
The The For Node 18, see above. I think the |
[email protected] was a bad release that accidentally broken v14 support. This updates the versions test to skip that release, and also reduces the number of versions tested from 27 to 8 by selecting just the current latest minor releases. Also, drop the pretest config that (a) was never used because of the wrong indentation and (b) is no longer valid since the move to npm workspaces. Refs: open-telemetry#1860
The beforeEach() hook was not awaited, so afterEach() could run before it completed, resulting in a client.disconnect() that rejects, and a mocha hook that calls done() twice. Refs: open-telemetry#1860
[email protected] was a bad release that accidentally broken v14 support. This updates the versions test to skip that release, and also reduces the number of versions tested from 27 to 8 by selecting just the current latest minor releases. Also, drop the pretest config that (a) was never used because of the wrong indentation and (b) is no longer valid since the move to npm workspaces. Refs: #1860
The beforeEach() hook was not awaited, so afterEach() could run before it completed, resulting in a client.disconnect() that rejects, and a mocha hook that calls done() twice. Refs: #1860 Co-authored-by: Marc Pichler <[email protected]>
This tests just the latest in each minor version. Refs: #1860 Co-authored-by: Marc Pichler <[email protected]>
|
After those changes ^^, there are still two failures in the TAV tests with Node v18. |
Actually, I mean open-telemetry/opentelemetry-js#4357 which was the update to [email protected] which added support for Node v18.19.0. There have been more udpates to import-in-the-middle that I hadn't yet seen. |
|
Current status update (also updated in the description):
Those issues cover all the current TAV failures that I know about. They are better to follow up on. I don't think there is much more value in this issue, so I'll close it now. |
This is a working issue to attempt to get test-all-versions (TAV) tests passing again. They had been failing for a long time due to #1477 and other issues.
current status
https://github.com/open-telemetry/opentelemetry-js-contrib/actions/runs/7174025293/job/19534639796
is a run after some aws-sdk and cucumber TAV fixes were merged.
The Node 16 run actually passed! The jobs for Node.js 14 and 18 failed. I don't yet know why.
They all took ~1h40m.
Update (2023-12-20):
redis@4release, so is exactly the kind of issue that TAV is being used for.timings from logs
I downloaded the logs archive (122MB) and unzipped to "~/Downloads/tav-pass".
This grep roughly shows the runtimes and failures for each TAV module:
mongoose(30 minutes), and maybemongodb(12 minutes) are the best candidates for improvement.Timings details
failures
This section lists each of the failures.
Tip: Can search for
-- fatalin the TAV log output to find the failures.fastify
Fastify TAV failure
pino
pino TAV failure
redis-4
redis-4 TAV failure
num versions
Here is a command that shows the number of versions of each module being tested by each of the ".tav.yml" files. In this example
ttav2is an alias to this change totest-all-versionsto add a dry-run command and include the number of versions in verbose output: watson/test-all-versions#12Highlights:
3. mongoose is teseting 204 versions. Let's reduce that.
4. mongodb is testing 88 versions. Let's see if we can reduce that.
5. other candidates for reducing the number of tested versions: cucumber, lru-memoizer, tedious, mysql2, nestjs-core, and redis-4
command output showing number of module versions
The text was updated successfully, but these errors were encountered: