test,tools: update wpt weekly #50567
Conversation
To keep the Web API behavior synchronized with the latest Web specs and WPT tests, add a GitHub Action to automatically update WPT fixtures with WPT weekly epoch. The status files are updated with the test results of the main branch as well.
|
Review requested:
|
nodejs-github-bot
added
build
There was a problem hiding this comment.
I am -1 on automated WPT fixture updates. They change the status json expectations without notice to the subsystem maintainers, making tests skipped and lowering our chance of finding regressions.
This could get bundled with automated issue creation for newly skipped tests but I think that would create too much noise resulting in unresolved and ignored bot issues anyway.
What about, instead of this, have the daily WPT job do something similar to what's done in nodejs/reliability and create / update a single issue with a list of unexpected passes & unexpected failures, that's effectively a link to this query
|
+1 to @panva's suggestion |
| - uses: gr2m/create-or-update-pull-request-action@77596e3166f328b24613f7082ab30bf2d93079d5 | ||
| # Creates a PR or update the Action's existing PR, or | ||
| # no-op if the base branch is already up-to-date. | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GH_USER_TOKEN }} | ||
| with: | ||
| author: Node.js GitHub Bot <[email protected]> | ||
| body: This is an automated patch update of wpt to https://github.com/web-platform-tests/wpt/commit/${{ env.WPT_REVISION }}. | ||
| branch: actions/update-wpt # Custom branch *just* for this Action. | ||
| commit-message: 'deps: update wpt to ${{ env.WPT_SHORT_REVISION }}' | ||
| labels: test | ||
| title: 'deps: update wpt to ${{ env.WPT_SHORT_REVISION }}' | ||
| update-pull-request-title-and-body: true |
There was a problem hiding this comment.
Could this log the failed tests in the PR body?
To keep the Web API behavior synchronized with the latest Web specs and
WPT tests, add a GitHub Action to automatically update WPT fixtures with
WPT weekly epoch. The status files are updated with the test results of
the main branch as well.
Example of automatic PR: legendecas#3