Fix flaky playwright tests (#28975) #4078
Annotations
4 errors, 5 warnings, and 1 notice
Run Playwright tests:
playwright/e2e/oidc/index.ts#L34
1) [Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery
Error: expect(received).toHaveLength(expected)
Expected length: 1
Received length: 0
Received array: []
Call Log:
- Test timeout of 30000ms exceeded
at oidc/index.ts:34
32 | messages = await mailhog.messages();
33 | expect(messages.items).toHaveLength(1);
> 34 | }).toPass();
| ^
35 | expect(messages.items[0].to).toEqual(`${username} <${email}>`);
36 | const [, code] = messages.items[0].text.match(/Your verification code to confirm this email address is: (\d{6})/);
37 |
at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:34:8)
at /home/runner/work/element-web/element-web/playwright/e2e/crypto/backups-mas.spec.ts:34:9
|
Run Playwright tests:
playwright/e2e/oidc/index.ts#L34
2) [Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere
Error: expect(received).toHaveLength(expected)
Expected length: 1
Received length: 0
Received array: []
Call Log:
- Test timeout of 30000ms exceeded
at oidc/index.ts:34
32 | messages = await mailhog.messages();
33 | expect(messages.items).toHaveLength(1);
> 34 | }).toPass();
| ^
35 | expect(messages.items[0].to).toEqual(`${username} <${email}>`);
36 | const [, code] = messages.items[0].text.match(/Your verification code to confirm this email address is: (\d{6})/);
37 |
at registerAccountMas (/home/runner/work/element-web/element-web/playwright/e2e/oidc/index.ts:34:8)
at /home/runner/work/element-web/element-web/playwright/e2e/crypto/backups-mas.spec.ts:58:9
|
Run Playwright tests:
playwright/e2e/login/utils.ts#L37
3) [Chrome] › login/soft_logout_oauth.spec.ts:40:9 › Soft logout with SSO user › shows the soft-logout page when a request fails, and allows a re-login
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByRole('heading', { name: 'Create your account' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByRole('heading', { name: 'Create your account' })
at login/utils.ts:37
35 |
36 | // Synapse prompts us to pick a user ID
> 37 | await expect(page.getByRole("heading", { name: "Create your account" })).toBeVisible();
| ^
38 | await page.getByRole("textbox", { name: "Username (required)" }).fill("alice");
39 |
40 | // wait for username validation to start, and complete
at doTokenRegistration (/home/runner/work/element-web/element-web/playwright/e2e/login/utils.ts:37:78)
at Object.user (/home/runner/work/element-web/element-web/playwright/e2e/login/soft_logout_oauth.spec.ts:30:26)
|
Run Playwright tests:
playwright/e2e/timeline/timeline.spec.ts#L624
4) [Chrome] › timeline/timeline.spec.ts:589:13 › Timeline › message displaying › should set inline start padding to a hidden event line @screenshot
Error: expect(locator).toHaveScreenshot(expected)
472385 pixels (ratio 0.77 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/timeline/timeline.spec.ts/hidden-event-line-zero-padding-irc-layout-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/timeline-timeline-Timeline-08b5e-ding-to-a-hidden-event-line-Chrome/hidden-event-line-zero-padding-irc-layout-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/timeline-timeline-Timeline-08b5e-ding-to-a-hidden-event-line-Chrome/hidden-event-line-zero-padding-irc-layout-diff.png
Call log:
- expect.toHaveScreenshot(hidden-event-line-zero-padding-irc-layout.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for locator('.mx_MainSplit')
- locator resolved to <div class="mx_MainSplit">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 472383 pixels (ratio 0.77 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for locator('.mx_MainSplit')
- locator resolved to <div class="mx_MainSplit">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 472385 pixels (ratio 0.77 of all image pixels) are different.
622 | };
623 |
> 624 | await expect(page.locator(".mx_MainSplit")).toMatchScreenshot(
| ^
625 | "hidden-event-line-zero-padding-irc-layout.png",
626 | screenshotOptions,
627 | );
at /home/runner/work/element-web/element-web/playwright/e2e/timeline/timeline.spec.ts:624:61
|
Run Playwright tests:
[Chrome] › login/login-consent.spec.ts#L1
[Chrome] › login/login-consent.spec.ts took 55.8s
|
Run Playwright tests:
[Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 30.9s
|
Run Playwright tests:
[Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 23.7s
|
Run Playwright tests:
[Chrome] › app-loading/guest-registration.spec.ts#L1
[Chrome] › app-loading/guest-registration.spec.ts took 23.3s
|
Run Playwright tests:
[Chrome] › register/register.spec.ts#L1
[Chrome] › register/register.spec.ts took 16.2s
|
Run Playwright tests
4 flaky
[Chrome] › crypto/backups-mas.spec.ts:31:9 › Encryption state after registration › user is prompted to set up recovery
[Chrome] › crypto/backups-mas.spec.ts:48:9 › Key backup reset from elsewhere › Key backup is disabled when reset from elsewhere
[Chrome] › login/soft_logout_oauth.spec.ts:40:9 › Soft logout with SSO user › shows the soft-logout page when a request fails, and allows a re-login
[Chrome] › timeline/timeline.spec.ts:589:13 › Timeline › message displaying › should set inline start padding to a hidden event line @screenshot
67 passed (8.5m)
|
Loading