Failures for awaited assertions are marked by Allure as Broken, rather than a Failure

[Maximus1285]

Using allure-playwright

Having awaited assertions like this:
await expect(page.getByTestId('locator')).toBeEnabled();

Playwright will attempt this expectation every certain millisec util it meets the expected behavior or it times out trying, in which case it would be considered by Playwright as a failed test. If if times out, but Allure treats it rather as a test defect and marks it as Broken.

I think this is a bug on how test results are interpreted for this kind of expectations and the package doesn't expose any api to override the criteria to categorize the different test results according to each scenario

[epszaw]

Hey, @Maximus1285! Usually, we mark tests as broken when something went wrong outside of test context (runtime errors, etc.). You're right, that we set BROKEN status for timed out tests, but I need some time to figure out, why we handle this in this way.

[Maximus1285]

@epszaw I found what the issue was. I figured both global timeout and expect timeout had the same value, so it was always defaulting to the global timeout.

I've decreased the expect timeout and now tests are successfully being marked as Failed
I think we can then close this issue

[epszaw]

Good job here :)