Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What could be the reason for "mismatch between the number of scenarios" warning? #22

Open
Jazzyekim opened this issue Oct 28, 2020 · 1 comment

Comments

@Jazzyekim
Copy link

Jazzyekim commented Oct 28, 2020

Hello,

We execute hundreds of tests in 16 threads. Tests are fetched from Jira(Xray). If all the tests that about to be executed are linked to one story - everything is fine, all tests will be executed. When we run tests linked to two or more stories some of the tests might not be executed. It is always different tests and different number of tests. I noticed the following warning in serenity log:

There is a mismatch between the number of scenarios included in this test run (21) and the expected number of scenarios loaded (17). This suggests that the scenario filtering is not working correctly or feature file(s) of an unexpected structure are being run

What could be the cause of such warning? The feature files seems to be valid. I have checked source code and found that this warning is thrown when the number of filteredInScenarioCount and number of weightedCucumberScenarios is different. Could you please clarify what are these two entities represent?

@Jazzyekim
Copy link
Author

Have just tried to execute tests linked to only one story (it put all the tests into one feature file) and compared the feature file with the run where in addition some other tests were executed linked to another story ( two feature files were executed). It turned out that the feature files from these to runs are identical. So, it is not the problem with the structure of the file, but with the way how serenity handles splitting the tests from multiple feature files to run them in parallel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant