You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The performance test stability seems to be somewhat improved by warming up the endpoints. However, they can still be inconsistent and lead to flaky tests.
Determine a way to make the tests less flaky or perhaps even find an alternative for what we're trying to accomplish with having the tests around (which is at least to prevent a performance degradation when making changes).
Another promising option might be to spread the tests out over a longer period of time. Right now all the tests are fairly short mainly to keep the build time low. However, spreading them out over a longer period of time could even out the results.
I think right now the performance tests are still on the quick side so we have some leeway to make them longer without it being much of an impact. It might also be an option to remove them from the main build and have them run nightly if they are going to run for an extended period of time.
The text was updated successfully, but these errors were encountered:
The performance test stability seems to be somewhat improved by warming up the endpoints. However, they can still be inconsistent and lead to flaky tests.
Determine a way to make the tests less flaky or perhaps even find an alternative for what we're trying to accomplish with having the tests around (which is at least to prevent a performance degradation when making changes).
Another promising option might be to spread the tests out over a longer period of time. Right now all the tests are fairly short mainly to keep the build time low. However, spreading them out over a longer period of time could even out the results.
I think right now the performance tests are still on the quick side so we have some leeway to make them longer without it being much of an impact. It might also be an option to remove them from the main build and have them run nightly if they are going to run for an extended period of time.
The text was updated successfully, but these errors were encountered: