Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace httpbin in integration tests with different application #661

Open
5 tasks
barchw opened this issue Mar 8, 2024 · 0 comments
Open
5 tasks

Replace httpbin in integration tests with different application #661

barchw opened this issue Mar 8, 2024 · 0 comments
Labels
area/ci Issues or PRs related to CI related topics area/tests Issues or PRs related to tests

Comments

@barchw
Copy link
Collaborator

barchw commented Mar 8, 2024

Description

Running httpbin as a target for integration and performance tests provides a huge overhead in terms of startup and response bottleneck. Replacing it with an application that could handle the traffic and startup better would allow us to get more meaningful results from performance tests, as well as decrease the time that integration tests take.

Reasons

  • Httpbin startup takes a long time
  • We don't use the full capabilities of httpbin application
  • Httpbin memory and cpu load hits maximum during performance tests and the autoscaling does not compensate for this, meaning that the tests don't test our setup limits, but httpbin limits instead

DoD:

  • Agree on a different application for target
  • Perfomance tests use new implementation:
    • New application is used
    • Make sure that mem and CPU aren't maxing out
  • Integration tests use new implementation

Follow up(s)

  • Consider setting up thresholds for performance tests and inform on internal slack when threshold is breached
  • Allow running more sophisticated scenarios (e.g. testing JWT scenarios with provider number > 50)
@barchw barchw added area/ci Issues or PRs related to CI related topics area/tests Issues or PRs related to tests labels Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci Issues or PRs related to CI related topics area/tests Issues or PRs related to tests
Projects
None yet
Development

No branches or pull requests

1 participant