E2E Test System integration #369
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #369 +/- ##
=======================================
Coverage 98.54% 98.54%
=======================================
Files 67 67
Lines 2742 2742
Branches 148 148
=======================================
Hits 2702 2702
Misses 40 40 ☔ View full report in Codecov by Sentry. |
.circleci/config.yml
Outdated
| command: | | ||
| E2E_RESULT="{}" | ||
| E2E_STATUS="running" | ||
| while [[ $E2E_STATUS != "failed" && $E2E_STATUS != "canceled" && $E2E_STATUS != "success" ]] |
There was a problem hiding this comment.
What are the other states 🤔
Any risk of an endless loop? or waiting for a timeout when the process could fail fast
There was a problem hiding this comment.
If you head to the CircleCI API ref and expand the response schema:
You bring up a good point: not just that this should probably handle all the states, but also one problem I encountered while testing this. If the token is incorrect, E2E_STATUS will end up being set to null - and that could result in an infinite loop. See for example this run.
.circleci/config.yml
Outdated
| E2E_STATUS="running" | ||
| while [[ $E2E_STATUS != "failed" && $E2E_STATUS != "canceled" && $E2E_STATUS != "success" ]] | ||
| do | ||
| sleep 10 |
There was a problem hiding this comment.
What is the reasoning behind 10?
Also adding a specifier suffix can make it clearer for new bash devs
| sleep 10 | |
| sleep 10s |
There was a problem hiding this comment.
Wow didn't know you could do that! Great suggestion. As for 10 seconds: no specific reason. Our tests take a minimum of 1min to run. We could probably bump this delay up to 30s without a huge impact to us and be slightly nicer to our friends at CircleCI.
…e more circleci job states.
Adds a CircleCI config to kick off end-to-end platform integration tests and report results back, passing in the branch ref of the deno-slack-sdk as a parameter into the tests.
This should execute on both PRs as well as on
main, and pass the correct branch in either case.