Add parallel tests #192
Add parallel tests #192
Conversation
There was a problem hiding this comment.
Look good, great changes! Added some small comments. For the CI we may modify run-tests-linux.sh and run-tests-mac.sh files. to run the parallel tests as well
| # if tests are parallel we only do this for the begin tests as we won't be dropping the database until the end | ||
| # begin will handle initialization specific to the tests but expects the database already exists | ||
| if [ "$PARALLEL" -eq 0 ]; then | ||
| psql "$@" -U ${DB_USER} -d postgres -v ECHO=none -q -c "DROP DATABASE IF EXISTS ${TEST_CASE_DB};" 2>/dev/null |
There was a problem hiding this comment.
should this not happen in both branches of the if?
| if [ "$PARALLEL" -eq 0 ]; then | ||
| psql "$@" -U ${DB_USER} -d postgres -v ECHO=none -q -c "DROP DATABASE IF EXISTS ${TEST_CASE_DB};" 2>/dev/null | ||
| db_init | ||
| psql "$@" -U ${DB_USER} -d ${TEST_CASE_DB} -v ECHO=none -q -f utils/common.sql 2>/dev/null |
There was a problem hiding this comment.
should this not happen in both branches as well?
There was a problem hiding this comment.
good catch, I'll rework this
| else | ||
| TEST_FILES=$(cat schedule.txt | grep '^test:' | sed -e 's/^test://' | tr " " "\n" | sed -e '/^$/d') | ||
| if [[ "$pgvector_installed" == "1" ]]; then | ||
| TEST_FILES=$(cat $SCHEDULE | grep -E '^(test:|test_pgvector:)' | sed -E -e 's/^test:|test_pgvector://' | tr " " "\n" | sed -e '/^$/d') |
There was a problem hiding this comment.
Unrelated to this PR, but still important: Do we currently run pgvector compat tests anywhere in ci/cd or release pipeline?
There was a problem hiding this comment.
Yes we are running them in the pipeline, as we're installing the pgvector there
| @@ -0,0 +1,11 @@ | |||
| CREATE OR REPLACE FUNCTION random_int_array(dim integer, min integer, max integer) RETURNS integer[] AS $BODY$ | |||
There was a problem hiding this comment.
I feel like this would be useful in regular tests as well. Should be in the utils for regular tests.
… standard tests, cleanup logic in test runner
|
merged the two branches in the test runner and made copies of a couple utilities |
Uses pg_regress to run tests in parallel against the database. Allows custom DB initialization and finalization which can be used to load relevant data in the beginning and check relevant invariants in the end
Uses pg_regress to run tests in parallel against the database. Allows custom DB initialization and finalization which can be used to load relevant data in the beginning and check relevant invariants in the end
Uses pg_regress to run tests in parallel against the database. Allows custom DB initialization and finalization which can be used to load relevant data in the beginning and check relevant invariants in the end
This PR adds support for a set of parallel tests to ensure that lantern behaves properly under a more realistic workload. At the moment they are not especially interesting, just a PoC, but this includes the changes to support the concept more generally. I have for the time being added these tests to
make testto see how they'll interact with CI on this PR. They should eventually be restricted tomake test-parallel@var77 if you have ideas about how best to include them in the github actions workflow any advice would be awesome.Some notes:
make test-parallel(currently they'll also run if you runmake testbut I'd like to change this before mergingtest/parallel_schedule.txttest/parallelthey have the same layout as the existing regression testsFixes #177