Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pytest-based testing framework #143

Merged
merged 1 commit into from
Sep 2, 2024
Merged

Add pytest-based testing framework #143

merged 1 commit into from
Sep 2, 2024

Conversation

JelteF
Copy link
Collaborator

@JelteF JelteF commented Aug 21, 2024

pgregress is easy to use an is part of Postgres, so it's often the first thing to use testing a Postgres extension. But it also quite limiting, since you can only execute SQL and a test fails as soon as any of the output is different.

This adds a Python based testing framework to the project, which allows to do more "application" style testing. It's based on the testing framework I wrote for PgBouncer, but I removed a bunch of stuff that only made sense when testing PgBouncer. Postgres has a similar framework included, but that uses Perl and I really don't want to write any Perl. Also pytest is a much nicer to use testing framework than what is available in Perl.

As an example of a test that cannot easily be written using pgregress I added a test for EXPLAIN ANALYZE. Support for EXPLAIN ANALYZE was added in #136, but writing a test was not possible because the Timing information would be flaky.

@JelteF JelteF force-pushed the pytest-tests branch 14 times, most recently from afa2cee to e4876b1 Compare August 21, 2024 15:38
JelteF added a commit that referenced this pull request Aug 21, 2024
The PostgreSQL license is very liberal but, like MIT and BSD, it does
require attribution and requires that the original license is included.
This adds a NOTICE file containing the license, because we've vendored
in a few files/functions from the Postgres repository and we'll likely
need
to vendor in more.

If/when we decide to to merge #143 we should also include PgBouncer its
license. Which is the ISC license, which is also attribution-only.
@JelteF JelteF requested a review from mkaruza August 22, 2024 06:49
Copy link
Collaborator

@mkaruza mkaruza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind adding this for testing purposes - so approving. Only thing that i would like to understand why we need ruff.

@JelteF
Copy link
Collaborator Author

JelteF commented Sep 2, 2024

Ruff is for auto formatting

@JelteF JelteF merged commit db88897 into main Sep 2, 2024
3 checks passed
@JelteF JelteF deleted the pytest-tests branch September 2, 2024 14:15
mkaruza pushed a commit that referenced this pull request Sep 11, 2024
The PostgreSQL license is very liberal but, like MIT and BSD, it does
require attribution and requires that the original license is included.
This adds a NOTICE file containing the license, because we've vendored
in a few files/functions from the Postgres repository and we'll likely
need
to vendor in more.

If/when we decide to to merge #143 we should also include PgBouncer its
license. Which is the ISC license, which is also attribution-only.
mkaruza pushed a commit that referenced this pull request Sep 11, 2024
`pgregress` is easy to use an is part of Postgres, so it's often the
first thing to use testing a Postgres extension. But it also quite
limiting, since you can only execute SQL and a test fails as soon as any
of the output is different.

This adds a Python based testing framework to the project, which allows
to do more "application" style testing. It's based on the testing
framework I wrote for PgBouncer, but I removed a bunch of stuff that
only made sense when testing PgBouncer. Postgres has a similar framework
included, but that uses Perl and I really don't want to write any Perl.
Also `pytest` is a much nicer to use testing framework than what is
available in Perl.

As an example of a test that cannot easily be written using `pgregress`
I added a test for `EXPLAIN ANALYZE`. Support for `EXPLAIN ANALYZE` was
added in #136, but writing a test was not possible because the Timing
information would be flaky.
@Tishj
Copy link
Collaborator

Tishj commented Sep 18, 2024

I love this, pytest is great and really powerful for testing additional side-effects that would not be possible with the postgres regression tests

@Tishj
Copy link
Collaborator

Tishj commented Sep 18, 2024

This could also be used to make #88 testable, verifying that files are correctly downloaded, multiple processes can make use of it and not break etc..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants