Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop a better testing strategy #55

Open
wkearn opened this issue Jul 26, 2024 · 0 comments
Open

Develop a better testing strategy #55

wkearn opened this issue Jul 26, 2024 · 0 comments

Comments

@wkearn
Copy link
Contributor

wkearn commented Jul 26, 2024

I like the property-based testing approach that we have in libtopotoolbox, but it doesn't make sense to verify all of these properties in pytopotoolbox. It's slow, and we know that libtopotoolbox passes its tests. On the other hand, we want to ensure that issues like #50 don't happen. More generally, we want to make sure that things that used to work don't suddenly stop working.

A kind of snapshot testing could work: generate and save input-output pairs that we know pass the libtopotoolbox tests. We can do that directly from the libtopotoolbox tests or by verifying the properties slowly with pytopotoolbox. Then the pytopotoolbox tests can just read the input, run the appropriate functions and compare against the expected output.

The hard part of this is how to save the input and output data so that we don't have to generate and verify it repeatedly. Maybe we can add the test data to the TopoToolbox/DEMs repository? Or create a separate test data repository? If we do that, then we can easily access the test data from any of the packages, and we wouldn't have to worry about committing a bunch of DEMs and test outputs to the pytopotoolbox repo.

This would also be a good way to ensure that functions that get implemented in libtopotoolbox match the expected behavior of the existing MATLAB code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant