Reduce execution time of MAST remote test suite #3036
Conversation
snbianco
changed the title
There are a couple of old ones, like e.g. #2505, but also please open new ones for specific issues or in any way that you find useful. You can search for the Nothing is really urgent, as you see some of the issues are open for year now. I just got super excited to see renewed activity and jumped on a few issues that can make the generic maintainer life easier (e.g. this one). |
|
@pllim - please add |
|
Added. Hope it works! 🤞 |
There was a problem hiding this comment.
Thank you so much, this is a great improvement.
However, locally I run into timeouts for the last 5 tests. However, I see the same timeouts on the current main, so would go ahead merging this PR as is as leave digging into those timeouts for a follow-up.
=============================================== short test summary info ================================================
FAILED astroquery/mast/tests/test_mast_remote.py::TestMast::test_zcut_get_surveys - requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://mast.stsci.edu/zcut/api/v0.1/survey
FAILED astroquery/mast/tests/test_mast_remote.py::TestMast::test_zcut_download_cutouts - requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://mast.stsci.edu/zcut/api/v0.1/ast...
FAILED astroquery/mast/tests/test_mast_remote.py::TestMast::test_zcut_get_cutouts - requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://mast.stsci.edu/zcut/api/v0.1/ast...
FAILED astroquery/mast/tests/test_mast_remote.py::TestMast::test_hapcut_download_cutouts - requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://mast.stsci.edu/hapcut/api/v...
FAILED astroquery/mast/tests/test_mast_remote.py::TestMast::test_hapcut_get_cutouts - requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://mast.stsci.edu/hapcut/api/v...
================================ 5 failed, 96 passed, 42 warnings in 1328.38s (0:22:08) ================================
| @@ -87,6 +87,7 @@ mast | |||
| ^^^^ | |||
|
|
|||
| - Fix bug in which the ``local_path`` parameter for the ``mast.observations.download_file`` method does not accept a directory. [#3016] | |||
| - Optimize remote test suite to improve performance and reduce execution time. [#3036] | |||
There was a problem hiding this comment.
General rule of thumb: we do changelog for user facing changes, so no need for one for 1) doc changes 2) test changes 3) internal refactorings that has no consequence to the user facing api
There was a problem hiding this comment.
I'll remove this at release time.
Closes #2209
@bsipocz @astrojimig
Although the issue only mentions one particular test, I ended up making a more changes to the entire suite to reduce running time. I got it down to around 6 minutes on my machine. The bottlenecks are still in the tests that download files, but I think this is a large improvement.
I will look for (or make) a separate issue about the documentation tests. How urgent would this be?