Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single Zenodo record reference #195

Merged
merged 11 commits into from
Nov 2, 2023
Merged

Single Zenodo record reference #195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
565fdac
__init__: add zenodo_record var
jeffjennings Nov 1, 2023
089386c
conftest: update zenodo record ref
jeffjennings Nov 1, 2023
27158de
download_external_files: update zenodo record ref
jeffjennings Nov 1, 2023
3437434
crossvalidation tutorial: update zenodo record ref
jeffjennings Nov 1, 2023
20ad0e9
fakedata tutorial: update zenodo record ref
jeffjennings Nov 1, 2023
4d9882f
gridder tutorial: update zenodo record ref
jeffjennings Nov 1, 2023
5f13770
dirtyimage tutorial: update zenodo record ref
jeffjennings Nov 1, 2023
39ecda5
loose-vistutorial: update zenodo record ref
jeffjennings Nov 1, 2023
db6225e
optimization tutorial: update zenodo record ref
jeffjennings Nov 1, 2023
b271778
reference __init__ in tests workflow
jeffjennings Nov 1, 2023
55aad3b
tests.yml: test local install
jeffjennings Nov 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,11 @@ jobs:
with:
python-version: 3.8
- name: Install dependencies needed to download files
# we're just installing mpol here to reference the zenodo record number
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why the zenodo reference number is needed at this point in the tests? Is it for the later download_external_files.py stage? I think this is a fine solution, but if you really wanted to defer installation of the package until the next stage of the workflow, you could modify the download files to take a path to a zenodo reference, and you would have a script that reads that path from the init.py source file (something similar to the single source Python version but this is probably overkill).

Copy link
Contributor Author

@jeffjennings jeffjennings Nov 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's right, the reference number is needed for download_external_files.py to reference. It's a little clunky, and your reference is a good one. But if you're ok with a bit of redundancy at this point in the workflow, I'm happy to keep the current workaround.

# in __init__. below we'll reinstall for the tests.
run: |
pip install astropy
pip install .
- name: Cache/Restore the .mpol folder cache
uses: actions/cache@v3
env:
Expand Down
5 changes: 3 additions & 2 deletions docs/ci-tutorials/crossvalidation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,11 @@ from mpol import (
losses,
precomposed,
)
from mpol.__init__ import zenodo_record

# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down Expand Up @@ -361,7 +362,7 @@ For the purposes of comparison, here is the image produced by the tclean algorit

```{code-cell}
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.tclean.fits",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.tclean.fits",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
3 changes: 2 additions & 1 deletion docs/ci-tutorials/fakedata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,10 +271,11 @@ Therefore, we always recommend generating fake data using $u,v$ distributions fr

```{code-cell} ipython3
from astropy.utils.data import download_file
from mpol.__init__ import zenodo_record

# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
3 changes: 2 additions & 1 deletion docs/ci-tutorials/gridder.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,13 @@ It's important to remember that MPoL follows the standard baseline convention as
import matplotlib.pyplot as plt
import numpy as np
from astropy.utils.data import download_file
from mpol.__init__ import zenodo_record
```

```{code-cell}
# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
3 changes: 2 additions & 1 deletion docs/ci-tutorials/initializedirtyimage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,15 @@ from astropy.utils.data import download_file

```{code-cell}
from mpol import coordinates, gridding, losses, precomposed, utils
from mpol.__init__ import zenodo_record
```

When saving and loading a model, it is important to make sure that ``cell_size``, ``nchan``, and ``npix`` remain the same. More info on coordinates can be found in {class}`mpol.coordinates.GridCoords`.

```{code-cell}
# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
3 changes: 2 additions & 1 deletion docs/ci-tutorials/loose-visibilities.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,15 @@ and the relevant MPoL modules

```{code-cell}
from mpol import coordinates, gridding, losses, precomposed, utils, images, fourier
from mpol.__init__ import zenodo_record
```

and loading the dataset

```{code-cell}
# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
3 changes: 2 additions & 1 deletion docs/ci-tutorials/optimization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,13 @@ from IPython.display import SVG, display

```{code-cell} ipython3
from mpol import coordinates, fourier, gridding, losses, precomposed, utils
from mpol.__init__ import zenodo_record
```

```{code-cell} ipython3
# load the mock dataset of the ALMA logo
fname = download_file(
"https://zenodo.org/record/10059491/files/logo_cube.noise.npz",
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
show_progress=True,
pkgname="mpol",
Expand Down
4 changes: 2 additions & 2 deletions docs/download_external_files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from astropy.utils.data import download_file
from mpol.__init__ import zenodo_record

version = 10059491
slug = "https://zenodo.org/record/{:d}/files/{:}"

fnames = [
Expand All @@ -10,7 +10,7 @@
]

for fname in fnames:
url = slug.format(version, fname)
url = slug.format(zenodo_record, fname)
download_file(
url,
cache=True,
Expand Down
1 change: 1 addition & 0 deletions src/mpol/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
__version__ = "0.1.13"
zenodo_record = 10064221
5 changes: 2 additions & 3 deletions test/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
from astropy.utils.data import download_file

from mpol import coordinates, gridding

zenodo_version = 10059491
from mpol.__init__ import zenodo_record

# We need a fixture which provides mock visibilities of the sort we'd
# expect from visread, but *without* the CASA dependency.
Expand All @@ -14,7 +13,7 @@
def mock_visibility_archive():
# use astropy routines to cache data
fname = download_file(
"https://zenodo.org/record/{:d}/files/logo_cube.noise.npz".format(zenodo_version),
f"https://zenodo.org/record/{zenodo_record}/files/logo_cube.noise.npz",
cache=True,
pkgname="mpol",
)
Expand Down