Skip to content

Remove the dependency of netCDF4 #3643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 29, 2025
Merged

Remove the dependency of netCDF4 #3643

merged 7 commits into from
Apr 29, 2025

Conversation

seisman
Copy link
Member

@seisman seisman commented Nov 22, 2024

Comment on lines 156 to 159
return load_dataarray(fname)
with Session() as lib:
with lib.virtualfile_out(kind="grid") as voutgrd:
lib.call_module(
module="read", args=["@static_earth_relief.nc", voutgrd, "-Tg"]
)
grid = lib.virtualfile_to_raster(vfname=voutgrd, kind="grid")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know what? Maybe we could think of refactoring pygmt.io.load_dataarray to use this with block? It would be a breaking change, but that could mean we can use GMT's I/O directly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, actually, I'm thinking if we should provide a wrapper for the special read module, which can read a table, grid, or image, and return pd.DataFrame or xarray.DataArray. The function looks like:

def read(file, kind=None, **kwargs):
    if kind == None:
        # Try our best to decide the data kind. Or make 'kind' a required parameter
        pass 

    code = {"dataset": "d", "grid": "g", "image": "i"}[kind]
    func = 

    with Session() as lib:
        with lib.virtualfile_out(kind=kind) as voutfile:
            lib.call_module(
                module="read", args=[file, voutfile, f"-T{code}"]
            )
            if kind == "dataset":
                return lib.virtualfile_to_dataset(vfname=voutfile, **kwargs)
            elif kind in {"grid", "image"}
                return lib.virtualfile_to_raster(vfname=voutgrd, kind=kind)

For comparison, GMT.jl provides the gmtread/gmtwrite functions, but As I understand it, the functions return the GMT_DATASET/GMT_GRID/GMT_IMAGE containers, rather than data structures in the Julia world.

@seisman seisman changed the base branch from main to feature/read December 9, 2024 10:10
@seisman seisman changed the base branch from feature/read to main December 9, 2024 11:12
@seisman seisman changed the title Remove netCDF4 to see how if affects our tests Remove the dependency of netCDF4 Dec 9, 2024
@seisman seisman mentioned this pull request Dec 19, 2024
49 tasks
@weiji14 weiji14 added this to the 0.15.0 milestone Dec 19, 2024
@weiji14 weiji14 mentioned this pull request Dec 30, 2024
9 tasks
@seisman seisman removed this from the 0.15.0 milestone Dec 31, 2024
@seisman seisman added the maintenance Boring but important stuff for the core devs label Feb 18, 2025
@seisman seisman changed the base branch from main to feature/read April 16, 2025 09:16
@seisman seisman changed the base branch from feature/read to backendentrypoint/gmtread April 18, 2025 09:29
Base automatically changed from backendentrypoint/gmtread to main April 26, 2025 01:21
@weiji14
Copy link
Member

weiji14 commented Apr 29, 2025

Not sure if removing a required dependency counts as a breaking change (i.e. if we should should we put **Breaking** in the title), the main non-test-related user-facing change was to _load_earth_relief_holes in pygmt/dataset/samples.py in #3922 which is quite minor. I suppose we could just label this under 'deprecation' instead of 'maintenance' to make it a bit more prominent in the changelog?

@seisman seisman marked this pull request as ready for review April 29, 2025 06:17
Copy link
Member

@weiji14 weiji14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a test failing on ubuntu latest - Python 3.12. Need to add a @pytest.mark.skipif(condition=not _HAS_NETCDF4, reason="netCDF4 is not installed") marker for this line:

grid.to_netcdf(tmpfile.name)

because .to_netcdf() still relies on the netCDF4 engine.

@seisman seisman added deprecation Deprecating a feature needs review This PR has higher priority and needs review. and removed maintenance Boring but important stuff for the core devs labels Apr 29, 2025
Copy link
Member

@weiji14 weiji14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, glad to have one less dependency to handle! Remember to remove netCDF4 from the conda-forge feedstock for the next release 🔥

@seisman seisman merged commit 7741d06 into main Apr 29, 2025
29 checks passed
@seisman seisman deleted the dep/netcdf branch April 29, 2025 10:05
@seisman seisman removed the needs review This PR has higher priority and needs review. label Apr 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deprecation Deprecating a feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants