Skip to content

feat(netcdf): support creating and loading model netcdf input files #2417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 15 commits into
base: develop
Choose a base branch
from

Conversation

mjreno
Copy link
Contributor

@mjreno mjreno commented Jan 13, 2025

Initial support for NetCDF model utility classes:

  • currently focused on input arrays (no timeseries)
  • dependent on FloPy discretization classes
  • support creating, loading and modifying grid associated array data
  • create or load "layered mesh" (structured(DIS)/vertex(DISV)) or "structured" (structured(DIS)) format types
  • creates files suitable as MODFLOW 6 inputs when dataset is associated with a MODFLOW 6 model
  • integrated into mf6 core module: reads and writes MODFLOW 6 input files that follow NETCDF variable formats as described in the mf6io guide

@mjreno mjreno marked this pull request as draft January 13, 2025 19:13
Copy link

codecov bot commented Jan 13, 2025

Codecov Report

Attention: Patch coverage is 90.04093% with 73 lines in your changes missing coverage. Please review.

Project coverage is 76.3%. Comparing base (7eb7c88) to head (95bcd97).
Report is 9 commits behind head on develop.

Files with missing lines Patch % Lines
flopy/utils/model_netcdf.py 89.2% 65 Missing ⚠️
flopy/mf6/data/mfdataarray.py 92.0% 2 Missing ⚠️
flopy/mf6/mfmodel.py 94.8% 2 Missing ⚠️
flopy/mf6/mfpackage.py 83.3% 2 Missing ⚠️
flopy/mf6/data/mffileaccess.py 94.7% 1 Missing ⚠️
flopy/mf6/utils/codegen/filters.py 0.0% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           develop   #2417     +/-   ##
=========================================
+ Coverage     76.1%   76.3%   +0.2%     
=========================================
  Files          296     297      +1     
  Lines        64215   64934    +719     
=========================================
+ Hits         48885   49564    +679     
- Misses       15330   15370     +40     
Files with missing lines Coverage Δ
flopy/mf6/data/mfdataplist.py 79.8% <100.0%> (+<0.1%) ⬆️
flopy/mf6/data/mfdatastorage.py 75.7% <100.0%> (+0.3%) ⬆️
flopy/mf6/data/mfstructure.py 74.0% <100.0%> (+0.2%) ⬆️
flopy/mf6/mfsimbase.py 75.6% <100.0%> (ø)
flopy/mf6/data/mffileaccess.py 76.9% <94.7%> (+0.3%) ⬆️
flopy/mf6/utils/codegen/filters.py 0.0% <0.0%> (ø)
flopy/mf6/data/mfdataarray.py 68.4% <92.0%> (+1.2%) ⬆️
flopy/mf6/mfmodel.py 84.4% <94.8%> (+0.4%) ⬆️
flopy/mf6/mfpackage.py 82.1% <83.3%> (+0.1%) ⬆️
flopy/utils/model_netcdf.py 89.2% <89.2%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mjreno mjreno force-pushed the create_load_netcdf branch 2 times, most recently from afb01ba to a29363e Compare January 31, 2025 18:29
@mjreno mjreno force-pushed the create_load_netcdf branch from 1f7ae2f to f280aa7 Compare February 4, 2025 16:07
@mjreno mjreno force-pushed the create_load_netcdf branch from e536392 to 4187da8 Compare February 11, 2025 17:39
@mjreno mjreno force-pushed the create_load_netcdf branch 2 times, most recently from e966728 to 8cec938 Compare February 21, 2025 21:13
@mjreno mjreno force-pushed the create_load_netcdf branch from 8cec938 to 2ddc04b Compare April 30, 2025 17:47
@mjreno mjreno force-pushed the create_load_netcdf branch from 2ddc04b to 4c1e0ff Compare April 30, 2025 17:50
"""
Open an existing dataset. Assumes the dataset has been annotated
with the necessary attributes to read and update, including global
attributes modflow_model and modflow_grid.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a specification of the necessary annotations (modflow_model, modflow_grid, etc) somewhere? flopy4 will need to know about them too, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically our own convention about what's expected in a netcdf dataset, on top of whichever other conventions we may use internally, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The annotations have been in flux with the dynamic (timeseries) input work ongoing in MODFLOW 6 now (GHBA, etc). After that PR goes in (or with it), documentation will be updated to formalize and share these annotations. The first place might be the MODFLOW 6 wiki but could also be mf6io/markdown files, etc. And yes, these are are own annotations, the most important of which are associated with individual variables, e.g. modflow_input which represents the mempath. These annotations identify the file as a valid input source for MODFLOW 6.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as alternative approaches- I would say that sort of approach is consistent with pyphoenix and maybe would help with integration or a cutover. xarray itself wouldn't be sufficient though as this utility also supports creating UGRID compliant inputs- we would have needed to also use xugrid or something similar. At the time, it was unclear what the path would be in pyphoenix, and I took what I thought was an Flopy3 consistent approach. Maybe we should talk about this more before it is merged- I'll need to update some tests first at any rate to bring what it there up to level with where MODFLOW 6 is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok- so we'll have to use xugrid objects. That makes sense.

I dunno if functions are the best way. It doesn't really matter if they stand alone or live on a class. I just feel like we should be wary of redefining a data model, and try to lean on xarray/xugrid/etc as much as possible. I'd say as soon as you get the tests up to date it can come in and we can refactor as we go?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that sounds good to me, updates will be needed soon for timeseries data and then whatever refactoring makes sense.

Copy link
Member

@wpbonelli wpbonelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this provides the functionality we need for now, I say bring it in.

But I wonder about introducing a new class hierarchy. Could we instead have factory functions creating xr.Dataset? What do we get here by wrapping a custom class around a dataset?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants