-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add Ocean Gliders format output option to seaxexplorer processing #212
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #212 +/- ##
==========================================
- Coverage 55.11% 53.77% -1.35%
==========================================
Files 9 9
Lines 1700 1750 +50
==========================================
+ Hits 937 941 +4
- Misses 763 809 +46 ☔ View full report in Codecov by Sentry. |
A quick skim - I'm not clear that a lot of this cannot be handled via the yaml. For the rest, it would be nice to keep new variables like platform id in the yaml as well. I've not looked but perhaps that requires a new variable type that just accepts a constant value. Maybe we should have a quick call, but I would rather see a general approach rather than special "og" carve outs |
outname = outdir + id0 + fnamesuffix + '.nc' | ||
_log.info('writing %s', outname) | ||
ds.to_netcdf(outname, 'w', | ||
encoding={'time': {'units': 'seconds since 1970-01-01T00:00:00Z', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For example, 'time'/'TIME' could be replaced by a time-encoder yaml entry. eg calls to ds.time
could be replaced by ds[time_var_name]
or similar?
Not sure about 'latitude' etc, but why do they need to be translated back and forth? For distance over ground? Maybe just use inside that function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can localise lon/lat to just change within those functions, I'll make a new PR with ds[time_var_name]
and see what it looks like
f"sea{ds.attrs['glider_serial'].zfill(3)}", | ||
attrs={"long_name": "glider serial number"}, | ||
) | ||
ds["DEPLOYMENT_TIME"] = np.nanmin(ds.TIME.values) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The yamls have a "conversion" field for each variable that tries to fine the method in utils.py
. So this could look like
netcdf_variables:
DEPLOYMENT_TIME:
"long_name": "date of deployment"
...
"conversion": "nanmin_time"
with the accompanying change in utils.py
. We may want to think of a way to make that more general (np.nan_min, "DEPLOYMENT_TIME)
if yaml accepts tuples.
6651931 (via #210) introduced a bunch of formatting changes that will likely cause merge conflicts. Here are merge and rebase workflows to avoid these conflicts (it depends on having I recommend first creating a # merge all the way up to the commit that introduced the formatting changes
git merge 66519318^1
# merge the commit that introduced the formatting changes, prioritising the working branch during conflicts
git merge -s ours 66519318 --no-commit
pre-commit run --all-files
git commit --no-edit
# merge the rest of the way
git merge main or by doing a rebase (if you want a clean history) # Rebase up to the commit introducing formatting changes
git rebase 66519318^1
# Rebase onto the commit introducing formatting changes.
# Accept working branch changes (i.e., "theirs" during rebase
# - different to merge. See git rebase docs) during conflicts.
# During the rebase we will run the formatter after applying
# changes from commits, before committing so that everything is compliant.
git rebase -X theirs -i 66519318
# Replace all "pick" with "edit" or "e". Start the rebase...
# Repeatedly run
git add -u && pre-commit run --all-files && git rebase --continue
# to run pre-commit tooling, stage modified files, and continue
# until rebase is complete. Sometimes pre-commit can't autofix
# issues, then you need to fix them manually.
# If you make a mistake you can always do `git rebase --abort`
# Finally, rebase onto main
git rebase main |
I'll try a different approach following Jody's suggestion of a new yaml entry that makes empty variables/variables with one value |
This PR is under development and is waiting on several currently active proposed edits to the Ocean Gliders format before it can be finalised
This will resolve #21