Skip to content

Feature/add preprocessing smoothing #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

alessandrofelder
Copy link
Collaborator

This PR is an attempt at incorporating preprocessing ("workflow steps B+C of the GEBCO cookbook") into remove-restore.py.
See #32 and #43 for details.

This PR mainly adds

  • a preprocessing function doing blockmedian, surface, grdfilter and grdsample on combined base and update grid data.
  • and incorporates this functionality into the main() remove-restore workflow

We need to think more about

  • how much this really helps us
  • if it does help, is there a better way to make the preprocessing part of the workflow?
    • do we want to control this via input arguments?
    • do we want a separate entry point for preprocessing?
    • how should it work in the case when the user provides several input grids?

remove unused import.
ensure diff_threshold arg is parsed to float.
will be resampled as part of preprocessing step instead.
this is needed because low-pass filter in preprocessing may
not have values everywhere.
@alessandrofelder
Copy link
Collaborator Author

close-up-comparison
This is a representative close-up comparison of using vs. not using the preprocessing steps B+C (in their current implementation).
I ran:

remove-restore --base gebco_2020_n54.0_s43.0_w-130.0_e-117.0.nc --output=vancouver-island.nc --plot --spacing 0.00028 NOAA_South_VancouverIsland_30m.tif --region_of_interest -123.4 -123 47.8 48.0 --diff_threshold 0.1

It doesn't seem to make any real difference on the smoothness of the boundary? 🤔
For orientation, this is here:
image

@ghost
Copy link

ghost commented May 21, 2021

Just to confirm:
Input base grid is GEBCO ~300m
Input update grid is for Vacouver ~30m
Output spacing ~ 30m

Now: (Step B +C p. 148 in GEBCO Cookbook)

  • All input data combined and blockmedian filtered at 30m*4 = 120m grid
  • Blockmedianed data gridded at 120m using surface
  • Gridded data smoothed at 120m*3 = 3600m using grdflter
  • Filtered data resampled at 30m using grdsample (i.e. Step C of Cookbook)
    Output at this stage is value at nodes of 30m grid.

Next: (remove-restore)

  • All input data combined and blockmedian filtered at 30m grid (Output here is cell-centred!?)
  • Find difference between blockmedianed data and resampled data (i.e. output of Step B+C)
  • If difference above a threshold, replace node data in 30m grid with blockmedian data.

@alessandrofelder
Copy link
Collaborator Author

@Devaraj-G essentially, yes, that is what is happening.

The main difference between your description and what is happening in the code is that only the update grid gets put back into the output of Step B+C.

(small corrections, not important: 3*120=360, not 3600, and Step C is the grdfilter not the grdsample step)

@alessandrofelder alessandrofelder self-assigned this Jun 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant