Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dtree constructor #221

Merged
merged 56 commits into from
Nov 2, 2024
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
a6bfaa0
refactoring dtree node apend using _attach_sweep_groups function
aladinor Oct 29, 2024
e71d046
refactoring iris datatree to use from_dict constructor
aladinor Oct 29, 2024
20d29a3
running pre-commit hook
aladinor Oct 29, 2024
e4c8c9c
refactoring calibration subgroup
aladinor Oct 29, 2024
a086389
Merge branch 'main' into dtree-constructor
aladinor Oct 29, 2024
8d694db
dropping common attributes present in all sweeps
aladinor Oct 29, 2024
06d37a7
refactoring to use from_dict datatree constructor
aladinor Oct 29, 2024
e45c673
refactoring loop by iterating only over sweeps
aladinor Oct 29, 2024
435d281
fixing failing test with attributes
aladinor Oct 29, 2024
ca659c5
addding comments
aladinor Oct 29, 2024
9cbbca9
Update xradar/io/backends/iris.py
kmuehlbauer Oct 30, 2024
62e9c62
fix lint
kmuehlbauer Oct 30, 2024
e400dc5
fixing typo
aladinor Oct 30, 2024
274c224
refactoring datamet backend to use from_dict datatree construnctor
aladinor Oct 30, 2024
70b8cbd
Merge branch 'dtree-constructor' of https://github.com/aladinor/xrada…
aladinor Oct 30, 2024
9a66b18
fixing issue with group test since datamet now has root, georefernce,…
aladinor Oct 30, 2024
15f0c17
adding some calibration variables from Furuno datasets
aladinor Oct 30, 2024
a185598
refactoring furuno backeds to use from_dict contructor
aladinor Oct 30, 2024
d1275c2
getting ride of unncessary attributes in the root dataset
aladinor Oct 30, 2024
393f586
updating global attrributes
aladinor Oct 30, 2024
811f6ae
adding gamic radar calibration parameters
aladinor Oct 30, 2024
4d4db75
refactoring gamic backend to use from_dict datatree constructor
aladinor Oct 30, 2024
c7b2ea6
refactoring code. moving _get_required_root_dataset, _get_subgroup, t…
aladinor Oct 30, 2024
3d9065c
using dtree.match to iterate over sweep nodes
aladinor Oct 30, 2024
978d15e
refactoring _get_subgroups and _get_required_root funcition
aladinor Oct 30, 2024
d6fed5d
fixing typo
aladinor Oct 30, 2024
63fb201
adding _get_radar_calibration fuction
aladinor Oct 31, 2024
ca0b8cc
refactoring and moving _get_requiered_root_Dataset, _get_subgroup, an…
aladinor Oct 31, 2024
2fffb9b
refactoring and moving _get_calibration function to commo.py file
aladinor Oct 31, 2024
dcdb542
refactoring hpl backends to use from_dict contructor and adding sweeps
aladinor Oct 31, 2024
70f36bd
Merge branch 'dtree-constructor' of github.com:aladinor/xradar into d…
aladinor Oct 31, 2024
c201576
using radar_calibration group from common.py in Datamet backend
aladinor Oct 31, 2024
f731eb8
using radar calibration subgrup fucntion from common.py in iris backend
aladinor Oct 31, 2024
0b0eab0
allowing metek to read lines from gzip file when using context manager
aladinor Oct 31, 2024
2e164c7
Merge branch 'main' into dtree-constructor
aladinor Oct 31, 2024
f21c7d5
refactoring _get_required_root function to fit with metek backend
aladinor Oct 31, 2024
723386c
refactoring metek backend to use from_dict constructor
aladinor Oct 31, 2024
869fd5d
Merge branch 'dtree-constructor' of github.com:aladinor/xradar into d…
aladinor Oct 31, 2024
119a7cd
fixing nexrad test that support the radar_calibration, georeference, …
aladinor Oct 31, 2024
fab8a3f
refactoring nexrad backend to use from_dict constructor
aladinor Oct 31, 2024
52ffd97
reformating odim test to include calibration, georeference, and radar…
aladinor Oct 31, 2024
4e2d96e
refactoring odim backend to use from_dict constructor
aladinor Oct 31, 2024
71470c7
refactoring odim export to iterate only over sweeps
aladinor Oct 31, 2024
6854491
refactoring rainbow test to only iterate over sweeps
aladinor Oct 31, 2024
9aec3e4
refactoring rainbow backend to use from_dict constructor
aladinor Oct 31, 2024
d408218
fixin typo
aladinor Oct 31, 2024
66c46e7
adding test for _get_requered_root_group, _get_subgroup, and _get_rad…
aladinor Nov 2, 2024
8abc2e7
Update xradar/model.py
aladinor Nov 2, 2024
9fd48e0
Update model.py
aladinor Nov 2, 2024
e6be32b
Update model.py
aladinor Nov 2, 2024
971ea59
Update model.py
aladinor Nov 2, 2024
da470d2
Update model.py
aladinor Nov 2, 2024
3cf21cd
Update xradar/model.py
aladinor Nov 2, 2024
064662a
Update model.py
aladinor Nov 2, 2024
468f502
Update xradar/model.py
kmuehlbauer Nov 2, 2024
9c626c4
Update history.md
aladinor Nov 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refactoring and moving _get_requiered_root_Dataset, _get_subgroup, an…
…d _get_calibration function to commo.py file
  • Loading branch information
aladinor committed Oct 31, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit ca0b8cc19df84d7ffe515a3358b9ef9f720b0137
61 changes: 1 addition & 60 deletions xradar/io/backends/iris.py
Original file line number Diff line number Diff line change
@@ -61,12 +61,10 @@
get_longitude_attrs,
get_range_attrs,
moment_attrs,
optional_root_vars,
radar_parameters_subgroup,
required_root_vars,
sweep_vars_mapping,
)
from .common import _assign_root, _attach_sweep_groups
from .common import _attach_sweep_groups, _get_required_root_dataset, _get_subgroup

#: mapping from IRIS names to CfRadial2/ODIM
iris_mapping = {
@@ -3969,63 +3967,6 @@ def _get_iris_group_names(filename):
return keys


def _get_required_root_dataset(ls_ds, optional=True):
"""Extract Root Dataset."""
# keep only defined mandatory and defined optional variables per default
# by checking in all nodes
data_var = {x for xs in [sweep.variables.keys() for sweep in ls_ds] for x in xs}
remove_root = set(data_var) ^ set(required_root_vars)
if optional:
remove_root ^= set(optional_root_vars)
remove_root ^= {"sweep_number", "fixed_angle"}
remove_root &= data_var
root = [sweep.drop_vars(remove_root) for sweep in ls_ds]
root_vars = {x for xs in [sweep.variables.keys() for sweep in root] for x in xs}
# rename variables
# todo: find a more easy method not iterating over all variables
for k in root_vars:
rename = optional_root_vars.get(k, None)
if rename:
root = [sweep.rename_vars({k: rename}) for sweep in root]

root_vars = {x for xs in [sweep.variables.keys() for sweep in root] for x in xs}
ds_vars = [sweep[root_vars] for sweep in ls_ds]
vars = xr.concat(ds_vars, dim="sweep").reset_coords()

# Creating the root group using _assign_root function
ls = ls_ds.copy()
ls.insert(0, xr.Dataset())
root = _assign_root(ls)

# merging both the created and the variables within each dataset
root = xr.merge([root, vars])

# Renaming variable
root = root.rename_vars({"sweep_number": "sweep_group_name"})
root["sweep_group_name"].values = np.array(
[f"sweep_{i}" for i in root["sweep_group_name"].values]
)
# override/fix dtype in encoding
root["sweep_group_name"].encoding["dtype"] = root["sweep_group_name"].dtype
return root


def _get_subgroup(ls_ds: list[xr.Dataset], subdict):
"""Get iris-sigmet root metadata group.
Variables are fetched from the provided Dataset according to the subdict dictionary.
"""
meta_vars = subdict
data_vars = {x for xs in [ds.variables.keys() for ds in ls_ds] for x in xs}
extract_vars = set(data_vars) & set(meta_vars)
subgroup = xr.concat([ds[extract_vars] for ds in ls_ds], "sweep")
for k in subgroup.data_vars:
rename = meta_vars[k]
if rename:
subgroup = subgroup.rename_vars({k: rename})
subgroup.attrs = {}
return subgroup


def _get_radar_calibration(ls_ds: list[xr.Dataset]) -> xr.Dataset:
"""Get radar calibration root metadata group."""
# radar_calibration is connected with calib-dimension