Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Fixing several issues, including 966 (bash octal issue); add new winter weather verification test with staged data #997

Merged
merged 22 commits into from
Jan 11, 2024
Merged

[develop] Fixing several issues, including 966 (bash octal issue); add new winter weather verification test with staged data #997

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
47a0092
Fix octal issue
mkavulich Nov 29, 2023
0d903ba
Give EXPT_SUBDIR a default value "experiment" (see issue 978)
mkavulich Nov 29, 2023
94084a9
Update some default values: FV3GFS_FILE_FMT_ICS and FV3GFS_FILE_FMT_L…
mkavulich Nov 30, 2023
9b359a5
Append "starttime_string" to experiment dictionary keys to make them …
mkavulich Dec 4, 2023
56e2141
Add check for duplicate working directories in monitor_jobs.py
mkavulich Dec 5, 2023
48f36f7
Output fields should be hourly for all CCPP suites. We really need a …
mkavulich Dec 5, 2023
fbcd3f9
Finally found source of problem: namelist variable "fhzero" controls …
mkavulich Dec 6, 2023
97c0fc7
Remove vestiges of unsupported CCPP suites
mkavulich Dec 7, 2023
013ed9b
Update all diag tables to output refl_10cm (simulated radar reflectiv…
mkavulich Dec 8, 2023
c791666
Per discussion in #939, "Derecho" is not actually needed here
mkavulich Dec 8, 2023
016c8bd
Move changing of default FV3GFS_FILE_FMT_ICS and FV3GFS_FILE_FMT_LBCS…
mkavulich Dec 8, 2023
a5b8082
Update dependences for verification tasks: "get_obs" tasks are always…
mkavulich Dec 8, 2023
f0f0149
Adding new 10-member ensemble verification test for winter wx
mkavulich Dec 20, 2023
2f59978
Update settings for new test
mkavulich Dec 20, 2023
228bac5
An actual use case for the WORKFLOW_ID variable: match it with the di…
mkavulich Dec 20, 2023
97a037b
Update comprehensive test lists, update machine files to include path…
mkavulich Dec 21, 2023
6e8cc9f
Remove hard-coded obs paths now that data is staged
mkavulich Dec 21, 2023
bcdb494
Forgot to add NOHRSC to the list of verification obs for populating t…
mkavulich Dec 21, 2023
548a2f7
Update some test descriptions
mkavulich Dec 21, 2023
8af835a
Big update to verification obs script that I almost forgot to commit!
mkavulich Dec 21, 2023
54ee587
Fix undefined variable problem in exregonal_run_met_pcpcombine.sh...t…
mkavulich Jan 10, 2024
cad113d
Remove ENSMEM_INDX definition from pcpcombine observation tasks
mkavulich Jan 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion parm/FV3.input.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,6 @@ FV3_GFS_v17_p8:
doGP_cldoptics_LUT: False
doGP_lwscat: False
dt_inner: 150
fhzero: 6
frac_grid: True
gwd_opt: 2
hybedmf: False
Expand Down
317 changes: 0 additions & 317 deletions parm/diag_table.FV3_GFS_2017_gfdlmp
View file
Open in desktop

This file was deleted.

359 changes: 0 additions & 359 deletions parm/diag_table.FV3_GFS_2017_gfdlmp_regional
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions parm/diag_table.FV3_GFS_v15_thompson_mynn_lam3km
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@
"gfs_phys", "t02min", "t02min", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "rh02max", "rh02max", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "rh02min", "rh02min", "fv3_history2d", "all", .false., "none", 2
# Reflectivity from Thompson microphysics
"gfs_phys", "refl_10cm", "refl_10cm", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "cldfra", "cldfra", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "pratemax", "pratemax", "fv3_history2d", "all", .false., "none", 2
Expand Down
2 changes: 2 additions & 0 deletions parm/diag_table.FV3_GFS_v15p2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,8 @@
"gfs_phys", "nbdsf_ave", "nbdsf_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "nddsf_ave", "nddsf_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "trans_ave", "trans_ave", "fv3_history2d", "all", .false., "none", 2
# Reflectivity from GFDL microphysics
"gfs_phys", "refl_10cm", "refl_10cm", "fv3_history2d", "all", .false., "none", 2
# Stochastic physics
"gfs_phys", "sppt_wts", "sppt_wts", "fv3_history", "all", .false., "none", 2
"gfs_phys", "skebu_wts", "skebu_wts", "fv3_history", "all", .false., "none", 2
Expand Down
2 changes: 2 additions & 0 deletions parm/diag_table.FV3_GFS_v16
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,8 @@
"gfs_phys", "nbdsf_ave", "nbdsf_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "nddsf_ave", "nddsf_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "trans_ave", "trans_ave", "fv3_history2d", "all", .false., "none", 2
# Reflectivity from GFDL microphysics
"gfs_phys", "refl_10cm", "refl_10cm", "fv3_history2d", "all", .false., "none", 2
# Stochastic physics
"gfs_phys", "sppt_wts", "sppt_wts", "fv3_history", "all", .false., "none", 2
"gfs_phys", "skebu_wts", "skebu_wts", "fv3_history", "all", .false., "none", 2
Expand Down
6 changes: 4 additions & 2 deletions parm/diag_table.FV3_GFS_v17_p8
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
#output files
"grid_spec", -1, "months", 1, "days", "time"
"atmos_static", -1, "hours", 1, "hours", "time"
"fv3_history", 3, "hours", 1, "hours", "time"
"fv3_history2d", 3, "hours", 1, "hours", "time"
"fv3_history", 0, "hours", 1, "hours", "time"
"fv3_history2d", 0, "hours", 1, "hours", "time"

#
#=======================
Expand Down Expand Up @@ -154,6 +154,8 @@
"gfs_phys", "BC_AOD_550", "bc_aod550", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "OC_AOD_550", "oc_aod550", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "SS_AOD_550", "ss_aod550", "fv3_history2d", "all", .false., "none", 2
# Reflectivity from Thompson microphysics
Copy link
Collaborator

@JeffBeck-NOAA JeffBeck-NOAA Dec 21, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be 'GFDL microphysics'? This is a GFS SDF.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GFSv17 P8 is testing with Thompson!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct @michelleharrold. I'm not sure if it was different previously, but from what I can see currently the FV3_GFS_v17_p8 suite uses Thompson microphysics: https://github.com/NOAA-EMC/fv3atm/blob/bba399053d3939241938f19ee598895eea54fd65/ccpp/suites/suite_FV3_GFS_v17_p8.xml#L76

"gfs_phys", "refl_10cm", "refl_10cm", "fv3_history2d", "all", .false., "none", 2

"gfs_sfc", "crain", "crain", "fv3_history2d", "all", .false., "none", 2
"gfs_sfc", "tprcp", "tprcp", "fv3_history2d", "all", .false., "none", 2
Expand Down
1 change: 1 addition & 0 deletions parm/diag_table.FV3_HRRR
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@
"gfs_dyn", "graupel", "grle", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "ps", "pressfc", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "hs", "hgtsfc", "fv3_history", "all", .false., "none", 2
# Reflectivity from Thompson microphysics
"gfs_phys", "refl_10cm" "refl_10cm" "fv3_history", "all", .false., "none", 2
"gfs_phys", "cldfra", "cldfra", "fv3_history", "all", .false., "none", 2
#"gfs_dyn", "ice_nc", "nicp", "fv3_history", "all", .false., "none", 2
Expand Down
3 changes: 2 additions & 1 deletion parm/diag_table.FV3_RAP
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,8 @@
"gfs_phys", "v-gwd_ave", "v-gwd_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "dusfc", "uflx_ave", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "dvsfc", "vflx_ave", "fv3_history2d", "all", .false., "none", 2
#"gfs_phys", "refl_10cm", "refl10cm", "fv3_history2d", "all", .false., "none", 2
# Reflectivity from Thompson microphysics
"gfs_phys", "refl_10cm", "refl_10cm", "fv3_history2d", "all", .false., "none", 2

"gfs_phys", "psurf", "pressfc", "fv3_history2d", "all", .false., "none", 2
"gfs_phys", "u10m", "ugrd10m", "fv3_history2d", "all", .false., "none", 2
Expand Down
348 changes: 0 additions & 348 deletions parm/diag_table.FV3_RRFS_v0
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions parm/diag_table.FV3_RRFS_v1beta
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@
"gfs_dyn", "graupel", "grle", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "ps", "pressfc", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "hs", "hgtsfc", "fv3_history", "all", .false., "none", 2
# Reflectivity from Thompson microphysics
"gfs_phys", "refl_10cm" "refl_10cm" "fv3_history", "all", .false., "none", 2
"gfs_phys", "cldfra", "cldfra", "fv3_history", "all", .false., "none", 2
#"gfs_dyn", "ice_nc", "nicp", "fv3_history", "all", .false., "none", 2
Expand Down
1 change: 1 addition & 0 deletions parm/diag_table.FV3_WoFS_v0
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@
"gfs_dyn", "ccn_nc", "ccn", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "ps", "pressfc", "fv3_history", "all", .false., "none", 2
"gfs_dyn", "hs", "hgtsfc", "fv3_history", "all", .false., "none", 2
# Reflectivity from NSSL microphysics
"gfs_phys", "refl_10cm" "refl_10cm" "fv3_history", "all", .false., "none", 2
#"gfs_phys", "cldfra", "cldfra", "fv3_history", "all", .false., "none", 2

Expand Down
38 changes: 0 additions & 38 deletions parm/field_table.FV3_GFS_2017_gfdlmp
View file
Open in desktop

This file was deleted.

38 changes: 0 additions & 38 deletions parm/field_table.FV3_GFS_2017_gfdlmp_regional
View file
Open in desktop

This file was deleted.

65 changes: 0 additions & 65 deletions parm/field_table.FV3_RRFS_v0
View file
Open in desktop

This file was deleted.

15 changes: 7 additions & 8 deletions parm/wflow/verify_det.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ metatask_GridStat_NOHRSC_all_accums_all_mems:
SLASH_ENSMEM_SUBDIR_OR_NULL: '{% if global.DO_ENSEMBLE %}{{ "/mem#mem#" }}{% endif %}'
walltime: 02:00:00
dependency:
and:
taskdep:
attrs:
task: get_obs_mrms
taskdep_pcpcombine_fcst:
attrs:
task: run_MET_PcpCombine_fcst_ASNOW#ACCUM_HH#h_mem#mem#
Expand All @@ -98,14 +102,9 @@ metatask_GridStat_MRMS_all_mems:
walltime: 02:00:00
dependency:
and:
or_get_obs: # Ensure get_obs task is complete if it's turned on
not:
taskvalid:
attrs:
task: get_obs_mrms
taskdep:
attrs:
task: get_obs_mrms
taskdep:
attrs:
task: get_obs_mrms
datadep_post_files_exist:
attrs:
age: 00:00:00:30
Expand Down
21 changes: 10 additions & 11 deletions parm/wflow/verify_ens.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,13 @@ metatask_GenEnsProd_EnsembleStat_NOHRSC:
<<: *envars_GenEnsProd_NOHRSC
MET_TOOL: 'ENSEMBLESTAT'
dependency:
taskdep_genensprod:
attrs:
task: run_MET_GenEnsProd_vx_ASNOW#ACCUM_HH#h
and:
taskdep:
attrs:
task: get_obs_nohrsc
taskdep_genensprod:
attrs:
task: run_MET_GenEnsProd_vx_ASNOW#ACCUM_HH#h

metatask_GenEnsProd_EnsembleStat_MRMS:
var:
Expand All @@ -101,14 +105,9 @@ metatask_GenEnsProd_EnsembleStat_MRMS:
OBTYPE: 'MRMS'
dependency:
and:
or_get_obs: # Ensure get_obs task is complete if it's turned on
not:
taskvalid:
attrs:
task: get_obs_mrms
taskdep:
attrs:
task: get_obs_mrms
taskdep:
attrs:
task: get_obs_mrms
metataskdep_post_files_exist: &post_files_exist
attrs:
metatask: check_post_output_all_mems
Expand Down
1 change: 0 additions & 1 deletion parm/wflow/verify_pre.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,6 @@ metatask_PcpCombine_obs:
OBTYPE: CCPA
OBS_DIR: '&CCPA_OBS_DIR;'
MET_TOOL: 'PCPCOMBINE'
ENSMEM_INDX: ''
dependency:
and:
datadep:
Expand Down
2 changes: 1 addition & 1 deletion scripts/exregional_check_post_output.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ time_lag=$( bc -l <<< "${ENS_TIME_LAG_HRS[$i]}*${SECS_PER_HOUR}" )
#
#-----------------------------------------------------------------------
#
ensmem_indx=$(printf "%0${VX_NDIGITS_ENSMEM_NAMES}d" "${ENSMEM_INDX}")
ensmem_indx=$(printf "%0${VX_NDIGITS_ENSMEM_NAMES}d" $(( 10#${ENSMEM_INDX})))
ensmem_name="mem${ensmem_indx}"
FCST_INPUT_FN_TEMPLATE=$( eval echo ${FCST_SUBDIR_TEMPLATE:+${FCST_SUBDIR_TEMPLATE}/}${FCST_FN_TEMPLATE} )
set_vx_fhr_list \
Expand Down
Loading
Loading