Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update grib2 tables with g2tmpl v1.10.2 and change soil moisture grib2 name #855

Merged
merged 2 commits into from
Jan 19, 2024
Merged

Update grib2 tables with g2tmpl v1.10.2 and change soil moisture grib2 name #855

merged 2 commits into from
Jan 19, 2024

Conversation

WenMeng-NOAA
Copy link
Collaborator

The main changes are:

  • Update grib2 tables in UPP with g2tmpl v1.10.2
  • Change soil moisture grib2 name from 'SOILM' into 'SOILMOI'

This was linked to issues Jan 16, 2024
Update grib2 name for soil moisture #854
Closed
Update grib2 tables in UPP with g2tmpl v1.10.2 #853
Closed
@WenMeng-NOAA WenMeng-NOAA added GFSV17 Ready for Review This PR is ready for code review. Baseline Change The baselines of the UPP regression tests are changed. labels Jan 18, 2024
@WenMeng-NOAA
Copy link
Collaborator Author

@FernandoAndrade-NOAA You may conduct the UPP RTs on Hera and Orion. There are no variable value changes except grib2 meta data update. For your reference, here are my test results on Hera at /scratch1/NCEPDEV/stmp2/Wen.Meng:
'''
[Wen.Meng@hfe06 ~/stmp2]$ ls -ltr */*diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGDAWP03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRD3D03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRDSF03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:25 rap_2020072316/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGDAWP03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:28 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRD3D03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRDSF03.tm00.Grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:33 gfs.20190830/gfs.t00z.sfluxgrbf006.grib2.diff
-rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:34 gfs.20190830_pe_test/gfs.t00z.sfluxgrbf006.grib2.diff
'''

@FernandoAndrade-NOAA
Copy link
Collaborator

FernandoAndrade-NOAA commented Jan 18, 2024 •
edited
Loading

It seems that Jenkins is still sitting in queue, I will run manually to see if that ends up being a bit faster on Orion and Hera.

@WenMeng-NOAA
Copy link
Collaborator Author

The grib2 name of soil moisture in gfs flux dataset is updated
From
21:45715792:d=2019083000:SOIL_M:0-2 m below ground:6 hour fcst:
into
21:45715792:d=2019083000:SOILMOI:0-2 m below ground:6 hour fcst:

@epic-cicd-jenkins
Copy link
Collaborator

Orion UPP Regression Tests FAILED

There are changes in results for case nmmb_pe_test in BGRDSF03.tm00.Grib2
There are changes in results for case nmmb_pe_test in BGRD3D03.tm00.Grib2
There are changes in results for case nmmb_pe_test in BGDAWP03.tm00.Grib2
There are changes in results for case rap in WRFPRS.GrbF16
There are changes in results for case gfs in gfs.t00z.sfluxgrbf006.grib2
There are changes in results for case nmmb in BGRDSF03.tm00.Grib2
There are changes in results for case nmmb in BGRD3D03.tm00.Grib2
There are changes in results for case nmmb in BGDAWP03.tm00.Grib2
Refer to .diff files in rundir: /work/noaa/epic/role-epic/jenkins/workspace/UPP_UPP-Pipeline_PR-855/ci/rundir/upp-ORION for details on differences in results for each case.

@FernandoAndrade-NOAA
Copy link
Collaborator

FernandoAndrade-NOAA commented Jan 19, 2024 •
edited
Loading

Those failures are not fully matching with all the expected changes, I believe Jenkins timed out. The manual runs are still in queue on both systems, apologies for the delayed queues I'll check in tomorrow to see where we are with increased epic allocation.

@epic-cicd-jenkins
Copy link
Collaborator

Hera UPP Regression Tests FAILED

There are changes in results for case gfs_pe_test in gfs.t00z.sfluxgrbf006.grib2
There are changes in results for case nmmb in BGDAWP03.tm00.Grib2
There are changes in results for case nmmb in BGRD3D03.tm00.Grib2
There are changes in results for case nmmb in BGRDSF03.tm00.Grib2
There are changes in results for case gfs in gfs.t00z.sfluxgrbf006.grib2
There are changes in results for case rap in WRFPRS.GrbF16
Refer to .diff files in rundir: /scratch2/NAGAPE/epic/role.epic/jenkins/workspace/UPP_UPP-Pipeline_PR-855/ci/rundir/upp-HERA for details on differences in results for each case.

@epic-cicd-jenkins
Copy link
Collaborator

Hercules UPP Regression Tests FAILED

There are changes in results for case gfs_pe_test in gfs.t00z.master.grb2f006
There are changes in results for case gfs_pe_test in gfs.t00z.sfluxgrbf006.grib2
There are changes in results for case fv3r_pe_test in NATLEV10.tm00
There are changes in results for case fv3r_pe_test in PRSLEV10.tm00
There are changes in results for case rtma in PRSLEV00.tm00
There are changes in results for case rtma in NATLEV00.tm00
There are changes in results for case hrrr_pe_test in WRFTWO.GrbF04
There are changes in results for case gfs in gfs.t00z.sfluxgrbf006.grib2
There are changes in results for case gfs in gfs.t00z.master.grb2f006
There are changes in results for case fv3r in PRSLEV10.tm00
There are changes in results for case fv3r in NATLEV10.tm00
There are changes in results for case rap_pe_test in WRFPRS.GrbF16
There are changes in results for case rap_pe_test in WRFNAT.GrbF16
There are changes in results for case nmmb in BGRDSF03.tm00.Grib2
There are changes in results for case nmmb in BGDAWP03.tm00.Grib2
There are changes in results for case nmmb in BGRD3D03.tm00.Grib2
There are changes in results for case rtma_pe_test in PRSLEV00.tm00
There are changes in results for case rtma_pe_test in NATLEV00.tm00
There are changes in results for case rap in WRFPRS.GrbF16
There are changes in results for case rap in WRFNAT.GrbF16
There are changes in results for case hrrr in WRFTWO.GrbF04
There are changes in results for case nmmb_pe_test in BGDAWP03.tm00.Grib2
There are changes in results for case nmmb_pe_test in BGRDSF03.tm00.Grib2
There are changes in results for case nmmb_pe_test in BGRD3D03.tm00.Grib2
Refer to .diff files in rundir: /work/noaa/epic/role-epic/jenkins/workspace/UPP_UPP-Pipeline_PR-855/ci/rundir/upp-HERCULES for details on differences in results for each case.

@WenMeng-NOAA
Copy link
Collaborator Author

The UPP RT was completed on WCOSS2. There will be baseline recreations for gfs, nmmb, rap with this PR.

@FernandoAndrade-NOAA
Copy link
Collaborator

Rerunning Hera manually with a different account, the test results look incomplete. Orion RTs will be rerun with windfall as it also didn't fully finish RTs. EPIC is still following up about receiving increased allocation for the accounts to minimize this sort of issue.

@FernandoAndrade-NOAA
Copy link
Collaborator

@FernandoAndrade-NOAA You may conduct the UPP RTs on Hera and Orion. There are no variable value changes except grib2 meta data update. For your reference, here are my test results on Hera at /scratch1/NCEPDEV/stmp2/Wen.Meng: ''' [Wen.Meng@hfe06 ~/stmp2]$ ls -ltr */*diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGDAWP03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRD3D03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRDSF03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:25 rap_2020072316/WRFPRS.GrbF16.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGDAWP03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:28 rap_2020072316_pe_test/WRFPRS.GrbF16.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRD3D03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRDSF03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:33 gfs.20190830/gfs.t00z.sfluxgrbf006.grib2.diff -rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:34 gfs.20190830_pe_test/gfs.t00z.sfluxgrbf006.grib2.diff '''

I see similar changes in my results on Hera and Orion, along with the change in gfs sflux file. I did notice a difference in the .diff output between hera and orion. Just something to be aware of:
Hera:

33:65050514:LANDFRC:surface:rpn_corr=-nan:rpn_rms=undefined
34:65640517:PAHFLX:surface:rpn_corr=-nan:rpn_rms=undefined
35:66230520:WATERSA:surface:rpn_corr=-nan:rpn_rms=undefined
36:66820523:EIWATER:surface:rpn_corr=-nan:rpn_rms=undefined
37:67410550:PLANTTR:surface:rpn_corr=-nan:rpn_rms=undefined
38:68000577:SOILSE:surface:rpn_corr=-nan:rpn_rms=undefined
39:68590604:PAHFLX:surface:rpn_corr=-nan:rpn_rms=undefined

Orion:

33:65050514:var discipline=2 center=7 local_table=1 parmcat=0 parm=233:surface:rpn_corr=-nan:rpn_rms=undefined
34:65640517:var discipline=2 center=7 local_table=1 parmcat=0 parm=235:surface:rpn_corr=-nan:rpn_rms=undefined
35:66230520:var discipline=2 center=7 local_table=1 parmcat=0 parm=236:surface:rpn_corr=-nan:rpn_rms=undefined
36:66820523:var discipline=2 center=7 local_table=1 parmcat=0 parm=237:surface:rpn_corr=-nan:rpn_rms=undefined
37:67410550:var discipline=2 center=7 local_table=1 parmcat=0 parm=238:surface:rpn_corr=-nan:rpn_rms=undefined
38:68000577:var discipline=2 center=7 local_table=1 parmcat=0 parm=239:surface:rpn_corr=-nan:rpn_rms=undefined
39:68590604:var discipline=2 center=7 local_table=1 parmcat=0 parm=235:surface:rpn_corr=-nan:rpn_rms=undefined

@WenMeng-NOAA
Copy link
Collaborator Author

WenMeng-NOAA commented Jan 19, 2024 •
edited
Loading

@FernandoAndrade-NOAA You may conduct the UPP RTs on Hera and Orion. There are no variable value changes except grib2 meta data update. For your reference, here are my test results on Hera at /scratch1/NCEPDEV/stmp2/Wen.Meng: ''' [Wen.Meng@hfe06 ~/stmp2]$ ls -ltr */*diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGDAWP03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRD3D03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:22 post_nmmb_meso_2014120818_Grib2/BGRDSF03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:25 rap_2020072316/WRFPRS.GrbF16.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGDAWP03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 75 Jan 16 22:28 rap_2020072316_pe_test/WRFPRS.GrbF16.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRD3D03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 0 Jan 16 22:28 post_nmmb_meso_2014120818_Grib2_pe_test/BGRDSF03.tm00.Grib2.diff -rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:33 gfs.20190830/gfs.t00z.sfluxgrbf006.grib2.diff -rw-r--r-- 1 Wen.Meng stmp 417 Jan 16 22:34 gfs.20190830_pe_test/gfs.t00z.sfluxgrbf006.grib2.diff '''

I see similar changes in my results on Hera and Orion, along with the change in gfs sflux file. I did notice a difference in the .diff output between hera and orion. Just something to be aware of: Hera:

33:65050514:LANDFRC:surface:rpn_corr=-nan:rpn_rms=undefined
34:65640517:PAHFLX:surface:rpn_corr=-nan:rpn_rms=undefined
35:66230520:WATERSA:surface:rpn_corr=-nan:rpn_rms=undefined
36:66820523:EIWATER:surface:rpn_corr=-nan:rpn_rms=undefined
37:67410550:PLANTTR:surface:rpn_corr=-nan:rpn_rms=undefined
38:68000577:SOILSE:surface:rpn_corr=-nan:rpn_rms=undefined
39:68590604:PAHFLX:surface:rpn_corr=-nan:rpn_rms=undefined

Orion:

33:65050514:var discipline=2 center=7 local_table=1 parmcat=0 parm=233:surface:rpn_corr=-nan:rpn_rms=undefined
34:65640517:var discipline=2 center=7 local_table=1 parmcat=0 parm=235:surface:rpn_corr=-nan:rpn_rms=undefined
35:66230520:var discipline=2 center=7 local_table=1 parmcat=0 parm=236:surface:rpn_corr=-nan:rpn_rms=undefined
36:66820523:var discipline=2 center=7 local_table=1 parmcat=0 parm=237:surface:rpn_corr=-nan:rpn_rms=undefined
37:67410550:var discipline=2 center=7 local_table=1 parmcat=0 parm=238:surface:rpn_corr=-nan:rpn_rms=undefined
38:68000577:var discipline=2 center=7 local_table=1 parmcat=0 parm=239:surface:rpn_corr=-nan:rpn_rms=undefined
39:68590604:var discipline=2 center=7 local_table=1 parmcat=0 parm=235:surface:rpn_corr=-nan:rpn_rms=undefined

The lower version of wgrib2 on Orion does not include the upgraded NCEP local table. From grib2 tables online, 'discipline=2 center=7 local_table=1 parmcat=0 parm=233' and 'LANDFRC' are the same, see
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-2-2-0.shtml

@WenMeng-NOAA
Copy link
Collaborator Author

@FernandoAndrade-NOAA You may go ahead to approve this PR. Thanks!

@FernandoAndrade-NOAA
Copy link
Collaborator

Thanks for explaining! And thank you for your patience regarding the account difficulties.

@WenMeng-NOAA
Copy link
Collaborator Author

This PR is ready for merging.

@WenMeng-NOAA WenMeng-NOAA merged commit 40a67b9 into NOAA-EMC:develop Jan 19, 2024
4 checks passed
@WenMeng-NOAA WenMeng-NOAA mentioned this pull request Jan 19, 2024
Closed
@WenMeng-NOAA
Copy link
Collaborator Author

@FernandoAndrade-NOAAThe PR was merged. You may update the baselines of the UPP RT on Hera and Orion. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HuiyaChuang-NOAA HuiyaChuang-NOAA HuiyaChuang-NOAA approved these changes

@FernandoAndrade-NOAA FernandoAndrade-NOAA FernandoAndrade-NOAA approved these changes

Assignees
No one assigned
Labels
Baseline Change The baselines of the UPP regression tests are changed. GFSV17 Ready for Review This PR is ready for code review.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Update grib2 name for soil moisture Update grib2 tables in UPP with g2tmpl v1.10.2
4 participants
@WenMeng-NOAA @FernandoAndrade-NOAA @epic-cicd-jenkins @HuiyaChuang-NOAA