Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add run and finalize methods to marine LETKF task #2944

Conversation

AndrewEichmann-NOAA
Copy link
Contributor

@AndrewEichmann-NOAA AndrewEichmann-NOAA commented Sep 19, 2024

Description

Adds run and finalize methods to marine LETKF task, experiment yaml for gw-ci in GDASApp, workflow additions, removes bugs found on the way, and completes the bulk of the work on LETKF task. Conversion of fields to increments pending.

Partially resolves NOAA-EMC/GDASApp#1091 and NOAA-EMC/GDASApp#1251

Mutual dependency with GDASApp PR NOAA-EMC/GDASApp#1287 and IC fix file issue #2944 (comment)

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? YES (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

Example:

  • Cycled test on Hera

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

@AndrewEichmann-NOAA
Copy link
Contributor Author

Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll probably have to rearrange things between your PR and mine at some point, but this looks good to me. Thanks for doing this @AndrewEichmann-NOAA .

ush/python/pygfs/task/marine_letkf.py Outdated Show resolved Hide resolved
workflow/rocoto/gfs_tasks.py Outdated Show resolved Hide resolved
workflow/rocoto/gfs_tasks.py Outdated Show resolved Hide resolved
jobs/JGLOBAL_MARINE_BMAT Outdated Show resolved Hide resolved
@@ -82,12 +82,14 @@ declare -rx COM_OCEAN_HISTORY_TMPL=${COM_BASE}'/model/ocean/history'
declare -rx COM_OCEAN_RESTART_TMPL=${COM_BASE}'/model/ocean/restart'
declare -rx COM_OCEAN_INPUT_TMPL=${COM_BASE}'/model/ocean/input'
declare -rx COM_OCEAN_ANALYSIS_TMPL=${COM_BASE}'/analysis/ocean'
declare -rx COM_OCEAN_LETKF_TMPL=${COM_BASE}'/analysis/ocean/letkf'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What files get written to the letkf directories vs the regular component analysis directories?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the analysis files from the LETKF, which are separate from the variational analysis files

ush/python/pygfs/task/marine_letkf.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/marine_letkf.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/marine_letkf.py Outdated Show resolved Hide resolved
workflow/rocoto/gfs_tasks.py Outdated Show resolved Hide resolved
@emcbot emcbot added the CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed label Oct 30, 2024
@emcbot
Copy link

emcbot commented Oct 30, 2024

CI Failed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/2944


Experiment C48mx500_3DVarAOWCDA_9fa78132 Completed 2 Cycles: *SUCCESS* at Tue Oct 29 20:28:28 UTC 2024
Experiment C48_ATM_9fa78132 Completed 2 Cycles: *SUCCESS* at Tue Oct 29 21:48:25 UTC 2024
Experiment C96_S2SWA_gefs_replay_ics_9fa78132 Completed 1 Cycles: *SUCCESS* at Tue Oct 29 21:48:25 UTC 2024
Experiment C48_S2SW_9fa78132 Completed 2 Cycles: *SUCCESS* at Tue Oct 29 22:50:28 UTC 2024
Experiment C96_atm3DVar_9fa78132 Completed 3 Cycles: *SUCCESS* at Tue Oct 29 23:15:15 UTC 2024
Experiment C48_S2SWA_gefs_9fa78132 Completed 1 Cycles: *SUCCESS* at Tue Oct 29 23:23:10 UTC 2024
Experiment C96C48_hybatmaerosnowDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Tue Oct 29 23:27:36 UTC 2024
Experiment C96C48_hybatmDA_9fa78132 Terminated with 0
FAIL
FAIL tasks failed and 1 dead at Wed Oct 30 00:59:18 UTC 2024
Experiment C96C48_hybatmDA_9fa78132 Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/2944/RUNTESTS/COMROOT/C96C48_hybatmDA_9fa78132/logs/2021122106/gdas_arch.log
Experiment C96C48_ufs_hybatmDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 03:32:35 UTC 2024

@WalterKolczynski-NOAA
Copy link
Contributor

Failed due to a system issue with modules we've seen before. Going to restart the test and also get started testing elsewhere since all other tests passed.

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Oct 30, 2024
@emcbot emcbot added CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Oct 30, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS label Oct 30, 2024
@WalterKolczynski-NOAA
Copy link
Contributor

Running in /lfs/h2/emc/ptmp/walter.kolczynski/PR/PR_2944

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Oct 30, 2024
@emcbot emcbot added CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully and removed CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress labels Oct 30, 2024
@emcbot
Copy link

emcbot commented Oct 30, 2024

CI Passed on Hercules in Build# 3
Built and ran in directory /work2/noaa/global/CI/HERCULES/2944


Experiment C48_ATM_9fa78132 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 05:12:25 CDT 2024
Experiment C96_S2SWA_gefs_replay_ics_9fa78132 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 05:36:51 CDT 2024
Experiment C96C48_hybatmDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 06:25:30 CDT 2024
Experiment C96_atm3DVar_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 06:31:22 CDT 2024
Experiment C48_S2SW_9fa78132 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 07:31:48 CDT 2024
Experiment C48_S2SWA_gefs_9fa78132 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 07:38:46 CDT 2024

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Oct 30, 2024
@emcbot
Copy link

emcbot commented Oct 30, 2024

CI Passed on Hera in Build# 4
Built and ran in directory /scratch1/NCEPDEV/global/CI/2944


Experiment C96_S2SWA_gefs_replay_ics_9fa78132 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 10:31:09 UTC 2024
Experiment C48_ATM_9fa78132 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 10:43:12 UTC 2024
Experiment C48mx500_3DVarAOWCDA_9fa78132 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 10:43:13 UTC 2024
Experiment C96C48_hybatmaerosnowDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 11:58:51 UTC 2024
Experiment C96C48_hybatmDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 12:04:20 UTC 2024
Experiment C96_atm3DVar_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 12:04:20 UTC 2024
Experiment C48_S2SWA_gefs_9fa78132 Completed 1 Cycles: *SUCCESS* at Wed Oct 30 12:17:28 UTC 2024
Experiment C48_S2SW_9fa78132 Completed 2 Cycles: *SUCCESS* at Wed Oct 30 12:47:31 UTC 2024
Experiment C96C48_ufs_hybatmDA_9fa78132 Completed 3 Cycles: *SUCCESS* at Wed Oct 30 12:47:31 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Oct 30, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 0b3304e into NOAA-EMC:develop Oct 30, 2024
10 of 11 checks passed
guillaumevernieres added a commit to NOAA-EMC/GDASApp that referenced this pull request Nov 7, 2024
Completes LETKF task, adds WCDA CI test

Mutually dependent on
NOAA-EMC/global-workflow#2944

---------

Co-authored-by: Cory Martin <[email protected]>
Co-authored-by: Guillaume Vernieres <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add marine letkf task
6 participants