Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add AWS functionality to workflow setup and forecast job #1708

Merged
merged 42 commits into from
Jun 23, 2023
Merged

Add AWS functionality to workflow setup and forecast job #1708

merged 42 commits into from
Jun 23, 2023

Conversation

HenryRWinterbottom
Copy link
Contributor

Description

This PR provides support for running UFS weather model forecasts, using the global-workflow, on the AWS Parallel Works (PW) platform. This PR accomplishes the following:

  • Building the UFS weather model within the global-workflow build system;
  • Running a UFS weather model atmosphere-only deterministic forecast using the global-workflow infrastructure.

Follow-up PRs will include the following:

  • Adding documentation for building and using the AWS PW UFSWM spack-stack environment; this is heavily borrowed from Sadegh Sadeghi Tabas's presentation 03 May 2023;
  • Addressing remaining TODO issues within this PR; several notes are made throughout as to how to better streamline the connections between the UFSWM and the global-workflow for the respective cloud platforms;
  • Adding data-transfer capabilities within the global-workflow for cloud deployments;
  • Testing additional UFSWM capabilities (e.g., coupling, etc.,) and resolutions (this PR has been tested at C48 and has not been configured for other cubed-sphere resolutions).

No dependencies other than the loading of the appropriate AWS PW modules are required. These are included in this PR.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

This PR has been tested as follows:

  • The global-workflow fork/branch has been deployed to the AWS PW platform;
  • The UFSWM is built using the global-workflow build system;
  • A C48 atmosphere-only forecast has been run to completion.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

HenryRWinterbottom and others added 30 commits June 6, 2023 11:03
@HenryRWinterbottom
Added cloud machine identification pieces.
dcac749
@HenryRWinterbottom
Updates for Rocoto.
27d63bc
@HenryRWinterbottom
Cloud platform modules.
61e4915
@HenryRWinterbottom
Cloud platform workflow information.
3fd30dd
@HenryRWinterbottom
Cloud platform workflow information.
3456297
@HenryRWinterbottom
UFS build script.
4bda331
@HenryRWinterbottom
Merge branch 'feature/cloud_aws_fcstonly' of http://www.github.com/He…
64de5ed 
…nryWinterbottom-NOAA/global-workflow into feature/cloud_aws_fcstonly
@HenryRWinterbottom
Build system updates.
5e09606
@HenryRWinterbottom
Bug fix.
0aebce6
@HenryRWinterbottom
Updates for GW atmosphere-only C48 forecast.
a2cd9e7
@HenryRWinterbottom
Removed left over debug statement.
9a1ff2c
@HenryRWinterbottom
Update build_ufs.sh
2a23028 
Fixed hell-norm detected bug.
@HenryRWinterbottom
Update detect_machine.sh
7cc09bf
@HenryRWinterbottom
Update detect_machine.sh
47ecb86
@HenryRWinterbottom
Update build_ufs.sh
798144d
@HenryRWinterbottom
Update fcst.sh
cbd65ee
@HenryRWinterbottom
Shell norm fixes.
c900c8f
@HenryRWinterbottom
Fixing another shellnorm detected error.
8b1bfc7
@HenryRWinterbottom
Updates for UFSWM develop branch.
0a1a4db
@HenryRWinterbottom
Merge pull request #3 from HenryWinterbottom-NOAA/test/ufswm_develop
01a16b4 
Updates for UFSWM develop branch.
@HenryRWinterbottom
Shellnorm fixes.
f15db2d
@HenryRWinterbottom
Removed no longer necessary files.
e3ce0f5
@HenryRWinterbottom
Shellnorm fixes.
c07d5d5
@HenryRWinterbottom
Used for platform detection.
5d0e4b0
@HenryRWinterbottom
Using PW_CSP to detect platform.
9891b91
@HenryRWinterbottom
Cleanup.
49b1980
@HenryRWinterbottom
Shellnorm fixes.
0a4a508
@HenryRWinterbottom
Merge branch 'develop' into feature/cloud_aws_fcstonly
b30b5af
@HenryRWinterbottom @aerorahul
Update jobs/rocoto/fcst.sh
5010175 
Co-authored-by: Rahul Mahajan <[email protected]>
@HenryRWinterbottom @aerorahul
Update jobs/rocoto/fcst.sh
0b973f4 
Co-authored-by: Rahul Mahajan <[email protected]>
@HenryRWinterbottom HenryRWinterbottom self-assigned this Jun 22, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as draft June 22, 2023 20:58
aerorahul
aerorahul reviewed Jun 22, 2023
View reviewed changes
sorc/build_ufs.sh Outdated Show resolved Hide resolved
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion label Jun 22, 2023
@emcbot emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jun 22, 2023
@emcbot
Copy link

emcbot commented Jun 22, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Jun 22 16:12:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Thu Jun 22 16:14:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Thu Jun 22 17:32:39 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:42 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:45 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:49 CDT 2023 for experiment C96C48_hybatmDA

@emcbot
Copy link

emcbot commented Jun 22, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Jun 22 16:12:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Thu Jun 22 16:14:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Thu Jun 22 17:32:39 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:42 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:45 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:49 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Thu Jun 22 18:07:14 CDT 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 23, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Jun 22 16:12:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Thu Jun 22 16:14:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Thu Jun 22 17:32:39 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:42 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:45 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:49 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Thu Jun 22 18:07:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Thu Jun 22 19:35:24 CDT 2023
with 73 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 23, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Jun 22 16:12:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Thu Jun 22 16:14:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Thu Jun 22 17:32:39 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:42 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:45 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:49 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Thu Jun 22 18:07:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Thu Jun 22 19:35:24 CDT 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Thu Jun 22 23:35:20 CDT 2023
with 308 successfully completed jobs

@emcbot emcbot added CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Jun 23, 2023
@emcbot
Copy link

emcbot commented Jun 23, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Jun 22 16:12:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Thu Jun 22 16:14:05 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Thu Jun 22 17:32:39 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:42 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:45 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Thu Jun 22 17:32:49 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Thu Jun 22 18:07:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Thu Jun 22 19:35:24 CDT 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Thu Jun 22 23:35:20 CDT 2023
with 308 successfully completed jobs

@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as ready for review June 23, 2023 06:11
@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title Feature/cloud aws fcstonly Add AWS functionality to workflow setup and forecast job Jun 23, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit a5d7146 into NOAA-EMC:develop Jun 23, 2023
8 checks passed
@HenryRWinterbottom HenryRWinterbottom deleted the feature/cloud_aws_fcstonly branch July 5, 2023 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aerorahul aerorahul aerorahul left review comments

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

Assignees

@HenryRWinterbottom HenryRWinterbottom

Labels
CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@HenryRWinterbottom @emcbot @aerorahul @WalterKolczynski-NOAA