Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create gw_setup script and extend to other machines #1706

Merged
merged 4 commits into from
Jun 26, 2023
Merged

Create gw_setup script and extend to other machines #1706

merged 4 commits into from
Jun 26, 2023

Conversation

WalterKolczynski-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA commented Jun 22, 2023 •
edited
Loading

Description

In an effort to make workflow setup easier for users, a new script is added to handle setting up the lmod environment needed to run all the setup for checkout and experiment generation. Corresponding modules had already been created for hera and orion for CI testing; these have now been extended to cover other supported machines.

checkout.sh now uses this script as more recent versions of git are needed on some machines. (Build scripts already load their own modules, so nothing new is needed there.)

Documentation has been updated to recommend this method on all machines rather than the previous instructions.

Closes #1677

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

  • Clone, build, and expt setup on orion
  • Clone, build, and expt setup on hera
  • Clone, build, and expt setup on wcoss2
  • Clone, build, and expt setup on jet
  • Clone, build, and expt setup on s4

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation. I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • New and existing tests pass with my changes

@WalterKolczynski-NOAA
Create gw_setup script and extend to other machines
e301b81 
In an effort to make workflow setup easier for users, a new script is added
to handle setting up the lmod environment needed to run all the setup for
checkout and experiment generation. Corresponding modules had already been
created for hera and orion for CI testing; these have now been extended to
cover other supported machines.

`checkout.sh` now uses this script as more recent versions of `git` are needed
on some machines. (Build scripts already load their own modules, so nothing
new is needed there.)

Documentation has been updated to recommend this method on all machines rather
than the previous instructions.

Closes NOAA-EMC#1677
@WalterKolczynski-NOAA WalterKolczynski-NOAA added documentation Improvements or additions to documentation feature New feature or request labels Jun 22, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA self-assigned this Jun 22, 2023
@WalterKolczynski-NOAA
Copy link
Contributor Author

@DavidHuber-NOAA I have no way of testing this on S4. Can you test and make any necessary modifications?

@WalterKolczynski-NOAA
Copy link
Contributor Author

WalterKolczynski-NOAA commented Jun 22, 2023 •
edited
Loading

Will remain draft until personal testing is complete before launching CI.

@WalterKolczynski-NOAA
Copy link
Contributor Author

Updated documentation build: https://global-workflow--1706.org.readthedocs.build/en/1706/

aerorahul
aerorahul approved these changes Jun 22, 2023
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

@WalterKolczynski-NOAA
Update clone instructions for UFSDA
a66da57 
Updates documentation to add the checkout with UFSDA option.

Refs: NOAA-EMC#1677
@WalterKolczynski-NOAA WalterKolczynski-NOAA marked this pull request as ready for review June 26, 2023 06:34
@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Jun 26, 2023
@emcbot emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Jun 26, 2023
@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jun 26 01:36:18 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 01:38:09 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 02:53:02 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:11 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:15 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:19 CDT 2023 for experiment C96C48_hybatmDA

@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Jun 26, 2023
@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jun 26 06:36:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 06:43:05 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 07:54:59 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:03 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:06 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:09 UTC 2023 for experiment C96_atm3DVar

@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jun 26 01:36:18 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 01:38:09 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 02:53:02 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:11 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:15 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:19 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 03:35:14 CDT 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jun 26 06:36:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 06:43:05 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 07:54:59 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:03 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:06 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:09 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 08:42:10 UTC 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jun 26 01:36:18 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 01:38:09 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 02:53:02 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:11 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:15 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:19 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 03:35:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 05:07:06 CDT 2023
with 73 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jun 26 06:36:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 06:43:05 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 07:54:59 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:03 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:06 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:09 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 08:42:10 UTC 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 10:56:16 UTC 2023
with 73 successfully completed jobs

DavidHuber-NOAA
DavidHuber-NOAA requested changes Jun 26, 2023
View reviewed changes
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works well on S4 after adding two additional module loads. Thanks for developing this!

modulefiles/module_gwsetup.s4.lua Show resolved Hide resolved
@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jun 26 01:36:18 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 01:38:09 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 02:53:02 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:11 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:15 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:19 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 03:35:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 05:07:06 CDT 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Mon Jun 26 08:28:07 CDT 2023
with 308 successfully completed jobs

@emcbot emcbot added CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Jun 26, 2023
@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Mon Jun 26 01:36:18 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 01:38:09 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 02:53:02 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:11 CDT 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:15 CDT 2023 for experiment C96_atm3DVar
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 02:53:19 CDT 2023 for experiment C96C48_hybatmDA
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 03:35:14 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 05:07:06 CDT 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Mon Jun 26 08:28:07 CDT 2023
with 308 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jun 26 06:36:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 06:43:05 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 07:54:59 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:03 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:06 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:09 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 08:42:10 UTC 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 10:56:16 UTC 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Mon Jun 26 13:57:12 UTC 2023
with 308 successfully completed jobs

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jun 26, 2023
@emcbot
Copy link

emcbot commented Jun 26, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Mon Jun 26 06:36:26 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Mon Jun 26 06:43:05 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Mon Jun 26 07:54:59 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:03 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:06 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Mon Jun 26 07:55:09 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Mon Jun 26 08:42:10 UTC 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Mon Jun 26 10:56:16 UTC 2023
with 73 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Mon Jun 26 13:57:12 UTC 2023
with 308 successfully completed jobs

@aerorahul aerorahul merged commit cfc3d9c into NOAA-EMC:develop Jun 26, 2023
@aerorahul aerorahul mentioned this pull request Jun 27, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aerorahul aerorahul aerorahul approved these changes

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA Awaiting requested review from TerrenceMcGuinness-NOAA

@DavidHuber-NOAA DavidHuber-NOAA Awaiting requested review from DavidHuber-NOAA DavidHuber-NOAA is a code owner

Assignees

@WalterKolczynski-NOAA WalterKolczynski-NOAA

Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully documentation Improvements or additions to documentation feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

RTD out-of-date with module instructions for setup.
4 participants
@WalterKolczynski-NOAA @emcbot @aerorahul @DavidHuber-NOAA