-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(PLATFORM-1077) Adding CI/CD capabilities and Unit Tests #76
Open
kbooker79
wants to merge
32
commits into
NOAA-PSL:master
Choose a base branch
from
kbooker79:platform-1077
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
a1b8d1f
test
kbooker79 75c2ff8
platform-1077, adding initial CI/CD for Jenkins
kbooker79 a6ba2a0
Removing typo.
kbooker79 ac9750c
Fixing other typo
kbooker79 22bcb97
Adding unit tests script
kbooker79 5705b9b
Adding label removal
kbooker79 0cf35af
Adding auth token from Jenkins.
kbooker79 697fdba
Checking env variables
kbooker79 2831d28
Troubleshooting
kbooker79 ba0d045
More troubleshooting
kbooker79 fd3b29a
Updating
kbooker79 3d62c00
Updating
kbooker79 43f20e1
Updating
kbooker79 967efff
Updating to use CHANGE_URL rather than GIT_URL
kbooker79 00d1289
Updating to use token for testing repo.
kbooker79 da3280c
Updating
kbooker79 bef267e
Updating auth
kbooker79 4e7af4c
PLATFORM-1077, removing pendatic comment file.
kbooker79 2c1b31d
Merge branch 'NOAA-PSL:master' into platform-1077
kbooker79 9c95c32
Update Jenkinsfile
zach1221 35c32c0
Update stochastic_physics.F90 fix do_sppt logic
zach1221 9f235a4
Create input.nml
zach1221 0de7a90
Update run_standalone.sh input-data and account
zach1221 ebb4758
Update standalone_stochy.F90 add new spp parameters and allocation
zach1221 7160eb4
Update Jenkinsfile
zach1221 b762de9
Update Jenkinsfile
zach1221 539923f
Update Jenkinsfile
zach1221 3f328d1
Update Jenkinsfile
zach1221 72fbbd9
Merge branch 'NOAA-PSL:master' into platform-1077
zach1221 3d4ee1c
Update input.nml enable SPP
zach1221 3849844
Update input.nml
zach1221 3e6b455
Update input.nml update n_var_spp
zach1221 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| pipeline { | ||
| agent none | ||
| environment { | ||
| GITHUB_TOKEN = credentials('GithubJenkinsNew') | ||
| AWS_ACCOUNT_ID = 211527314271 | ||
| } | ||
| stages { | ||
| stage('Run Unit Tests') { | ||
| agent { | ||
| label 'hera' | ||
| } | ||
| steps { | ||
| script { | ||
| cleanWs() | ||
| checkout scm | ||
| sh ''' | ||
| cd unit_tests | ||
| echo "Running Unit Tests on ${NODE_NAME}" | ||
| ./compile_standalone.hera_gnu | ||
| cp -r /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/FV3_input_data/INPUT INPUT | ||
| srun --label -A epic -n 24 standalone_stochy.x | ||
| mkdir stochy_out | ||
| mv workg* stochy_out | ||
| cd stochy_out/ | ||
| ls | ||
| ''' | ||
| } | ||
| } | ||
| } | ||
| } | ||
| post { | ||
| always { | ||
| node('built-in') { | ||
| sh ''' | ||
| GIT_OWNER=$(echo $CHANGE_URL | cut -d '/' -f4) | ||
| GIT_REPO_NAME=$(echo $CHANGE_URL | cut -d '/' -f5 | cut -d '.' -f1) | ||
| echo "Testing successfully completed...removing label(s)" | ||
| curl --silent -X DELETE -H "Accept: application/vnd.github.v3+json" -H "Authorization: Bearer ${GITHUB_TOKEN}" https://api.github.com/repos/${GIT_OWNER}/${GIT_REPO_NAME}/issues/${CHANGE_ID}/labels -d '{"labels":["run-unit-tests"]}' | ||
| ''' | ||
| } | ||
| } | ||
| success { | ||
| node('built-in') { | ||
| echo 'This will run only if successful.' | ||
| sh ''' | ||
| aws sns publish --topic-arn "arn:aws:sns:us-east-1:${AWS_ACCOUNT_ID}:Jenkins-CICD-Notifications" --region us-east-1 --message '{"version":"1.0","source":"custom","content":{"description":":sunny: Jenkins build *'"$JOB_NAME"' '"$BUILD_NUMBER"'* with *PR-'"$CHANGE_ID"'* *succeeded*"}}' | ||
| ''' | ||
| } | ||
| } | ||
| failure { | ||
| node('built-in') { | ||
| echo 'This will run only if the run was marked as unstable.' | ||
| sh ''' | ||
| aws sns publish --topic-arn "arn:aws:sns:us-east-1:${AWS_ACCOUNT_ID}:Jenkins-CICD-Notifications" --region us-east-1 --message '{"version":"1.0","source":"custom","content":{"description":":warning: Jenkins build *'"$JOB_NAME"' '"$BUILD_NUMBER"'* with *PR-'"$CHANGE_ID"'* *failed!*"}}' | ||
| ''' | ||
| } | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| &amip_interp_nml | ||
| interp_oi_sst = .true. | ||
| use_ncep_sst = .true. | ||
| use_ncep_ice = .false. | ||
| no_anom_sst = .false. | ||
| data_set = 'reynolds_oi', | ||
| date_out_of_range = 'climo', | ||
| / | ||
|
|
||
| &atmos_model_nml | ||
| blocksize = 32 | ||
| chksum_debug = .false. | ||
| dycore_only = .false. | ||
| fhout = 1 | ||
| fhmax = 800 | ||
| fhmaxhf = -1 | ||
| ccpp_suite = 'FV3_GFS_v15p2_coupled' | ||
| / | ||
|
|
||
| &diag_manager_nml | ||
| prepend_date = .F. | ||
| / | ||
|
|
||
| &fms_io_nml | ||
| checksum_required = .false. | ||
| max_files_r = 100, | ||
| max_files_w = 100, | ||
| / | ||
|
|
||
| &fms_nml | ||
| clock_grain = 'ROUTINE', | ||
| domains_stack_size = 40000000, | ||
| print_memory_usage = .false. | ||
| / | ||
|
|
||
| &fv_grid_nml | ||
| grid_file = 'INPUT/grid_spec.nc' | ||
| / | ||
|
|
||
| &fv_core_nml | ||
| layout = 1,4 | ||
| io_layout = 1,1 | ||
| npx = 97, | ||
| npy = 97, | ||
| ntiles = 6, | ||
| grid_type = -1 | ||
|
|
||
| / | ||
|
|
||
| &gfs_physics_nml | ||
| do_spp = .true. | ||
| do_sppt = .true. | ||
| do_shum = .true. | ||
| do_skeb = .true. | ||
| lndp_type = 2 | ||
| n_var_lndp = 2 | ||
| n_var_spp = 5 | ||
| / | ||
|
|
||
| &nam_sppperts | ||
| SPP_VAR_LIST: "pbl", "sfc", "mp", "rad", "gwd", "cu_deep" | ||
| SPP_MAG_LIST: 0.2, 0.2, 0.75, 0.2, 0.2 | ||
| SPP_LSCALE: 150000.0, 150000.0, 150000.0, 150000.0, 150000.0 | ||
| SPP_TSCALE: 21600.0, 21600.0, 21600.0, 21600.0, 21600.0 | ||
| SPP_SIGTOP1: 0.1, 0.1, 0.1, 0.1, 0.1 | ||
| SPP_SIGTOP2: 0.025, 0.025, 0.025, 0.025, 0.025 | ||
| SPP_STDDEV_CUTOFF: 1.5, 1.5, 2.5, 1.5, 1.5 | ||
| ISEED_SPP: 4, 5, 6, 7, 8 | ||
|
|
||
| / | ||
| &nam_stochy | ||
| SKEBNORM=1, | ||
| SKEB_NPASS=30, | ||
| SKEB_VDOF=5, | ||
| SKEB=0.5, | ||
| SKEB_TAU=2.16E4, | ||
| SKEB_LSCALE=1000.E3, | ||
| SHUM=0.01, | ||
| SHUM_TAU=21600, | ||
| SHUM_LSCALE=500000, | ||
| SPPT=0.8,0.4,0.1 | ||
| SPPT_TAU=21600,86400,216000 | ||
| SPPT_LSCALE=500000,1000000,2000000 | ||
| SPPT_LOGIT=.TRUE., | ||
| SPPT_SFCLIMIT=.TRUE., | ||
| ISEED_SHUM=1, | ||
| ISEED_SKEB=2, | ||
| ISEED_SPPT=3, | ||
| STOCHINI=.false. | ||
| / | ||
| &nam_sfcperts | ||
| lndp_type=2 | ||
| lndp_var_list='vgf','smc' | ||
| lndp_prt_list=1.0,0.5 | ||
| iseed_lndp=4,5 | ||
| lndp_tau= 86400, | ||
| lndp_lscale = 250000, | ||
| lndp_model_type = 1 | ||
| / | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here are the options for SPP (note that these are the generally recommended settings to run SPP at 3-km in the SRW App; I've not tested it with any other UFS App or at different grid resolutions):
DO_SPP: true
N_VAR_SPP: 5
SPP_VAR_LIST: "pbl", "sfc", "mp", "rad", "gwd"
SPP_MAG_LIST: 0.2, 0.2, 0.75, 0.2, 0.2
SPP_LSCALE: 150000.0, 150000.0, 150000.0, 150000.0, 150000.0
SPP_TSCALE: 21600.0, 21600.0, 21600.0, 21600.0, 21600.0
SPP_SIGTOP1: 0.1, 0.1, 0.1, 0.1, 0.1
SPP_SIGTOP2: 0.025, 0.025, 0.025, 0.025, 0.025
SPP_STDDEV_CUTOFF: 1.5, 1.5, 2.5, 1.5, 1.5
ISEED_SPP: 4, 5, 6, 7, 8
Note that EMC implemented SPP for the Grell-Frietas scheme, which can be turned on by adding "cu_deep" to the SPP_VAR_LIST variable and adding an additional value to the other namelist entries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, @JeffBeck-NOAA. I've added the changes to input.nml. I added "cu_deep" as an additional variable to SPP_VAR_LIST as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zach1221, did n_var_spp not work with 6?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JeffBeck-NOAA yes, looks like it failed with the below. I was going to try with 5, but Hera is down for maintenance currently.
At line 182 of file compns_stochy.F90 (unit = 21, file = 'input.nml')
1: Fortran runtime error: Cannot match namelist object name spp_var_list: