Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spin up b #28

Draft
wants to merge 28 commits into
base: main
Choose a base branch
from
Draft

Spin up b #28

wants to merge 28 commits into from

Conversation

JhanSrbinovsky
Copy link
Collaborator

@JhanSrbinovsky JhanSrbinovsky commented Dec 22, 2024

Include my branch including CABLE updates intended for spin up run


🚀 The latest prerelease access-esm1p6/pr28-24 at f4b555c is here: #28 (comment) 🚀

Copy link

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

  • !bump major for feature releases
  • !bump minor for bugfixes

@blimlim blimlim marked this pull request as draft December 22, 2024 22:29
@blimlim
Copy link

blimlim commented Dec 22, 2024

!redeploy

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-2 with commit 4dbe6eb

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-2 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-2

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-2 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky
Copy link
Collaborator Author

In these 2 simulations we use the same code base for the LSM/UM7. The script run uses Leuning, no soil thermal fix etc. An older cable.nml. The payu run updates all of these plus includes wombat-lite, generic tracers, Dave's iceberg fix. There is also a difference in netcdf versions, compiler versions. So on the one hand this comparison is like weather from 2 different models. It's a daily average screen Temp after a year. On the other hand, such a large difference scares me that we need to do a payu/script comparrson. Which I really dont want to do. BUT I recall that NRI have done this an it was all acceptable. is this your recollection @rml599gh?

Screenshot 2024-12-23 at 4 15 39 pm

@rml599gh
Copy link

@JhanSrbinovsky - maybe let me know where the output is and I can have a look in the morning. Is this a single time? Looking at some of my runs, I am typically seeing +/- 15 degrees in the December monthly mean, so if your plot isn't a time average, then maybe the differences you are seeing are plausible.

@JhanSrbinovsky
Copy link
Collaborator Author

daily averages. honestly though its the 1st of the three Tscrn fields in the .pe file.. I'm assuming it goes mean, max, min. Either way its mean-mean, max-max or min-min

@blimlim
Copy link

blimlim commented Dec 23, 2024

On the Payu vs script run question – For the ESM1.5 release with payu we regularly checked whether the output from short PI simulations matched the PI-02 simulation on p73. I just checked my longer simulation of the NRI release, and into year 20 the two match:
download-47

I hadn't looked this far into the simulation before, but surprisingly it starts diverging, with what looks like noise appearing in May of the 20th year:
download-48
The difference then grows from here. It's surprising to me that the two diverged after being exactly the same for so long.

@JhanSrbinovsky
Copy link
Collaborator Author

That is strange. It doesnt sound like a physical difference, pretending the simulation is physically real. I suppose you would know if the script model had been restarted or perturbed on restart. BUT the results of the tiny perturbation we sometimes applied would've caused even greater divergence than you've shown here. I can vaguely make out coast lines there, suggesting it was a land thing. I dont really want to chase it. Almost pointless as the version of CABLE is different. BUT there was (perhaps still is) a dodgy variable in CABLE canopy. This was 10 years ago but I remember running the same single-site simulation back-to-back 10 times (or more) in order to follow it. Was your simulation running in monthly chunks, annual? It very well might have only been 1 timestep in the 20th year that started the divergence. It just snowballs after that.

Copy link

github-actions bot commented Jan 6, 2025

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-3 with commit bde6366

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-3 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-3

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-3 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

github-actions bot commented Jan 6, 2025

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-5 with commit d9eec7a

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-5 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-5

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-5 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

github-actions bot commented Jan 8, 2025

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-7 with commit 0380de3

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-7 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-7

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-7 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

github-actions bot commented Jan 8, 2025

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-8 with commit 9361163

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-8 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-8

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-8 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-20 with commit 4a78afe

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-20 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-20

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-20 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-21 with commit f692a96

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-21 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-21

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-21 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky
Copy link
Collaborator Author

Summary of cases running to date

where * represents the difference from the pre-ceding simmulation (@rml599gh )

PI-case2a

  • PI-case2a/config.yaml: - access-esm1p6/pr28-2
  • 77 years (/scratch/....access-esm/archive/PI-case2a-expt-11659a9b/)
  • pre-Xmas code
    • cable.nml
      cable_user%FWSOIL_SWITCH = 'standard' ! Controls root water uptake function
      cable_user%access13roots = .TRUE. ! TRUE uses prescribed froot values
      cable_user%SOIL_THERMAL_FIX = .FALSE. ! dy defualt

PI-case2b

  • PI-case2b/config.yaml: - access-esm1p6/pr28-2
  • 45 years
  • pre-Xmas code
  • cable.nml
    cable_user%FWSOIL_SWITCH = 'Haverd2013' ! Controls root water uptake function
    cable_user%access13roots = .TRUE. ! TRUE uses prescribed froot values
    cable_user%SOIL_THERMAL_FIX = .TRUE. ! TRUE used from CM2 onwards
    snmin = 0.11 ! depth at which snow switches to 3-layer

PI-case2c

  • 82 years
  • PI-case2c/config.yaml: - access-esm1p6/pr28-5
  • pre-Xmas code
  • cable.nml same as PI-case2b
  • revised prescribed root fractions in PFT params code

PI-case2d

  • 27 years (ATM)
  • PI-case2d/config.yaml: - access-esm1p6/pr28-21
  • same config as PI-case2b
  • post-Xmas code
  • cable.nml same as PI-case2b
  • see also UM7 #57

@JhanSrbinovsky JhanSrbinovsky marked this pull request as ready for review January 12, 2025 21:37
Copy link

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

  • !bump major for feature releases
  • !bump minor for bugfixes

@JhanSrbinovsky
Copy link
Collaborator Author

@harshula , @CodeGat , @blimlim This job which has been going over the weekend has today stopped. I went to (hopefully) restart the job. Looking t he most recent *.e file suggests that it couldn't find. the executable. I accidentally this morning clicked on "Ready to Review". I suspect that this has moved the pre-release pr28-21 module which I need. Is there a way to restore the status and this pr?

@CodeGat
Copy link
Member

CodeGat commented Jan 13, 2025

Hi @JhanSrbinovsky - unfortunately when you closed the PR it cleaned up the existing environments...
Usually, you would need to recreate the environments that you want as new commits on this branch (which would lead to a bunch of new enivronment numbers). But I can recreate the enivironments that were lost, if you can tell me which ones you would like :)

@CodeGat CodeGat marked this pull request as draft January 13, 2025 04:42
@JhanSrbinovsky
Copy link
Collaborator Author

Thanks @CodeGat, I'll bet back to you if we want to run it longer. We have 28-29 years of data to look at for this pr28-21. Of the 4 cases we have so far. This is the one that is likely non-negotiable and has to go in - containing improved science changes in the model.

@JhanSrbinovsky
Copy link
Collaborator Author

@CodeGat it occurred to me that I won't be able to reuse this PR #28 unless you rebuild it. I was only considering the specific revision. If it is easier just to copy/branch from here or even we can just do that.

@CodeGat
Copy link
Member

CodeGat commented Jan 15, 2025

I'm happy to rebuild any of the deployments that you want by hand, just so this PR remains clean. What were the versions that you want rebuilt? Any from pr28-* are fine.

@JhanSrbinovsky
Copy link
Collaborator Author

I'm happy to rebuild any of the deployments that you want by hand, just so this PR remains clean. What were the versions that you want rebuilt? Any from pr28-* are fine.

Apart from the first few maybe, they were identical except for a #comment, to trigger re-building of new developments in the UM7 repo. The feature branch referenced in pr28-21 is now merged to the main branch so the next change will probably be changing back to that - SO pr-28-21 is probably the b best place to start. Doesnt matter if revision numbers aren't preserved.

@CodeGat
Copy link
Member

CodeGat commented Jan 15, 2025

Alright, I'll add a commit that changes that UM7 reference back to the dev-access-esm1.6 branch, in that case, and won't resurrect any of the old builds that were cleaned up when the PR was closed - is that okay?

@JhanSrbinovsky
Copy link
Collaborator Author

Alright, I'll add a commit that changes that UM7 reference back to the dev-access-esm1.6 branch, in that case, and won't resurrect any of the old builds that were cleaned up when the PR was closed - is that okay?

Yeah sure - thanks

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-23 with commit 7575772

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-23 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-23

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-23 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr28-24 with commit f4b555c

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr28-24 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr28-24

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr28-24 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2024.11.27

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2024.11.27

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky JhanSrbinovsky deployed to Gadi Prerelease January 23, 2025 21:15 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants