Consolidate manifests #3547
Consolidate manifests #3547
Conversation
Sbozzolo
force-pushed
the
gb/manifests
branch
from
8797e16 to
a315d0e
Compare
Sbozzolo
force-pushed
the
gb/manifests
branch
2 times, most recently
from
636928b to
206c33e
Compare
There was a problem hiding this comment.
I'm sorry about the merge conflict, I thought I had merged this before #3551, but I was mistaken.
This looks good to me, feel free to merge after rebasing, and thanks for updating this!
Manifests.toml are a burden to maintain and add a lot of git diff noise. The one benefit of Manifests.toml is that they help with tracking version dependencies. However: - For `docs`, the precise version of dependencies is not important, there is pretty much no version-dependent code being executed there - `perf` and `examples` can be consolidated into a single `.buildkite` environment. This environment only has to have one manifest checked, the one for which buildkite is run (and not one for each version of Julia). - The `perf` environment can be removed all together. It is a more niche environment: if reproducing a specific result is required, developers can use the `.buildkite` Manifest. If not, developers can also consider keeping the relevant tools (mainly JET) in the base environment. This is the same pattern we have in other repos such as ClimaCore.
charleskawczynski
force-pushed
the
gb/manifests
branch
from
206c33e to
624be38
Compare
|
I went ahead and rebased! |
|
Includes #3548 |
Sbozzolo
force-pushed
the
gb/manifests
branch
3 times, most recently
from
3529619 to
3932f19
Compare
Sbozzolo
force-pushed
the
gb/manifests
branch
3 times, most recently
from
5bb811c to
e97cee6
Compare
Sbozzolo
force-pushed
the
gb/manifests
branch
2 times, most recently
from
d2646d2 to
499f419
Compare
| @@ -1,16 +1,17 @@ | |||
| moist: "equil" | |||
| apply_limiter: true | |||
| z_elem: 25 | |||
| z_elem: 11 | |||
There was a problem hiding this comment.
Why is this changing?
There was a problem hiding this comment.
I think it's because this job failed: https://buildkite.com/clima/climaatmos-ci/builds/22496#0194af81-d116-4343-92f6-9b753e5763aa, which is likely related to the climatimestepper update.
You changed the wrong config file though I think?
There was a problem hiding this comment.
Ah, ok. I think it might be better to just increase the mem request
There was a problem hiding this comment.
Using low resolution is not really meaningful for performance monitoring jobs, so I think it'd be better to remove it instead of lowering the resolution.
There was a problem hiding this comment.
That's fine with me. I prefer to make it soft fail though so we will remember to come back to fix this.
There was a problem hiding this comment.
Ah, ok. I think it might be better to just increase the mem request
The memory is GPU memory, so we would need to request a different type of GPU, which would increase the queue time significantly.
There was a problem hiding this comment.
I changed it to a soft fail
There was a problem hiding this comment.
Sounds good. I think there is something wrong, though. The fieldvector is being allocated here, and it's definitely not that large, so I don't know why we're OOMing. Maybe the GPU is not freeing memory as it should be.
Sbozzolo
force-pushed
the
gb/manifests
branch
from
499f419 to
062d71c
Compare
Sbozzolo
force-pushed
the
gb/manifests
branch
from
062d71c to
5048573
Compare
|
It passed everything in the merge queue, but somehow the format check didn't start. |
I have an idea of what it might be. Let me start by merging this manually |
|
@Sbozzolo why is there a Manifest in the .buildkite/ directory but no Project? |
It was probably deleted by accident. We do need one becuase:
I'll open a PR. |
Manifests.toml are a burden to maintain and add a lot of git diff noise. The one benefit of Manifests.toml is that they help with tracking version dependencies.
However:
docs, the precise version of dependencies is not important, there is pretty much no version-dependent code being executed thereperfandexamplescan be consolidated into a single.buildkiteenvironment. This environment only has to have one manifest checked, the one for which buildkite is run (and not one for each version of Julia).perfenvironment can be removed all together. It is a more niche environment: if reproducing a specific result is required, developers can use the.buildkiteManifest. If not, developers can also consider keeping the relevant tools (mainly JET) in the base environment.This is the same pattern we have in other repos such as ClimaCore.
(Builds off #3546)
Closes #3548