allow using on-demand instead of spot only #622
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@markblee could you review please? I need this in order to test on internal on-demand capacity. |
|
@Ethanlm would love your thoughts too |
markblee
reviewed
Aug 22, 2024
| @@ -508,8 +516,9 @@ def _build_pod(self) -> Nested[Any]: | |||
| logging.info("Found tier=%s in env. Using reservation=%s", tier, cfg.reservation) | |||
| selector.update({"cloud.google.com/reservation-name": cfg.reservation}) | |||
| else: | |||
There was a problem hiding this comment.
What about using tier == "1" for spot and tier == "2" for on-demand? You can configure the quota file s.t. there's only on-demand quota like
[[total_resources]]
v5litepod = 0
[[total_resources]]
v5litepod = 0
[[total_resources]]
v5litepod = X
...
Which corresponds to 0 quota for tier 0/1 and X quota on tier 2. More generally we can also make the logic for tier -> selector mapping configurable, for more flexible schemes. WDYT?
ruomingp
approved these changes
Aug 28, 2024
Comment on lines
+519
to
+521
| if not cfg.enable_ondemand: | ||
| logging.info("Found tier=%s in env. Using spot quota", tier) | ||
| selector.update({"cloud.google.com/gke-spot": "true"}) |
There was a problem hiding this comment.
Is there a unittest that can be updated?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #621