Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to request extra boot disk space for Batch Tasks #57

Merged
merged 2 commits into from
Jan 12, 2024
Merged

Add option to request extra boot disk space for Batch Tasks #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7a6fe9a
Add option to request extra boot disk space for Batch Tasks
lathanh Jan 10, 2024
fd0d8a2
Add option to request extra boot disk space for Batch Tasks
lathanh Jan 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions buildstockbatch/gcp/gcp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -572,6 +572,7 @@ def start_batch_job(self, batch_info):
resources = batch_v1.ComputeResource(
cpu_milli=1000 * job_env_cfg.get("vcpus", 1),
memory_mib=job_env_cfg.get("memory_mib", 1024),
boot_disk_mib=job_env_cfg.get("boot_disk_mib", None),
)

# Give three minutes per simulation, plus ten minutes for job overhead
Expand Down
1 change: 1 addition & 0 deletions buildstockbatch/schemas/v0.3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ gcp-ar-spec:
gcp-job-environment-spec:
vcpus: int(min=1, max=224, required=False)
memory_mib: int(min=512, required=False)
boot_disk_mib: int(required=False)
lathanh marked this conversation as resolved.
Show resolved Hide resolved
machine_type: str(required=False)
use_spot: bool(required=False)

Expand Down
7 changes: 7 additions & 0 deletions docs/project_defn.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,12 @@ using `GCP Batch <https://cloud.google.com/batch>`_ and `Cloud Run <https://clou
* ``vcpus``: Number of CPUs to allocate for running each simulation. Default: 1.
* ``memory_mib``: Amount of RAM memory needed for each simulation in MiB. Default: 1024.
For large multifamily buildings this works better if set to 2048.
* ``boot_disk_mib``: Optional. Extra boot disk size in MiB for each task. This affects how
large the boot disk will be (see the `Batch OS environment docs`_ for details) of the
machine(s) running simulations (which is the disk used by simulations). This will likely need
to be set to at least 2,048 if more than 8 simulations will be run in parallel on the same
machine (i.e., when vCPUs per machine_type ÷ vCPUs per sim > 8). Default: None (which should
result in a 30 GB boot disk according to the docs linked above).
* ``machine_type``: GCP Compute Engine machine type to use. If omitted, GCP Batch will
choose a machine type based on the requested vCPUs and memory. If set, the machine type
should have at least as many resources as requested for each simulation above. If it is
Expand All @@ -315,6 +321,7 @@ using `GCP Batch <https://cloud.google.com/batch>`_ and `Cloud Run <https://clou

.. _GCP's default behavior: https://cloud.google.com/python/docs/reference/batch/latest/google.cloud.batch_v1.types.TaskGroup
.. _job limits: https://cloud.google.com/batch/quotas
.. _Batch OS environment docs: https://cloud.google.com/batch/docs/vm-os-environment-overview#default
.. _Number of CPUs: https://cloud.google.com/run/docs/configuring/services/cpu
.. _Amount of RAM: https://cloud.google.com/run/docs/configuring/services/memory-limits

Expand Down