Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to running ray for bulk experiments #333

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Updates to running ray for bulk experiments #333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
013ad8d
A pretty decent cut of parallelized Ray calc for score testing
ZmeiGorynych Dec 19, 2024
69ad99c
Merge branch 'py-why:main' into ray_nice
ZmeiGorynych Dec 20, 2024
145e6cd
Simple retry mechanism for ray.get()
whimo Dec 25, 2024
4b54349
Persistent runner
whimo Dec 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions notebooks/RunExperiments/cluster_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ cluster_name: default

# The maximum number of workers nodes to launch in addition to the head
# node.
max_workers: 2
max_workers: 20

# The autoscaler will scale up the cluster faster with higher upscaling speed.
# E.g., if the task requires adding more nodes then autoscaler will gradually
Expand Down Expand Up @@ -41,7 +41,7 @@ idle_timeout_minutes: 5
provider:
type: gcp
region: us-west1
availability_zone: us-west1-a
availability_zone: us-west1-b
project_id: motleys # Globally unique project id

# How Ray will authenticate with newly launched nodes.
Expand All @@ -60,7 +60,7 @@ auth:
available_node_types:
ray_head_default:
# The resources provided by this node type.
resources: {"CPU": 2}
resources: {"CPU": 0}
# Provider-specific config for the head node, e.g. instance type. By default
# Ray will auto-configure unspecified fields such as subnets and ssh-keys.
# For more documentation on available fields, see:
Expand Down Expand Up @@ -93,15 +93,15 @@ available_node_types:
min_workers: 1
# The maximum number of worker nodes of this type to launch.
# This takes precedence over min_workers.
max_workers: 5
max_workers: 20
# The resources provided by this node type.
resources: {"CPU": 2}
# Provider-specific config for the head node, e.g. instance type. By default
# Ray will auto-configure unspecified fields such as subnets and ssh-keys.
# For more documentation on available fields, see:
# https://cloud.google.com/compute/docs/reference/rest/v1/instances/insert
node_config:
machineType: n1-standard-2
machineType: n2-highmem-2
disks:
- boot: true
autoDelete: true
Expand Down
Loading