Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(backend): configurable project and location. Fixes #11212 #11211

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+6 −0
Open

feat(backend): configurable project and location. Fixes #11212 #11211

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
25bda6d
Add project and location
AlexanderLavelle Sep 13, 2024
d77686e
edit comma to align with list item formatting
AlexanderLavelle Sep 13, 2024
dd6cfb5
remember to add docstring and copy docstring hints from other files
AlexanderLavelle Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions components/google-cloud/google_cloud_pipeline_components/v1/custom_job/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ def create_custom_training_job_from_component(
labels: Optional[Dict[str, str]] = None,
persistent_resource_id: str = _placeholders.PERSISTENT_RESOURCE_ID_PLACEHOLDER,
env: Optional[List[Dict[str, str]]] = None,
project: Optional[str] = _placeholders.PROJECT_ID_PLACEHOLDER,
location: Optional[str] = "us-central1",
) -> Callable:
# fmt: off
"""Convert a KFP component into Vertex AI [custom training job](https://cloud.google.com/vertex-ai/docs/training/create-custom-job) using the [CustomJob](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.customJobs) API.
Expand Down Expand Up @@ -99,6 +101,8 @@ def create_custom_training_job_from_component(
labels: The labels with user-defined metadata to organize the CustomJob. See [more information](https://goo.gl/xmQnxf).
persistent_resource_id: The ID of the PersistentResource in the same Project and Location which to run. The default value is a placeholder that will be resolved to the PipelineJob [RuntimeConfig](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.pipelineJobs#PipelineJob.RuntimeConfig)'s persistent resource id at runtime. However, if the PipelineJob doesn't set Persistent Resource as the job level runtime, the placedholder will be resolved to an empty string and the custom job will be run on demand. If the value is set explicitly, the custom job will runs in the specified persistent resource, in this case, please note the network and CMEK configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.
env: Environment variables to be passed to the container. Takes the form `[{'name': '...', 'value': '...'}]`. Maximum limit is 100.
project: Project to run the CustomJob. Defaults to the project in which the PipelineJob is run, but may need specified depending on VPC.
location: Location to run the HyperparameterTuningJob in, defaults to `'us-central1'`.

Returns:
A KFP component with CustomJob specification applied.
Expand Down Expand Up @@ -203,6 +207,8 @@ def create_custom_training_job_from_component(
'labels': labels or {},
'encryption_spec_key_name': encryption_spec_key_name,
'persistent_resource_id': persistent_resource_id,
'project': project,
'location': location,
}

for param_name, default_value in custom_job_param_defaults.items():
Expand Down