Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sdk] parameterize PipelineTask's methods: set_cpu_requests, set_env_variables, set_memory_limit, etc #11140

Open
haiminh2001 opened this issue Aug 27, 2024 · 0 comments

Comments

@haiminh2001
Copy link

haiminh2001 commented Aug 27, 2024

Environment

  • KFP SDK version:
    2.8.0

Steps to reproduce

@pipeline(name='test_cpu_request'):
  def pipeline(cpu_request: str):

    task.set_cpu_request(cpu_request)

Error:

Traceback (most recent call last):
  File "pipeline.py", line 13, in <module>
    def pipeline(train_image_uri: str,
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_context.py", line 65, in pipeline
    return component_factory.create_graph_component_from_func(
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/component_factory.py", line 673, in create_graph_component_from_func
    return graph_component.GraphComponent(
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/graph_component.py", line 58, in __init__
    pipeline_outputs = pipeline_func(*args_list)
  File "pipeline.py", line 72, in pipeline
    .set_cpu_request(cpu_request)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 56, in wrapper
    return method(self, *args, **kwargs)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 362, in set_cpu_request
    cpu = self._validate_cpu_request_limit(cpu)
  File "/miniconda3/envs/dev-3-10/lib/python3.10/site-packages/kfp/dsl/pipeline_task.py", line 340, in _validate_cpu_request_limit
    if re.match(r'([0-9]*[.])?[0-9]+m?$', cpu) is None:
  File "/miniconda3/envs/dev-3-10/lib/python3.10/re.py", line 190, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or bytes-like object

Expected result

The method set_cpu_requests is expecting a hard-coded string, but I think the cpu requests should be an adjustable param that may vary among runs.


Impacted by this bug? Give it a 👍.

@haiminh2001 haiminh2001 changed the title [sdk] set_cpu_requests does not accept parameters. [sdk] parameterize set_cpu_requests, set_env_variables, set_memory_limit, etc Sep 10, 2024
@haiminh2001 haiminh2001 changed the title [sdk] parameterize set_cpu_requests, set_env_variables, set_memory_limit, etc [sdk] parameterize PipelineTask's methods: set_cpu_requests, set_env_variables, set_memory_limit, etc Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant