Enable or Disable cache for the Compiler

[diegolovison]

Description of your changes:

Support for enabling or disabling caching for the entire pipeline is being introduced. This functionality mirrors the one provided by the client::run method (see https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/client/client.py#L707). The aim is to offer a similar feature using the same parameter names to prevent confusion.

Note that this PR does not address changing the default value for structures.TaskSpec::enable_caching. That functionality will be introduced in a separate PR and is unrelated to this one.

Checklist:

• You have signed off your commits
• The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[google-oss-prow]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign chensun for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• sdk/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[DharmitD]

Make sure to use is_flag functionality here, to maintain precedence (just like type_check for instance).

[DharmitD]

Change this to --disable-default-caching to maintain precedence with other flags in this file. (like the type_check one)

[DharmitD]

once you update as per https://github.com/kubeflow/pipelines/pull/11209/files#r1759241709,
you would need to change this as well to something like:
enable_caching=not disable_caching_default

[DharmitD]

Not sure if we want to override compile time settings, we just want to have an option to change the caching default. When we add the flag, this needs to happen for instance:

Assume that we set our flag such that default caching is disabled.
When we are done with this feature, this is expected:
@dsl.pipeline(name='iris-training-pipeline')
def my_pipeline():
   task_1 = create_dataset()
   task_2 = create_dataset()
   task_3 = create_dataset()
   task_3.set_caching_options(True)
   # tasks 1 and 2 don’t try to use the cache. Task 3 does try to use the cache.

However, with your update of overriding in the client directly, task 3's caching might also get set to False. (Not entirely sure, something you would have to test and find out @diegolovison )

@gregsheremeta could elaborate