[FLINK-37720][table-planner] Enable sink reuse table optimizer by default and apply plan changes to existing test cases

[xiangyuf]

What is the purpose of the change

Enable sink reuse table optimizer by default and apply plan changes to existing test cases

Brief change log

Enable sink reuse table optimizer by default and apply plan changes to existing test cases

Verifying this change

This change is already covered by existing tests

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no

[flinkbot]

CI report:

• ffec5e3 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

@xiangyuf I am wondering what the migration implications are to change this to be default. In general it is not good practise to change defaults, but I see the flip has not being fully implemented so maybe this is not a concern. Please can you let me know your thinking; maybe this change in behaviour is always desirable and would not be seen as a break for existing users?

[xiangyuf]

@xiangyuf I am wondering what the migration implications are to change this to be default. In general it is not good practise to change defaults, but I see the flip has not being fully implemented so maybe this is not a concern. Please can you let me know your thinking; maybe this change in behaviour is always desirable and would not be seen as a break for existing users?

Hi David, as discussed in this thread https://lists.apache.org/thread/r1wo9sf3d1725fhwzrttvv56k4rc782m, the sink reuse is enabled by default in the first place for following reason:
"taking sink reuse as a long awaited feature with significant benefits for users, we choose to enable this in the first place. Similar features like table.optimizer.reuse-sub-plan-enabled and table.optimizer.reuse-source-enabled are also enabled by default. From this point of view, sink reuse should be the same."

Also, this PR is a draft PR, I need to change several existing IT cases to pass the CI.

[xiangyuf]

@davidradl Hi, this pr is ready to merge now. Welcome any reviews.

[xiangyuf]

@xuyangzhong @reswqa Kindly remind for review.