[Enhancement] Simplify date/datetime column predicate in certain cases

[kaijianding]

Why I'm doing:

Some user use date/datetime column as string, in this case sparse index is not used.
This pr will do these rewrite

 --if t is date
 date_format(t, '%Y%m%d') >= '20230327' -> `t` >= '20230327'
 date_format(t, '%Y-%m-%d') >= '2023-03-27' -> `t` >= '2023-03-27'
 substr(t, 1, 10) >= '2023-03-27' -> `t` >= '2023-03-27'
 substring(t, 1, 10) >= '2023-03-27' -> `t` >= '2023-03-27'
 replace(substring(t, 1, 10), "-", "") >= '20230327' -> `t` >= '20230327'
 
 --if t is datetime
 date_format(t, '%Y-%m-%d') >/>=/< '2023-03-27' -> `t` >/>=/< '2023-03-27'
 date_format(t, '%Y-%m-%d') <= '2023-03-27' -> `t` < days_add('2023-03-27', 1)

What I'm doing:

Fixes #issue

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 3.3
  • 3.2
  • 3.1
  • 3.0
  • 2.5

[starrocks-cr]

The most risky bug in this code is:
Mismatch of function name in ReplaceAndSubstrExtractor#check when checking the empty string.
You can modify the code like this:

+        @Override
+        public boolean check() {
+            if (!(call.getChild(0) instanceof CallOperator && isSubstrFn((CallOperator) call.getChild(0)))) {
+                return false;
+            }
+            if (!call.getChild(1).isConstantRef() || !((ConstantOperator) call.getChild(1)).getVarchar().equals("-")) {
+                return false;
+            }
-            if (!call.getChild(2).isConstantRef() || !((ConstantOperator) call.getChild(2)).getVarchar().isEmpty()) {
+            if (!call.getChild(2).isConstantRef() || !((ConstantOperator) call.getChild(2)).getVarchar().equals("")) {
                return false;
+            }
+            // yyyyMMdd
+            if (value.getVarchar().length() != 8) {
+                return false;
+            }
+            return new SubstrExtractor((CallOperator) call.getChild(0), value, 8).check();
+        }

Explanation:
The original condition for verifying that the third argument to replace is an empty string used .isEmpty() method. However, to ensure consistency and correctness with the other comparisons, we should use .equals("") to explicitly check for an empty string.

[packy92]

please test date_format(t, '%Y%m%d') >= '20230400'. If the right constant value is a illegal date string, is the rewriten result same with before?

[kaijianding]

please test date_format(t, '%Y%m%d') >= '20230400'. If the right constant value is a illegal date string, is the rewriten result same with before?

PREDICATES: date_format(2: t, '%Y%m%d') >= '2023030700' is the result, no rewritten.

But I will use regex to verify the right constant.

[packy92]

please test date_format(t, '%Y%m%d') >= '20230400'. If the right constant value is a illegal date string, is the rewriten result same with before?

PREDICATES: date_format(2: t, '%Y%m%d') >= '2023030700' is the result, no rewritten.

But I will use regex to verify the right constant.

I means clients may use 20230400 as the right constant value, this value seems to pass the regex check. But it's not a valid date value. Does the rule will rewrite it to dt >= cast('20230400' as date) ? If yes, the right cast returns a null may cause different behavior from before.

[kaijianding]

@packy92 U r right, if right constant is not valid date, the result is not consitent with that before rewrite. I will verify the right constant is valid date

[sonarcloud]

Quality Gate passed

Issues
21 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[github-actions]

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[FE Incremental Coverage Report]

✅ pass : 88 / 91 (96.70%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/sql/optimizer/rewrite/scalar/SimplifiedDateColumnPredicateRule.java	88	91	96.70%	[150, 222, 225]

[github-actions]

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[kaijianding]

@packy92 any more comment?

[github-actions]

@Mergifyio backport branch-3.3

[mergify]

backport branch-3.3

✅ Backports have been created

• #50960 [Enhancement] Simplify date/datetime column predicate in certain cases (backport #50643) has been created for branch branch-3.3