[Enhancement] Support Generated Column rewrite in complex Query #50398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![starrocks-cr[bot]](https://avatars.githubusercontent.com/in/404334?s=60&v=4){kind=small}
|
|
||
| GeneratedColumnVisitor visitor = new GeneratedColumnVisitor(); | ||
| visitor.process(node, viewScope); | ||
|
|
||
| return viewScope; | ||
| } | ||
|
|
| public Map<Expr, SlotRef> getGeneratedExprToColumnRef() { | ||
| return generatedExprToColumnRef; | ||
| } | ||
|
|
||
| @Override | ||
| public <R, C> R accept(AstVisitor<R, C> visitor, C context) { | ||
| return visitor.visitRelation(this, context); |
There was a problem hiding this comment.
The most risky bug in this code is:
Potential thread-safety issues with generatedExprToColumnRef.
You can modify the code like this:
import java.util.Collections;
import java.util.concurrent.ConcurrentHashMap;
//...
private Map<Expr, SlotRef> generatedExprToColumnRef = new ConcurrentHashMap<>();
public void setGeneratedExprToColumnRef(Map<Expr, SlotRef> generatedExprToColumnRef) {
this.generatedExprToColumnRef = Collections.synchronizedMap(generatedExprToColumnRef);
}
srlch
force-pushed
the
gc_column_rewrite2
branch
3 times, most recently
from
784e377
to
7583580
Compare
srlch
changed the title
srlch
force-pushed
the
gc_column_rewrite2
branch
2 times, most recently
from
f8b5ab8
to
05deed6
Compare
Seaven
previously approved these changes
Sep 6, 2024
In this pr:
1. Refactor Generated Column rewrite code:
a. Collect generated column expr mapping From QueryAnalyzer.
b. Introduce ReplaceScalarOperatorRule to replace the ScalarOperator match the
generated column expression.
2. Supprot View, JOIN with Subquery and CTE for Generated Column rewriting.
3. Support simple equivalent expression rewrite for generated column.
Note: All query structure support Generated Column rewriting obey a simple rule:
the Generated Column and its ref columns output directly into the outer scope.
Signed-off-by: srlch <[email protected]>
srlch
force-pushed
the
gc_column_rewrite2
branch
from
f145e83
to
3068c59
Compare
|
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[FE Incremental Coverage Report]✅ pass : 127 / 132 (96.21%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
Seaven
approved these changes
Sep 6, 2024
murphyatwork
approved these changes
Sep 9, 2024
|
@Mergifyio backport branch-3.3 |
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
Sep 9, 2024
Signed-off-by: srlch <[email protected]> (cherry picked from commit 96b97a7) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/sql/analyzer/QueryAnalyzer.java
42 tasks
srlch
added a commit
to srlch/starrocks
that referenced
this pull request
Sep 9, 2024
…Rocks#50398) Signed-off-by: srlch <[email protected]>
srlch
added a commit
to srlch/starrocks
that referenced
this pull request
Sep 9, 2024
…Rocks#50398) Signed-off-by: srlch <[email protected]>
srlch
added a commit
to srlch/starrocks
that referenced
this pull request
Sep 9, 2024
…Rocks#50398) Signed-off-by: srlch <[email protected]>
srlch
added a commit
to srlch/starrocks
that referenced
this pull request
Sep 9, 2024
…port StarRocks#50398) Signed-off-by: srlch <[email protected]>
murphyatwork
pushed a commit
that referenced
this pull request
Sep 9, 2024
…port #50398) (#50830) Signed-off-by: srlch <[email protected]>
murphyatwork
pushed a commit
that referenced
this pull request
Sep 9, 2024
…port #50398) (#50831) Signed-off-by: srlch <[email protected]>
HangyuanLiu
pushed a commit
to HangyuanLiu/starrocks
that referenced
this pull request
Sep 12, 2024
…Rocks#50398) Signed-off-by: srlch <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this pr:
a. Collect generated column expr mapping From QueryAnalyzer.
b. Introduce ReplaceScalarOperatorRule to replace the ScalarOperator match the generated column expression.
Note: All query structure support Generated Column rewriting obey a simple rule: the Generated Column and its ref columns output directly into the outer scope.
Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: