Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Support List partition pruning #2660

Open
shiyuhang0 opened this issue Mar 22, 2023 · 0 comments
Open

[Draft] Support List partition pruning #2660

shiyuhang0 opened this issue Mar 22, 2023 · 0 comments

Comments

@shiyuhang0
Copy link
Member

shiyuhang0 commented Mar 22, 2023

Is your feature request related to a problem? Please describe.

Support List partition pruning.

Describe the solution you'd like

I write a demo for it in my branch https://github.com/pingcap/tispark/compare/master...shiyuhang0:tispark:listpar_prune?expand=1.
It works but it is only a draft . Things missing:

  • design doc and user doc
  • partition pruning with IN expression
  • forbid partition pruning in some expressions (like >,<) when there have functions in partition expression. For example, we can not perform partition pruning when the partition expression is year(column) and the predicate is column > '2000-01-01'. because we will rewrite as year(column) > 2000, however year('2000-01-02') is also 2000. This may cause an error when choosing the partition. (TiDB also does not support)
  • fully tests (including different expressions and different data types)
  • Optimize the code. There are some codes copied from the range partition pruning, I think we can reuse them; I think we can also combine the logic of list partition pruning and list column partition pruning; more clear comments on the code

I currently don't have time to work on this any further for at least a couple of weeks. but feel free to use this branch as the basis for proper implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant