Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: adaptive cost model functionality working #174

Merged
merged 31 commits into from
Apr 30, 2024
Merged

feat: adaptive cost model functionality working #174

merged 31 commits into from
Apr 30, 2024

Conversation

wangpatrick57
Copy link
Member

Summary: We are now executing the query in DataFusion after getting the estimated cardinality, so the functionality of the adaptive cost model is working.

Demo:
This is joblight 4a,4b,4c without --adaptive:
Screenshot 2024-04-30 at 10 07 34

This is joblight 4a,4b,4c with --adaptive:
Screenshot 2024-04-30 at 10 07 50

Details:

  • We had to use our own optimizer instead of DataFusion's so that we could correctly take the results and put them back into the memo.

wangpatrick57 and others added 30 commits April 27, 2024 14:53
@wangpatrick57
added actual set of working job light queries
3c7aba5
@wangpatrick57
fmt and clip
c2c99d8
@wangpatrick57
changed precision of mg and hll, getting us from 35 -> 40 queries ahead
ec81ca6
@wangpatrick57
test_inner_redundant_predicate -> test_add_edge_to_multi_equal_graph_…
af00dd6 
…which_maintains_mst
@wangpatrick57
wrote test_three_table_join_for_join1_on_cond test. not passing yet
28a3d26
@wangpatrick57
turned get_join_selectivity_from_most_selective_predicates into get_j…
9104a81 
…oin_selectivity_from_most_selective_columns
@wangpatrick57
fixed bug of adding new predicate that touches one col of existing pred
91791f7
@wangpatrick57
generalized three table join test to allow for arbitrary initial join…
d9a8f29 
… conds
@wangpatrick57
added test_join_which_connects_two_components_together
42a9162
@wangpatrick57
fmt and clip
0474fcd
@wangpatrick57
changed comment and name of what is now get_join_selectivity_adjustme…
4fc54c1 
…nt_when_adding_to_multi_equality_graph()
@wangpatrick57
inclusion principle comment
e5b2adf
@wangpatrick57
poc working group card caching
f9dda6b
@wangpatrick57
skeleton for load data or create table
90105b2
@wangpatrick57
wrote create tables for job
9974637
@Gun9niR
load job data no stats
2258af5
@wangpatrick57
added execute query
2d0b848
@wangpatrick57
Merge remote-tracking branch 'refs/remotes/origin/phw0/df-execute' in…
f96224b 
…to phw2/df-execute
@wangpatrick57
moved execute query
ef458de
@wangpatrick57
Merge branch 'main' into phw2/df-execute
ececf10
@wangpatrick57
integrated adaptive option with code
ab20bc9
@wangpatrick57
added adaptive to which_queries_work.sh
0eed5f2
@wangpatrick57
removed debug from log_explain
349b6b5
@Gun9niR
specify schema when creating external table
7750438
@Gun9niR
Merge branch 'phw2/df-execute' of https://github.com/cmu-db/optd into…
11e00fb 
… phw2/df-execute
@Gun9niR
use register_csv
2fbfe9b
@Gun9niR
misc
4ce15c9
@wangpatrick57
now creating table in temporary schema
3b82853
@wangpatrick57
fixed job
ca9ddc1
@wangpatrick57
moved execute to after estcard
37197af
@wangpatrick57 wangpatrick57 marked this pull request as ready for review April 30, 2024 14:11
@wangpatrick57 wangpatrick57 merged commit a368616 into main Apr 30, 2024
1 check passed
@wangpatrick57 wangpatrick57 deleted the phw2/df-execute branch April 30, 2024 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wangpatrick57 @Gun9niR