Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit nested loop join record batch size #12634

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Limit nested loop join record batch size #12634

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b794a63
create smaller output batches in nested loop join
mhilton Sep 25, 2024
b5c7e72
buffer output batches in nested loop join
mhilton Sep 26, 2024
e0dadee
nested loop join batch size test
mhilton Sep 26, 2024
0fa1977
nested loop join issue 12633 test
mhilton Sep 27, 2024
2153150
fix clippy errors
mhilton Sep 27, 2024
8dd7412
inner join review suggestion
mhilton Sep 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion datafusion/core/tests/memory_limit/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,10 @@ async fn join_by_expression() {
TestCase::new()
.with_query("select t1.* from t t1 JOIN t t2 ON t1.service != t2.service")
.with_expected_errors(vec![
"Resources exhausted: Additional allocation failed with top memory consumers (across reservations) as: NestedLoopJoinLoad[0]",
"Resources exhausted: Additional allocation failed with top memory consumers (across reservations) as:",
"NestedLoopJoinLoad[0] consumed 0 bytes",
"NestedLoopJoin[0] consumed 0 bytes",
"NestedLoopJoinBuffer[0] consumed 0 bytes",
])
.with_memory_limit(1_000)
.run()
Expand Down
Loading