Skip to content

Commit

Permalink
Merge pull request #3577 from szarnyasg/nits-20240909b
Browse files Browse the repository at this point in the history
Nit
  • Loading branch information
szarnyasg committed Sep 9, 2024
2 parents c5efbbc + 7b18fe4 commit 19d0265
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion _posts/2024-09-09-announcing-duckdb-110.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ This release adds a feature where DuckDB [automatically decides](https://github.

### Parallel Streaming Queries

[**Parallel Result Streaming.**](https://github.com/duckdb/duckdb/pull/11494). DuckDB has two different methods for fetching results: *materialized* results, and *streaming* results. Materialized results fetch all of the data that is present in a result at once, and return it. Streaming results instead allow iterating over the data in incremental steps. Streaming results are critical when working with large result sets – as they do not require the entire result set to fit in memory. However, in previous releases, the final streaming phase was limited to a single thread.
[**Parallel Result Streaming.**](https://github.com/duckdb/duckdb/pull/11494) DuckDB has two different methods for fetching results: *materialized* results, and *streaming* results. Materialized results fetch all of the data that is present in a result at once, and return it. Streaming results instead allow iterating over the data in incremental steps. Streaming results are critical when working with large result sets – as they do not require the entire result set to fit in memory. However, in previous releases, the final streaming phase was limited to a single thread.

Parallelism is critical for obtaining good query performance on modern hardware, and this release adds support for parallel streaming of query results. The system will use all available threads to fill up a query result buffer of a limited size (a few megabytes). When data is consumed from the result buffer, the threads will restart and start filling up the buffer again. The size of the buffer can be configured through the `streaming_buffer_size` parameter.

Expand Down

0 comments on commit 19d0265

Please sign in to comment.