Skip to content

Commit

Permalink
Update docs/source/user-guide/lazy/execution.md
Browse files Browse the repository at this point in the history
Co-authored-by: Rodrigo Girão Serrão <[email protected]>
  • Loading branch information
edwinvehmaanpera and rodrigogiraoserrao authored Sep 24, 2024
1 parent 0ac3f52 commit ea5cd0c
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion docs/source/user-guide/lazy/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ We look at [streaming in more detail here](streaming.md).

While you're writing, optimizing or checking your query on a large dataset, querying all available data may lead to a slow development process.

You can instead limit the number of scanned partitions or use .head early in the query when testing. Keep in mind that aggregations and filters may behave unpredictably on subsets of data.
Instead, you can scan a subset of your partitions or use `.head`/`.collect` at the beginning and end of your query, respectively.
Keep in mind that the results of aggregations and filters on subsets of your data may not be representative of the result you would get on the full data.

{{code_block('user-guide/lazy/execution','partial',['scan_csv','collect','head'])}}

Expand Down

0 comments on commit ea5cd0c

Please sign in to comment.