Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parallelize Stateless (CSV/JSON) File Write Serialization #7452

Merged
merged 15 commits into from
Sep 9, 2023
Merged

Parallelize Stateless (CSV/JSON) File Write Serialization #7452

merged 15 commits into from
Sep 9, 2023

Commits on Aug 30, 2023

  1. implement tokio task spawning

    @devinjdangelo
    devinjdangelo committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    86aba96 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. improve error handling

    @devinjdangelo
    devinjdangelo committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    d793ec5 View commit details
    Browse the repository at this point in the history

  2. parallelize single_file true

    @devinjdangelo
    devinjdangelo committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    b0d5c1e View commit details
    Browse the repository at this point in the history

  3. cargo fmt

    @devinjdangelo
    devinjdangelo committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    1dd0e98 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. Update datafusion/core/src/datasource/file_format/write.rs 

    Co-authored-by: Metehan Yıldırım <[email protected]>
    @devinjdangelo @metesynnada
    devinjdangelo and metesynnada authored Sep 5, 2023
    Configuration menu
    Copy the full SHA
    bcb2baf View commit details
    Browse the repository at this point in the history

  2. add channel

    @devinjdangelo
    devinjdangelo committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    584d925 View commit details
    Browse the repository at this point in the history

  3. cargo fmt

    @devinjdangelo
    devinjdangelo committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    6abbdfc View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. yield_now if input is fifo unbounded

    @devinjdangelo
    devinjdangelo committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    64d7692 View commit details
    Browse the repository at this point in the history

  2. fix unbounded option parse

    @devinjdangelo
    devinjdangelo committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    cf1b1a5 View commit details
    Browse the repository at this point in the history

  3. cargo fmt

    @devinjdangelo
    devinjdangelo committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    d8b84dd View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Remove redundant source

    @alamb
    alamb committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    e6f1b2f View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'apache/main' into parallelize_csv_json_… 

    …writes
    @alamb
    alamb committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    ab1fe3e View commit details
    Browse the repository at this point in the history

  3. Apply suggestions from code review 

    Co-authored-by: Andrew Lamb <[email protected]>
    @devinjdangelo @alamb
    devinjdangelo and alamb authored Sep 8, 2023
    Configuration menu
    Copy the full SHA
    c65d429 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2023

  1. address comments, add type aliases

    @devinjdangelo
    devinjdangelo committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    0df35b1 View commit details
    Browse the repository at this point in the history

  2. cargo fmt

    @devinjdangelo
    devinjdangelo committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    b8f2f0a View commit details
    Browse the repository at this point in the history