Skip to content

Support metadata columns (location, size, last_modified) in ListingTableProvider #15181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
4f150eb
Support for metadata columns (`location`, `size`, `last_modified`) i…
phillipleblanc Mar 10, 2025
5a2584f
Working implementation
phillipleblanc Mar 12, 2025
8b97099
Add tests for apply_filters and prune_partitions
phillipleblanc Mar 12, 2025
d23bab0
Add tests for the metadata filtering
phillipleblanc Mar 12, 2025
a806d2e
Fix lint
phillipleblanc Mar 12, 2025
ce58b7c
Fix file limit
phillipleblanc Mar 12, 2025
06ea966
Add tests for the metadata in datasource
phillipleblanc Mar 12, 2025
3400f48
Add tests for extended column projector
phillipleblanc Mar 12, 2025
f1cf368
Add tests for the ExtendedColumnProjector in file_scan_config.rs
phillipleblanc Mar 12, 2025
27860a2
Don't allow duplicate metadata columns
phillipleblanc Mar 12, 2025
3200687
cargo fmt
phillipleblanc Mar 12, 2025
ff3fa44
Fix taplo
phillipleblanc Mar 12, 2025
e654993
Fix cargo doc
phillipleblanc Mar 12, 2025
c750331
Fix doctest
phillipleblanc Mar 12, 2025
b2d3476
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 8, 2025
429f792
Fix path_partition tests
phillipleblanc Apr 8, 2025
8e8f8d7
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 9, 2025
a479ea3
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 9, 2025
765c508
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 11, 2025
e904ee4
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 14, 2025
d60d1ea
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 17, 2025
5ef715d
Fix u64 to usize
phillipleblanc Apr 17, 2025
fb2fd9e
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc Apr 21, 2025
7ac8aa8
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc May 1, 2025
7d6fe48
Fix file_scan_config.rs
phillipleblanc May 1, 2025
c6d2f17
fix
phillipleblanc May 1, 2025
1c8ed08
fix
phillipleblanc May 1, 2025
a6bce18
Fix clippy
phillipleblanc May 1, 2025
71273c5
Merge branch 'main' into phillip/250312-listing-table-metadata-cols
phillipleblanc May 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions datafusion/catalog-listing/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ object_store = { workspace = true }
tokio = { workspace = true }

[dev-dependencies]
chrono = { workspace = true }
tempfile = { workspace = true }

[lints]
Expand Down
Loading
Loading