Skip to content

Rename DataSource and FileSource fields for consistency #14898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 27, 2025

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Feb 26, 2025

Which issue does this PR close?

Rationale for this change

While working on upgrading DataFusion 46 in several projects, I found the name source super confusing as there are two similarly named traits, FileSource and DataSource but several functions / fields are named just source so it is not clear which they refer to

This was especially confusing as DataSourceExec has a DataSource which is a FileScanConfig not the FileSOurce (which is a``ParquetSource` 🤯

What changes are included in this PR?

  1. rename the fields and accessors to be explicitly data_source or file_source to avoid the confusion

The documentation on how all these traits are related to each other is somewhat sparse. I am hoping to make a PR to improve it too

Are these changes tested?

By CI

Are there any user-facing changes?

While this is an API change, the releveant APIs were introduced in #14224 and thus not yet released, so I don't think this is explicitly a breaking change.

@github-actions github-actions bot added core Core DataFusion crate substrait Changes to the substrait crate proto Related to proto crate labels Feb 26, 2025
@alamb alamb force-pushed the alamb/rename_properties branch from c773b30 to 2834d58 Compare February 26, 2025 15:29
}

/// Return the source object
pub fn source(&self) -> &Arc<dyn DataSource> {
&self.source
pub fn data_source(&self) -> &Arc<dyn DataSource> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the actual API change -- the rest of the PR is making local variable names consistent

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alamb, making sense

@berkaysynnada berkaysynnada merged commit 1ae06a4 into apache:main Feb 27, 2025
25 checks passed
@alamb alamb deleted the alamb/rename_properties branch February 27, 2025 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate proto Related to proto crate substrait Changes to the substrait crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants