Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export object_store integration tests #5709

Merged

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented May 2, 2024

Which issue does this PR close?

Closes #5685.

Rationale for this change

The object_store crate has quite a comprehensive test suite, to better enable third-party implementations we can export this for external consumption.

I debated making this a separate crate, but I felt a feature flag was more discoverable, and avoids all the complexities associated with needing to create a separate object_store workspace to contain both crates, and then release both in sync.

What changes are included in this PR?

Are there any user-facing changes?

No

@github-actions github-actions bot added the object-store Object Store Interface label May 2, 2024
@tustvold tustvold force-pushed the export-object-store-integration-tests branch from d7b731e to cf45dd0 Compare May 2, 2024 13:10
}

#[cfg(any(feature = "azure", feature = "aws"))]
pub(crate) async fn signing<T>(integration: &T)
pub async fn signing<T>(integration: &T)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to leave signing and tagging behind as they're relatively niche, and created some funky feature flag/dependency shenanigans.

@@ -25,15 +25,9 @@ use url::Url;

#[derive(Debug, Snafu)]
enum Error {
#[snafu(display("Unable to convert URL \"{}\" to filesystem path", url))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated clippy lint fix

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this an API change (as the type enums have changed?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These enumerations are module private

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tustvold
Copy link
Contributor Author

tustvold commented May 2, 2024

Looks like a new clippy is better at finding unused code, so rolled that into this PR

@alamb alamb added the api-change Changes to the arrow API label May 2, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @tustvold

I think we should:

  1. Document the new feature flag (along with an example of how to run the integration tests -- I didn't see this anywhere but may have missed it) Perhaps add to docs here
  2. Figure out if this is an API change or not, and if it is an API change split those out from the extraction to a new feature flag (to allow finer grained control of when breaking changes are needed)

This is a really neat PR BTW

@tustvold tustvold removed the api-change Changes to the arrow API label May 2, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think documenting how to use this integration test is really important, but I think it could be done as a follow on PR too.

Thanks @tustvold

@@ -70,21 +70,9 @@ enum Error {
#[snafu(display("Configuration key: '{}' is not known.", key))]
UnknownConfigurationKey { key: String },

#[snafu(display("Bucket '{}' not found", bucket))]
BucketNotFound { bucket: String },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
object-store Object Store Interface
Projects
None yet
2 participants