Skip to content

Infer default credential providers #414

Open
@kylebarron

Description

@kylebarron

I'm thinking something like this in store.py:

(Not sure if all these exist credential providers exist in obstore, but assume that when called with no arguments that these return the vendor's default credential.)

def from_url(...):
    ...
    scheme = _parse_scheme(url)
    vendor_default_credential_providers = {
        "s3": S3CredentialProvider,
        "gcs": GoogleCredentialProvider,
        "azure": AzureCredentialProvider,
    }

    vendor_default_async_credential_providers = {
        "s3": S3AsyncCredentialProvider,
        "gcs": GoogleAsyncCredentialProvider,
        "azure": AzureAsyncCredentialProvider,
    }

    if credential_provider is None and vendor_default_credential_provider:
        if credential_provider := vendor_default_credential_providers.get(scheme):
            credential_provider = credential_provider()
    elif credential_provider is None and vendor_default_async_credential_provider:
        if credential_provider := vendor_default_async_credential_providers.get(scheme):
            credential_provider = credential_provider()

    if scheme == "s3":
        ...

Implications:

  • Existing behaviour of from_url is unchanged when new keywords are omitted.
  • Two new keywords added to from_url: vendor_default_credential_provider and vendor_default_async_credential_provider. These allow the user to opt into using an automatically selected vendor default credential provider, and allow the user to specify whether they want it to be synchronous or asynchronous.
  • If one of the new keywords is specified, and credential_provider is not explicitly specified, then the appropriate vendor default credential provider will be used.
  • There is a one to one mapping from scheme to default vendor credential providers. Users must still explicitly specify if they need credential providers other than the vendor default credential provider.

Originally posted by @daviewales in #267

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions