Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs.info() and fs.ls(detail=True) return different etag formats #461

Open
pmrowla opened this issue Feb 15, 2024 · 0 comments
Open

fs.info() and fs.ls(detail=True) return different etag formats #461

pmrowla opened this issue Feb 15, 2024 · 0 comments

Comments

@pmrowla
Copy link
Contributor

pmrowla commented Feb 15, 2024

I'm not sure this is something that needs to be addressed in adlfs, but I think it's worth noting that etags returned by fs.info() and fs.ls(detail=True) have different formats. When using fs.info() the etag will be quoted (wrapped in " quotes) but when using fs.ls() the etag will not be quoted. This means that the caller has to correct the quote format before being able to compare etags to see whether a file has been modified.

With adlfs 2024.2.0:

>>> from adlfs import AzureBlobFileSystem
>>> fs = AzureBlobFileSystem()
>>> fs.info("az://test-deletion/2024.2.0/foo", refresh=True)["etag"]
'"0x8DC2DFC9E520378"'
>>> [(f["name"], f["etag"]) for f in fs.ls("az://test-deletion/2024.2.0/", detail=True, refresh=True)]
[('test-deletion/2024.2.0/foo', '0x8DC2DFC9E520378')]

I did some investigation and it looks like this is due to differences in what the azure API returns for different calls. When using BlobClient.get_blob_properties() the returned etag property is wrapped in " quotes. However, when using BlobContainer.walk_blobs() the etag property for iterated blob properties is not wrapped in quotes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant