Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XAmzContentSHA256Mismatch writing to storage bucket #932

Open
JWDobken opened this issue Jan 20, 2025 · 2 comments
Open

XAmzContentSHA256Mismatch writing to storage bucket #932

JWDobken opened this issue Jan 20, 2025 · 2 comments

Comments

@JWDobken
Copy link

When writing to a Hetzner Storage Bucket, which supports same AWS and boto clients, I received an XAmzContentSHA256Mismatch

import s3fs

bucket_client = s3fs.S3FileSystem(
    key="***",
    secret="***",
    endpoint_url="**",
)

with bucket_client.open("s3://mybucker/test.txt", "wb") as f:
    f.write(b"eebhdbedbehdbehdbehj")

Logs:

File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:2290](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=2289), in S3File._call_s3(self, method, *kwarglist, **kwargs)
   2289 def _call_s3(self, method, *kwarglist, **kwargs):
-> 2290     return self.fs.call_s3(method, self.s3_additional_kwargs, *kwarglist, **kwargs)

File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:118](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=117), in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    115 @functools.wraps(func)
    116 def wrapper(*args, **kwargs):
    117     self = obj or args[0]
--> 118     return sync(self.loop, func, *args, **kwargs)

File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:103](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=102), in sync(loop, func, timeout, *args, **kwargs)
    101     raise FSTimeoutError from return_result
    102 elif isinstance(return_result, BaseException):
--> 103     raise return_result
    104 else:
    105     return return_result

File [~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py:56](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/fsspec/asyn.py#line=55), in _runner(event, coro, result, timeout)
     54     coro = asyncio.wait_for(coro, timeout=timeout)
     55 try:
---> 56     result[0] = await coro
     57 except Exception as ex:
     58     result[0] = ex

File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:371](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=370), in S3FileSystem._call_s3(self, method, *akwarglist, **kwargs)
    369 logger.debug("CALL: %s - %s - %s", method.__name__, akwarglist, kw2)
    370 additional_kwargs = self._get_s3_method_kwargs(method, *akwarglist, **kwargs)
--> 371 return await _error_wrapper(
    372     method, kwargs=additional_kwargs, retries=self.retries
    373 )

File [~/miniconda/lib/python3.12/site-packages/s3fs/core.py:146](http://localhost:8888/lab/workspaces/auto-P/tree/~/miniconda/lib/python3.12/site-packages/s3fs/core.py#line=145), in _error_wrapper(func, args, kwargs, retries)
    144         err = e
    145 err = translate_boto_error(err)
--> 146 raise err

OSError: [Errno 5] An error occurred (XAmzContentSHA256Mismatch) when calling the PutObject operation: None

Versions:

  • Python 3.12.2
  • s3fs==2024.12.0
  • aioboto3==13.4.0
  • aiobotocore==2.18.0
  • boto==2.49.0
  • boto3==1.36.1
  • botocore==1.36.1
@JWDobken
Copy link
Author

I found that going back to aiobotocore==2.16.0 resolves the issue

@martindurant
Copy link
Member

I believe this is the same as #931

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants