Skip to content

Commit

Permalink
allow for roundtrips of cloudpaths through pickle serialization
Browse files Browse the repository at this point in the history
This avoids an exception thrown because the _client is not serialized
into the pickled object, and thus when __getstate__ is called the second
time, there is no _client field to delete.

Closes #450
  • Loading branch information
kujenga committed Jul 21, 2024
1 parent 08b018b commit 514d447
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 1 deletion.
1 change: 1 addition & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## UNRELEASED

- Allow `CloudPath` objects to be loaded/dumped through pickle format repeatedly. (Issue [#450](https://github.com/drivendataorg/cloudpathlib/issues/450))
- Fixes typo in `FileCacheMode` where values were being filled by envvar `CLOUPATHLIB_FILE_CACHE_MODE` instead of `CLOUDPATHLIB_FILE_CACHE_MODE`. (PR [#424](https://github.com/drivendataorg/cloudpathlib/pull/424)
- Fix `CloudPath` cleanup via `CloudPath.__del__` when `Client` encounters an exception during initialization and does not create a `file_cache_mode` attribute. (Issue [#372](https://github.com/drivendataorg/cloudpathlib/issues/372), thanks to [@bryanwweber](https://github.com/bryanwweber))
- Drop support for Python 3.7; pin minimal `boto3` version to Python 3.8+ versions. (PR [#407](https://github.com/drivendataorg/cloudpathlib/pull/407))
Expand Down
3 changes: 2 additions & 1 deletion cloudpathlib/cloudpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,8 @@ def __getstate__(self) -> Dict[str, Any]:
state = self.__dict__.copy()

# don't pickle client
del state["_client"]
if "_client" in state:
del state["_client"]

return state

Expand Down
14 changes: 14 additions & 0 deletions tests/test_cloudpath_serialize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import pickle

from cloudpathlib import CloudPath


def test_pickle_roundtrip():
path1 = CloudPath("s3://bucket/key")
pkl1 = pickle.dumps(path1)

path2 = pickle.loads(pkl1)
pkl2 = pickle.dumps(path2)

assert path1 == path2
assert pkl1 == pkl2

0 comments on commit 514d447

Please sign in to comment.