py-v0.4.0

See also a blog post announcing this release.

New Features 🪄

• Support for pickling & always manage store prefix by @kylebarron in #185, #239, #223
• Add top-level obstore.store.from_url function, which delegates to each store's from_url constructor by @kylebarron in #179, #201
• Add option to return Arrow from list_with_delimiter by @kylebarron in #238, #244
• (Provisional) Enhanced loading of s3 credentials using aws-config crate by @kylebarron in #203
• Access config values out from stores by @kylebarron in #210
• LocalStore updates:
  • Enable automatic cleanup for local store, when deleting directories by @kylebarron in #175
  • Optionally create root dir in LocalStore by @kylebarron in #177
• File-like object updates:

  • Add support for writable file-like objects by @kylebarron in #167

  • Updates to readable file API:

    • Support user-specified capacity in readable file-like objects by @kylebarron in #174
    • Expose ObjectMeta from readable file API by @kylebarron in #176
• Merge config and kwargs and validate that no configuration parameters have been passed multiple times. (#180, #182, #218)
• Add __repr__ to Bytes class by @jessekrubin in #173

Breaking changes 🔧

• get_range, get_range_async, get_ranges, and get_ranges_async now require named parameters for start, end, and length to make the semantics of the range request fully explicit. by @kylebarron in #156
• Previously, individual stores did not manage a prefix path within the remote resource and PrefixStore was used to enable this. As of 0.4.0, PrefixStore was removed and all stores manage an optional mount prefix natively.
• obstore.open has been renamed to obstore.open_reader.
• The from_env constructor has been removed from S3Store, GCSStore, and AzureStore. Now all constructors will read from environment variables. Use __init__ or from_url instead. #189
• obstore.exceptions.ObstoreError renamed to obstore.exceptions.BaseError #200

Bug fixes 🐛

• Fix pylance finding exceptions module by @kylebarron in #183
• Allow passing in partial retry/backoff config by @kylebarron in #205
• Fix returning None from async functions by @kylebarron in #245
• Fix LocalStore range request past end of file, by @kylebarron in #230

Documentation 📖

• Update wording for fsspec docstring by @kylebarron in #195
• Add documentation about AWS region by @kylebarron in #213
• Add developer documentation for functional API choice by @kylebarron in #215
• Add tqdm progress bar example by @kylebarron in #237
• Add contributor, performance, integrations docs by @kylebarron in #227
• Add minio example by @kylebarron in #241

Other

• Use manylinux 2_24 for aarch64 linux wheels by @kylebarron in #225

New Contributors

• @vincentsarago made their first contribution in #168
• @jessekrubin made their first contribution in #173

Full Changelog: py-v0.3.0...py-v0.4.0

py-v0.3.0

New Features 🪄

• Streaming uploads. obstore.put now supports iterable input, and obstore.put_async now supports async iterable input. This means you can pass the output of obstore.get_async directly into obstore.put_async. by @kylebarron in #54
• Allow passing config options directly as keyword arguments. Previously, you had to pass all options as a dict into the config parameter. Now you can pass the elements directly to the store constructor. by @kylebarron in #144
• Readable file-like objects. Open a readable file-like object with obstore.open and obstore.open_async. by @kylebarron in #33
• Fsspec integration by @martindurant in #63
• Prefix store by @kylebarron in #117
• Python 3.13 wheels by @kylebarron in #95
• Support python timedelta objects as duration config values by @kylebarron in #146
• Add class constructors for store builders. Each store now has an __init__ method, for easier construction. by @kylebarron in #141

Breaking changes 🔧

• get_range, get_range_async, get_ranges, and get_ranges_async now use start/end instead of offset/length. This is for consistency with the range option of obstore.get. by @kylebarron in #71

• Return Bytes from GetResult.bytes() by @kylebarron in #134

Bug fixes 🐛

• boto3 region name can be None by @kylebarron in #59
• add missing py.typed file by @gruebel in #115

Documentation 📖

• FastAPI/Starlette example by @kylebarron in #145
• Add conda installation doc to README by @kylebarron in #78
• Document suggested lifecycle rules for aborted multipart uploads by @kylebarron in #139
• Add type hint and documentation for requester pays by @kylebarron in #131
• Add note that S3Store can be constructed without boto3 by @kylebarron in #108
• HTTP Store usage example by @kylebarron in #142

New Contributors

• @willemarcel made their first contribution in #64
• @martindurant made their first contribution in #63
• @norlandrhagen made their first contribution in #107
• @gruebel made their first contribution in #115

Full Changelog: py-v0.2.0...py-v0.3.0

py-v0.2.0

What's Changed

• Streaming list results. list now returns an async or sync generator. by @kylebarron in #35
• Optionally return list result as arrow. The return_arrow keyword argument returns chunks from list as Arrow RecordBatches, which is faster than materializing Python dicts/lists. by @kylebarron in #38
• Return buffer protocol object from get_range and get_ranges. Enables zero-copy data exchange from Rust into Python. by @kylebarron in #39
• Add put options. Enables custom tags and attributes, as well as "put if not exists". by @kylebarron in #50
• Rename to obstore by @kylebarron in #45
• Add custom exceptions. by @kylebarron in #48

Full Changelog: py-v0.1.0...py-v0.2.0

py-v0.1.0

What's Changed

• Initial Release

Full Changelog: https://github.com/developmentseed/obstore/commits/py-v0.1.0