Open
Description
Async functions should not open files with blocking methods like open
.
obstore/obstore/python/obstore/fsspec.py
Lines 195 to 198 in 9ca85e7
obstore/obstore/python/obstore/fsspec.py
Lines 201 to 206 in 9ca85e7
Instead we should use something like:
async def _put_file(
self,
lpath: str,
rpath: str,
mode: str = "overwrite", # noqa: ARG002
**_kwargs: Any,
) -> None:
local_store = LocalStore("/")
local_file = await obs.get_async(local_store, lpath)
await obs.put_async(self.store, rpath, local_file)
-
We may want to cache the construction of
LocalStore("/")
? It looks like constructing that only takes1.22 μs
on my machine, so maybe not important. -
It looks like the leading
"/"
on a path is not an issue. I.e. this works even though there's a leading"/"
in the path:lpath = "/Users/kyle/github/developmentseed/obstore/README.md" local_store = LocalStore("/") local_file = obs.get(local_store, lpath) print(local_file.bytes().to_bytes().decode())
-
This may deadlock in the case that the async implementation is used synchronously, but that's something we can come back to in the future.