Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python 3.13 compatibility #474

Merged
merged 17 commits into from
Oct 18, 2024
2 changes: 1 addition & 1 deletion .github/workflows/docs-master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11
cache: "pip" # caching pip dependencies
cache-dependency-path: |
pyproject.toml
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs-preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11
cache: "pip" # caching pip dependencies
cache-dependency-path: |
pyproject.toml
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.11

- name: Install dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
python-version: [3.8, 3.9, "3.10", "3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4
Expand Down
12 changes: 12 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,18 @@

- Added support for custom schemes in CloudPath and Client subclases. (Issue [#466](https://github.com/drivendataorg/cloudpathlib/issues/466), PR [#467](https://github.com/drivendataorg/cloudpathlib/pull/467))
- Fixed `ResourceNotFoundError` on Azure gen2 storage accounts with HNS enabled and issue that some Azure credentials do not have `account_name`. (Issue [#470](https://github.com/drivendataorg/cloudpathlib/issues/470), Issue [#476](https://github.com/drivendataorg/cloudpathlib/issues/476), PR [#478](https://github.com/drivendataorg/cloudpathlib/pull/478))
- Added support for Python 3.13 (Issue [#472](https://github.com/drivendataorg/cloudpathlib/issues/472), [PR #474](https://github.com/drivendataorg/cloudpathlib/pull/474)):
- [`.full_match` added](https://docs.python.org/3.13/library/pathlib.html#pathlib.PurePath.full_match)
- [`.from_uri` added](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.from_uri)
- [`follow_symlinks` kwarg added to `is_file`](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.is_file) added as no-op
- [`follow_symlinks` kwarg added to `is_dir`](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.is_dir) added as no-op
- [`newline` kwarg added to `read_text`](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.read_text)
- [`recurse_symlinks` kwarg added to `glob`](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.glob) added as no-op
- [`pattern` parameter for `glob` can be PathLike](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.glob)
- [`recurse_symlinks` kwarg added to `rglob`](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.rglob) added as no-op
- [`pattern` parameter for `rglob` can be PathLike](https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.rglob)
- [`.parser` property added](https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.parser)

pjbull marked this conversation as resolved.
Show resolved Hide resolved

## v0.19.0 (2024-08-29)

Expand Down
6 changes: 0 additions & 6 deletions cloudpathlib/azure/azblobpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,6 @@ class AzureBlobPath(CloudPath):
def drive(self) -> str:
return self.container

def is_dir(self) -> bool:
return self.client._is_file_or_dir(self) == "dir"

def is_file(self) -> bool:
return self.client._is_file_or_dir(self) == "file"

def mkdir(self, parents=False, exist_ok=False):
self.client._mkdir(self, parents=parents, exist_ok=exist_ok)

Expand Down
104 changes: 76 additions & 28 deletions cloudpathlib/cloudpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@
PosixPath,
PurePosixPath,
WindowsPath,
_PathParents,
)

import shutil
import sys
from types import MethodType
from typing import (
BinaryIO,
Literal,
Expand Down Expand Up @@ -56,21 +56,29 @@
else:
from typing_extensions import Self

if sys.version_info >= (3, 12):

if sys.version_info < (3, 12):
from pathlib import _posix_flavour # type: ignore[attr-defined] # noqa: F811
from pathlib import _make_selector as _make_selector_pathlib # type: ignore[attr-defined] # noqa: F811
from pathlib import _PathParents # type: ignore[attr-defined]

def _make_selector(pattern_parts, _flavour, case_sensitive=True): # noqa: F811
return _make_selector_pathlib(tuple(pattern_parts), _flavour)

elif sys.version_info[:2] == (3, 12):
from pathlib import _PathParents # type: ignore[attr-defined]
from pathlib import posixpath as _posix_flavour # type: ignore[attr-defined]
from pathlib import _make_selector # type: ignore[attr-defined]
else:
from pathlib import _posix_flavour # type: ignore[attr-defined]
from pathlib import _make_selector as _make_selector_pathlib # type: ignore[attr-defined]
elif sys.version_info >= (3, 13):
from pathlib._local import _PathParents
import posixpath as _posix_flavour # type: ignore[attr-defined] # noqa: F811

def _make_selector(pattern_parts, _flavour, case_sensitive=True):
return _make_selector_pathlib(tuple(pattern_parts), _flavour)
from .legacy.glob import _make_selector # noqa: F811


from cloudpathlib.enums import FileCacheMode

from . import anypath

from .exceptions import (
ClientMismatchError,
CloudPathFileExistsError,
Expand Down Expand Up @@ -194,7 +202,12 @@
and getattr(getattr(Path, attr), "__doc__", None)
):
docstring = getattr(Path, attr).__doc__ + " _(Docstring copied from pathlib.Path)_"
getattr(cls, attr).__doc__ = docstring

if isinstance(getattr(cls, attr), (MethodType)):
getattr(cls, attr).__func__.__doc__ = docstring
else:
getattr(cls, attr).__doc__ = docstring

if isinstance(getattr(cls, attr), property):
# Properties have __doc__ duplicated under fget, and at least some parsers
# read it from there.
Expand Down Expand Up @@ -383,16 +396,6 @@
"""For example "bucket" on S3 or "container" on Azure; needs to be defined for each class"""
pass

@abc.abstractmethod
def is_dir(self) -> bool:
"""Should be implemented without requiring a dir is downloaded"""
pass

@abc.abstractmethod
def is_file(self) -> bool:
"""Should be implemented without requiring that the file is downloaded"""
pass

@abc.abstractmethod
def mkdir(self, parents: bool = False, exist_ok: bool = False) -> None:
"""Should be implemented using the client API without requiring a dir is downloaded"""
Expand Down Expand Up @@ -427,24 +430,44 @@
def exists(self) -> bool:
return self.client._exists(self)

def is_dir(self, follow_symlinks=True) -> bool:
return self.client._is_file_or_dir(self) == "dir"

def is_file(self, follow_symlinks=True) -> bool:
return self.client._is_file_or_dir(self) == "file"

@property
def fspath(self) -> str:
return self.__fspath__()

def _glob_checks(self, pattern: str) -> None:
if ".." in pattern:
@classmethod
def from_uri(cls, uri: str) -> Self:
return cls(uri)

def _glob_checks(self, pattern: Union[str, os.PathLike]) -> str:
if isinstance(pattern, os.PathLike):
if isinstance(pattern, CloudPath):
str_pattern = str(pattern.relative_to(self))
else:
str_pattern = os.fspath(pattern)
else:
str_pattern = str(pattern)

if ".." in str_pattern:
raise CloudPathNotImplementedError(
"Relative paths with '..' not supported in glob patterns."
)

if pattern.startswith(self.cloud_prefix) or pattern.startswith("/"):
if str_pattern.startswith(self.cloud_prefix) or str_pattern.startswith("/"):
raise CloudPathNotImplementedError("Non-relative patterns are unsupported")

if self.drive == "":
raise CloudPathNotImplementedError(
".glob is only supported within a bucket or container; you can use `.iterdir` to list buckets; for example, CloudPath('s3://').iterdir()"
)

return str_pattern

def _build_subtree(self, recursive):
# build a tree structure for all files out of default dicts
Tree: Callable = lambda: defaultdict(Tree)
Expand Down Expand Up @@ -488,9 +511,9 @@
yield (self / str(p)[len(self.name) + 1 :])

def glob(
self, pattern: str, case_sensitive: Optional[bool] = None
self, pattern: Union[str, os.PathLike], case_sensitive: Optional[bool] = None
) -> Generator[Self, None, None]:
self._glob_checks(pattern)
pattern = self._glob_checks(pattern)

pattern_parts = PurePosixPath(pattern).parts
selector = _make_selector(
Expand All @@ -505,9 +528,9 @@
)

def rglob(
self, pattern: str, case_sensitive: Optional[bool] = None
self, pattern: Union[str, os.PathLike], case_sensitive: Optional[bool] = None
) -> Generator[Self, None, None]:
self._glob_checks(pattern)
pattern = self._glob_checks(pattern)

pattern_parts = PurePosixPath(pattern).parts
selector = _make_selector(
Expand Down Expand Up @@ -812,8 +835,13 @@
with self.open(mode="rb") as f:
return f.read()

def read_text(self, encoding: Optional[str] = None, errors: Optional[str] = None) -> str:
with self.open(mode="r", encoding=encoding, errors=errors) as f:
def read_text(
self,
encoding: Optional[str] = None,
errors: Optional[str] = None,
newline: Optional[str] = None,
) -> str:
with self.open(mode="r", encoding=encoding, errors=errors, newline=newline) as f:
return f.read()

def is_junction(self):
Expand Down Expand Up @@ -904,6 +932,19 @@
def name(self) -> str:
return self._dispatch_to_path("name")

def full_match(self, pattern: str, case_sensitive: Optional[bool] = None) -> bool:
if sys.version_info < (3, 13):
raise NotImplementedError("full_match requires Python 3.13 or higher")

# strip scheme from start of pattern before testing
if pattern.startswith(self.anchor + self.drive):
pattern = pattern[len(self.anchor + self.drive) :]

# remove drive, which is kept on normal dispatch to pathlib
return PurePosixPath(self._no_prefix_no_drive).full_match( # type: ignore[attr-defined]
pattern, case_sensitive=case_sensitive
)

def match(self, path_pattern: str, case_sensitive: Optional[bool] = None) -> bool:
# strip scheme from start of pattern before testing
if path_pattern.startswith(self.anchor + self.drive + "/"):
Expand All @@ -916,6 +957,13 @@

return self._dispatch_to_path("match", path_pattern, **kwargs)

@property
def parser(self) -> Self:
if sys.version_info < (3, 13):
raise NotImplementedError("parser requires Python 3.13 or higher")

Check warning on line 963 in cloudpathlib/cloudpath.py

View check run for this annotation

Codecov / codecov/patch

cloudpathlib/cloudpath.py#L963

Added line #L963 was not covered by tests

return self._dispatch_to_path("parser")

@property
def parent(self) -> Self:
return self._dispatch_to_path("parent")
Expand Down
6 changes: 0 additions & 6 deletions cloudpathlib/gs/gspath.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,6 @@ class GSPath(CloudPath):
def drive(self) -> str:
return self.bucket

def is_dir(self) -> bool:
return self.client._is_file_or_dir(self) == "dir"

def is_file(self) -> bool:
return self.client._is_file_or_dir(self) == "file"

def mkdir(self, parents=False, exist_ok=False):
# not possible to make empty directory on cloud storage
pass
Expand Down
Loading
Loading