-
-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add functional API #303
Merged
Merged
Add functional API #303
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
189d15f
Apply CPython PR, sans docs and changelogs
encukou 1e98e35
Adapt to importlib_resources
encukou 558f5bf
Formatting nitpicks
encukou 8fdadde
Port tests to Python 3.8
encukou 2df6ced
Use Ruff style, rather than PEP 8
encukou fa60969
Add news fragment.
jaraco File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
"""Simplified function-based API for importlib.resources""" | ||
|
||
import warnings | ||
|
||
from ._common import files, as_file | ||
|
||
|
||
_MISSING = object() | ||
|
||
|
||
def open_binary(anchor, *path_names): | ||
"""Open for binary reading the *resource* within *package*.""" | ||
return _get_resource(anchor, path_names).open('rb') | ||
|
||
|
||
def open_text(anchor, *path_names, encoding=_MISSING, errors='strict'): | ||
"""Open for text reading the *resource* within *package*.""" | ||
encoding = _get_encoding_arg(path_names, encoding) | ||
resource = _get_resource(anchor, path_names) | ||
return resource.open('r', encoding=encoding, errors=errors) | ||
|
||
|
||
def read_binary(anchor, *path_names): | ||
"""Read and return contents of *resource* within *package* as bytes.""" | ||
return _get_resource(anchor, path_names).read_bytes() | ||
|
||
|
||
def read_text(anchor, *path_names, encoding=_MISSING, errors='strict'): | ||
"""Read and return contents of *resource* within *package* as str.""" | ||
encoding = _get_encoding_arg(path_names, encoding) | ||
resource = _get_resource(anchor, path_names) | ||
return resource.read_text(encoding=encoding, errors=errors) | ||
|
||
|
||
def path(anchor, *path_names): | ||
"""Return the path to the *resource* as an actual file system path.""" | ||
return as_file(_get_resource(anchor, path_names)) | ||
|
||
|
||
def is_resource(anchor, *path_names): | ||
"""Return ``True`` if there is a resource named *name* in the package, | ||
|
||
Otherwise returns ``False``. | ||
""" | ||
return _get_resource(anchor, path_names).is_file() | ||
|
||
|
||
def contents(anchor, *path_names): | ||
"""Return an iterable over the named resources within the package. | ||
|
||
The iterable returns :class:`str` resources (e.g. files). | ||
The iterable does not recurse into subdirectories. | ||
""" | ||
warnings.warn( | ||
"importlib.resources.contents is deprecated. " | ||
"Use files(anchor).iterdir() instead.", | ||
DeprecationWarning, | ||
stacklevel=1, | ||
) | ||
return (resource.name for resource in _get_resource(anchor, path_names).iterdir()) | ||
|
||
|
||
def _get_encoding_arg(path_names, encoding): | ||
# For compatibility with versions where *encoding* was a positional | ||
# argument, it needs to be given explicitly when there are multiple | ||
# *path_names*. | ||
# This limitation can be removed in Python 3.15. | ||
if encoding is _MISSING: | ||
if len(path_names) > 1: | ||
raise TypeError( | ||
"'encoding' argument required with multiple path names", | ||
) | ||
else: | ||
return 'utf-8' | ||
return encoding | ||
|
||
|
||
def _get_resource(anchor, path_names): | ||
if anchor is None: | ||
raise TypeError("anchor must be module or string, got None") | ||
return files(anchor).joinpath(*path_names) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,242 @@ | ||
import unittest | ||
import os | ||
import contextlib | ||
|
||
try: | ||
from test.support.warnings_helper import ignore_warnings, check_warnings | ||
except ImportError: | ||
# older Python versions | ||
from test.support import ignore_warnings, check_warnings | ||
|
||
import importlib_resources as resources | ||
|
||
# Since the functional API forwards to Traversable, we only test | ||
# filesystem resources here -- not zip files, namespace packages etc. | ||
# We do test for two kinds of Anchor, though. | ||
|
||
|
||
class StringAnchorMixin: | ||
anchor01 = 'importlib_resources.tests.data01' | ||
anchor02 = 'importlib_resources.tests.data02' | ||
|
||
|
||
class ModuleAnchorMixin: | ||
from . import data01 as anchor01 | ||
from . import data02 as anchor02 | ||
|
||
|
||
class FunctionalAPIBase: | ||
def _gen_resourcetxt_path_parts(self): | ||
"""Yield various names of a text file in anchor02, each in a subTest""" | ||
for path_parts in ( | ||
('subdirectory', 'subsubdir', 'resource.txt'), | ||
('subdirectory/subsubdir/resource.txt',), | ||
('subdirectory/subsubdir', 'resource.txt'), | ||
): | ||
with self.subTest(path_parts=path_parts): | ||
yield path_parts | ||
|
||
def test_read_text(self): | ||
self.assertEqual( | ||
resources.read_text(self.anchor01, 'utf-8.file'), | ||
'Hello, UTF-8 world!\n', | ||
) | ||
self.assertEqual( | ||
resources.read_text( | ||
self.anchor02, | ||
'subdirectory', | ||
'subsubdir', | ||
'resource.txt', | ||
encoding='utf-8', | ||
), | ||
'a resource', | ||
) | ||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
self.assertEqual( | ||
resources.read_text( | ||
self.anchor02, | ||
*path_parts, | ||
encoding='utf-8', | ||
), | ||
'a resource', | ||
) | ||
# Use generic OSError, since e.g. attempting to read a directory can | ||
# fail with PermissionError rather than IsADirectoryError | ||
with self.assertRaises(OSError): | ||
resources.read_text(self.anchor01) | ||
with self.assertRaises(OSError): | ||
resources.read_text(self.anchor01, 'no-such-file') | ||
with self.assertRaises(UnicodeDecodeError): | ||
resources.read_text(self.anchor01, 'utf-16.file') | ||
self.assertEqual( | ||
resources.read_text( | ||
self.anchor01, | ||
'binary.file', | ||
encoding='latin1', | ||
), | ||
'\x00\x01\x02\x03', | ||
) | ||
self.assertEqual( | ||
resources.read_text( | ||
self.anchor01, | ||
'utf-16.file', | ||
errors='backslashreplace', | ||
), | ||
'Hello, UTF-16 world!\n'.encode('utf-16').decode( | ||
errors='backslashreplace', | ||
), | ||
) | ||
|
||
def test_read_binary(self): | ||
self.assertEqual( | ||
resources.read_binary(self.anchor01, 'utf-8.file'), | ||
b'Hello, UTF-8 world!\n', | ||
) | ||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
self.assertEqual( | ||
resources.read_binary(self.anchor02, *path_parts), | ||
b'a resource', | ||
) | ||
|
||
def test_open_text(self): | ||
with resources.open_text(self.anchor01, 'utf-8.file') as f: | ||
self.assertEqual(f.read(), 'Hello, UTF-8 world!\n') | ||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
with resources.open_text( | ||
self.anchor02, | ||
*path_parts, | ||
encoding='utf-8', | ||
) as f: | ||
self.assertEqual(f.read(), 'a resource') | ||
# Use generic OSError, since e.g. attempting to read a directory can | ||
# fail with PermissionError rather than IsADirectoryError | ||
with self.assertRaises(OSError): | ||
resources.open_text(self.anchor01) | ||
with self.assertRaises(OSError): | ||
resources.open_text(self.anchor01, 'no-such-file') | ||
with resources.open_text(self.anchor01, 'utf-16.file') as f: | ||
with self.assertRaises(UnicodeDecodeError): | ||
f.read() | ||
with resources.open_text( | ||
self.anchor01, | ||
'binary.file', | ||
encoding='latin1', | ||
) as f: | ||
self.assertEqual(f.read(), '\x00\x01\x02\x03') | ||
with resources.open_text( | ||
self.anchor01, | ||
'utf-16.file', | ||
errors='backslashreplace', | ||
) as f: | ||
self.assertEqual( | ||
f.read(), | ||
'Hello, UTF-16 world!\n'.encode('utf-16').decode( | ||
errors='backslashreplace', | ||
), | ||
) | ||
|
||
def test_open_binary(self): | ||
with resources.open_binary(self.anchor01, 'utf-8.file') as f: | ||
self.assertEqual(f.read(), b'Hello, UTF-8 world!\n') | ||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
with resources.open_binary( | ||
self.anchor02, | ||
*path_parts, | ||
) as f: | ||
self.assertEqual(f.read(), b'a resource') | ||
|
||
def test_path(self): | ||
with resources.path(self.anchor01, 'utf-8.file') as path: | ||
with open(str(path), encoding='utf-8') as f: | ||
self.assertEqual(f.read(), 'Hello, UTF-8 world!\n') | ||
with resources.path(self.anchor01) as path: | ||
with open(os.path.join(path, 'utf-8.file'), encoding='utf-8') as f: | ||
self.assertEqual(f.read(), 'Hello, UTF-8 world!\n') | ||
|
||
def test_is_resource(self): | ||
is_resource = resources.is_resource | ||
self.assertTrue(is_resource(self.anchor01, 'utf-8.file')) | ||
self.assertFalse(is_resource(self.anchor01, 'no_such_file')) | ||
self.assertFalse(is_resource(self.anchor01)) | ||
self.assertFalse(is_resource(self.anchor01, 'subdirectory')) | ||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
self.assertTrue(is_resource(self.anchor02, *path_parts)) | ||
|
||
def test_contents(self): | ||
with check_warnings((".*contents.*", DeprecationWarning)): | ||
c = resources.contents(self.anchor01) | ||
self.assertGreaterEqual( | ||
set(c), | ||
{'utf-8.file', 'utf-16.file', 'binary.file', 'subdirectory'}, | ||
) | ||
with contextlib.ExitStack() as cm: | ||
cm.enter_context(self.assertRaises(OSError)) | ||
cm.enter_context(check_warnings((".*contents.*", DeprecationWarning))) | ||
|
||
list(resources.contents(self.anchor01, 'utf-8.file')) | ||
|
||
for path_parts in self._gen_resourcetxt_path_parts(): | ||
with contextlib.ExitStack() as cm: | ||
cm.enter_context(self.assertRaises(OSError)) | ||
cm.enter_context(check_warnings((".*contents.*", DeprecationWarning))) | ||
|
||
list(resources.contents(self.anchor01, *path_parts)) | ||
with check_warnings((".*contents.*", DeprecationWarning)): | ||
c = resources.contents(self.anchor01, 'subdirectory') | ||
self.assertGreaterEqual( | ||
set(c), | ||
{'binary.file'}, | ||
) | ||
|
||
@ignore_warnings(category=DeprecationWarning) | ||
def test_common_errors(self): | ||
for func in ( | ||
resources.read_text, | ||
resources.read_binary, | ||
resources.open_text, | ||
resources.open_binary, | ||
resources.path, | ||
resources.is_resource, | ||
resources.contents, | ||
): | ||
with self.subTest(func=func): | ||
# Rejecting None anchor | ||
with self.assertRaises(TypeError): | ||
func(None) | ||
# Rejecting invalid anchor type | ||
with self.assertRaises((TypeError, AttributeError)): | ||
func(1234) | ||
# Unknown module | ||
with self.assertRaises(ModuleNotFoundError): | ||
func('$missing module$') | ||
|
||
def test_text_errors(self): | ||
for func in ( | ||
resources.read_text, | ||
resources.open_text, | ||
): | ||
with self.subTest(func=func): | ||
# Multiple path arguments need explicit encoding argument. | ||
with self.assertRaises(TypeError): | ||
func( | ||
self.anchor02, | ||
'subdirectory', | ||
'subsubdir', | ||
'resource.txt', | ||
) | ||
|
||
|
||
class FunctionalAPITest_StringAnchor( | ||
unittest.TestCase, | ||
FunctionalAPIBase, | ||
StringAnchorMixin, | ||
): | ||
pass | ||
|
||
|
||
class FunctionalAPITest_ModuleAnchor( | ||
unittest.TestCase, | ||
FunctionalAPIBase, | ||
ModuleAnchorMixin, | ||
): | ||
pass |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
The functions | ||
``is_resource()``, | ||
``open_binary()``, | ||
``open_text()``, | ||
``path()``, | ||
``read_binary()``, and | ||
``read_text()`` are un-deprecated, and support | ||
subdirectories via multiple positional arguments. | ||
The ``contents()`` function also allows subdirectories, | ||
but remains deprecated. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this module be private or should importing from
importlib_resources.functional
be allowed?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I don't recall if it was public before.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call. Private is probably better.