The _check_module script's find does not work correctly with namespace packages

[johnfb]

For personal tools I will often use a namespace package to put my packages into my own namespace.

For example, I might write a command line tool to help with containers, so I make something like johnfb.container, where johnfb is a namespace package. Then I register a console script with something like johnfb.container.__main__:main.

The issue basically comes down to the line here

    names = name.split(".")
    spec = find_spec(names[0])

This returns the spec for the namespace package johnfb which does not exist and spec.has_location is false.

But if I pass find_spec the full package path johnfb.container then I get a spec back where has_location is true and everything works as I would expect it to.

I am not sure why some of the things in that find function do what they do but if I apply the following diff everything works for me:

--- a/_check_module.py	2025-02-07 15:15:35.837493599 -0600
+++ b/_check_module.py	2025-02-07 15:21:26.801972338 -0600
@@ -42,18 +42,19 @@
 
 def find(name, return_package=False):
     names = name.split(".")
-    spec = find_spec(names[0])
+    package = ".".join(names[:-1])
+    spec = find_spec(package)
     if spec is None:
-        raise ArgcompleteMarkerNotFound('no module named "{}"'.format(names[0]))
+        raise ArgcompleteMarkerNotFound('no module named "{}"'.format(package))
     if not spec.has_location:
         raise ArgcompleteMarkerNotFound("cannot locate file")
     if spec.submodule_search_locations is None:
         if len(names) != 1:
-            raise ArgcompleteMarkerNotFound("{} is not a package".format(names[0]))
+            raise ArgcompleteMarkerNotFound("{} is not a package".format(package))
         return spec.origin
     if len(spec.submodule_search_locations) != 1:
         raise ArgcompleteMarkerNotFound("expecting one search location")
-    path = os.path.join(spec.submodule_search_locations[0], *names[1:])
+    path = os.path.join(spec.submodule_search_locations[0], names[-1])
     if os.path.isdir(path):
         filename = "__main__.py"
         if return_package:

My read of the original is that it makes the assumption name is formatted as package.file, and this change then makes the assumption package.subpackage1[.subpackageN].file There may be other things that this breaks because I don't know all the differences between different python versions and how they have changed the importlib metadata loading and spec finding. There may be a better way to solve this issue.

[kislyuk]

Thanks for the report. I agree that support for namespace packages is lacking. I have to think more about the change being proposed here. Ideally we need to just replicate the machinery that the python -m loader is using.

[kislyuk]

OK, this should be fixed in v3.6.0. Please test.

[johnfb]

I updated argcomplete (python -m pip install -U argcomplete) then retried the global argcomplete setup again and everything worked with my namespace package!

That was fast. Thank you!