Pyright can't resolve same-directory imports when having a pyproject.toml and a .pth file.

[rgoya]

Describe the bug
Having a pyproject.toml and a .pth file stops pyright from resolving absolute imports from the same directory as the importing file. (Step 6 in import resolution order)

Reproduce the bug
(Edit: I've added full repro code in followup #9610 (comment))

Consider a folder structure like so:

• pyright_test/
  • pyright_test/org/
    • pyright_test/org/foo.py
    • pyright_test/org/bar.py

Contents of pyright_test/org/bar.py are:

baz = "Baz!"

Contents of pyright_test/org/foo.py are:

import bar

print(bar.baz)

There is also a pyright_test.pth file in ~/.conda/lib/python3.*/site-packages that contains:

/home/rgoya/pyright_test

At this point things seem fine:

$ cd /home/rgoya/pyright_test/
$ pyright .
0 errors, 0 warnings, 0 informations
$ cd org/
$ python foo.py
Baz!

I then create a file pyright_test/pyproject.toml to add some pyright configurations:

[tool.pyright]
exclude = []

The actual contents don't seem to matter, as long as [tool.pyright] is there, the bug is triggered:

$ cd /home/rgoya/pyright_test/
$ pyright .
/home/rgoya/pyright_test/org/foo.py
  /home/rgoya/pyright_test/org/foo.py:1:8 - error: Import "bar" could not be resolved (reportMissingImports)
1 error, 0 warnings, 0 informations

If I remove either the pyproject.toml or the pyright_test.pth file, pyright works again.

pyright CLI version
👍 In versions 1.1.365 and prior (tested 351, 361, 365) having both pyproject.toml and pyright_test.pth causes no problems.

👎 Starting in version 1.1.366 and forward (tested 366, 370, 383, 391) having both pyproject.toml and pyright_test.pth triggers the bug.

Debug traces
Last working version (1.1.365):

$ pyright . --verbose
Loading pyproject.toml file at /home/rgoya/pyright_test/pyproject.toml
Assuming Python version 3.10.12.final.0
Auto-excluding **/node_modules
Auto-excluding **/__pycache__
Auto-excluding **/.*
Search paths for file:///home/rgoya/pyright_test
  /home/rgoya/.cache/pyright-python/1.1.365/node_modules/pyright/dist/typeshed-fallback/stdlib
  /home/rgoya/pyright_test
  /home/rgoya/pyright_test/typings
  /home/rgoya/.cache/pyright-python/1.1.365/node_modules/pyright/dist/typeshed-fallback/stubs/...
  /home/rgoya/.conda/lib/python3.10
  /home/rgoya/.conda/lib/python3.10/lib-dynload
  /home/rgoya/.conda/lib/python3.10/site-packages
  /home/rgoya/pyright_test
Found 2 source files
pyright 1.1.365
0 errors, 0 warnings, 0 informations 
Completed in 0.727sec
WARNING: there is a new pyright version available (v1.1.365 -> v1.1.391).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`

Bug in latest version (1.1.391)

$ pyright . --verbose
Loading pyproject.toml file at /home/rgoya/pyright_test/pyproject.toml
Auto-excluding **/node_modules
Auto-excluding **/__pycache__
Auto-excluding **/.*
Execution environment: python
  Extra paths:
    (none)
  Python version: 3.13
  Python platform: Darwin
  Search paths:
    /home/rgoya/.cache/pyright-python/1.1.391/node_modules/pyright/dist/typeshed-fallback/stdlib
    /home/rgoya/pyright_test
    /home/rgoya/pyright_test/typings
    /home/rgoya/.cache/pyright-python/1.1.391/node_modules/pyright/dist/typeshed-fallback/stubs/...
    /home/rgoya/.conda/lib/python3.10
    /home/rgoya/.conda/lib/python3.10/lib-dynload
    /home/rgoya/.conda/lib/python3.10/site-packages
    /home/rgoya/pyright_test
Found 2 source files
Could not import 'bar' in file '/home/rgoya/pyright_test/org/foo.py'
  Looking in stubPath 'file:///home/rgoya/pyright_test/typings'
  Attempting to resolve stub package using root path 'file:///home/rgoya/pyright_test/typings'
  Attempting to resolve using root path 'file:///home/rgoya/pyright_test/typings'
  Looking in root directory of execution environment 'file:///home/rgoya/pyright_test'
  Attempting to resolve stub package using root path 'file:///home/rgoya/pyright_test'
  Attempting to resolve using root path 'file:///home/rgoya/pyright_test'
  Looking for typeshed stdlib path
  Looking for typeshed stdlib path
  Attempting to resolve using root path 'file:///home/rgoya/.cache/pyright-python/1.1.391/node_modules/pyright/dist/typeshed-fallback/stdlib'
  Typeshed path not found
  Finding python search paths
  Executing interpreter: 'python3'
  Skipping '/home/rgoya/.conda/lib/python310.zip' because it is not a valid directory
  Received 4 paths from interpreter
    file:///home/rgoya/.conda/lib/python3.10
    file:///home/rgoya/.conda/lib/python3.10/lib-dynload
    file:///home/rgoya/.conda/lib/python3.10/site-packages
    file:///home/rgoya/pyright_test
  Looking in python search path 'file:///home/rgoya/.conda/lib/python3.10'
  Attempting to resolve stub package using root path 'file:///home/rgoya/.conda/lib/python3.10'
  Attempting to resolve using root path 'file:///home/rgoya/.conda/lib/python3.10'
  Looking in python search path 'file:///home/rgoya/.conda/lib/python3.10/lib-dynload'
  Attempting to resolve stub package using root path 'file:///home/rgoya/.conda/lib/python3.10/lib-dynload'
  Attempting to resolve using root path 'file:///home/rgoya/.conda/lib/python3.10/lib-dynload'
  Looking in python search path 'file:///home/rgoya/.conda/lib/python3.10/site-packages'
  Attempting to resolve stub package using root path 'file:///home/rgoya/.conda/lib/python3.10/site-packages'
  Attempting to resolve using root path 'file:///home/rgoya/.conda/lib/python3.10/site-packages'
  Looking in python search path 'file:///home/rgoya/pyright_test'
  Attempting to resolve stub package using root path 'file:///home/rgoya/pyright_test'
  Attempting to resolve using root path 'file:///home/rgoya/pyright_test'
  Looking for typeshed third-party path
  Looking for typeshed stubs path
  Typeshed path not found
pyright 1.1.391
/home/rgoya/pyright_test/org/foo.py
  /home/rgoya/pyright_test/org/foo.py:1:8 - error: Import "bar" could not be resolved (reportMissingImports)
1 error, 0 warnings, 0 informations 
Completed in 0.607sec

[rgoya]

Quick followup to note that it also happens with folder imports.

To the above example I add folder org/qux with a file bar.py. The full test setup can be created as follows:

echo "/home/rgoya/pyright_test" > $CONDA_PREFIX/lib/python3.*/site-packages/pyright_test.pth
mkdir ~/pyright_test/
cd ~/pyright_test/
mkdir org
echo 'baz = "Baz!"' > org/bar.py
echo -e "import bar\n\nprint(bar.baz)" > org/foo.py
mkdir org/qux
echo 'baz = "Baz in qux.bar!"' > org/qux/bar.py
echo -e "import qux.bar\n\nprint(qux.bar.baz)" > org/foo_qux.py

Then we run the tests:

$ cd ~/pyright_test/
$ find .
.
./org
./org/foo_qux.py
./org/qux
./org/qux/bar.py
./org/bar.py
./org/foo.py
$ python org/foo.py
Baz!
$ python org/foo_qux.py 
Baz in qux.bar!
$ pyright .
0 errors, 0 warnings, 0 informations

$ # Let's trigger the bug
$ echo -e "[tool.pyright]\nexclude = []" > pyproject.toml
$ pyright .
/home/rgoya/pyright_test/org/foo.py
  /home/rgoya/pyright_test/org/foo.py:1:8 - error: Import "bar" could not be resolved (reportMissingImports)
/home/rgoya/pyright_test/org/foo_qux.py
  /home/rgoya/pyright_test/org/foo_qux.py:1:8 - error: Import "qux.bar" could not be resolved (reportMissingImports)
2 errors, 0 warnings, 0 informations

$ # ... and back
$ rm pyproject.toml 
$ pyright .
0 errors, 0 warnings, 0 informations

$ # ... and removing pyright_test.pth instead, re-establish bug
$ echo -e "[tool.pyright]\nexclude = []" > pyproject.toml
$ pyright .  # sanity check for failure
/home/rgoya/pyright_test/org/foo.py
  /home/rgoya/pyright_test/org/foo.py:1:8 - error: Import "bar" could not be resolved (reportMissingImports)
/home/rgoya/pyright_test/org/foo_qux.py
  /home/rgoya/pyright_test/org/foo_qux.py:1:8 - error: Import "qux.bar" could not be resolved (reportMissingImports)
2 errors, 0 warnings, 0 informations

$ # remove pth file... which we require so it's not an option for us
$ rm $CONDA_PREFIX/lib/python3.*/site-packages/pyright_test.pth
$ pyright .
0 errors, 0 warnings, 0 informations 

[erictraut]

Thanks for the bug report. I'm able to repro this.

@rchiodo, this looks to be a regression related to your Uri changes from earlier in the year. In particular, it came in through this pylance pull request. It looks like the logic for getParentImportResolutionRoot changed. This, in turn, causes resolveImportInternal to early return and skip the logic that searches up the directory hierarchy for an absolute import. As indicated in the bug report, this happens only when a ".pth" file is present that provides a link back to the root of the workspace. I don't have the full context for the change in this PR. I presume this was your change, so perhaps you have some additional context. Could you please take a look at it?

@rgoya, strictly speaking, this import should fail to resolve because the ".pth" is pointing to pyright_test rather than pyright_test/org. The only reason it doesn't is because pyright contains some logic that attempts to resolve an import in cases like this by walking up the directory hierarchy to find a directory that happens to work. This logic is being skipped when a ".pth" is present. You can avoid this bug by providing the correct import resolution paths. You can do this by either changing your ".pth" to point to pyright_test/org or by adding an extraPaths = ["org"] to your configuration file.

[rchiodo]

I believe @heejaechang made this change. This commit here on our side:
https://github.com/microsoft/pyrx/commit/74a3c41f1ba7ba9a0a7bf23146a5d596a4f3c1e4

I assume it was to fix a problem with imports in a standalone file?

[rgoya]

@rgoya, strictly speaking, this import should fail to resolve because the ".pth" is pointing to pyright_test rather than pyright_test/org. The only reason it doesn't is because pyright contains some logic that attempts to resolve an import in cases like this by walking up the directory hierarchy to find a directory that happens to work. This logic is being skipped when a ".pth" is present. You can avoid this bug by providing the correct import resolution paths. You can do this by either changing your ".pth" to point to pyright_test/org or by adding an extraPaths = ["org"] to your configuration file.

Thanks for reviewing, @erictraut!

Unfortunately both your suggestions are not easily doable in our environment.

The codebase is in a mono repo shared with several dozen developers, data scientists, and other user types. pyright_test/ as the root folder really has two main folders with several thousand python files each at varying depths.

The two main folders are a large "production" code folder (let's call it org) and a R&D-testing-analysis-interactive folder (let's call it org_dev).

• Code in org can import from org, but not from org_dev. This code always has long form imports (e.g. from org.pk1.mod1 import func1).
• Code in org_dev can import from org and org_dev. This code can have long imports from both (e.g. import org.pkg1.mod1, import org_dev.pkg1.mod1). There are also notebooks where users will use local imports to put utility functions, etc (e.g. import util)

Having the .pth file pointing to pyright_test/ allows us to have the org.pkg1.mod1 and org_dev.pkg1.mod1 importing structure from anywhere in our mono repo, which we rely on for interactive anaysis/coding (ipython, notebooks, etc)

Modifying pyproject.toml with extraPaths would require that every subfolder where a local import exists be added to it, which would be cumbersome to keep track of, and likely interfere with interactive development (pyproject.toml is not writeable by all). Additionally, if multiple users rely on a local import util.py, the order in which pyright searches extraPaths might have negative effects.

Going back to 1.1.365 behaviour would be ideal. That said, on our side we are also considering enforcing long form imports on org_dev as well.