Skip to content

Use robust shebang manipulation also for (data)/scripts (e.g. to support spaces in venv paths) #13389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task done
flying-sheep opened this issue May 9, 2025 · 9 comments · May be fixed by #13390
Open
1 task done
Labels
type: bug A confirmed bug or unintended behavior

Comments

@flying-sheep
Copy link

flying-sheep commented May 9, 2025

Description

pip install adds the target interpreter’s path to script shebangs both for scripts defined via entry points and for scripts in (data)/scripts (i.e. the wheel path pkg_name-1.2.2.data/scripts/).

However for the latter, it uses more brittle code to create the shebang.

So this:

exename = sys.executable.encode(sys.getfilesystemencoding())
firstline = b"#!" + exename + os.linesep.encode("ascii")

should be changed to use this code:

def _get_shebang(self, encoding, post_interp=b'', options=None):

See: ranger/ranger#2941 (comment)
See also:

Expected behavior

Shebang is correctly modified

pip version

25.0.1

Python version

3.13

OS

Linux/MacOS

How to Reproduce & Output

$ mkdir "path with spaces"
$ cd "path with spaces"
$ python -m venv .venv
$ .venv/bin/pip install ranger-fm==1.9.4
$ head -n1 .venv/bin/ranger
#!/home/me/path with spaces/.venv/bin/python
$ .venv/bin/ranger
zsh:1: .venv/bin/ranger: bad interpreter: /home/me/path: no such file or directory

Code of Conduct

@flying-sheep flying-sheep added type: bug A confirmed bug or unintended behavior S: needs triage Issues/PRs that need to be triaged labels May 9, 2025
@pypa pypa locked as spam and limited conversation to collaborators May 10, 2025
@ichard26
Copy link
Member

I've reported the spam to GitHub Support. In the meanwhile, I'll keep this issue locked. Sorry about the noise!

@pypa pypa unlocked this conversation May 11, 2025
@pradyunsg
Copy link
Member

(unlocked since the accounts have been deleted)

@pradyunsg pradyunsg removed the S: needs triage Issues/PRs that need to be triaged label May 11, 2025
@flying-sheep
Copy link
Author

I can make a PR here and will probably start by just calling that private function I linked to.

Is the process here to promote things like this to public functions in distlib before a PR can be merged or do we care little or not because we vendor things anyway and therefore don’t risk breaking when using private functions?

@pfmoore
Copy link
Member

pfmoore commented May 11, 2025

Personally, I'm uncomfortable relying on undocumented internals of our dependencies. We insist that people only use our documented API, so it seems hypocritical to not follow the same rule with regard to our dependencies.

@notatallshaw
Copy link
Member

notatallshaw commented May 11, 2025

No idea if you already have, but I do think it's worth comparing to what uv does, which I've seen go through a few rounds of trying to handle spaces in venvs.

@flying-sheep
Copy link
Author

uv uses the '''exec' … workaround for both [project.scripts] and *.data/scripts/.

@flying-sheep
Copy link
Author

How does a proper dev setup for pip work?

Seems like not only does nox not do a dev install (which makes breakpoints set in VS Code useless), tests like the one I modify copy pip into the venv which also makes breakpoints useless.

Can I make it so pip dev-installs itself into the venvs it creates during testing or otherwise just make breakpoints work normally?

@pfmoore
Copy link
Member

pfmoore commented May 11, 2025

By the way, I'll note that script files are generally considered to be deprecated (although no standard has formally done so yet). I'm not convinced we should be spending effort on adding support to script files as opposed to directing people to use entry points.

On Windows, scripts with a shebang file are not natively supported by the OS, they require workarounds using file associations and the py launcher, which don't work in all circumstances. They very definitely don't provide a production-quality user experience, unlike entry points (which use a native executable).

Any Python script can very easily be converted into a callable function within the application, and exposed via an entry point, so it seems like there are very few reasonable use cases which would demand a script rather than an entry point. While "it's an unnecessary cost" is a prefectly good reason for not just wholesale rewriting scripts to entry points, doing so to provide a more robust user experience does seem like a good reason - and IMO better than expecting changes to pip which will allow the script approach to last for a little longer.

I appreciate that scripts with a shebang are a much more reasonable solution on POSIX, and not everything has to be cross-platform, but conversely, pip's features do have to be and that makes supporting things in pip harder.

@flying-sheep
Copy link
Author

pipx is held back from using standard directories for its data files (and maybe also cache files?) on MacOS because of this issue. I don’t think they’ll consider breaking things, so unless things get formally deprecated within the next weeks, I think we should go forward here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A confirmed bug or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

19 participants
@flying-sheep @pfmoore @pradyunsg @notatallshaw @ichard26 and others