Allow to skip dependencies (--no-deps) for individual requirements

[blueyed]

It would be useful if you could specify --no-deps for some dependencies, where you do not want pip-compile to include the dependencies, but control it manually.

pip's own options are handled via https://github.com/pypa/pip/blob/f8732ac06dcb7c6af81dcdc0cc3158196b796baf/src/pip/_internal/req/req_file.py#L283 (e.g. --hash).

echo '--no-deps channels' | pip-compile -q - -o -

This might also make sense as a global option to pip-compile, but usually is more important for single deps only (i.e. in a line in the requirements.in file).

[graingert]

I think this is close to #215 eg you want to make sure a package is not installed and a replacement is installed instead, or you want to force a specific version when a package is too strict about its deps.

I believe --no-deps is a bit of a sledgehammer solution to #215

[blueyed]

For me --no-deps is useful since e.g. "channels" pulls in "daphne", which then pulls in twisted and much more (django/channels#1285), which are really optional deps.

[atugushev]

FTR, --no-deps is not supported option for pip install -r requiremtns.txt. See:

$ cat requirements.txt
--no-deps django
django

$ pip install -r requirements.txt
Usage: pip [options]

ERROR: Invalid requirement: --no-deps django
pip: error: no such option: --no-deps

[blueyed]

Yeah, it should not end up there, but get consumed / used by pip-compile.
It should also apply to a single dep only, not to all of them.

[atugushev]

The --no-deps is global option. So what's the point? It's not gonna work if there are N packages in reqs.in with deps, because it'll be all installed without deps.

[blueyed]

I've thought about pip-compile using this internally only.
I do not know the internals, but I've thought that pip-compile could resolve those with the "--no-deps" option then, and merging them later - assuming that all reqs are resolved in one batch currently.
Pip's resolver understands the option: https://github.com/pypa/pip/blob/f8732ac06dcb7c6af81dcdc0cc3158196b796baf/src/pip/_internal/resolve.py#L41-L55

[atugushev]

Pip's resolver understands the option: https://github.com/pypa/pip/blob/f8732ac06dcb7c6af81dcdc0cc3158196b796baf/src/pip/_internal/resolve.py#L41-L55

Looks like ignore_dependencies works globally.

[blueyed]

@atugushev
ignore_dependencies is an option to the resolver.
Could another resolver be used here?

pip-tools/piptools/repositories/pypi.py

Line 221 in 0da08e5

resolver = PipResolver(**resolver_kwargs)

(for all requirements that have a --no-deps option (similar to the existing -e option that can be specified with requirements.in entries))

[AndydeCleyre]

Can we get a clearer picture of the feature requirement?

Does the problem go beyond particular packages specifying hard deps instead of using extras groups?

[AndydeCleyre]

EDIT: This was wrong!

Original

I don't think this particular workaround (which I think achieves the desired behavior) has been articulated here yet:

requirements.in:

-r reqs-with-deps.txt
req-without-dep1
req-without-dep2

reqs-with-deps.in:

req1
req2

$ pip-compile reqs-with-deps.in && pip-compile --pip-args --no-deps requirements.in

[UncleGoogle]

@AndydeCleyre

pip-compile --pip-args --no-deps requirements.in

this seems to not work as expected for compilation. "deps without dependencies" are still resolved to full list of dependencies given in requrirements.in.

I works only with pip install or pip-sync as you mentioned earlier: #817 (comment)

My use case is that:

• we have some pre-compiled packages in private index (eg wxpython) but without its all dependencies uploaded to private index
• we want to have a lock file with hashes
• we want to install our own dependencies along with external (from public pypi)

As --index-url are global options, my idea was to have a common, final lock file with hashes so that we can install with
--index-url=private --extra-index-url=https://pypi.org/simple safely (hashes should make sure that the correct version of eg wxpython will be taken during installation).

So thought about something similar to yours approach

# external.in
-r internal.txt
ext-pkg1

pip-compile internal.in --generate-hashes --index-url=https://private --pip-args --no-deps
pip-compile external.in --generate-hashes

(the first commad just to generate hashes of our pkges, then prepare all deps tree)
But the first one doesn't work

Is there any other way/workaround to dump hashes of given list of requrieemtns WITHOUT resolving all their dependencies?

Before "hashes approach" I was using sed tricks to cleanup stuff from final requirements file, but with --hashes it becomes cumbersome.

[AndydeCleyre]

@UncleGoogle

You're right, I'm sorry!

---

I don't work much with private indexes, so forgive my ignorance here, but why do you need to avoid resolving the dependencies? Why not compile them all, passing --index-url=PRIVATE --extra-index-url=PYPI to pip-compile? Maybe that doesn't work as I expect.

---

I'll mention that nowadays uv can generate TOML lockfiles, which you could grab data from. Here's a hacky example of grabbing just requests with its version and hash from a uv.lock, using tools I like (Zsh, NestedTextTo, yamlpath, wheezy.template), but you'd probably choose some different ones:

$ wheezy.template =(<<<'@require(version, wheels)
requests==@version\
@for wheel in wheels:
 --hash=@wheel["hash"]
@end') =(toml2nt uv.lock | nt2json | yaml-get -p 'package.[name=requests]')

yielding:

requests==2.32.3 --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6