Skip to content

Commit 412a5d9

Browse files
author
Dos Moonen
committed
Make it possible to request a keyring provider: auto, disabled, import or subprocess
Refactored `_get_index_url()` to get integration tests for the subprocess backend working. Keyring support via the 'subprocess' provider can only retrieve a password, not a username-password combo. The username therefor MUST come from the URL. If the URL obtained from the index does not contain a username then the username from a matching index is used. `_get_index_url()` does that matching. The problem this refactoring solves is that the URL where a wheel or sdist can be downloaded from does not always start with the index url. Azure DevOps Artifacts Feeds are an example since it replaces the friendly name of the Feed with the GUID of the Feed. Causing `url.startswith(prefix)` to evaluate as `False`. The new behaviour is to return the index which matches the netloc and has the longest common prefix of the `path` property of the value returned by `urllib.parse.urlsplit()`. The behaviour for resolving ties is unspecified.
1 parent 26d914f commit 412a5d9

File tree

9 files changed

+495
-97
lines changed

9 files changed

+495
-97
lines changed

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ repos:
4747
additional_dependencies: [
4848
'keyring==23.0.1',
4949
'nox==2021.6.12',
50-
'pytest==7.1.1',
50+
'pytest',
5151
'types-docutils==0.18.3',
5252
'types-setuptools==57.4.14',
5353
'types-freezegun==1.1.9',

docs/html/topics/authentication.md

Lines changed: 109 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,12 +66,118 @@ man pages][netrc-docs].
6666
## Keyring Support
6767

6868
pip supports loading credentials stored in your keyring using the
69-
{pypi}`keyring` library.
69+
{pypi}`keyring` library which can be enabled py passing `--keyring-provider`
70+
with a value of `auto`, `disabled`, `import` or `subprocess`. The default value
71+
is `auto`. `auto` will respect `--no-input` and not query keyring at all if that
72+
option is used. The `auto` provider will use the `import` provider if the
73+
`keyring` module can be imported. If that is not the case it will use the
74+
`subprocess` provider if a `keyring` executable can be found. Otherwise, the
75+
`disabled` provider will be used.
76+
77+
### Configuring Pip
78+
Passing this as a command line argument will work, but is not how the majority
79+
of this feature's users will use it. They instead will want to overwrite the
80+
default of `disabled` in the global, user of site configuration file:
81+
```bash
82+
$ pip config set --global global.keyring-provider subprocess
83+
84+
# A different user on the same system which has PYTHONPATH configured and and
85+
# wanting to use keyring installed that way could then run
86+
$ pip config set --user global.keyring-provider import
87+
88+
# For a specific virtual environment you might want to use disable it again
89+
# because you will only be using PyPI and the private repo (and mirror)
90+
# requires 2FA with a keycard and a pincode
91+
$ pip config set --site global.index https://pypi.org/simple
92+
$ pip config set --site global.keyring-provider disabled
93+
94+
# configuring it via environment variable is also possible
95+
$ export PIP_KEYRING_PROVIDER=disabled
96+
```
97+
98+
### Installing and using the keyring python module
99+
100+
Setting it to `import` tries to communicate with `keyring` by importing it
101+
and using its Python api.
70102

71103
```bash
72-
$ pip install keyring # install keyring from PyPI
104+
# install keyring from PyPI
105+
$ pip install keyring --index-url https://pypi.org/simple
73106
$ echo "your-password" | keyring set pypi.company.com your-username
74-
$ pip install your-package --index-url https://pypi.company.com/
107+
$ pip install your-package --keyring-provider import --index-url https://pypi.company.com/
108+
```
109+
110+
### Installing and using the keyring cli application
111+
112+
Setting it to `subprocess` will look for a `keyring` executable on the PATH
113+
if one can be found that is different from the `keyring` installation `import`
114+
would be using.
115+
116+
The cli requires a username, therefore you MUST put a username in the url.
117+
See the example below or the basic HTTP authentication section at the top of
118+
this page.
119+
120+
```bash
121+
# install keyring from PyPI using pipx, which we assume if installed properly
122+
# you can also create a venv somewhere and add it to the PATH yourself instead
123+
$ pipx install keyring --index-url https://pypi.org/simple
124+
125+
# install the keyring backend for Azure DevOps for example
126+
# VssSessionToken is the username you MUST use for this backend
127+
$ pipx inject keyring artifacts-keyring --index-url https://pypi.org/simple
128+
129+
# or the one for Google Artifact Registry
130+
$ pipx inject keyring keyrings.google-artifactregistry-auth --index-url https://pypi.org/simple
131+
$ gcloud auth login
132+
133+
$ pip install your-package --keyring-provider subprocess --index-url https://[email protected]/
134+
```
135+
136+
### Here be dragons
137+
138+
The `auto` provider is conservative and does not query keyring at all when
139+
`--no-input` is used because the keyring might require user interaction such as
140+
prompting the user on the console. Third party tools frequently call Pip for
141+
you and do indeed pass `--no-input` as they are well-behaved and don't have
142+
much information to work with. (Keyring does have an api to request a backend
143+
that does not require user input.) You have more information about your system,
144+
however!
145+
146+
You can force keyring usage by requesting a keyring provider other than `auto`
147+
(or `disabled`). Leaving `import` and `subprocess`. You do this by passing
148+
`--keyring-provider import` or one of the following methods:
149+
150+
```bash
151+
# via config file, possibly with --user, --global or --site
152+
$ pip config set global.keyring-provider subprocess
153+
# or via environment variable
154+
$ export PIP_KEYRING_PROVIDER=import
155+
```
156+
157+
```{warning}
158+
Be careful when doing this since it could cause tools such as Pipx and Pipenv
159+
to appear to hang. They show their own progress indicator while hiding output
160+
from the subprocess in which they run Pip. You won't know whether the keyring
161+
backend is waiting the user input or not in such situations.
162+
```
163+
164+
Pip is conservative and does not query keyring at all when `--no-input` is used
165+
because the keyring might require user interaction such as prompting the user
166+
on the console. You can force keyring usage by passing `--force-keyring` or one
167+
of the following:
168+
169+
```bash
170+
# possibly with --user, --global or --site
171+
$ pip config set global.force-keyring true
172+
# or
173+
$ export PIP_FORCE_KEYRING=1
174+
```
175+
176+
```{warning}
177+
Be careful when doing this since it could cause tools such as Pipx and Pipenv
178+
to appear to hang. They show their own progress indicator while hiding output
179+
from the subprocess in which they run Pip. You won't know whether the keyring
180+
backend is waiting the user input or not in such situations.
75181
```
76182

77183
Note that `keyring` (the Python package) needs to be installed separately from

news/8719.feature.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add ``--keyring-provider`` flag. See the Authentication page in the documentation for more info.

src/pip/_internal/cli/cmdoptions.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,19 @@ class PipOption(Option):
244244
help="Disable prompting for input.",
245245
)
246246

247+
keyring_provider: Callable[..., Option] = partial(
248+
Option,
249+
"--keyring-provider",
250+
dest="keyring_provider",
251+
choices=["auto", "disabled", "import", "subprocess"],
252+
default="disabled",
253+
help=(
254+
"Enable the credential lookup via the keyring library if user input is allowed."
255+
" Specify which mechanism to use [disabled, import, subprocess]."
256+
" (default: disabled)"
257+
),
258+
)
259+
247260
proxy: Callable[..., Option] = partial(
248261
Option,
249262
"--proxy",
@@ -1019,6 +1032,7 @@ def check_list_path_option(options: Values) -> None:
10191032
quiet,
10201033
log,
10211034
no_input,
1035+
keyring_provider,
10221036
proxy,
10231037
retries,
10241038
timeout,

src/pip/_internal/cli/req_command.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,14 @@ def _build_session(
151151

152152
# Determine if we can prompt the user for authentication or not
153153
session.auth.prompting = not options.no_input
154+
session.auth.keyring_provider = options.keyring_provider
155+
# We won't use keyring when --no-input is passed unless
156+
# a specific provider is requested because it might require
157+
# user interaction
158+
session.auth.use_keyring = (
159+
session.auth.prompting
160+
or options.keyring_provider not in ["disabled", "auto"]
161+
)
154162

155163
return session
156164

0 commit comments

Comments
 (0)