pip no longer prompts for user credentials

[krobb-idnow]

Description

When installing from an index url containing environment variables for user authentication, pip no longer prompts for credentials in case the environment variables aren't set. This used to work up until v20.3.4.

Note: In our particular use case, packages are located on artifactory which returns 404 rather than 401 in case of incorrect credentials. Keyring is not an option since this needs to work in different environments where no keyring might be available.

Expected behavior

If environment variables needed for user authentication aren't set, pip should prompt for credentials.

pip version

21.1.1

Python version

3.6.9

OS

Ubuntu 18.04 LTS

How to Reproduce

This works for pip version <= 20.3.4:

> echo $USER

> echo $PASSWORD

> pip install -i https://${USER}:${PASSWORD}@my.private.index -r requirements.txt
Looking in indexes: https://:****@my.private.index
User for my.private.index:

Instead now I get this:

> echo $USER

> echo $PASSWORD

> pip install -i https://${USER}:${PASSWORD}@my.private.index -r requirements.txt
Looking in indexes: https://:****@my.private.index
ERROR: Could not find a version that satisfies the requirement XXX

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

Note: In our particular use case, packages are located on artifactory which returns 404 rather than 401 in case of incorrect credentials.

Please take this up with Artifactory folks -- providing a 401 will result in better behaviour from newer/future versions of pip (IIRC, it has a clearer error message).

url: https://:@foo.com/

Well, what a fun edge case. :)

While there's been a unintended change in behaviour here, I'm not sure that the older behaviour was correct even.

Given that in about two months, we've seen exactly one instance of this being reported, I'm inclined to say that it's unlikely that we'd be rolling back the change. Could you clarify why this change in behaviour is problematic and should be considered an issue?

[uranusjr]

url: https://:@foo.com/

While there's been a unintended change in behaviour here, I'm not sure that the older behaviour was correct even.

I wouldn’t even say it’s unintended. This is similar to the https://[email protected] vs https://foo:@example.com debate we had in another issue; a blank username/password is not the same as not providing a username/password (according to how Basic Authentication is defined), so they should be treated differently.

[krobb-idnow]

Given that in about two months, we've seen exactly one instance of this being reported, I'm inclined to say that it's unlikely that we'd be rolling back the change. Could you clarify why this change in behaviour is problematic and should be considered an issue?

At the moment, there is no way we can get a user prompt, not even if we send the user name and blank password or no password at all. Keyrings are not an option for us since we run the pip installation in different environments where no keyring might be present. We do not want to have user credentials in the terminal history, which is why we are using env vars. These env vars are only allowed to be set in certain environments; whenever they are not set, we rely on the prompt.

I'll check to see if there is anything we can do on the artifactory side, but if not, it would be great if you could take this up.

[uranusjr]

There was also a similar situation with Google Artifact Registry (they returned 403) and a user was successful asking them to change it to 401. This could be useful information for Artifactory. Please do also post an update here when you reach out toe Arfitactory so we can also join the discussion there if needed.

(Edit: forgot the link the Artifact Registry issue #9870)

[krobb-idnow]

I wouldn’t even say it’s unintended. This is similar to the https://[email protected] vs https://foo:@example.com debate we had in another issue; a blank username/password is not the same as not providing a username/password (according to how Basic Authentication is defined), so they should be treated differently.

Could you link that issue here?

[krobb-idnow]

After some more investigation, I found out that artifactory does indeed return 404 for empty credentials, which we might be able to change to 401 on our side by changing the configuration (still to be investigated).

However, when sending a username and no/empty password, artifactory returns 401 but pip still does not prompt for user credentials. This makes pip >= 21.1 unusable for us and we are stuck with the older version.

I can provide more insights if needed, just let me know.

[uranusjr]

I wouldn’t even say it’s unintended. This is similar to the https://[email protected] vs https://foo:@example.com debate we had in another issue; a blank username/password is not the same as not providing a username/password (according to how Basic Authentication is defined), so they should be treated differently.

Could you link that issue here?

This thread: #9974 (comment)

[krobb-idnow]

Just to follow up, would you consider investigating the failure to prompt in case of 401 for https://foo:@example.com, independently of env vars? Should I open another ticket for that?

[uranusjr]

I think it’s correct to not prompt. You supplied a username/password pair with foo:, so when that fails to authenticate, pip is expected to fail as in all other cases when a username/password pair is suplied. If you want pip to prompt for a password, you should not write that :.