VCS editable regresses to using deprecated `egg=` syntax

[mgmarino]

Issue description

This is basically the same issue noted here #5626, it seems to be have been reintroduced by #5928.

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyjwt = {extras = ["crypto"], editable = true, ref = "2.6.0", git = "ssh://git@github.com/jpadilla/pyjwt.git"}

[dev-packages]

[requires]
python_version = "3.9"

The easiest way to see this is, after lock, the older pipenv returns the expected new syntax:

> pipenv --version
pipenv, version 2023.9.1
 > pipenv requirements
-i https://pypi.org/simple
cffi==1.16.0; python_version >= '3.8'
cryptography==41.0.4
pycparser==2.21
pyjwt[crypto]@ git+ssh://git@github.com/jpadilla/pyjwt.git@7665aa625506a11bae50b56d3e04413a3dc6fdf8

The newest version does not:

> pipenv --version
pipenv, version 2023.10.3
> pipenv requirements
-i https://pypi.org/simple
cffi==1.16.0; python_version >= '3.8'
cryptography==41.0.4
pycparser==2.21
-e git+ssh://git@github.com/jpadilla/pyjwt.git@7665aa625506a11bae50b56d3e04413a3dc6fdf8#egg=pyjwt[crypto]

The newer version has the deprecated warnings, as well, that have now returned, e.g.:

> pipenv sync --dev
PipDeprecationWarning: DEPRECATION: git+ssh://****@github.com/jpadilla/pyjwt.git@7665aa625506a11bae50b56d3e04413a3dc6fdf8#egg=pyjwt[crypto] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617
  deprecated(

EDIT:
As a follow-up, I should note, also, that without editable, it also generates a deprecation warning on lock:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyjwt = {extras = ["crypto"], ref = "2.6.0", git = "ssh://git@github.com/jpadilla/pyjwt.git"}

[dev-packages]

[requires]
python_version = "3.9"
~

And then lock:

pipenv lock
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
⠴ Locking...Warning: INFO:pip.subprocessor:Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/jpadilla/pyjwt.git' /private/var/folders/jh/0_l94d8n51n2gjzt_5l8mv8r0000gq/T/pip-temp-tbxaqx59/pyjwt_2de1d1d332894184a987c8ebdc9489d2
INFO:pip.subprocessor:Running command git checkout -q 7665aa625506a11bae50b56d3e04413a3dc6fdf8
INFO:pip.subprocessor:Running command git clone --filter=blob:none 'ssh://****@github.com/jpadilla/pyjwt.git' /var/folders/jh/0_l94d8n51n2gjzt_5l8mv8r0000gq/T/tmplbwvrzyq
INFO:pip.subprocessor:Cloning into '/var/folders/jh/0_l94d8n51n2gjzt_5l8mv8r0000gq/T/tmplbwvrzyq'...
INFO:pip.subprocessor:Updating files:   2% (2/87)
INFO:pip.subprocessor:Updating files:   3% (3/87)
INFO:pip.subprocessor:Updating files:   4% (4/87)
INFO:pip.subprocessor:Updating files:   5% (5/87)
INFO:pip.subprocessor:Updating files:   6% (6/87)
INFO:pip.subprocessor:Updating files:   8% (7/87)
INFO:pip.subprocessor:Updating files:   9% (8/87)
INFO:pip.subprocessor:Updating files:  10% (9/87)
INFO:pip.subprocessor:Updating files:  11% (10/87)
INFO:pip.subprocessor:Updating files:  12% (11/87)
INFO:pip.subprocessor:Updating files:  13% (12/87)
INFO:pip.subprocessor:Updating files:  14% (13/87)
INFO:pip.subprocessor:Updating files:  16% (14/87)
INFO:pip.subprocessor:Updating files:  17% (15/87)
INFO:pip.subprocessor:Updating files:  18% (16/87)
INFO:pip.subprocessor:Updating files:  19% (17/87)
INFO:pip.subprocessor:Updating files:  20% (18/87)
INFO:pip.subprocessor:Updating files:  21% (19/87)
INFO:pip.subprocessor:Updating files:  22% (20/87)
INFO:pip.subprocessor:Updating files:  24% (21/87)
INFO:pip.subprocessor:Updating files:  25% (22/87)
INFO:pip.subprocessor:Updating files:  26% (23/87)
INFO:pip.subprocessor:Updating files:  27% (24/87)
INFO:pip.subprocessor:Updating files:  28% (25/87)
INFO:pip.subprocessor:Updating files:  29% (26/87)
INFO:pip.subprocessor:Updating files:  31% (27/87)
INFO:pip.subprocessor:Updating files:  32% (28/87)
INFO:pip.subprocessor:Updating files:  33% (29/87)
INFO:pip.subprocessor:Updating files:  34% (30/87)
INFO:pip.subprocessor:Updating files:  35% (31/87)
INFO:pip.subprocessor:Updating files:  36% (32/87)
INFO:pip.subprocessor:Updating files:  37% (33/87)
INFO:pip.subprocessor:Updating files:  39% (34/87)
INFO:pip.subprocessor:Updating files:  40% (35/87)
INFO:pip.subprocessor:Updating files:  41% (36/87)
INFO:pip.subprocessor:Updating files:  42% (37/87)
INFO:pip.subprocessor:Updating files:  43% (38/87)
INFO:pip.subprocessor:Updating files:  44% (39/87)
INFO:pip.subprocessor:Updating files:  45% (40/87)
INFO:pip.subprocessor:Updating files:  47% (41/87)
INFO:pip.subprocessor:Updating files:  48% (42/87)
INFO:pip.subprocessor:Updating files:  49% (43/87)
INFO:pip.subprocessor:Updating files:  50% (44/87)
INFO:pip.subprocessor:Updating files:  51% (45/87)
INFO:pip.subprocessor:Updating files:  52% (46/87)
INFO:pip.subprocessor:Updating files:  54% (47/87)
INFO:pip.subprocessor:Updating files:  55% (48/87)
INFO:pip.subprocessor:Updating files:  56% (49/87)
INFO:pip.subprocessor:Updating files:  57% (50/87)
INFO:pip.subprocessor:Updating files:  58% (51/87)
INFO:pip.subprocessor:Updating files:  59% (52/87)
INFO:pip.subprocessor:Updating files:  60% (53/87)
INFO:pip.subprocessor:Updating files:  62% (54/87)
INFO:pip.subprocessor:Updating files:  63% (55/87)
INFO:pip.subprocessor:Updating files:  64% (56/87)
INFO:pip.subprocessor:Updating files:  65% (57/87)
INFO:pip.subprocessor:Updating files:  66% (58/87)
INFO:pip.subprocessor:Updating files:  67% (59/87)
INFO:pip.subprocessor:Updating files:  68% (60/87)
INFO:pip.subprocessor:Updating files:  70% (61/87)
INFO:pip.subprocessor:Updating files:  71% (62/87)
INFO:pip.subprocessor:Updating files:  72% (63/87)
INFO:pip.subprocessor:Updating files:  73% (64/87)
INFO:pip.subprocessor:Updating files:  74% (65/87)
INFO:pip.subprocessor:Updating files:  75% (66/87)
INFO:pip.subprocessor:Updating files:  77% (67/87)
INFO:pip.subprocessor:Updating files:  78% (68/87)
INFO:pip.subprocessor:Updating files:  79% (69/87)
INFO:pip.subprocessor:Updating files:  80% (70/87)
INFO:pip.subprocessor:Updating files:  81% (71/87)
INFO:pip.subprocessor:Updating files:  82% (72/87)
INFO:pip.subprocessor:Updating files:  83% (73/87)
INFO:pip.subprocessor:Updating files:  85% (74/87)
INFO:pip.subprocessor:Updating files:  86% (75/87)
INFO:pip.subprocessor:Updating files:  87% (76/87)
INFO:pip.subprocessor:Updating files:  88% (77/87)
INFO:pip.subprocessor:Updating files:  89% (78/87)
INFO:pip.subprocessor:Updating files:  90% (79/87)
INFO:pip.subprocessor:Updating files:  91% (80/87)
INFO:pip.subprocessor:Updating files:  93% (81/87)
INFO:pip.subprocessor:Updating files:  94% (82/87)
INFO:pip.subprocessor:Updating files:  95% (83/87)
INFO:pip.subprocessor:Updating files:  96% (84/87)
INFO:pip.subprocessor:Updating files:  97% (85/87)
INFO:pip.subprocessor:Updating files:  98% (86/87)
INFO:pip.subprocessor:Updating files: 100% (87/87)
INFO:pip.subprocessor:Updating files: 100% (87/87), done.
INFO:pip.subprocessor:Running command git checkout -q 7665aa625506a11bae50b56d3e04413a3dc6fdf8
/.../.pyenv/versions/3.9.17/lib/python3.9/site-packages/pipenv/patched/pip/_internal/models/link.py:445: PipDeprecationWarning: DEPRECATION: git+ssh://****@github.com/jpadilla/pyjwt.git@7665aa625506a11bae50b56d3e04413a3dc6fdf8[crypto]#egg=pyjwt[crypto] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617
  deprecated(
Locking [dev-packages] dependencies...
Updated Pipfile.lock (a9d82110488b564f4c7da14f298280fd55109dab9f3889067b3104fc9227efeb)!

[matteius]

I don't see where the pipenv utils would be using the old format for non-edtiable; if it was editable: when I took this deprecation warning up originally, it appears that the new format doesn't work for editable.

[mgmarino]

if it was editable: when I took this deprecation warning up originally, it appears that the new format doesn't work for editable.

I would agree that the support for this is currently confusing, see e.g. pypa/pip#11676 (comment) and related links. It's not clear to me what the pip devs intend to do.

That being said, I actually am not sure if the editable requirement is even needed for most VCS installs, I think this is possibly a relic of old pipenv docs that "suggested" doing this.

I don't see where the pipenv utils would be using the old format for non-edtiable;

Ok, I had a look here, I think the issue was when running pipenv lock where there was an old Pipfile.lock that had the editable settings still (even if removed from the Pipfile), it then throws the warning. This would then imply it uses the settings from the old Pipfile.lock at some point. You can reproduce as the following:

>  grep pyjwt Pipfile
pyjwt = {extras = ["crypto"], ref = "2.6.0", editable = true, git = "ssh://git@github.com/jpadilla/pyjwt.git"}
> pipfile lock
# outputs with warnings, but acceptable
> # modify Pipfile to remove editable
> grep pyjwt Pipfile
pyjwt = {extras = ["crypto"], ref = "2.6.0", git = "ssh://git@github.com/jpadilla/pyjwt.git"}
> pipfile lock -vvv

will output at the end:

INFO:pip.subprocessor:Running command git checkout -q 7665aa625506a11bae50b56d3e04413a3dc6fdf8
INFO:pipenv.patched.pip._internal.vcs.git:Resolved ssh://****@github.com/jpadilla/pyjwt.git to commit 7665aa625506a11bae50b56d3e04413a3dc6fdf8
/Users/mgmarino/.pyenv/versions/3.9.17/lib/python3.9/site-packages/pipenv/patched/pip/_internal/models/link.py:445: PipDeprecationWarning: DEPRECATION:
git+ssh://****@github.com/jpadilla/pyjwt.git@7665aa625506a11bae50b56d3e04413a3dc6fdf8#egg=pyjwt contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A
possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617

subsequent pipfile lock calls will not.

I'm not sure if this is an issue, tbh, even though I would argue it's unexpected, so I'm happy to close.

[matteius]

I actually am not sure if the editable requirement is even needed for most VCS installs

I wasn't sure of that either when I did the refactor away from requirementslib, but it turns out it affects which directory in the virtualenv it gets installed into, and some users have expectations around that.

Sounds like the lock file retains a prior editable setting even if it was removed in subsequent lock? If so that sounds like an edge case we should clean up