Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

poetry lock takes forever #46

Open
mho291 opened this issue Dec 12, 2024 · 8 comments
Open

poetry lock takes forever #46

mho291 opened this issue Dec 12, 2024 · 8 comments

Comments

@mho291
Copy link
Contributor

mho291 commented Dec 12, 2024

I'm attempting to update the dependencies using poetry lock. However, it still doesn't write the lock file after more than 20 thousand seconds. I stopped it because it was using an increasing amount of swap memory.

Then I tried poetry cache clear --all pypi before running poetry lock again, but it still doesn't update.

Is there anything we can do?

Hoping @alecandido and @scarrazza can provide some suggestions. Thank you!

@alecandido
Copy link
Member

alecandido commented Dec 12, 2024

Which version of Poetry are you using? And which branch?

For me, even a pretty old version on main actually locks in a reasonable amount of time:

: poetry --version
Poetry (version 1.7.1)

❯ : poetry lock
Updating dependencies
Resolving dependencies... (13.3s)

Writing lock file

EDIT: an old version of main, I'm retrying

@mho291
Copy link
Contributor Author

mho291 commented Dec 12, 2024

I'm on Poetry (version 1.8.5)

The branch I am working on is not up yet -- it's still local.
The only change to the pyproject.toml is to add line 53, then run pytest tests/ to test for errors.
Once successful, I'll push and create a new PR.

 45    [tool.poetry.group.docs.dependencies]
 46    sphinx = "^7.2.6"
 47    recommonmark = "^0.7.1"
 48    sphinx-copybutton = "^0.5.2"
 49    nbsphinx = "^0.9.3"
 50    furo = "^2023.9.10"
 51    qiskit_ibm_runtime = "^0.17"
 52    qiskit_ibm_provider = "^0.11"
 53 +  amazon-braket-sdk = "^1.83.0"

@alecandido
Copy link
Member

Apparently the problem is generated here:

qiskit_ibm_runtime = { version = "^0.17", optional = true }
qiskit_ibm_provider = { version = "^0.11", optional = true }
amazon-braket-sdk = { version = "^1.83.0", optional = true }

if you comment either the two Qiskit-related deps, or the Braket one, then it will lock blazing fast.

I believe that the problem is (somehow) related to the (somehow) huge version space that Poetry has to take into account.

Specifically, the problem seems to be fully related to amazon-braket-sdk, since the Qiskit version specifiers (despite outdated) are effectively pinpointing to individual versions.
Indeed, if you change version to the last one (1.88.3), it locks in no time:

: poetry --version
Poetry (version 1.8.5)

❯ : poetry lock
Updating dependencies
Resolving dependencies... (5.2)

Writing lock file

What ^1.83.0 means is actually >=1.83.0, <2.0.0, which is not small (I wouldn't have called it huge, but maybe it comes with a lot of deps to analyze...).

Even replacing with just ^1.88.0 it takes much longer than ^1.88.3 (i.e. more than one minute, not sure exactly how much...).

@mho291
Copy link
Contributor Author

mho291 commented Dec 13, 2024

Thank you @alecandido , let me investigate and give it a try! Appreciate your time and advice! ;)

@BrunoLiegiBastonLiegi
Copy link
Contributor

On my end amazon-braket-sdk is blocking everything. Even commenting out the qiskit-* dependencies and fixing the version of amazon-braket-sdk to 1.88.3

[tool.poetry.dependencies]
python = ">=3.9,<3.13"
qibo = "^0.2.14"
qibo_client = "^0.0.15"
#qiskit_ibm_runtime = { version = "^0.34.0", optional = true }
#qiskit_ibm_provider = { version = "^0.11", optional = true }
amazon-braket-sdk = { version = "1.88.3", optional = true }

won't bring me through the lock generation.

@alecandido
Copy link
Member

alecandido commented Jan 8, 2025

Uhm...

: git diff -- pyproject.toml
pyproject.toml --- TOML
28 amazon-braket-sdk = { version = "^1.83.0", optional = true }       28 amazon-braket-sdk = { version = "^1.88.3", optional = true }

❯ : poetry lock
Updating dependencies
Resolving dependencies... (31.1s)

Writing lock file

❯ : poetry --version
Poetry (version 1.7.1)

❯ : git restore poetry.lock

❯ : poetry lock
Updating dependencies
Resolving dependencies... (7.9s)

Writing lock file

❯ : poetry --version
Poetry (version 1.8.5)

Just tried, not sure what is different in your case. Maybe you could try even uncommenting those lines (but I would not expect improvements).

P.S.: the apparent improvement with the Poetry version switch is almost for sure due to caching - the purpose was just to show that the timing is reasonable with both

@BrunoLiegiBastonLiegi
Copy link
Contributor

Ok apparently I had to run poetry lock --regenerate otherwise it was getting stuck, not sure about the reason though...
This way, lock generation was pretty fast (~5 s) with all the qiskit and braket dependencies set to the latest release.

@alecandido
Copy link
Member

alecandido commented Jan 9, 2025

It seems like you're already using Poetry v2.0.0, --regenerate has been introduced in there.
It seems like it was the default behavior in v1.x.

P.S.: most likely, since it was already demanding-enough without the suitable version constraints, the --no-update equivalent mode was too hard to resolve, since it was attempting to resolve with old versions of other packages - I guess...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants