Has anyone looked into using the uv package manager, yet? #2175
Comments
creative-resort
added
the
type:Enhancement
|
Mamba is a re-implementation of conda, so although it would be nice for the mamba docs to be self-contained it seems OK to refer to the conda docs.
I agree that uv sounds very interesting and it's something I'm planning to try, but given it's still relatively new (announced less than a year ago) I don't think it should be installed by default yet, especially since it's trivial to install yourself. |
What makes you say that?
That's true, but I don't think there is a much better solution.
Unfortunately, there are some big downsides to using
That being said,
Sure, let's keep this issue open for a month to at least gather some feedback. |
|
|
|
Hopping on this discussion with a related question/concern. How do you suggest that we lock Python package versions that we're install on top of your images? For example, if I'm building an image on top of the SciPy Dockerfile here, some Python packages are installed using However, the recommended way to add more Python packages to an image built on-top is just to use |
|
Layered approaches like these 'stacks' are not very well suited to lockfiles, which typically assume a single install command for everything. You might have a better experience going all-in on tooling like pixi/Poetry/etc. to manage a whole env, rather than adding to an existing one, whether you start from these images or not. But if you do want to pip install a few packages on top of one of these base images and lock just what you install, I think the way to go is to generate a constraints file from your chosen base image and include it as an input to your lock file generation. You can generate the constraints file by running in the base environment: And then put your dependencies in a For example, this script will run pip-compile to generate a locked # lock.sh:
#!/bin/sh
set -eux
# must match FROM in Dockerfile to work correctly,
# and must be a strictly tagged base image
base_image=quay.io/jupyter/minimal-notebook:2024-07-08
# generate constraints file from base image
# then run `pip-compile -c base-env.txt` to generate locked requirements.txt
# constraining any dependencies to what's already in the base env
docker run --rm -it -v$PWD:/io -w /io $base_image sh -c "pip list --format=freeze > base-env.txt; pip install pip-tools; pip-compile -c base-env.txt -r requirements.in"# requirements.in
altairThen your Dockerfile uses the locked requirements.txt and the same base image tag (maybe sync in your build automation with # must always use a pinned image for locking to work
ARG base_image=quay.io/jupyter/minimal-notebook:2024-07-08
FROM $base_image
# the locked requirements.txt generated by lock.sh
COPY requirements.txt /requirements.txt
RUN pip install --no-cache -r /requirements.txtThe output of the You will need to re-run |
For example, a clone of an environment is supposed to be saving space by symlinking resources from the cloned (base) environment, just like with conda. That's not what is happening, though, with mamba.
There's more (at first sight) hickups, that have had unresolved issues open for years, that one stumbles upon in practice, that actually amount to roadblocks and time wasters, that one needs to figure out tediously, in order to find eventually at least workarounds for. That's why I'm saying it's still in its infancy. |
I would be more than willing to use Poetry, I use it over the place - however if I want to get the functionality from the Jupyter images (startup scripts etc.) I'm somewhat hamstrung into using Conda and trying to make the best of it. Currently toying with a solution where I maintain a Poetry project containing my extra dependencies and creating a version locked |
|
@james-ro-williams have you tried the foundation image? |
I don't think that's the case. Python itself may have been installed by conda, but that's about it if you want to create and use an env with poetry in the image. You don't even have to use that Python, but it seems like a bit of a waste of space not to. As @manics pointed out, docker-stacks-foundation is probably the lightest image to use if you aren't going to be using the stack envs. Here's a sample of building an env entirely controlled with poetry, using only the Python installed in the foundation image: https://gist.github.com/minrk/063fe9798315738bed48df822c671615 with all the benefits of startup scripts, entrypoints, etc. in these images. Pip's support for constraint files probably does make But with all that said, you may well be better off creating your own images if you prefer a different set of tools, and just using these as a reference for how you might go about it. I think there is sometimes a misconception that these images are the "one true way" to run Jupyter in a container, when they are really just "one way." It really is fine and often preferable to build your own images to meet your own needs. It does not need to be as complex as these. |
What docker image(s) is this feature applicable to?
docker-stacks-foundation
What change(s) are you proposing?
https://github.com/astral-sh/uv
https://docs.astral.sh/uv/
I'm seeing Mamba / Micromamba not being able to get off the ground, not even with proper documentation – one always has to refer to the conda docs.
UV offers the package + code project handling, Python programmers are looking for, especially those working with multiple python kernel environments.
How does this affect the user?
It is not going to be a replacement for Mamba, as it doesn't support non-Python packages,
but may be the right step in the direction of well appreciated future support for Python applications with Jupyter, for Python developers looking to get the benefits from uv, that they are soon going to be used to.
Anything else?
Having uv pre-installed in the stack for python projects is the idea to explore.
The text was updated successfully, but these errors were encountered: