Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce more docker tags for uv #6053

Merged
merged 1 commit into from
Sep 3, 2024

Conversation

samypr100
Copy link
Contributor

@samypr100 samypr100 commented Aug 13, 2024

Summary

Closes #5610

This PR introduces additional images with the uv/uvx binaries from scratch for both amd64/arm64 and make the mapping easy to configure by generating the Dockerfile on the fly. This approach focuses on minimizing CI time by taking advantage of dedicating a worker per mapping (20-30s~ per job).

This PR also fixes org.opencontainers.image.version for all tags (including the one from scratch) to contain the right release version instead of branch name main (default when no tag patterns are specified).

For example, on release x.y.z, this will publish the following image tags with format ghcr.io/astral-sh/uv:{tag} with manifests for both amd64/arm64. This also include x.y tags for each respective additional tag.

  • From scratch: latest, x.y.z, x.y (currently being published)
  • From alpine:3.20: alpine, alpine3.20, x.y.z-alpine, x.y.z-alpine3.20
  • From debian:bookworm-slim: debian-slim, bookworm-slim, x.y.z-debian-slim, x.y.z-bookworm-slim
  • From buildpack-deps:bookworm: debian, bookworm, x.y.z-debian, x.y.z-bookworm
  • From python:3.12-alpine: python3.12-alpine, x.y.z-python3.12-alpine
  • From python:3.11-alpine: python3.11-alpine, x.y.z-python3.11-alpine
  • From python:3.10-alpine: python3.10-alpine, x.y.z-python3.10-alpine
  • From python:3.9-alpine: python3.9-alpine, x.y.z-python3.9-alpine
  • From python:3.8-alpine: python3.8-alpine, x.y.z-python3.8-alpine
  • From python:3.12-bookworm: python3.12-bookworm, x.y.z-python3.12-bookworm
  • From python:3.11-bookworm: python3.11-bookworm, x.y.z-python3.11-bookworm
  • From python:3.10-bookworm: python3.10-bookworm, x.y.z-python3.10-bookworm
  • From python:3.9-bookworm: python3.9-bookworm, x.y.z-python3.9-bookworm
  • From python:3.8-bookworm: python3.8-bookworm, x.y.z-python3.8-bookworm
  • From python:3.12-slim-bookworm: python3.12-bookworm-slim, x.y.z-python3.12-bookworm-slim
  • From python:3.11-slim-bookworm: python3.11-bookworm-slim, x.y.z-python3.11-bookworm-slim
  • From python:3.10-slim-bookworm: python3.10-bookworm-slim, x.y.z-python3.10-bookworm-slim
  • From python:3.9-slim-bookworm: python3.9-bookworm-slim, x.y.z-python3.9-bookworm-slim
  • From python:3.8-slim-bookworm: python3.8-bookworm-slim, x.y.z-python3.8-bookworm-slim

@samypr100
Copy link
Contributor Author

Below is what https://github.com/astral-sh/uv/pkgs would end up looking like with the additional tags (just for 0.2.35).

uv-tags

@samypr100 samypr100 marked this pull request as ready for review August 13, 2024 12:24
@zanieb
Copy link
Member

zanieb commented Aug 13, 2024

Epic! Thank you! I'll review this one.

@zanieb zanieb self-assigned this Aug 13, 2024
Copy link
Member

@zanieb zanieb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I might play with the job names so they're more readable? but otherwise it seems great. I like the approach you took.

@zanieb
Copy link
Member

zanieb commented Aug 13, 2024

I would like to hear a bit more about what motivating adding quite so many tags, I was imagining a less :)

@zanieb zanieb added the releases Related to building and distributing release artifacts of uv label Aug 13, 2024
@samypr100
Copy link
Contributor Author

This looks good to me. I might play with the job names so they're more readable? but otherwise it seems great. I like the approach you took.

Happy to rename the jobs, lmk

@samypr100
Copy link
Contributor Author

samypr100 commented Aug 13, 2024

I would like to hear a bit more about what motivating adding quite so many tags, I was imagining a less :)

I primarily mimicked supported patterns from docker hub official images for python and general use (non-bundled python ones) and the tags they support. I considered the feedback / use cases from @konstin and yours on #5610 (comment) as well. I thought this was a good starting point.

@zanieb
Copy link
Member

zanieb commented Aug 16, 2024

Cool, thanks for explaining. Just waiting for consensus from more of the team here.

zanieb pushed a commit that referenced this pull request Aug 23, 2024
## Summary

This PR parallelizes multi-platform builds using multiple workers (hence
the new docker-build / docker-publish jobs), this seems to save about ~8
minutes.

This is partial work extracted from
#6053 than is standalone
@jvacek
Copy link

jvacek commented Aug 28, 2024

@samypr100 do you have these pushed anywhere by any chance? Would love to test them out, I could write some docs on how to implem these for GitLab

@samypr100
Copy link
Contributor Author

samypr100 commented Aug 28, 2024

@samypr100 do you have these pushed anywhere by any chance? Would love to test them out, I could write some docs on how to implem these for GitLab

I've published the ones I've built in https://github.com/samypr100/uv/pkgs/container/uv, but do not use these for anything official as I might delete them eventually.

That being said, it would very helpful if you can confirm they resolve your problem in Gitlab.

@zanieb
Copy link
Member

zanieb commented Aug 28, 2024

@samypr100 does this need a rebase after the other changes?

I might merge this for the release today. Nobody else on the team is expressing opinions...

@samypr100
Copy link
Contributor Author

@samypr100 does this need a rebase after the other changes?

I might merge this for the release today. Nobody else on the team is expressing opinions...

@zanieb It does not need a rebase, unless we also want to support tags without patch version (e.g. 0.3) on the additional extra images, That can be done as a follow up though.

@samypr100
Copy link
Contributor Author

samypr100 commented Aug 29, 2024

@samypr100 does this need a rebase after the other changes?
I might merge this for the release today. Nobody else on the team is expressing opinions...

@zanieb It does not need a rebase, unless we also want to support tags without patch version (e.g. 0.3) on the additional extra images, That can be done as a follow up though.

@zanieb The latest version now also handles major.minor tags as well. I've also added the details of the new tags to the new docs section introduced in #6768.

@jvacek
Copy link

jvacek commented Aug 29, 2024

@samypr100 Will test, thanks!

Also, any chance you could introduce the same changes to ruff?

@samypr100
Copy link
Contributor Author

@samypr100 Will test, thanks!

Also, any chance you could introduce the same changes to ruff?

I can take a look at supporting after this is merged, although I don't think the python-based images make sense over there so it will likely be just distroless, alpine, and debian.

@jvacek
Copy link

jvacek commented Aug 30, 2024

I've got success on GitLab, amazing! Thanks @samypr100

Here's my config:

.uv_compile:
  stage: analysis
  image:
    name: ghcr.io/samypr100/uv:0.4-python3.12-alpine
  script: >
    cd $CODE_LOCATION

    if [ -z "$FILES" ]; then
      FILES=$(find . -iname requirements*.in -maxdepth 1)
    fi

    for file in $FILES; do
      output_file=${file%.in}.txt
      uv pip compile $file --extra-index-url $EXTRA_INDEX_URL --no-strip-extras --output-file $output_file
    done

    git diff HEAD --exit-code

Side note: I have to call to git diff to complete my use-case for checking if the requirements are compiled, #3021 is relevant here in my opinion.

@zanieb zanieb merged commit 37e25e2 into astral-sh:main Sep 3, 2024
61 checks passed
@samypr100 samypr100 deleted the more-docker-images branch September 3, 2024 14:23
@samypr100
Copy link
Contributor Author

It worked 🥳, starting with 0.4.4 new extra docker tags have been published.

@zanieb
Copy link
Member

zanieb commented Sep 4, 2024

Haha oops I meant to do that release so I could watch it.. glad it worked! Thank you!

tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Sep 4, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.4.0` -> `0.4.4` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.4.4`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#044)

[Compare Source](astral-sh/uv@0.4.3...0.4.4)

##### Enhancements

-   Allow customizing the project environment path with `UV_PROJECT_ENVIRONMENT` ([#&#8203;6834](astral-sh/uv#6834))
-   Warn when `VIRTUAL_ENV` is set but will not be respected in project commands ([#&#8203;6864](astral-sh/uv#6864))
-   Add `--no-hashes` to `uv export` ([#&#8203;6954](astral-sh/uv#6954))
-   Make HTTP headers title case for backward compatibility ([#&#8203;6887](astral-sh/uv#6887))
-   Pin `.python-version` in `uv init` ([#&#8203;6869](astral-sh/uv#6869))
-   Support `file://` URLs for `UV_PYTHON_INSTALL_MIRROR` ([#&#8203;6950](astral-sh/uv#6950))
-   Introduce more docker tags for uv ([#&#8203;6053](astral-sh/uv#6053))

##### Bug fixes

-   Avoid canonicalizing the cache directory ([#&#8203;6949](astral-sh/uv#6949))
-   Show all PyPy versions in `uv python list --all-versions` ([#&#8203;6917](astral-sh/uv#6917))
-   Avoid incorrect `requires-python` marker simplifications ([#&#8203;6268](astral-sh/uv#6268))

##### Documentation

-   Add documentation for `UV_PROJECT_ENVIRONMENT` ([#&#8203;6987](astral-sh/uv#6987))
-   Add optional dependencies section to the lockfile document ([#&#8203;6982](astral-sh/uv#6982))
-   Document use of the `file://` scheme in Python installation mirrors ([#&#8203;6984](astral-sh/uv#6984))
-   Fix outdated references to the help menu documentation in the first steps page ([#&#8203;6980](astral-sh/uv#6980))
-   Show env option in CLI reference documentation ([#&#8203;6863](astral-sh/uv#6863))
-   Add bind mount example to `docker.md` ([#&#8203;6921](astral-sh/uv#6921))

### [`v0.4.3`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#043)

[Compare Source](astral-sh/uv@0.4.2...0.4.3)

##### Enhancements

-   Show build backend output when `--verbose` is provided ([#&#8203;6903](astral-sh/uv#6903))
-   Allow `uv sync --frozen --package` without copying member `pyproject.toml` ([#&#8203;6943](astral-sh/uv#6943))

##### Bug fixes

-   Avoid panic with missing temporary directory ([#&#8203;6929](astral-sh/uv#6929))
-   Avoid updating incorrect dependencies for sorted `uv add` ([#&#8203;6939](astral-sh/uv#6939))
-   Use lower-bound semantics for all Python compatibility comparisons ([#&#8203;6882](astral-sh/uv#6882))

### [`v0.4.2`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#042)

[Compare Source](astral-sh/uv@0.4.1...0.4.2)

##### Enhancements

-   Adding support for `.pyc`  files in `uv run` ([#&#8203;6886](astral-sh/uv#6886))
-   Treat missing `top_level.txt` as non-fatal ([#&#8203;6881](astral-sh/uv#6881))

##### Bug fixes

-   Fix `is_disjoint` check for supported environments ([#&#8203;6902](astral-sh/uv#6902))
-   Remove dangling archives in `uv cache clean ${package}` ([#&#8203;6915](astral-sh/uv#6915))
-   Error when discovered Python is incompatible with `--isolated` workspace ([#&#8203;6885](astral-sh/uv#6885))
-   Warn when discovered Python is incompatible with PEP 723 script ([#&#8203;6884](astral-sh/uv#6884))

### [`v0.4.1`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#041)

[Compare Source](astral-sh/uv@0.4.0...0.4.1)

##### Enhancements

-   Add `uv export --format requirements-txt` ([#&#8203;6778](astral-sh/uv#6778))
-   Allow `@` references in `uv tool install --from` ([#&#8203;6842](astral-sh/uv#6842))
-   Normalize version specifiers by sorting ([#&#8203;6333](astral-sh/uv#6333))
-   Respect the user's upper-bound in `requires-python` ([#&#8203;6824](astral-sh/uv#6824))
-   Use Windows registry to discover Python on Windows directly ([#&#8203;6761](astral-sh/uv#6761))
-   Hint at `--no-workspace` in `uv init` failures ([#&#8203;6815](astral-sh/uv#6815))
-   Update to last PyPy releases ([#&#8203;6784](astral-sh/uv#6784))

##### Bug fixes

-   Avoid deadlocks when multiple uv processes lock resources ([#&#8203;6790](astral-sh/uv#6790))
-   Expand tildes when matching against `PATH` ([#&#8203;6829](astral-sh/uv#6829))
-   Fix `uv init --no-project` alias ([#&#8203;6837](astral-sh/uv#6837))
-   Ignore pre-release segments when discovering via `requires-python` ([#&#8203;6813](astral-sh/uv#6813))
-   Support inline optional tables in `uv add` and `uv remove` ([#&#8203;6787](astral-sh/uv#6787))
-   Update default `hello.py` to pass `ruff format` ([#&#8203;6811](astral-sh/uv#6811))
-   Avoid stripping root for user path display ([#&#8203;6865](astral-sh/uv#6865))
-   Error when user-provided environments are disjoint with Python ([#&#8203;6841](astral-sh/uv#6841))
-   Retain alphabetical sorting for `pyproject.toml` in `uv add` operations ([#&#8203;6388](astral-sh/uv#6388))))

##### Documentation

-   Add a link to the multiple index docs in the alternative index guide ([#&#8203;6826](astral-sh/uv#6826))
-   Add docs for inline exclude newer in PEP 723 scripts ([#&#8203;6831](astral-sh/uv#6831))
-   Enumerate available Docker tags ([#&#8203;6768](astral-sh/uv#6768))
-   Omit `[pip]` section from configuration file docs ([#&#8203;6814](astral-sh/uv#6814))
-   Update `project.urls` in `pyproject.toml`  ([#&#8203;6844](astral-sh/uv#6844))
-   Add docs for AWS CodeArtifact usage ([#&#8203;6816](astral-sh/uv#6816))

##### Other changes

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
releases Related to building and distributing release artifacts of uv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Publish variants of Docker image
3 participants