Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Private Registry Mirrors #285

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jabrown85
Copy link
Contributor

@jabrown85 jabrown85 self-assigned this May 24, 2023
@buildpack-bot
Copy link
Member

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

(none)

@loewenstein
Copy link
Contributor

What images are requested inside of the lifecycle?

@hone
Copy link
Member

hone commented Jun 9, 2023

What images are requested inside of the lifecycle?

If the mirrors are available, I believe it would use the image from the mirror while the image will be written with the original source.

@jabrown85
Copy link
Contributor Author

What images are requested inside of the lifecycle?

Lifecycle talks directly to the registries about various images during the execution. One being the destined run-image for the image being built. For example, during the export phase, lifecycle creates an image manifest from the target run-image's manifest. Lifecycle has to talk to the run-image's registry to do this. If the run-image's registry is a public registry like dockerhub, it would consume request limits from that IP and be anonymous by default.

Lifecycle would also talk to the registry of the image being built in the same way as well as extension referenced images during that phase.

@loewenstein
Copy link
Contributor

Understood. Thanks. I was wondering if there is anything else but run images that are pulled by the lifecycle, because we just recently contributed buildpacks/spec#357 and buildpacks/lifecycle#1024 to improve the handling of stack provided run image mirrors.

Should we add a similar provision here and state that

  1. Multiple mirrors could be defined
  2. The first site that is both reachable and readable is taken
  3. The original site is considered last

cc @natalieparellano @pbusko @c0d1ngm0nk3y

# Unresolved Questions
[unresolved-questions]: #unresolved-questions

- How will will teach kaniko-style extensions to use the mirrors?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With luck I think the lifecycle could handle all of the mirror translations. We don't pull any images when running kaniko...

docker.io=https://docker.mirror.example.com;quay.io=https://quay.mirror.example.com
```

When `lifecycle` requests an image during any phase (`analyze`, `export`, `extend`, etc.), it will first check the `CNB_REGISTRY_MIRRORS` environment variable. If the requested image's registry is configured in the `CNB_REGISTRY_MIRRORS` environment variable, it will use the mirror URL instead of the original registry URL.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we outline how this intersects with run.toml/stack.toml and "run image mirrors"?

Co-authored-by: Natalie Arellano <[email protected]>
Signed-off-by: Jesse Brown <[email protected]>
docker.io=https://docker.mirror.example.com;quay.io=https://quay.mirror.example.com
```

When `lifecycle` attempts to resolve the `public/stack:run-image` image, `lifecycle` will attempt to fetch the image from `docker.mirror.example.com/public/stack:run-image`. If the [Run Image Resolution](https://github.com/buildpacks/spec/blob/main/platform.md#run-image-resolution) resulted in `quay.io/public/stack:run-image` being chosen, `lifecycle` will attempt to fetch the image from `quay.mirror.example.com/public/stack:run-image` instead. This new processing happens AFTER the [Run Image Resolution](https://github.com/buildpacks/spec/blob/main/platform.md#run-image-resolution) has executed. The run image selection will NOT take `CNB_REGISTRY_MIRRORS` into account, but the final image resolution will. Think of `CNB_REGISTRY_MIRRORS` as a just-in-time override of the final image resolution.
Copy link
Member

@natalieparellano natalieparellano Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just thinking out loud here, I'm not sure if this is relevant or not...

There exists today a pack config run-image-mirrors that will

There is also a pack config registry-mirrors that will

  • opaquely translate the registry before fetching ANY image for ANY operation
  • transparently translate the registry for the run image reference AFTER run-image-mirrors have been applied

It's worth noting that while these translations update the -run-image provided to the lifecycle, they do NOT update stack.toml/run.toml i.e., these mirrors are "hidden" today in the sense that they are not persisted as metadata on the app image.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I remember - it would be helpful to describe how CNB_REGISTRY_MIRRORS env var for the lifecycle would be exposed in pack given the above stuff that exists today. I would really hope that we could avoid adding yet another toggle here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@buildpacks/platform-maintainers @jjbustamante do you have any thoughts here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to search in past issues and pull request to understand the reason for pack config mirrors commands. I found the following:

The order of preference applied for for run image when pack build is executed:

  1. --run-image flag
  2. default run image mirror, as discussed in the --publish/daemon cases above
    2.1 when publish, default to using the run-image mirror in the registry you are publishing the image to
    2.2 when daemon, defaults to using the run-image mirror in the same registry (colocated) as the builder
  3. mirrors set using pack config run-image-mirrors
  4. mirrors defined on the builder

Based on this, the new CNB_REGISTRY_MIRRORS seems to be like a new option to override point 4, right? something like:

  1. expose some way to set CNB_REGISTRY_MIRRORS and pass it through the lifecycle
  2. mirrors defined on the builder

text/0000-private-mirrors.md Outdated Show resolved Hide resolved
text/0000-private-mirrors.md Outdated Show resolved Hide resolved
jabrown85 and others added 2 commits June 17, 2024 09:46
Co-authored-by: Ed Morley <[email protected]>
Signed-off-by: Jesse Brown <[email protected]>
Co-authored-by: Ed Morley <[email protected]>
Signed-off-by: Jesse Brown <[email protected]>
@natalieparellano natalieparellano self-requested a review June 25, 2024 15:04
@natalieparellano
Copy link
Member

Whoops, I mean to review this, not approve it (yet)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants