Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release/8.0.4xx] Multi-arch OCI images export as tarballs #46467

Open
wants to merge 51 commits into
base: release/8.0.4xx
Choose a base branch
from

Conversation

surayya-MS
Copy link
Member

@surayya-MS surayya-MS commented Feb 2, 2025

Context

Fixes dotnet/sdk-container-builds#609

This is a feature request from partner team - support publishing multi-arch OCI image as one combined tarball. Currently we publish separate tarballs for each architecture.

Details

Current behavior for multi-arch image publishing

  • remote registry - create single-arch image for each arch and publish them to the remote registry, then create image index based on them and load to remote registry
  • local daemon (docker/podman) - create single-arch image for each arch and publish them to the local daemon
  • tarball - create single-arch image for each arch and publish them as separate tarballs

Behaviour after this PR

  • remote registry - create single-arch image for each arch and publish them to the remote registry, then create image index based on them and load to remote registry (same as before)
  • local daemon (docker) - create one multi-arch image and publish it to the local daemon
  • local daemon (podman) - create single-arch image for each arch and publish them to the local daemon (same as before; support for podman will be added in the future)
  • tarball - create one multi-arch image and publish it as a tarball

Note: in order to load a multi-arch oci image tarball, containerd image store should be enabled for Docker.

Changes made

  1. Added support for multi-arch OCI tarballs creation in DockerCli.
  2. Refactored the code to extract publishing logic from CreateNewImage task into a separate class, to be able to reuse it in CreateImageIndex task.
  3. Added SkipPublishing parameter in CreateNewImage task, in order to create an image but skip publish it (this is needed for multi-arch tarball publishing because we need to publish one combined tarball)
  4. More refactoring

Testing

Modified end to end multi-arch tests to reflect the new behavior. Tested manually as well.

@surayya-MS surayya-MS requested a review from a team as a code owner February 2, 2025 19:30
@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Containers Related to dotnet SDK containers functionality untriaged Request triage from a team member labels Feb 2, 2025
@surayya-MS surayya-MS added DO NOT MERGE and removed untriaged Request triage from a team member labels Feb 2, 2025
Copy link
Member

@JanKrivanek JanKrivanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good.
But you might want to get as well review from someone with closer domain knowledge

@surayya-MS
Copy link
Member Author

surayya-MS commented Feb 6, 2025

The changes made from the last review:

  1. Fixed multi-arch oci tarball by prefixing name annotation by 'docker.io/library/', so that docker would show one image instead of 2 after loading the tarball. (Now containers can be run by referencing image name)
  2. Fixed serialization of image index so that symbols won't be escaped, and ignore nulls
  3. Fixed setting ContainerArchiveOutputPath to a file with .tar.gz extension.
  4. Fixed setting ContainerArchiveOutputPath to a file with no extension to be treated as a directory
  5. Added condition to custom test attributes, to skip the test if containerd image store is not enabled.
  6. Brought back GeneratedImageIndex output parameter to CreateImageIndex

A test is failing on macos leg - which is unrelated to the changes in this PR.

The PR is ready for review.

@surayya-MS surayya-MS changed the title Multi-arch OCI images export as tarballs [release/8.0.4xx] Multi-arch OCI images export as tarballs Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Containers Related to dotnet SDK containers functionality DO NOT MERGE
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants