Open internal docker registry for customers

[pbochynski]

Description
Provide a way to use in-cluster docker registry for hosting customer's images.
As the registry is accessible with the node-port it should be possible to push the images from the developer's machine or from CI/CD agent using port forwarding. Provide a command in the kyma CLI to push images.

AC

• The registry backend could be also replaced with durable highly available storage
  • Use NFS Storage as a backend storage for docker registry docker-registry#51
  • Configure s3-compliant object store as a backend via DockerRegistry CR  docker-registry#25
• Allow access to internal docker registry via docker CLI docker-registry#38
• Document "first steps" guide docker-registry#82

Reasons
Pushing images directly to the in-cluster registry simplifies the development, testing and even production releases.
Users should be able to easily deploy their code and run their tests without the need for additional BTP or hyperscaler services.
Currently a problem arises when users want to deploy their code to the cluster, due to the fact that there is no SAP registry available. Unlike with CF where a cf push does all the work necessary to deploy the code.

Related Issues
Enable Registry Cache Extension

[kwiatekus]

Allow serverless module users to configure custom HA store as serverless docker registry backend.

• Configure s3-compliant object store as a backend via DockerRegistry CR  docker-registry#25

[kwiatekus]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?
We would like to try it out and do it in a way that serverless (untill its still dependant on internal docker registry) ux doesnt suffer
kyma-project/docker-registry#19

[pbochynski]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume.
What do you think?

[kwiatekus]

Thanks @pbochynski . That would mean that serverless runs pre-built containers as function runtimes with code dependencies mounted. dependencies resolving would need to be moved from built time into runtime ( start time ). There is a consequence in function scaling scenario: i.e when a new replica starts it would need to resolve dependencies ( i.e npm install ) into a mounted volume. In theory, replicas could have different versions of dependencies. But if we consider serverless being dev module only that should be ok.

[kwiatekus]

Opening internal docker registry should be considered as extending toolset for developers to run/test their applications on kyma only in develoment mode. We should keep recommending full-fledged HA external docker registry for production grade installations of customer applications.

[kwiatekus]

We aim to separate docker registry reconciliation to a dedicated operator, which could be used both in and out of sereverles context.

• Reconcile internal docker registry via dedicated operator docker-registry#19
• POC: Serverless uses separated internal docker registry module  serverless#849

[kwiatekus]

• Separate reconciler for docker registry has already a first working version developed in the dedicated repository https://github.com/kyma-project/docker-registry

The team is also looking into using the latest version of distribution (a.k.a registry) from cncf ->

• Use latest distribution image as internal docker registry docker-registry#18 [

[pPrecel]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume. What do you think?

I think, in this case, it would be nice to have one PVC for all functions, not to reserve space for every function. The problem is that almost all types of PVC are namespace-scoped so it would be impossible to use simple PVC. The only solution I see right now is to use nfs (network-file-system) that allows to share volume between namespaces. It's something to investigate

[kwiatekus]

Following recent feedback we will look into allowing using the module via docker CLI
kyma-project/docker-registry#38

[kwiatekus]

Currently working on exposing docker regisrty via https ( kyma-project/docker-registry#37) and configuring s3-compliant backend for storage (kyma-project/docker-registry#25)

ETA On track

[kwiatekus]

We work on allowing setting up AWS or Azure storage for docker registry.
Next will be allowing GCS buckets & allowing configuring BTP secret as a Object Storage.
We are finalising exposing docker registry via https.

ETA on track

[kwiatekus]

Status update 15.07.2024

• we are close to finish exposing docker registry via https (and thus enabling it to be used by docker CLI)
• we enriched the documentation with use case description and hints about what target environments is docker-registry module suitable for
• docker-registry (main) supports GCS as a storage backend

Next plans:

• support BTP Object store as a backend
• new 0.1.0 release and promotion to experimental channel

[kwiatekus]

Status update 29.07.2024

• @kyma-project/otters completed the task to enable bto object store as a backend and allowing to use docker registry via docker CLI
• we are now focusing on:
  • enabling NFS storage as a storage backend for docker-registry
  • hardening our CI/CD workflows on GH actions
  • publishing docker-registry to DEV and EXPERIMENTAL channels

[kwiatekus]

Status update 26.08.2024

@kyma-project/otters are working on feature completeness:

• UI configuration for kyma dashboard
• NFS storage
• (currently in progress) enabling custom gateway for external access (Allow configuring custom gateway when exposing internal kyma docker registry  docker-registry#106)

Additionally, from the DoD we are working on

• docker registry availability in DEV channel
• E2E integration test of docker registry on SAP BTP (Introduce basic integration test of docker-registry on btp platform docker-registry#127)
• docker registry availability in EXPERIMENTAL channel

We estimate we need extra week to finish
New forecast: 13.09.2024

[kwiatekus]

Status update 09.09.2024

• automated e2e test for docker registry (using extracted tf module) in review

[kwiatekus]

The docker registry is available for internal customers on canary landscape (via experimental channel ).

What's new announcement
Internal docs