registry.md

Configure Image Registry

This document describes the method to configure the image registry for containerd for use with the cri plugin.

Configure Registry Endpoint

With containerd, docker.io is the default image registry. You can also set up other image registries similar to docker.

To configure image registries create/modify the /etc/containerd/config.toml as follows:

# Config file is parsed as version 1 by default.
# To use the long form of plugin names set "version = 2"
[plugins.cri.registry.mirrors]
  [plugins.cri.registry.mirrors."docker.io"]
    endpoint = ["https://registry-1.docker.io"]
  [plugins.cri.registry.mirrors."test.https-registry.io"]
    endpoint = ["https://HostIP1:Port1"]
  [plugins.cri.registry.mirrors."test.http-registry.io"]
    endpoint = ["http://HostIP2:Port2"]
  # wildcard matching is supported but not required.
  [plugins.cri.registry.mirrors."*"]
    endpoint = ["https://HostIP3:Port3"]

The default configuration can be generated by containerd config default > /etc/containerd/config.toml.

The endpoint is a list that can contain multiple image registry URLs split by commas. When pulling an image from a registry, containerd will try these endpoint URLs one by one, and use the first working one. Please note that if the default registry endpoint is not already specified in the endpoint list, it will be automatically tried at the end with scheme https and path v2, e.g. https://gcr.io/v2 for gcr.io.

As an example, for the image gcr.io/library/busybox:latest, the endpoints are:

• gcr.io is configured: endpoints for gcr.io + default endpoint https://gcr.io/v2.
• * is configured, and gcr.io is not: endpoints for * + default endpoint https://gcr.io/v2.
• None of above is configured: default endpoint https://gcr.io/v2.

After modify this config, you need restart the containerd service.

Configure Registry TLS Communication

cri plugin also supports configuring TLS settings when communicating with a registry.

To configure the TLS settings for a specific registry, create/modify the /etc/containerd/config.toml as follows:

# The registry host has to be a domain name or IP. Port number is also
# needed if the default HTTPS or HTTP port is not used.
[plugins.cri.registry.configs."my.custom.registry".tls]
    ca_file   = "ca.pem"
    cert_file = "cert.pem"
    key_file  = "key.pem"

In the config example shown above, TLS mutual authentication will be used for communications with the registry endpoint located at https://my.custom.registry. ca_file is file name of the certificate authority (CA) certificate used to authenticate the x509 certificate/key pair specified by the files respectively pointed to by cert_file and key_file.

cert_file and key_file are not needed when TLS mutual authentication is unused.

[plugins.cri.registry.configs."my.custom.registry".tls]
    ca_file   = "ca.pem"

To skip the registry certificate verification:

[plugins.cri.registry.configs."my.custom.registry".tls]
  insecure_skip_verify = true

Configure Registry Credentials

cri plugin also supports docker like registry credential config.

To configure a credential for a specific registry, create/modify the /etc/containerd/config.toml as follows:

# The registry host has to be a domain name or IP. Port number is also
# needed if the default HTTPS or HTTP port is not used.
[plugins.cri.registry.configs.auths."https://gcr.io"]
  username = ""
  password = ""
  auth = ""
  identitytoken = ""

The meaning of each field is the same with the corresponding field in .docker/config.json.

Please note that auth config passed by CRI takes precedence over this config. The registry credential in this config will only be used when auth config is not specified by Kubernetes via CRI.

After modify this config, you need restart the containerd service.

Configure Registry Credentials Example - GCR with _json_key Authentication

Create a gcp account with gcr, do all the steps to enable receiving a pushed image for a gcr instance, including the generation and download of a new _json_key (for a new service account user.) To make sure your gcr registry is working with _json_key authentication let's login and push an image to your gcr instance: This step is not necessary if you have already pushed an image to your gcr instance.

$ docker login -u _json_key -p "$(cat key.json)" gcr.io
$ docker push gcr.io/your-gcr-instance-id/busybox
$ docker logout gcr.io

Generate a single line for the _json_key file that you downloaded:

jq -c . key.json

Edit the containerd config (default location is at /etc/containerd/config.toml) to add your _json_key authentication for gcr.io domain image pull requests:

version = 2

[plugins."io.containerd.grpc.v1.cri".registry]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
      endpoint = ["https://registry-1.docker.io"]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gcr.io"]
      endpoint = ["https://gcr.io"]
  [plugins."io.containerd.grpc.v1.cri".registry.configs]
    [plugins."io.containerd.grpc.v1.cri".registry.configs."gcr.io".auth]
      username = "_json_key"
      password = 'paste output from jq'

Restart containerd

$ service containerd restart

Pull an image from your gcr instance with crictl:

$ sudo crictl pull gcr.io/your-gcr-instance-id/busybox
DEBU[0000] get image connection
DEBU[0000] connect using endpoint 'unix:///run/containerd/containerd.sock' with '3s' timeout
DEBU[0000] connected successfully using endpoint: unix:///run/containerd/containerd.sock
DEBU[0000] PullImageRequest: &PullImageRequest{Image:&ImageSpec{Image:gcr.io/your-gcr-instance-id/busybox,},Auth:nil,SandboxConfig:nil,}
DEBU[0001] PullImageResponse: &PullImageResponse{ImageRef:sha256:78096d0a54788961ca68393e5f8038704b97d8af374249dc5c8faec1b8045e42,}
Image is up to date for sha256:78096d0a54788961ca68393e5f8038704b97d8af374249dc5c8faec1b8045e42