Releases: containerd/cri
cri v1.0.0-rc.1
TARBALL: https://storage.googleapis.com/cri-containerd-release/cri-containerd-1.1.0-rc.1.linux-amd64.tar.gz
SHA256: d499826f8206da101d7be90784212bf9e6da000e2a1be2baa809eba36448881e
Welcome to the v1.0.0-rc.1 release of containerd cri plugin!
Notable Changes
- Fix a potential panic in event monitor. (#701, @Random-Liu)
- Use systemd
containerd.slicecgroup andOOMScoreAdjustfor GCE cluster. (#704, @Random-Liu) - Support Kubernetes alpha feature
RunAsGroup(#710, @Random-Liu) - Remove dependency on
libapparmororlibapparmor-dev. (#711, @tklauser) - Use self-signed CA for container streaming. (#713, @mikebrow)
- Disable TLS container streaming (exec, attach etc.) by default, and add the
enable_tls_streamingconfig option to enable it. (#714, @mikebrow)
Change List: v1.0.0-rc.0...v1.0.0-rc.1
Try It Out
- For a production quality cluster on GCE brought up with
kube-up.sh, see here. - For a multi-node cluster installer and bring up steps using ansible and kubeadm, see here.
- For creating a cluster from scratch on Google Cloud, see Kubernetes the Hard Way.
- For a custom installation from release tarball, see here.
- To install use LinuxKit on a local VM, see here.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri v1.0.0-rc.0
TARBALL: https://storage.googleapis.com/cri-containerd-release/cri-containerd-1.1.0-rc.0.linux-amd64.tar.gz
SHA256: 396189f25a37d04e84b62fe1615d5a5bdb13056a0433b4b1faaf98ff23062294
Welcome to the v1.0.0-rc.0 release of containerd cri plugin! The containerd CRI support is GA now!
In this release, the project was moved from the kubernetes-incubator organization, and renamed to cri.
Highlights
CRI Plugin
This release of cri is a native plugin of containerd.It is built into containerd v1.1 and the CRI support is enabled by default.
You can now use Kubernetes, with containerd directly, without having to use the intermediate cri-containerd daemon. The cri-containerd daemon is end-of-life.
Note: Please drain your node before upgrading from older versions of cri-containerd to containerd v1.1.
You can use a containerd config file to configure the cri plugin.
Untrusted Workload Runtime
To run an untrusted pod on a runtime for untrusted workload e.g. katacontainers and clearcontainers, you can:
- Configure a runtime for untrusted workload with the config option
plugins.cri.containerd.untrusted_workload_runtime. - Create an untrusted pod by setting the annotation
io.kubernetes.cri.untrusted-workloadto"true", for example:
apiVersion: v1
kind: Pod
metadata:
name: nginx
annotations:
io.kubernetes.cri.untrusted-workload: "true"
spec:
containers:
- name: nginx
image: nginxBy default, cri will run pods with the default runtime. However, if a pod has the io.kubernetes.cri.untrusted-workload annotation, the cri plugin will run the pod with the runtime for untrusted workloads.
Unless configured otherwise, the default runtime is set to runc.
Container Runtime Interface v1alpha2
The supported CRI (Container Runtime Interface) version for Kubernetes v1.10 is now v1alpha2. This release of cri has been updated to use CRI v1alpha2, so it only works with Kubernetes v1.10+.
New CRI features added in v1alpha2 are all supported:
- Container log rotation: Kubelet rotates container logs.
- Shared pid namespace: Support sharing pid namespace inside a pod.
Registry Mirror
You can now setup registry configurations with the config option plugins.cri.registry.
Currently only the mirrors option is supported. With it, you can specify registry mirrors and insecure registry. (doc)
Notable Changes
- The
exec,attachandportforwardconnection between Kubernetes apiserver and containerd is now encrypted. (#681) - If you are using native cni plugins, v0.7.0 is required. (#674)
Test
End-To-End Test
In terms of testing, we've passed:
- ALL CRI validation tests
- ALL node e2e tests
- ALL e2e tests
The containerd test coverage on GCE is equivalent with Docker now.
All the test results are public: https://k8s-testgrid.appspot.com/sig-node-containerd.
Performance
We significantly improved pod start latency and cpu/memory usage of cri plugin this release.
The continuous benchmark result is published on http://node-perf-dash.k8s.io/. Job ci-kubernetes-node-kubelet-benchmark is for Docker 17.03, and ci-cri-containerd-node-e2e-benchmark is for containerd with cri plugin.
All metrics of containerd are either better or comparable with Docker 17.03.
Try It Out
- For a production quality cluster on GCE brought up with
kube-up.sh, see here. - For a multi-node cluster installer and bring up steps using ansible and kubeadm, see here.
- For creating a cluster from scratch on Google Cloud, see Kubernetes the Hard Way.
- For a custom installation from release tarball, see here.
- To install use LinuxKit on a local VM, see here.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri-containerd v1.0.0-beta.1 release
SHA256: 86b2415d9fe3b55ef72e290a7dd68adb956a5a8ab7ea58b4271348f30e23324e
Welcome to the v1.0.0-beta.1 release of cri-containerd!
Note: This repository will be moved into containerd organization after this release.
Highlights
This release we mainly focused on bug fix. Notable bug fixes:
- Get rid of default
RLIMIT_NOFILEfor containers. (Issue: #515, PR: #516, @Random-Liu) - Fix an issue that privileged pod can't run on kernel with version <3.16. (Issue: #517, PR: #518, @Random-Liu)
- Cache pod IP for performance and also unblock VM-based container support. (Issue: #524, PR: #525, @abhi)
- Add a flag
skip-imagefs-uuidto skip retrieving image filesystem UUID. Users will be able to trycri-containerdon node withmdevorZFS. Note: kubelet will not be able to get imagefs capacity or perform imagefs disk eviction whenskip-imagefs-uuid=true. (Issue: #325, #399, #509, PR: #510, @Random-Liu)
Test dashboard: https://k8s-testgrid.appspot.com/sig-node-containerd.
External Dependencies
- containerd: v1.0.0
- runc: 74a17296470088de3805e138d3d87c62e613dfc4
- cni: v0.6.0
- Kubernetes: v1.9+ (cri-containerd v1.0.0-beta.1)
- crictl: v1.0.0.alpha.0
Note: Because of Kubernetes container runtime interface change, cri-containerd v1.0.0-beta.1 requires Kubernetes v1.9 or later. With older Kubernetes version, container exec has known issue #417, and container logging won't work.
Try It Out
- For a production quality cluster on GCE brought up with
kube-up.sh, see here. - For a multi-node cluster installer and bring up steps using ansible and kubeadm, see here.
- For creating a cluster from scratch on Google Cloud, see Kubernetes the Hard Way.
- For a custom installation from release tarball, see here.
- For a installation with LinuxKit on a local VM, see here.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri-containerd v1.0.0-beta.0 release
SHA256: 00a98c14081a7d090bfdd44f3c16812a046c2642925fce0ec76bd3c94273eee1
Welcome! cri-containerd has graduated to v1.0.0-beta!!!
Highlights
In this release we focused on test, bug fix, and usability.
Installation
Using kube-up.sh to bring up a production quality Kubernetes cluster on GCE was complete. This would enable users to use containerd and cri-containerd in their production Kubernetes environments as the container runtime.
Steps to use kube-up.sh can be found here.
Debuggability
We've tried to provide docker like debug experience using crictl to debug, inspect, and manage pods, containers, and images. The user guide for crictl could be found here.
Example:
$ crictl ps -a
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
cde98b50b1155 sha256:5d049a8c4eec92b21ca4be399c260166d96569a1a52d497f4a0365bb55c1a18c 2 weeks ago CONTAINER_EXITED kubedns 0
3fae43b5d5eb3 sha256:5feec37454f45d060c5f528c7d0bd4958df39e7ffd2e65ae42aae68bf78f69a5 2 weeks ago CONTAINER_EXITED dnsmasq 0
b41a6661c98a3 sha256:db76ee297b8597fc007b23a90619314b8405bb1df6dcad189df0a123a09e7ecc 2 weeks ago CONTAINER_EXITED sidecar 0
5a4a348431b69 gcr.io/kubernetes-e2e-test-images/redis-amd64@sha256:3e01bcaf67cb9b5c9fa7f57ba92539c8962d59c9647b91e9ec5047a89e2bc49a 2 weeks ago CONTAINER_EXITED master 0
0245c61fdf85d sha256:69854bafc1214f1a7f88c32f193dd0112e4d89d5bd9da9a85d95d5735acbc397 2 weeks ago CONTAINER_EXITED nginx 0Monitoring
Containerd integration with cadvisor was complete. Now Kubernetes Summary API is supported.
The only exception is container log stats, which will be supported in next release.
Test
In terms of testing we have passed:
- ALL CRI validation tests
- ALL regular node e2e tests
- ALL regular e2e tests
All the test results are public: https://k8s-testgrid.appspot.com/sig-node-containerd.
External Dependencies
- containerd: v1.0.0
- runc: 74a17296470088de3805e138d3d87c62e613dfc4
- cni: v0.6.0
- Kubernetes: v1.9+ (cri-containerd v1.0.0-beta.0)
- crictl: v1.0.0.alpha.0
Note: Because of Kubernetes container runtime interface change, cri-containerd v1.0.0-beta.0 requires Kubernetes v1.9 or later. With older Kubernetes version, container exec has known issue #417, and container logging won't work.
Try It Out
- For a production quality cluster on GCE brought up with
kube-up.sh, see here. - For a multi-node cluster installer and bring up steps using ansible and kubeadm, see here.
- For creating a cluster from scratch on Google Cloud, see Kubernetes the Hard Way.
- For a custom installation from release tarball, see here.
- For a installation with LinuxKit on a local VM, see here.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri-containerd v1.0.0-alpha.1 release
Welcome to the v1.0.0-alpha.1 release of cri-containerd!
Highlights
Test Dashboard
A new sig-node-containerd tab is added in Kubernetes test dashboard: https://k8s-testgrid.appspot.com/sig-node-containerd.
All Kubernetes containerd integration test result will be accessible there. Now it's running node e2e test, and the e2e test will be added soon.
Load Docker Image from Tarball
Now you could use cri-containerd load to load a docker image from a tarball created by docker save:
$ sudo cri-containerd load busybox.tar
Loaded image: docker.io/library/busybox:latestNew Command Line Interface
We added a more user friendly command line interface for cri-containerd:
$ cri-containerd --help
_ __ _ __
__________(_) _________ ____ / /_____ _(_)____ ___ _________/ /
/ ___/ ___/ /______/ ___/ __ \/ __ \/ __/ __ `/ // __ \/ _ \/ ___/ __ /
/ /__/ / / //_____/ /__/ /_/ / / / / /_/ /_/ / // / / / __/ / / /_/ /
\___/_/ /_/ \___/\____/_/ /_/\__/\__,_/_//_/ /_/\___/_/ \__,_/
A containerd based Kubernetes CRI implementation.
Usage:
cri-containerd [flags]
cri-containerd [command]
Available Commands:
default-config Print default toml config of cri-containerd.
help Help about any command
load Load an image from a tar archive.
version Print cri-containerd version information.
Flags:
--alsologtostderr log to standard error as well as files
--cgroup-path string The cgroup that cri-containerd is part of. Cri-containerd is not placed in a cgroup if none is specified.
--config string Path to the config file. (default "/etc/cri-containerd/config.toml")
--containerd-endpoint string Path to the containerd endpoint. (default "/run/containerd/containerd.sock")
--containerd-root-dir string Root directory path where containerd stores persistent data. (default "/var/lib/containerd")
--containerd-runtime string The runtime used by containerd. (default "io.containerd.runtime.v1.linux")
--containerd-runtime-engine string Runtime engine used by containerd. Defaults to containerd's default if not specified.
--containerd-runtime-root string The directory used by containerd for runtime state. Defaults to containerd's default if not specified.
--containerd-snapshotter string The snapshotter used by containerd. (default "overlayfs")
--enable-selinux Enable selinux support. By default not enabled.
-h, --help help for cri-containerd
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory
--logtostderr log to standard error instead of files
--network-bin-dir string The directory for putting network binaries. (default "/opt/cni/bin")
--network-conf-dir string The directory for putting network plugin configuration files. (default "/etc/cni/net.d")
--oom-score int Adjust the cri-containerd's oom score. (default -999)
--root-dir string Root directory path for cri-containerd managed files (metadata checkpoint etc). (default "/var/lib/cri-containerd")
--sandbox-image string The image used by sandbox container. (default "gcr.io/google_containers/pause:3.0")
--socket-path string Path to the socket which cri-containerd serves on. (default "/var/run/cri-containerd.sock")
--stats-collect-period int The period (in seconds) of snapshots stats collection. (default 10)
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
--stream-addr string The ip address streaming server is listening on. The default host interface is used if not specified.
--stream-port string The port streaming server is listening on. (default "10010")
--systemd-cgroup Enables systemd cgroup support. By default not enabled.
-v, --v Level log level for V logs
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
Use "cri-containerd [command] --help" for more information about a command.External Dependencies
- containerd: v1.0.0-beta.2
- runc: 74a17296470088de3805e138d3d87c62e613dfc4
- cni: v0.6.0
- Kubernetes: v1.7+
Features & Bug Fixes
- #235 Add truncated index support. Container/sandbox/image could be indexed with truncated id now. @yanxuean
- #319 Fix
UpdateContainerResourcesso that the CPU manager Kubernetes alpha feature works with cri-containerd now. @Random-Liu - #322 Support
unconfinedAppArmor profile. @miaoyq - #327 Fix a bug that cri-containerd can't start a container without directory created for image volume during build. @Random-Liu
- #328 Fix a panic in
ListContainerStats. @Random-Liu - #335 Use device number to more reliably find image filesystem partition uuid. @Random-Liu
- #345 Check whether an image is corrupted during restart recovery. @yanxuean
- #347 Add
--oom-scoreflag in cri-containerd. @yanxuean - #366 Add stack dump support. Now user could send
SIGUSR1signal tocri-containerdprocess to trigger a stack dump. @Random-Liu - #371 Fix a bug that once a container fails to be deleted, it could never be deleted again. @Random-Liu
- #372 Generate log path in
ContainerStatus, so thatcrictl logscould work. @Random-Liu - #374 Put containerd and cri-containerd into a specific cgroup
/runtimein the ansible setup, and point kubelet to monitor that cgroup. @Random-Liu
Try It Out
- For a multi-node cluster installer and bring up steps using ansible and kubeadm, see here.
- For creating a cluster from scratch on Google Cloud, see Kubernetes the Hard Way.
- For a custom installation from release tarball, see here.
- For a installation with LinuxKit on a local VM, see here.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri-containerd v1.0.0-alpha.0 release
So we have come a long way from the cri-containerd v0.1.0 release. We are excited to announce the cri-containerd v1.0.0-alpha.0 release today!
Highlights
- Feature Complete. ALL Kubernetes features are supported with CRI-Containerd.
- Cluster Installer. An installer is created to bring up multi-node kubernetes cluster with cri-containerd and containerd.
External Dependencies
- containerd: v1.0.0-beta.1
- runc: v1.0.0-rc4
- cni: v0.6.0
- Kubernetes: v1.7+
Features
In this release, lots of significant missing features have been added:
- Container Streaming: Exec(#115), Attach(#128), Portforward(#130).
- Security Context: RunAsUser(#168), Selinux(#157), Apparmor(#159), Seccomp(#219), Sysctl(#119).
- Container Metrics: CPU/Memory/Disk usage of a container(#265).
- Image Filesystem Metrics: Disk and inodes usage of image filesystem(#257).
- Live Restore: cri-containerd and containerd could be restarted, containers will not be affected(#209).
- More...
Test
In terms of testing we have passed:
- ALL CRI validation tests
- ALL regular node e2e tests
Try It Out
We have created a cluster installer to bring up multi node Kubernetes cluster with cri-containerd and containerd. The installer is based on ansible and kubeadm. We hope to receive a lot of feedback and suggestions in addition to help on validating and improving the installer on various distros.
We have compiled all the steps to bring up the cluster in here
Known Issues
- Kubelet Summary API: Because of known issue in Kubelet, it can't correctly consume container and image filesystem stats from container runtime. This is going to be fixed in Kubernetes v1.8.1.
- Image Removal: Removing an image only removes the image reference, the underlying storage is not removed. This will be fixed in next release, after containerd image garbage collection (containerd/containerd#1398) is implemented.
- Alpha CPU Manager: The alpha feature CPU manager is not supported yet because of a known issue #316. The support will be added in next patch release.
Contributors
We'd like to extend a thanks to the following people who contributed to this release:
cri-containerd v0.1.0 release
External Dependencies
Significant changes to containerd and runc are underway in the development (master) branches. If you want to try out cri-containerd please take care to checkout the specified commits for the below listed dependencies. We will be syncing up to the current master branches soon. Thank you for taking a look at our release!
- containerd: v0.2.3-1098-g8ed1e24
- runc: v1.0.0-rc3
- cni: v0.4.0
- Kubernetes: v1.7+
Features
cri-containerd v0.1.0 supports all basic functionalities including:
- Sandbox/container lifecycle management;
- Image management;
- Sandbox networking;
- Container logging;
- Run command synchronously in container etc.
Missing Features
Following features are not supported in v0.1.0. They will be added in future version.
- Container streaming: Exec, Attach, Portforward;
- Security context: RunAsUser, Selinux, Apparmor, Seccomp, Sysctl;
- Container metrics: CPU/memory/disk usage of a container.
- Image filesystem metrics: Image filesystem usage.
- Host port: Expose container port on a host port.
Known Issues
- cri-containerd doesn't checkpoint state. cri-containerd will lose all sandbox/container/image information across restart.
- Frequently containerd restart may cause state mismatch between cri-containerd and containerd.
- Pause container getting killed unexpectedly will cause network resource leakage.
Try it Out
Please follow the getting started instruction.
cri-containerd v0.1.0-alpha.1 release
Dependencies
Significant changes to containerd and runc are underway in the development (master) branches. If you want to try out cri-containerd please take care to checkout the specified commits for the below listed dependencies. We will be syncing up to the current master branches soon. Thank you for taking a look at our alpha release!
- containerd: containerd/containerd@8ed1e24
- runc: opencontainers/runc@6394544
- cni: https://github.com/containernetworking/cni/tree/v0.4.0
- Kubernetes: v1.7
New Features
- Sandbox
/etc/hosts(@Random-Liu, #60) - Sandbox
/dev/shm(@Random-Liu, #67) - Sanbox
DNSOptions(@Crazykev, #50) - Privileged container (@heartlock, #51)
- Container logging (@Random-Liu, #56)
- Container
ExecSync(@Random-Liu, #72) - Mount cgroup inside container (@Random-Liu, #70)
- Container capabilities (@Random-Liu, #71)
- Schema 1 docker image (@Random-Liu, #81)
- Pull image authentication (@Random-Liu, #88)
- Stop/remove containers when stop/remove sandbox (@Random-Liu, #77)
Missing Features
Try it out
- Build and install
containerd,runc,cni(including cni config), andcri-containerd. - Start a local cluster:
$ sudo containerd &
$ sudo cri-containerd &
$ CONTAINER_RUNTIME=remote CONTAINER_RUNTIME_ENDPOINT=/var/run/cri-containerd.sock hack/local-up-cluster.shcri-containerd 0.1.0-alpha.0 release
Dependencies
Significant changes to containerd and runc are underway in the development (master) branches. If you want to try out cri-containerd please take care to checkout the specified commits for the below listed dependencies. We will be syncing up to the current master branches soon. Thank you for taking a look at our alpha release!
- containerd: containerd/containerd@2562aca
- runc: opencontainers/runc@50401b5
- cni: https://github.com/containernetworking/cni/tree/v0.4.0
- Kubernetes: kubernetes/kubernetes@1369a26
Features Support
- Basic container lifecycle;
- Basic sandbox lifecycle;
- Basic image management.
Missing Features
Try it out
- Build and install
containerd,runc,cni(including cni config), andcri-containerd. - Start a local cluster:
$ containerd &
$ cri-containerd &
$ CONTAINER_RUNTIME=remote CONTAINER_RUNTIME_ENDPOINT=/var/run/cri-containerd.sock hack/local-up-cluster.sh