Skip to content
This repository was archived by the owner on Oct 22, 2024. It is now read-only.

Commit

Permalink
docs: storage capacity trackig in Kubernetes 1.21
Browse files Browse the repository at this point in the history
Now that we actually can use the feature, the scheduler extensions are
not needed anymore for Kubernetes >= 1.21.
pohly committed Jun 11, 2021
1 parent 0ea1f41 commit 7874146
Showing 2 changed files with 16 additions and 21 deletions.
7 changes: 4 additions & 3 deletions docs/design.md
Original file line number Diff line number Diff line change
@@ -172,10 +172,11 @@ provisioning"](https://github.com/kubernetes-csi/external-provisioner/tree/v2.1.
- The scheduler extensions help the scheduler with picking nodes where
volumes can be created. Without them, the risk of choosing nodes
without PMEM may be too high and manual pod scheduling may be needed
to avoid long delays when starting pods. In the future with
Kubernetes >= 1.21, [storage capacity
to avoid long delays when starting pods. Starting with
Kubernetes 1.21, [storage capacity
tracking](https://kubernetes.io/docs/concepts/storage/storage-capacity/)
will be another solution for that problem.
is used to solve this problem and the scheduler extensions are not
needed anymore.
- For volumes with storage classes that use immediate binding, the
different `external-provisioner` instances compete with each for
ownership of the volume by setting the "selected node"
30 changes: 12 additions & 18 deletions docs/install.md
Original file line number Diff line number Diff line change
@@ -62,11 +62,11 @@ storage class also chooses which filesystem is used (xfs or ext4) and
enables [Kata Containers support](#kata-containers-support).

Optionally, the administrator can enable [the scheduler
extensions](#enable-scheduler-extensions) (recommended) and monitoring
of resource usage via the [metrics support](#metrics-support).
extensions](#enable-scheduler-extensions) and monitoring of resource
usage via the [metrics support](#metrics-support).

It is [recommended](./design.md#dynamic-provisioning-of-local-volumes)
to enable the scheduler extensions in combination with
to enable the scheduler extensions and use
`volumeBindingMode: WaitForFirstConsumer` as in the
[`pmem-storageclass-late-binding.yaml`](/deploy/common/pmem-storageclass-late-binding.yaml)
example. This ensures that pods get scheduled onto nodes that have
@@ -76,6 +76,12 @@ immediate binding (the default volume binding mode) might work
better. However, then pods might not be able to run when the node
where volumes were created are overloaded.

Starting with Kubernetes 1.21, PMEM-CSI uses [storage capacity
tracking](https://kubernetes.io/docs/concepts/storage/storage-capacity/)
to handle Pod scheduling and the scheduler extensions are not needed
anymore. `WaitForFirstConsumer` still is the recommended volume
binding mode.

Optionally, the log output format can be changed from the default
"text" format (= the traditional glog format) to "json" (= output via
[zap](https://github.com/uber-go/zap/blob/master/README.md)) for easier
@@ -1257,7 +1263,7 @@ $ kubectl create --kustomize my-webhook
[Kubernetes
1.19](https://kubernetes.io/blog/2020/09/01/ephemeral-volumes-with-storage-capacity-tracking/)
introduces support for publishing and using storage capacity
information for pod scheduling. PMEM-CSI must be deployed
information for pod scheduling. It became beta in 1.21. PMEM-CSI must be deployed
differently to use this feature:

- `external-provisioner` must be told to publish storage capacity
@@ -1266,21 +1272,9 @@ differently to use this feature:
scheduler, otherwise it ignores that information when considering
pods with unbound volume.

Because the `CSIStorageCapacity` feature is still alpha in 1.19 and
driver deployment would fail on a cluster without support for it, none
of the normal deployment files nor the operator enable that. Instead,
special kustomize variants are provided in
`deploy/kubernetes-1.19-alpha*`.

They can be used for example in the QEMU test cluster with:
The deployments for Kubernetes >= 1.21 do this automatically. The
alpha API in 1.19 and 1.20 is no longer supported.

```console
$ TEST_KUBERNETES_VERSION=1.19 make start
...
$ TEST_KUBERNETES_FLAVOR=-alpha test/setup-deployment.sh
INFO: deploying from /nvme/gopath/src/github.com/intel/pmem-csi/deploy/kubernetes-1.19-alpha/lvm/testing
...
```

### Metrics support

0 comments on commit 7874146

Please sign in to comment.