Blog URL: https://kubernetes.dennyzhang.com/challenges-k8s-storage, Category: concept
File me Issues or star this repo.
See more Kubernetes sharing from Denny: denny-kubernetes
1.1.1 Update configmap, after it has already been mounted to pod. Will the pod get the new version or the old version?
https://github.com/kubernetes/kubernetes/tree/v1.11.3/pkg/volume
FlexVolume Allows Creating Volume Plugins FlexVolume enables users to develop Kubernetes volume plugins for vendor-provided storage.
vsphereVolume VMDK Stands for a virtual machine disk (VMDK) provided by the vSphere (VMware).
kubernetes/pkg/cloudprovider/providers/gce/gce_disks.go
One big benefit of containerize is supporting fast changes. But I won’t upgrade DB version that often.
Why volume mount may result in db performance penalty?
Discussions for why it gets deprecated:
- https://kubernetes.io/docs/concepts/storage/volumes/#gitrepo
- kubernetes/kubernetes#17676 (comment)
- kubernetes/kubernetes#60999
Use initContainer instead
- https://github.com/kubernetes/git-sync
- https://gist.github.com/tallclair/849601a16cebeee581ef2be50c351841
# Example of using an InitContainer in place of a GitRepo volume.
# Unilke GitRepo volumes, this approach runs the git command in a container,
# with the associated hardening.
apiVersion: v1
kind: Pod
metadata:
name: git-repo-demo
annotations:
seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
spec:
initContainers:
# This container clones the desired git repo to the EmptyDir volume.
- name: git-clone
image: alpine/git # Any image with git will do
args:
- clone
- --single-branch
- --
- https://github.com/kubernetes/kubernetes # Your repo
- /repo # Put it in the volume
securityContext:
runAsUser: 1 # Any non-root user will do. Match to the workload.
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
volumeMounts:
- name: git-repo
mountPath: /repo
containers:
# Replace with your actual workload.
- name: busybox
image: busybox
args: ['sleep', '100000'] # Do nothing
volumeMounts:
- name: git-repo
mountPath: /repo
volumes:
- name: git-repo
emptyDir: {}
https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes
https://supergiant.io/blog/persistent-storage-with-persistent-volumes-in-kubernetes
| Volume Name | Storage Type | Description |
|---|---|---|
| gcePersistentDisk | Block Storage | A Google Compute Engine (GCE) Persistent Disk that provides SSD and HDD storage attached to nodes and pods in a K8s cluster. |
| awsElasticBlockStore | Block Storage | Amazon EBS volume is a persistent block storage volume offering consistent and low-latency performance. |
| azureFile | Network File Shares | Microsoft Azure file volumes are fully managed file shares in Microsoft Azure accessible via the industry standard Server Message Block (SMB) protocol. |
| azureDisk | Block Storage | A Microsoft Azure data disk provides block storage with SSD and HDD options. |
| fc | Data Center Storage and Storage Area Networks (SAN) | Fibre channel is a high-speed networking technology for the lossless delivery of raw block data. FC is primarily used in Storage Area Networks (SAN) and commercial data centers. |
| FlexVolume | Allows Creating Volume Plugins | FlexVolume enables users to develop Kubernetes volume plugins for vendor-provided storage. |
| flocker | Container Data Storage and Management | Flocker is an open-source container data volume manager for Dockerized applications. The platform supports container portability across diverse storage types and cloud environments. |
| nfs | Network File System | NFS refers to a distributed file system protocol that allows users to access files over a computer network. |
| iscsi | Networked Block Storage | iSCSI (Internet Small Computer Systems Interface) is an IP-based storage networking protocol for connecting data storage facilities). It is used to facilitate data transfer over intranets and to manage storage over long distances by enabling location-independent data storage. |
| rbd | Ceph Block Storage | Ceph RADOS Block Device (RBD) is a building block of Ceph Block Storage that leverages RADOS capabilities such as snapshotting, consistency, and replication. |
| cephfs | Object Storage and Interfaces for Block and File Storage | Ceph is a storage platform that implements object storage on a distributed computer cluster. |
| cinder | Block Storage | Cinder is a block storage service for openstack designed to provide storage resources to end users that can be used by the OpenStack Compute Project (Nova). |
| glusterfs | Networked File System | Gluster is a distributed networked file system that aggregates storage from multiple servers into a single storage namespace. |
| vsphereVolume | VMDK | Stands for a virtual machine disk (VMDK) provided by the vSphere (VMware). |
| quobyte | Data Center File System | Quobyte volume plugin mounts Quobyte data center file system. |
| hostPath | Local Cluster File System | hostPath volumes mounts directories from the host node’s filesystem into a pod. |
| portworxVolume | Block Storage | A portworxVolume is a Portworx’s elastic block storage layer that runs hyperconverged with Kubernetes. Portworx’s storage system is designed to aggregate capacity across multiple servers similarly to Gluster. |
| scaleIO | Shared Block Networked Storage | ScaleIO is a software-defined storage product from Dell EMC that creates a server-based Storage Area Network (SAN) from local server storage. It is designed to convert direct-attached storage into shared block storage. |
| storageos | Block Storage | StorageOS aggregates storage across a cluster of servers and exposes it as high-throughput and low-latency block storage. |
Minikube supports PersistentVolumes of type hostPath. These PersistentVolumes are mapped to a directory inside the Minikube VM.
https://github.com/kubernetes/minikube/blob/master/docs/persistent_volumes.md
https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume
https://scalablesystem.design/ds101/local-volume/
There are many problems with hostPath, just to name a few:
- Unmanaged volume lifecycle
- Possible path collisions
- Too many privileges
- Not portable
