Skip to content

Commit

Permalink
Added support for non-virtual (hardware-based) CAN interfaces
Browse files Browse the repository at this point in the history
- changed the resource name to k8s.collabora.com/vcan
- expanded the Limitations section in the README
- assumed microk8s in the quick start example
  • Loading branch information
jpc committed Mar 8, 2022
1 parent d0b3d21 commit fe7a1bd
Show file tree
Hide file tree
Showing 4 changed files with 282 additions and 34 deletions.
74 changes: 59 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,62 @@
# Virtual SocketCAN Kubernetes device plugin
# SocketCAN Kubernetes device plugin

This plugins enables you to create virtual [SocketCAN](https://en.wikipedia.org/wiki/SocketCAN) interfaces inside your Kubernetes Pods.
`vcan` allows processes inside the pod to communicate with each other using the full Linux SocketCAN API.

## Usage example

Assuming you have a microk8s Kubernetes cluster with `kubectl` configured properly you can install the SocketCAN plugin:
Assuming you have a [microk8s](https://microk8s.io) Kubernetes cluster you can install the SocketCAN plugin:

kubectl apply -f https://raw.githubusercontent.com/jpc/k8s-socketcan/main/k8s-socketcan-daemonset-microk8s.yaml
microk8s kubectl apply -f https://raw.githubusercontent.com/Collabora/k8s-socketcan/main/k8s-socketcan-daemonset.yaml

There is also a YAML file for Azure AKS. Using it with other k8s providers may required an adjustment of the
`run-containerd` hostPath volume to point to the containerd control socket.
NOTE: Using it with other k8s providers should require only an adjustment to the `init` container script to add a new
search path for the `containerd.sock` control socket and/or install the `vcan` kernel module.

Next, you can create a simple Pod that has two vcan interfaces enabled:
Next, you can create a simple Pod that has two `vcan` interfaces enabled:

kubectl apply -f https://raw.githubusercontent.com/jpc/k8s-socketcan/main/k8s-socketcan-client-example.yaml
microk8s kubectl apply -f https://raw.githubusercontent.com/Collabora/k8s-socketcan/main/k8s-socketcan-client-example.yaml

Afterwards you can run these two commands in two separate terminals to verify it's working correctly:

kubectl exec -it k8s-socketcan-client-example -- candump vcan0
kubectl exec -it k8s-socketcan-client-example -- cansend vcan0 5A1#11.2233.44556677.88
microk8s kubectl exec -it k8s-socketcan-client-example -- candump vcan0
microk8s kubectl exec -it k8s-socketcan-client-example -- cansend vcan0 5A1#11.2233.44556677.88

Adding SocketCAN support to an existing Pod is as easy as adding a resource limit in the container spec:

```yaml
resources:
limits:
k8s.collabora.com/vcan: 1
```
## Hardware CAN interfaces
The SocketCAN device plugin also supports hardware CAN interfaces which is useful if you want to use (for example)
[K3s](https://k3s.io) to manage your embedded system software. It allows you to improve security by moving a SocketCAN network
interface into a single container and fully isolating it from any other applications on the system. It's a perfect
solution if you have a daemon that arbitrates all access to the CAN bus and you wish to containerize it.
To move a hardware CAN interface into a Pod you have to modify the DaemonSet to specify the names of the interfaces

This comment has been minimized.

Copy link
@zoq

zoq Mar 8, 2022

Contributor

Should we format "DaemonSet" -> DaemonSet?

This comment has been minimized.

Copy link
@jpc

jpc Mar 9, 2022

Author Contributor

But then should we do the same for Pods?

This comment has been minimized.

Copy link
@zoq

zoq Mar 9, 2022

Contributor

Yeah, to be consistent, I think either is fine.

you wish to make available. The names should be passed as a space separated list in the `SOCKETCAN_DEVICES` environment
variable:

```yaml
containers:
- name: k8s-socketcan
image: ghcr.io/collabora/k8s-socketcan:latest
env:
- name: SOCKETCAN_DEVICES
value: "can1 can2"
```

Afterwards, in the client container definition, instead of `k8s.collabora.com/vcan` you can specify the name of
the interface you wish to use (adding the `socketcan-` prefix to make sure it's unambiguous):

```yaml
resources:
limits:
k8s.collabora.com/socketcan-can1: 1
```

## Learn more

Expand All @@ -37,20 +74,27 @@ Other resources:

## Limitations

The plugin requires kernel support for SocketCAN (compiled in or as a module) on the cluster Nodes.
This is a package installation away on Ubuntu (so microk8s and Azure AKS work great) but unfortunatelly does not seem
possible at all on Alpine (so for example Rancher Desktop <= v1.1.1 does not work).

Currently each Pod get it's own isolated virtual SocketCAN network. There is no support for bridging
this to other Pods on the same node or to other nodes in the cluster.
this to other Pods on the same node or to other nodes in the cluster. [Adding local bridging would be possible with
the `vxcan` functionality in the kernel and the `cangw` tool.](https://www.lagerdata.com/articles/forwarding-can-bus-traffic-to-a-docker-container-using-vxcan-on-raspberry-pi) Transparent bridging to other cluster nodes over
the network should be possible manually with [cannelloni](https://github.com/mguentner/cannelloni). Pull requests to either of these cases automatically
are more then welcome.

[Adding local bridging would be possible with the `vxcan` functionality in the kernel and the `cangw` tool.](https://www.lagerdata.com/articles/forwarding-can-bus-traffic-to-a-docker-container-using-vxcan-on-raspberry-pi)
Transparent bridging to other cluster nodes over the network should be possible manually with [cannelloni](https://github.com/mguentner/cannelloni).
Pull requests to integrate both of these automatically are more then welcome.
Currently the plugin only work with clusters based on containerd, which includes most production clusters but

This comment has been minimized.

Copy link
@zoq

zoq Mar 8, 2022

Contributor

Currently, the plugin only works with clusters based on containerd, which includes most production clusters but

not Docker Desktop (we recommend using [microk8s](https://microk8s.io) instead). Pull requests to support dockerd are of course welcome.

This comment has been minimized.

Copy link
@zoq

zoq Mar 8, 2022

Contributor

"dockerd" -> dockerd just to be consistent with the style.


## Other solutions

This project was inspired by the [k8s-device-plugin-socketcan](https://github.com/mpreu/k8s-device-plugin-socketcan) project by [Matthias Preu](https://www.matthiaspreu.com) but it was written
from scrach and has some significant improvements:

- it has a valid Open Source license (MIT)
- it supports `containerd` (which is used by default in most k8s clusters, like AKS, these days) instead of the Docker daemon
- it supports `containerd` (which is used by default in most k8s clusters, like AKS, these days) instead of the `dockerd`
- it is capable of handling multiple Pods starting at the same time, which avoids head-of-the-line blocking issues when you have Pods that take a long time to start
- it supports exclusive use of real CAN interfaces

Both projects currently support only separate per-Pod SocketCAN networks.
Neither project currently supports sharing a single SocketCAN interface among multiple Pods.
2 changes: 1 addition & 1 deletion k8s-socketcan-client-example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ spec:
name: k8s-socketcan-client-example
resources:
limits:
k8s.collabora.com/socketcan: 2
k8s.collabora.com/vcan: 2
tolerations:
- key: "sku"
operator: "Equal"
Expand Down
3 changes: 3 additions & 0 deletions k8s-socketcan-daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ spec:
containers:
- name: k8s-socketcan
image: ghcr.io/collabora/k8s-socketcan:latest
env:
- name: SOCKETCAN_DEVICES
value: ""
securityContext:
privileged: true
capabilities:
Expand Down
Loading

0 comments on commit fe7a1bd

Please sign in to comment.