Monitoring course

This contains the NestJS application and Kubernetes resource files for the Monitoring course at Polytech Montpellier. This is the repository of Alexandre Sollier.

The application is running at a2.serpentard.dopolytech.fr.

Usage

Backend

You need Node.js 20 to run this application. It is available in the api/ directory.

This application is dependent on a Postgres database running. You can run the Docker Compose stack for this:

docker compose up -d

First, you'll need to install the dependencies:

npm i

Then, you can simply start the application in development mode:

npm run start:dev

The server will be running on port 3000.

Deployment

The Kubernetes deployment files are available in the kube/ directory.

First, deploy the Postgres database to your cluster using the Bitnami postgresql chart:

helm repo add bitnami https://charts.bitnami.com/bitnami --force-update
helm install postgres -f db/values.yml -n api-monitoring --create-namespace --version 15.5.7 bitnami/postgresql

You will also need to deploy the CRDs for the Gateway API:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml

Finally, you can deploy the application to the cluster:

kubectl apply -n api-monitoring -f api/

Services

The Kubernetes distribution used is RKE2 running on two Rocky Linux 9 nodes with SELinux enabled.

Cilium

The CNI installed in this cluster is Cilium. The configuration files are available in the kube/cilium/ directory.

The following configuration has been applied:

• The Envoy DaemonSet is enabled
• The Hubble Relay and UI are enabled
• Support for the Gateway API is enabled
• Replacement for kube-proxy is enabled
• Prometheus metrics are enabled

Because of a conflict with the internal network at Polytech, the pool of IPv4 addresses allocated to the pods are set to 10.42.0.0/16.

An IPv4 address pool has also been deployed to the cluster for the Cilium Load Balancer to attribute public IPv4 addressed to LoadBalancer services (notably, for gateways and ingress controllers).

Local Path Provisioner

To provide a storage class for PVs, Rancher's Local Path Provisioner was installed in the cluster. There was no specific configuration applied to this install.

I encountered a problem with this provisioner and SELinux, where the provisioner couldn't create directories in the provisioning path (/opt/local-path-provisioner) because of a "Permission Denied" error.

To fix this, I needed to run the following commands on each node:

mkdir /opt/local-path-provisioner
chcon -Rt container_file_t /opt/local-path-provisioner

This creates the provisioning path, and set the correct type of security context (container_file_t) to it so it can be used by the provisioner.

cert-manager

To generate and manage the TLS certificates, cert-manager was installed in the cluster. Its configuration files are available in kube/cert-manager/.

First, I applied a ClusterIssuer custom resource as usual, that is used to issue certificates using Let's Encrypt and HTTP01 challenges. For the HTTP01 challenge, instead of the usual Ingress solver, the Gateway API solver was used with the Gateway deployed for this application, as described here.

Finally, I modified the Gateway resource as described here to add the cert-manager.io/cluster-issuer annotation, the HTTP listener for the HTTP01 challenges, and the HTTPS listener that will use the TLS certificate generated by cert-manager.