This repository contains all the relevant configuration files and scripts for my remote homelab in a declarative state. The homelab is hosted on a single bare metal server hosted on Hetzner Cloud running Proxmox VE 7.4-17 as the hypervisor. The server is provisioned through hetzner and configured using Ansible. Currently, the homelab is used for the following purposes:
- Running a OPNsense Virtual Firewall
- Running a Kubernetes Cluster
- Running a Docker Server
The repository is organized as follows:
ansible/: Contains the Ansible playbooks used to install proxmox and configure the server and its VMs.kubernetes/: Contains the Kubernetes configuration files for the cluster.
The ansible/ directory contains one playbook for each of the wrapper roles below, each wrapper role is a collection of roles with some additional tasks that are used to configure the host server/VMs with a specific purpose. The site.yml file is the main playbook that triggers the installation and configuration of the server and VMs.
common: Installs common packages and applies common configurations. Wraps the following roles:singleplatform-eng.users: Creates users and groups.dev-sec.ssh-hardening: Hardens the SSH configuration.Oefenweb.apt: Updates the apt cache and upgrades the system.geerlingguy.ntp: Installs and configures NTP.Oefenweb.fail2ban: Installs and configures fail2ban.
proxmox: Installs Proxmox VE 7.4-17 on the host server, configures networking, storage and provisions the VMs. Wraps the following roles:lae.proxmox: Installs Proxmox VE and configures proxmox.
docker: Installs Docker on the docker and configures it with the necessary settings, also manages deployment of applications. Wraps the following roles:geerlingguy.docker: Installs Docker.
For more information on how to use the playbooks/roles, please refer to the README file in the ansible directory.
The kubernetes/ directory contains the configuration files for the Kubernetes cluster in a declarative way. The cluster nodes are provisioned automatically on proxmox by the proxmox role in the ansible directory and talos is used as the operating system for the nodes. The cluster is managed by flux and the configuration files are organized as follows:
kubernetes/
├── cluster # Contains the kustomization files for the cluster
│ ├── apps # Contains the kustomization files for the applications running on the cluster
│ ├── base # Contains the base kustomization files for the cluster
│ ├── boootstrap # Contains the kustomization file to bootstrap the cluster with Flux along with the necessary secrets encrypted with sops.
├── talos # Contains the talhelper configuration files for the cluster along with the cluster secrets encrypted with sops.
│ ├── clusterconfig # empty directory to store the cluster configuration files generated by talhelper
│ ├── cni # Contains the cni configuration files for the cluster
└── README.md # Kubernetes READMEThe homelab is designed to be a flexible and scalable environment that can be used for various purposes. The following features are currently supported:
| Feature | Description |
|---|---|
| OPNSense | OPNSense is used as virtual firewall for the homelab. |
| Kubernetes | Kubernetes is used to manage the applications running on the cluster. |
| Docker | Docker is used to manage additional applications running on the host server. |
The OPNSense VM is provisioned automatically by the proxmox role in the ansible directory and it is currently manually configured to act as a virtual firewall for the homelab. OPNSense allows us to make use of the single IPv4 public IP address provided by Hetzner and route traffic to the different services and VMs running on the homelab. It also enables us to create a VPN and provide DHCP and DNS services for the homelab. I do have plans to automate the configuration of OPNSense using Ansible in the future or explore other alternatives like VyOS.
The docker VM is provisioned automatically by the proxmox role and later configured by the docker role in the ansible directory. It is currently used to run the following applications:
- Portainer: A lightweight management UI which allows you to easily manage your different Docker environments.
- Traefik: A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.
- Nexus Repository: A repository manager to store and retrieve artifacts.
The Kubernetes cluster nodes are provisioned automatically by the proxmox role present in the ansible directory with the Talos OS as the base operating system. The cluster is managed by Flux and it is currently used to provision the following applications/components:
- Talos OS: A modern OS for Kubernetes, designed to be secure, immutable, and minimal.
- Talhelper: A tool to help manage and provision Talos clusters.
- Flux: A tool that automatically ensures that the state of a Kubernetes cluster matches the configuration defined in Git.
- SOPS (Mozilla SOPS): A tool to manage secrets in Git repositories.
- Cilium: A modern networking and security layer for Kubernetes based on BPF.
- Mayastor: A cloud-native storage system for Kubernetes.
- CSI-SMB-Driver A Kubernetes CSI driver for SMB (Used to provision Kubernetes PVCs using Hetzner Storage Box).
Secrets are encrypted using SOPS and commited to the repository. The sops tool is used to encrypt/decrypt the secrets and it is integrated with flux to automatically decrypt the secrets when they are applied to the cluster. The sops tool is also integrated with talhelper to automatically decrypt the secrets when they are applied to the cluster.
Tests use Pester, yq and kubeconform to validate yaml files and kubernetes resources. The tests are triggered by github actions during CI.
Thanks to the following projects for providing the tools and resources that make this homelab possible:
- Budimanjojo for the Talhelper tool and inspiration from his home-cluster
- Flux for the GitOps toolkit that makes it easy to automate the deployment and lifecycle of applications in Kubernetes.
- Ansilbe for the automation tool that makes it easy to automate the provisioning of proxmox and the VMs and VM configuration.
- Talos OS for the modern OS for Kubernetes, designed to be secure, immutable, and minimal.