Build a high availability kubernetes cluster on your Raspberry Pi computers with K3s, as well as managing volume mounts and exports.

• Requirements
• Usage
  • Getting Started
  • Deploying
  • Post Deployment
• Next Steps

It is recommended that you have at least 1x Pi 4, although it is recommended to use a Pi 5 with an SSD or NVMe for boot.

Before running this playbook it is also assumed that:

• You have ansible installed on your provisioning machine
• Has your SSH Public key for passwordless SSH
• Has the latest version of Raspberry Pi OS (recommended Lite 64-bit version)
• (optional) Have at least one harddisk connected to your Pi

First, we need to create inventory for the playbook.

cp -Rv ./inventory/example ./inventory/your-cluster

In this newly created directory we need to edit the ./inventory/your-cluster/hosts.ini file to assign roles to devices. You can either use hostnames, or IP addresses. By adding multiple master hosts you can enable a high availability cluster to be provisioned.

In the ./inventory/your-cluster/group_vars/ directory exists two files containing host specific configuration.

Some useful variables to modify:

• all.yaml
  • ansible_user - pi is the default username for Raspberry Pi OS
  • k3s_install_args - anything to pass to the INSTALL_K3S_EXEC variable when installing the server, for example --disable=traefik if you wanted to manage traefik outside of k3s. Consult K3s documentation for more on this value.
• fileserver.yaml
  • mounts - these are any drives you want to mount to the Pi
  • export_dirs - directories to export with NFS
• k3s_cluster.yaml
  • labels - custom labels to add to nodes
  • taints - taints to add to nodes

It is worth looking at the variable files themselves to see all possible config and values, which includes a lot of documentation in the comments.

1. Install ansible module requirements with ansible-galaxy collection install -r requirements.yaml
2. Install python dependencies argcomplete / pyyaml

You can then deploy your cluster using the following command

ansible-playbook -i inventory/your-cluster install.yaml

Alternatively, by replacing install.yaml with reset.yaml you can reset all ansible commands

You'll probably want to grab your kubeconfig file from the server so that you can control your cluster from your machine.

If you will only manage this cluster, you can simply use scp pi@master:.kube/config ~/.kube/config. Or, to manage multiple you can:

1. Download the kubeconfig file for your cluster (where master is the master node)

   scp pi@master:.kube/config ~/.kube/pi-cluster

2. Change your KUBECONFIG environment variable to discover the new config

   export KUBECONFIG="${HOME}/.kube/config:${HOME}/.kube/pi-cluster

3. You can either place this KUBECONFIG export in your .zshrc or .bashrc file, or you could merge and flatten the files with the following:

   kubectl config view --flatten > /tmp/config
   mv /tmp/config "${HOME}/.kube/config"

From your host machine, run the following to verify the nodes are available and k3s has been set up correctly

$ kubectl get nodes
NAME        STATUS   ROLES                  AGE     VERSION
mccartney   Ready    control-plane,master   3m34s   v1.27.4+k3s1
lennon      Ready    <none>                 2m20s   v1.27.4+k3s1
harrison    Ready    <none>                 2m20s   v1.27.4+k3s1
starr       Ready    <none>                 2m36s   v1.27.4+k3s1

Check out the sister project home-server-kubernetes for home server related resources to deploy to your cluster.