So I have finally gave it a go at setting up a Raspberry Pi cluster. It was a rather simple process, but maybe because of my IT nerd background. There were a few pit-stops to do in a few places to have the rights environments and configurations.
I used a lot of content from NetworkChuck video and adjusted to my needs.
We are going to use Rancher k3s version as it is lightweight and perfect for Raspberry Pi. We will also setup their Rancher dashboard to monitor easily the cluster.
These is the list of material and environments needed to complete the creation of the cluster
You need at least a Raspberry Pi, obviously 😀
But here is my full list:
- 1 x Raspberry Pi 4 Model B – 8GB RAM
- 2 x Raspberry Pi 4 Model B – 4GB RAM
- 1 x small LAN hub 5 ports
- 3 x 50cm LAN cables
- 1 x 2m LAN cable (use the length you need to get to your router)
- 1 x 60W USB power supply. My model is the “White Xiaomi USB 60W Fast Charger 6 Ports”
- 3 x USB to Type-C cables
- 1 x Raspberry Pi 4 4-Layers Acrylic Cluster Case (you can use the type and model you want, I will probably print a nicer looking one in the future)
- SD-Card reader/writer
You will also need some software:
- Raspberry Pi Imager or similar tool to write the SD-Card
- An Ubuntu version 18.04 or 20.04 with docker installed (see Rancher support matrix). I am using on a VirtualBox, the Ubuntu Server version (not desktop) and selected the
Brideged Adapterfor the Network.
- A text editor as a pad place to copy/paste
Setting up the RPis
Let’s get done. First you’ll need to assemble the main components. Here is a work-in-progress that connects the RPIs to the LAN hub.
Baking the base stack
Next, use the Raspberry Pi imager to install the OS version. Select
Then the Raspberry Pi Lite version.
Then Press CTRL+SHIFT+X (your OS may vary) to bring the customization popup. Enable SSH. You can also provide the hostname if you want.
Once you got your SD-Card prepared, put it in the RPi and let it boot. Don’t connect any HDMI or keyboard to it, just let it boot and wait about 1 minute.
After 1 minute, check the green LED doesn’t blink (or very slowly) then disconnect the power. YES! You heard me 😀 Don’t worry, there is nothing running that requires a clean shutdown. It’s a Raspberry Pi!
Take the SD-Card and put it back on your computer. Then go to the partition marked as boot. You will need to modify 2 files on his partition:
Add the following at the end of the line:
cgroup_memory=1 cgroup_enable=memory hdmi_blanking=2 ip=<the static IP address>::<your network gateway>:<your network mask>:<the RPi hostname>:eth0:off
For example that could be
cgroup_memory=1 cgroup_enable=memory hdmi_blanking=2 ip=192.168.1.10::192.168.1.1:255.255.255.0:k3s-main:eth0:off
What does all of this do? Let’s have a look:
- cgroup_enable=memory: enable in the kernel to use the Control Groups to limit memory per process. It is disabled by default.
- cgroup_memory=1: I couldn’t find more information about it, but it seems this is no longer a kernel flag and could be ignored. It is supposedly replaced by the
- hdmi_blanking=2: this is to disable fully the HDMI ports. A value of 1 would allow to re-enable by software/command line and 0 (default) enables the HDMI ports.
- ip=[…]: sets the IP Address, the dns serveer (I skipped so there are double
:), the netmask, the hostname in the /etc/hosts file, the name of the NIC and whether we want to turn on or off the auto configuration.
- Add after the line
# Additional overlays and parameters are documented /boot/overlays/README
This will disable WiFi and Bluetooth
- Modify on the line
This will disable the audio.
- Add on the last line after the
This will enable
- Add after the line
Now disconnect the SD-Card then place it back in the RPi. Connect the power.
After a while you should be able to ping the configured IP address. When you can, you’re ready for the next step.
Setting up the OS
It is time to configure the RPi base stack before we put k3s on it. Let’s SSH to the RPi. Adjust the username and IP address to your setup:
Then let’s update and configure the base. Run a command at a time if you want to follow the progress or troubleshoot an issue. You should be able to run these commands multiple times in case something doesn’t work as expected
sudo apt update sudo apt upgrade -y sudo apt install iptables -y sudo iptables -F sudo update-alternatives --set iptables /usr/sbin/iptables-legacy sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
The first 2 commands simply update the OS to latest packages.
iptables to configure the network and allow connectivity in/out of nodes. But it doesn’t support he newer version so we need to use the
iptables-legacy instead. The
update-alternatives commands help to do that elegantly by mapping the command
iptables to the
Then reboot (
sudo reboot) your RPi. That’s it, repeat these steps for all your RPis, or clone the SD-Card (as long as they are all same RPi models)
Installing Kubernetes (k3s version)
Once all your RPis are up and running you are ready to create your kubernetes cluster.
Setting up the main node
SSH to the RPi you want to be the main node. Then execute the following commands:
sudo su - curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s - cat /var/lib/rancher/k3s/server/node-token
The last command will display a token that looks like this:
This will be used to configure the nodes to attach to the main. That’s it, it’s ready. Give it a try by typing
kubectl get nodes
It should display something like this
Setting up the worker nodes
Now let’s SSH to the worker nodes and run these commands to attach them to the main, after replacing the relevant parts as:
- use your server K3S-TOKEN
- use your server node IP address
- use a different name per each node (in this example k3s-node1)
sudo su - curl -sfL https://get.k3s.io | K3S_TOKEN="K104123450ec5bbe9d3b412a6b3de2d241b2a64c0d273685446e35171e79f94a38f::server:7512345c1234abbb0f65ea32d9874edc" K3S_URL="https://192.168.1.10:6443" K3S_NODE_NAME="k3s-node1" sh -
And voilà! Your cluster is ready. From the main node, you can run the
kubectl get nodes command again and you will get:
The next steps are to install my default applications there: