So I have finally gave it a go at setting up a Raspberry Pi cluster. It was a rather simple process, but maybe because of my IT nerd background. There were a few pit-stops to do in a few places to have the rights environments and configurations.

I used a lot of content from NetworkChuck video and adjusted to my needs.

We are going to use Rancher k3s version as it is lightweight and perfect for Raspberry Pi. We will also setup their Rancher dashboard to monitor easily the cluster.

Pre-Requisites

These is the list of material and environments needed to complete the creation of the cluster

Hardware

You need at least a Raspberry Pi, obviously 😀
But here is my full list:

  • 1 x Raspberry Pi 4 Model B – 8GB RAM
  • 2 x Raspberry Pi 4 Model B – 4GB RAM
  • 1 x small LAN hub 5 ports
  • 3 x 50cm LAN cables
  • 1 x 2m LAN cable (use the length you need to get to your router)
  • 1 x 60W USB power supply. My model is the “White Xiaomi USB 60W Fast Charger 6 Ports”
White Xiaomi USB 60W Fast Charger 6 Ports
Xiaomi USB 60W Fast Charger 6 Ports
  • 3 x USB to Type-C cables
QUANTUM 3.0 port 2 m USB Type C Cable - QUANTUM : Flipkart.com
USB to Type-C cable
  • 1 x Raspberry Pi 4 4-Layers Acrylic Cluster Case (you can use the type and model you want, I will probably print a nicer looking one in the future)
  • SD-Card reader/writer

Software

You will also need some software:

  • Raspberry Pi Imager or similar tool to write the SD-Card
  • An Ubuntu version 18.04 or 20.04 with docker installed (see Rancher support matrix). I am using on a VirtualBox, the Ubuntu Server version (not desktop) and selected the Brideged Adapter for the Network.
  • A text editor as a pad place to copy/paste

Setting up the RPis

Let’s get done. First you’ll need to assemble the main components. Here is a work-in-progress that connects the RPIs to the LAN hub.

RPi Cluster - WIP
RPi Cluster – WIP

Baking the base stack

Next, use the Raspberry Pi imager to install the OS version. Select

Raspberry Pi OS (Other)

Then the Raspberry Pi Lite version.

Then Press CTRL+SHIFT+X (your OS may vary) to bring the customization popup. Enable SSH. You can also provide the hostname if you want.

Advanced options

Once you got your SD-Card prepared, put it in the RPi and let it boot. Don’t connect any HDMI or keyboard to it, just let it boot and wait about 1 minute.

After 1 minute, check the green LED doesn’t blink (or very slowly) then disconnect the power. YES! You heard me 😀 Don’t worry, there is nothing running that requires a clean shutdown. It’s a Raspberry Pi!

Take the SD-Card and put it back on your computer. Then go to the partition marked as boot. You will need to modify 2 files on his partition:

  • cmdline.txt:
    Add the following at the end of the line:
    cgroup_memory=1 cgroup_enable=memory hdmi_blanking=2 ip=<the static IP address>::<your network gateway>:<your network mask>:<the RPi hostname>:eth0:off

For example that could be
cgroup_memory=1 cgroup_enable=memory hdmi_blanking=2 ip=192.168.1.10::192.168.1.1:255.255.255.0:k3s-main:eth0:off

What does all of this do? Let’s have a look:

  1. cgroup_enable=memory: enable in the kernel to use the Control Groups to limit memory per process. It is disabled by default.
  2. cgroup_memory=1: I couldn’t find more information about it, but it seems this is no longer a kernel flag and could be ignored. It is supposedly replaced by the cgroup_enable=memory setting.
  3. hdmi_blanking=2: this is to disable fully the HDMI ports. A value of 1 would allow to re-enable by software/command line and 0 (default) enables the HDMI ports.
  4. ip=[…]: sets the IP Address, the dns serveer (I skipped so there are double :), the netmask, the hostname in the /etc/hosts file, the name of the NIC and whether we want to turn on or off the auto configuration.
  • config.txt
    • Add after the line # Additional overlays and parameters are documented /boot/overlays/README
      dtoverlay=disable-wifi
      dtoverlay=disable-bt
      This will disable WiFi and Bluetooth
    • Modify on the line
      dtparam=audio=on
      to
      dtparam=audio=off
      This will disable the audio.
    • Add on the last line after the [all] section
      arm_64bit=1
      This will enable

Now disconnect the SD-Card then place it back in the RPi. Connect the power.
After a while you should be able to ping the configured IP address. When you can, you’re ready for the next step.

Setting up the OS

It is time to configure the RPi base stack before we put k3s on it. Let’s SSH to the RPi. Adjust the username and IP address to your setup:

ssh pi@192.168.1.10

Then let’s update and configure the base. Run a command at a time if you want to follow the progress or troubleshoot an issue. You should be able to run these commands multiple times in case something doesn’t work as expected

sudo apt update
sudo apt upgrade -y
sudo apt install iptables -y
sudo iptables -F
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

The first 2 commands simply update the OS to latest packages.
k3s needs iptables to configure the network and allow connectivity in/out of nodes. But it doesn’t support he newer version so we need to use the iptables-legacy instead. The update-alternatives commands help to do that elegantly by mapping the command iptables to the iptables-legacy binary.

Then reboot (sudo reboot) your RPi. That’s it, repeat these steps for all your RPis, or clone the SD-Card (as long as they are all same RPi models)

Installing Kubernetes (k3s version)

Once all your RPis are up and running you are ready to create your kubernetes cluster.

Setting up the main node

SSH to the RPi you want to be the main node. Then execute the following commands:

sudo su -
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" sh -s -
cat /var/lib/rancher/k3s/server/node-token

The last command will display a token that looks like this:

K104123450ec5bbe9d3b412a6b3de2d241b2a64c0d273685446e35171e79f94a38f::server:7512345c1234abbb0f65ea32d9874edc

This will be used to configure the nodes to attach to the main. That’s it, it’s ready. Give it a try by typing
kubectl get nodes

It should display something like this

Main node

Setting up the worker nodes

Now let’s SSH to the worker nodes and run these commands to attach them to the main, after replacing the relevant parts as:

  • use your server K3S-TOKEN
  • use your server node IP address
  • use a different name per each node (in this example k3s-node1)
sudo su -
curl -sfL https://get.k3s.io | K3S_TOKEN="K104123450ec5bbe9d3b412a6b3de2d241b2a64c0d273685446e35171e79f94a38f::server:7512345c1234abbb0f65ea32d9874edc" K3S_URL="https://192.168.1.10:6443" K3S_NODE_NAME="k3s-node1" sh -

And voilà! Your cluster is ready. From the main node, you can run the kubectl get nodes command again and you will get:

Cluster nodes

The next steps are to install my default applications there:
– Pi-hole
– transmission
– Plex

raspberrypirpi
By RaspberryPi, Tutorial