A small cluster.
A lot to learn from.
Three physical machines for learning how Linux infrastructure is configured, observed, and recovered when something breaks.
The purpose
I built this lab to get practical experience operating infrastructure: automating Linux hosts, deploying applications, inspecting traffic and storage, and testing what happens when a machine goes offline.
The machines are three Lenovo ThinkCentre mini PCs connected through a Gigabit Ethernet switch. Each has four logical CPU cores, about 8 GB of RAM, and a 256 GB NVMe SSD.
How it fits together
Ubuntu Server runs on all three hosts. Ansible manages the baseline, Kubernetes prerequisites, and containerd configuration. Kubernetes is bootstrapped with kubeadm, with Flannel providing pod networking over the LAN.
A two-replica Nginx application exercises scheduling and pod replacement. The monitoring stack uses Prometheus, Grafana, and Alertmanager, installed through Helm. Local Path Provisioner gives monitoring persistent storage on individual nodes.
An HDMI monitor attached to node01 displays Grafana through a lightweight browser kiosk and a systemd-managed local port forward.
The failure that explained the storage
I powered off node03 while Grafana was running there. The dashboard stayed visible in the browser, but its requests failed. Kubernetes reported the node as unavailable.
Replacing a pod does not move its data. Grafana’s persistent volume was a directory on node03’s disk. A replacement pod on another node could not use that directory while node03 was offline.
This was a useful distinction between compute recovery and data availability. Local storage survives pod replacement on the same machine, but it does not replicate data to the other workers. Restoring node03 brought the service back.
The next storage milestone is to back up and migrate monitoring to node01, then test worker shutdowns separately. That deliberately shares the control plane’s failure domain; it does not make monitoring highly available.
What I have exercised
- Bootstrapped a three-node cluster and brought all nodes to Ready.
- Accessed an Nginx application through a NodePort Service.
- Deleted a pod and observed Kubernetes create a replacement.
- Deployed persistent monitoring storage and the HDMI Grafana kiosk.
- Powered off a worker, observed the local-storage limitation, and restored service after the node returned.
These are individual setup and recovery checks. I have not established an availability percentage or measured a recovery-time objective.
This website, running here
RUNNING ON KUBERNETESThis static site is packaged in a non-root Nginx OCI image. GitHub Actions builds, tests, and publishes the image, then two replicas run on my Kubernetes homelab with verified access across my local network.
The current release is pinned by image digest in the homelab repository. Deployment updates are still applied manually while GitOps delivery, secure public access, monitoring, and recovery tests remain planned.