Replacing a Broken HDD in my Ceph Cluster

Back in July, I was greeted by this error on my Ceph dashboard while visiting family: A Ceph error you generally don’t want to see while you’re 400 km away from your Homelab. This error meant that during the nightly scrub, Ceph detected an error that was not trivially resolvable. ...

September 29, 2025 · 14 min · Michael

Updating my Kubeadm k8s Cluster from 1.30 to 1.33

Wherein I talk about updating my kubeadm Kubernetes cluster from 1.30 to 1.33 using Ansible. I’ve been a bit lax on my Kubernetes cluster updates, and I was still running Kubernetes v1.30. I’m also currently on a trip to fix a number of the smaller tasks in my Homelab, paying down a bit of technical debt before tackling the next big projects. I already did one update, from my initial Kubernetes 1.29 to 1.30 in the past, using an Ansible playbook I wrote to codify the kubeadm upgrade procedure. But I never wrote a proper post about it, which I’m now rectifying. ...

September 21, 2025 · 17 min · Michael
The CloudNativePG logo. It shows the name 'CloudNativePG' as well as a silhouetted elephant next to it.

Migrating my CNPG backups to the Barman Cloud Plugin

Wherein I migrate my CloudNativePG setup to the Barman Cloud Plugin. During my migration from Nomad to Kubernetes, I started using CNPG for my database needs. For more details, have a look at this post. I configured their backup solution right away. It consists of a component which runs in the same Pod as the main Postgres and backs up both, the Write Ahead Log (WAL) and the full database, all while the instance is kept up and running. Those can then be copied to an S3 bucket for long term storage. ...

September 10, 2025 · 9 min · Michael
The Bookwyrm logo. It shows a wyrm, meaning a dragon but without the wings, slithering around a book.

Setting up Bookwyrm

Wherein I’m adding Bookwyrm to my Homelab. I used to read novels. A lot. On school days, I would spend the approximately twenty minutes between the end of my morning routine and having to head off with a novel. Ditto for lazy Sunday evenings. During my service as a conscript, I would always find space for a book in my pack when we went on a training exercise. At University, the most difficult decision while packing for a trip home would be judging how many books I would need to pack to ensure I would not run out. ...

August 31, 2025 · 23 min · Michael

Improving Multi-Arch Image Build Performance by not Emulating

Wherein I update my container image build pipeline in Woodpecker with buildah. A couple of weekends ago, I naively thought: Hey, how about stepping away from my Tinkerbell experiments for a weekend and quickly setting up a Bookwyrm instance? As such things tend to turn out, that rookie move turned into a rather deep rabbit hole, mostly on account of my container image build pipeline not really being up to snuff. The current setup Before going into details on the problem and ultimate solution, I’d like to sketch out my setup. For a detailed view, have a look at this post. ...

August 16, 2025 · 20 min · Michael