A Grafana gauge panel. It is red, with the text 98.5% below it.

Prometheus Metrics Cleanup

I had to clean up my Prometheus data, and it got pretty darned close there. When it comes to my metrics, I’m very much a data hoarder. Metrics gathering was what got me into Homelabbing as a hobby, instead of just a means to an end. Telegraf/Influx/Grafana were the first new services on my Homeserver in about five years. And I really do like looking at my dashboards, including looking at past data....

June 28, 2024 · 8 min · Michael

Homelab Backup Operator Part II: Basic Framework

In the last post of my Backup Operator series, I lamented the state of permissions in the kopf Kubernetes Operator framework. After some thinking, I decided to go ahead with kopf and just accept the permission/RBAC ugliness. I’ve just finished implementing the first cluster state change in the operator, so I thought this is a good place to write a post about my approach and setup. The journey up to now has been pretty interesting....

May 25, 2024 · 21 min · Michael

Homelab Backup Operator Part I: RBAC permission issues

As I’ve mentioned in my last k8s migration post, I’m working on writing a Homelab backup operator for my Kubernetes cluster. And I’ve run into some RBAC/permission issues I can’t quite figure out. So let’s see whether writing about it helps. 🙂 First, a short overview of the plan. I’m using the kopf framework to build a Kubernetes operator. This operator’s main goal is to handle HomelabServiceBackup resources. These will contain a list of PersitentVolumeClaims and S3 buckets which need to be backed up....

May 12, 2024 · 5 min · Michael
The HashiCorp Nomad and Kubernetes logos, connected with an arrow pointing from Nomad to Kubernetes

Nomad to k8s, Part 12: Backup Plan

Wherein it seems I need a new backup strategy. This is part 13 of my k8s migration series. During the last week, I’ve started to work on implementing my backup strategy for the new Kubernetes cluster. The original plan was to stay with what I’m already doing in my Nomad cluster. But it turns out I can’t, so I need a new strategy. If you’re prone to suffering from IT-related nightmares, you might wish to skip this one....

May 5, 2024 · 10 min · Michael
The HashiCorp Nomad and Kubernetes logos, connected with an arrow pointing from Nomad to Kubernetes

Nomad to k8s, Part 11: Container Registry with Harbor

Wherein I migrate my internal container registry to Harbor. This is part 12 of my k8s migration series. Let’s start by answering the obvious question: Why even have an internal container registry? For me, there are two reasons: Some place to put my own container images A cache for external images Most of my internal images are slightly changed external images. A prime example is my Fluentd image. I’ve extended the official image with a couple of additional plugins....

April 27, 2024 · 22 min · Michael