K8s-Migration

Wherein I migrate my internal container registry to Harbor. This is part 12 of my k8s migration series. Let’s start by answering the obvious question: Why even have an internal container registry? For me, there are two reasons: Some place to put my own container images A cache for external images Most of my internal images are slightly changed external images. A prime example is my Fluentd image. I’ve extended the official image with a couple of additional plugins. And I needed some place to store them. ...

Wherein I migrate my Grafana instance over to k8s. This is part 11 of my k8s migration series. I already wrote about my love for metrics in the companion post about the Prometheus setup, so I will spare you my excitement about pretty graphs this time. 😉 For the Grafana setup, I used the kube-prometheus-stack’s integration of the Grafana Helm Chart. Database setup First step is to setup the database for Grafana. You can also run it locally, without an external database. Then, Grafana uses an SQLite DB. But the Postgres database made more sense to me. This was the first deployment of a production database with CloudNativePG and looked like this: ...

I recently started migrating my Grafana instance from Nomad to k8s and hit some very weird errors in the CloudNativePG DB after letting it run for a short while. This is an addendum to my previous post on CloudNativePG. The initial issue The first issue came during the first setup of Grafana. A couple of minutes after Grafana started running and writing data to the DB, the two database pods (primary and replica) suddenly stopped working and just threw this error: ...

Wherein I set up Prometheus for metrics gathering in the k8s cluster. This is part 10 of my k8s migration series. Let me tell you something about me: I love metrics. And pretty charts. The more the better. Back in 2020, setting up Prometheus+Grafana was what brought me to Homelabbing as a hobby, instead of just a means to an end, running some services I wanted to use. I had just gotten an updated ISP connection and found my old FritzBox not working anymore. Instead of just buying a newer one, I decided to try out OPNsense. This meant that I now had two hosts in the Homelab. My old home server running a handful of services, and the new OPNsense box. And I wanted metrics, especially about CPU and network usage. ...

Wherein I set up cloud-native-pg to supply Postgres clusters in my k8s cluster. This is part nine of my k8s migration series. PostgreSQL is currently the only DBMS in my Homelab. My initial plan was to just copy and paste the deployment over from my Nomad setup. But then I was pointed towards CloudNativePG, which is an operator for managing Postgres deployments in Kubernetes. But before I go into details on CloudNativePG, a short overview of my current setup in Nomad. I’ve got only a single Postgres instance, hosting several databases for a variety of apps. By far the largest DB at the moment is for my Mastodon instance, with something over 1 GB in size. It runs on a CSI volume provided by my Ceph cluster, located on a couple of SSDs. All apps use this one Postgres instance, and there’s no High Availability or failover. ...