Wherein I document how I migrated some S3 buckets over to the Ceph Rook cluster and with that, made it load-bearing. This is part six of my k8s migration series. So why write a post about migrating S3 buckets, and why do it at this point of the Nomad -> k8s migration? In short, it just fit in here very well. I already planned to make Ceph Rook one of the first services to set up anyway....
While working on some internal documentation of my Rook Ceph setup, I found that my pool’s Placement Groups were still at size 1, even though I had transferred about 350GB of data already. I have the PG Autoscaler enabled by default on all pools, so I won’t have to have an eye on the PG counts. But for some reason, scaling wasn’t happening. Digging into the issue, I finally found the following log lines in the MGR logs:...
Wherein I talk about the setup of Ceph Rook on my k8s cluster. This is part five of my k8s migration series. The current setup I’ve been running Ceph as my storage layer for quite a while now. In my current Nomad setup, it provides volumes for my jobs as well as S3 for those apps which support it. In addition, most of my Raspberry Pis are diskless, netbooting off of Ceph’s RBD block devices as their root....
Wherein I talk about the Ingress setup for my Homelab’s k8s cluster with Traefik. This is part four of my k8s migration series. After the initial setup of some infrastructure like external-dns and external-secrets, I went to work on the Ingress implementation for my cluster. I chose Traefik as my Ingress controller. This was mostly driven by the fact that I’m already using Traefik as the proxy in front of my current Nomad cluster, and I’ve become quite familiar with it....
Wherein I talk about migrating from Cilium’s L2 announcements for LoadBalancer services to BGP. This is an addendum to the third part of my k8s migration series. BGP instead of L2 announcements? In the last post, I described my setup to make LoadBalancer type services functional in my k8s Homelab with Cilium’s L2 Announcements feature. While working on the next part of my Homelab, introducing Ingress with Traefik, I ran into the issue that the source IP is not necessarily preserved during in-cluster routing....