Ceph

Wherein I figure out why my Ceph S3 is so slow and think about potential hardware upgrades. As part of my goaccess post, I had to copy around almost 60 GB of logs, from my laptop to my desktop. I decided to do that via my Ceph S3. And it was very, very slow. There were 185 files to copy, with a total size just shy of 60 GiB. The majority of that size comes from two Traefik log files, both around 30 GiB in size. I used Rclone to sync the files to an empty directory on my desktop with this command: ...

First post in the new sysadmin category. I’ve recently realized that I’ve never written about how I’m running my desktop setup on Linux. Triggered by having to revamp one of my Gentoo portage ebuilds for my desktop, let me remedy this and tell you a bit about what my personal computing outside the Homelab looks like. My Linux journey started back in 2007. I was about to start my computer science degree. And I had allowed myself one month between finishing the then mandatory military service and some jobs to bridge the time before University and my program starting in October. At the beginning of September, I got my first University laptop, an Acer TravelMate 5720 Core 2 Duo with an ATI GPU and Windows Vista. Receiving that laptop, I thought to myself: Any real CS student should be running Linux. ...

Wherein I set up some Prometheus metrics gathering from Ceph’s S3 RGW and build a dashboard to show the data. I like metrics. And dashboards. And plots. And one of the things I’ve been missing up to now was data from Ceph’s RadosGateway. That’s the Ceph daemon which provides an S3 (and Swift) compatible API for Ceph clusters. While Rook, the tool I’m using to deploy Ceph in my k8s cluster, already wires up Ceph’s own exporters to be scraped by a Prometheus Operator, that does not include S3 data. My main interest here is the development of bucket sizes over time, so I can see early when something is misconfigured. Up to now, the only indicator I had was the size of the pool backing the RadosGW, which currently stands at 1.42 TB, which makes it the second-largest pool in my cluster. ...

Back in July, I was greeted by this error on my Ceph dashboard while visiting family: A Ceph error you generally don’t want to see while you’re 400 km away from your Homelab. This error meant that during the nightly scrub, Ceph detected an error that was not trivially resolvable. ...

Wherein I write down things that don’t feel like they should be their own post. My blogging notes are starting to really fill up with small topics I’d like to write about, but which don’t feel like they warrant their own post. On the other hand, they also don’t feel ephemeral enough to just be a Fediverse post. So I decided to introduce the Sammelsurium, which is the German word for a random collection of things. ...