S3

Wherein I figure out why my Ceph S3 is so slow and think about potential hardware upgrades. As part of my goaccess post, I had to copy around almost 60 GB of logs, from my laptop to my desktop. I decided to do that via my Ceph S3. And it was very, very slow. There were 185 files to copy, with a total size just shy of 60 GiB. The majority of that size comes from two Traefik log files, both around 30 GiB in size. I used Rclone to sync the files to an empty directory on my desktop with this command: ...

Wherein I document how I migrated some S3 buckets over to the Ceph Rook cluster and with that, made it load-bearing. This is part six of my k8s migration series. So why write a post about migrating S3 buckets, and why do it at this point of the Nomad -> k8s migration? In short, it just fit in here very well. I already planned to make Ceph Rook one of the first services to set up anyway. And then the logical next step is to have a look at what I can then migrate over without any other dependencies. And the answer to that was: Some non-service S3 buckets. With “non-service” I mean those buckets which are not directly tied to specific services running on the cluster, like Mastodon’s media files bucket or Loki’s log storage bucket. Those I will migrate over with their respective services. ...