Consul Connect certificate problems

I updated my Nomad and Consul clusters recently. Especially Consul was a large jump, getting from 1.13.5 to 1.15.1. After about three days, I suddenly started getting 500: Internal server error from a lot of my services. In this article, I will be going into the debugging process and explain a little bit about what Consul Connect is and how it works. Why Consul Connect? So Consul Connect, or Consul Service Mesh, creates a sort-of overlay network to connect services running on multiple machines, in a secure manner....

April 2, 2023 · 9 min · Michael

Handling service configuration files in Nomad

I’ve just had a major success: My docker-compose like Nomad script can now use the nomad binary with the job run -output command to transform a HCL file into JSON for use in the Nomad API. Before, my tool was using the Nomad API’s /v1/jobs/parse endpoint. This meant that I was not able to make use of any of the HCL2 functions recently introduced. I’m mostly interested in using the file and fileset functions, and I want to tell you why....

January 12, 2023 · 9 min · Michael

Migrating to 3-node HA for Consul/Vault/Nomad

As mentioned in my previous post on migrating the Consul/Vault/Nomad servers from a VM to a Raspberry Pi, I was still waiting for some more Pis to arrive to extend the Nomad/Consul/Vault clusters to a HA configuration for all three. The main reason for this is not necessarily fault tolerance, but rather gaining the ability to restart the controllers without taking down the entire Nomad cluster. Now I’d like to give a short overview of the experience, and end with a bit of an overview on the resource consumption (spoiler: Raspberry Pi 4 4GB are absolutely sufficient)....

December 25, 2022 · 7 min · Michael

Cluster Controller Migration to a Raspberry Pi 4

I am currently working on distributing my Homelab a little bit more. My main driver is high availability. Do I need high availability in a homelab setup? No, not really. But I was getting annoyed by having to take down the entire Homelab whenever I was doing an update on my single server. The newest part of that project is my cluster controller. That is the machine running the servers for my Vault, Consul and Nomad cluster....

December 10, 2022 · 9 min · Michael