As part of this weekend’s regular service update, I also came across Mosquitto’s new 2.1.2 release. This is my tale…
I’m using Mosquitto as the MQTT broker for my IoT thermostats and smart plugs. If you’re interested, you can find more details on my setup in this and this post.
The changelog of the new release contained a few interesting points:
- The acl_file option is deprecated in favour of the acl-file plugin, which is the same code but moved into a plugin. The acl_file option will be removed in 3.0.
- The password_file option is deprecated in favour of the password-file plugin, which is the same code but moved into a plugin. The password_file option will be removed in 3.0.
I’m using both of these options, so because I was doing the update on a lazy
Sunday morning instead of Friday evening after work, I decided to be a good
sysadmin and replace the acl_file and password_file options now, instead of
waiting for the update where they’re ultimately getting removed.
The first hurdle was that there doesn’t seem to be any good docs on how to use
either the password-file or the acl-file plugins. How do I configure them?
How do I use them? How do I even get them?
After not having any success in finding any examples, I finally hit upon an idea: Look at the source code. Yet again, three Hurrah! for open source software. I found this commit, and in it this example Mosquitto config for internal testing:
listener 1883
allow_anonymous true
plugin ./mosquitto_acl_file.so
plugin_opt_acl_file ./acl_file
So first step: Figuring out whether those shared objects are actually delivered as part of the image. I had a look into the Mosquitto container and found that at lest those two libs are delivered as part of the image:
podman run -it eclipse-mosquitto:2.1.2-alpine ash
find . -iname "*.so"
[...]
./usr/lib/mosquitto_password_file.so
./usr/lib/mosquitto_acl_file.so
[...]
I ended up with this final config:
plugin /usr/lib/mosquitto_acl_file.so
plugin_opt_acl_file /mosquitto/config/acl.conf
plugin /usr/lib/mosquitto_password_file.so
plugin_opt_password_file /hl/passwd
But I kept getting an error:
1772364168: Error: Unable to open acl_file "/mosquitto/config/acl.conf".
1772364168: Error: Plugin returned 13 when initialising.
The acl.conf file is mapped into the container via a k8s ConfigMap, so I thought
that perhaps there’s something going wrong here?
I checked in a running Pod and saw this:
/ # ls -Al mosquitto/
total 16
drwxrwsrwx 3 root 1000 4096 Mar 1 11:26 config
drwxrwsr-x 3 mosquitto 1000 4096 Mar 1 11:25 data
drwxr-xr-x 2 mosquitto mosquitto 4096 Feb 9 20:01 log
/ # ls -Al mosquitto/config/
total 4
drwxr-sr-x 2 root 1000 4096 Mar 1 11:26 ..2026_03_01_11_26_50.4252382031
lrwxrwxrwx 1 root 1000 32 Mar 1 11:26 ..data -> ..2026_03_01_11_26_50.4252382031
lrwxrwxrwx 1 root 1000 15 Mar 1 11:26 acl.conf -> ..data/acl.conf
lrwxrwxrwx 1 root 1000 21 Mar 1 11:26 mosquitto.conf -> ..data/mosquitto.conf
So I didn’t see any issue, the files definitely existed.
After digging even more, I finally found this GitHub issue.
Somebody had the same issue as me. It looks like it was created by some sort of
security measure, disabling following of symlinks by default. After setting the
env variable to disable that behavior, MOSQUITTO_UNSAFE_ALLOW_SYMLINKS=1,
Mosquitto finally started up again and has been running nicely since then.
So be a bit cautious when running Mosquitto in a k8s cluster, the update to v2.1.2 might not work without some small changes.