2025-10-07 19:48:00
A few weeks ago I set up a locally hosted Mailcow instance, with proper TLS certificates from Let's Encrypt. Good stuff.
At the time I'd put off automating certificate renewal, because I wasn't ready to deal with my DNS provider's API yet. Turns out that TransIP is dead simple to use with the official Acme.sh solution!
There's a ready-to-run implementation of Acme.sh for TransIP DNS. Here's the documentation for it.
All you need is to follow TransIP's documentation of opening up and securing the API for your account.
The script I'd showed before only needs a tiny adjustment to work:
#!/bin/bash
#
# Documentation:
# https://github.com/acmesh-official/acme.sh/wiki/dnsapi2#dns_transip
if [[ ($# -gt 0) && ($1 == "--force") ]]
then
Force="--force"
else
Force=""
fi
export TRANSIP_Username="break524"
export TRANSIP_Key_File="./api.pem"
acme.sh --set-default-ca --server https://acme-v02.api.letsencrypt.org/directory
acme.sh --issue --dns dns_transip --dnssleep 300 -d mail.${Domain} ${Force}
acme.sh --install-cert -d mail.broehaha.nl \
--cert-file data/assets/ssl/nochain.pem \
--key-file data/assets/ssl/key.pem \
--fullchain-file data/assets/ssl/cert.pem
kilala.nl tags: work, sysadmin,
View or add comments (curr. 0)
2025-10-05 16:50:00
Last year I did my traditional comparison of incoming and outgoing exam objectives for the Linux+ exam. It's good for students to know the changes from XK0-005 to XK0-006.
The latest version, also known as v8, went live in June. I hadn't looked at the official objectives document yet, but now I can confirm: there are no real changes between the XK1-006 draft objectives and the official, final XK0-006 objectives. There's only a few small corrections, like objective 3.3 which stated that sshd is solid state hybrid drive, instead of the SSH daemon. :)
I'm still in the midst of teaching v7 to my current group of students. They will take their exams in early December, with the final deadline being January 15th 2026. In the mean time I'm creating the new slide decks for next year's group.
kilala.nl tags: work,
View or add comments (curr. 0)
2025-09-29 21:43:00
Running Proxmox on an older Optiplex 7020 Micro, my networking is limited to the onboard e1000 Intel Ethernet card. So far it's treated me well, except ...
Except for those situations where I'm synchronizing tens of gigabytes with Syncthing, that's running on an Ubuntu VM in Proxmox. :)
e1000e 0000:00:1f.6 enp0s31f6: Detected Hardware Unit Hang
Is what the e1000 kernel module starts throwing at me, upon which point all network connectivity fully dies until I reboot.
It seems this has been going on at least since 2019 and it's a known issue with the older e1000 hardware, working with modern-day throughput. Here's a more recent thread on Reddit.
Supposedly a work-around for the issue lies in disabling certain on-card functionality, forcing it back to the CPU. Let's see if my connection remains stable now.
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
2025-09-23 22:12:00
After getting a proof-of-concept running for my self-hosted email server, I'm ready for my next step in "stepping away from big tech". A lot of my files are on OneDrive and I'm getting antsy about it.
Ideally I'd use Synology Drive and Synology MailPlus as the enterprise grade replacements of MS365 and OneDrive. But getting the gear ready has some requirements I can't fulfil just now.
So I'm testing alternatives, just like Ernie Smith in 2022.
With a few hours of fiddling I have another hardened Ubuntu server, running as centralized server for Syncthing. I've also set up a local/private Syncthing Discovery server, to make it easier for the server and systems in the ecosystem to find each other. The alternative would be to use public Syncthing Discovery servers, through NAT and port forwards, while opening up my infrastructure to the Internet.
I don't want that. I want it all to work via my VPN. And it does. :)
My Macbook is currently synced with the Ubuntu server at the office. The small amount of data I chucked back and forth looks okay. Before I start putting important data on there, it's time to ensure full drive encryption on the server and to also have backups up and running.
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
2025-09-23 22:01:00
While setting up all the DNS records needed for the Mailcow server, I ran into a few small snags. Troubleshooting was made so much easier, thanks to three tools:
Setting up my DANE record was eased by using the tool at Huque.com.
Michael Lucas' "Run your own mail server" book has been an indispensible read! It's clarified quite a few things I didn't understand yet, about the email ecosystem.
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
2025-09-21 22:56:00
Another hour, another case of PEBCAK.
Emails from my self-hosted Mailcow instance to GMail were getting bounced with this error report:
status=bounced (host gmail-smtp-in.l.google.com said:
550-5.7.26 Unauthenticated email from broehaha.nl is not accepted due to
550-5.7.26 domain's DMARC policy. Please contact the administrator of
550-5.7.26 broehaha.nl domain if this was a legitimate mail. To learn about the
550-5.7.26 DMARC initiative, go to 550 5.7.26 https://support.google.com/mail/?p=DmarcRejection
It took a little fiddling and when I tested with the MXToolbox.com email deliverability tool I found out what was wrong.
While Mailcow was signing my emails with DKIM, the recipient wasn't able to match the signature to my published key. I had erroneously followed the example of another domain of mine, publishing the key as "s1._domainkey". That was wrong.
Mailcow's admin interface specifically tells you the selector in the DNS name. It should be "dkim._domainkey" (or is it "email._domainkey"??). I made both of those and now it works! Gmail validates my signatures, MXToolbox says it's fine too... and even another domain of mine no longer puts the emails in the junk folder. Nice!
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
2025-09-21 22:45:00
I'm setting up Mailcow for self-hosted email. Lots of moving parts and one of the stranger things was this bounce message when emailing to my MS365 account.
<tess@${MyDomain}>: host ${MyDomain}.e-v1.mx.microsoft[52.101.68.18] said:
550 5.7.1 Service unavailable, Client host [178.zzz.yyy.xxx] blocked using
Spamhaus.
That IP address was not something I'd outright recognized as my own. I know exactly which IP address is tied to my static fiber connection, so I was confused about this one! I also couldn't do a reverse DNS lookup, the IP did not link back to a name.
It was only while doing some Rubber Ducky Debugging on the Angry Nerds Discords that I realized what was happening: that IP address belongs to my fallback 4G GSM router!
More than two years ago I'd setup a Teltonika RUT241 as the secondary WAN interface for my Unifi router. It works like a charm and load balancing makes it possible to have faster downloads than my single fiber allows.
But load balancing also meant that some emails were being sent over the GSM WAN. And that's not cool. :D
It was easily fixed by adding a policy-based route, ensuring that all hosts in the DMZ would only communicate with the Internet via the fiber on WAN1.
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
2025-09-21 21:13:00
The past few days I've been slowly piecing together a self-hosted email and productivity platform, just to see if I could. This project has a lot of moving pieces, which all need to sync up, so I don't look like a spammer to the outside world.
One of the things I need, is a properly trusted certificate. One free and easy way to get those is through Lets Encrypt. Their Certificate Authority uses a very hands-off approach, insofar that it's aimed at high automation through the Acme tooling.
With Acme there are a few ways you can prove that you're actually requesting a certificate for a domain you own. By far the easiest, is to have Acme run as a scheduled job on your public web server. If that's not an option you can also provide Acme with API access to your DNS records. And if that's not an option? Well...
As it usually goes, I'm making things more difficult than I have to.
I could use the easiest approach, running Acme with the web server for my email platform. The trouble is that this web server also offers the webmail as well as the admin functionality for the platform. I want neither of those on the web, I want them 100% in my private network only. And there's no easy way to split them.
I could also look into API access to my TransIP DNS account and in the long term I probably will. But not for now.
So I had to go the third way: the Acme manual DNS registration. This means that I need to manually run the acme.sh client, then manually register the DNS record their API expects, then rerun acme.sh to get the certificate. And I need to repeat that at least every fifty days. Here's some useful acme.sh documentation.
The process on my server was as follows. First I edited mailcow.conf to set "SKIP_LETS_ENCRYPT=y". This is per Mailcow's own documentation.
I then ran:
git clone https://github.com/acmesh-official/acme.sh.git /opt/acme.sh
echo 'export PATH="${PATH}:/opt/acme.sh"' > ~/.bashrc
chmod +x /opt/acme.sh/acme.sh
acme.sh --issue -d mail.${MyDomain} --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please
Then I went ahead and created the requested DNS record. After waiting for it to propagate, I run the following. The export/install instructions were found here.
acme.sh --set-default-ca --server https://acme-v02.api.letsencrypt.org/directory
acme.sh --issue -d mail.${MyDomain} --dns --yes-I-know-dns-manual-mode-enough-go-ahead-please --renew
acme.sh --install-cert -d mail.${MyDomain} \--cert-file data/assets/ssl/nochain.pem \--key-file data/assets/ssl/key.pem \--fullchain-file data/assets/ssl/cert.pem
docker compose restart
In the ideal case, I'd have a way of the built-in Acme registration to work without exposing the web interface(s) but the developers have already indicated they won't be building that any time soon.
kilala.nl tags: sysadmin,
View or add comments (curr. 0)
All content, with exception of "borrowed" blogpost images, or unless otherwise indicated, is copyright of Tess Sluijter. The character Kilala the cat-demon is copyright of Rumiko Takahashi and used here without permission.