Moving to a mature mTLS infrastructure

02.03.2026

In late October last year I joined my local defcon chapter to meet real security professionals and ask them for advice.

A long discussion at a bar about my current project and ambitions over a few beers ended up procuring this simple, but important piece of scrap paper.

Proxmox running on baremetal, OPNsense VM as the primary firewall, and HAproxy as a reverse proxy server used to forward to other hosts/server VMs, with a DMZ for public facing servers.

The last few months have been spent turning that simple piece of paper into reality.

I had to migrate everything from my trusty dusty old laptop, to a powerful offsite virtual environment in a datacenter that allows instant spinning up of testing environments.

Instead of using a VPN to connect to my backends, this new infrastructure would use mTLS (mutual TLS) using certificates and a reverse proxy to connect, mirroring modern infrastructure security practices.

Before

After

This was enormously challenging as I had to learn several new complicated tools and technologies — Hetzner's Robot console, Proxmox, OPNsense, HAproxy, create my own PKI — and combine them all together to make a working final product.

This page walks through my journey of setting all of this up.

Part 1 — Deciding on hardware and initial Hetzner setup

I initially pushed back on the idea of moving everything to a datacenter. I really enjoyed the idea of truly selfhosting everything in my own house. However, given the size of my tiny Geneva apartment, the price of electricity, and the noise that a server of this power would generate, a dedicated datacentre was a better idea.

It also taught me to be extra careful when touching network settings, as any misstep could mean getting kicked out of my remote session. I had to be really sure about what I was doing and get comfortable using the remote KVM rescue system.

I set myself a budget around 40 bucks a month. Using Hetzner's auction system I found a server with a decent CPU, 128GB of RAM, and 2x1TB NVME. Thankfully, I managed to grab this server before the massive recent spike in hardware prices. This server allowed me a ton of RAM for as many client and server VMs as I would need with a bit leftover.

The server is physically located in Helsinki, which adds a tiny bit of latency, but not anything dramatic. The cold weather there keeps the costs of running the server down.

Hetzner so far has been a great service. Within an hour I had access to my new server using their Console system.

To get started, all I had to do was SSH remotely into the server using the key they provided me. Hetzner provides a "rescue system" — a very light Debian-based image that runs on the system's RAM.

Basic hardening of the server

As a security best practice: "assume you're pwned." Since I know nothing about the history of this refurbished server, the first thing I did was run a secure wipe on the disks. Thankfully with NVME this takes a few seconds using the command:

nvme format /dev/nvme0n1 -s 1

Next, I needed to lock down the firewall. I created a simple rule that only allows TCP connections on port 22 from my homelab's public IP. Everything else is blocked.

Now it was time to start installing Proxmox. I followed this guide from Hetzner to get started.

However, when I tried to wget the latest ISO, my previous firewall rules were working too well — they weren't allowing my new server to establish a connection with Proxmox's servers. So I added a rule temporarily that whitelisted their IP. I'll need to review this later to see how auto-updates will work.

Besides my own rules blocking me, the Hetzner community guide worked great:

I went on to set up the disks. I decided to go with the fairly standard ZFS RAID1 setup with default settings — in case one disk fails I have a backup.

After that, I did basic Linux hardening: created a non-root user to SSH in via key-pair, disabled password login via SSH, installed updates, and gave the server a reboot. I also configured this non-root user as an admin in the Proxmox web GUI, then disabled the root account in Proxmox.

I also ran into an issue where the Robot firewall was blocking apt updates completely. Since the firewall is stateless, I had to explicitly tell it to accept incoming DNS and TCP ACK, otherwise the packets were dropped.

Then I thought it was still blocked, because I was getting a 401 error — but that was because apt was pointing updates to the enterprise subscription-based repositories. After some googling, I learned I had to edit /etc/apt/sources.list.d/proxmox.sources to point to a public repo, as well as /etc/apt/sources.list.d/ceph.sources.

For my next hardening step I set up fail2ban. Installation and configuration was pretty simple, except for the fact I mistakenly set jail enabled = true in the conf file, which caused all the template jails to default to enabled, then fail to start because they had no filters.

Here's the basic steps:

1. apt install fail2ban

2. cp /etc/fail2ban/jail.conf jail.local

3. Edit jail.local to your preference
4. Add the following to jail.local for fail2ban to work with Proxmox:

   [proxmox]
   enabled = true
   port = https,http,8006
   filter = proxmox
   backend = systemd
   maxretry = 3
   findtime = 2d
   bantime = 6h

5. Add the following filter so that fail2ban can parse the IP from Proxmox logs:

   [Definition]
   failregex = pvedaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.*
   ignoreregex =
   journalmatch = _SYSTEMD_UNIT=pvedaemon.service

6. systemctl restart fail2ban

7. I put some fake credentials into the Proxmox web UI and after a few tries — the Web UI stopped loading. fail2ban-client status proxmox shows it successfully banned me:
8. I whitelisted my IP in the jail.local config file, restarted again, and verified I was no longer banned.

Part 2 — My first Proxmox VM: OPNsense (28.12.2025)

Now that Proxmox was up and running it was time to set up my first VM.

I started by following the Hetzner guide and getting my Proxmox host network config ready for network bridging. I edited /etc/network/interfaces with the following:

auto lo10
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address MyPublicIP/26
        gateway MyGateway
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

Then I downloaded the latest OPNsense ISO using wget.

I also decided it would be best to verify the integrity of the downloaded ISO. I did a wget of their .pub and .sig files, then performed the following steps:

1. Decode the .sig file:

   openssl base64 -d -in OPNsense-25.7-checksums-amd64.sha256.sig -out checksum.sig

2. Verify the checksum file with their public key:

   openssl dgst -sha256 -verify OPNsense-25.7.pub -signature checksum.sig OPNsense-25.7-checksums-amd64.sha256

   Result: Verified OK
3. Verify the ISO:

   sha256sum -c OPNsense-25.7-checksums-amd64.sha256 --ignore-missing

   Result: OPNsense-25.7-dvd-amd64.iso.bz2: OK

Once it was verified, I created the VM in the Proxmox web GUI with the following settings:

For networking, I assigned the virtual MAC of my secondary Hetzner IP to this interface and also from the Proxmox host created an additional Linux network interface (vmbr1) and assigned it to the VM before booting it up.

Wrapping my brain around all these interfaces, layers of firewalls and how everything interacts has been quite the challenge — but reached a decent milestone here! Got my subdomain to point to the WAN of OPNsense. Wanted to test before going further. Now on to HAproxy configuration!

The only way I was able to access my backend GUI via security.luke.yt was if I temporarily disabled the OPNsense firewall. It took me a lot of tinkering and troubleshooting to figure out what exactly was blocking in the firewall. I first looked at the LAN rules, and I thought since my IP was whitelisted on Hetzner's end, and that LAN to LAN connections were OK, that I should be able to access my WAN interface.

Well, it turns out there are separate rules for the WAN firewall. I created a rule that allowed my public IP into the interface, but nowhere else for now.

Finally, I was able to access my interface across the internet. But this setup is not ideal and not professional. I can't rely on my home public IP staying the same, and there's no additional verification by the server to confirm my identity. Onto mTLS configuration.

Building my first PKI

At the beginning of this project I had heard of TLS but never mTLS. TLS is what allows HTTPS to happen between a browser and a server. The server and client create a secure connection using asymmetric encryption — the server tells the client "I am who I really say I am", and the client says OK. Once that connection is established, a symmetric encrypted tunnel is created between the browser and the server, securing any data being transferred between the two.

mTLS, or Mutual TLS, takes this one step further. It verifies that the client connecting has a certificate that has been signed by a trusted CA and that the certificate comes from this client by decrypting the private signature with the enclosed matching public key.

Right now when I try to access my OPNsense interface, I get a self-signed certificate error:

This is because the web GUI only has a self-signed certificate. The certificate has not been signed by a Certificate Authority with a chain of trust that is trusted by Firefox. Firefox doesn't even allow "accept anyway" anymore.

So, in order to do this, I needed to create a full PKI — Public Key Infrastructure. Thankfully, OPNsense has everything to do so.

I started with the most important part: the Root Certificate Authority (CA). Any certificate verification will follow a chain of trust down to this CA.

From the admin page I went to System > Trust > Authorities and clicked +:

I decided on ECC 384 and SHA384 for a good security/speed balance:

Lifetime: I ended up setting 10 years for the Root, 5 years for the Intermediate, and 1 year for any leaves. I named it "LukeYT Internal Root CA".

Then I created the Intermediate CA and selected the previously made Root as the issuer:

Now that the Intermediate was signed by the Root, I downloaded the Root CA's certificate and private key onto an encrypted USB drive, then deleted it entirely from OPNsense — effectively storing it offline.

This way, even if my Intermediate certificate becomes compromised, I can delete and re-issue one by re-importing the Root CA using the cert and private key stored on my USB key.

Now it was time to issue a leaf certificate with the Intermediate CA to be used by the Web GUI:

Tonight I made some finishing touches on my system. After struggling for some time and feeling lost, everything suddenly just clicked.

Here's a high-level picture of how it works. I rent a physical Hetzner box in Helsinki. On this box lives my Proxmox server, Shima. Shima has a physical network card named eno1. This physical network card has been assigned my Hetzner public IP.

Next, on that Proxmox server lives a VM. That VM has the network card vmbr0. vmbr0 has been assigned a Hetzner-supplied "sub IP and MAC" inside the VM.

These two interfaces are bridged together, allowing network access to and from the sub IP, based on the firewall rules configured in both OPNsense and Hetzner. The physical network card acts as a sort of "switch" that knows to forward packets to the sub IP.

Now that I've set up HAproxy and mTLS — the only way to connect to my backend is with a certificate from my CA!