I wanted to use HTTPS with my OctoPrint setup so I set up a local certificate authority with Step, which you will need for this guide to work. Much of this is based on guides from both HAProxy and Step, combined with some elbow grease on my end to adapt them for my particular setup. Assuming you have both the local certificate authority and a working instance of HAProxy 2.6 or newer (whatever's in Debian stable right now) already set up, read on:
step ca bootstrap --ca-url https://YOUR_CA_AUTHORITY --fingerprint THE_FINGERPRINT_FROM_THE_CA_AUTHORITY