LMStudio is pretty much the standard. I think it’s opensource except for the UI. Even if you don’t end up using it long-term, it’s great for getting used to a lot of the models.
Otherwise there’s OpenWebUI that I would imagine would work as a docker compose, as I think there’s ARM images for OWU and ollama
Just glancing through that guide:
OPNsense instead of Pfsense, because pfsense is going to rugpull, it’s just a matter of time. I wouldn’t trust the twats that run it farther than I could throw them because they’re pretty silly people. Rossman suggests exactly this in the intro to the router section, he would change if he hadn’t been using it for a decade already. Unfortunately, a lot of this guide is focussed on how to do it via pfsense and if you’re brand new, you’re going to have to figure out how to do it in OPNsense yourself.
Wireguard/Tailscale instead of openvpn. Faster and way easier to set up. Don’t even try to set up a full LAN routed VPN, just use Tailscale for the services you want. And use it for everything and everyone instead of punching holes in the firewall.
He’s definitely right about mailcow; if you’re reading that guide for information, you are not a person that should be self-hosting email.
Meshtastic node?