Self-hosting email the hard way from your own routable IPv4 block up
55 points by ryangibb
55 points by ryangibb
Thank you for this.
I continue to study the gatekeepers who swear that the thing we've been doing for literally decades can't be done. I wonder where they get their silly ideas, why they seem so adamant about telling others to not do a thing...
Really, setting up reverse DNS, SPF, DMARC and MTA-STS, and you're good. I'm constantly telling people that it's not that expensive, nor that difficult.
My example email server: https://poofydoof.zia.io/
we've been doing for literally decades
I think that's the key thing:
setting up reverse DNS, SPF, DMARC and MTA-STS, and you're good
This is 100% true for domains and IP addresses that already have good reputation. If you're adding a new domain to a mail server that's already trusted by the big players, it will accrue reputations quickly. If you're moving an existing domain to a new mail-server IP and have DKIM set up, it's fine.
From what I've heard, the situation is quite different if you're starting up with a new domain and a new mail server IP from scratch. You'll be automatically spam filtered until you do whatever makes their machine-learning system happy (having people on their systems send messages to your domain and pull replies out of spam seems to commonly work well).
When I set up my own domain many years ago:
That solved most delivery problems, although DKIM with mailing lists was a problem: they insert a footer that breaks the signature. I added body-length-limit to work around thatz but now I learned that some servers might actually block emails signed that way: https://certified-senders.org/blog/dkim-body-length-a-serious-weakness-with-the-rise-of-domain-based-reputation/
Still, sometimes I end up in gmail's spam box. This is impossible to debugz because mail is delivered (no DMARC failure), and I can't reproduce with my own Gmail account.
From what I've heard, the situation is quite different if you're starting up with a new domain and a new mail server IP from scratch.
I think the exact TLD matters quite a lot here. fresh org/com/net seem much easier to bootstrap than some random new TLD.
I can imagine. I got my partner a .ninja domain a few years ago and it worked fine because it was on the same mail server that hosted a .org I'd had for 20 years (and a few other things, including our .wedding domain).
If you don't want the hassle of developing reputation, just smarthost through a reputable provider. Or smarthost while also developing reputation, so you can get important messages delivered and can keep testing less important ones until everything is right.
I've been running my own self-hosted email since ~2008, and I continue to be baffled by how many people tell me it's impossible!
Currently on debian + postfix, dovecot, rspamd (so very similar to the article!), and I continue to have more issues with my work's google workplace than my own setup :)
A lot of the issues are around reputation and deliverability. That said I'm gonna try this too, even though my ISP rotates my IP occasionally. Wish me luck!
From my own experience of running a mailserver for about 3 years, the hard part is not getting setup with DNS, SPF, DMARC and MTA-STS, it's not getting blasted into blacklists when you also send transactional mail or just not being whitelisted in some email providers i had to interact with. Chasing all that crap is not worth my time, I'd rather pay someone a fiver a month and have them deal with that.
My example email server: https://poofydoof.zia.io/
That is awesome, as is https://dmesgd.nycbug.org/dmesgd?do=view&id=8929
Any more details on this? I see the Mango Pi MQ-Pro is hard to buy. Are there other very cheap devices with decent NetBSD/Linux support?
I bought a Mango Pi MQ-Pro when I first saw it because it was pink :)
My other test email servers have been a Mac LC III+ and a Raspberry Pi Zero. The Raspberry Pis <=4 have good NetBSD support.
One additional reason to run your own mail server: you might discover that someone else is already using your domain for spam!
Anecdote: when one of my domains was transferred to Squarespace (because Google Domains was sold to them), they automatically added MX/SPF/DKIM DNS records for Mailgun, even though I didn't have a Mailgun account. Someone else claimed the account on Mailgun and sent me spam from my own domain. Thanks, Google!
You just reminded me of the time our CEO hosed our entire company infra because some web hosting service told him in their tutorial to change our nameservers to theirs.
Interesting, particularly around the IPv4 allocation.
If you want to do this yourself and are based in Europe, then pay the annual RIPE NCC membership fee to open a Local Internet Registry (LIR) account.
https://www.ripe.net/membership/payment/ shows this costs 1800 EUR/year. Ouch! Though I suppose the cheaper it is, the more readily it will be abused by spammers.
Yeah it's not cheap, but about £12/address/year means it's not so bad as we spread it out among local friends and family's services. The fun of writing an OCaml BGP server: priceless :-)
The fun of writing an OCaml BGP server: priceless :-)
i would love to see some against preexisting linux routing daemons like bird or frr (esp. on low spec machines)
Is it realistically possible to use IPv6 for sending/receiving email?
That's a good question; I took a look when writing the article but decided to defer that until I had more experience with it as I disabled it a few years ago in my mail server.
From https://dn.org/ipv6-and-domain-reputation-in-anti-spam-filters:
Email providers like Gmail, Microsoft, and Yahoo apply proprietary spam filters that weigh domain and IP reputation differently, and their support for IPv6 continues to mature. Gmail, for example, requires valid PTR records, SPF, and DKIM for IPv6 mail and actively recommends the use of DMARC. Messages sent over IPv6 without these elements often land in spam or are deferred. Microsoft’s filtering systems incorporate IPv6 into their SNDS and JMRP programs but may throttle or defer IPv6 mail if the sending reputation is unknown. It is crucial for domain owners to enroll in these programs, monitor their IPv6 traffic, and address any deliverability issues proactively.
IOW, it's yet another thing that can go wrong and probably a bad idea to setup a mixed IPv4/6 deliverry domain when starting out! There are no V6-only SMTP endpoints that I've found yet.
Yes, the big email providers, and many small ones, support IPv6 for receiving mail and will often send mail over IPv6. Having said that, enough providers only support IPv4 that you will fail to send and receive some mail if you use IPv6 only, unless you use a third party relay.