Self hosting email is great, until it isn’t. There’s plenty of options on where to get your email whether you’re bringing your own domain or not. Moving from a self hosted mail server to a managed email (or between two managed services) is primarily just DNS changes. This is a quick summary of what in DNS you need to change, and why.
Ups and Downs of self hosting email.
I’ve been self hosting my email for quite a while, but it was always in the back of my mind that it would be worth offloading to avoid the inevitable maintenance and possible downtime. Really, once I had email setup, there wasn’t much maintenance outside of upgrades and reboots. Sure, there was extra work for DKIM, SPF, and the occasional log review to dial in Fail2Ban, but outside that it all just worked. The good thing about email is that even if the server is down for a bit, the sender will try again for some period of time. So I don’t consider a bit of downtime here and there as an issue.
Catch-All
Because I’m the only user on my domain, I’ve gotten into the habit of handing out an email address relevant to the company/business/use, so that I can find out who gave out or lost my email address, and set up filtering easier. For example, if I’m handing out my email address to a salesman at a car dealer in Omaha, I would give them the email address of omaha-cardealer@my-domain.com. This tends to get a confused response (many times I’ve been asked “Do you work for us?”), but a quick blurb of “Its for mail forwarding” is enough for people to lose interest. All it takes to do this is to enable a catch-all email address (see your host/server configuration) and then handle mail forwarding based on the To: field.
Logs
Having access to your own maillogs are pretty handy. Let’s face it, if you’re self hosting anything, you’re going to be digging into logs at some point. But if you didn’t enjoy that at least a bit you wouldn’t be self hosting. How great is it to dial in your Fail2Ban, or manually drop the IP that won’t stop knocking on your server?
Someone didn’t get an email? A quick search and you can provide that their email server got it or didn’t. But, that doesn’t mean much to a lot of people, and unless you’re emailing another self-hoster, providing the log information or even just the general “your server got it” doesn’t mean a whole lot. Telling your email recipient that their mail server received it is effectively the same as it not being sent at all, other than maybe a push to check spam folders.
Blocks from Providers
This is the worst, and what finally got me to stop self hosting. Running a mail server on a VPS runs into a whole host of problems, even if your configuration (DNS, DKIM, SPF, and DMARC) is set up perfectly. On creation of a VPS, there’s no way to know what it was used before you were allocated that IP. It may have been used and flagged as a spammer before you got it. Even if not that IP, another IP (or multiple) in your new network block may have been flagged resulting in a provider not accepting email from any IP address within that block. Even if everything is fine on creation of the server, this can change at any time.
Diagnostic-Code: smtp; 550 5.7.1 Unfortunately, messages from [YOUR.IP.ADD.RESS]
weren't sent. Please contact your Internet service provider since part of
their network is on our block list (<REMOVED>). You can also refer your
provider to http://mail.live.com/mail/troubleshooting.aspx#errors.
Typically your VPS provider will go to bat for you and work with the mail administrator where you’re getting blocked. But again, this can change at any time. Some mail providers won’t even respond with this information, and instead just silently mark your email as spam.
Where to get email service
A quick search at your favorite internet search engine provides plenty of options for hosted email. These can include anything from a Shared Hosting platform, a VPS provider with a managed email service, or one of the “big” guys such as Outlook or Gmail. Both have advantages and disadvantages. Going with a Shared or VPS host gets rid of the management aspect, but can still run into issues with blocks from other providers. Using Outlook or Gmail may cost a little more, but with the bonus of being a huge email provider and additional business tools.
There’s way more to the decision than what I’ve given above if you have specific business needs, multiple users, etc. Definitely dig a little deeper if you’re looking for more than just someone else to host your email.
How to migrate
This is quite high level as there are too many mail and DNS providers to attempt to cover anything more than the basic steps. I use Rackspace DNS, and was previously hosting my email on a Linode server. The steps are fairly simple for the migration itself.
- Sign up with your new email host
- Update DNS to the new host
- Enable SPF
- Enable DKIM
- Enable DMARC
Sign up for your new email host
I went with Google because of the ability to add catch-all email forwarding, and because the integration with Android was worth more to me than the access to Office features with using Outlook. Prices were comparable between the two for my use.
Update DNS to point to the new mail host
This will be provided by your new email host. DNS records for your domain use MX records to direct mail. You may have one MX record (most likely if you’re self hosting), but multiple are possible and common with specified priorties. Your new provider should tell you exactly what needs to be entered.
TTL is a very important concept with DNS here. Time To Live basically says “this record is good for X amount of time.” If your record’s TTL is 24 hours, that means that it could be up to 24 hours before your new MX records will be used and your mail be sent to the new provider. I always keep my TTLs 1 hour or less (typically 5 minutes), but just sure to not tear down your existing mail setup until at least the TTL from the previous DNS record is over.
$ dig dankolb.net MX +short
1 ASPMX.L.GOOGLE.COM.
5 ALT2.ASPMX.L.GOOGLE.COM.
10 ALT3.ASPMX.L.GOOGLE.COM.
5 ALT1.ASPMX.L.GOOGLE.COM.
10 ALT4.ASPMX.L.GOOGLE.COM.
More DNS – SPF
Sender Policy Framework, or SPF is a DNS record that essentially states where email can be sent from. This is a method of preventing spam mail by allowing the receiving mail server to validate that email received for a domain came from a mail server that is authorized to send that email. There’s plenty of options for SPF, take a look at the documentation for more information. Your new host should provide this as well in order to mark their sending servers as valid for your domain.
$ dig dankolb.net TXT +short | grep spf
"v=spf1 include:_spf.google.com ~all"
This record states to use _spf.google.com to find the list of approved senders, and SoftFail (accept the email but mark it). The SoftFail allows email to be sent from other places without flat out rejecting (such as directly from a WordPress installation).
Even More DNS – DKIM
DomainKeys Identified Mail or DKIM is a method for identifying email messages by signing and verifying based on the public key provided in the domain’s DNS. This signs the email with a header field that provides a hash of the message and DNS record to query for the public key, as well as additional information. This provides to verification that the email was generated from an authorized server (or at least one that had the private key!). Again, your host would provide this, but it may require additonal steps such as the key being generated.
$ dig google._domainkey.dankolb.net TXT +short
"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAgWG5dWv8XqN9UUqDsoi3F5wW8SwCahdslYbtygLHZageCccyNKM5ux7IhDG1sHKVM4ASG+jV6NvaMlxxIWMAAEQ3gQjZSVzsGzPXAdoaVJL73x+VfxuAmhpz8NPp4GLZMzGuMAH/Aq1w0IsCPzPGwd0jmZ1A8pOGPBDnlpYKAklTm+Rb/iv+8xUMy3O/jLLZj" "xK9/0Zo0+K28dB2QgozgIIABXFDSoYNUkg9yH4ag1cZmhSkaQpJ17TwLTqymHO6sw4pkm7EcIRYhPtjdmwunPEm53n6ObuT/fRK3UFNqjpRp2vb6VPdHmK8MjFZVOumsy+FMjaZaJhytoSICkNlfQIDAQAB"
A DKIM email would actually specify the DNS record to use here, in this example google._domainkey.dankolb.net, so there could multiple records. Because this is just a header value, this is compatible with an email server that does not do DKIM validation.
Interesting side note, DKIM initially failed to be enable after a new GSuite account creation giving a Error #1000 pop up message after a 500 from the backend service. After reviewing with support, I received further information and an eventual follow up email. Enabling DKIM can take some time for new GSuite accounts, and after two days I was able to enable and continue. As mentioned above, for my personal email this wasn’t an issue, but when migrating multiple users this could be a problem. This can be easily worked around (and would probably be more realistic) to set up the new account, give it some time, then modify DNS.
And Even More DNS – DMARC
Domain-based Message Authentication, Reporting, and Conformance DMARC provides a specified action to receiving mail servers based on SPF and DKIM results. Essentially, this allows the mail administrator to definitively state what to do with email that doesn’t match SPF/DKIM, and where to send aggregated mail reports.
$ dig _dmarc.dankolb.net TXT +short
"v=DMARC1; p=none; rua=mailto:postmaster@dankolb.net"
This is not a terribly exciting example of dmarc, as p=none is providing no specific action to be done by the receiving mail server. But, to keep with the SPF record allowing emails to be received from elsewhere, this is needed to prevent rejection of the email (or it could use quarantine).
Additional (Optional) Steps
There’s plenty of other things to do depending on your email and how you use it, but the above outlines the initial setup. Further steps to consider not described here include:
- Reconfiguring email clients (mobile, desktop)
- Migrating existing emails from the old host to the new host
- Updating email filtering and forwarding