efa-project.org

just thinking out loud asking for general feedback on how others handle this issue.


The scenario is this:

- small company
- EFA appliance in the office
- MX record points to a static IP we get from our ISP which points to EFA
- EFA relays incoming emails to an Exchange server on premises
- Exchange relays outgoing emails to EFA which then sends them out
- the firewall masks outgoing SMTP with a fixed IP which has proper RDNS setup

Problem:
- yesterday we lost internet access for 8h
- after 2h I had a backup solution setup
- no incoming emails the whole time as the MX record was pointing to the IP of the ISP which was down
- outgoing emails were sometimes refused as I had to remove the IP masking so emails went out through an ISP without proper RDNS

Not sure how to prevent this in future. I could setup a second ISP with a fixed IP then setup a second MX which points to this IP, and have the firewall also route all SMTP coming in through this IP to EFA. The only manual step would be to change the masking with a static IP for outgoing SMTP when the 1st ISP is down to the Ip of the 2nd ISP.

The limiting factor is my current IP has no more free ports so I can't add another ISP right now

Alternatively, would it be possible to have a second EFA instance on the net then connect some sort of webmailer i.e. roundcube so employees can read7write emails on this second EFA instance in case of an outage? of course the 2nd EFA wil automatically send the emails to the main one once the ISP is up again.

I have two solutions to the problem

1/ all mail is filtered through messagelabs. they act as my primary mx host, so if I'm down, they'll hold the mail for me until I'm back up. They're a large company so I I expect their stability to be much, much greater than mine.

2/ at the office level, I have 3 separate fixed ip broadband circuits from 3 different providers (with independent networks) all configured properly for sending mail (rdns, etc) and they are configured as my secondary mx hosts in the extremely unlikely event that the messagelabs service is unavailable.

As far as I know, we don't lose mail because of connectivity issues.

Now, in your case (and in most people's cases for that matter) is I might consider spinning up a vm host with some provider to act as a secondary mx host. The secondary mx host would be configured to store the mail and deliver it to the primary mx host when it becomes available again.

I'd also configured the vm host to allow sending mail through its fixed ip for my domain (spf records, dmarc, etc) so if I could get any kind of internet access, I could send mail through the vm host smtp server while my primary server was down.

Just my 2c.

Thanks for the feedback, this is exactly what I was looking for, I mean get an insight into what other people chose as their solution.

let me add some more info:

- management doesn't believe in cloud services so messagelabs and the likes are not an option
- self-hosted would be ok-ish so say a 2nd EFA in the cloud should be OK
- we're too small to have 2 ISPs with static IPs (those are very expensive around here) and going with a cheaper solution means dynamic IP

I'm not excessively concerned about losing emails as much as about the continuity of the email service, meaning to be able to receive and send emails. So how would I get my users who're not on premises to be able to send and receive emails as my VPN won't work when my primary ISP is down and I use specific DNS records aka owa.mydomain.tld to access OWA and mail.mydomain.tld for Outlook Anywhere and those DNS records point to my primary ISP's public IP.

I'd also configured the vm host to allow sending mail through its fixed ip for my domain (SPF records, dmarc, etc) so if I could get any kind of internet access, I could send mail through the vm host SMTP server while my primary server was down.

Could you elaborate on this a bit more?
In my case, exchange is set up to relay through my local EFA which signs with DKIM and sends out. Our outgoing firewall masks SMTP with one of our IPs with working RDNS. How would Exchange or for that matter EFA know that my usual ISP is down and the backup ISP is active on my firewall so it relays through the 2nd EFA in the cloud? The only way I see is if I tell my local EFA to always relay through my second EFA in the cloud?

Ok, you have to make a decision as to what is important and what is not.

If getting a high availability connection to the internet is not possible, or not important enough to the necessary money on, then your options are very limited. Think about it for a moment.

You have several scenarios you have to account for

1/ no internet connectivity for your mail system
2/ efa server down
3/ exchange server down

your goals seem to be

(a) provide the ability to send mail at all times for all the users
(b) provide the ability to receive mail at all times for all the users
(c) never lose mail sent to you when the main server is inaccessible because of 1, 2, or 3.

Without a highly available setup and configuration, you can forget (a) and (b). The best you can manage is (c) by setting up a secondary mx server to hold your mail until your system is back online. The remote mx server can also be configured to allow users to send mail via smtp, but you'll have to set them up with a different mail configuration, to allow that. Most users will have difficulty understanding using "different profiles depending on circumstances" so that won't be as easy as you think.

If you get a second dynamic IP for sending mail will tend to get your mail blocked as spam if you send from it directly. However, you can use that line to forward mail to the secondary mx server and allow that server to send on your behalf as long as it's configured with some kind of authentication.

For users of your exchange server, if you lose network connectivity, they won't have access to their mailboxes. Nor can you set up a "backup" mailbox easily, unless you have another internet connection they can use to access the server (dynamic ip is fine for this).

...

So, here is what I might consider doing if "cloud" is not an option

exchange server hosted in the office, preferably with redundant network connections and a good firewall for handling multiple connections (I like pfsense). EFA hosted with a VM provider and configured as the primary MX, with the fixed IP for your exchange server configured as a secondary MX (or secondary EFA in front of the exchange server as secondary MX). Then use a fixed ip line, and a dynamic ip line from different providers.

exchange server configured to use primary MX host in cloud as smart host

scenarios:

i) primary mx host down? exchange server reconfigured to send via secondary mx host (locally host efa) via the fixed ip line. secondary MX record means mail is sent to your secondary mail server instead ( to the machine listening to the fixed ip line)

ii) fixed ip line down? don't care - mail goes out to the primary mx host via the dynamic ip line. if you set up dynamic dns, the your owa.*.tld and mail.*.tld domains will still resolve to the dynamic ip line (they should resolve to both fixed and dynamic). Worst case, you update the dynamic host records to remove the fixed ip line by hand. Thus users can still access their mail on your exchange server.

iii) dynamic ip line down? don't care. everything still works. worst case, remove the dynamic ip record from the owa|mail.*.tld domains

iv) secondary MX (local efa) down? don't care. just point your firewall at the exchange server instead of the efa box until problems resolved and live with the temporary increase of spam

v) exchange server down? you're fucked. go home until the problem is resolved, or avoid this by getting a highly available exchange server setup.

vi) both internet lines down? you're fucked. go home until any connection is restored. consider getting a third line in the future like I've done (yes, I've had two lines down at the same time)

The only real problem is maintaining two efa instances. It'd be nice if two instances could share configurations so you only have to configure one.

At the end of the day, if you don't have redundant internet connectivity for your locally hosted mail server, there is not much you can do.

If you need that Exchange send email from your primary efa box and only if not available use a secondary box you must do this :

NOT WORKING :
The thing is define 2 smarthosts in Exchange with different 'cost'
But Exchange doesnt consider the cost as a preference so it use both for sending

WORKING
define only one smarthost in Exchange naming it with an internal dns name (smarthost.company.local)
Create 2 mx records like this :
smarthost MX 10 efaprimary.company.local.
smarthost MX 20 efabackup.company.local.
Create A records for efapriimary and efabackup
In this way Exchange use Always efaprimary if available

Thanks everyone for the amazing feedback. I'm busy putting my current mail flow to paper in form of a drawing to help me visualize things. I'm almost done with my new drawing and will post both here for feedback once done.

Thanks for all the feedback, there is some really interesting and relevant info in here.

In the end we have decided to go with a backup solution by provider of our main connection.
Out main line is fiber optics, the backup will be DSL via a guaranteed disjunct line to our building.

We'll have one set of IPs and the failover to my backup line will happen on their side, so no need for new IPs or any routing.