exchange server and EFA setting

General E.F.A. discussion
Post Reply
fatima2007
Posts: 19
Joined: 03 Jan 2018 11:14

exchange server and EFA setting

Post by fatima2007 » 03 Jan 2018 11:21

hello
i have exchange server 2016
i want to use efa for spam filtering. i see 2 youtube videos for setting but there was not enough. could you have a complete configuration learning video or documnet for both exchange server and efa that i could use?
could anyone please help me?

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 06 Jan 2018 05:26

That's a pretty open ended question, but I must admit I had to do a bit of digging to figure out what to do.

For the record I've got 2 EFA servers (both virtual, one's a cold standby weekly clone of the first), 2 (virtual) Zen Load Balancers for HA for SMTP/HTTPS, 3 Exchange servers in a DAG (2 onsite (thus the need for Zen), 1 offsite).

Though I have the clone of EFA it needs to manually be spun up, so that's currently a point of failure that has no automatic resolution. I'll probably eventually have two EFAs running and equal weighted MX records pointing to external IPs for each, but the annoyance of that is we've then got 2 servers to check for mail troubleshooting/tracing. Either that or 3 MX records; priority 10 to EFA1, 20 to EFA2, 30 to EFA1 again.

Anyway, on with what I've got noted down/can remember.

SSH into EFA;
  • Recommended: IP Settings > Primary and Secondary DNS. Set to internal Active Directory DNS. Also set domain name to your AD domain.local name. Whatever you do, don't use a public DNS, it'll be blocked from doing RBL lookups for you.
  • Optional: MailWatch > Enable/Disable greylisting. I have this on, but I have modified the greylisting time to 3 minutes to reduce the delay. Some poor SMTP implementations (printers for example) won't retry so you may have to add exceptions if you enable this (done via CLI).
  • Optional: MailWatch > Quarantine retention. I've set it to 14 days. Depends on how much storage you have, how much volume you receive, etc.
  • Required: Mail Settings > Outbound mail relay. Add anything that needs to relay through your server.
  • Optional: Mail Settings > Outbound smarthost. If you need to relay out through something for sending emails, configure it here.
  • Required: Mail Settings > Transport settings. Add every domain you want to accept. Set the internal IP to your Exchange server (or load balancer's virtual IP if, like me, you run Zen Load Balancer).
    -- If you've got a lot of domains to add then you can drop to shell and edit "sudo vi /etc/postfix/transport", then run "sudo postmap /etc/postfix/transport" and "sudo service postfix reload".
  • Recommended: Spam Settings > Non Spam Settings.
    Disable storing non spam: No (this allows training of false positives plus a recent archive/recovery).
    Disable non spam signatures: Yes (This setting depends on whether you want the EFA watermark, I disabled it as we (the IT admins) manage the spam, not the users. Additionally most (99%) of our users are external and I haven't exposed EFA to the internet.)
  • Recommended: Spam Settings > Spam Settings. Enable spam delivery: Yes. (This, in combination of other rules mentioned later, allows suspect spam to go to the user's junk e-mail folder.
  • Optional: Recommended: Spam Settings > MailScanner Max Message Size: 10240 (spammers generally don't send large emails but I did increase this from the default)
  • Recommended: Spam Settings > Mailwatch hide high spam/mcp: Yes. If you want users to see really suspect stuff then have it as no, but I block high spam.
  • Optional: Virus settings > Cleaned Message Delivery: Yes or no. Up to you.
Then drop to shell.

sudo vi /etc/MailScanner/MailScanner.conf

Code: Select all

Spam Actions = store deliver header "X-Spam-Status:Yes"
Yours may be a little different depending on your settings, but it's important it has "deliver" and "X-Spam-Status: Yes" in it.
I also changed;

Code: Select all

Notify Senders = yes
Notify Senders Of Blocked Size Attachments = yes
sudo vi /var/www/html/mailscanner/conf.php

Code: Select all

Added "<style>" to ALLOWED_TAGS
sudo vi /etc/MailScanner/defaults

Code: Select all

ms_cron_av=1
Note: This shouldn't be needed, pretty sure it's been fixed (https://github.com/E-F-A/v3/issues/328) but my config notes mentioned it, so mentioning it here.

sudo vi /etc/MailScanner/filename.rules.conf ... and archives.filename.rules.conf
-- Before the final deny add in "Allow \.ics$ - -". Tab separated, make sure you don't get these as spaces. I added this as ics (calendar) files can have odd names which trip up mailscanner.
I actually also commented out the last deny, found it actually caused too many issues. For some reason we get lots of files with domain names in them. e.g. "file from forum.efa-project.org.doc" would have been blocked.

sudo vi /etc/postfix/main.cf
Changed;

Code: Select all

smtpd_client_restrictions = permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org
To;

Code: Select all

smtpd_client_restrictions = permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org, reject_rbl_client b.barracudacentral.org, reject_rbl_client bl.spamcop.net
Depends on what RBLs you like to use.

If you need to add any IP exceptions to greylisting then sudo vi /etc/sqlgrey/clients_ip_whitelist.local

Once all the edits are done then;

sudo service MailScanner restart (might be all lower case in latest version?)
sudo service postfix restart

If EFA (or equivalent) is the only SMTP entry point into the organisation then on Exchange run the Exchange PowerShell command "Set-SenderIDConfig -Enabled $false". No point running senderid checks against connections from your EFA box.

I haven't mentioned LDAP lookups here. I haven't configured it as we (as in IT) manage the EFA box as well as blocking/releasing emails and training SA.

Now, to "integrate" EFA into Exchange it's vital you;
Log into ECP
  • mail flow > rules
  • New rule. My rule is unimaginatively called "Tag 'X-Spam-Status: Yes' emails as SCL 8".
  • Apply this rule if... A message header matches "X-Spam-Status" header matches "Yes".
  • Do the following... Set the spam confidence level (SCL) to... 8.
  • Enforce.
  • Defer the message if the rule processing doesn't complete.
  • Audit... I wouldn't bother to audit.
Order the rule however you like, but make sure it's above any rules that stop processing any rules under it.
Exchange PowerShell

Code: Select all

Set-ContentFilterConfig -SCLRejectThreshold 9 -SCLRejectEnabled $false -SCLDeleteEnabled $false -SCLQuarantineEnabled $false
-- You can mess with these if you like, but if you set something to 8 then the rule above (and the command below) needs to be 7, and so on.

Code: Select all

Set-OrganizationConfig SCLJunkThreshold 8
In a nutshell EFA will deliver suspect spam emails with a header of "X-Spam-Status: Yes". The Exchange rule then gives these emails a SCL confidence of 8. The organisation config then sees this SCL of 8 and puts the email into the user's "Junk E-mail" folder.

That's it. If I can read my notes properly that should have EFA filtering for and integrating into Exchange.

Besides getting the actual configuration right the most important thing to do is train Bayes. viewtopic.php?f=5&t=2400&p=10016#p10016 should help. Without training Bayes, and this spamassassin, is going to be fairly useless.

Good luck!

fatima2007
Posts: 19
Joined: 03 Jan 2018 11:14

Re: exchange server and EFA setting

Post by fatima2007 » 14 May 2018 06:30

excuse me. i have dag with 2 exchnage server. which setting in transport setting should i do. is this setting correct and enough to transfer traffic to mail servers? efa-configure >> mail setting >> transport setting>>add new domain

172.16.5.72 is my first exchange server
172.16.5.74 is my second exchange server


mydomain.com 172.16.5.72
mydomain.com 172.16.5.74

is it correct? which policy is used to send traffic to mail servers? round robin? does it have priority?

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 14 May 2018 09:46

Ah, no it's not correct. You can only configure one destination per domain.

So you'll need to set mydomain.com to either 172.16.5.72 or 172.16.5.72. This obviously will give you no automatic failover, and if you've if you've got a lot of domains in EFA it's a pain to reconfigure (unless you drop to shell and edit /etc/postfix/transport directly).

What I've done is installed a pair of (free!) network load balancers which serve as the connection point for incoming HTTPS and SMTP connections. You can point EFA to the NLB IP and that'll handle distributing the load however you see fit.

Load balancers might be a bit much. Alternatively you can do a basic DNS failover.

Configure EFA to use your internal DNS server(s). Configure a DNS entry to resolve to 172.16.5.72 (e.g. smtp-in.domain.local). Then configure EFA to use "smtp-in.domain.local" for all domains. If 172.16.5.72 ever falls down then reconfigure the DNS entry to resolve to 172.16.5.74.

You can also do basic load balancing by having DNS RR. Add A records for "smtp-in.domain.local" for both 172.16.5.72 and 172.16.5.74. If one server ever fails then remove the A record for the failed one.

Personally I have all incoming SMTP delivery going to my "primary" Exchange server. Makes tracing emails a lot easier and faster.

User avatar
pdwalker
Posts: 1135
Joined: 18 Mar 2015 09:16

Re: exchange server and EFA setting

Post by pdwalker » 15 May 2018 06:07

Another thing to consider is how your exchange servers are configured. For example, do they serve separate domains, or do they serve the same domains? Do they know how to route mail to the other server if necessary?

fatima2007
Posts: 19
Joined: 03 Jan 2018 11:14

Re: exchange server and EFA setting

Post by fatima2007 » 19 May 2018 07:36

jase72 wrote:
14 May 2018 09:46
Ah, no it's not correct. You can only configure one destination per domain.

So you'll need to set mydomain.com to either 172.16.5.72 or 172.16.5.72. This obviously will give you no automatic failover, and if you've if you've got a lot of domains in EFA it's a pain to reconfigure (unless you drop to shell and edit /etc/postfix/transport directly).

What I've done is installed a pair of (free!) network load balancers which serve as the connection point for incoming HTTPS and SMTP connections. You can point EFA to the NLB IP and that'll handle distributing the load however you see fit.

Load balancers might be a bit much. Alternatively you can do a basic DNS failover.

Configure EFA to use your internal DNS server(s). Configure a DNS entry to resolve to 172.16.5.72 (e.g. smtp-in.domain.local). Then configure EFA to use "smtp-in.domain.local" for all domains. If 172.16.5.72 ever falls down then reconfigure the DNS entry to resolve to 172.16.5.74.

You can also do basic load balancing by having DNS RR. Add A records for "smtp-in.domain.local" for both 172.16.5.72 and 172.16.5.74. If one server ever fails then remove the A record for the failed one.

Personally I have all incoming SMTP delivery going to my "primary" Exchange server. Makes tracing emails a lot easier and faster.


you mean it is just enough to do this: in my internal dns i set a record mail.mydomain.com with 172.16.5.72 and mail.mydomain.com with 172.16.5.74 ?? and it is not necessary to change configuration in efa ???? yes ?

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 19 May 2018 11:45

That sounds like you're proposing to use your real domain for the DNS entry. Normally you wouldn't do that, you'd create something completely separate.

If "mydomain.com" is your real domain then I'd be very careful about creating local mail.mydomain.com records, unless the server you're editing is the authoritative name server then doing so will create split DNS.

Generally you'll use a local domain, meaning something that only resolves internally. In Windows land this would be the "mydomain.local" domain, so you'd create mail.mydomain.local.

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 11 Aug 2018 22:20

dag dual mta idea.gif
dag dual mta idea.gif (215.42 KiB) Viewed 1621 times

Hi,

I have no experience using Exchange in DAG configuration, but I would like to test a geographic failover configuration using 2 server far from each other.


Do you think it will possible to realize a configuration as designed in the draw?

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 11 Aug 2018 22:31

It is also possible to configure a "failover configuration" in EFA so that a domain can be relayed in 2 different server (enchange server in that case) when the first target doesn't respond?

1) EFAMTA01 send email to EXCHANGE01 that has been setup as the main target for relaying
2) Failure occur in EXCHANGE01 so that it doesn't respond to EFAMTA01
3) EFAMTA01 after a timeout route the email in cache to EXCHANGE02

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 13 Aug 2018 15:25

Getting way beyond EFA here, but yes. Few suggestions/pointers/comments;
  • Exchange 02 will only ever be good as a DR box. It'll never work for production, if the link goes down the databases will dismount.
    It's not a bad thing, that's what my DR server is for.
  • I'd make 3 MX records, the highest and lowest being EFA MTA 01. Spammers will often hit the lowest record. If the MTA 02 only ever sees spam it'll get so jaded about emails that if you ever fail over to it spamassassin will probably think everything's spam (note I'm not a SA expert but I believe that's what can happen).
  • As you've got an even number of Exchange boxes you'll have a witness server somewhere for resolving deadlocks, which you'll have to put in site 01. Whichever site has the witness server will get the majority vote (2 of 3) if the link or VPN ever goes down and that's the site that will remain operational.
  • You need a DC in site 02.
  • You can have issues with witness servers on DCs (and it's strongly recommended against) so you'll need something that can do file shares in site 01 as the witness server, and the same for site 02 if you ever want to be able to fail over to it.
  • Speaking of failing over, if you lose site 01 and you need to fail over to site 02 you'll need to update Exchange to use a different witness server in site 02, then you'll have majority vote for site 02.
    Extra tip: Don't think that "alternate witness server" will automatically cover this for you, it actually does nothing until you manually fail over.
Anyway as you can see mostly Exchange issues, not EFA. EFA will work fine, any emails that flow into site 02 will get routed back through the VPN so it can be written to the active database in site 01.

As for your other questions;
Gate Array wrote:
11 Aug 2018 22:31
It is also possible to configure a "failover configuration" in EFA so that a domain can be relayed in 2 different server (enchange server in that case) when the first target doesn't respond?
Not in EFA, no. You can set the relay server for your domains to be a DNS entry and can then edit that DNS entry if you ever need to change it (rather than possibly having to edit dozens of domains in EFA), or the best solution is to use a network load balancer, something like Zevenet load balancer (the free one is fine). With the load balancer you can then set a preferred server and then a failover.

I use two HA Zevent servers to do that and it works brilliantly. Can't remember the last time I did anything with them.

Good luck!

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 14 Aug 2018 13:30

Thank for the answer.

You are talking about a "DR" configuration with a "master/slave" configuration or "primary/secondary" approach...

I would like to get an active/active configuration with online DB replica/sync so that theoretically SITE01 = SITE02 in term of functionality.

Is that possible?
Attachments
dag dual mta idea 2.gif
dag dual mta idea 2.gif (170.35 KiB) Viewed 1546 times

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 15 Aug 2018 02:02

In short it's not possible with your configuration.

I'm assuming Exchange 02 will have a passive copy of the active database on Exchange 01, and Exchange 01 will have a passive copy of the active database on Exchange 02. Users in site 01 will connect to Exchange 01 and users in site 02 will connect to Exchange 02.

With a DAG there's constantly health checks going on between members to figure out the status of the servers and databases. If the private link goes down Exchange 01 can't contact Exchange 02 (and visa-versa) so what does each server do about their passive copies (and their active copies)? Basically they hold a vote to see which site is deemed healthy, which is where the aforementioned witness server comes in. Both servers will have a vote each so it'll be a deadlock, the witness server has the deciding vote on which site is healthy, and assuming it is located in site 01 then site 01 will be deemed healthy. All databases will become active on Exchange 01. Exchange 02, only having 1 out of 3 votes in its site, will declare itself bad and make all databases passive and unavailable.

You'll actually need two certificates for Outlook Anywhere (one for each site). Autodiscover DNS will be configured to hit one of your Exchange servers (presumably 01) and it'll then refer the client to itself or the DNS entry associated with Exchange 02 if the active copy is over there.

I've deliberately always avoided the situation you're proposing so I might be a bit off for this next bit... but if you've got that configured properly then if Exchange 02 goes offline and their copies become active on Exchange 01 the users in site 02 will get updated autodiscover information pointing them to the Outlook Anywhere address for site 01 and they'll eventually come back online. Once the link comes back you can then reactivate the database on Exchange 02 and the users will then start talking back to the local Exchange server.

But as you can see a minor link hiccup can lead to a lot of annoyance. Even communication delays can lead to Exchange doing pleasant things like deciding the cluster isn't healthy (which a DAG uses in the backend) and rebooting the entire server. I love that one. Especially as it does it via a BSOD.

If you truly want active/active that'll survive private link issues then the simple thing is to remove the DAG component. If you want the HA and active/active then it'll probably be four Exchange boxes, two DAGs, two witness servers (one in each site).

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 24 Aug 2018 10:08

Hi,

I finally got that my scenario with "active-active" database is not available in DAG exchange configuration.

"Within a database availability group (DAG) each database can have one active copy, and up to 15 passive copies at any given time. Changes that occur in the active database copy are replicated to the passive copies by a process of continuous replication. The active copy can be dismounted, and one of the passive copies mounted to become the active copy, when a switchover or failover occurs.

In a split brain scenario, two copies of a database would be active at the same time, mounted on DAG members that are unable to communicate with each other due to a network problem, causing the databases to diverge. This is a situation that must be avoided, because it creates a difficult recovery scenario and will likely result in data loss
."

Anyway I started a test ambience to understand the right solution.

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 24 Aug 2018 10:23

I still thinking how to use EFA in a dual (or multiple) site .... not easy....

jase72
Posts: 20
Joined: 21 Jul 2017 09:06

Re: exchange server and EFA setting

Post by jase72 » 25 Aug 2018 04:23

Simplify it. Take out the DAG component (leaving you with two standard Exchange boxes, one in each site with one database each) and I'd also ditch the HA appliances, I can't really see how that'll help.

Leave the EFA boxes in each site delivering to the local Exchange server. Exchange will know where the recipient's mailbox is homed and push the mail delivery to that box (via your private link if it's a server in another site).

DAG will give you HA but it sounds like you just need an active database in all sites.

Gate Array
Posts: 23
Joined: 30 Aug 2017 09:36

Re: exchange server and EFA setting

Post by Gate Array » 25 Aug 2018 12:14

mmmmm

The problem is the cache in EFA...

If the exchange go down, EFA will continue to accept email from SMTP server.

The right functionality should be this one (in the pitcture)
Attachments
itpro_transport_shadowmessagingoverview.gif
itpro_transport_shadowmessagingoverview.gif (34.89 KiB) Viewed 1330 times

Post Reply