SQLGrey and excessive delay receiving email from Office365, etc.

Report bugs and workarounds
Post Reply
dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 06 Jul 2016 18:21

We've enabled greylisting but have noticed that emails from Office365 and others are getting delayed for hours or days. The problem occurs when the retry comes from different server IP, after different server IP, after different server IP. Whitelisting the sender domain works but only for known clients. If you receive email from a new customer, it's not nice to ignore their request to purchase something for hours or days. 2 customers of ours had to have greylisting disabled as they were losing business.

Code: Select all

Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0121.outbound.protection.outlook.com [104.47.33.121])
What is best way to not greylist anything from *.outlook.com? Their server IP list is very large!

User avatar
shawniverson
Posts: 3104
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by shawniverson » 06 Jul 2016 22:14

sqlgrey is not suitable in this situation. You may want to consider disabling greylisting or looking into postscreen instead.
Version eFa 4.0.2 now available!

ovizii
Posts: 463
Joined: 11 May 2016 08:08

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by ovizii » 13 Jul 2016 13:59

are you sure sqlgrey is not suitable to whitelist i.e. outbound.protection.outlook.com?

as far as I can see you have the options of whitelisting with /etc/sqlgrey/clients_fqdn_whitelist.local (see /etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.

If you like you could add IP ranges too: /etc/sqlgrey/clients_ip_whitelist.local

and /etc/sqlgrey/discrimination.regexp for more discrimination :-)

User avatar
shawniverson
Posts: 3104
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by shawniverson » 15 Jul 2016 16:50

Possibly, although I have not tried it since I have not had this particular situation.
Version eFa 4.0.2 now available!

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 22 Jul 2016 21:49

The number of people using Office 365 is growing exponentially! Other sources of grief for SQLGrey are any Cloud email security solution like Symantec Cloud, McAfee/Intel's MXLogic (soon to retire), Barracuda, etc. where users route outbound email through their Cloud solution. MXLogic alone has 208.65.144.0/21 and 208.81.64.0/22 for mail servers.

Thanks for the heads-up on these options of whitelisting with /etc/sqlgrey/clients_fqdn_whitelist.local and via IP ranges in
/etc/sqlgrey/clients_ip_whitelist.local.

cdburgess75
Posts: 48
Joined: 11 Jun 2014 21:43

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by cdburgess75 » 28 Jul 2016 16:14

dbrunt, yeah agreed, office365 has crippled sqlgrey. Here is a list of "Exchange Online Protection IP addresses" They hop around like rabbits when initiating connections.

23.103.132.0/22
23.103.136.0/21
23.103.144.0/20
23.103.198.0/23
23.103.200.0/21
40.92.0.0/14
40.107.0.0/16
65.55.88.0/24
65.55.169.0/24
94.245.120.64/26
104.47.0.0/17
134.170.101.0/24
134.170.140.0/24
134.170.171.0/24
157.55.133.0/25
157.56.87.192/26
157.56.110.0/23
157.56.112.0/24
157.56.116.0/25
157.56.120.0/25
207.46.51.64/26
207.46.100.0/24
207.46.108.0/25
207.46.163.0/24
213.199.154.0/24
213.199.180.128/26
216.32.180.0/23


2a01:111:f400:7c00::/54
2a01:111:f400:fc00::/54

ovizii
Posts: 463
Joined: 11 May 2016 08:08

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by ovizii » 28 Jul 2016 16:19

I'm still waiting for someone to confirm if it isn't easier doing it this way:
/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.
rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 17:03

Here is Symantec Cloud Security Services IP ranges:
http://images.messagelabs.com/EmailReso ... net_IP.pdf

216.82.240.0/20
216.82.240
216.82.241
216.82.242
216.82.243
216.82.244
216.82.245
216.82.246
216.82.247
216.82.248
216.82.249
216.82.250
216.82.251
216.82.252
216.82.253
216.82.254
216.82.255

67.216.240.0/20
67.219.240
67.219.241
67.219.242
67.219.243
67.219.244
67.219.245
67.219.246
67.219.247
67.219.248
67.219.240
67.219.250
67.219.251
67.219.252
67.219.253
67.219.254
67.219.255

85.158.136.0
85.158.136
85.158.137
85.158.138
85.158.139
85.158.140
85.158.141
85.158.142
85.158.143

95.131.104.0/21
95.131.104
95.131.105
95.131.106
95.131.107
95.131.108
95.131.109
95.131.110
95.131.111

46.226.48.0/21
46.226.48
46.226.49
46.226.50
46.226.51
46.226.52
46.226.53
46.226.54
46.226.55

117.120.16.0/21
117.120.16
117.120.17
117.120.18
117.120.19
117.120.20
117.120.21
117.120.22
117.120.23

103.9.96.0/22
103.9.96
103.9.97
103.9.98
103.9.99

193.109.254.0/23
193.109.254
193.109.255

194.106.220.0/23
194.106.220
194.106.221

195.245.230.0/23
195.245.230
195.245.231

** Edit **
Instead of adding all of these IP's, add *.messageslabs.com to /etc/sqlgrey/clients_fqdn_whitelist.local
Last edited by dbrunt on 29 Jul 2016 16:18, edited 1 time in total.

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 17:12

ovizii wrote:I'm still waiting for someone to confirm if it isn't easier doing it this way:
/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.
rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.
I've added outbound.protection.outlook.com and *.outbound.protection.outlook.com and will see what happens...

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 17:44

# Barracuda:
64.235.144
64.235.145
64.235.146
64.235.147
64.235.148
64.235.149
64.235.150
64.235.151
64.235.152
64.235.153
64.235.154
64.235.155
64.235.156
64.235.157
64.235.158
64.235.159

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 21:22

dbrunt wrote:
ovizii wrote:I'm still waiting for someone to confirm if it isn't easier doing it this way:
/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.
rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.
I've added outbound.protection.outlook.com and *.outbound.protection.outlook.com and will see what happens...
Confirmed.

After adding those two entries, we received an email which had previously been auto-whitelisted by SQLGrey:
Image

The new email header now has this:
Image

This would indicate to me that the outbound.protection.outlook.com entry kicked in...
Last edited by dbrunt on 28 Jul 2016 21:46, edited 1 time in total.

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 21:45

However a new (possible) issue I'm seeing now is most emails from outbound.protection.outlook.com without the X-Greylist: header meaning SQLGrey did not process the email?

These messages do not have X-Greylist: header:
Image

These ones do:
Image

This particular appliance is 3.0.0.8

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 28 Jul 2016 22:55

I just ran this command and look what's been added to the main file:

[root@efa sqlgrey]# update_sqlgrey_config
updating /etc/sqlgrey/clients_fqdn_whitelist:
--- /etc/sqlgrey/clients_fqdn_whitelist 2015-02-26 18:45:56.317999767 -0800
+++ clients_fqdn_whitelist 2016-06-27 08:02:37.000000000 -0700
@@ -100,6 +100,14 @@
# GL-group: no retry
mail.gl-group.com

+# StartSSL: no retry
+*.startcom.org
+*.startssl.com
+
[b]+# Outlook.com users, retries do not come from the same server.
+*.outbound.protection.outlook.com
[/b]+
+
# Do not add anything here (this file can be overwritten by SQLgrey updates and
# update_sqlgrey_config), create a "clients_fqdn_whitelist.local" file
# and add your own entries in there
updating /etc/sqlgrey/smtp_server.regexp:
--- /etc/sqlgrey/smtp_server.regexp 2015-02-26 18:45:56.422999767 -0800
+++ smtp_server.regexp 2005-03-01 16:29:45.000000000 -0800
@@ -1 +1 @@
-^(.+[._-])*(apache|bounce|bulk|delay|d?ns|external|extranet|filter|firewall|forward|gateway|gw|m?liste?s?|(bulk|dead|mass|send|[eqw])?mail(er)?|e?mail(agent|host|hub|scan(ner)?)|messagerie|mta|v?mx|out(bound)?|pop|postfix|w?proxy|rela(is|y)|serveu?r|smarthost|v?smtp|web|www)(gate|mail|mx|pool|out|server)?[0-9]*[._-]
\ No newline at end of file
+^(.+[._-])*(apache|bounce|bulk|delay|d?ns|external|extranet|filter|firewall|forward|gateway|gw|m?liste?s?|(bulk|dead|mass|send|[eqw])?mail(er)?|e?mail(agent|host|hub|scan(ner)?)|messagerie|mta|v?mx|out(bound)?|pop|postfix|w?proxy|rela(is|y)|serveu?r|smarthost|v?smtp|web|www)(gate|mail|mx|pool|out|server)?[0-9]*[._-]
[root@efa sqlgrey]#

ovizii
Posts: 463
Joined: 11 May 2016 08:08

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by ovizii » 29 Jul 2016 05:57

hehe, seems we were on the right track ;-)

dbrunt
Posts: 64
Joined: 28 Nov 2015 00:09

Re: SQLGrey and excessive delay receiving email from Office365, etc.

Post by dbrunt » 29 Jul 2016 16:19

ovizii wrote:hehe, seems we were on the right track ;-)
Yes it would seem so.
So instead of adding all of Symantec's & MXLogic's IPs, I've added *.messageLabs.com and *.MXLogic.net to /etc/sqlgrey/clients_fqdn_whitelist.local

Post Reply