SQLGrey and excessive delay receiving email from Office365, etc.

[dbrunt]

We've enabled greylisting but have noticed that emails from Office365 and others are getting delayed for hours or days. The problem occurs when the retry comes from different server IP, after different server IP, after different server IP. Whitelisting the sender domain works but only for known clients. If you receive email from a new customer, it's not nice to ignore their request to purchase something for hours or days. 2 customers of ours had to have greylisting disabled as they were losing business.

Code: Select all

Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0121.outbound.protection.outlook.com [104.47.33.121])

What is best way to not greylist anything from *.outlook.com? Their server IP list is very large!

[shawniverson]

sqlgrey is not suitable in this situation. You may want to consider disabling greylisting or looking into postscreen instead.

[ovizii]

are you sure sqlgrey is not suitable to whitelist i.e. outbound.protection.outlook.com?

as far as I can see you have the options of whitelisting with /etc/sqlgrey/clients_fqdn_whitelist.local (see /etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.

If you like you could add IP ranges too: /etc/sqlgrey/clients_ip_whitelist.local

and /etc/sqlgrey/discrimination.regexp for more discrimination

[shawniverson]

Possibly, although I have not tried it since I have not had this particular situation.

[dbrunt]

The number of people using Office 365 is growing exponentially! Other sources of grief for SQLGrey are any Cloud email security solution like Symantec Cloud, McAfee/Intel's MXLogic (soon to retire), Barracuda, etc. where users route outbound email through their Cloud solution. MXLogic alone has 208.65.144.0/21 and 208.81.64.0/22 for mail servers.

Thanks for the heads-up on these options of whitelisting with /etc/sqlgrey/clients_fqdn_whitelist.local and via IP ranges in
/etc/sqlgrey/clients_ip_whitelist.local.

[cdburgess75]

dbrunt, yeah agreed, office365 has crippled sqlgrey. Here is a list of "Exchange Online Protection IP addresses" They hop around like rabbits when initiating connections.

23.103.132.0/22
23.103.136.0/21
23.103.144.0/20
23.103.198.0/23
23.103.200.0/21
40.92.0.0/14
40.107.0.0/16
65.55.88.0/24
65.55.169.0/24
94.245.120.64/26
104.47.0.0/17
134.170.101.0/24
134.170.140.0/24
134.170.171.0/24
157.55.133.0/25
157.56.87.192/26
157.56.110.0/23
157.56.112.0/24
157.56.116.0/25
157.56.120.0/25
207.46.51.64/26
207.46.100.0/24
207.46.108.0/25
207.46.163.0/24
213.199.154.0/24
213.199.180.128/26
216.32.180.0/23


2a01:111:f400:7c00::/54
2a01:111:f400:fc00::/54

[ovizii]

I'm still waiting for someone to confirm if it isn't easier doing it this way:

/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.

rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.

[dbrunt]

Here is Symantec Cloud Security Services IP ranges:
http://images.messagelabs.com/EmailReso ... net_IP.pdf

216.82.240.0/20
216.82.240
216.82.241
216.82.242
216.82.243
216.82.244
216.82.245
216.82.246
216.82.247
216.82.248
216.82.249
216.82.250
216.82.251
216.82.252
216.82.253
216.82.254
216.82.255

67.216.240.0/20
67.219.240
67.219.241
67.219.242
67.219.243
67.219.244
67.219.245
67.219.246
67.219.247
67.219.248
67.219.240
67.219.250
67.219.251
67.219.252
67.219.253
67.219.254
67.219.255

85.158.136.0
85.158.136
85.158.137
85.158.138
85.158.139
85.158.140
85.158.141
85.158.142
85.158.143

95.131.104.0/21
95.131.104
95.131.105
95.131.106
95.131.107
95.131.108
95.131.109
95.131.110
95.131.111

46.226.48.0/21
46.226.48
46.226.49
46.226.50
46.226.51
46.226.52
46.226.53
46.226.54
46.226.55

117.120.16.0/21
117.120.16
117.120.17
117.120.18
117.120.19
117.120.20
117.120.21
117.120.22
117.120.23

103.9.96.0/22
103.9.96
103.9.97
103.9.98
103.9.99

193.109.254.0/23
193.109.254
193.109.255

194.106.220.0/23
194.106.220
194.106.221

195.245.230.0/23
195.245.230
195.245.231

** Edit **
Instead of adding all of these IP's, add *.messageslabs.com to /etc/sqlgrey/clients_fqdn_whitelist.local

[dbrunt]

I'm still waiting for someone to confirm if it isn't easier doing it this way:

/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.

rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.

I've added outbound.protection.outlook.com and *.outbound.protection.outlook.com and will see what happens...

[dbrunt]

# Barracuda:
64.235.144
64.235.145
64.235.146
64.235.147
64.235.148
64.235.149
64.235.150
64.235.151
64.235.152
64.235.153
64.235.154
64.235.155
64.235.156
64.235.157
64.235.158
64.235.159

[dbrunt]

I'm still waiting for someone to confirm if it isn't easier doing it this way:

/etc/sqlgrey/clients_fqdn_whitelist for examples) where you can add outbound.protection.outlook.com and *.outbound.protection.outlook.com as far as I can see.

rather than via that huge list of IPs which can and will will grow eventually. I don't get enough traffic to be able to reliably test this.

I've added outbound.protection.outlook.com and *.outbound.protection.outlook.com and will see what happens...

Confirmed.

After adding those two entries, we received an email which had previously been auto-whitelisted by SQLGrey:


The new email header now has this:


This would indicate to me that the outbound.protection.outlook.com entry kicked in...

[dbrunt]

However a new (possible) issue I'm seeing now is most emails from outbound.protection.outlook.com without the X-Greylist: header meaning SQLGrey did not process the email?

These messages do not have X-Greylist: header:


These ones do:


This particular appliance is 3.0.0.8

[dbrunt]

I just ran this command and look what's been added to the main file:

[root@efa sqlgrey]# update_sqlgrey_config
updating /etc/sqlgrey/clients_fqdn_whitelist:
--- /etc/sqlgrey/clients_fqdn_whitelist 2015-02-26 18:45:56.317999767 -0800
+++ clients_fqdn_whitelist 2016-06-27 08:02:37.000000000 -0700
@@ -100,6 +100,14 @@
# GL-group: no retry
mail.gl-group.com

+# StartSSL: no retry
+*.startcom.org
+*.startssl.com
+
[b]+# Outlook.com users, retries do not come from the same server.
+*.outbound.protection.outlook.com
[/b]+
+
# Do not add anything here (this file can be overwritten by SQLgrey updates and
# update_sqlgrey_config), create a "clients_fqdn_whitelist.local" file
# and add your own entries in there
updating /etc/sqlgrey/smtp_server.regexp:
--- /etc/sqlgrey/smtp_server.regexp 2015-02-26 18:45:56.422999767 -0800
+++ smtp_server.regexp 2005-03-01 16:29:45.000000000 -0800
@@ -1 +1 @@
-^(.+[._-])*(apache|bounce|bulk|delay|d?ns|external|extranet|filter|firewall|forward|gateway|gw|m?liste?s?|(bulk|dead|mass|send|[eqw])?mail(er)?|e?mail(agent|host|hub|scan(ner)?)|messagerie|mta|v?mx|out(bound)?|pop|postfix|w?proxy|rela(is|y)|serveu?r|smarthost|v?smtp|web|www)(gate|mail|mx|pool|out|server)?[0-9]*[._-]
\ No newline at end of file
+^(.+[._-])*(apache|bounce|bulk|delay|d?ns|external|extranet|filter|firewall|forward|gateway|gw|m?liste?s?|(bulk|dead|mass|send|[eqw])?mail(er)?|e?mail(agent|host|hub|scan(ner)?)|messagerie|mta|v?mx|out(bound)?|pop|postfix|w?proxy|rela(is|y)|serveu?r|smarthost|v?smtp|web|www)(gate|mail|mx|pool|out|server)?[0-9]*[._-]
[root@efa sqlgrey]#

[ovizii]

hehe, seems we were on the right track

[dbrunt]

hehe, seems we were on the right track

Yes it would seem so.
So instead of adding all of Symantec's & MXLogic's IPs, I've added *.messageLabs.com and *.MXLogic.net to /etc/sqlgrey/clients_fqdn_whitelist.local