SPF problems?

Questions and answers about how to do stuff
Post Reply
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

SPF problems?

Post by stusmith » 14 Mar 2017 13:30

Does anyone else have the problem that almost all of the small companies that they work with have terrible SPF records? The number one problem I have with the EFA at the moment ( in terms of e-mail being blocked ) is that I see so many terrible SPF records. Things like ?all. or -all" or two DNS TXT records for a domain or sixteen different includes.

It's bad enough that I've written custom SpamAssassin rules to reverse the SPF validation results for exceeding void lookups, too many txt records, and now unknown mechanisms.

How have others addressed this problem?

stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: SPF problems?

Post by stusmith » 14 Mar 2017 13:33

/etc/mail/spamassassin/local.cf

Code: Select all

#       ---------Handle Broken SPF----------
header __SPF_FF_PERMERROR Authentication-Results =~ /spf=permerror/i
header __SPF_FF_TOOMANYRECORDS Authentication-Results =~ /two or more type TXT spf records found/i
header __SPF_FF_TOOMANYLOOKUPS Authentication-Results =~ /too many DNS lookups/i
header __SPF_FF_VOIDLOOKUPS Authentication-Results =~ /Void lookup limit of . exceeded/i
header __SPF_FF_UNKNOWN_MECHANISM Authentication-Results =~ /Unknown mechanism/i
meta FF_SPF_MISCONFIGURATION ((__SPF_FF_PERMERROR + __SPF_FF_TOOMANYRECORDS + __SPF_FF_TOOMANYLOOKUPS + __SPF_FF_VOIDLOOKUPS + __SPF_FF_UNKNOWN_MECHANISM) > 1)
describe FF_SPF_MISCONFIGURATION Everyone makes my life difficult
score FF_SPF_MISCONFIGURATION -4.0

User avatar
shawniverson
Posts: 2880
Joined: 13 Jan 2014 23:30
Location: Rushville, Indiana, USA
Contact:

Re: SPF problems?

Post by shawniverson » 14 Mar 2017 13:47

Alas, the joys of bad SPF records. Really not an eFa specific problem, as a bad or malformed SPF is *supposed* to be a indicator of something fishy. But yes, too many organizations stink at writing good SPF records and/or fail to maintain them, meaning you have to do silly things to let the mail through if you need it from those organizations.

I would be interested in incorporating some of these rules into the next eFa. We have a lot of good things in the forum for various SA rules, and it would be cool to have options you can enable and disable from the GUI. I especially like your description "Everyone makes my life difficult" :lol:
Version eFa 4.0.0 now available!

stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: SPF problems?

Post by stusmith » 14 Mar 2017 16:03

shawniverson wrote:
14 Mar 2017 13:47
Alas, the joys of bad SPF records. Really not an eFa specific problem, as a bad or malformed SPF is *supposed* to be a indicator of something fishy. But yes, too many organizations stink at writing good SPF records and/or fail to maintain them, meaning you have to do silly things to let the mail through if you need it from those organizations.

I would be interested in incorporating some of these rules into the next eFa. We have a lot of good things in the forum for various SA rules, and it would be cool to have options you can enable and disable from the GUI. I especially like your description "Everyone makes my life difficult" :lol:
Glad you got a kick out of it! :D

I've been writing domain specific spam assassin rules to counterbalance the SPF records for 'poorly maintained' organizations. For example, when the MX record is in place and corresponds to the SMTP host but the SPF record is off in space. That would be useful to add to the GUI since it seems to be the most common scenario. I imagine that it wouldn't be too difficult to add since the [ Add to Whitelist | Add to Blacklist ] functionality is already next to the From: field, so I'd have something to work from.

In related news, I'm considering campaigning for sainthood and automating an unsubscribe e-mail option. You know, coupled with a spam score. :dance: So I guess I'd have to write a perl or python script to implement a custom SpamAssassin action? I'm still looking into it, but no matter what it takes? Probably worth it.

Since putting the EFA in, I've got our SPAM down to under 5%, but unsubscribing everyone by hand from all the e-mail lists that show up that they don't want to see is super painful. I wish I could just use a header check to reject them, but we have business services that contain the unsubscribe link -- which is a good thing! -- but which prevents that solution. So I use SpamAssassin rules to match the List-Unsubscribe header and then write rules to allow the specific notifications through. Slow and painful, but if I could put that into the GUI then maybe it would be beneficial.

User avatar
darky83
Site Admin
Posts: 531
Joined: 30 Sep 2012 11:03
Location: eFa
Contact:

Re: SPF problems?

Post by darky83 » 18 Mar 2017 10:19

Generaly there is hardly anything you can do as it is up to the senders mail admin, I try not to fix items in eFa but try to point mail admins to the SPF syntax page if their mail is marked as spam, and I instruct my users to call the sender and tell them their mails are not received because they have a badly configured mailserver.

The next problem you mostly see after that is that the admins change their spf records to end with an +all... :doh:
Version eFa 4.0.0 now available!

stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: SPF problems?

Post by stusmith » 23 Mar 2017 15:18

darky83 wrote:
18 Mar 2017 10:19
Generaly there is hardly anything you can do as it is up to the senders mail admin, I try not to fix items in eFa but try to point mail admins to the SPF syntax page if their mail is marked as spam, and I instruct my users to call the sender and tell them their mails are not received because they have a badly configured mailserver.

The next problem you mostly see after that is that the admins change their spf records to end with an +all... :doh:
Good advice. I've been reaching out to organizations that we deal with - finding contact information on their websites or calling and asking to speak to their helpdesk. So far, I've actually received some "thank you's" as well as follow up calls from some state organizations. I doubt that my luck will hold, but you never know. At least I've managed to have very polite and pleasant interactions thus far. It's a lot more work than just writing a rule to work around the problems, but I guess you've got to "be the change."

Yeeesssss... neutral SPF. Like screen doors on a submarine.

+all -- /shudders involuntarily

Post Reply