Tuning filters

Questions and answers about how to do stuff
Post Reply
csum77
Posts: 9
Joined: 26 Dec 2014 01:41

Tuning filters

Post by csum77 » 15 Apr 2015 04:30

Hi all,

When I first set up EFA last year we noticed a significant drop in the spam we were receiving. However, in the last month or two people are complaining about the volume of spam making it through to their mailboxes. Then, oddly, about two weeks ago I started getting hit with heavy spam in my inbox that has been persistent. I'm probably getting 20-30 messages each day that are spam. I haven't signed up for anything or put my email address out there anywhere in that time span. So it seems like we're getting hit heavily...at least more heavily than we used to be.

How should I go about tuning EFA to get more aggressive on spam? I've read up on tuning stand alone Spam Assassin, but I'm not really clear on if I should be modifying SA's config files directly, or if there is an "EFA Approved" way to tweak & tune. I know I can pull reports & run message operations to mark ones that are getting through as spam, but it's so heavy that it feels like I'm barely making a dent.

Any specific things you'd recommend I tune? And where is the proper place to make changes? Perhaps there is a How-To that I'm missing, but some quick searches didn't reveal it to me.

Thanks in advance for your help & feedback.

-Charlie

User avatar
pdwalker
Posts: 1255
Joined: 18 Mar 2015 09:16

Re: Tuning filters

Post by pdwalker » 15 Apr 2015 05:38

Quick question, are you training spam assassin with the new spam?

What are the scorings from spam assassin of the new spam?

DaN
Posts: 240
Joined: 19 Nov 2014 10:04
Location: Earth

Re: Tuning filters

Post by DaN » 15 Apr 2015 09:20

Good questions from pdwalker. If the answere for the first question is "yes" (you train spam assassin), the second answere could lead to:
shawniverson wrote:/etc/MailScanner/MailScanner.conf:

Code: Select all

# This replaces the SpamAssassin configuration value 'required_hits'.
# If a message achieves a SpamAssassin score higher than this value,
# it is spam. See also the High SpamAssassin Score configuration option.
# This can also be the filename of a ruleset, so the SpamAssassin
# required_hits value can be set to different values for different messages.
Required SpamAssassin Score = 3

# If a message achieves a SpamAssassin score higher than this value,
# then the "High Scoring Spam Actions" are used. You may want to use
# this to deliver moderate scores, while deleting very high scoring messsages.
# This can also be the filename of a ruleset.
High SpamAssassin Score = 7
If the answere for the first question is "no" you should begin to train SA!

User avatar
darky83
Site Admin
Posts: 537
Joined: 30 Sep 2012 11:03
Location: eFa
Contact:

Re: Tuning filters

Post by darky83 » 15 Apr 2015 12:02

Also which DNS servers are you using?

If you can send an screenshot from the spam that is getting through (EFA status page) that would also be helpfull (if you don't want to disclose it to the public just PM me)
Version eFa 4.x now available!

csum77
Posts: 9
Joined: 26 Dec 2014 01:41

Re: Tuning filters

Post by csum77 » 15 Apr 2015 15:10

Thank you both for the quick replies! I really appreciate it.

Shortly after posting this last night I was digging through the EFA web admin and I noticed on the Spam Assassin Rule Hits report that I'm getting this:

URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/Dns ... nsbl-block for more information.

So, I've signed up for URIBL's datafeed & I'm setting that up now. I'm guessing that probably has a lot to do with it. Is there a guide or How To on what configuration changes I need to make to use their Datafeed over DNS service?

Additionally, when it comes to training SA, I need to spend more time going through the Message Operations report & marking Spam as such.

Thanks again! I'll keep you posted as to how this works.

-Charlie

User avatar
pdwalker
Posts: 1255
Joined: 18 Mar 2015 09:16

Re: Tuning filters

Post by pdwalker » 15 Apr 2015 18:06

Charlie,

Two things:

1/ There are a couple (few? 1? many?) of DNS block lists that Spam Assassin checks. If you use your ISPs DNS server, chances are, that IP will have gone over quota for free usage.

IF you setup your own internal Caching DNS nameserver for use by EFA, you'll probably get faster responses, and your IP will be less likely blocked from the free services.

There is some documentation on that, perhaps in these forums. It's one of the first problems I had to deal with with my efa installation.

2/ For messages that are making it past spam, look at the message details. You'll find two lines on the left, SpamAssassin Score and SpamAssassin Spam Report.

The score will tell you how "spammy" the message is, and the report will tell you which rules were triggered. You'll want to look at the details in case you find some rules you don't agree with. I had to make some minor tweaks to stop legitimate mail has being marked as spam.

Here's a report from a "high" spam message as an example.

Code: Select all

SpamAssassin Score: 11.88
Spam Report:
Score   Matching Rule                 Description
3.50   BAYES_99	                    Bayes spam probability is 99 to 100%
0.20   BAYES_999                      Bayes spam probability is 99.9 to 100%
1.10   DCC_CHECK                      Detected as bulk mail by DCC (dcc-servers.net)
0.29   DIGEST_MULTIPLE                Message hits more than one network digest check
0.10   DKIM_SIGNED                    Message has a DKIM or DK signature, not necessarily valid
0.00   HEADER_FROM_DIFFERENT_DOMAINS	 
1.09   HTML_IMAGE_ONLY_16             HTML: images with 1200-1600 bytes of words
0.00   HTML_MESSAGE                   HTML included in message
0.00   LOTS_OF_MONEY
0.50   RAZOR2_CF_RANGE_51_100         Razor2 gives confidence level above 50%
1.89   RAZOR2_CF_RANGE_E8_51_100      Razor2 gives engine 8 confidence level above 50%
0.92   RAZOR2_CHECK                   Listed in Razor2 (http://razor.sf.net/)
-0.00  SPF_HELO_PASS                  SPF: HELO matches SPF record
0.67   SPF_SOFTFAIL                   SPF: sender does not match SPF record (softfail)
0.01   T_DKIM_INVALID	 
1.61   URIBL_WS_SURBL                 Contains an URL listed in the WS SURBL blocklist
The message details can be seen by clicking on "Recent Messages" and clicking on the space between the [ ] characters on the leftmost column.

User avatar
darky83
Site Admin
Posts: 537
Joined: 30 Sep 2012 11:03
Location: eFa
Contact:

Re: Tuning filters

Post by darky83 » 16 Apr 2015 07:01

Yea I never use public DNS servers (google, opendns, from ISP etc..) so never had the issue before but some users have pointed out before that using an public DNS server for your query's might give you the URIBL_BLOCKED messages.

See Michaelv's post: viewtopic.php?t=934

We are looking into using an resolving DNS server on the E.F.A. image itself using unbound or bind and don't give the users the option to use forwarding DNS servers anymore. (but not all people might like this so...)
Version eFa 4.x now available!

csum77
Posts: 9
Joined: 26 Dec 2014 01:41

Re: Tuning filters

Post by csum77 » 29 Aug 2015 03:18

Hi all! Sorry to revive an old thread, but I handed this off to someone else and they've been struggling & not really getting anywhere.

Per the suggestions, I did check the DNS blocks, and made some changes to address it, but that hasn't really helped improve the situation. However, just recently we found something that I wonder if it isn't part of the problem. When we go to the Bayes Database Information page in EFA it says our Last Journal Sync was Wed, Dec 31 1969!

I'm searching the forums to try to find information on this, but I thought I'd update with this info as I'm guessing it's related.

If anyone has insight into this, or can point me in a direction to help fix it, that would be greatly appreciated.

Thanks!

-Charlie

User avatar
shawniverson
Posts: 3087
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Tuning filters

Post by shawniverson » 29 Aug 2015 05:10

This is normal.

The journal isn't enabled by default. It is useful in high traffic situations to reduce contention on the bayes database.
Version eFa 4.0.2 now available!

csum77
Posts: 9
Joined: 26 Dec 2014 01:41

Re: Tuning filters

Post by csum77 » 29 Aug 2015 05:35

Got it. Thanks for the clarification.

Back to the drawing board...

User avatar
pdwalker
Posts: 1255
Joined: 18 Mar 2015 09:16

Re: Tuning filters

Post by pdwalker » 31 Aug 2015 06:47

Perhaps you can tell us a bit more about what problem you are actually having?

Post Reply