Page 1 of 1

Tuning filters

Posted: 15 Apr 2015 04:30
by csum77
Hi all,

When I first set up EFA last year we noticed a significant drop in the spam we were receiving. However, in the last month or two people are complaining about the volume of spam making it through to their mailboxes. Then, oddly, about two weeks ago I started getting hit with heavy spam in my inbox that has been persistent. I'm probably getting 20-30 messages each day that are spam. I haven't signed up for anything or put my email address out there anywhere in that time span. So it seems like we're getting hit heavily...at least more heavily than we used to be.

How should I go about tuning EFA to get more aggressive on spam? I've read up on tuning stand alone Spam Assassin, but I'm not really clear on if I should be modifying SA's config files directly, or if there is an "EFA Approved" way to tweak & tune. I know I can pull reports & run message operations to mark ones that are getting through as spam, but it's so heavy that it feels like I'm barely making a dent.

Any specific things you'd recommend I tune? And where is the proper place to make changes? Perhaps there is a How-To that I'm missing, but some quick searches didn't reveal it to me.

Thanks in advance for your help & feedback.

-Charlie

Re: Tuning filters

Posted: 15 Apr 2015 05:38
by pdwalker
Quick question, are you training spam assassin with the new spam?

What are the scorings from spam assassin of the new spam?

Re: Tuning filters

Posted: 15 Apr 2015 09:20
by DaN
Good questions from pdwalker. If the answere for the first question is "yes" (you train spam assassin), the second answere could lead to:
shawniverson wrote:/etc/MailScanner/MailScanner.conf:

Code: Select all

# This replaces the SpamAssassin configuration value 'required_hits'.
# If a message achieves a SpamAssassin score higher than this value,
# it is spam. See also the High SpamAssassin Score configuration option.
# This can also be the filename of a ruleset, so the SpamAssassin
# required_hits value can be set to different values for different messages.
Required SpamAssassin Score = 3

# If a message achieves a SpamAssassin score higher than this value,
# then the "High Scoring Spam Actions" are used. You may want to use
# this to deliver moderate scores, while deleting very high scoring messsages.
# This can also be the filename of a ruleset.
High SpamAssassin Score = 7
If the answere for the first question is "no" you should begin to train SA!

Re: Tuning filters

Posted: 15 Apr 2015 12:02
by darky83
Also which DNS servers are you using?

If you can send an screenshot from the spam that is getting through (EFA status page) that would also be helpfull (if you don't want to disclose it to the public just PM me)

Re: Tuning filters

Posted: 15 Apr 2015 15:10
by csum77
Thank you both for the quick replies! I really appreciate it.

Shortly after posting this last night I was digging through the EFA web admin and I noticed on the Spam Assassin Rule Hits report that I'm getting this:

URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/Dns ... nsbl-block for more information.

So, I've signed up for URIBL's datafeed & I'm setting that up now. I'm guessing that probably has a lot to do with it. Is there a guide or How To on what configuration changes I need to make to use their Datafeed over DNS service?

Additionally, when it comes to training SA, I need to spend more time going through the Message Operations report & marking Spam as such.

Thanks again! I'll keep you posted as to how this works.

-Charlie

Re: Tuning filters

Posted: 15 Apr 2015 18:06
by pdwalker
Charlie,

Two things:

1/ There are a couple (few? 1? many?) of DNS block lists that Spam Assassin checks. If you use your ISPs DNS server, chances are, that IP will have gone over quota for free usage.

IF you setup your own internal Caching DNS nameserver for use by EFA, you'll probably get faster responses, and your IP will be less likely blocked from the free services.

There is some documentation on that, perhaps in these forums. It's one of the first problems I had to deal with with my efa installation.

2/ For messages that are making it past spam, look at the message details. You'll find two lines on the left, SpamAssassin Score and SpamAssassin Spam Report.

The score will tell you how "spammy" the message is, and the report will tell you which rules were triggered. You'll want to look at the details in case you find some rules you don't agree with. I had to make some minor tweaks to stop legitimate mail has being marked as spam.

Here's a report from a "high" spam message as an example.

Code: Select all

SpamAssassin Score: 11.88
Spam Report:
Score   Matching Rule                 Description
3.50   BAYES_99	                    Bayes spam probability is 99 to 100%
0.20   BAYES_999                      Bayes spam probability is 99.9 to 100%
1.10   DCC_CHECK                      Detected as bulk mail by DCC (dcc-servers.net)
0.29   DIGEST_MULTIPLE                Message hits more than one network digest check
0.10   DKIM_SIGNED                    Message has a DKIM or DK signature, not necessarily valid
0.00   HEADER_FROM_DIFFERENT_DOMAINS	 
1.09   HTML_IMAGE_ONLY_16             HTML: images with 1200-1600 bytes of words
0.00   HTML_MESSAGE                   HTML included in message
0.00   LOTS_OF_MONEY
0.50   RAZOR2_CF_RANGE_51_100         Razor2 gives confidence level above 50%
1.89   RAZOR2_CF_RANGE_E8_51_100      Razor2 gives engine 8 confidence level above 50%
0.92   RAZOR2_CHECK                   Listed in Razor2 (http://razor.sf.net/)
-0.00  SPF_HELO_PASS                  SPF: HELO matches SPF record
0.67   SPF_SOFTFAIL                   SPF: sender does not match SPF record (softfail)
0.01   T_DKIM_INVALID	 
1.61   URIBL_WS_SURBL                 Contains an URL listed in the WS SURBL blocklist
The message details can be seen by clicking on "Recent Messages" and clicking on the space between the [ ] characters on the leftmost column.

Re: Tuning filters

Posted: 16 Apr 2015 07:01
by darky83
Yea I never use public DNS servers (google, opendns, from ISP etc..) so never had the issue before but some users have pointed out before that using an public DNS server for your query's might give you the URIBL_BLOCKED messages.

See Michaelv's post: viewtopic.php?t=934

We are looking into using an resolving DNS server on the E.F.A. image itself using unbound or bind and don't give the users the option to use forwarding DNS servers anymore. (but not all people might like this so...)

Re: Tuning filters

Posted: 29 Aug 2015 03:18
by csum77
Hi all! Sorry to revive an old thread, but I handed this off to someone else and they've been struggling & not really getting anywhere.

Per the suggestions, I did check the DNS blocks, and made some changes to address it, but that hasn't really helped improve the situation. However, just recently we found something that I wonder if it isn't part of the problem. When we go to the Bayes Database Information page in EFA it says our Last Journal Sync was Wed, Dec 31 1969!

I'm searching the forums to try to find information on this, but I thought I'd update with this info as I'm guessing it's related.

If anyone has insight into this, or can point me in a direction to help fix it, that would be greatly appreciated.

Thanks!

-Charlie

Re: Tuning filters

Posted: 29 Aug 2015 05:10
by shawniverson
This is normal.

The journal isn't enabled by default. It is useful in high traffic situations to reduce contention on the bayes database.

Re: Tuning filters

Posted: 29 Aug 2015 05:35
by csum77
Got it. Thanks for the clarification.

Back to the drawing board...

Re: Tuning filters

Posted: 31 Aug 2015 06:47
by pdwalker
Perhaps you can tell us a bit more about what problem you are actually having?