Release a blocked file

Questions and answers about how to do stuff
Post Reply
danitaz
Posts: 24
Joined: 24 Jan 2014 17:56

Release a blocked file

Post by danitaz »

I received a PST from a client yesterday (expected), and it was blocked as a "Bad" file. Try as I like I cannot see a way to release this message so that it will deliver to me. Am I just blind?

Thanks.

Danita
User avatar
darky83
Site Admin
Posts: 540
Joined: 30 Sep 2012 11:03
Location: eFa
Contact:

Re: Release a blocked file

Post by darky83 »

first:
Blocked files are not stored on the system.
This is an default setting based of MailScanner based on the Regulation of Investigatory Powers Act 2000.

So releasing is not possible.

Second:
PST files are blocked by default (this is an default MailScanner Setting and I must say I agree. PST files are normally huge so having a small .pst file might mean viruses...)

If you want to allow the pst file you can edit /etc/MailScanner/filename.rules.conf and remove the deny for .pst, then reload mailscanner.
Version eFa 4.x now available!
danitaz
Posts: 24
Joined: 24 Jan 2014 17:56

Re: Release a blocked file

Post by danitaz »

Hmm - I guess I don't really like the idea that a file is just blocked with no recourse. There are plenty of times when you want to make exceptions regarding file types to allow through. I've worked with many different anti-spam/anti-virus systems that have quarantines, and this is the first system that has not given the ability to block a file by type, and yet have the ability to retrieve it if desired.

The other issue, is that I have my quarantine report set to show "spam" (I dont need to have the good mail mixed in it - although if it could be sorted by spam, ham, blocked, etc. that would be useful) and I had no idea from the quarantine digest that this file had been blocked. The sender asked me if I had received the file, and I had not. The sender said he also did not receive a notification of the block. So in essence, an expected email just went into "lala land".

Danita
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Bear in mind that E.F.A. is built on several components, one of which is MailScanner.

This is a default setting in MailScanner.

You should review the file extensions configured here and make sure none are listed that you really want to receive.

If there is a way to improve this behavior, by all means, please let us know. I agree with darky83 though, that not very many people are emailing each other .pst files, so for most this would be a potentially harmful file.
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote: ...
This is a default setting in MailScanner.
...
I have an issue currently with the same thing. Overall I have no issue with these default settings. However we have one company (financial institution) we deal with that will only send Password-Protected zip files of their info. No point arguing with them. This is the rules and they won't change.
I don't want to enable sending of Password-Protected zip attachments from everyone for the obvious security risks involved. Just from these guys.

I have spent hours reading and trying to create custom rules to just allow these specific attachments, but am failing miserably.

Any chance anyone can give me a clear guideline on what to include, in which files in /etc/MailScanner, to achieve the desired outcome? I assume I need to edit /etc/MailScanner/MailScanner.conf and add a line to a custom rule. But I can't even get this bit to work. Let alone the actual custom Rule.
Any assistance greatly appreciated.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Let me see what I can come up with...

It *might* require some modification to mailscanner code...not sure yet...will do some digging...
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote:Let me see what I can come up with...
It *might* require some modification to mailscanner code...not sure yet...will do some digging...
Ta Shawn,
the reading and experimenting to date suggests that if I create a line in the /etc/MailScanner/MailScanner.conf to replace

Code: Select all

# Should archives which contain any password-protected files be allowed?
# Leaving this set to "no" is a good way of protecting against all the
# protected zip files used by viruses at the moment.
# This can also be the filename of a ruleset.
Allow Password-Protected Archives = no
with a new ruleset name *should* work. The Contents of that ruleset are stumping me also though.

My main direction so far, has been to change the above to

Code: Select all

Allow Password-Protected Archives = /etc/MailScanner/rules/P-P.rules


Then within that to have

Code: Select all

From:   *@trusteddomain.com   and   to:   recipient@mydomain.com   yes
FromOrTo:   default   no
But alas, I can't seem to get this to work.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Do this...

Code: Select all

Allow Password-Protected Archives = %rules-dir%/P-P.rules
Careful with your case sentitivity and spaces/tabs in the rules...I recommend using a single space between each object in the rule....

Code: Select all

From: *@trusteddomain.com and To: recipient@mydomain.com yes
FromOrTo: default no
Restart MailScanner

Code: Select all

sudo service MailScanner restart
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote:Do this...

...
Careful with your case sentitivity and spaces/tabs in the rules...I recommend using a single space between each object in the rule....
...
Restart MailScanner

Code: Select all

sudo service MailScanner restart
OK, the 3 things I have done differently are
  • used the %rules-dir% environment variable (I should have known that one)
  • changed the single Tab Stops to single Spaces (I think from the reading this one is optional as long as it is only a single character)
  • and restarted using "sudo service MailScanner restart" instead of "sudo kill -HUP 'cat /var/run/MailScanner.pid'" (No idea why I was using this stupid method!)
I'll test again today and see if this has solved the issue.
Thanks again Shawn. I really appreciate all the assistance.
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

ramtech wrote: [I'll test again today and see if this has solved the issue.
Yep! This works... :dance:
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

Here's an unexpected eventuality...
When MailScanner crafts a message to the sender to tell them their email never made it, it rejects it's own message because it is not filled out properly it would seem.

Code: Select all

Received on:	26/02/14 15:22:47
Received by:	mail.xxx.com
Received from:	
127.0.0.1	[Add to Whitelist | Add to Blacklist]
Received Via:	127.0.0.1
ID:	41524101512.ABC53
Message Headers:	Received: by mail.xxx.com (Postfix, from userid 89)
     id 41524101512; Wed, 26 Feb 2014 15:22:45 +1000 (EST)
From: "MailScanner" <postmaster@xxx.com>
To: noreply@<OriginalSenderDomain>.com
Subject: Potentially dangerous email rejected
X-VIP-MailScanner: generated
Message-Id: <20140226052245.41524101512@vipmail.vine-ip.com>
Date: Wed, 26 Feb 2014 15:22:45 +1000 (EST)
From:	
	[Add to Whitelist | Add to Blacklist]
To:	noreply@<OriginalSenderDomain>.com
Subject:	Potentially dangerous email rejected
Size:	1.4Kb
Anti-Virus/Dangerous Content Protection
Virus:	 N 
Blocked File:	 N 
Other Infection:	 N 
SpamAssassin
Spam:	 Y   Action(s): store
High Scoring Spam:	 Y   Action(s): store
SpamAssassin Spam:	 N 
Listed in RBL:	 N 
Spam Whitelisted:	 N 
Spam Blacklisted:	 N 
SpamAssassin Autolearn:	 N 
SpamAssassin Score:	10.00
Spam Report:	spam(no watermark or sender address)
As it turns out it wouldn't have made it anyway, as it is sent from a noreply@ address, but still... It shouldn't be rejecting it's own messages surely. I thought if 127.0.0.1 was whitelisted, it would not go through MailScanner.

Or am I mis-interpreting what I am reading...
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Interesting...so you do have 127.0.0.1 --> default in you whitelist? According to this report, Spam Whitelisted=N?
:?
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

Yep. It's there alright. Unchanged from out of the box. It sounds like I may have buggered something up in my install somewhere. I might remove and replace it and see what happens.
Update:
I have removed and replaced it in the GUI and it looks no different than before.
My /etc/MailScanner/MailScanner.conf has the whitelist set as follows out of the box...

Code: Select all

Is Definitely Not Spam = &SQLWhitelist
I have no SQL skills (Network Engineer... :/ ) so I am not sure how to check if the entry is there in the db OK.
Any assistance appreciated as always.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Try this, at least temporarily, and see what happens...I am fighting another issue related to this, wondering if it is doing the same thing in the other direction...

/etc/MailScanner/MailScanner.conf

Code: Select all

Treat Invalid Watermarks with No Sender as Spam = nothing

Code: Select all

sudo service MailScanner restart
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote:Try this, at least temporarily, and see what happens...
Will Do. This is presumable the OoO issue you refer to.
What I can't get my head around is why it even hits MailScanner if it is WhiteListed.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

Well, since MailScanner makes the determination whether it is whitelisted or not, my theory is that the Treat Invalid Watermarks with No Sender as Spam setting is taking precedence over the whitelists in the MailScanner code.
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote:Well, since MailScanner makes the determination whether it is whitelisted or not, my theory is that the Treat Invalid Watermarks with No Sender as Spam setting is taking precedence over the whitelists in the MailScanner code.
Makes Perfect sense.
what I actually meant is, why is it processing it, and assigning the 10 score, as opposed to, why is it hitting MailScanner (which is how i incorrectly phrased it sorry...).
I realise that it is Mail Scanner that makes the decision. I just (wrongly) assumed that it would check WhiteList first and process no further.
Apparently not. So How would I find an order of precedence for mail Scanner decisions?
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

I am testing this...

Code: Select all

Treat Invalid Watermarks with No Sender as Spam = 1
Idea is not to push it over to spam right away, let it traverse further into MailScanner but nudge up the spam score a little to treat it with greater suspicion or make it to whitelist/blacklist processing...
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

shawniverson wrote:I am testing this...

Code: Select all

Treat Invalid Watermarks with No Sender as Spam = 1
Idea is not to push it over to spam right away, let it traverse further into MailScanner but nudge up the spam score a little to treat it with greater suspicion or make it to whitelist/blacklist processing...
It would appear that MailScanner doesn't just go "DIng Ding Ding! We have a winner" as soon as it reaches 4 on the score, as it totals the score and gives an over all result. so it must process further through the precedence logic. However if this the case the "Invalid Watermark with no From:" must take some sort of over ruling precendence as the whitelisting as classed as "Never treat as Spam". Logic would suggest this should be the precedence.

My point being, it seems it have broken logic.
if we had a means of tracing through the logic process it would be easier to solve.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Release a blocked file

Post by shawniverson »

That is true in
ramtech wrote: It would appear that MailScanner doesn't just go "DIng Ding Ding! We have a winner" as soon as it reaches 4 on the score, as it totals the score and gives an over all result. so it must process further through the precedence logic. However if this the case the "Invalid Watermark with no From:" must take some sort of over ruling precendence as the whitelisting as classed as "Never treat as Spam". Logic would suggest this should be the precedence.

My point being, it seems it have broken logic.
if we had a means of tracing through the logic process it would be easier to solve.
However, MailScanner is stopping at that rule and is doing no futher processing. SpamAssassin and ClamAV never see the message, for example. You can watch the logs and also see this happening...

I agree, I don't think it should behave this way...I may dive into the MailScanner code and see if I can trace the logic flow.
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

Thanks Again Shawn.
We are way out of my league now. Over to you and others of your skill set.
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Release a blocked file

Post by ramtech »

I am having success with changing ...

/etc/MailScanner/MailScanner.conf

Code: Select all

    Treat Invalid Watermarks with No Sender as Spam = 3
It sends it off to SpamAssasin for subsequent processing and seems happier with the world. I will leave that in for the time being and see how it goes. The OoO are coming through and the Postmaster initiated notifications coming from LocalHost are being sent.
It remains to be seen what happens with genuine SPAM, but so far it is promising.
Post Reply