efa-project.org

Hi there,

we have been using EFA from past few years and came to conclusion that it is very effective product. However, few days back we came across a problem that PDF attachments from some domains were getting corrupted. We have checked logs and found that Mailscanner is not blocking these attachments but corrupting it some how.Logs is pasted for reference:

May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E 1289.PDF
May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E 911.PDF
May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E 900.PDF
May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E image001.jpg
May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E msg-32058-185.html (no rule matched)
May 21 10:58:16 MailScanner[32058]: Filename Checks: Allowing 5AC74120DA7.AF17E msg-32058-184.txt
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E msg-32058-185.html
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E msg-32058-184.txt
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E 900.PDF (no match found)
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E 911.PDF (no match found)
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E 1289.PDF (no match found)
May 21 10:58:16 MailScanner[32058]: Filetype Checks: Allowing 5AC74120DA7.AF17E image001.jpg (no match found)
May 21 10:58:16 MailScanner[32058]: HTML Img tag found in message 5AC74120DA7.AF17E from ******@domain.com
May 21 10:58:21 MailScanner[32058]: Requeue: 5AC74120DA7.AF17E to 09C57120DAA
May 21 10:58:21 MailScanner[32058]: MailWatch: Logging message 5AC74120DA7.AF17E to SQL
May 21 10:58:21 MailScanner[5848]: MailWatch: 5AC74120DA7.AF17E: Logged to MailWatch SQL


May 16 10:22:44 MailScanner[9386]: Filename Checks: Allowing 90606120D9E.A9951 msg-9386-210.txt
May 16 10:22:44 MailScanner[9386]: Filename Checks: Allowing 90606120D9E.A9951 msg-9386-209.txt
May 16 10:22:44 MailScanner[9386]: Filename Checks: Allowing 90606120D9E.A9951 120377_BROKRAGE BILL.PDF.pdf
May 16 10:22:44 MailScanner[9386]: Filename Checks: Allowing 90606120D9E.A9951 msg-9386-208.html (no rule matched)
May 16 10:22:44 MailScanner[9386]: Filename Checks: Allowing 90606120D9E.A9951 msg-9386-207.txt
May 16 10:22:44 MailScanner[9386]: Filetype Checks: Allowing 90606120D9E.A9951 msg-9386-210.txt
May 16 10:22:44 MailScanner[9386]: Filetype Checks: Allowing 90606120D9E.A9951 msg-9386-209.txt
May 16 10:22:44 MailScanner[9386]: Filetype Checks: Allowing 90606120D9E.A9951 msg-9386-207.txt
May 16 10:22:44 MailScanner[9386]: Filetype Checks: Allowing 90606120D9E.A9951 msg-9386-208.html
May 16 10:22:44 MailScanner[9386]: Filetype Checks: Allowing 90606120D9E.A9951 120377_BROKRAGE BILL.PDF.pdf (no match found)
May 16 10:22:49 MailScanner[9386]: Requeue: 90606120D9E.A9951 to 35926120DA7
May 16 10:22:49 MailScanner[9386]: MailWatch: Logging message 90606120D9E.A9951 to SQL
May 16 10:22:49 MailScanner[10606]: MailWatch: 90606120D9E.A9951: Logged to MailWatch SQL

We have also checked by allowing .PDF and .PDF.pdf extensions in filename.rules.conf and whitelisting those domains from which we were having problem but all in vain.
Also to confirm that EFA is causing this corruption in files , we have bypassed EFA and all attachments received successfully. We are using EFA version 3.0.2.6
It has also been observed that these blocked pdf files have .PDF or .PDF.pdf extensions. Files with extension *.pdf(in small letters) received successfully.

Kindly suggest how to identify the root cause of this issue.

Hi Rapid,

As only some domains have this issue, you could first try modifying your TNEF decoder settings in /etc/MailScanner/MailScanner.conf
to get the root cause of this issue. for the source that sends you problematic TNEF attachments.

Password protected files can give issues to.

Do you use Sohos?
Long time ago there was also an issue with Sophos and pdf files

Also, is the original pdf actually a valid pdf?

henk wrote: 21 May 2018 09:08 Hi Rapid,

As only some domains have this issue, you could first try modifying your TNEF decoder settings in /etc/MailScanner/MailScanner.conf
to get the root cause of this issue.

Code: Select all

TNEF Contents = replace ==> Use TNEF Contents = no 

for the source that sends you problematic TNEF attachments.

Password protected files can give issues to.

Do you use Sohos?
Long time ago there was also an issue with Sophos and pdf files

Code: Select all

The fix was to set "Allowed Sophos Error Messages = corrupt" in the configuration file which means that Sophos will simply ignore any
"damaged" attachment that generates the Sophos error string "(corrupt)".

Thank you for your response. We will change TNEF settings as suggested and will test emails again.
No we don't use Sophos, so I don't think this could be the issue an Password protected files are already allowed on server.
Thank You

pdwalker wrote: 21 May 2018 10:27 Also, is the original pdf actually a valid pdf?

Hi pdwalker,

These pdf files are valid. As on receiving these files on gmail, hotmail it didn't get corrupted. Also we have checked it after bypassing EFA and emails were received without corruption on our Mail Server.

henk wrote: 21 May 2018 09:08 Hi Rapid,

As only some domains have this issue, you could first try modifying your TNEF decoder settings in /etc/MailScanner/MailScanner.conf
to get the root cause of this issue.

Code: Select all

TNEF Contents = replace ==> Use TNEF Contents = no 

for the source that sends you problematic TNEF attachments.

Password protected files can give issues to.

Do you use Sohos?
Long time ago there was also an issue with Sophos and pdf files

Code: Select all

The fix was to set "Allowed Sophos Error Messages = corrupt" in the configuration file which means that Sophos will simply ignore any
"damaged" attachment that generates the Sophos error string "(corrupt)".

Hi Henk,

We have applied TNEF decoder settings as suggested but still all PDF in email got corrupted, blow are logs:

May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 1289.PDF
May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 911.PDF
May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 900.PDF
May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 image001.jpg
May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 msg-14188-9.html (no rule matched)
May 22 14:42:18 MailScanner[14188]: Filename Checks: Allowing 3F528120D9E.A1B15 msg-14188-8.txt
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 900.PDF (no match found)
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 msg-14188-8.txt
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 911.PDF (no match found)
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 1289.PDF (no match found)
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 image001.jpg (no match found)
May 22 14:42:19 MailScanner[14188]: Filetype Checks: Allowing 3F528120D9E.A1B15 msg-14188-9.html
May 22 14:42:19 MailScanner[14188]: HTML Img tag found in message 3F528120D9E.A1B15 from *******@domain.com
May 22 14:42:19 MailScanner[14188]: Message 3F528120D9E.A1B15 from X.X.X.X (*******@domain.com) is whitelisted
May 22 14:42:19 MailScanner[14188]: Requeue: 3F528120D9E.A1B15 to C0E93120DA7
May 22 14:42:19 MailScanner[14188]: MailWatch: Logging message 3F528120D9E.A1B15 to SQL
May 22 14:42:19 MailScanner[14192]: MailWatch: 3F528120D9E.A1B15: Logged to MailWatch SQL

Any other suggestion how to sort out this issue ?

You did restart MailScanner?

It's a bit strange that no other EFA user reported this behaviour. Seems you are the only one having this issue.
To be sure: By corrupted you mean unreadable or moved to quarantaine?

last thing I can think off: In your log an HTML tag is found. To be sure, you can temporary disable disarming tags, just to find the cause of the problem.

Take a look at viewtopic.php?f=13&t=3069

Try fo figure out what is the difference between the pdf files from domains that are processed ok, as I'm out of options.

henk wrote: 23 May 2018 07:22 You did restart MailScanner?

It's a bit strange that no other EFA user reported this behaviour. Seems you are the only one having this issue.
To be sure: By corrupted you mean unreadable or moved to quarantaine?

last thing I can think off: In your log an HTML tag is found. To be sure, you can temporary disable disarming tags, just to find the cause of the problem.

Take a look at viewtopic.php?f=13&t=3069

Try fo figure out what is the difference between the pdf files from domains that are processed ok, as I'm out of options.

Yes I Did start mailscanner service. Its is strange to me also, by corrupted I mean The file is unbale to open. Screen shot of error is attached.
We have already tested by disabling "disarming" from All options to check if this it is causing this issue. But logs are still same and nothing is showed in logs.
The worst thing is yesterday we received another PDF which is unable to open from a new domain.

May 24 16:39:29 postfix/cleanup[14293]: DF8DC120DAD: message-id=<005a01d3f353$e351f270$a9f5d750$@com.pk>
May 24 16:39:35 MailScanner[6131]: Filename Checks: Allowing DF8DC120DAD.A1BBF 1437.pdf
May 24 16:39:35 MailScanner[6131]: Filename Checks: Allowing DF8DC120DAD.A1BBF msg-6131-30.html (no rule matched)
May 24 16:39:35 MailScanner[6131]: Filename Checks: Allowing DF8DC120DAD.A1BBF msg-6131-29.txt
May 24 16:39:36 MailScanner[6131]: Filetype Checks: Allowing DF8DC120DAD.A1BBF 1437.pdf (no match found)
May 24 16:39:36 MailScanner[6131]: Filetype Checks: Allowing DF8DC120DAD.A1BBF msg-6131-29.txt
May 24 16:39:36 MailScanner[6131]: Filetype Checks: Allowing DF8DC120DAD.A1BBF msg-6131-30.html
May 24 16:39:42 MailScanner[6131]: Requeue: DF8DC120DAD.A1BBF to 45A30120D9E
May 24 16:39:42 MailScanner[6131]: MailWatch: Logging message DF8DC120DAD.A1BBF to SQL
May 24 16:39:42 MailScanner[6135]: MailWatch: DF8DC120DAD.A1BBF: Logged to MailWatch SQL

Please post the complete message detail of this mail. As clamav does not modify attachments, as far as I know.

Take al look at viewtopic.php?t=3102
--the sa-learn part- replacing filename / date/message-ID.

sa-learn -D --spam /var/spool/MailScanner/quarantine/20180522/spam/AB76B403AC.A94ED &> /tmp/henk1.log
Check the log

Run clamscan --debug on the pdf.

Second option: Let's try, From theMailwatch Gui-> message details to download and save the pdf.
And then open it from Acrobat reader -> open file.
The filename will be something like viewpart.php, but it's a pdf.
Somewhere in the deliver process it's renamed to a .pdf extension

I had some (classified as spam) issues with password protected attachments (pdf and Office docx).

The proposed -bad- solution https://bugzilla.clamav.net/show_bug.cgi?id=11911:
"Currenty, with Clam, the only way to send a password protected PDF is to disable the "ArchiveBlockEncrypted" check, allowing malware inside encrypted archives."

Solved by setting:
%rules-dir%/hilary.archives.rules The whitespaces are tabs.

Last option: hope someone else on this forum can step in.

Standing by to assist