Missing entries for "Bad content detected"

[SelfMan]

Hi guys,
I am running: eFa-4.0.4 - with
MailWatch Version: 1.2.16
Operating System Version: CentOS Linux 7 (Core)
Postfix Version: 3.5.9
MailScanner Version: 5.4.1
ClamAV Version: 0.103.3
SpamAssassin Version: 3.4.6
PHP Version: 7.4.23
MySQL Version: 10.2.30-MariaDB
GeoIP Database Version: GeoLite2 Country database 2021-09-21 00:56:02
With all offered updates

The issue I am having for the past 7 days is that neither the "Recent messages" or "Search and reports" list show recent "Bad Content Detected" or "Other Bad Content Detected" entries in the listing. I am getting the notification mail, that these were processed though and should be there.

All other entry types are normally visible. If I search for "contained an Unacceptable Attachment (>0 = TRUE) is greater than '0'", I get only older entries.
This is preventing me from "releasing" safe items.

Any tips what can cause this and where to look?
THANKS

SelfMan

[SelfMan]

For few days it was fine and today the situation repeated.

Code: Select all

The following e-mails were found to have: Bad Filename Detected

    Sender: admin@uniba.sk
IP Address: 23.237.5.146
 Recipient: xxxxx@xxxxxx.xxx
   Subject: ŽIADOSŤ O CENOVÚ PONUKU (Univerzita Komenského v Bratislave) EUI894/SK4633
 MessageID: 4HVzd01xHSzZkM
Quarantine: /var/spool/MailScanner/quarantine/20211015/4HVzd01xHSzZkM
    Report: MailScanner: Executable DOS/Windows programs are dangerous in email (7RequestForQuote15-10-2021úpdf.exe)
            No programs allowed (7RequestForQuote15-10-2021úpdf.exe)
    Report: MailScanner: Executable DOS/Windows programs are dangerous in email (7RequestForQuote15-10-2021úpdf.exe)
            No programs allowed (7RequestForQuote15-10-2021úpdf.exe)

Full headers are:

 Received: from ns1.omnis.com ([23.237.5.146] [23.237.5.146])
 	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 	(no client certificate requested)
 	by efa43.xxxxx.xxx (MailScanner Milter) with SMTP id 4HVzd01xHSzZkM
 	for <xxxxx@xxxxx.xxx>; Fri, 15 Oct 2021 10:16:37 +0200 (CEST)
 DMARC-Filter: OpenDMARC Filter v1.4.1 efa43.xxxxx.xxx 4HVzd01xHSzZkM
 Authentication-Results: efa43.xxxxx.xxx; dmarc=fail (p=none dis=none) header.from=uniba.sk
 Authentication-Results: efa43.xxxxx.xxx; spf=fail smtp.mailfrom=uniba.sk
 DKIM-Filter: OpenDKIM Filter v2.11.0 efa43.xxxxx.xxx 4HVzd01xHSzZkM
 Received: from [216.38.8.189] (port=62300)
 	by ns1.omnis.com with esmtpsa  (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 	(Exim 4.94.2)
 	(envelope-from <admin@uniba.sk>)
 	id 1mbIOF-0006fT-SA
 	for rvydra@xxxxx.xxx; Fri, 15 Oct 2021 04:16:32 -0400
 From: =?UTF-8?B?VW5pdmVyeml0YSBLb21lbnNrw6lobyB2IEJyYXRpc2xhdmU=?= <admin@uniba.sk>
 To: rvydra@xxxxx.xxx
 Subject: =?UTF-8?B?xb1JQURPU8WkIE8gQ0VOT1bDmiBQT05VS1UgKFVuaXZlcnppdGEgS29tZW5za8OpaG8gdiBCcmF0aXNsYXZlKSBFVUk4OTQvU0s0NjMz?=
 Date: 15 Oct 2021 01:16:28 -0700
 Message-ID: <20211015011627.7486CB8D43741A48@uniba.sk>
 MIME-Version: 1.0
 Content-Type: multipart/mixed;
 	boundary="----=_NextPart_000_0012_2E735B9E.C81F5B2E"
 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
 X-AntiAbuse: Primary Hostname - ns1.omnis.com
 X-AntiAbuse: Original Domain - xxxxx.xxx
 X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
 X-AntiAbuse: Sender Address Domain - uniba.sk
 X-Get-Message-Sender-Via: ns1.omnis.com: authenticated_id: smtp36@aws.amazon.com
 X-Authenticated-Sender: ns1.omnis.com: smtp36@aws.amazon.com


-- 
EFA
Email Filter Appliance
www.efa-project.org

[shawniverson]

/etc/MailScanner/MailScanner.conf

Code: Select all

Quarantine Infections = yes

[SelfMan]

Hi, that variable is set all the time to yes.

[SelfMan]

Today, for some unknown reason, eFa started to block most of the incoming e-mails as "Other bad content detected".
All the e-mails were stored in "/var/spool/MailScanner/quarantine", so I was able to recover them and store as *.eml
These e-mails are not listed in the "Recent messages" tab, nor are they searchable in "Search and reports".

Even when I stored the blocked message as eml and then forwarded it from my external account, it got blocked.
These messages were of different types. Some of them included embedded images, others were just text.
We have restarted the server and that fixed it. I am now able to receive mail. The previous blocked emails are still not visible, yet in quarantine.

[SelfMan]

Further investigation revealed the following:
Mail (anonimized)

Code: Select all

Received: from mail.xxx.xx (mail.xxx.xx [123.456.789.10])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(no client certificate requested)
	by efa43.xxxxxxx.zzz (MailScanner Milter) with SMTP id 4HZdXy4LMbzqqL;
	Thu, 21 Oct 2021 08:55:48 +0200 (CEST)
DMARC-Filter: OpenDMARC Filter v1.4.1 efa43.xxxxxxx.zzz 4HZdXy4LMbzqqL
Authentication-Results: efa43.xxxxxxx.zzz; dmarc=none (p=none dis=none) header.from=aaaaaaa.zzz
Authentication-Results: efa43.xxxxxxx.zzz; spf=pass smtp.mailfrom=aaaaaaa.zzz
DKIM-Filter: OpenDKIM Filter v2.11.0 efa43.xxxxxxx.zzz 4HZdXy4LMbzqqL
IronPort-Data: A9a23: removed =
X-IronPort-AV: E=Sophos;i="5.87,169,1631570400"; 
   d="scan'208";a="68598309"
Received: from mx.aaaaaaa.zzz (HELO msx1.upvsp.xxx.xx) ([100.112.210.210])
  by g2inmail.xxx.xx with ESMTP/TLS/ECDHE-RSA-AES256-SHA384; 21 Oct 2021 08:55:46 +0200
Received: from EDEMCASN521 (10.100.130.5) by msx1.upvsp.xxx.xx (10.20.2.221)
 with Microsoft SMTP Server id 14.3.498.0; Thu, 21 Oct 2021 08:55:44 +0200
MIME-Version: 1.0
From: <enotify@aaaaaaa.zzz>
To: <mail@xxxxxxx.zzz>
Date: Thu, 21 Oct 2021 08:55:44 +0200
Subject: =?utf-8 removed ?=
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
Message-ID: <ae826a51-ab77-4642-999f-352811a1a406@MSX1.upvsp.xxx.xx>

< removed base 64 encoded html message >

Caused following error (from the maillog):

Code: Select all

Oct 21 08:55:45 efa43 postfix/smtpd[16314]: connect from mail.xxx.xx[123.456.789.10]
Oct 21 08:55:46 efa43 postfix/smtpd[16314]: Anonymous TLS connection established from mail.xxx.xx[123.456.789.10]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Oct 21 08:55:46 efa43 postfix/smtpd[16314]: 4HZdXy4LMbzqqL: client=mail.xxx.xx[123.456.789.10]
Oct 21 08:55:48 efa43 postfix/cleanup[16320]: 4HZdXy4LMbzqqL: message-id=<ae826a51-ab77-4642-999f-352811a1a406@MSX1.upvsp.xxx.xx>
Oct 21 08:55:48 efa43 opendkim[9716]: 4HZdXy4LMbzqqL: mail.xxx.xx [123.456.789.10] not internal
Oct 21 08:55:48 efa43 opendkim[9716]: 4HZdXy4LMbzqqL: not authenticated
Oct 21 08:55:48 efa43 opendkim[9716]: 4HZdXy4LMbzqqL: no signature data
Oct 21 08:55:48 efa43 opendmarc[9718]: 4HZdXy4LMbzqqL: SPF(mailfrom): aaaaaaa.zzz pass
Oct 21 08:55:48 efa43 opendmarc[9718]: 4HZdXy4LMbzqqL: aaaaaaa.zzz none
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Whitelist refresh time reached
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Starting up MailWatch SQL Whitelist
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Read 36 whitelist entries
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Blacklist refresh time reached
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Starting up MailWatch SQL Blacklist
Oct 21 08:55:48 efa43 MSMilter[16318]: MailWatch: Read 516 blacklist entries
Oct 21 08:55:49 efa43 postfix/cleanup[16320]: 4HZdXy4LMbzqqL: milter-discard: END-OF-MESSAGE from mail.xxx.xx[123.456.789.10]: milter triggers DISCARD action; from=<prvs=921a09965=enotify@aaaaaaa.zzz> to=<mail@xxxxxxx.zzz> proto=ESMTP helo=<mail.xxx.xx>
Oct 21 08:55:54 efa43 postfix/smtpd[16314]: disconnect from mail.xxx.xx[123.456.789.10] ehlo=2 starttls=1 mail=1 rcpt=3 data=1 quit=1 commands=9
Oct 21 08:55:55 efa43 MailScanner[10070]: New Batch: Scanning 1 messages, 2837 bytes
Oct 21 08:55:55 efa43 MailScanner[10070]: Virus and Content Scanning: Starting
Oct 21 08:55:55 efa43 MailScanner[10070]: <A> tag found in message 4HZdXy4LMbzqqL from prvs=921a09965=enotify@aaaaaaa.zzz
Oct 21 08:55:55 efa43 MailScanner[10070]: Spam Checks: Starting
Oct 21 08:55:55 efa43 MailScanner[10070]: MailWatch: Blacklist refresh time reached
Oct 21 08:55:55 efa43 MailScanner[10070]: MailWatch: Starting up MailWatch SQL Blacklist
Oct 21 08:55:55 efa43 MailScanner[10070]: MailWatch: Read 516 blacklist entries
Oct 21 08:55:56 efa43 MailScanner[10070]: Unable to initialise database connection: Access denied for user 'efa'@'localhost' (using password: YES)
Oct 21 08:55:57 efa43 MailScanner[10070]: HTML disarming died, status = 13
Oct 21 08:55:57 efa43 MailScanner[10070]: Content Checks: Detected and have disarmed denialofservice tags in HTML message in 4HZdXy4LMbzqqL from prvs=921a09965=enotify@aaaaaaa.zzz
Oct 21 08:55:57 efa43 MailScanner[10070]: Quarantined message 4HZdXy4LMbzqqL as it caused MailScanner to crash several times
Oct 21 08:55:57 efa43 MailScanner[10070]: Saved entire message to /var/spool/MailScanner/quarantine/20211021/4HZdXy4LMbzqqL
Oct 21 08:55:57 efa43 postfix/pickup[7394]: 4HZdY90f1DzqqL: uid=89 from=<postmaster>
Oct 21 08:55:57 efa43 MailScanner[10070]: Notices: Warned about 1 messages
Oct 21 08:55:57 efa43 MailScanner[10070]: Deleted 1 messages from processing-database
Oct 21 08:55:57 efa43 opendmarc[9718]: ignoring connection from localhost
Oct 21 08:55:57 efa43 postfix/cleanup[16320]: 4HZdY90f1DzqqL: message-id=<4HZdY90f1DzqqL@efa43.xxxxxxx.zzz>
Oct 21 08:55:57 efa43 opendkim[9716]: 4HZdY90f1DzqqL: no signature data
Oct 21 08:55:57 efa43 postfix/qmgr[2597]: 4HZdY90f1DzqqL: from=<postmaster@xxxxxxx.zzz>, size=3213, nrcpt=1 (queue active)
Oct 21 08:55:57 efa43 postfix/smtp[16337]: 4HZdY90f1DzqqL: to=<postmaster@xxxxxxx.zzz>, orig_to=<postmaster>, relay=10.100.1.71[10.100.20.70]:25, delay=0.27, delays=0.14/0.02/0/0.12, dsn=2.6.0, status=sent (250 2.6.0 <4HZdY90f1DzqqL@efa43.xxxxxxx.zzz> [InternalId=64076617089062, Hostname=EXCHANGE.xxxxxxx.zzz] 4479 bytes in 0.104, 41.770 KB/sec Queued mail for delivery)
Oct 21 08:55:57 efa43 postfix/qmgr[2597]: 4HZdY90f1DzqqL: removed

[SelfMan]

Oct 21 08:55:57 efa43 MailScanner[10070]: Quarantined message 4HZdXy4LMbzqqL as it caused MailScanner to crash several times

Where can i find the details of the crash?

[SelfMan]

I think that this emoji in mail problem is related
viewtopic.php?f=13&t=3692&p=14380&hilit ... ion#p14380

[shawniverson]

Temporary workaround while I fix the upstream code:

/etc/MailScanner/MailScanner.conf

Code: Select all

Ignore Denial of Service = yes

[SelfMan]

Ok, thanks will do.
Was any of the samples useful for you?
Thanks

[shawniverson]

Not sure I need any samples, there's a bug in the code that causes a race condition depending on how long the HTML disarm takes.

[SelfMan]

Ok, thank you. The "Ignore Denial of Service = yes" seems to be working.