[FRQ] Integration of Malware Patrol
[FRQ] Integration of Malware Patrol
Hello,
I would wish me an integration of Malware Patrol in the great email filter appliance with SpamAssassin and ClamAV to beat the spam even better. What you think about this idea?
Regards,
Patrik
I would wish me an integration of Malware Patrol in the great email filter appliance with SpamAssassin and ClamAV to beat the spam even better. What you think about this idea?
Regards,
Patrik
Last edited by Patschi on 21 Apr 2015 07:55, edited 1 time in total.
- shawniverson
- Posts: 3649
- Joined: 13 Jan 2014 23:30
- Location: Indianapolis, Indiana USA
- Contact:
Re: [FR] Integration of Malware Patrol
Sounds great, but it looks like it requires a subscription to use it, including the free version?
Re: [FR] Integration of Malware Patrol
Yes, both requires a registration at this site. But the free version is really free, you just need to sign up and you get a access to the free version of this service (I am currently using that, but thinking about to switch to an payed subscription). After that you receive something like an API key where you can retrieve the according data via an URL like lists.malwarepatrol.net/cgi/getfile?receipt=API_KEY&product=8&list=spamassassin.
This information have to be downloaded in an specific interval, parsed and integrated with the services. Cybercizi for example provided a simple way how I can be integrated with postfix (but the malwarepatrol URL is not working anymore in this article): http://www.cyberciti.biz/tips/howto-set ... lware.html
This information have to be downloaded in an specific interval, parsed and integrated with the services. Cybercizi for example provided a simple way how I can be integrated with postfix (but the malwarepatrol URL is not working anymore in this article): http://www.cyberciti.biz/tips/howto-set ... lware.html
Re: [FR] Integration of Malware Patrol
[edit: warning - stream of consciousness posting]
If you subscribe to the free version, you'll get access to to the block lists.
They provide preconfigured files for:
They also provide a script (https://www.malwarepatrol.net/MalwarePatrolDownload.sh - you may have to log in to get the script) to aid you in downloading the data. You only have to configure two things in the script before you run it.
As the free data is only updated every 2 to 3 days, don't abuse their hospitality and run it more often than that.
Having tried it, it now takes spamassassin lint from 5 seconds to over 735, so I won't be doing that check too often with the rules enabled.
---
Looking further, it appears that ClamAV used to use the Malware Patrol db. In fact, I can see in the log /var/log/clamav-unofficial-sigs.log that it tries to download a block list to: /usr/unofficial-dbs/mbl-dbs/mbl.ndb with the following content:
Other info:
cron job:
/etc/cron.d/clamav-unofficial-sigs-cron
script file (https://sourceforge.net/projects/unofficial-sigs/files/ - not updated since 2013-12-02)
/usr/local/bin/clamav-unofficial-sigs.sh
configuration file
/usr/local/etc/clamav-unofficial-sigs.conf
We should comment out lines 184-186 to avoid trying to download a list we cannot get until the "new" way of getting the data is working.
Also, we should set mbl_update_hours from 6 to 60.
---
The /usr/local/bin/clamav-unofficial-sigs.sh script will need updating to allow for the new way of downloading the MalwarePatrol block list. The relevant lines are between 1265 and 1380.
If you subscribe to the free version, you'll get access to to the block lists.
They provide preconfigured files for:
- - ClamAV
- Postfix MTA
- SpamAssassin
They also provide a script (https://www.malwarepatrol.net/MalwarePatrolDownload.sh - you may have to log in to get the script) to aid you in downloading the data. You only have to configure two things in the script before you run it.
As the free data is only updated every 2 to 3 days, don't abuse their hospitality and run it more often than that.
Having tried it, it now takes spamassassin lint from 5 seconds to over 735, so I won't be doing that check too often with the rules enabled.
---
Looking further, it appears that ClamAV used to use the Malware Patrol db. In fact, I can see in the log /var/log/clamav-unofficial-sigs.log that it tries to download a block list to: /usr/unofficial-dbs/mbl-dbs/mbl.ndb with the following content:
So, I guess that ClamAV should be the way to integrate the MalwarePatrol block list.###################################################################
# WARNING: this block list was discontinued on Oct/07/2013
# according to our previous announcements.
# Users are advised to visit https://www.malwarepatrol.net/
# for information on how to continue using our data feed.
###################################################################
Other info:
cron job:
/etc/cron.d/clamav-unofficial-sigs-cron
script file (https://sourceforge.net/projects/unofficial-sigs/files/ - not updated since 2013-12-02)
/usr/local/bin/clamav-unofficial-sigs.sh
configuration file
/usr/local/etc/clamav-unofficial-sigs.conf
We should comment out lines 184-186 to avoid trying to download a list we cannot get until the "new" way of getting the data is working.
Also, we should set mbl_update_hours from 6 to 60.
---
The /usr/local/bin/clamav-unofficial-sigs.sh script will need updating to allow for the new way of downloading the MalwarePatrol block list. The relevant lines are between 1265 and 1380.
Re: [FR] Integration of Malware Patrol
solved.
writing up the details now
writing up the details now
[HOWTO] Enable Integration of Malware Patrol in efa 3.0.0.9
ok, I have it working, and I fixed a minor problem with the ClamAV installation.
First, the minor problem
/etc/clamd.conf has the database directory set to:
/usr/local/etc/clamav-unofficial-sigs.conf has it set to:
Change the latter to match the former so the unofficial databases will be used, like so:
(that's how it was configured in my installation - check yours to see what you have configured - the problem just may have been with me - do let me know, I'm curious)
Moderator: The correct path is /var/clamav . EPEL update tries to configure clamd to /var/lib/clamav.
Second, re-enable MailPatrol support with ClamAV
subscribe to the MailPatrol service
diff clamav-unofficial-sigs.conf clamav-unofficial-sigs.conf.org
Moderator: The correct path is /var/clamav . EPEL update tries to configure clamd to /var/lib/clamav.
Don't forget to update "YOUR RECEIPT PASSWORD HERE" with your proper receipt id for the Malware Patrol service
fix /usr/local/bin/clamav-unofficial-sigs.sh update script
diff clamav-unofficial-sigs.sh clamav-unofficial-sigs.sh.org
test the command by running it and looking for errors
send yourself a bad message
and send it to your efa protected account. You should see this message flag (make sure you use a current evil url)
Fix the Freshclam config to make it consistent
vi /etc/freshclam.conf
finally, force one last update to make sure everything is behaving
Voila! Malware Patrol enabled.
Unresolved Issues
First, the minor problem
/etc/clamd.conf has the database directory set to:
Code: Select all
DatabaseDirectory /var/lib/clamav
Code: Select all
clam_dbs="/var/clamav"
Code: Select all
clam_dbs="/var/lib/clamav"
Moderator: The correct path is /var/clamav . EPEL update tries to configure clamd to /var/lib/clamav.
Second, re-enable MailPatrol support with ClamAV
subscribe to the MailPatrol service
- go to https://www.malwarepatrol.net/lists.shtml and choose either the free or premium subscription
- you will get a receipt/password string.
- save this.
diff clamav-unofficial-sigs.conf clamav-unofficial-sigs.conf.org
Moderator: The correct path is /var/clamav . EPEL update tries to configure clamd to /var/lib/clamav.
Code: Select all
53c53
< clam_dbs="/var/lib/clamav"
---
> clam_dbs="/var/clamav"
175c175
< si_update_hours="167" # Most of these are hardly updated anymore.
---
> si_update_hours="4" # Default is 4 hours (6 update checks daily).
188,189d187
< mbl_receipt="YOUR RECEIPT PASSWORD HERE"
<
198c196
< mbl_update_hours="60" # Every 2.5 days.
---
> mbl_update_hours="6" # Default is 6 hours (4 downloads daily).
fix /usr/local/bin/clamav-unofficial-sigs.sh update script
diff clamav-unofficial-sigs.sh clamav-unofficial-sigs.sh.org
Code: Select all
19c19
< default_config="/usr/local/etc/clamav-unofficial-sigs.conf"
---
> default_config="/etc/clamav-unofficial-sigs.conf"
756c756
< mbl_url="lists.malwarepatrol.net"
---
> mbl_url="www.malwarepatrol.net"
1284d1283
< # [Note: new format DB not available to free subscriptions]
1287c1286
< # test -e $clam_dbs/$db_file -o -e $clam_dbs/$db_file-bak && rm -f -- "$clam_dbs/mbl.d*"
---
> test -e $clam_dbs/$db_file -o -e $clam_dbs/$db_file-bak && rm -f -- "$clam_dbs/mbl.d*"
1294,1295c1293
< --max-time "$curl_max_time" -o $mbl_dir/$db_file --compressed \
< "https://$mbl_url/cgi/getfile?receipt=$mbl_receipt&product=8&list=clamav_basic"
---
> --max-time "$curl_max_time" -o $mbl_dir/$db_file http://$mbl_url/cgi/submit?action=list_clamav_ext
Code: Select all
/usr/local/bin/clamav-unofficial-sigs.sh
Code: Select all
To: me
Subject: clamd test
Body:
this is an evil url:
http://update.ss838.com/
Fix the Freshclam config to make it consistent
vi /etc/freshclam.conf
Code: Select all
DatabaseDirectory /var/lib/clamav
Code: Select all
# force unofficial-dbs refresh
rm /usr/unofficial-dbs/configs/last-mbl-update.txt
rm /usr/unofficial-dbs/configs/last-si-update.txt
/usr/bin/freshclam
/usr/local/bin/clamav-unofficial-sigs.sh
Unresolved Issues
- both /var/clamav and /var/lib/clamav exist. There should be only 1 directory; I think it should be /var/lib/clamav as that appear to be the default. Did the other directory come from unofficial-sigs.sh or was that an efa decision?
- which id? clam:clam or clamav:clamav. efa wants the former and clamav wants the latter.
Moderator: The correct path is /var/clamav . EPEL update tries to configure clamd to /var/lib/clamav. Correct user is clamav. Again, same reason - http://sanesecurity.com/usage/signatures/ has more signatures available than are being used. The list should be reviewed and the /usr/local/etc/clamav-unofficial-sigs.conf updated to use/exclude those databases
- https://www.securiteinfo.com/ has also gone to a subscription model. They do allow some updates, but the better updates require an account. On the plus side, it is updated hourly. The unofficial-sigs.sh script should be changed to handle these.
- note to self: list of other services that might ingegrate?
- shawniverson
- Posts: 3649
- Joined: 13 Jan 2014 23:30
- Location: Indianapolis, Indiana USA
- Contact:
- shawniverson
- Posts: 3649
- Joined: 13 Jan 2014 23:30
- Location: Indianapolis, Indiana USA
- Contact:
Re: [FRQ] Integration of Malware Patrol
This appears to be self-inflicted.both /var/clamav and /var/lib/clamav exist. There should be only 1 directory; I think it should be /var/lib/clamav as that appear to be the default. Did the other directory come from unofficial-sigs.sh or was that an efa decision?
which id? clam:clam or clamav:clamav. efa wants the former and clamav wants the latter.
Using EPEL, by chance?
EPEL uses /var/lib/clamav and clam user instead of /var/clamav and clamav user
Re: [FRQ] Integration of Malware Patrol
Stock Centos 6 and build from scratch.
Should i reset everything to use /var/clamav?
Should i reset everything to use /var/clamav?
- shawniverson
- Posts: 3649
- Joined: 13 Jan 2014 23:30
- Location: Indianapolis, Indiana USA
- Contact:
Re: [FRQ] Integration of Malware Patrol
I am thinking so. I am working through this issue right now using epel...
- shawniverson
- Posts: 3649
- Joined: 13 Jan 2014 23:30
- Location: Indianapolis, Indiana USA
- Contact: