YARA

Questions and answers about how to do stuff
Post Reply
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

YARA

Post by stusmith »

Is anyone using Yara as part of their eFa configuration? I've been following the WannCry malware saga, and have recently signed up for US-CERT alerts, which include Yara rulesets for some things. It's a tool I'm not familiar with, but I've been reading about it. It seems that you can plug Yara rulesets into ClamAV signatures...

I've been searching the web to try and figure out how to incorporate this, but if anyone has a jump-start method it would be much appreciated.
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

It's already incorporated.

Look at /etc/clamav-unofficial-sigs/master.conf

This file includes the yara signatures as well as some others you can enable for your clamav instance. You can also control which yara rulesets you want to include.

Perhaps this will give you what you are looking for?
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

pdwalker wrote: 15 May 2017 13:17 It's already incorporated.

Look at /etc/clamav-unofficial-sigs/master.conf

This file includes the yara signatures as well as some others you can enable for your clamav instance. You can also control which yara rulesets you want to include.

Perhaps this will give you what you are looking for?
That's exactly it! Thanks! Part of what i wanted was to be able to scrape the notifications from US-CERT that include Yara signatures and feed those directly into the eFa Yara ruleset. Now I've got my jump-off point. Thanks again!
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

do let us know how you get on.
ovizii
Posts: 463
Joined: 11 May 2016 08:08

Re: YARA

Post by ovizii »

after reading this thread I checked and I see there are new YARA rules not integrated by this script. Please share the list of those you have added.
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

which ones are not added?
ovizii
Posts: 463
Joined: 11 May 2016 08:08

Re: YARA

Post by ovizii »

Easy to check. Inside /etc/clamav-unofficial-sigs/master.conf I see:

Code: Select all

# Yara Rules Project Database(s)
# ========================
# Add or remove database file names between quote marks as needed.  To
# disable any Yara Rule database downloads, remove the appropriate
# lines below.
yararulesproject_dbs="
### Yara Rules https://github.com/Yara-Rules/rules
#
# Some rules are now in sub-directories. To reference a file in a sub-directory
# use subdir/file
# LOW
email/EMAIL_Cryptowall.yar|LOW # CryptoWall Resume phish
Antidebug_AntiVM/antidebug_antivm.yar|LOW # anti debug and anti virtualization techniques used by malware
Exploit-Kits/EK_Angler.yar|LOW # Angler Exploit Kit Redirector
Exploit-Kits/EK_Blackhole.yar|LOW # BlackHole2 Exploit Kit Detection
Exploit-Kits/EK_BleedingLife.yar|LOW # BleedingLife2 Exploit Kit Detection
Exploit-Kits/EK_Crimepack.yar|LOW # CrimePack Exploit Kit Detection
Exploit-Kits/EK_Eleonore.yar|LOW # Eleonore Exploit Kit Detection
Exploit-Kits/EK_Fragus.yar|LOW # Fragus Exploit Kit Detection
Exploit-Kits/EK_Phoenix.yar|LOW # Phoenix Exploit Kit Detection
Exploit-Kits/EK_Sakura.yar|LOW # Sakura Exploit Kit Detection
Exploit-Kits/EK_ZeroAcces.yar|LOW # ZeroAccess Exploit Kit Detection
Exploit-Kits/EK_Zerox88.yar|LOW # 0x88 Exploit Kit Detection
Exploit-Kits/EK_Zeus.yar|LOW # Zeus Exploit Kit Detection
# MEDIUM
Malicious_Documents/maldoc_somerules.yar|MEDIUM # documents with malicious code
Malicious_Documents/Maldoc_Hidden_PE_file.yar|MEDIUM # Detect a hidden PE file inside a sequence of numbers (comma$
Packers/Javascript_exploit_and_obfuscation.yar|MEDIUM # JavaScript Obfuscation Detection
Packers/packer.yar|MEDIUM # well-known sofware packers
CVE_Rules/CVE-2010-0805.yar|MEDIUM # CVE 2010 0805
CVE_Rules/CVE-2010-0887.yar|MEDIUM # CVE 2010 0887
CVE_Rules/CVE-2010-1297.yar|MEDIUM # CVE 2010 1297
CVE_Rules/CVE-2013-0074.yar|MEDIUM # CVE 2013 0074
CVE_Rules/CVE-2013-0422.yar|MEDIUM # CVE 2013 0422
CVE_Rules/CVE-2015-5119.yar|MEDIUM # CVE 2015 5119
# HIGH
Crypto/crypto.yar|HIGH # detect the existence of cryptographic algoritms
" #END yararulesproject DATABASES
at a first glance at: https://github.com/Yara-Rules/rules/tre ... /CVE_Rules I already see severall CVE_Rules are not added. I haven't checked all the other sections yet.
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

pdwalker wrote: 17 May 2017 11:53 which ones are not added?

Code: Select all

CVE_Rules/CVE-2012-0158.yar|MEDIUM # CVE 2012 0158
CVE_Rules/CVE-2015-1701.yar|MEDIUM # CVE 2015 1701
CVE_Rules/CVE-2015-2426.yar|MEDIUM # CVE 2015 2426
CVE_Rules/CVE-2016-5195.yar|MEDIUM # CVE 2016 5195
Additionally,

Code: Select all

[@mx clamav]$ ls -l *.yar
-rw-r--r-- 1 clam clam 47013 Apr  8 18:38 antidebug_antivm.yar
-rw-r--r-- 1 clam clam 10889 Feb  6 18:15 EK_Angler.yar
-rw-r--r-- 1 clam clam 14659 Feb  6 18:15 EK_Blackhole.yar
-rw-r--r-- 1 clam clam  3401 Feb  6 18:15 EK_BleedingLife.yar
-rw-r--r-- 1 clam clam  1349 Feb  6 18:15 EK_Crimepack.yar
-rw-r--r-- 1 clam clam  4688 Feb  6 18:15 EK_Eleonore.yar
-rw-r--r-- 1 clam clam  8268 Feb  6 18:15 EK_Fragus.yar
-rw-r--r-- 1 clam clam 16842 Feb  6 18:15 EK_Phoenix.yar
-rw-r--r-- 1 clam clam  1860 Feb  6 18:15 EK_Sakura.yar
-rw-r--r-- 1 clam clam  8488 Feb  6 18:15 EK_ZeroAcces.yar
-rw-r--r-- 1 clam clam  1435 Feb  6 18:15 EK_Zerox88.yar
-rw-r--r-- 1 clam clam   800 Feb  6 18:15 EK_Zeus.yar
 -rw-r--r-- 1 clam clam  1412 Feb  6 18:15 EMAIL_Cryptowall.yar
I sort of expected that the CVE rules that were enabled to be downloaded here. However, when I try to use the URL defined for Yararulesproject_url="https://raw.githubusercontent.com/Yara- ... les/master" in a broswer, I receive 400: Invalid request.

Looking at the index.yar file on GitHub:

Code: Select all

/*
Generated by Yara-Rules
On 08-05-2017
*/
include "./malware/APT_FVEY_ShadowBrokers_Jan17_Screen_Strings.yar"
include "./malware/RAT_Gh0st.yar"
include "./malware/POS_Mozart.yar"
include "./malware/APT_Mirage.yar"
include "./malware/MALW_Buzus_Softpulse.yar"
include "./malware/RANSOM_Satana.yar"
include "./malware/MALW_DirtJumper.yar"
include "./malware/APT_Backspace.yar"
include "./malware/APT_Sofacy_Jun16.yar"
include "./malware/MALW_Backoff.yar"
include "./malware/MALW_Furtim.yar"
include "./malware/RAT_Sakula.yar"
include "./malware/APT_Ke3Chang_TidePool.yar"
include "./malware/APT_WildNeutron.yar"
include "./malware/RANSOM_TeslaCrypt.yar"
include "./malware/RAT_Indetectables.yar"
include "./malware/TOOLKIT_Dubrute.yar"
include "./malware/RANSOM_Petya.yar"
include "./malware/MALW_Wabot.yar"
include "./malware/MALW_Sayad.yar"
include "./malware/RAT_Inocnation.yar"
include "./malware/MALW_Lateral_Movement.yar"
include "./malware/RANSOM_Crypren.yar"
include "./malware/APT_PCclient.yar"
include "./malware/APT_Irontiger.yar"
include "./malware/MALW_Chicken.yar"
include "./malware/RAT_Njrat.yar"
include "./malware/MALW_Miscelanea_Linux.yar"
include "./malware/APT_Winnti.yar"
include "./malware/APT_HackingTeam.yar"
include "./malware/RAT_Adzok.yar"
include "./malware/MALW_Cxpid.yar"
include "./malware/APT_C16.yar"
include "./malware/APT_Greenbug.yar"
include "./malware/APT_CheshireCat.yar"
include "./malware/MALW_Wimmie.yar"
include "./malware/MALW_Lenovo_Superfish.yar"
include "./malware/POS_FastPOS.yar"
include "./malware/MALW_Cloaking.yar"
include "./malware/MALW_Favorite.yar"
include "./malware/RANSOM_Cryptolocker.yar"
include "./malware/MALW_Andromeda.yar"
include "./malware/POS_LogPOS.yar"
include "./malware/APT_APT3102.yar"
include "./malware/MALW_Olyx.yar"
include "./malware/MALW_XOR_DDos.yar"
include "./malware/RANSOM_Locky.yar"
include "./malware/RAT_Bozok.yar"
include "./malware/APT_Blackenergy.yar"
include "./malware/MALW_F0xy.yar"
include "./malware/RAT_jRAT.yar"
include "./malware/TOOLKIT_Wineggdrop.yar"
include "./malware/MALW_PubSab.yar"
include "./malware/MALW_Torte_ELF.yar"
include "./malware/MALW_Athena.yar"
include "./malware/MALW_OSX_Leverage.yar"
include "./malware/MALW_Mirai.yar"
include "./malware/MALW_Magento_backend.yar"
include "./malware/MALW_Tinba.yar"
include "./malware/MALW_Miancha.yar"
include "./malware/APT_Casper.yar"
include "./malware/APT_Careto.yar"
include "./malware/APT_eqgrp_apr17.yar"
include "./malware/RAT_xRAT20.yar"
include "./malware/TOOLKIT_Chinese_Hacktools.yar"
include "./malware/MALW_Bublik.yar"
include "./malware/MALW_Corkow.yar"
include "./malware/APT_Sofacy_Fysbis.yar"
include "./malware/RAT_Adwind.yar"
include "./malware/MALW_Derkziel.yar"
include "./malware/MALW_Odinaff.yar"
include "./malware/MALW_FakeM.yar"
include "./malware/RAT_ShadowTech.yar"
include "./malware/POS_Bernhard.yar"
include "./malware/MALW_Zeus.yar"
include "./malware/APT_OPCleaver.yar"
include "./malware/MALW_Elknot.yar"
include "./malware/APT_Mongall.yar"
include "./malware/APT_APT29_Grizzly_Steppe.yar"
include "./malware/RAT_Glass.yar"
include "./malware/APT_TradeSecret.yar"
include "./malware/APT_Sofacy_Bundestag.yar"
include "./malware/MALW_xDedic_marketplace.yar"
include "./malware/APT_Minidionis.yar"
include "./malware/APT_Cloudduke.yar"
include "./malware/APT_Snowglobe_Babar.yar"
include "./malware/MALW_LURK0.yar"
include "./malware/RANSOM_GoldenEye.yar"
include "./malware/RANSOM_Alpha.yar"
include "./malware/APT_PutterPanda.yar"
include "./malware/MALW_Dexter.yar"
include "./malware/APT_APT10.yar"
include "./malware/APT_Pipcreat.yar"
include "./malware/MALW_Warp.yar"
include "./malware/RAT_CyberGate.yar"
include "./malware/APT_APT1.yar"
include "./malware/MALW_Rovnix.yar"
include "./malware/MALW_Ezcob.yar"
include "./malware/MALW_Sakurel.yar"
include "./malware/MALW_Rockloader.yar"
include "./malware/MALW_NSFree.yar"
include "./malware/MALW_DDoSTf.yar"
include "./malware/MALW_Cookies.yar"
include "./malware/MALW_Gozi.yar"
include "./malware/APT_Stuxnet.yar"
include "./malware/RANSOM_Cerber.yar"
include "./malware/MALW_Spora.yar"
include "./malware/MALW_Exploit_UAC_Elevators.yar"
include "./malware/MALW_Yayih.yar"
include "./malware/APT_UP007_SLServer.yar"
include "./malware/APT_Kaba.yar"
include "./malware/APT_Grizzlybear_uscert.yar"
include "./malware/APT_Derusbi.yar"
include "./malware/APT_furtim.yar"
include "./malware/APT_KeyBoy.yar"
include "./malware/APT_Waterbug.yar"
include "./malware/POS_MalumPOS.yar"
include "./malware/APT_Scarab_Scieron.yar"
include "./malware/MALW_Hsdfihdf_banking.yar"
include "./malware/MALW_Kovter.yar"
include "./malware/RAT_ZoxPNG.yar"
include "./malware/RAT_Xtreme.yar"
include "./malware/APT_FiveEyes.yar"
include "./malware/MALW_Safenet.yar"
include "./malware/MALW_Mailers.yar"
include "./malware/RAT_PoisonIvy.yar"
include "./malware/APT_Shamoon_StoneDrill.yar"
include "./malware/APT_Emissary.yar"
include "./malware/RAT_xRAT.yar"
include "./malware/POS_BruteforcingBot.yar"
include "./malware/MALW_Quarian.yar"
include "./malware/MALW_Stealer.yar"
include "./malware/MALW_Korplug.yar"
include "./malware/MALW_Elex.yar"
include "./malware/MALW_IMuler.yar"
include "./malware/APT_DeputyDog.yar"
include "./malware/MALW_Magento_suspicious.yar"
include "./malware/MALW_MacControl.yar"
include "./malware/MALW_Bangat.yar"
include "./malware/APT_OpPotao.yar"
include "./malware/RAT_Havex.yar"
include "./malware/APT_Duqu2.yar"
include "./malware/MALW_PE_sections.yar"
include "./malware/GEN_PowerShell.yar"
include "./malware/MALW_Rooter.yar"
include "./malware/RAT_NetwiredRC.yar"
include "./malware/POS_Easterjack.yar"
include "./malware/APT_Turla_RUAG.yar"
include "./malware/MALW_Naikon.yar"
include "./malware/TOOLKIT_PassTheHash.yar"
include "./malware/EXPERIMENTAL_Beef.yar"
include "./malware/MALW_Atmos.yar"
include "./malware/APT_WoolenGoldfish.yar"
include "./malware/MALW_TreasureHunt.yar"
include "./malware/APT_Seaduke.yar"
include "./malware/RAT_Bolonyokte.yar"
include "./malware/MALW_Jolob_Backdoor.yar"
include "./malware/MALW_Upatre.yar"
include "./malware/TOOLKIT_Pwdump.yar"
include "./malware/MALW_BlackWorm.yar"
include "./malware/MALW_Batel.yar"
include "./malware/RAT_BlackShades.yar"
include "./malware/MALW_Regsubdat.yar"
include "./malware/RAT_DarkComet.yar"
include "./malware/MALW_BackdoorSSH.yar"
include "./malware/MALW_Shifu.yar"
include "./malware/APT_Hellsing.yar"
include "./malware/RANSOM_Stampado.yar"
include "./malware/RAT_Gholee.yar"
include "./malware/POS.yar"
include "./malware/APT_Equation.yar"
include "./malware/APT_Bestia.yar"
include "./malware/RAT_Cerberus.yar"
include "./malware/APT_Windigo_Onimiki.yar"
include "./malware/APT_Passcv.yar"
include "./malware/TOOLKIT_Gen_powerkatz.yar"
include "./malware/MALW_Magento_frontend.yar"
include "./malware/RAT_Meterpreter_Reverse_Tcp.yar"
include "./malware/RANSOM_DMALocker.yar"
include "./malware/MALW_Ponmocup.yar"
include "./malware/RAT_Terminator.yar"
include "./malware/RAT_Ratdecoders.yar"
include "./malware/MALW_CAP_HookExKeylogger.yar"
include "./malware/MALW_Iexpl0ree.yar"
include "./malware/RANSOM_Comodosec.yar"
include "./malware/Operation_Blockbuster/RomeoCharlie.yara"
include "./malware/Operation_Blockbuster/RomeoWhiskey.yara"
include "./malware/Operation_Blockbuster/LimaBravo.yara"
include "./malware/Operation_Blockbuster/WhiskeyCharlie.yara"
include "./malware/Operation_Blockbuster/KiloAlfa.yara"
include "./malware/Operation_Blockbuster/IndiaWhiskey.yara"
include "./malware/Operation_Blockbuster/SierraJuliettMikeTwo.yara"
include "./malware/Operation_Blockbuster/RomeoBravo.yara"
include "./malware/Operation_Blockbuster/LimaCharlie.yara"
include "./malware/Operation_Blockbuster/IndiaCharlie.yara"
include "./malware/Operation_Blockbuster/IndiaJuliett.yara"
include "./malware/Operation_Blockbuster/TangoBravo.yara"
include "./malware/Operation_Blockbuster/SierraBravo.yara"
include "./malware/Operation_Blockbuster/WhiskeyDelta.yara"
include "./malware/Operation_Blockbuster/RomeoEcho.yara"
include "./malware/Operation_Blockbuster/RomeoGolf_mod.yara"
include "./malware/Operation_Blockbuster/suicidescripts.yara"
include "./malware/Operation_Blockbuster/UniformAlfa.yara"
include "./malware/Operation_Blockbuster/cert_wiper.yara"
include "./malware/Operation_Blockbuster/IndiaHotel.yara"
include "./malware/Operation_Blockbuster/DeltaCharlie.yara"
include "./malware/Operation_Blockbuster/IndiaDelta.yara"
include "./malware/Operation_Blockbuster/WhiskeyBravo_mod.yara"
include "./malware/Operation_Blockbuster/UniformJuliett.yara"
include "./malware/Operation_Blockbuster/SierraJuliettMikeOne.yara"
include "./malware/Operation_Blockbuster/IndiaBravo.yara"
include "./malware/Operation_Blockbuster/WhiskeyAlfa.yara"
include "./malware/Operation_Blockbuster/RomeoHotel.yara"
include "./malware/Operation_Blockbuster/SierraCharlie.yara"
include "./malware/Operation_Blockbuster/TangoAlfa.yara"
include "./malware/Operation_Blockbuster/IndiaAlfa.yara"
include "./malware/Operation_Blockbuster/RomeoDelta.yara"
include "./malware/Operation_Blockbuster/LimaDelta.yara"
include "./malware/Operation_Blockbuster/PapaAlfa.yara"
include "./malware/Operation_Blockbuster/HotelAlfa.yara"
include "./malware/Operation_Blockbuster/RomeoAlfa.yara"
include "./malware/Operation_Blockbuster/general.yara"
include "./malware/Operation_Blockbuster/IndiaEcho.yara"
include "./malware/Operation_Blockbuster/LimaAlfa.yara"
include "./malware/Operation_Blockbuster/sharedcode.yara"
include "./malware/Operation_Blockbuster/SierraAlfa.yara"
include "./malware/Operation_Blockbuster/IndiaGolf.yara"
include "./malware/MALW_Naspyupdate.yar"
include "./malware/MALW_NetTraveler.yar"
include "./malware/MALW_LinuxMoose.yar"
include "./malware/APT_Poseidon_Group.yar"
include "./malware/MALW_T5000.yar"
include "./malware/APT_OpClandestineWolf.yar"
include "./malware/MALW_NionSpy.yar"
include "./malware/MALW_Intel_Virtualization.yar"
include "./malware/MALW_Vidgrab.yar"
include "./malware/APT_Prikormka.yar"
include "./malware/MALW_Skeleton.yar"
include "./malware/APT_LotusBlossom.yar"
include "./malware/APT_Grasshopper.yar"
include "./malware/MALW_Empire.yar"
include "./malware/MALW_Korlia.yar"
include "./malware/APT_Terracota.yar"
include "./malware/MALW_Boouset.yar"
include "./malware/APT_Dubnium.yar"
include "./malware/MALW_Surtr.yar"
include "./malware/MALW_Install11.yar"
include "./malware/MALW_LostDoor.yar"
include "./malware/RAT_FlyingKitten.yar"
include "./malware/MALW_DiamondFox.yar"
include "./malware/MALW_Genome.yar"
include "./malware/MALW_Kraken.yar"
include "./malware/MALW_KINS.yar"
include "./malware/RANSOM_.CRYPTXXX.yar"
include "./malware/RAT_PlugX.yar"
include "./malware/APT_Sphinx_Moth.yar"
include "./malware/MALW_CAP_Win32Inet.yara"
include "./malware/APT_Carbanak.yar"
include "./malware/MALW_viotto_keylogger.yar"
include "./malware/MALW_Miscelanea.yar"
include "./malware/RANSOM_777.yar"
include "./malware/MALW_Pony.yar"
include "./malware/RANSOM_Tox.yar"
include "./malware/MALW_Zegost.yar"
include "./malware/MALW_PittyTiger.yar"
include "./malware/RAT_Crimson.yar"
include "./malware/APT_Oilrig.yar"
include "./malware/MALW_AdGholas.yar"
include "./malware/MALW_Madness.yar"
include "./malware/APT_Unit78020.yar"
include "./malware/MALW_Grozlex.yar"
include "./malware/MALW_Citadel.yar"
include "./malware/TOOLKIT_FinFisher_.yar"
include "./malware/MALW_Alina.yar"
include "./malware/APT_Hikit.yar"
include "./malware/APT_fancybear_dnc.yar"
include "./malware/MALW_Urausy.yar"
include "./malware/MALW_Fareit.yar"
include "./malware/MALW_Scarhikn.yar"
include "./malware/RAT_Hizor.yar"
include "./malware/APT_Bluetermite_Emdivi.yar"
include "./malware/RAT_Shim.yar"
include "./malware/APT_EQUATIONGRP.yar"
include "./malware/APT_DeepPanda_Anthem.yar"
include "./malware/MALW_Sendsafe.yar"
include "./malware/MALW_Shamoon.yar"
include "./malware/MALW_Pyinstaller.yar"
include "./malware/APT_OpDustStorm.yar"
include "./malware/MALW_Hajime.yar"
include "./malware/APT_ThreatGroup3390.yar"
include "./malware/MALW_Sqlite.yar"
include "./malware/APT_Regin.yar"
include "./malware/APT_Platinum.yar"
include "./malware/MALW_LuckyCat.yar"
include "./malware/APT_NGO.yar"
include "./malware/MALW_Enfal.yar"
include "./malware/APT_Molerats.yar"
include "./malware/APT_Codoso.yar"
include "./malware/TOOLKIT_exe2hex_payload.yar"
include "./malware/MALW_Notepad.yar"
include "./malware/TOOLKIT_THOR_HackTools.yar"
include "./malware/RAT_Nanocore.yar"
include "./malware/MALW_BlackRev.yar"
include "./malware/MALW_Retefe.yar"
include "./malware/MALW_Glasses.yar"
include "./malware/MALW_Kelihos.yar"
include "./malware/MALW_Cythosia.yar"
include "./malware/MALW_MiniAsp3_mem.yar"
include "./malware/APT_APT9002.yar"
include "./malware/APT_APT17.yar"
include "./malware/MALW_Tedroo.yar"
include "./Packers/peid.yar"
include "./Packers/Javascript_exploit_and_obfuscation.yar"
include "./Packers/JJencode.yar"
include "./Packers/packer_compiler_signatures.yar"
include "./Packers/packer.yar"
include "./Exploit-Kits/EK_ZeroAcces.yar"
include "./Exploit-Kits/EK_BleedingLife.yar"
include "./Exploit-Kits/EK_Angler.yar"
include "./Exploit-Kits/EK_Sakura.yar"
include "./Exploit-Kits/EK_Blackhole.yar"
include "./Exploit-Kits/EK_Zeus.yar"
include "./Exploit-Kits/EK_Fragus.yar"
include "./Exploit-Kits/EK_Phoenix.yar"
include "./Exploit-Kits/EK_Eleonore.yar"
include "./Exploit-Kits/EK_Crimepack.yar"
include "./Exploit-Kits/EK_Zerox88.yar"
include "./CVE_Rules/CVE-2015-5119.yar"
include "./CVE_Rules/CVE-2012-0158.yar"
include "./CVE_Rules/CVE-2010-0805.yar"
include "./CVE_Rules/CVE-2013-0422.yar"
include "./CVE_Rules/CVE-2015-1701.yar"
include "./CVE_Rules/CVE-2016-5195.yar"
include "./CVE_Rules/CVE-2010-0887.yar"
include "./CVE_Rules/CVE-2010-1297.yar"
include "./CVE_Rules/CVE-2015-2426.yar"
include "./CVE_Rules/CVE-2015-2545.yar"
include "./CVE_Rules/CVE-2013-0074.yar"
include "./Antidebug_AntiVM/antidebug_antivm.yar"
include "./email/EMAIL_Cryptowall.yar"
include "./email/attachment.yar"
include "./email/image.yar"
include "./email/email_Ukraine_BE_powerattack.yar"
include "./email/urls.yar"
include "./email/bank_rule.yar"
include "./email/scam.yar"
include "./Webshells/WShell_THOR_Webshells.yar"
include "./Webshells/Wshell_fire2013.yar"
include "./Webshells/Wshell_ChineseSpam.yar"
include "./Webshells/WShell_PHP_Anuna.yar"
include "./Webshells/WShell_APT_Laudanum.yar"
include "./Webshells/WShell_PHP_in_images.yar"
include "./Malicious_Documents/Maldoc_CVE-2017-0199.yar"
include "./Malicious_Documents/Maldoc_UserForm.yar"
include "./Malicious_Documents/Maldoc_Contains_VBE_File.yar"
include "./Malicious_Documents/Maldoc_Dridex.yar"
include "./Malicious_Documents/Maldoc_APT_OLE_JSRat.yar"
include "./Malicious_Documents/Maldoc_PDF.yar"
include "./Malicious_Documents/Maldoc_Hidden_PE_file.yar"
include "./Malicious_Documents/Maldoc_malrtf_ole2link.yar"
include "./Malicious_Documents/Maldoc_VBA_macro_code.yar"
include "./Malicious_Documents/Maldoc_MIME_ActiveMime_b64.yar"
include "./Malicious_Documents/maldoc_somerules.yar"
include "./Crypto/base64.yar"
include "./Crypto/crypto_signatures.yar"
So I'm guessing that the method of pulling down rules on the eFa is broken...
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

I see what you mean. You appear to be right.

Needs investigation. I will look at it in a couple of days when I have time to debug this one, unless someone does it for me first.
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

pdwalker wrote: 17 May 2017 13:41 I see what you mean. You appear to be right.

Needs investigation. I will look at it in a couple of days when I have time to debug this one, unless someone does it for me first.
I can pull down the rules directory by installing git, etc. My guess is that I could either:
  • Include the index.yar file and pull all the rules into ClamAV with a single include
  • Perform the equivalent of find . *.yar -print and add that text block to the /etc/clamav-unofficial-sigs/master.conf file
I believe that git clone https://github.com/yara-rules/rules /var/lib/clamav/ will update the files and could be added to a cron job to perform an update. One of my concerns is that reading the announcement for clamav v0.99 http://blog.clamav.net/2015/11/clamav-0 ... osted.html, I found a mention that although yara rules are supported, "private" rules are not and you may not be allowed to reference other rules ( preempting the use of "include" directives? ).

I don't really know much about the workings of clamav, so I'm playing catch-up.
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

Additionally, there seems to be a problem in GitHub for the ClamAV-Unofficial-Sigs project:

https://github.com/extremeshok/clamav-u ... issues/133

Difficulty in pulling down files in subdirectories?
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

Okay, so I can pull down the index using wget with

Code: Select all

wget https://github.com/Yara-Rules/rules/raw/master/index.yar
That gets me the raw file such that:

Code: Select all

/*
Generated by Yara-Rules
On 08-05-2017
*/
include "./malware/APT_FVEY_ShadowBrokers_Jan17_Screen_Strings.yar"
include "./malware/RAT_Gh0st.yar"
include "./malware/POS_Mozart.yar"
include "./malware/APT_Mirage.yar"
include "./malware/MALW_Buzus_Softpulse.yar"
include "./malware/RANSOM_Satana.yar"
include "./malware/MALW_DirtJumper.yar"
include "./malware/APT_Backspace.yar"
include "./malware/APT_Sofacy_Jun16.yar"
include "./malware/MALW_Backoff.yar"
include "./malware/MALW_Furtim.yar"
include "./malware/RAT_Sakula.yar"
include "./malware/APT_Ke3Chang_TidePool.yar"
include "./malware/APT_WildNeutron.yar"
include "./malware/RANSOM_TeslaCrypt.yar"
include "./malware/RAT_Indetectables.yar"
include "./malware/TOOLKIT_Dubrute.yar"
include "./malware/RANSOM_Petya.yar"
...
testing a number of files in my home directory with clamscan -d /var/lib/clamav/index.yar *

Code: Select all

LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 19 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 71 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 95 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 119 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 142 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Mirai.yar line 166 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Odinaff.yar line 243 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_xDedic_marketplace.yar line 16 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grizzlybear_uscert.yar line 46 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_KeyBoy.yar line 70 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_KeyBoy.yar line 219 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_KeyBoy.yar line 237 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Shamoon_StoneDrill.yar line 27 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Elex.yar line 58 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Elex.yar line 76 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Elex.yar line 93 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Elex.yar line 110 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Turla_RUAG.yar line 266 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Batel.yar line 19 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/RANSOM_Stampado.yar line 127 empty string
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/RANSOM_Stampado.yar line 21 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoCharlie.yara line 36 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoWhiskey.yara line 36 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoWhiskey.yara line 64 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/LimaBravo.yara line 26 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/WhiskeyCharlie.yara line 49 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/KiloAlfa.yara line 74 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaWhiskey.yara line 45 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraJuliettMikeTwo.yara line 69 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoBravo.yara line 36 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/LimaCharlie.yara line 42 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaJuliett.yara line 54 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/TangoBravo.yara line 29 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraBravo.yara line 46 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraBravo.yara line 78 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/WhiskeyDelta.yara line 43 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoGolf_mod.yara line 35 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/UniformAlfa.yara line 26 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaHotel.yara line 22 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaDelta.yara line 29 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/WhiskeyBravo_mod.yara line 50 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraJuliettMikeOne.yara line 31 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaBravo.yara line 42 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaBravo.yara line 79 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/WhiskeyAlfa.yara line 47 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoHotel.yara line 48 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraCharlie.yara line 29 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoDelta.yara line 32 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/LimaDelta.yara line 45 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/HotelAlfa.yara line 26 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/RomeoAlfa.yara line 43 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaEcho.yara line 34 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/LimaAlfa.yara line 28 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 32 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 60 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 90 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 119 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 159 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 184 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 206 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 230 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 277 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/sharedcode.yara line 311 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/SierraAlfa.yara line 68 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/Operation_Blockbuster/IndiaGolf.yara line 29 undefined identifier "pe"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 59 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 23 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 30 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 37 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 44 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 51 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 58 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 65 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 72 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 79 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 86 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 93 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 100 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 107 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 114 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 121 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 128 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 135 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/APT_Grasshopper.yar line 142 undefined identifier "hash"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_Miscelanea.yar line 84 undefined identifier "uint16be"
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_AdGholas.yar line 15 out of space in lex_buf
LibClamAV Error: yyerror(): /var/lib/clamav/./malware/MALW_AdGholas.yar line 80 syntax error, unexpected $end, expecting _REGEXP_
LibClamAV Error: cli_loadyara: failed to parse rules file /var/lib/clamav/index.yar, error count 87
So... I need to parse each rule and determine if it uses a module ( not supported by ClamAV, or a global rule, or one of several other things ).

I wonder if it's actually easier to just install Yara and add that to MailScanner rather than trying to include ClamAV rules?
Last edited by stusmith on 17 May 2017 15:52, edited 1 time in total.
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

Good question. The clamav-unofficial-sigs project doesn't look very active.

Where is the "Yara" package? I don't actually know anything about it, so I don't know what it can actually do.
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

pdwalker wrote: 17 May 2017 15:34 Good question. The clamav-unofficial-sigs project doesn't look very active.

Where is the "Yara" package? I don't actually know anything about it, so I don't know what it can actually do.

Code: Select all

sudo yum search yara
https://securityintelligence.com/signat ... with-yara/
http://resources.infosecinstitute.com/y ... ware/#gref
http://virustotal.github.io/yara/

There is also:
https://github.com/ineedblood/postfix-yara but I'm still looking at the code to understand how it works. I haven't written mail filters before, but I imagine I can use the python implementation of policyd-spf for comparison to see roughly how it should behave and how to configure a wrapper for yara.

It seems like it's pretty fast - at least running it from the command line. Using yara directly would also solve the problem of not being able to use modules through ClamAV. ( hashes and PE )
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

So... I disabled all the rules throwing errors because of modules, globals, and undefined types.

AND discovered that Yara has a rule called .../drumroll ...YARA.contentis_base64.UNOFFICIAL .../rimshot

And then I remembered that I'd left index.yar in master.conf. 386 deleted email messages due to a "Virus".

SMH. This is why we don't test on production servers. /spins up new VM.
User avatar
pdwalker
Posts: 1553
Joined: 18 Mar 2015 09:16

Re: YARA

Post by pdwalker »

SMH. This is why we don't test on production servers. /spins up new VM.

Oh, right! *cough cough*
stusmith
Posts: 63
Joined: 27 Jan 2017 15:24

Re: YARA

Post by stusmith »

So, I still think that adding a Python wrapper around Yara is going to be the way to go. It looks like we'll want to look at yara-extend, as well. Let's Yara process compressed archives. Seems useful. The wrapper above is making more sense to me, after looking at the python-policyd-spf implementation. It's only about 800 or so lines, so not too bad to go through.

An interesting thing about the wrapper already written is that it decodes all the base64 encoding prior to feeding each part to yara. There's a python wrapper for yara itself, which would be super useful, but it doesn't seem happy coming to live on my eFa. I expect it's a python3 vs. python2.7 problem?

In the event that we use a python wrapper to decode the base64 mime parts prior to feeding them to yara ( or scanning them using the yara python module ), does it still seem useful to keep the base64 content rule in place? My assumption is that double-encoding something would be a simple ( script-kiddie ) method of obfuscating content. I vaguely remember an Outlook exploit with the preview pane that made use of double-encoding...
Post Reply