Bro Agent for Sguil - Now supports Intel.log

Post date: Oct 29, 2014 6:06:06 PM

Intro

I created the Bro agent for Sguil so that I could insert certain events from Bro (the notice log) into my workflow and access them via squert. This puts these events in a nice spot for perusal, allows correlation with the other data sources I can access from the interface and also gives me the ability to pull a transcript if need be. 

The stock notice.log has been great as both a complimentary piece and standalone source however this week I started playing with the Bro Intel Framework and Wow; opportunities galore! 

Inspired by the release of Mandiant's APT28 report INDICATORS! and propelled by the help from the folks in #snort-gui I managed to get things up and running. More importantly though, and the reason for this post is I also modified the Bro Agent so that it can work with both logs simultaneously.

Keep in mind, this isn't about logs but events. This agent is not supposed to be processing everything Bro is logging, it simply acts as a tripwire.

Setup

I am going to be very generic here and use a very simple (perhaps not the best) example: a domain blacklist. I will use the one from here: http://mirror1.malwaredomains.com/files/justdomains

This file contains a bunch of entries like:

38zu.cn

brenz.pl

gumblar.cn

blog-salopes.com

forum-cs.net76.net

iseyh.com

To use them with the Intel Framework, they need to be formatted like this:

#fields    indicator    indicator_type    meta.source    meta.do_notice

38zu.cn    Intel::DOMAIN    Malware Domains    F

brenz.pl    Intel::DOMAIN    Malware Domains    F

Note: meta.do_notice is set to false (F) because I don't need to see these in the notice log as well. The agent will skip Intel entries it finds in the notice.log so you don't get duplicates.

Very Important: field separators are a single tab, this rule applies in the header as well. Bro is very picky about this format (I wasted a lot of time figuring this out). If things aren't working, check your reporter.log.

To achieve the required format I just did this:

~$ fetch http://mirror1.malwaredomains.com/files/justdomains

~$ awk '{print $1"\011""Intel::DOMAIN""\011""Malware Domains""\011""F"}' justdomains > intel_domains.dat

Lastly, I just needed to add these lines to my local.bro:

# Intel Framework

@load frameworks/intel/seen

@load frameworks/intel/do_notice

redef Intel::read_files += { 

  "/usr/local/bro/share/bro/site/intel_domains.dat",

  "/usr/local/bro/share/bro/site/intel_emails.dat",

  "/usr/local/bro/share/bro/site/intel_apt28.dat",

  "/usr/local/bro/share/bro/site/intel_apt1-certs.dat",

  "/usr/local/bro/share/bro/site/intel_apt1-fqdn.dat",

  "/usr/local/bro/share/bro/site/intel_apt1-md5.dat"

};

Take a look here: http://blog.bro.org/2014/01/intelligence-data-and-bro_4980.html   for more details on the Intel Framework, options and fields. 

The Agent

You need a working sguil and bro install to use this agent. This is trivial to setup if you are running  Securityonion

Get the agent:

~$ git clone https://github.com/int13h/bro_agent

Configure the agent:

~$ cp bro_agent.conf.sample bro_agent.conf

Edit the settings in the conf file to match your sguil setup, it is well documented. There are some other options in there as well if you wish to change the event priorities and classifications.

Lastly, fire it up:

~$ bro_agent.tcl -c bro_agent.conf -f "path_to_notice.log path_to_intel.log"

Results

Here we can see the events from both the intel and notice logs:

The payload looks like this:

And here's a transcript:

I think this is a great addition to the existing data; adding one more piece of context or perhaps even revealing gaps in coverage. I am pretty excited to start pouring more Intel into this and see what I can net.