Firing up OpenAppID

The purpose of this post is to provide guidance to Snort users who would like to try out Snort 2.9.7.0 and the OpenAppID features that it comes with. It is not intended to guide the reader through setting up Snort from scratch, there are plenty of docs on how to set up Snort at http://www.snort.org/docs.

Snort

Before we can download the latest Snort source code and compile it, we have a new prerequisite to fulfill before we can compile Snort. You'll need to install Luajit, which is used to define application detectors with the new OpenAppID.

On my Ubuntu system this was as easy as the following command.

sudo apt-get install libluajit-5.1-2 libluajit-5.1-common libluajit-5.1-dev luajit

Now we’re all set to download and install the alpha version of Snort. At the time this was written that was 2.9.7.0 alpha (snort-2.9.7.0_alpha.tar.gz). Grab the latest source from snort.org’s Download Snort page. Then we can extract and build.

tar zxvf snort-2.9.7.0_alpha.tar.gz
cd snort-2.9.7.0.alpha
./configure --prefix=/usr/local/snort --enable-sourcefire --enable-open-appid
make
sudo make install
Note the new flag on the configure command. We are adding that switch to enable OpenAppID.

Application Detector Package

The next step is to download the Application Detector Package. This entirely new content contains what is at the heart of the OpenAppID feature. You can download it from the Download Snort page on snort.org (currently: snort-openappid-detectors.2014-02-22.187-0.tgz).

This package contains the Lua libraries for detecting applications, as well as the application detectors themselves.

sudo tar -xzf snort-openappid-detectors.2014-02-22.187-0.tgz -C /usr/local/snort

Once that is unpacked you will see your Snort directory now has an odp directory. Here you will see the appMapping.data file, which includes metadata about the application detectors. A simple way to explore what application detectors are included in this first release is to examine the second column of this file. If you’d like to see the list, try the following command.

cat appMapping.data | cut -f2

For example, if you would like to see what micro-applications that can be identified within Facebook, search for that data.

cat appMapping.data | cut -f2 | grep Facebook
Facebook Apps
Facebook
Facebook Chat
Facebook Comment
Facebook Read Email
Facebook Send Email
Facebook Status Update
Facebook search
Facebook event
Facebook post
Facebook video chat
Facebook message
Facebook video

You will also see the libs subdirectory, which contains the Lua libraries needed to run application detection. The port and lua subdirectories contain the applications detectors themselves. The port detectors are simple YAML files to patch port and application. The lua directory contains the bulk of the detectors. These lua scripts perform far more powerful detection.

Configure snort.conf

There are several changes to make in our snort.conf file. We must add the preprocessor command for OpenAppID and configure the output command.

Configure OpenAppID Preprocessor

Our next step is to add the configuration for the OpenAppID preprocessor to the snort.conf file. Find the lines for the reputation preprocessor. This should be followed by commented text for “Step 6.” Just after the reputation preprocessor and before Step 6 we will add another preprocessor setting.

preprocessor appid: app_stats_filename appstats-u2.log, \    
app_stats_period 60, \
app_detector_dir /usr/local/snort

This will turn on the OpenAppID preprocessor. The first step simply names the configuration file to which application statistics will be logged. The second gives the time period used to sample this data. Finally, you must point out the directory which contains the odp directory we extracted from the Open App ID Detector package.

Configure Output
The final step in configuring our snort.conf file will be to change the output command. Everyone should be using the Unified2 output module already, but there is a new command to add new the new event field for the application in use.

Look into Step 6 find the lines explaining the unified2 output type. In that section add the following line.

output unified2: filename snort.log, limit 128, appid_event_types

Testing Snort

Now let’s fire up Snort.

/usr/local/snort/bin/snort -c /usr/local/snort/etc/snort.conf -i eth#

Where eth# is which ever interface you will be monitoring with (e.g. eth0).

If there aren't any errors you will see the final line output by Snort indicating a successful startup, as “Commencing packet processing.” If you examine the output above that you will see a new section for AppID Configuration. This will include the configuration given to the preprocessor for reference.

To test the new deployment, fire up a web browser to test out application detection. For my test I used Chrome to visit cnn.com.

If you are still examining the console output of the Snort process you will see something to the effect of “Opening /var/log/snort/appstats-u2.log.1393807981 for output.” Ending the Snort process will also write what data has not yet been output.

Examining Application Statistics

When you downloaded the Snort package, it included a tools subdirectory. In here you will find a couple of programs for changing the unified2 binary format to readable text. In this case we will use the u2openappid program to print the application statistics that Snort is collecting for us.

u2openappid /var/log/snort/appstats-u2.log.1393807981
statTime="1393807860",appName="chrome",txBytes="6043",rxBytes="111267"
statTime="1393807860",appName="dns",txBytes="8708",rxBytes="38103"
statTime="1393807860",appName="http",txBytes="200399",rxBytes="1444070"
statTime="1393807860",appName="cnn.com",txBytes="198478",rxBytes="1557970"
statTime="1393807860",appName="doubleclick",txBytes="5543",rxBytes="2598"
statTime="1393807860",appName="truste",txBytes="1829",rxBytes="12208"
statTime="1393807860",appName="washington_time",txBytes="2210",rxBytes="1401"
statTime="1393807860",appName="turner_broadcas",txBytes="1785",rxBytes="1316"
statTime="1393807860",appName="moat",txBytes="5707",rxBytes="3453"

From this you can see a number of entries. In this case the above were generated from my visit to cnn.com. You can see that they share the same statTime. OpenAppID identifies the client application (Chrome), DNS request, protocol (http), and web app (cnn.com). The rest of the entries are other web apps embedded within CNN’s web page.

To further illustrate the data we can get form OpenAppID, here is the output from when I instead use Firefox to visit Netflix.

statTime="1393809840",appName="google",txBytes="3500",rxBytes="5171"
statTime="1393809840",appName="firefox",txBytes="11695",rxBytes="20818"
statTime="1393809840",appName="http",txBytes="11695",rxBytes="20818"
statTime="1393809840",appName="netflix",txBytes="1072",rxBytes="1262"
statTime="1393809840",appName="https",txBytes="18330",rxBytes="33984"
statTime="1393809840",appName="verisign",txBytes="1450",rxBytes="2668"

Happy Snorting! Check out the new stuff. There are a few new features to explore here.