Better application logging with Snort3

By Costas Kleopa.

With the introduction of OpenAppID in SNORT®, we started to provide application-based information for our network flows. A user could enable the AppID preprocessor, load our Open Detector Package (snort-openappid.tgz) from the Snort Downloads page and — with the integration of any third-party tools — we could provide a deeper graphical representation of what’s running over a network. (See the blog here for an example showing Integration with Splunk.) The app_stats logging configuration allowed us to report some basic statistics on what type of traffic we can see per application and the overall traffic size we see during a specific recurring time interval.  

We also provide additional AppID-based control via the IPS rules. These IPS rules were allowing us to block/alert the actual application and ultimately log this information on a per-packet basis. The combination of alert/logging in IPS rules partially met a use case that the field has been asking for, which is logging the application per connection. Unfortunately, this was not the best solution, since this was causing us to report this information per packet and could cause some performance issues with a lot of duplicate data. 

With Snort 3, we introduced a new capability that allows us to overcome this limitation. We have created a new plugin/inspector to work on top of Snort 3 called the appid_listener. This new inspector is part of our snort3_extra repository which is a collection of extra plugins that can enhance the overall capabilities of Snort 3 for each user’s needs. 

Installation

 

To download and install the code, follow the instructions of the README file on how to compile and build the plugins. 

 

Configuration

 

Use the following steps and configurations to enable the appid_listener in Snort 3.

Include the following in your Snort 3 Lua configuration to enable appid_listener: 

appid_listener = { }

 

Running Examples

 

In the example below, we wanted to log our connection in the output for debugging reasons, as well as exporting each flow into a JSON format, log output.  

As a result, in our appid_listener.lua file, we added the following configurations: 

appid =

{

    app_detector_dir = "~/appid_listener_demo",

}

 

appid_listener =

{

    json_logging = true,

    file = "~/appid_listener_demo/appid-output.log",

}

 

Snort3 command used for the example:

snort -c ~/appid_listener_demo/appid_listener.lua --daq-dir=[daqs-folders] --plugin-path=[plugin-paths] -i eth0 -z 1 -k none

 

The following logs are some examples of the appid-output.log of these connections:

 

{ "session_num": "0.42", "pkt_time": "2020-10-08 13:18:11.967790", "client_ip": "10.10.1.165", "client_port": 36512, "server_ip": "151.101.66.167", "server_port": 80, "proto": "TCP", "packet_num": 2503, "apps": { "service": "HTTP", "client": "Wget", "payload": "TwitchTV", "misc": null, "referred": null }, "tls_host": null, "dns_host": null, "http": { "http2_stream": null, "host": "twitch.tv", "url": "http://twitch.tv/", "user_agent": "Wget/1.20.3 (linux-gnu)", "response_code": null, "referrer": null, "client_version": "1.20.3 (linux-gnu)" } }

{ "session_num": "0.43", "pkt_time": "2020-10-08 13:18:12.163964", "client_ip": "10.10.1.165", "client_port": 60042, "server_ip": "151.101.66.167", "server_port": 443, "proto": "TCP", "packet_num": 2527, "apps": { "service": "HTTPS", "client": "SSL client", "payload": "TwitchTV", "misc": null, "referred": null }, "tls_host": "twitch.tv", "dns_host": null, "http": { "http2_stream": null, "host": null, "url": null, "user_agent": null, "response_code": null, "referrer": null, "client_version": null } }

{ "session_num": "0.5", "pkt_time": "2020-10-08 13:16:25.718023", "client_ip": "10.10.1.165", "client_port": 54986, "server_ip": "13.249.78.230", "server_port": 443, "proto": "TCP", "packet_num": 28, "apps": { "service": "HTTPS", "client": null, "payload": "unknown", "misc": null, "referred": null }, "tls_host": "static.twitchcdn.net", "dns_host": null, "http": { "http2_stream": null, "host": null, "url": null, "user_agent": null, "response_code": null, "referrer": null, "client_version": null } }

Elasticsearch integration

When we integrate these JSON-formatted logs with Elasticsearch, we can visualize data like so:

Custom application detectors

One of the things worth pointing out here is that we are missing one of the HTTPS patterns for Twitch TV. To fix this, we can run the appid_detector_builder.sh script under snort3/tools to create our own custom detector.  

See the example below: 

[snort@snort3demo tools]# ./appid_detector_builder.sh 

Snort Application Id - Detector Creation Tool

 

Enter below, the AppId string to be associated with the Detector.

(e.g. "CNN.com", "Yahoo!", "Avira Download/Update", etc.)

AppId strings MUST NOT INCLUDE tab, backslash, apostrophe, or double-quote.

 

Enter AppId string: TwitchTV

 

Enter its optional description: 

 

Detection Protocol:

1) TCP

2) UDP

3) HTTP

4) SSL

5) SIP

6) RTMP

Selection: 4

SSL Pattern Type:

1) Host

2) Common Name

3) Organizational Unit

Selection: 1

Enter Host pattern: twitchcdn.net

 

Choose "Save Detector" or choose an additional Detection Protocol:

1) Save Detector

2) TCP

3) UDP

4) HTTP

5) SSL

6) SIP

7) RTMP

Selection: 1

Successfully completed construction of:

   /home/vdber/snort3/tools/TwitchTV.lua

When you add the .lua file, the AppId,

   "TwitchTV",

   will be the name reported as detected.

 

[snort@snort3demo tools]#

 

 

At this point, we have a new detector created into the file TwitchTV.lua.  

The next step is to create a new folder under our detector’s directory in which we can copy this new file there for Snort to load. 

The folder that accepts custom detectors will need to be called “custom/lua.” 

Note: The app_detector_dir was set to "~/appid_listener_demo" in the configuration. 

  

 

[snort@snort3demo tools]# mkdir -p ~/appid_listener_demo/custom/lua

[snort@snort3demo tools]# cp TwitchTv.lua ~/appid_listener_demo/custom/lua

 

 

When running the same tests again with Snort, you will now see Twitch TV being correctly identified with this traffic: 

If you're interested about learning more about Snort3 or its application detection capabilities through OpenAppID, feel free to check out the rest of our blogs and subscribe to the Snort OpenAppID Mailing list to participate in the discussion.