Thursday, February 26, 2015

Snort 3.0's new http_inspect preprocessor!

One of the major undertakings for Snort 3.0 is developing a completely new HTTP inspector. It is incomplete right now but you can examine the work-in-progress. You can configure it by adding:
    new_http_inspect = {}
to your snort.lua configuration file. Or you can read it in the source code under src/service_inspectors/nhttp_inspect.

The classic HTTP preprocessor is still available in the alpha release as http_inspect. It’s probably the better choice for now if you just want to do some work and do not feel like experimenting. Be sure not to configure both old and new HTTP inspectors at the same time.

So why a new HTTP inspector?

For starters it is object-oriented. That’s good for us because we maintain this software. But it should also be really nice for open-source developers. You can make meaningful changes and additions to HTTP processing without having to understand the whole thing. In fact much of the new HTTP inspector’s knowledge of HTTP is centralized in a series of tables where it can be easily reviewed and modified. Many significant changes can be made just by updating these tables.

New_http_inspect is the first inspector written specifically for the new Snort 3.0 architecture. That provides access to one of the very best features of Snort 3.0: purely PDU-based inspection. Classic http_inspect processes HTTP messages, but even while doing so it is constantly aware of IP packets and how they divide up the TCP data stream. The same HTTP message might be processed differently depending on how the sender (bad guy) divided it up into IP packets.

New_http_inspect is free of this burden and can focus exclusively on HTTP. That makes it much more simple, easier to test, and less prone to false positives. It also greatly reduces the opportunity for adversaries to probe the inspector for weak spots by adjusting packet boundaries to disguise bad behavior.

Dealing solely with HTTP messages also opens the door for developing major new features. The new_http_inspect design supports true stateful processing. Want to ask questions that involve both the client request and the server response? Or different requests in the same session? These things are possible.

Another new feature on the horizon is HTTP/2 analysis. HTTP/2 derives from Google’s SPDY project and is in the process of being standardized. Despite the name, it is better to think of HTTP/2 not as a newer version of HTTP/1.1, but rather a separate protocol layer that runs under HTTP/1.1 and on top of TLS or TCP. It’s a perfect fit for the new Snort 3.0 architecture because a new HTTP/2 inspector would naturally output HTTP/1.1 messages but not any underlying packets. Exactly what the new_http_inspect wants to input.

New_http_inspect is taking a very different approach to HTTP header fields. Classic http_inspect divides all the HTTP headers following the start line into cookies and everything else. It normalizes the two pieces using a generic process and puts them in buffers that one can write rules against. There is some limited support for examining individual headers within the inspector but it is very specific.

The new concept is that every header should be normalized in an appropriate and specific way and individually made available for the user to write rules against it. If for example a header is supposed to be a date then normalization means put that date in a standard format.

There is still a great deal of work to be done to make all this happen. One major open area is what to do with all this power? Protocol processing is getting ahead of rule-writing capabilities.

What do you want the new_http_inspect to do for you? What kind of rules do you want to be able to write that are currently difficult or impossible? What correlations would you like to be able to examine? Send your ideas to the open source mailing lists. We take them very seriously.

Snort++ Update

Just pushed build 139 to github (snortadmin/snort3):
  • sync 297 http XFF, SWF, and PDF updates
  • additional http_inspect cleanup
  • documented gotcha regarding rule variable definitions in Lua

Snort Subscriber Rule Set Update for 02/26/2015

Just released:
Snort Subscriber Rule Set Update for 02/26/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 43 new rules and made modifications to 199 additional rules.

There were no changes made to the snort.conf in this release.

Talos's rule release:
Talos has added and modified multiple rules in the blacklist, browser-ie, browser-webkit, file-flash, file-image, file-other, file-pdf, malware-backdoor, malware-cnc, os-other, protocol-voip, pua-p2p, server-mysql and server-webapp rule sets to provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Wednesday, February 25, 2015

Possible Packet Loss issues during reassembly for Snort

William Parker was nice enough to write up this quick white paper on how to diagnose and remediate some possible packet loss issues found during reassembly for Snort.

I've posted the white paper on our documentation page on Snort.org (under Additional Resources) and here's a link to it directly.

Thanks Bill for always contributing great content for the site!

Snort OpenAppID Detectors have been updated!

An update has been released today for the Snort OpenAppID Detector content.

This release, build 233, includes

  • A total of 2,629 detectors.
  • This was a maintenance release with some minor fixes and improvements

Available now for download from our downloads page, we look forward to you downloading and using the new features of 2.9.7.0's OpenAppID preprocessor and sharing your experiences with the community.

The OpenAppID community has a mailing list specifically dedicated to the exchange and discussion of detector content.  Please visit the mailing lists page to sign up.

Tuesday, February 24, 2015

Snort Subscriber Rule Set Update for 02/24/2015

Just released:
Snort Subscriber Rule Set Update for 02/24/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 24 new rules and made modifications to 464 additional rules.

There were no changes made to the snort.conf in this release.

Talos's rule release:
Talos has added and modified multiple rules in the browser-firefox, browser-ie, browser-plugins, file-flash, file-identify, file-image, file-multimedia, file-office, file-other, malware-cnc, protocol-dns, pua-adware, server-iis, server-mail, server-other and server-webapp rule sets to provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Monday, February 23, 2015

Cisco touts OpenAppID for internal application traffic visibility

A great interview with Jason Brvenik about OpenAppId's growth.

On its one-year anniversary, use of Cisco's OpenAppID application-layer traffic-detection tool is still modest compared to Snort, but the networking giant says it can help enterprises improve internal application traffic visibility.

Check out the article!

Friday, February 20, 2015

Snort++ Update

Just pushed build 138 to github (snortadmin/snort3):
  • sync ftp with 297
  • replace stream event callbacks with FlowData virtuals

Thursday, February 19, 2015

Snort Subscriber Rule Set Update for 02/19/2015

Just released:
Snort Subscriber Rule Set Update for 02/19/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 17 new rules and made modifications to 5 additional rules.

There were no changes made to the snort.conf in this release.

Talos would like to thank the following individuals for their contributions, their rules are included in the Community Ruleset:

Avery Tarasov
33560

Yaser Mansour
33547


Talos's rule release:
Talos has added and modified multiple rules in the blacklist, browser-plugins, file-flash, file-office, malware-cnc and server-other rule sets to provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Tuesday, February 17, 2015

Snort Subscriber Rule Set Update for 02/17/2015

Just released:
Snort Subscriber Rule Set Update for 02/17/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 68 new rules and made modifications to 327 additional rules.

There were no changes made to the snort.conf in this release.

Talos would like to thank the following individuals for their contributions, their rules are included in the Community Ruleset:

James Lay
33513

Avery Tarasov
33519
33520

Yaser Mansour
33522
33523
33524


Talos's rule release:
Talos has added and modified multiple rules in the blacklist, browser-firefox, browser-ie, browser-plugins, file-flash, file-identify, file-image, file-java, file-multimedia, file-office, file-other, file-pdf, indicator-compromise, indicator-obfuscation, indicator-scan, malware-cnc, malware-other, os-windows, policy-other, protocol-scada, pua-adware, pua-toolbars, server-mail, server-other and sql rule sets to provide coverage for emerging threats from these technologies.

In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Snort++ Nets and Ports

One of the fundamental differences between Snort and Snort++ concerns configuration related to networks and ports.  Here is a brief review of Snort's configuration for network and service related components:
  • Snort's configuration has a default policy and optional policies selected by VLAN or network (with config binding).
  • Each policy contains a user defined set of preprocessor configurations.
  • Each preprocessor has a default configuration and some support non-default configurations selected by network.
  • Most preprocessors have port configurations.
  • The default policy may also contain a list of ports to ignore.
In Snort++, the above configurations are done in a single module called the binder.  Here is an example:

binder =
{
    -- allow all tcp port 22:
    -- (similar to snort 2.X config ignore_ports)
    { when = { proto = 'tcp', ports = '22' }, use = { action = 'allow' } },

    -- select a config file by vlan
    -- (similar to snort 2.X config binding by vlan)
    { when = { vlans = '1024' }, use = { file = 'vlan.lua' } },

    -- use a non-default HTTP inspector for port 8080:
    -- (similar to a snort 2.X targeted preprocessor config)
    { when = { nets = '192.168.0.0/16', proto = 'tcp', ports = '8080' },
      use = { name = 'alt_http', type = 'http_inspect' } },

    -- use the default inspectors:
    -- (similar to a snort 2.X default preprocessor config)
    { when = { proto = 'tcp' }, use = { type = 'stream_tcp' } },
    { when = { service = 'http' }, use = { type = 'http_inspect' } },

    -- figure out which inspector to run automatically:
    { use = { type = 'wizard' } }
}

Bindings are evaluated when a session starts and again if and when service is identified on the session.  Essentially, the bindings are a list of when-use rules evaluated from top to bottom.  The first matching network and service configurations are applied.  binder.when can contain any
combination of criteria and binder.use can specify an action, config file, or inspector configuration.

Using the wizard enables port-independent configuration and the detection of malware command and control channels.  If the wizard is bound to a session, it peeks at the initial payload to determine the service.  For example, 'GET' would indicate HTTP and 'HELO' would indicate SMTP.  Upon
finding a match, the service bindings are reevaluated so the session can be handed off to the appropriate inspector.  The wizard is still under development; if you find you need to tweak the defaults please let us know.

Additional Details:
  • If the wizard and one or more service inspectors are configured w/o explicitly configuring the binder, default bindings will be generated which should work for most common cases.
  • Also note that while Snort bindings can only be configured in the default policy, each Snort++ policy can contain a binder leading to an arbitrary hierarchy.
  • The entire configuration can be reloaded and hot-swapped during run-time via signal or command in both Snort and Snort++.  Ultimately, Snort++ will support commands to update the binder on the fly, thus enabling incremental reloads of individual inspectors.
  • Both Snort and Snort++ support server specific configurations via a hosts table (XML in Snort and Lua in Snort++).  The table allows you to map network, protocol, and port to a service and policy.  This table can be reloaded and hot-swapped separately from the config file.
  • You can find the specifics on the binder, wizard, and hosts tables in the manual or command line like this:  snort --help-module binder, etc.

Thursday, February 12, 2015

Snort Subscriber Rule Set Update for 02/12/2015

Just released:
Snort Subscriber Rule Set Update for 02/12/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 41 new rules and made modifications to 12 additional rules.

There were no changes made to the snort.conf in this release.

Talos would like to thank the following individuals for their contributions, their rules are included in the Community Ruleset:

Avery Tarasov
33443
33444
33449
33450
33453
33457

Talos's rule release:
Talos has added and modified multiple rules in the blacklist, browser-plugins, file-flash, file-multimedia, file-office, file-other, malware-cnc, protocol-telnet, protocol-voip, pua-toolbars and server-webapp rule sets to provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Snort++ Update

Just pushed build 137 to github (snortadmin/snort3):

  • updated manual from blog posts and emails
  • normalization refactoring, renaming
  • fixed icmp4 encoding
  • methods in codec_events and ip_util namespaces are now protected Codec
  • methods
  • 297 sync of active and codecs


Tuesday, February 10, 2015

Snort Subscriber Rule Set Update for 02/10/2015, MSTuesday

Just released:
Snort Subscriber Rule Set Update for 02/10/2015


We welcome the introduction of the newest rule release from Talos. In this release we introduced 127 new rules and made modifications to 18 additional rules.

There were no changes made to the snort.conf in this release.


Talos's rule release:
Synopsis: Talos is aware of vulnerabilities affecting products from Microsoft
Corporation.

Details:
Microsoft Security Bulletin MS15-009:
Microsoft Internet Explorer suffers from programming errors that may
lead to remote code execution.

Rules to detect attacks targeting these vulnerabilities are included in
this release and are identified with GID 1, SIDs 33312 through 33325,
33331 through 33338, 33340 through 33341, 33345 through 33349, 33352
through 33354, 33356 through 33361, 33365 through 33366, and 33412
through 33428.

Microsoft Security Bulletin MS15-010:
A coding deficiency exists in the Microsoft Windows Kernel Mode Driver
that may lead to remote code execution.

Rules to detect attacks targeting these vulnerabilities are included in
this release and are identified with GID 1, SIDs 33343 through 33344,
33355, 33363 through 33364, and 33436 through 33437.

Microsoft Security Bulletin MS15-012:
A coding deficiency exists in Microsoft Office that may lead to remote
code execution.

Rules to detect attacks targeting these vulnerabilities are included in
this release and are identified with GID 1, SIDs 33350 through 33351
and 33362.

Microsoft Security Bulletin MS15-014:
A coding deficiency exists in Microsoft SMB that lead to remote code
execution.

A rule to detect attacks targeting this vulnerability is included in
this release and is identified with GID 1, SID 33429.

Talos has added and modified multiple rules in the app-detect,
blacklist, browser-ie, exploit-kit, file-flash, file-office,
file-other, file-pdf, indicator-shellcode, malware-cnc, os-linux,
os-other, os-windows, protocol-scada and server-other rule sets to
provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Friday, February 6, 2015

Building a Snort3.0 Codec

The Snort3.0 Codecs decipher raw packets. These Codecs are now completely pluggable; almost every Snort3.0 Codec can be built dynamically and replaced with an alternative, customized Codec. The pluggable nature has also made it easier to build new Codecs for protocols without having to touch the Snort3.0 code base.

The first step in creating a Codec is defining its class and protocol. Every Codec must inherit from the Snort3.0 Codec class defined in "framework/codec.h". The following is an example Codec named "example" and has an associated struct that is 14 bytes long.

#include <cstdint>
#include <arpa/inet.h>
#include “framework/codec.h”
#include "main/snort_types.h"

#define EX_NAME “example”
#define EX_HELP “example codec help string”

struct Example
{
    uint8_t dst[6];
    uint8_t src[6];
    uint16_t ethertype;

    static inline uint8_t size()
    { return 14; }
}

class ExCodec : public Codec
{
public:
    ExCodec() : Codec(EX_NAME) { }
    ~ExCodec() { }

    bool decode(const RawData&, CodecData&, DecodeData&) override;
    void get_protocol_ids(std::vector<uint16_t>&) override;

}; /* class ExCodec : public Codec */

After defining ExCodec, the next step is adding the Codec's decode functionality. The function below does this by implementing a valid decode function. The first parameter, which is the RawData struct, provides both a pointer to the raw data that has come from a wire and the length of that raw data. The function takes this information and validates that there are enough bytes for this protocol. If the raw data's length is less than 14 bytes, the function returns false and Snort3.0 discards the packet; the packet is neither inspected nor processed. If the length is greater than 14 bytes, the function populates two fields in the CodecData struct, next_prot_id and lyr_len. The lyr_len field tells Snort3.0 the number of bytes that this layer contains. The next_prot_id field provides Snort3.0 the value of the next EtherType or IP protocol number.

bool ExCodec::decode(const RawData& raw, CodecData& codec, DecodeData&)
{
   if ( raw.len < Example::size() )
         return false;

    const Example* const ex = reinterpret_cast<const Example*>(raw.data);
    codec.next_prot_id = ntohs(ex->ethertype);
    codec.lyr_len = ex->size();
    return true;
}

For instance, assume this decode function receives the following raw data with a validated length of 32 bytes:

    00 11 22 33 44 55 66 77    88 99 aa bb 08 00 45 00
    00 38 00 01 00 00 40 06    5c ac 0a 01 02 03 0a 09

The Example struct's EtherType field is the 13 and 14 bytes. Therefore, this function tells Snort that the next protocol has an EtherType of 0x0800. Additionally, since the lyr_len is set to 14, Snort knows that the next protocol begins 14 bytes after the beginning of this protocol. The Codec with EtherType 0x0800, which happens to be the IPv4 Codec, will receive the following data with a validated length of 18 ( == 32 – 14):

    45 00 00 38 00 01 00 00    40 06 5c ac 0a 01 02 03
    0a 09

How does Snort3.0 know that the IPv4 Codec has an EtherType of 0x0800? The Codec class has a second virtual function named get_protocol_ids(). When implementing the function, a Codec can register for any number of values between 0x0000 - 0xFFFF. Then, if the next_proto_id is set to a value for which this Codec has registered, this Codec's decode function will be called. As a general note, the protocol ids between [0, 0x00FF] are IP protocol numbers, [0x0100, 0x05FF] are custom types, and [0x0600, 0xFFFF] are EtherTypes.

For example, in the get_protocol_ids function below, the ExCodec registers for the protocols numbers 17, 787, and 2054. 17 happens to be the protocol number for UDP while 2054 is ARP's EtherType. Therefore, this Codec will now attempt to decode UDP and ARP data. Additionally, if any Codec sets the next_protocol_id to 787, ExCodec's decode function will be called. Some custom protocols are already defined in the file "protocols/protocol_ids.h"

void ExCodec::get_protocol_ids(std::vector<uint16_t>&v)
{
    v.push_back(0x0011); // == 17  == UDP
    v.push_back(0x1313); // == 787  == custom
    v.push_back(0x0806); // == 2054  == ARP
}

To register a Codec for Data Link Type's rather than protocols, the function get_data_link_type() can be similarly implemented.

The final step to creating a pluggable Codec is the snort_plugins array. This array is important because when Snort3.0 loads a dynamic library, the program only find plugins that are inside the snort_plugins array. In other words, if a plugin has not been added to the snort_plugins array, that plugin will not be loaded into Snort3.0.

Although the details will not be covered in this post, the following code snippet is a basic CodecApi that Snort3.0 can load. This snippet can be copied and used with only three minor changes. First, in the function ctor, ExCodec should be replaced with the name of the Codec that is being built. Second, EX_NAME must match the Codec's name or Snort will be unable to load this Codec. Third, EX_HELP should be replaced with the general description of this Codec. Once this code snippet has been added, ExCodec is ready to be compiled and plugged into Snort3.0.

static Codec* ctor(Module*)
{ return new ExCodec; }

static void dtor(Codec *cd)

{ delete cd; }

static const CodecApi ex_api =

{
    {
        PT_CODEC,
        EX_NAME,
        EX_HELP,
        CDAPI_PLUGIN_V0,
        0,
        nullptr,
        nullptr,
    },
    nullptr, // pointer to a function called during Snort's startup.
    nullptr, // pointer to a function called during Snort's exit.
    nullptr, // pointer to a function called during thread's startup.
    nullptr, // pointer to a function called during thread's destruction.
    ctor, // pointer to the codec constructor.
    dtor, // pointer to the codec destructor.
};

SO_PUBLIC const BaseApi* snort_plugins[] =

{
    &ex_api.base,
    nullptr
};

Two example Codecs are available in the extra directory on git and the extra tarball on the Snort3.0 page. One of those examples is the Token Ring Codec while the other example is the PIM Codec.

As a final note, there are four more virtual functions that a Codec should implement: encodeformatupdate, and log. If the functions are not implemented Snort will not throw any errors. However, Snort may also be unable to accomplish some of its basic functionality.
  • encode is called whenever Snort actively responds and needs to builds a packet, i.e. whenever a rule using an IPS ACTION like react, reject, or rewrite is triggered. This function is used to build the response packet protocol by protocol.
  • format is called when Snort is rebuilding a packet. For instance, every time Snort reassembles a TCP stream or IP fragment, format is called. Generally, this function either swaps any source and destination fields in the protocol or does nothing.
  • update is similar to format in that it is called when Snort is reassembling a packet. Unlike format, this function only sets length fields.
  • log is called when either the log_codecs logger or a custom logger that calls PacketManager::log_protocols is used when running Snort3.0.


Happy (de)coding!

Snort++ Update

Pushed build 136 to github (snortadmin/snort3):
  • misc encoder bug fixes
  • sync stream with 297
  • sync normalizatons with 297

Thursday, February 5, 2015

OpenAppID Detection Webinar has been posted!

Hello All,

Thanks to those who attended the successful webinar in the East Coast this morning! Special thanks to the Developer himself, Costas Kleopa and his team member Cliff Judge. It was a great turnout and we look forward to having many more OpenAppID webinars in the near future. Thank you for all of your support.

The slides are available under the Webinar section on our documents page on Snort.org, or directly available via this link.


You can access the recorded piece of the presentation by clicking the link below:

Streaming recording link:
https://cisco.webex.com/ciscosales/lsr.php?RCID=f62d3d62d68067863ff920bca19f2697


Snort Subscriber Rule Set Update for 02/05/2015

Just released:
Snort Subscriber Rule Set Update for 02/05/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 22 new rules and made modifications to 5 additional rules.

There were no changes made to the snort.conf in this release.

Talos would like to thank the following individuals for their contributions, their rules are included in the Community Ruleset:

Yaser Mansour
32976
32977
33281
33282

Talos's rule release:
Talos has added and modified multiple rules in the blacklist, exploit-kit, file-flash, file-other, malware-cnc and server-webapp rule sets to provide coverage for emerging threats from these technologies.


In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!

Tuesday, February 3, 2015

Snort Subscriber Rule Set Update for 02/03/2015

Just released:
Snort Subscriber Rule Set Update for 02/03/2015

We welcome the introduction of the newest rule release from Talos. In this release we introduced 14 new rules and made modifications to 2 additional rules.

There were no changes made to the snort.conf in this release.

Talos's rule release:
Talos has added and modified multiple rules in the blacklist, browser-ie, exploit-kit, malware-cnc and server-webapp rule sets to provide coverage for emerging threats from these technologies.

In order to subscribe now to Talos's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at https://www.snort.org/products. Make sure and stay up to date to catch the most emerging threats!