Rule Category Reorganization

Sometimes the new becomes old, sometimes the old becomes new.  But then sometimes, the old is just plain old.

Snort was started over 10 years ago.  Since the ability to write rules to Snort was added, its rules have been organized into categories in different files.  For years they have been added to and added to, but now there has come a time when the ruleset is a bit large that we want to and have to refocus it, add features, and clear the old cruft away.

That time is now.

What we are going to do over the next few months is slowly progress from our old categories (rule files) into a new set of categories.

Let me provide you a bit of background.  When we sat down and started discussing what a new ruleset would look like, we started thinking about what we wanted to accomplish with the ruleset.  What information we, as rule writers, wanted to convey to you, the end user.

How could we make your lives simpler and get more of value out of the rules.

What were our goals?

To communicate to the user the:

• Intent of the rule
• Impact of the rule
• Target of the attack which the rule covers
• Mechanism of attack that the rule covers
• Technique of the attack or attacker that the rule covers

This is going to fan out over several parts, two of which have happened already:

• File-identify

This rule category was created to be able to identify and track rules that deal with file (client) based attacks.  Centrally locating all rules that "set" flowbits based off of two things, the download method, and the detection of the file actually coming down through characteristics of the file itself.

We will continually expand in this category and have done so with every rule release.  All of these rules are enabled based on the "balanced" policy, which is now our standard default policy.  For other policies (for instance security) you should use a tool named PulledPork to fix everything for you.  The Sourcefire product also handles this "flowbit resolution" for you seamlessly, I say this to illustrate that Oinkmaster does not provide either of these functions unfortunately.

The second part of this project has been happening quietly in the background:

• Cleanup

This is an on-going project to modernize our ruleset with newer keywords and functionality that did not exist before.  For instance, adding file_data to all the rules that need it for better decoding, (making the IDS harder to evade) and pointer placement.

Standardized naming across all rule message names.  For instance, not using "Microsoft IE" or "IE" or "Windows IE", but instead adopting a the naming sequence "Microsoft Internet Explorer", and "Microsoft Windows".  "Apple QuickTime" instead of just plain "quicktime".  This allows you to search the ruleset easily and turn on what is in your network through the use of easy pcre inside of the enablesid.conf file using PulledPork.   We even went so far as to get rid of things like "from_server" and move to "to_client" inside of the flow statement for easier readability.

We started this cleanup back in October and since then we've averaged over 600 changes per rule release.

The third portion of this project is our biggest.  We call it:

• Rule re-categorization

Moving all rules from their old categories:

attack-responses.rules

backdoor.rules

bad-traffic.rules

blacklist.rules

botnet-cnc.rules

chat.rules

content-replace.rules

ddos.rules

deleted.rules

dns.rules

dos.rules

experimental.rules

exploit.rules

file-identify.rules

finger.rules

ftp.rules

icmp-info.rules

icmp.rules

imap.rules

info.rules

local.rules

misc.rules

multimedia.rules

mysql.rules

netbios.rules

nntp.rules

oracle.rules

other-ids.rules

p2p.rules

phishing-spam.rules

policy.rules

pop2.rules

pop3.rules

rpc.rules

rservices.rules

scada.rules

scan.rules

shellcode.rules

smtp.rules

snmp.rules

specific-threats.rules

spyware-put.rules

sql.rules

telnet.rules

tftp.rules

virus.rules

voip.rules

web-activex.rules

web-attacks.rules

web-cgi.rules

web-client.rules

web-coldfusion.rules

web-frontpage.rules

web-iis.rules

web-misc.rules

web-php.rules

x11.rules

Into a new set of categories.  The new set will not only make us more agile and able to add new categories as we expand in future years, more professional, but will also be simpler for the end user.

Obviously all the new categories are not created yet.  We have a base set of names that we are going to start with, but we know for a fact that once we get into moving these rules, new categories will need to be created.

For example, we are going to create a series of rules based off of protocols, so that you may enable or disable them based upon your network needs.  The rules contained in here will not be software related, but related to vulnerabilities in the protocol itself.

PROTOCOL-FTP (protocol-ftp.rules)

PROTOCOL-FINGER (protocol-finger.rules)

PROTOCOL-RSERVICES (protocol-rservices.rules)

PROTOCOL-ICMP (protocol-icmp.rules)

PROTOCOL-IMAP (protocol-imap.rules)

PROTOCOL-POP3 (protocol-pop3.rules)

PROTOCOL-RPC (protocol-rpc.rules)

PROTOCOL-SMTP (protocol-smtp.rules)

PROTOCOL-SNMP (protocol-snmp.rules)

PROTOCOL-OTHER (protocol-other.rules)

We'll create a series of categories based upon file type allowing us to consolidate all rules that cover attack vectors based on those file types from many different rule files into one:

FILE-PDF (file-pdf.rules)

FILE-OFFICE (file-office.rules)

FILE-IMAGE (file-image.rules)

FILE-MULTIMEDIA (file-multimedia.rules)

FILE-FLASH (file-flash.rules)

FILE-JAVA (file-java.rules)

FILE-EXECUTABLE (file-executable.rules)

FILE-OTHER (file-other.rules)

You notice that we always have an "OTHER" at the end of the series.  This is our category where rules that just don't fit in any of the other categories will go, until there is a sufficient amount of them to warrant their own category.

For example, PDF wasn't a gigantic threat 5-8 years ago.  Maybe 10 or so rules that covered PDF files.  Now it'll have it's own category as we have hundreds.

Along the way we'll be revisiting every single rule for syntax, cleanliness, and speed.  If there are some quick optimizations we can make, then we'll make them.  I made several just the planning process for the first category.

Speaking of the first category, the first one we'll be breaking out is POLICY.  When looking through the policy.rules category, it looks like it'll be broken out in the following:

POLICY-SOCIAL

POLICY-MULTIMEDIA

POLICY-OTHER

INDICATOR-OBFUSCATION

INDICATOR-COMPROMISE

PUA-TOOLBARS

PUA-P2P

SERVER-MAIL

FILE-PDF

FILE-OFFICE

FILE-FLASH

FILE-OTHER

The good news is, if you are using the Sourcefire product, PulledPork, or Oinkmaster, the vast majority of you should be unaffected.  These products will handle the transition just fine.  The only way you will be affected using PulledPork (or Oinkmaster's related tools) is if you use enablesid.conf or disablesid.conf to enable or disable entire categories of rules. We will also notify you of any changes that will be made to the snort.conf to incorporate these changes.  

With each category that we transition, we'll do a blog post about it so that you are well aware before, during, and after the transition.

We will not be deleting the old category names until they are completely empty and local.rules will stay as is.

There are two more parts to this project that will begin after this transition is complete, introducing new functionality and intelligence into the ruleset that exists absolutely no where else in any product.  These two projects involve a total revamp of the classification and priority system, and finally additions to the metadata system within the rules.  These modifications are in addition to the significant changes we have set for the Snort engine itself in 2012.

If there are any questions, please contribute to the thread on the Snort-users mailing list here:
https://lists.sourceforge.net/lists/listinfo/snort-users

An exciting year is here!  Stay tuned!