Thursday, February 26, 2015

Snort 3.0's new http_inspect preprocessor!

One of the major undertakings for Snort 3.0 is developing a completely new HTTP inspector. It is incomplete right now but you can examine the work-in-progress. You can configure it by adding:
    new_http_inspect = {}
to your snort.lua configuration file. Or you can read it in the source code under src/service_inspectors/nhttp_inspect.

The classic HTTP preprocessor is still available in the alpha release as http_inspect. It’s probably the better choice for now if you just want to do some work and do not feel like experimenting. Be sure not to configure both old and new HTTP inspectors at the same time.

So why a new HTTP inspector?

For starters it is object-oriented. That’s good for us because we maintain this software. But it should also be really nice for open-source developers. You can make meaningful changes and additions to HTTP processing without having to understand the whole thing. In fact much of the new HTTP inspector’s knowledge of HTTP is centralized in a series of tables where it can be easily reviewed and modified. Many significant changes can be made just by updating these tables.

New_http_inspect is the first inspector written specifically for the new Snort 3.0 architecture. That provides access to one of the very best features of Snort 3.0: purely PDU-based inspection. Classic http_inspect processes HTTP messages, but even while doing so it is constantly aware of IP packets and how they divide up the TCP data stream. The same HTTP message might be processed differently depending on how the sender (bad guy) divided it up into IP packets.

New_http_inspect is free of this burden and can focus exclusively on HTTP. That makes it much more simple, easier to test, and less prone to false positives. It also greatly reduces the opportunity for adversaries to probe the inspector for weak spots by adjusting packet boundaries to disguise bad behavior.

Dealing solely with HTTP messages also opens the door for developing major new features. The new_http_inspect design supports true stateful processing. Want to ask questions that involve both the client request and the server response? Or different requests in the same session? These things are possible.

Another new feature on the horizon is HTTP/2 analysis. HTTP/2 derives from Google’s SPDY project and is in the process of being standardized. Despite the name, it is better to think of HTTP/2 not as a newer version of HTTP/1.1, but rather a separate protocol layer that runs under HTTP/1.1 and on top of TLS or TCP. It’s a perfect fit for the new Snort 3.0 architecture because a new HTTP/2 inspector would naturally output HTTP/1.1 messages but not any underlying packets. Exactly what the new_http_inspect wants to input.

New_http_inspect is taking a very different approach to HTTP header fields. Classic http_inspect divides all the HTTP headers following the start line into cookies and everything else. It normalizes the two pieces using a generic process and puts them in buffers that one can write rules against. There is some limited support for examining individual headers within the inspector but it is very specific.

The new concept is that every header should be normalized in an appropriate and specific way and individually made available for the user to write rules against it. If for example a header is supposed to be a date then normalization means put that date in a standard format.

There is still a great deal of work to be done to make all this happen. One major open area is what to do with all this power? Protocol processing is getting ahead of rule-writing capabilities.

What do you want the new_http_inspect to do for you? What kind of rules do you want to be able to write that are currently difficult or impossible? What correlations would you like to be able to examine? Send your ideas to the open source mailing lists. We take them very seriously.