Showing posts with label downloads. Show all posts
Showing posts with label downloads. Show all posts

Wednesday, October 26, 2016

Crontabs, and how to fix them

In previous blog entires you've heard me talk about the need to stagger your crontabs to lighten the load on Snort.org at certain times of the day.

We've taken the liberty of creating a section on the oinkcode page about how to configure your crontab.

If you log into Snort.org, click on your user account email address (found at the top right of the page).

Navigate to "Oinkcode" on the left hand side:



Follow the link to:



You will see instructions for how to use your oinkcode, however, we've added a new section under:



This will give you some default syntax for your crontab entry, along with a randomized time (it changes every time you refresh the page, so you can actually place a different time on all your sensors if you so choose) for pulledpork to execute.

Please replace your crontab entry with one of our randomized times from the website, and that should lower the loads on downloads.

Thanks!

Thursday, June 23, 2016

Snort Rule Downloads, Crontabs, and you.

At Snort we have an extensive amount of monitoring taking place to make sure the health of Snort.org is as optimal as we can make it.

One of the things we monitor is response time, or how long it takes, from the time your browser requests Snort.org, to the time we fulfill the entire page or whatever is being loaded.  We strive for a sub-100ms response time.

We'd like to go faster, but look, this is reality, nothing is perfect, and Snort is a very complex beast.

Setting aside the millions of hits a day at Snort.org gets, lets concentrate on the people that have PulledPork and Oinkmaster checking for new rules, automatically, in a crontab.  We have nearly 500,000 PulledPork requests a day, and this "GET" request is very quick.  Since we generally release rule packs on Tuesdays and Thursdays, most of the people hitting Snort.org for the md5 of the rulepack, find out the md5 hasn't changed, and move on.

Unless of course, we deploy a new rule pack, that md5 changes, then you grab the full rule pack.  Working exactly as intended.  We love pulledpork for this, and we wish the rest of the oinkmaster users would move off of oinkmaster, as it helps us alleviate a lot of load on the server.

We use load balancing, and even Cloudflare in front of Snort.org to cache the majority of requests to the site.  In fact, about 85% of the content served from Snort.org is cached.

The remainder of this traffic, for the most part, is document and rule downloads.

This only becomes a problem, basically, at the top of the hour.  (Our downloaders love 12pm and 4pm the most for some reason).  At every hour, we have huge spikes of traffic, caused by people running pulledpork (or, for some reason, oinkmaster) in a cron to download the ruleset on the hour.

It's perfectly fine that you do this.

However, if we can encourage, say, 10% of you, to randomize your crontab's time, even to 10 minutes past the hour, the response time on our servers would drop tremendously.  (Now, don't everyone go set their crontab to 10 past the hour, it was just an example!)

Please keep in mind that no one has complained about the response time of the site, and we aren't overly concerned with the issue.  We just prefer to head this off at the pass, before it becomes an issue.

We add over 1,000 new users to the site every week, and with well over 500,000 active users on Snort.org now, and we show no signs of slowing down.  In fact, by all the metrics we track, activity is increasing.  This is fantastic, and we love that the fact that our community is strong.

However, if we can adjust some of our crontab run times for the rule update software that you all are running, we can keep the experience as optimal as we can for everyone for a long time to come.

I appreciate you doing so, thanks a lot!

Keep Snorting!

Friday, March 8, 2013

Snort syntax highlighting for the Nano text editor

Caleb Jaren,  a regular Snort user and contributor has recently put up a blog post about some work he's done for Snort syntax highlighting for the Nano text editor.

If the name sounds familiar on the blog, it's because he recently performed the same work recently with the Notepad++ editor which we lighted back here: http://blog.snort.org/2012/08/snort-syntax-highlighting-and-more-in.html

For his most recent work on Nano check out his post here: http://www.tropismgroup.org/2013/03/07/snort-syntax-highlighting-for-the-nano-text-editor/


Friday, August 3, 2012

Snort syntax highlighting and more in Notepad++

Friend of the VRT, Caleb Jaren, recently showed me some cool work he's done creating a Snort "User Defined Language" in Notepad++. This UDL provides flexible syntax highlighting, rule validation, and general assistance as you're getting familiar with the Snort rules language and its capabilities. It's freely available on his site, and he's looking for feedback and testing from the community.

Thanks, Caleb, for the great contribution to the community. Windows users, have fun with this useful new bit of code!

Tuesday, April 3, 2012

VRT Rule Update for 4/3/2012, Rule-Recategorization

Join us as we welcome the introduction of the newest rule release for today from the VRT. In this release we introduced 30 new rules and made modifications to 169 additional rules.

The following changes made to the snort.conf in this release, these can be added to the bottom of the snort.conf where the rule declarations are made:

include $RULE_PATH/file-office.rules
include $RULE_PATH/file-other.rules
include $RULE_PATH/file-pdf.rules
include $RULE_PATH/indicator-compromise.rules
include $RULE_PATH/indicator-obfuscation.rules
include $RULE_PATH/policy-multimedia.rules
include $RULE_PATH/policy-other.rules
include $RULE_PATH/policy-social.rules
include $RULE_PATH/pua-p2p.rules
include $RULE_PATH/pua-toolbars.rules
include $RULE_PATH/server-mail.rules


 In VRT's rule release:
Synopsis: This release introduces eleven new rule categories and contains new and modified rules in several categories. 
Details: This release introduces eleven new rule categories: 
File-Office
File-Other
File-PDF
Indicator-Compromise
Indicator-Obfuscation
Policy-Multimedia
Policy-Other
Policy-Social
PUA-P2P
PUA-Toolbars
Server-Mail
These categories have been populated with rules that were formerly in policy.rules, leaving 36 rules in that category.  These will be moved in the near future 
This release contains new and modified rules in the backdoor, botnet-cnc, dos, exploit, file-identify, file-office, file-other, file-pdf, indicator-compromise, indicator-obfuscation, mysql, policy-multimedia, policy-other, policy-social, pua-p2p, pua-toolbars, server-mail, specific-threats, spyware-put, voip, web-client and web-php rule sets to provide coverage for emerging threats from these technologies.
In order to subscribe now to the VRT's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at http://www.snort.org/store. Make sure and stay up to date to catch the most emerging threats!

Wednesday, September 7, 2011

Snez: New Snort GUI has been posted

If you head over to our "additional-downloads" page on Snort.org, you'll notice a new project at the bottom of the list named "Snez".  From the project's Sourceforge page:

SNEZ is a web interface to the popular open source IDS program SNORT® . The main design feature of SNEZ is the ability to filter (or dismiss) alerts without having to delete.
 Take a look at this new project and help the author out by providing feedback!

Tuesday, August 23, 2011

Snort 2.9.1 has been released, including Protocol Aware Flushing and IP Reputation Preprocessor

Snort 2.9.1 has been released!

Now available at our download link here:  https://www.snort.org/downloads


Please start downloading and using Snort 2.9.1.  You should be aware that you'll get some new alerts and things will behave a bit differently with the file_data rule option now because of PAF.  For more on PAF please read the README.stream5 documentation file.

The Snort 2.9.1 manual will be up on http://manual.snort.org and http://www.snort.org/docs in a few minutes.

Every Friday for the next few weeks we will be posting a new blog post covering the new features of 2.9.1 directly from the Snort Developers.  So stay tuned!

Below are the Release Notes and Changelog for everything since the release of Snort 2.9.0.5:


Snort 2.9.1 introduces the following new capabilities:

* Protocol aware reassembly support for HTTP and DCE/RPC
preprocessors.  Updates to Stream5 allowing Snort to more
intelligently inspect HTTP and DCE/RPC requests and responses.
See README.stream5 subsection related to Protocol Aware Flushing
(PAF).

* SIP preprocessor to identify SIP call channels and provide
rule access via new rule option keywords.  Also includes new
preprocessor rules for anomalies in the SIP communications.
See the Snort Manual and README.sip for details.

* POP3 & IMAP preprocessors to decode email attachments in
Base64, Quoted Printable, and uuencode formats, and updates
to SMTP preprocessor for decoding email attachments encoded
as Quoted Printable and uuencode formats.  See the Snort
Manual, README.pop, README.imap, and README.SMTP for details.

* Support for reading large pcap files.

* Logging of HTTP URL (host and filename), SMTP attachment
filenames and email recipients to unified2 when Snort generates
events on related traffic.

* IP Reputation preprocessor, allowing Snort to blacklist or
whitelist packets based on their IP addresses. This preprocessor
is still in an experimental state, so please report any issues
to the Snort team.  See README.reputation for more information.

Additionally, the following updates and improvements have been made:

* Updates to give shared library rules direct access to gzip
decoding capabilities.

* Rule Option Improvements:

- Updates to content modifier http_cookie to not include
the HTTP header names themselves in the buffer.  This change
may affect existing rules that leverage this keyword.

- Updates to the file_data and base64_data rule option keywords
and added a pkt_data rule option keyword that sets the buffer
to be used for subsequent content/pcre/etc rule options.

- Updates to the tcp flag rule option keyword to support 'C'
and 'E' for CWR and ECN bits.

- Updates to byte_extract rule option keyword to support
the same string formats as with byte_test and byte_jump.

* Updates to Snort's build infrastructure and autoconf script
for portability and improved checks for library dependencies.
To facilitate easier building of Snort on many of the different
platforms supported, Snort now uses pkg-config to check for
certain library locations.  Obtain pkg-config from freedesktop.org.

* Many updates and improvements to the Snort documentation.  Special
thanks to all of the contributors from the Snort community for
working with us and making the documentation more accurate and
usable.

* Updates to the sensitive data preprocessor for handling HTTP
traffic and reducing false positives.

* Updates to Snort's config parsing to provide more meaningful
error messages relating to snort.conf errors and configuration
display at startup.

* Updates to Snort's active response packets whether via response
keyword or part of inline normalization.

* Improvements to HTTP Inspect processing of chunked HTTP data.
Additional HTTP Inspect alerts for evasion attempts such as small
chunks and excessive whitespace in folded headers.

* Updates to the statistics Snort prints to console or syslog
at exit for different preproessors.




2.9.1.0 Changelog:

Snort 2.9.1
* src/build.h:
Updated build number to 71.

* etc/gen-msg.map, preproc_rules/decoder.rules, src/decode.c,
src/decode.h, src/generators.h, src/snort.c,
src/dynamic-plugins/sf_engine/sf_snort_packet.h:
Fixed an issue with decoding large numbers of IPv6 extension headers.
Added rule 116:456 to safeguard against too many IPv6 extension headers.
Thanks to Martin Schutte for reporting the issue.

* src/detection-plugins/sp_urilen_check.c,
src/detection-plugins/sp_urilen_check.h:
Fixed the urilen rule option to look at reassembled packets.
Added an extra parameter to specify whether to check raw or normalized uri buffer. Will check raw uri buffer by default.

* src/: dynamic-preprocessors/dcerpc2/sf_dce2.dsp,
dynamic-preprocessors/dns/sf_dns.dsp,
dynamic-preprocessors/ftptelnet/sf_ftptelnet.dsp,
dynamic-preprocessors/imap/sf_imap.dsp,
dynamic-preprocessors/isakmp/sf_isakmp.dsp,
dynamic-preprocessors/pop/sf_pop.dsp,
dynamic-preprocessors/reputation/sf_reputation.dsp,
dynamic-preprocessors/sdf/sf_sdf.dsp,
dynamic-preprocessors/sip/sf_sip.dsp,
dynamic-preprocessors/smtp/sf_smtp.dsp,
dynamic-preprocessors/ssh/sf_ssh.dsp,
dynamic-preprocessors/ssl/sf_ssl.dsp,
win32/WIN32-Prj/sf_engine.dsp:
Fixed a bug where the sensitive_data preprocessor gave an error while loading sensitive data rules.

* doc/README.http_inspect, etc/gen-msg.map,
preproc_rules/preprocessor.rules, src/generators.h,
src/preprocessors/snort_httpinspect.c,
src/preprocessors/HttpInspect/event_output/hi_eo_log.c,
src/preprocessors/HttpInspect/include/hi_eo_events.h,
src/preprocessors/HttpInspect/utils/hi_paf.c:
Added two HTTP Inspect preprocessor rules:
119:28 - post w/o content-length or transfer-encoding: chunked
120:8 - message with invalid content-length or chunk size

* src/preprocessors/spp_httpinspect.c:
Fixed a bug where Snort wouldn't reload, giving the error that
"Changing decompress_depth requries a restart".

* etc/gen-msg.map:
Commented out four rules from gen-msg.map, 133:44 through 133:47,
because they were not yet implemented.

* preproc_rules/preprocessor.rules:
Added a CVE reference for Rule 119:19.
Added a reference to SMTP preprocessor rule 124:4.
Added a preprocessor rule, 125:9, for an FTPTelnet preprocessor
alert that was missing the corresponding rule.

* src/dynamic-plugins/sf_engine/sf_snort_plugin_api.c:
PAF tweak for single-segment full PDUs matching only-stream

* src/snort.c:
Fixed a bug where Snort wouldn't reload on SIGHUP with OpenBSD.
Set default paf_max to 16K.

* doc/: README.reputation, snort_manual.pdf, snort_manual.tex:
Added a use case in the IP Reputation preprocessor documentation.

* src/: dynamic-preprocessors/reputation/reputation_config.c,
dynamic-preprocessors/reputation/sf_reputation.dsp,
win32/WIN32-Prj/snort.dsw, win32/WIN32-Prj/snort_installer.nsi:

Fixed the IP Reputation preprocessor so that it would build on Windows.

* src/preprocessors/HttpInspect: client/hi_client.c, include/hi_client.h,
server/hi-server.c, utils/hi_paf.c:
Support up to full 32-bit content-lengths

* src/preprocessors/Stream5/stream5_paf.c:
Fixed compilation with the options "--disable-target-based --enable-paf".

* src/preprocessors/Stream5/snort_stream5_tcp.c:
Fixed an error in IDS mode when segments overlap and the sequence
number wraps.

* tools/u2spewfoo/Makefile.am:
Added the u2spewfoo Windows project file to the Snort source tarball.

Snort 2.9.1 RC
* doc/README.sip, doc/snort_manual.pdf, doc/snort_manual.tex,
preproc_rules/preprocessor.rules,
src/dynamic-preprocessors/sip/sip_parser.c,
src/dynamic-preprocessors/sip/spp_sip.h, etc/gen-msg.map:
Added three new SIP preprocessor alerts.

* src/preprocessors/Stream5/: snort_stream5_tcp.c, stream5_paf.c,
stream5_paf.h:
Allow multiple preprocs to scan for PDUs on the same port.
This fixes a problem with DCE autodetect using the same
ports as HTTP.

* src/build.h:
Updated build number to 63.

* src/: fpcreate.c, log.c, detection-plugins/sp_byte_extract.c,
detection-plugins/sp_tcp_win_check.c,
dynamic-plugins/sf_engine/sf_snort_plugin_content.c,
dynamic-plugins/sf_engine/sf_snort_plugin_pcre.c,
preprocessors/spp_normalize.c:
Fixed some compiler warnings.

* src/: detection-plugins/detection_options.c,
detection-plugins/sp_flowbits.h,
dynamic-plugins/sf_engine/sf_snort_detection_engine.c,
dynamic-plugins/sf_engine/examples/Makefile.am,
dynamic-plugins/sf_engine/examples/flowbits_test.c,
dynamic-plugins/sf_engine/examples/rules.c,
dynamic-plugins/sf_engine/examples/web-client_test.c:
Only set/clear/toggle/unset a flowbit when all of the rule
matches, including the IPs and Ports. Thanks to Eoin Miller
for reporting the issue.

* src/dynamic-preprocessors/: Makefile.am, dcerpc2/Makefile.am,
dns/Makefile.am, ftptelnet/Makefile.am, imap/Makefile.am,
pop/Makefile.am, reputation/Makefile.am, rzb_saac/Makefile.am,
sdf/Makefile.am, sip/Makefile.am, smtp/Makefile.am,
ssh/Makefile.am, ssl/Makefile.am:
Fixed dynamic preprocesor Makefiles so that they can be built
in parallel.

* doc/README.http_inspect, doc/snort_manual.pdf,
doc/snort_manual.tex, etc/gen-msg.map,
preproc_rules/preprocessor.rules, src/generators.h,
src/preprocessors/snort_httpinspect.c,
src/preprocessors/snort_httpinspect.h,
src/preprocessors/HttpInspect/client/hi_client.c,
src/preprocessors/HttpInspect/event_output/hi_eo_log.c,
src/preprocessors/HttpInspect/include/hi_eo_events.h,
src/preprocessors/HttpInspect/include/hi_ui_config.h,
src/preprocessors/HttpInspect/include/hi_util.h,
src/preprocessors/HttpInspect/user_interface/hi_ui_config.c,
src/sfutil/util_unfold.c:

Added a new HTTP Inspect preprocessor rule, GID 119 SID 26.  This rule checks for 200+ whitespaces in a folded header line from an HTTP request. A new config option was added to configure the allowable amount whitespace.

Added a new configuration option to http_inspect server configuration:
"small_chunk_length { <chunk_size> <num_consec_chunks> }", with preprocessor rules for both client and server. Consecutive chunk lengths less than or equal to <chunk_size> will cause an event to be generated.

See README.http_inspect for more information.

* src/: dynamic-preprocessors/dcerpc2/sf_dce2.dsp,
dynamic-preprocessors/dns/sf_dns.dsp,
dynamic-preprocessors/ftptelnet/sf_ftptelnet.dsp,
dynamic-preprocessors/imap/sf_imap.dsp,
dynamic-preprocessors/isakmp/sf_isakmp.dsp,
dynamic-preprocessors/sdf/sf_sdf.dsp,
dynamic-preprocessors/sf_dynamic_initialize/sf_dynamic_initialize.dsp,
dynamic-preprocessors/sip/sf_sip.dsp,
dynamic-preprocessors/smtp/sf_smtp.dsp,
dynamic-preprocessors/ssh/sf_ssh.dsp,
dynamic-preprocessors/ssl/sf_ssl.dsp,
win32/WIN32-Prj/sf_engine.dsp,
win32/WIN32-Prj/sf_engine_initialize.dsp,
win32/WIN32-Prj/sf_testdetect.dsp, win32/WIN32-Prj/snort.dsp:
Fixed the Win32 build to (1) not use .pch, and (2) correct sed
patterns on ipv6_port.h.

* src/output-plugins/spo_alert_sf_socket.c:
Fixed a problem where Snort's generic IP address structure was being sent by the socket output plugin.
The output plugin now only generates events for IPv4 packets, and is guaranteed to use uint32_t IPv4 addresses for interoperability.

* src/sfutil/: sfrt.c, sfrt.h:
Optimized some memory usage.

* configure.in:
Add check for pkg-config and provide instructions to get it if pkg-config is not installed.

* src/preprocessors/Stream5/: snort_stream5_tcp.c,
stream5_common.h:
Show single segment PAF packets and only short-circuit at
correct sequence.
When aborting PAF, flush at paf_max.
Tweaked retransmission check to use actual sequence numbers
instead of the adjusted sequence numbers.
Changed the pseudo-random flush point after each flush.

* src/snort.c:
Fixed a compilation error when active response is disabled.

* src/snort.h:
Fixed a bug where Snort wouldn't daemonize on OpenBSD if the process was running as root. Thanks to Olaf Schreck for reporting this issue.

* src/preprocessors/: perf-base.c, perf-base.h, perf-event.c,
perf-event.h, perf-flow.c, perf-flow.h, perf.c, perf.h,
spp_perfmonitor.c:
Split out Perfmon submodule Init and Reset, so that everything is
initialized when the Perfmonitor preprocessor is initialized.
Previously, some data was initialized on the first packet.

* src/detection-plugins/sp_tcp_flag_check.c:
Fixed a couple spots where the "1" and "2" flags weren't renamed to "C" and "E". Thanks to Joshua Kinard for reporting the issue and supplying a patch.

* doc/README.sip, doc/snort_manual.pdf, doc/snort_manual.tex,
src/dynamic-preprocessors/sip/sip_parser.c,
src/dynamic-preprocessors/sip/spp_sip.h,
preproc_rules/preprocessor.rules, etc/gen-msg.map:
Added a new SIP preprocessor alert for missing content type headers.
Fixed an issue where the SIP preprocessor checked for Stream5 even if the SIP preprocessor was disabled.

* etc/unicode.map:
Updated unicode.map to match the unicode standard on Windows 7 SP1.

* etc/snort.conf:
Sync'ed to VRT's latest snort.conf.

* src/: decode.c, detect.c:
Tweaked the preprocessing loop to bypass app preprocs if no app data.

* src/sfutil/sf_ip.c, src/sfutil/sf_ip.h, src/sfutil/sfrt_dir.c,
src/dynamic-preprocessors/reputation/Makefile.am,
src/dynamic-preprocessors/reputation/reputation_config.h,
src/dynamic-preprocessors/reputation/reputation_utils.c,
src/dynamic-preprocessors/reputation/sf_reputation.dsp,
src/dynamic-preprocessors/reputation/spp_reputation.c,
src/dynamic-preprocessors/reputation/spp_reputation.h,
src/dynamic-preprocessors/reputation/reputation_config.c,
src/dynamic-preprocessors/reputation/reputation_debug.h,
src/dynamic-preprocessors/reputation/reputation_utils.h,
doc/README.reputation, doc/Makefile.am, doc/snort_manual.pdf,
doc/snort_manual.tex, preproc_rules/preprocessor.rules,
src/dynamic-preprocessors/Makefile.am, configure.in,
src/preprocids.h, etc/gen-msg.map:
Added the IP Reputation preprocessor. This preprocessor provides the ability to whitelist and blacklist packets based on IP addresses.
See README.reputation for more information.

* src/: sf_types.h, dynamic-plugins/sf_dynamic_plugins.c,
dynamic-preprocessors/dcerpc2/Makefile.am,
dynamic-preprocessors/dcerpc2/dce2_config.c,
dynamic-preprocessors/dcerpc2/dce2_debug.h,
dynamic-preprocessors/dcerpc2/dce2_paf.c,
dynamic-preprocessors/dcerpc2/dce2_paf.h,
dynamic-preprocessors/dcerpc2/sf_dce2.dsp,
dynamic-preprocessors/dcerpc2/snort_dce2.c:
Added protocol-aware flushing support for the dcerpc2 preprocessor.

* src/dynamic-plugins/sf_convert_dynamic.c:
Added the ability to convert shared object rules that use the preprocessor rule option.

* src/preprocessors/: snort_httpinspect.c, spp_httpinspect.c,
HttpInspect/include/hi_paf.h, HttpInspect/utils/hi_paf.c,
Stream5/snort_stream5_tcp.c:
Don't enable paf unless stream ports configured for the given direction; add "(PAF)" to http inspect ports output to indicate when enabled; and only register port for given direction if corresponding flow depth is set.

Support full 32-bit content-lengths and chunk sizes, and flush/abort when exceeded.

* doc/README.SMTP, doc/snort_manual.tex,
src/dynamic-preprocessors/smtp/smtp_config.h,
src/dynamic-preprocessors/smtp/smtp_util.c,
src/dynamic-preprocessors/smtp/snort_smtp.c,
src/dynamic-preprocessors/smtp/snort_smtp.h,
src/dynamic-preprocessors/smtp/spp_smtp.c:
Fixed performance issue: allocate the buffers used for filename, mailfrom and rcptto logging using mempool ('memcap' used to allocate the mempool).
Added a fatal error when b64_decode_depth is used with enable_mime_decoding.

* src/dynamic-plugins/sf_engine/examples: all rule files:
Fixed compiler warnings.


* configure.in:
Updates to configure.in.
Fix zlib checks to use correctly named variable for checking zlib header and library existence.
Enable IPv6 by default in builds.  Can use --disable-ipv6 to turn it off.
Using --enable-zlib, configure should fail.  snort -V should show IPv6 by default and VRT config should load without modification.

Added a new option, "--enable-large-pcap", which allows Snort to read pcap files that are larger than 2 GB.
Changed the default ./configure options to match the requirements for the bundled snort.conf
* doc/: INSTALL, README.imap, README.pop,
README.SMTP, README.stream5, README.sip, README.tag,
README.http_inspect, README.counts, README.normalize,
snort_manual.pdf, snort_manual.tex:
Updated documentation for Snort 2.9.1:

Added documentation for new SIP, POP and IMAP preprocessors
Updated README.stream5 with documentation for Protocol Aware Flushing (PAF)
Updated README.http_inspect with memcap information, clarified "http_cookie" information, and documentation for "log_uri" and "log_hostname".
Fixed a typo in README.counts
Updated "byte_extract" section to reflect syntax changes
Improved the explanation of "max_queued_events"
Added documentation for the ESP decoder, which is now configurable
Improved the explanation of "rawbytes"
Fixed an incorrect example in README.tag.
* etc/snort.conf:
Synced snort.conf with VRT's latest version.

Added configurations for new preprocessors.
* preproc_rules/: decoder.rules, preprocessor.rules
Added new preprocessor rules for SIP, SMTP, POP, and IMAP.

Added decoder rules 116:453, 116:454, and 116:455. These rules
were formerly covered by VRT rules.
* src/build.h: Updated build number to 46
* src/decode.c:
TCP and UDP decoder rules that require a fully-decoded packet will only fire if the checksum is correct and the port number is not ignored.

ESP decoding is now configurable, and off by default.

The "config enable_decode_oversized_alerts" option now applies to packets where the UDP header claims there is more data than actually exists.
The Teredo decoder now only processes packets in the Teredo prefix
(2001:0000::/32) or the link-local prefix (fe80::/16).
* src/detection-plugins/sp_cvs.c:
Fixed a false positive in the CVS detection plugin.
* doc/snort_manual.tex, src/detection-plugins/sp_byte_extract.c:
Made some changes to the byte_extract syntax:
Writing "string" without a number type defaults to decimal.
The "string" and "hex/dec/oct" options are now independent of each other, like in byte_test and byte_jump. You can write "string,dec", "hex,string", "string,relative,oct", etc.
Specifying one of "hex", "dec", and "oct" without using "string"
results in an error.
byte_extract options can no longer be delimited by spaces. This does not affect "align <num>" or "multiplier <num>".
* src/: parser.c, util.c, util.h,
detection-plugins/sp_base64_decode.c,
dynamic-plugins/sf_dynamic_plugins.c,
dynamic-plugins/sf_dynamic_preprocessor.h,

dynamic-plugins/sp_dynamic.c,
dynamic-preprocessors/smtp/smtp_util.c,
preprocessors/HttpInspect/client/hi_client.c,
preprocessors/HttpInspect/server/hi_server.c,
sfutil/sf_base64decode.c, sfutil/sf_base64decode.h:
Changes include the following:
- Attempt dechunkind only when transfer-encoding: chunked is present.
- Override the content length with transfer encoding
- SnortStrcasestr uses slen now.
- unfolding : trim spaces when required.
* src/: pcap_pkthdr32.h, preprocessors/spp_frag3.c,
preprocessors/Stream5/snort_stream5_tcp.c,
preprocessors/Stream5/stream5_common.h, sfutil/sf_ipvar.c,
sfutil/sf_ipvar.h, sfutil/sf_vartable.c:
Update Frag3/Stream5 to print bound addresses, better descriptsions of detect anomalies and port lists.
- Updated Frag3/Stream5 to print bound addresses for IPv6 enabled builds
- Updated Frag3 to print meaningful detect anomalies configuration
- Updated Stream5 to print that there are more ports than those printed.
* src/dynamic-plugins/sf_engine/: Makefile.am, sf_decompression.c,
sf_decompression.h, sf_snort_detection_engine.c,
sf_snort_plugin_api.h:
Added a Decompression API that wraps Zlib for use with dynamic
plugins. See sf_decompression.h for more details.
* src/: fpcreate.c, fpdetect.c, treenodes.h:
Update pattern matcher and sort functions to correctly sort by priority as well as implement sorting by content_length (which was never done with 2.8.2 addition of rule option tree).

Added a warning when max-pattern-len is defined twice.

Packets will no longer be tagged or logged if they are filtered or passed.
* src/preprocessors/Stream5:
Ensured that reassembly doesn't require packet dropping in IPS mode.
The message "additional ports configured but not printed" is only printed when that is actually the case.
* src/snort.c:
fix output of filename / shutdown alerts sequence when iterating over multiple pcaps with --pcap-show --pcap-reset and console alerts (eg -A cmg or
-A console:test).

Fixed an issue with reloading Snort while the default output options were used.

When reading several pcap files with --pcap-dir, Snort will move on
to the next file if one fails to load.
* src/output-plugins/spo_alert_full.c:
Update alert_full to print rule references, regardless of whether
there is TCP/UDP/etc.
* src/output-plugins/spo_log_tcpdump.c:
convert DLT_IPV{4,6} to DLT_RAW for compatibility with libpcap 1.0.0
fix 'mixed decls and code' compiler warning
* src/: decode.h, detect.c, detection_util.c, detection_util.h,
fpcreate.c, fpdetect.c, log.c, log_text.c, parser.h, plugbase.c,
rule_option_types.h, detection-plugins/Makefile.am,
detection-plugins/detection_options.c,
detection-plugins/sp_base64_data.c,
detection-plugins/sp_byte_check.c,
detection-plugins/sp_byte_extract.c,
detection-plugins/sp_byte_jump.c,
detection-plugins/sp_file_data.c,
detection-plugins/sp_ftpbounce.c,
detection-plugins/sp_isdataat.c,
detection-plugins/sp_pattern_match.c,
detection-plugins/sp_pcre.c, detection-plugins/sp_pkt_data.c,
detection-plugins/sp_pkt_data.h,
dynamic-plugins/sf_convert_dynamic.c,
dynamic-plugins/sf_dynamic_common.h,
dynamic-plugins/sf_dynamic_define.h,
dynamic-plugins/sf_dynamic_engine.h,
dynamic-plugins/sf_dynamic_plugins.c,
dynamic-plugins/sf_dynamic_preprocessor.h,
dynamic-plugins/sp_dynamic.c, dynamic-plugins/sp_dynamic.h,
dynamic-plugins/sf_engine/sf_snort_detection_engine.c,
dynamic-plugins/sf_engine/sf_snort_packet.h,
dynamic-plugins/sf_engine/sf_snort_plugin_api.c,
dynamic-plugins/sf_engine/sf_snort_plugin_content.c,
dynamic-plugins/sf_engine/sf_snort_plugin_pcre.c,
dynamic-plugins/sf_engine/examples/detection_lib_meta.h,
dynamic-preprocessors/ftptelnet/pp_ftp.c,
dynamic-preprocessors/ftptelnet/pp_telnet.c,
dynamic-preprocessors/ftptelnet/snort_ftptelnet.c,
dynamic-preprocessors/smtp/smtp_util.c,
dynamic-preprocessors/smtp/snort_smtp.c,
dynamic-preprocessors/smtp/snort_smtp.h,
preprocessors/snort_httpinspect.c,
preprocessors/snort_httpinspect.h,
preprocessors/spp_rpc_decode.c,
preprocessors/HttpInspect/server/hi_server.c,
preprocessors/HttpInspect/server/hi_server_norm.c,
preprocessors/Stream5/snort_stream5_tcp.c:
The "file_data" and "base64_data" rule options now set the buffer
for any rule options that follow them. This applies to both relative and non-relative rule options.

The detection code now uses 3 separate buffers:
- "Alt Detect": set by file_data, base64_data, etc.
- "Alt Decode": set by preprocessor normalization, e.g. HTTP Inspect
- Raw packet data

The AltDetect buffer can also be set by custom .so rules.
* src/parser.c, src/parser.h, src/snort.h, src/output-plugins/spo_unified2.c,
src/sfutil/Unified2_common.h:
IPv6 source and destination addresses are now logged in Unified2 as extra data events. This is configured with "config log_ipv6_extra_data".
* src/dynamic-preprocessors/sip/Makefile.am,
src/dynamic-preprocessors/sip/sf_sip.dsp,
src/dynamic-preprocessors/sip/sip_config.c,
src/dynamic-preprocessors/sip/sip_config.h,
src/dynamic-preprocessors/sip/sip_debug.h,
src/dynamic-preprocessors/sip/sip_dialog.c,
src/dynamic-preprocessors/sip/sip_dialog.h,
src/dynamic-preprocessors/sip/sip_parser.c,
src/dynamic-preprocessors/sip/sip_parser.h,
src/dynamic-preprocessors/sip/sip_roptions.c,
src/dynamic-preprocessors/sip/spp_sip.c,
src/dynamic-preprocessors/sip/spp_sip.h,
src/dynamic-preprocessors/sip/sip_roptions.h,
src/dynamic-preprocessors/sip/sip_utils.c,
src/dynamic-preprocessors/sip/sip_utils.h, doc/README.sip,
etc/gen-msg.map, src/dynamic-preprocessors/sip/test/Makefile.am,
src/dynamic-preprocessors/sip/test/sip_test.c, configure.in,
src/dynamic-preprocessors/Makefile.am:
Added a new preprocessor for SIP traffic.
See README.sip and the Snort Manual for more information.
* src/: dynamic-preprocessors/dcerpc2/dce2_utils.c,
dynamic-preprocessors/dcerpc2/spp_dce2.c,
preprocessors/spp_frag3.c:
Make Frag3 OpenBSD Vuln alert only happen if the frag policy is 'linux' (which includes OpenBSD).  The 'bsd' policy is NOT used for OpenBSD, which is the only OS on which the vulnerability was present.

This reduces false positives to only occur when frag3 policy is linux and its an actual linux system, rather than the alert occurring regardless of frag policy.
* src/: detection-plugins/Makefile.am,
detection-plugins/sp_byte_extract.c,
detection-plugins/sp_byte_extract.h,
dynamic-plugins/sf_convert_dynamic.c,
dynamic-plugins/sf_engine/Makefile.am,

dynamic-plugins/sf_engine/sf_snort_detection_engine.c,
dynamic-plugins/sf_engine/sf_snort_detection_engine.h,
dynamic-plugins/sf_engine/sf_snort_plugin_api.c,
dynamic-plugins/sf_engine/sf_snort_plugin_api.h,
dynamic-plugins/sf_engine/sf_snort_plugin_byte.c,
dynamic-plugins/sf_engine/sf_snort_plugin_content.c,
dynamic-plugins/sf_engine/sf_snort_plugin_hdropts.c,
dynamic-plugins/sf_engine/sf_snort_plugin_loop.c,
dynamic-plugins/sf_engine/sf_snort_plugin_pcre.c,
Added support for ByteExtract variables to the .so rule versions of
Content, ByteTest, ByteJump, and isdataat.
* src/: encode.c, preprocessors/spp_normalize.c,
preprocessors/Stream5/snort_stream5_tcp.c,
preprocessors/Stream5/stream5_common.c:
Fixed the TTL on encoded response packets.
* src/: fpcreate.c, fpdetect.c,
detection-plugins/sp_pattern_match.c,
detection-plugins/sp_pattern_match.h,
dynamic-plugins/sf_dynamic_define.h,
dynamic-plugins/sf_engine/sf_snort_detection_engine.c,
dynamic-plugins/sf_engine/sf_snort_plugin_api.h:
Update to not inspect HTTP method buffer with Snort's fast pattern engine.
Rules with only HTTP method content end up as non-content rules.
This eliminates a short cycle of searches with fast pattern on every initial HTTP request.
* src/dynamic-preprocessors/pop/: all files
Added a new preprocessor for POP traffic.
See README.pop for more information.
* src/dynamic-preprocessors/imap/: all files
Added a new preprocessor for IMAP traffic.
See README.imap for more information.
* src/sfutil/: sf_email_attach_decode.c, sf_email_attach_decode.h:
Base64 decoding was moved to its own section in sfutil, for use by the new email preprocessors.

Added support for uuencoded email attachments.
* src/dynamic-preprocessors/sdf/spp_sdf.c:
The Sensitive Data preprocessor now inspects the "file_data" buffer, used for HTTP response bodies & decoded email attachments.
* src/: snort.c, preprocessors/spp_stream5.c,
preprocessors/stream_api.h:
Update Snort to return a DAQ verdict of whitelist (meaning don't send Snort any more packets) for sessions that are being ignored in both directions or ports that are configured to ignore.  For DAQ modules and hardware that supports it, this should result in a performance gain because Snort no longer has to decode packets that are part of that connection.
* src/util.c:
Added an error message when opening a pid file fails.
* src/preprocessors/HttpInspect/: client/hi_client.c,
server/hi_server.c:
The Set-Cookie: and Cookie: headers wont be included in the cookie buffers.
* configure.in, src/active.c, src/active.h, src/decode.h,
src/encode.c, src/encode.h, src/log_text.c, src/log_text.h,
src/parser.c, src/parser.h, src/sf_types.h, src/sfdaq.c,
src/sfdaq.h, src/snort.h, src/snort_debug.h,
src/detection-plugins/sp_react.c,
src/detection-plugins/sp_respond3.c,
src/dynamic-plugins/sf_dynamic_define.h,
src/dynamic-plugins/sf_engine/sf_snort_packet.h,
src/preprocessors/snort_httpinspect.c,
src/preprocessors/spp_httpinspect.c,
src/preprocessors/spp_stream5.c, src/preprocessors/stream_api.h,
src/preprocessors/HttpInspect/Makefile.am,
src/preprocessors/HttpInspect/include/Makefile.am,
src/preprocessors/HttpInspect/include/hi_paf.h,
src/preprocessors/HttpInspect/mode_inspection/hi_mi.c,
src/preprocessors/HttpInspect/server/hi_server.c,
src/preprocessors/HttpInspect/utils/Makefile.am,
src/preprocessors/HttpInspect/utils/hi_paf.c,
src/preprocessors/Stream5/Makefile.am,
src/preprocessors/Stream5/snort_stream5_icmp.c,
src/preprocessors/Stream5/snort_stream5_session.c,
src/preprocessors/Stream5/snort_stream5_tcp.c,
src/preprocessors/Stream5/snort_stream5_tcp.h,
src/preprocessors/Stream5/snort_stream5_udp.c,
src/preprocessors/Stream5/stream5_common.c,
src/preprocessors/Stream5/stream5_common.h,
src/preprocessors/Stream5/stream5_paf.c,
src/preprocessors/Stream5/stream5_paf.h, src/sfutil/sf_textlog.h:
Added support in Stream5 for Protocol Aware Flushing (PAF). PAF allows Snort to statefully scan a stream and reassemble a complete PDU regardless of segmentation.

Added PAF support to HTTP Inspect, allowing the preprocessor to determine when HTTP sessions are flushed by Stream5.

See README.stream5 for more details.
* src/preprocessors/: stream_ignore.h, stream_ignore.c,
Stream5/snort_stream5_udp.c:
Added support for ignoring UDP channels. Light weight session will be created to track UDP channel, even ports are not monitored.
* src/win32/: most files
Updated Snort and its libraries to build/link against MFC.


Monday, August 15, 2011

New Snort 3rd Party project is listed: iBlock

The author of this project Roberto Zarrelli wrote me last week while I was at the GFirst conference, and notified me of the listing of his new project "iBlock"'s (For "Intrusion Block") listing on Sourceforge.

A short description of the project:

This tool is a small Linux Daemon that greps the Snort Alert file and blocks the offending hosts via iptables for a given amount of time. iBlock supports the whitelisting of IP addresses so those IPs will never be blocked.
iBlock is now listed on our 3rd Party projects page on Snort.org, and a link to Roberto's project directly on Sourceforge is here.

Thanks Roberto for your submission, we're all hoping that your project does well!

Thanks to all of the 3rd party projects surrounding Snort!

Tuesday, June 28, 2011

Sourcefire Recognizes Seventh Annual SNORT Cybersecurity Scholarship Winners

Columbia, MD – June 28, 2011 – Sourcefire, Inc. (Nasdaq: FIRE), the creator of Snort® and a leader in intelligent cybersecurity solutions, today announced that it has selected Darcie Cohee and Daniel Freer as the recipients of the 2011 Snort Scholarship. The scholarships, each worth up to $15,000, are awarded to university students around the world that use Snort to further their education and gain hands-on experience in network security.

“As hackers continue to find new ways to access sensitive corporate and customer data, we need to groom a new generation of security professionals to identify and combat these exploits,” said Martin Roesch, CTO and founder of Sourcefire. “Snort and Sourcefire are built on the foundation of community development and these scholarships allow us to recognize the next great security professionals.”

To qualify, applicants must be enrolled in a university that uses Snort or Sourcefire products to protect its network or uses Snort as part of the curriculum in the classroom. The scholarships assist the winning students in completing their degrees and covering educational costs. Sourcefire selected Darcie and Daniel from a pool of hundreds of applicants:


  • Darcie Cohee is a Bachelor of Science candidate in Information Systems Technologies at Southern Illinois University Carbondale. Darcie has worked on several projects using Snort to protect SharePoint deployments and is interested in the intersection of the Web and security.
  • Daniel Freer is a Bachelor of Science candidate in Networking at Indiana Tech. Daniel relied on Snort as an important weapon in his arsenal when he competed in the National Collegiate Cyber Defense Competition and is committed to exploring how Snort can help prevent evolving attacks.


To assist the winning students in completing their degrees, Sourcefire has awarded each a $5,000 scholarship for educational costs at the students’ respective universities. The winners also receive a $10,000 credit to use toward any training course or certification exam in the Sourcefire Security Education Program. The Sourcefire Security Education and Certification Programs deliver training and testing for IT staff on Sourcefire’s products and open source security solutions, either on-site or at dedicated locations around the world.

Sourcefire developed the Snort Scholarship in 2004 as a way to give back to the open source and security communities. Since the inception of the Snort Scholarship program seven years ago, Sourcefire has recognized university students from around the world, including the United States, Australia, Turkey, Mexico, the Netherlands and Rwanda. Martin Roesch founded Sourcefire in 2001 to deliver commercial security solutions that leverage his open source innovation, Snort. Snort is the world’s most widely deployed intrusion detection and prevention technology with more than 300,000 registered users and nearly 4 million downloads to date. As the de facto standard for intrusion detection and prevention, Snort is used extensively by Fortune 100 enterprises and government agencies.

About Sourcefire
Sourcefire, Inc. (Nasdaq:FIRE), is a world leader in intelligent cybersecurity solutions.  Sourcefire is transforming the way Global 2000 organizations and government agencies manage and minimize network security risks. Sourcefire’s IPS, Real-time Network Awareness and Real-time Adaptive Security solutions equip customers with an efficient and effective layered security defense – protecting network assets before, during and after an attack. Through the years, Sourcefire has been consistently recognized for its innovation and industry leadership by customers, media and industry analysts alike – with more than 50 awards and accolades. Today, the name Sourcefire has grown synonymous with innovation and network security intelligence. For more information about Sourcefire, please visit http://www.sourcefire.com.

Sourcefire, the Sourcefire logo, Snort, the Snort and Pig logo, ClamAV, Immunet and certain other trademarks and logos are trademarks or registered trademarks of Sourcefire, Inc. in the United States and other countries. Other company, product and service names may be trademarks or service marks of others.

Tuesday, May 31, 2011

Sguil Version 0.8.0 released

In it's first release since 2008, Sguil Version 0.8.0 has been released.

Bamm states on the Sguil News site:


Okay, new direction. Time has been escaping me and Sguil development has suffered. When I do have time to spend on Sguil, I would rather be adding new features and fixing bugs versus testing installs and writing documentation. So starting with this release, I am going to focus on getting code out the door and hope our small community will document their experiences through blogs, wikis, mailing lists, tweets, and #snort-gui. 

Go out and download Sguil 0.8.0. Install it. Test it. Break it. And find some bad guys. 
 Downloads are available here.

Thursday, January 13, 2011

GUIs for Snort

I asked for people to send me topics that they'd like to learn more about in Snort, and I received a good amount of responses.  So I thought I'd get started on one of them.  (BTW - if you'd like to get our input on something Snort related for the blog, please feel free to email me at joel [at] snort.org)

Every so often (probably twice a year) there seems to be an uptick in the amount of people emailing the mailing lists asking about GUIs for Snort.  Many of them repeat offenders.  So I am guessing that either people don't know about the GUI options for Snort or people don't like the ones they have.  So let's start off with a few in alphabetical order:

BASE
BASE, the Basic Analysis and Security Engine was based off of the old ACID code codebase.  The ACID GUI interface (which is now dead, and has been for about five or six years) was a college project written by an attendee of Carnegie Mellon.  It hasn't been actively developed since about 2003.  BASE, a fork of the ACID code, picked up where the original author left off, added a bunch of new features, and made it easy to use, multi-language, and a  highly functional GUI.  There were plans for a redesign of BASE, including the database format that it reads from, but Kevin Johnson, the original BASE project manager has since left the project and turned the project over to new management.  However, it remains the most popular Snort GUI interface with over 215,000 downloads.  BASE is written in PHP, and has several dependencies.  BASE has it's own IRC channel #secureideas, although there is rarely anyone there, so most people come to the default #snort for help.

OSSIM
OSSIM, made by AlienVault stands for "Open Source Security Information Management".  Not only can it take the logs from Snort and display them in a great looking interface, but it also integrates with many other tools (p0f, arpwatch, pads, nessus, ntop, nagios, etc) for a consistant user interface.  I've personally never used this tool, but I've heard from the people that use do use it, and find it really a joy to use.

PLACID
Standing for "Phil Loathes ACID", it was originally made as a super stripped down way of simply looking at Snort Events in the Snort DB.  It has stayed that way.  There is a certain demographic of Snort users that like simple, text based interfaces, and PLACID serves that need.

SGUIL
(Pronounced "Squeel")  SGUIL started off as the "Snort GUI for Lamers".  The project, maintained by Bamm Vischer, is a multi part system consisting of a "Sensor", "Server", and "Client".  Not only is SGUIL a GUI for Snort, but it also integrates other technologies into the recording of data for use by the analyst as well (including fulltime, full packet capture).  This is a heavy weight technology, is written in TCL, and is a very well performing engine.  Most people start off with a GUI like BASE and move into SGUIL.  SGUIL also has it's own IRC channel #snort-gui.

Snorby
A relative newcomer to the Snort GUI area, Snorby uses a lot of "Web 2.0" effects and rendering providing the user with a very sharp and beautifully functioning tool.   This seems to be the current "go-to" web interface for Snort.  While it has many of the features of BASE (and a lot more, hotkeys, classifications, an iOS interface, and actual pdf reporting), and not as featured as SGUIL (in terms of architecture), it's extremely easy to deploy, looks fantastic, and functions as an alert browser very well.  Snorby's code is hosted on Github, here.  Another advantage of Snorby is that it integrates with the OpenFPC project.  Functioning similar to how SGUIL collects all information on the network using Full Packet Capture (FPC), Snorby gives you the ability to not only view the Snort alert, but also to view the alerts in context with the rest of the packet flow on the network.  Snorby's IRC channel can be found at #snorby.

SQueRT
Paul wrote in about SQueRT.  SQueRT uses the SGuil database format and is also web based.  You can see the screenshots and download it at the link above.

This is by no means complete, these are just the most common that I see people using.  If I have missed a free Snort GUI that you enjoy, please feel free to respond in the comments.  The more complete your post, the better.  Give people links to your favorite tool.

Update:  http://blog.snort.org/2011/10/comparison-of-3-popular-snort-guis.html

FirePOWER
While not free by any means, the FirePOWER system is the commercial system that we develop here at Cisco.  Not only making the administration and analysis of events from Snort (the engine embedded into FirePOWER) extremely simple, it couples hundreds of more features into an extremely complex system with a simple to understand and navigate GUI.  Made to keep large deployments simple, and small deployments even easier, this is by far, the best system made. (We're biased)  But, is not free.

Thursday, January 6, 2011

VRT Rule Update Available Now and EOL of Snort 2.8.6.0

The Vulnerability Research Team of Sourcefire just released another rule pack for the following versions of Snort:

  • 2.8.6.1
  • 2.9.0.0
  • 2.9.0.1
  • 2.9.0.2
  • 2.9.0.3

As announced back in October, and in accordance with our End of Life Policy, January 2nd was the end of life for VRT rules for 2.8.6.0.

We encourage all Snort Users that are using legacy versions of Snort (2.8.6.0 and below) to upgrade to the most current version (2.9.0.3).

Snort is available for download at https://www.snort.org/downloads.

In order to subscribe now to the VRT's newest rule detection functionality, you can subscribe for as low as $29 US dollars a year for personal users, be sure and see our business pricing as well at http://www.snort.org/store.  Be sure and stay up to date to catch the most current threats!