Using OSSEC Open Source Host Based Intrusion Detection

righteousgaggleData Management

Jan 31, 2013 (4 years and 7 months ago)

354 views

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Using OSSEC Open Source Host
Based Intrusion Detection

Justin C. Klein Keane

University of Pennsylvania

School of Arts & Sciences

Information Security and Unix Systems

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Solutions


OSSEC is a service you can utilize internally
or offer your stakeholders


OSSEC allows you to extend your security
impact


OSSEC provide you greater visibility into your
security posture


Customization options allow OSSEC to meet
your specific needs

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Presentation Format


Top down

-
Definitions

-
How OSSEC works

-
Customization

-
Our Implementation

-
Logistics and considerations

-
Resources

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Network vs. App Security


We've won network security (sort of)


Firewalls are becoming less effective as
applications standardize on port 80


Defending applications requires new
approaches


OSSEC fits easily and naturally into an
application security program

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

About OSSEC


Open source host based intrusion detection
system (HIDS)


Written by Daniel Cid, who continues to
participate


Pronunciation varies


Acquired by Third Brigade in 2008, which was
then acquired by Trend Micro in 2009

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

What is an IDS


Intrusion detection system (IDS)


Distinct from an intrusion prevention system
(IPS) which is an active system


Monitor data points to identify malicious
behaviour


Anomaly detection or pattern matching


Basically just automated traffic monitoring

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Differences of HIDS vs. NIDS


HIDS


host based IDS


NIDS


network based IDS


NIDS uses data from the wire


HIDS can use filesystem as well as network


Log files, file contents


Filesystem attributes


Main difference: visibility

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Advantages of HIDS


Low(er) false positive rate


If your HIDS detected the traffic it was
definitely parsed by the target


Leverages existing hardware


Not very resource intensive


No interruption (complication) of networking
architecture

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Disadvantages of HIDS


Blind spots:


If it isn't logged, or stored on the filesystem,
it's invisible


Unable to parse unknown or unlogged traffic


Can't evaluate egress


Can't spot probes that don't hit active services


Requires an agent install on endpoints


Can be defeated with a host compromise

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Advantages of NIDS


All traffic becomes visible, ingress and egress


Requires just one point of installation


Failure doesn't affect endpoints

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Disadvantages of NIDS


High false positive rate


No way to tell if attack was “real”


Easy to overwhelm with bad traffic or evade


Packet fragmentation, etc.


Can't see encrypted traffic

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Beyond HIDS


Filesystem integrity checking


Registry monitoring on Windows


Active response

-
Commonly used for RTBL

-
Can be scripted for almost any behaviour


Rootkit detection

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

How OSSEC Works


Three modes

-
Local, client, server


Client server model (we'll focus on this)

-
Clients receive configuration from server

-
Clients send logs to server over an encrypted channel

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Configuration


OSSEC is configured in two main areas:


Server configuration


Global configurations applied to all hosts


Client configuration


Configuration options specific to a certain
machine

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Network Communication


UDP port 1514


Compressed messages (using zlib)


Blowfish based encryption using pre
-
shared
keys


Logs specified on client are collected and sent
to manager for analysis

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Log File Monitoring


OSSEC monitors specific logs by default,
including:


Syslog


Apache http logs


Mail logs


OSSEC can be configured to monitor any log
it can gain access to

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Data Flow

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Decoders


OSSEC uses decoders to parse log files


Decoders are written as XML


Extracts useful data fields from log entries to
use for rule and alert matching including:


Source IP and/or port


Destination IP and/or port


Program name or user name


And more...

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Rules


OSSEC rules are stored as XML files


Rules are hierarchical


By default OSSEC includes rules for:


apache, arpwatch, asterisk, cisco
-
ios, courier, firewalls,
ftpd, horde/imp, IDS systems, IMAP, McAfee antivirus,
MS auth, MS DHCP, Exchange, Microsoft FTPD,
MySQL, Bind, Netscreen, PAM, postfix, Postgres,
ProFTP, Roundcube, sendmail, samba, Squid, SSH,
Symantec AV, Syslog, Telnet, VMWare, VSFTP,
Wordpress, and more...

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Rule Format


Rules are assigned priority levels, usually from 1
(lowest) to 15


Rules trigger based on:

-
Pattern matching in strings

-
Timing between matches (x hits on rule y in z interval)

-
Dependence on other rules (x rule already fired)

-
Time of day

-
Hostnames

-
Applications

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Alerts


Default settings include alerting on:


Web attacks


SSH brute force


Buffer overflows and program crashes


Firewall events


Users using sudo


Many more...


Creating custom alerts is relatively easy

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Alert Behavior


When a rule triggers an alert several actions
can be configured:


Logging (to the alert.log or MySQL)


Default behavior


Sending an e
-
mail alert


Usually alerts over a threshold level trigger email


Default is level 7


Sending a SMS alert


Executing an active response script

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Typical Alert

OSSEC HIDS Notification.

2010 Aug 04 12:10:08


Received From: webdev
-
>/var/log/httpd/access_log

Rule: 31106 fired (level 12)
-
> "A web attack returned code 200
(success)."

Portion of the log(s):


172.16.46.1
-

-

[04/Aug/2010:12:10:07
-
0400] "GET /drupal
-
4.7.11/?q=user/autocomplete/%3Cscript%3Ealert(%27title%27)%3
B%3C%2Fscript%3E HTTP/1.1" 200 140
"http://172.16.46.129/drupal
-
4.7.11/?q=node/add/page"
"Mozilla/5.0 (X11; U; Linux x86_64; en
-
US; rv:1.9.1.11)
Gecko/20100723 Fedora/3.5.11
-
1.fc12 Firefox/3.5.11"

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Active Response


Initiates scripts


Default installations include firewall rules

-
Time based


Whitelists prevent self denial of service


Active response can be delegated to multiple
hosts

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Generating Reports


OSSEC includes ossec
-
reportd

-
Can be used to generate summary reports

-
Ex. show all brute force attempts and usernames
used and number of times attempted


OSSEC can also log to a database so that SQL
can be used for reporting


Custom scripts can be used to parse alert logs


Web based user interface (WUI) reporting

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Beyond HIDS: Monitoring Scripts


OSSEC can be used to monitor the output of
custom scripts


For instance, OSSEC can generate alerts
based on changes to NMAP scan results of
specific hosts


http://www.ossec.net/wiki/Know_How:Nmap


Can also log scripted alerts to common log
(syslog) and write a rule to augment existing
log monitoring

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Custom Decoders and Logs


OSSEC will monitor many common services
out of the box


OSSEC can be extended to monitor custom
applications


OSSEC can detect events in custom
application logs and trigger custom active
response scripts

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Case Study


Penn makes heavy use of the open souce
Drupal content management system


Drupal is a complex web application


Easy to spot some attacks via Apache logs


Abuse of native Drupal features isn't visible at
the Apache level


Apache only logs GET variables

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Drupal Logging


Drupal 6 includes a syslog module that writes
logs to standard syslog


Aug 2 16:00:48 webdev drupal: 172.16.46.129
http://172.16.46.129/drupal
-
6.16|1280779248|update|172.16.46.1|http://172.16.46.12
9/drupal
-
6.16/admin/reports/updates/check?destination=admin%2Fb
uild%2Fmodules|http://172.16.46.129/drupal
-
6.16/admin/build/modules|1|view|Unable to fetch any
information about available new releases and updates.

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Parsing the Log


Drupal uses a non
-
standard log format so we
have to build a custom decoder:


<decoder name="drupal">


<program_name>^drupal</program_name>


<prematch>
\
d+.
\
d+.
\
d+.
\
d
\
S+|
\
d+|
\
w+|</prematch>


<regex
offset="after_prematch">(
\
d+.
\
d+.
\
d+.
\
d+)
\
|(
\
.+)
\
|
\
.*
\
|
\
d+
\
|
\
.*
\
|(
\
.+)</regex>



<order>srcip,url,data</order>

</decoder>

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Custom Rules

<rule id="104110" level="3">

<decoded_as>drupal</decoded_as>

<description>Drupal syslog message</description>

</rule>

<rule id="104120" level="6">

<if_sid>104110,1002</if_sid>

<match>Login attempt failed</match>

<description>Drupal failed login!</description>

</rule>

<rule id="104225" level="11">

<if_sid>104120</if_sid>

<match>Login attempt failed for administrator.</match>

<description>Drupal failed attempt to log in as administrator!</description>

</rule>

<rule id="104130" level="10" frequency="4" timeframe="360">

<if_matched_sid>104120</if_matched_sid>

<description>Possible Drupal brute force attack </description>

<description>(high number of logins).</description>

</rule>

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Why Penn is Using OSSEC


Found OSSEC in a search for open source
SEM and SIM solutions


No access to network taps or span ports


Looking for a sensor net to correlate alerts


Using OSSEC for over 3 years now


Open source (allows for 10% customization)


Lightweight


Active development and support community

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

How Penn is Using OSSEC


Server installed on a bastion host


Agents deployed on critical hosts


Separate server installed on cluster back ends


Alerts go to queue monitored by security staff


Per machine alerts go to individual sysadmins


Using OSSEC to monitor custom alert scripts

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Logistical Considerations


As with any intrusion detection system, expect
a timeframe for tuning


Be extremely careful with active response to
avoid self denial of service


Installation may require coordination with
sysadmins

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Overall Impact


Develop metrics to justify security resource
allocation


Verify effectiveness of security
countermeasures


Track outlying machines, especially virtual
machines


Extend service offerings to client groups


Centralized log reporting

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Pitfalls of OSSEC


Difficulty in upgrades between versions


Coordinating pre
-
shared keys can be
problematic


Sometimes agents become unresponsive


Volume of alerts


In testing OSSEC is great for early warning
but not so good in a post compromise situation

©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

OSSEC Community


Extremely active user community

-
Developer mailing list


OSSEC mailing list (and Google group)

-
http://groups.google.com/group/ossec
-
list


OSSEC wiki

-
http://www.ossec.net/wiki

Commercial support from Trend Micro


ossec.purchase@trendmicro.com


OSSEC HIDS GUIDE

by Hay, Cid, and Bray. Syngress
2008



©Justin C. Klein Keane
<jukeane@sas.upenn.edu>

Questions

Thank you.


http://www.MadIrish.net


justin@MadIrish.net


@madirish2600