Installing an Up-to-date open source IDS to monitor network security on a tightly

hamburgerfensuckedΑσφάλεια

20 Νοε 2013 (πριν από 3 χρόνια και 4 μήνες)

195 εμφανίσεις




Installing an Up
-
to
-
date open source IDS to monitor network security on a tightly
controlled Operating System



By

Greg Williams


CS 691

University of Colorado Colorado Springs




Network attacks, viruses, and malware are
prevalent across the internet today.
Organizations must monitor and detect these threats to ensure their data is safe from
accidental or malicious agents. One monitoring and detection engine put into place in many
organizations is called a Network Intrus
ion Detection System.
The purpose for building this
project and building an IDS was to find out what was going on inside the network aside from
what we already block on the firewall side.
This system can detect malicious activity, track
viruses, and moni
tor networks.
The system can be implemented a number of different ways,
but the most secure, in my opinion, is a system built upon a tested, production operating
system, and that has up to date code.
In any organization, information technology
department
s have a set of standards that are thoroughly tested before they are put into a
production environment. The reason they are tested is because they don’t know how the
software is going to react with the rest of the environment. There are compatibility iss
ues to
work out, security hole
s that may need to be patched,
and performance issues both on the
hardware and network side that need to be looked at. For the University of Colorado Colorado
Springs, this is no exception. We thoroughly test our software

a
nd hardware before we put it

into production. We do not tend to install beta software and perhaps can be a version behind
on some software updates except in the cases of security patches which are always up to date.
This paper is going to talk about the
steps that have been taken to ensure that a network
intrusion detection system (Snort) is installed on a thoroughly
tested, known to be compatible,
security patched information technology built image. This causes problems along the way
because some of the

software that is required of the NIDS is not standard for our base image.
There are plenty of pre
-
packaged appliances out there that can instantly install the software we
need, however they are typically not customized
systems

based upon the organization
’s base
operating system.
We’ll talk about the base operating system and what features it has on it
that we must have in any production system, the network intrusion detection system itself and
the installation process, and how we have tweaked our NIDS.

F
irst let’s talk about the operating system. The operating system that we have chosen is
CentOS The major version of CentOS, version 5.0, came out in 2007. The latest version of
CentOS is version 6.0, which came out July 10, 2011. The installation of Cen
tOS and the NIDS
took place the week of
June 20, 2011, when the latest version of CentOS was 5.6. The reason
we have chosen CentOS is because it is 100% binary compatible with Red Hat Enterprise Linux.
Many
organizations

use CentOS because it is based up
on Red Hat Enterprise Linux. Red Hat
Enterprise Linux is thoroughly tested before it is released to market. Since Linux is open source
and there are many developers working on different packages that are included in Linux,
thus
the base OS must be tightl
y controlled. Repositories of software may not include the latest
versions of software because they haven’t

been fully tested by Red Hat
to be included in the
release. Another reason
organization’
s

use CentOS is because it is free. Red Hat Enterprise
L
inux costs $50/year/license. If organizations have hundreds of servers, this could
add up to a
hefty expenditure. The repositories of CentOS and RHEL (Red Hat Enterprise Linux) include
many different software packages that are used for security and auditi
ng.

In any organization information security and auditing must be thought of first and
foremost in order to have a safe computing environment. Organizations such as UCCS develop
server images to
guarantee

that base server operating systems are the same from installation
to installation and are compatible with their environme
nt, while maintaining strict access to the
server. UCCS includes many packages to ensure that security and auditing remain the same
f
rom server to server. While I cannot talk about exactly what is installed in the base image,
some components are standard such as log auditing packages, user access packages,
and
security packages.
These packages help us control the server environment ag
ainst malicious
attacks, accidental data breaches, and also help user access and control. All these packages are
standard and are available from the main CentOS and EPEL repositories. Customizing servers
can then be the challenging part since many packag
es that are offered do not come in a
repository or standard package or the packages are older and we need a newer version.

When
customizing a server to serve a specific purpose, the software that is installed may require
packages or binaries that are not
a part of any repository or may not be compati
ble with the
kernel of the operating system.
Such is the case with some of the packages needed for the
installation of Snort.



Snort is a Network Intrusion System (NIDS) or can be used as a Network Intrusio
n
Prevention System (NIPS).
Snort was created in 1998 by Martin Roesch and is still being actively
maintained today. One of the primary drivers why I chose Snort is because it has a large
community following and allows a user to integrate their own custo
m rules. Snort identifies
intrusions via signature matching, protocol analysis and anomaly based packet inspection. This
is key for an environment like UCCS where you must monitor protocol activity, intrusions based
off signatures and be alerted to anoma
lies in the network.
Since attacks on a network and
users are always changing it is best to keep up to date with the latest IDS/IPS signatures and
major Snort software
releases.

Snort’s latest version is at 2.9.0.5 which was released in April of 2011.
Version 2.9.0 was
released in September 2010 and changed the way packets are processed therefore resulting in
software requirements that were outside RHEL
-
like repositories. Snort version 2.9.0 introduced
the Data AcQuisition library for packet input and
output. The library replaces direct use of
older pcap libraries with an abstraction layer that makes it easier to add additional software or
hardware packet capture implementations. Basically it allows you to turn IDS into IPS without
recompiling. It al
so adds AFPACKET support which allows you to quickly turn the system into an
IPS without messing around with iptables and packet queues. While this new library helps with
packet processing, it requires that we use a libpcap library that is newer than the
newest
libpcap in a RHEL repository. This means that we must download and compile libpcap on our
own.

Before we start to download and recompile libpcap, let’s first remove the old libpcap.

yum remove libpcap libpcap
-
devel

Next let’s start downloading the
packages necessary for the installation of libpcap and Snort
.
These packages are easy to install because they are a part of the RHEL repository.


yum install gcc gcc
-
c++ mysql mysql
-
server libtool.x86_64

While we’re at it, let’s also install packages that

will be necessary to look at our output of Snort
.

The following packages

are necessary to install for BASE, which is the web front end for viewing
and analyzing Snort
alerts
.


yum install httpd php mod
-
ssl php
-
mysql mysql
-
shared
-
compat mod
-
auth
-
mysql

php
-
pear_numbers_roman php
-
pear_words php
-
pear_image_color php
-
pear_image_canvas php
-
pear_image_graph

Now we need to download
and install
the source code of all the other packages that we can’t
get via a RPM or via repository.

Now let’s configure a couple of
things before we actually start installing code. First we’ll need to
create a new user and group for all snort activities and files. We don’t want them running as a
normal user.

groupadd snort useradd

g

snort snort

s /sbin/nologin

Next we’ll need to configure the mysql database.

echo “SET PASSWORD FOR root@localhost=PASSWORD(‘xxxxxxxx’);

| mysql

u root
-
p

mysql

u root

p

D snort <./schemas/create_mysql

echo “grant create,insert on root.* to snort@localhost” | mysql

u root

p

ec
ho “SET PASSWORD FOR snort@localhost=PASSWORD(‘xxxxxxxx’);

| mysql

u root

p

echo “grant create, insert, select, delete, update on snort.* to
snort@localhost” | mysql

u root

p

We always keep downloaded source code in the same place so we can
retrieve it

later if
necessary.

cd /usr/src

Libpcap 1.1.1 source code



wget http://www.tcpdump.org/release/libpcap
-
1.1.1.tar.gz

tar
-
zxvf libpcap
-
1.1.1.tar.gz

cd libpcap
-
1.1.1

./configure

prefix=/usr

make all

make install

Data Acqusistion soruce code



wget http:/
/www.snort.org/downloads/860

tar

zxvf daq
-
0.5

cd daq
-
0.5

./configure

with
-
libpcap
-
libraries=/usr/lib64/

Snort 2.9.0 source code

wget
http://www.snort.org/downloads/867

tar

zxvf snort
-
2.9.0

cd
snort
-
2.9.0

./configure

with
-
mysql
-
libraries=/usr/lib64/mysql/

enable
-
dynamicplugin

with
-
libpcap
-
libraries=/usr/lib64

with
-
daq
-
libraries=/usr/local/lib/daq

enable
-
zlib

make

make install


We’ll also need to download and install the rules for processin
g. There are 2 places I like
to get the rules. First is from Snort. These are 30 days old, but they still work.
You will
have to register for them at
http://www.snort.org

and then download them to another
machine and then scp them over to your snort di
rectory.
Second is from Emerging
Threats

at http://www.emergingthreats.net
.

cd /etc/snort/

wget
http://rules.emergingthreats.net/open
-
nogpl/snort
-
2.9.0/emerging.rules.tar.gz

tar

zxvf snortrules
-
snapshot
-
290.tar.gz

tar

zxvf emerging.rules.tar.gz


Both of these will unzip the files
in the /etc/snort/rules/ folder, except for the so_rules
which we need to copy over

since they are specific to the OS architecture
.

cp ../so_rules/precompiled/Cent)S
-
5.0/i386/2.9.0/* /etc/snort/so_rules

We now need to con
figure Snort where to look for these rules and add the emerging
threats rules.

vi /etc/snort/snort.conf

Change the following lines:

“var RULE_PATH ../rules” to “var RULE_PATH /etc/snort/rules”

“var SO_RULE_PATH ../so_rules” to “var SO_RULE_PATH /etc/snort/
so_rules”

Add the following line past “Output”

output unified2: filename snort.log, limit 128

A
dd the following lines to the bottom of the file

include $RULE_PATH/classification.config

include $RULE_PATH/reference.config

include $RULE_PATH/emerging
-
trojan.
rules

include $RULE_PATH/emerging
-
pop3.rules

include $RULE_PATH/emerging
-
user_agents.rules

include $RULE_PATH/emerging
-
activex.rules

include $RULE_PATH/emerging
-
rpc.rules

include $RULE_PATH/emerging
-
virus.rules

include $RULE_PATH/emerging
-
attack_response.rules

include $RULE_PATH/emerging
-
icmp.rules

include $RULE_PATH/emerging
-
scan.rules

include $RULE_PATH/emerging
-
scada.rules

include $RULE_PATH/emerging
-
voip.rules

include $RULE_PATH/emerging
-
chat.rules

include $RULE_PATH/emerging
-
icmp_info
.rules

include $RULE_PATH/emerging
-
shellcode.rules

include $RULE_PATH/emerging
-
web_client.rules

include $RULE_PATH/emerging
-
imap.rules

include $RULE_PATH/emerging
-
web_server.rules

include $RULE_PATH/emerging
-
current_events.rules

include $RULE_PATH/emergi
ng
-
smtp.rules

include $RULE_PATH/emerging
-
web_specific_apps.rules

include $RULE_PATH/emerging
-
deleted.rules

i
nclude $RULE_PATH/emerging
-
malware.rules

include $RULE_PATH/emerging
-
snmp.rules

include $RULE_PATH/emerging
-
worm.rules

include $RULE_PATH/emergin
g
-
dns.rules

include $RULE_PATH/emerging
-
misc.rules

include $RULE_PATH/emerging
-
sql.rules

include $RULE_PATH/emerging
-
dos.rules

include $RULE_PATH/emerging
-
netbios.rules

include $RULE_PATH/emerging
-
telnet.rules

include

$RULE_PATH/emerging
-
exploit.rules

include $RULE_PATH/emerging
-
tftp.rules

include $RULE_PATH/emerging
-
mobile_malware.rules

include $RULE_PATH/emerging
-
dshield
-
BLOCK.rules

include $RULE_PATH/emerging
-
rbn.rules

include $RULE_PATH/emerging
-
ciarmy.rules


A
dodb. Adodb is a database abstraction library for PHP. This is for use with BASE.


wget
http://sourceforge.net/projects/adodb/files/adodb
-
php5
-
only/adodb
-
511
-
for
-
php5/adodb511.tgz/download

tar

zvxf adodb511.tgz

mv adodb5/
/var/www/
adodb/


BASE 1.4.5.

BASE is a front end GUI for reviewing Snort alerts.

wget
http://sourceforge.net/projects/secureideas/files/BASE/base
-
1.4.5/base
-
1.4.5.tar.gz/download

mkdir /var/www/base/

tar

zvxf base
-
1.4.5 /var/www/html/base/

cd /var/www/base/

mv base_conf.php.dist ba
se_conf.php

vi base_conf.php


Add the following lines:


$BASE_urlpath=’/base’;

$DBlib_path = ‘/var/www/adodb’

$DBtype = ‘mysql’;

$alert_dbname = ‘snort’

$alert_host = ’localhost’

$alert_port = ‘’;

$alert_user = ‘snort’;

$alert_password = ‘xxxxxx’


Next we
need to go to a web browser and continue the setup


https://xxx.xxx.xxx.xxx/base


Click on “Setup Page” then “Create BASE AG”

I really don’t want anyone looking at these alerts except me, so I
’ll lock it down with
a
.htpasswd file.

mkdir /opt/password/

/usr/bin/htpasswd

c /opt/password/passwords base


You’ll be prompted to enter a password.


Next you’ll have to edit your httpd.conf file.


vi /etc/httpd/conf/httpd.conf


Add the following lines:

<Directory "/var/www/html/base">


AuthType Basic


AuthName "SnortIDS"


AuthUserFile /var/www/passwords/passwords


Require user base

</Directory>


Now restart Apache.

service httpd restart

Barnyard2 1.9

source code
Barnyard2 off
loads alert
processing to a separate process so that
snort can focus on processing packets



wget
http://www.securixlive.com/download/barnyard2/barnyard2
-
1.9.tar.gz

tar xvzf barnyard2
-
1.9.tar.gz

cd barnyard2
-
1.9

./configure
--
with
-
mysql
-
libraries=/usr/lib
64/mysql
--
with
-
libpcap
-
libraries=/usr/lib64/libpcap1/lib64/

make

make install

cp /etc/barnyard2.conf /etc/snort

mkdir /var/log/barnyard2

chmod 666 /var/log/barnyard2

touch /var/log/snort/barnyard2.waldo

chown snort:snort /var/log/snort/barnyard2.waldo

vi
/etc/snort/barnyard2.conf



Find and change the following entries


config hostname: localhost


config interface:eth0


output database: log,mysql, user=snort password=’xxxxxxxxx’
dbname=snort host=localhost

All we have left is to create the processes for starting Snort and Barnyard2. We’ll put these in
the
OS
startup scripts in case the server dies.


/usr/local/bin/snort

D

u snort

g snort

c /etc/snort/snort.conf

I eth0

/usr/local/bin/barnyard2

c /etc/s
nort/barnyard2.conf

d /var/log/snort

f
snort.log

w /var/log/snort/barnyard2.waldo
-
D


We can now start the installation by typing in /etc/rc.local start

After Snort is running, we can see the interface by typing in
http://xxx.xxx.xxx.xxx/base
. Next
we must enter our username and password in that we created with htpassword. Then we’ll
come to a screen like this:


From here we can see all alerts that are coming through without looking through the logs.
After w
e have collected the logs for a while we can start to tweak our installation of Snort.

There are two ways in which I have tweaked my installation. The first way is
suppressing alerts that are either too chatty and are false positives, or are alerts that I

really
don’t care about, meaning they are alerts that I don’t mind if they are

real or not. The problem
with
an IDS that is signature based is that it will produce quite a few false positives. These false
positives can easily be filtered out within the
threshold.conf file. The threshold.conf file is
located in /etc/snort/. We want to suppress findings, so the lines that we’ll put in look like this:
suppress gen_id 1, sig_id 16008

where 16008 is the
signature ID
. We can also rate limit
alerts in this file as well.
The second way I have tweaked my installation is building my own
custom rule sets and networks.

Snort allows you to create your own signatures in the local.rules file located within
/etc/snort/rules f
older. A user can always add more files, but they will just need to include
them in the snort.conf file for processing. Building custom rules is fairly straight forward. If you
know what you want to search for within a packet, you can create a rule with

the following:

alert tcp $HOME_NET any
-
> any $HTTP_PORTS (msg:custom message alert";
content:”content of the packet"; nocase; uricontent: "part of the uri
string”; sid:xxxxxxx;)

msg: What message do you want displayed

on the alert

content: What is the c
ontent contained with the packet that you are alerting for

uricontent: is there are particular
string that you are searching for within the URL

sid: this is the
unique signature id of the rule
. All
rules

must have a different sid.


Another aspect of
the
Snort rules

is the network (NET shown above). I have also created new
networks to monitor for different data in different places within our large
r

network. For
example, I only want to monitor for sensitive data in a couple different subnets, not the enti
re
network. I
have

create
d

a new network within snort.conf called
$
CC_NET. This is defined as
var $CC_NET xxx.xxx.xxx.xxx/24. I
have then
replace
d

the $HOME_NET string found in most of
the

rules

with my custom $CC_NET network and Snort will only look f
or those signatures
matching the
alert

and the network.

Snort always

must be updated with new rules to keep up
with new threats. Overall it has taken weeks of tweaking to fine tune my Snort installation.
This will always be an ongoing process. In addit
ion to tweaking the system, you must also view
alerts and act upon those alerts.


A network intrusion detection system is only one piece of many in our security arsenal.
Catching viruses that antivirus can’t catch or botnets is one thing an IDS can do wel
l, however
you have to be monitoring your IDS for this information then act upon them.
Sometimes
monitoring security incidents from outside a system is the only way to truly know you have an
incident. The next phases of the project will be two
-
fold. The

first phase will be to implement a
commercial IPS which will be in front of the open
-
source IDS. The reason for this is that I can
still create custom rules with the open
-
source IDS to monitor different network activity. It will
also help gauge how effe
ctive
the commercial IPS really is. The final phase will be to implement
a honeynet and create rules based upon known
honeynet
activity.

All organizations need to know what is going on with their network. An IDS

is an
excellent way of knowing what is going on with your network. It can help identify problem
areas within your network, such as systems with ports open that shouldn’t be, or it can help
identify where a rogue system is sending out massive amounts of s
pam. An edge firewall
cannot always block everything, so together with an IDS/IPS, your network is a little safer.