Intrusion Detection & Network Forensics

sweetlipscasteSecurity

Nov 2, 2013 (3 years and 11 months ago)

83 views

1

Intrusion Detection

&

Network Forensics


Lucius L. Millinder Jr.

security@secureitconsulting.us

Chief Technology Officer

Secure
-
IT Consulting, Inc.

2

An ounce of
prevention is worth a
pound of detection

3

Why Talk about IDS?


Emerging new technology


Very interesting

...but...


About to be over
-
hyped


Being informed is the best weapon in
the security analyst’s arsenal


It also helps keep vendors honest!


4

What is an Intrusion?!


Difficult to define


Not everyone agrees


This is a
big

problem


How about someone telneting your system?


And trying to log in as “root”?


What about a ping sweep?


What about them running an ISS scan?


What about them trying phf on your webserver?


What about succeeding with phf and logging in?

5

What is IDS?


The ideal Intrusion Detection System
will notify the system/network manager
of a successful attack in progress:


With 100% accuracy


Promptly (in under a minute)


With complete diagnosis of the attack


With recommendations on how to block it

…Too bad it doesn’t exist!!

6

Objectives: 100% Accuracy
and 0% False Positives


A False Positive is when a system
raises an incorrect alert


“The boy who cried ‘wolf!’” syndrome


0% false positives is the goal


It’s easy to achieve this:
simply detect
nothing


0% false negatives is another goal:
don’t let an attack pass undetected

7

Objectives: Prompt
Notification


To be maximally accurate the system
may need to “sit on” information for a
while until all the details come in


e.g.: Slow
-
scan attacks may not be
detected for hours


This has important implications for how
“real
-
time” IDS can be!


IDS should notify user as to detection lag


8

Objectives: Prompt
Notification
(cont)


Notification channel must be protected


What if attacker is able to sever/block
notification mechanism?


An IDS that uses E
-
mail to notify you is
going to have problems notifying you that
your E
-
mail server is under a denial of
service attack!

9

Objectives: Diagnosis


Ideally, an IDS will categorize/identify
the attack


Few network managers have the time to
know intimately how many network attacks
are performed


This is a difficult thing to do


Especially with things that “look weird” and
don’t match well
-
known attacks

10

Objectives: Recommendation


The ultimate IDS would not only identify
an attack, it would:


Assess the target’s vulnerability


If the target is vulnerable it would notify the
administrator


If the vulnerability has a known “fix” it
would include directions for applying the fix


This requires huge, detailed knowledge

11

IDS: Pros


A reasonably effective IDS can identify


Internal hacking


External hacking attempts


Allows the system administrator to
quantify the level of attack the site is
under


May act as a backstop if a firewall or
other security measures fail

12

IDS: Cons


IDS’ don’t typically act to prevent or
block attacks


They don’t replace firewalls, routers, etc.


If the IDS detects trouble on your
interior network what are you going to
do?


By definition
it is already too late

13

Paradigms for Deploying IDS


Attack Detection


Intrusion Detection

14

Internal

Network

Internet

Router

w/some

screening

Firewall

DMZ

Network

WWW

Server

Desktop

Attack Detection

IDS detects (and counts) attacks against

the Web Server and firewall

IDS

15

Attack Detection


Placing an IDS outside of the security
perimeter records
attack level


Presumably if the perimeter is well
designed the
attacks should not affect it!


Still useful information for management
(“we have been attacked 3,201 times this
month…)


Prediction: AD Will generate a lot of noise
and be ignored quickly

16

Internal

Network

Internet

Router

w/some

screening

Firewall

DMZ

Network

WWW

Server

Desktop

Intrusion Detection

IDS detects hacking activity WITHIN

the protected network,
incoming or outgoing

IDS

17

Intrusion Detection


Placing an IDS within the perimeter will
detect instances of clearly improper
behavior


Hacks via backdoors


Hacks from staff against other sites


Hacks that got through the firewall


When the IDS alarm goes off, it’s a red
alert

18

Attack vs Intrusion Detection


Ideally do
both


Realistically, do ID first then AD


Or, deploy AD to justify security effort to
management, then deploy ID (more of a
political problem than a technical one)


The real question here is one of staffing
costs to deal with alerts generated by
AD systems

19

IDS Data Source Paradigms


Host Based


Network Based

20

Host Based IDS


Collect data usually from within the
operating system


C2 audit logs


System logs


Application logs


Data collected in very compact form


But application / system specific

21

Host Based: Pro


Quality of information is very high


Software can “tune” what information it
needs (e.g.: C2 logs are configurable)


Kernel logs “know” who user is


Density of information is very high


Often logs contain pre
-
processed
information (e.g.: “badsu” in syslog)


22

Host Based: Con


Capture is often highly system specific


Usually only 1, 2 or 3 platforms are
supported (“you can detect intrusions on
any platform you like as long as it’s Solaris
or NT!”)


Performance is a wild
-
card


To unload computation from host logs are
usually sent to an external processor
system

23

Host Based: Con
(cont)


Hosts are often the target of attack


If they are compromised their logs may be
subverted


Data sent to the IDS may be corrupted


If the IDS runs on the host itself it may be
subverted

24

Host Based IDS


Signature log analysis


application and system


File integrity checking


MD5 checksums


Enhanced Kernel Security


API access control


Stack security


Network Monitoring Hybrids

25

Host Based IDS Limitations


Places load on system


Disabling system logging


Kernel modifications to avoid file
integrity checking (and other stuff)


Management overhead


Network IDS Limitations


26

messages

xfer

access_log

secure

sendmail

27

messages

xfer

access_log

secure

sendmail

One

Security

Log

28

Network IDS


Searches for patterns in packets


Searches for patterns of packets


Searches for packets that shouldn't be
there


May ‘understand’ a protocol for effective
pattern searching and anomaly detection


May passively log, alert with SMTP/SNMP
or have real
-
time GUI

29

Network IDS Limitations


Obtaining packets
-

topology & encryption


Number of signatures


Quality of signatures


Performance


Network session integrity


Understanding the observed protocol


Disk storage

30

/cgi
-
bin/phf

Jane used

the PHF

attack!

31

NMAP

Jane did

a port

sweep!

32

Network Based IDS


Collect data from the network or a hub /
switch


Reassemble packets


Look at headers


Try to determine what is happening
from the contents of the network traffic


User identities, etc inferred from actions

33

Network Based: Pro


No performance impact


More tamper resistant


No management impact on platforms


Works across O/S’


Can derive information that host based
logs might not provide (packet
fragmenting, port scanning, etc.)


34

Network Based: Con


May lose packets on flooded networks


May mis
-
reassemble packets


May not understand O/S specific
application protocols (e.g.: SMB)


May not understand obsolete network
protocols (e.g.: anything non
-
IP)


Does not handle encrypted data


35

IDS Paradigms


Anomaly Detection
-

the AI approach


Misuse Detection
-

simple and easy


Burglar Alarms
-

policy based detection


Honey Pots
-

lure the hackers in


Hybrids
-

a bit of this and that

36

Anomaly Detection


Goals:


Analyse the network or system and infer
what is normal


Apply statistical or heuristic measures to
subsequent events and determine if they
match the model/statistic of “normal”


If events are outside of a probability
window of “normal” generate an alert
(tuneable control of false positives)

37

Anomaly Detection

(cont)


Typical anomaly detection approaches:


Neural networks
-

probability
-
based pattern
recognition


Statistical analysis
-

modelling behavior of
users and looking for deviations from the
norm


State change analysis
-

modelling system’s
state and looking for deviations from the
norm

38

Anomaly Detection: Pro


If it works it could conceivably catch any
possible attack


If it works it could conceivably catch
attacks that we haven’t seen before


Or close variants to previously
-
known
attacks


Best of all it won’t require constantly
keeping up on hacking technique

39

Anomaly Detection: Con


Current implementations don’t work
very well


Too many false positives/negatives


Cannot categorize attacks very well


“Something looks abnormal”


Requires expertise to figure out what
triggered the alert


Ex: Neural nets can’t say
why

they trigger

40

Anomaly Detection: Examples


Most of the research is in anomaly
detection


Because it’s a harder problem


Because it’s a more interesting problem


There are many examples, these are
just a few


Most are at the proof of concept stage

41

Misuse Detection


Goals:


Know what constitutes an attack


Detect it


42

Misuse Detection

(cont)


Typical misuse detection approaches:


“Network grep”
-

look for strings in network
connections which might indicate an attack
in progress


Pattern matching
-

encode series of states
that are passed through during the course
of an attack


e.g.: “change ownership of
/etc/passwd

-
>
“open

/etc/passwd

for write”
-
>
alert

43

Misuse Detection: Pro


Easy to implement


Easy to deploy


Easy to update


Easy to understand


Low false positives


Fast

44

Misuse Detection: Con


Cannot detect something previously
unknown


Constantly needs to be updated with
new rules


Easier to fool


45

Burglar Alarms


A burglar alarm is a misuse detection
system that is carefully targeted


You may not care about people port
-
scanning your firewall from the outside


You may care profoundly about people
port
-
scanning your mainframe from the
inside


Set up a misuse detector to watch for
misuses violating site policy

46

Burglar Alarms

(cont)


Goals:


Based on site policy alert administrator to
policy violations


Detect events that may not be “security”
events which may indicate a policy
violation


New routers


New subnets


New web servers

47

Burglar Alarms

(cont)


Trivial burglar alarms can be built with
tcpdump and perl


Netlog and NFR are useful event
recorders which may be used to trigger
alarms

http://www.nswc.navy.mil/ISSEC/Docs/loggingproject.html

ftp://coast.cs.purdue.edu/pub/tools/unix/netlog/

http://www.nfr.net/download

48

Burglar Alarms

(cont)


The ideal burglar alarm will be situated
so that it fires when an attacker
performs an action that they normally
would try once they have successfully
broken in


Adding a userid


Zapping a log file


Making a program setuid root

49

Burglar Alarms

(cont)


Burglar alarms are a big win for the
network manager:


Leverage local knowledge of the local
network layout


Leverage knowledge of commonly used
hacker tricks

50

Burglar Alarms: Pro


Reliable


Predictable


Easy to implement


Easy to understand


Generate next to no false positives


Can (sometimes) detect previously
unknown attacks

51

Burglar Alarms: Con


Policy
-
directed


Requires knowledge about your network


Requires a certain amount of stability
within your network


Requires care not to trigger them
yourself

52

Honey Pots


A honey pot is a system that is
deliberately named and configured so
as to invite attack


swift
-
terminal.bigbank.com


www
-
transact.site.com


source
-
r
-
us.company.com


admincenter.noc.company.net

53

Honey Pots

(cont)


Goals:


Make it look inviting


Make it look weak and easy to crack


Instrument every piece of the system


Monitor all traffic going in or out


Alert administrator whenever someone
accesses the system

54

Honey Pots

(cont)


Trivial honey pots can be built using
tools like:


tcpwrapper


Burglar alarm tools (see “burglar alarms”)


restricted/logging shells (sudo, adminshell)


C2 security features (ugh!)


See Cheswick’s paper “An evening with
Berferd” for examples

55

Honey Pots: Pro


Easy to implement


Easy to understand


Reliable


No performance cost

56

Honey Pots: Con


Assumes hackers are really stupid


They aren’t

57

Firewalls as an IDS


Excellent source of network probe,
attack and misuse information


Detect policy deviations based on
access control lists


Some have “NIDS” capabilities

58

Network Honeypots


Sacrificial system(s) or sophisticated
simulations


Any traffic to the honeypot is considered
suspicious


If a scanner bypassed the NIDS, HIDS
and firewalls, they still may not know
that a Honeypot has been deployed

59

honeypot

HTTP

DNS

Firewall

60

Hybrid IDS


The current crop of commercial IDS are
mostly hybrids


Misuse detection (signatures or simple
patterns)


Expert logic (network
-
based inference of
common attacks)


Statistical anomaly detection (values that
are out of bounds)

61

Hybrid IDS

(cont)


At present, the hybrids’ main strength
appears to be the misuse detection
capability


Statistical anomaly detection is useful more
as backfill information in the case of
something going wrong


Too many false positives
-

many sites turn
anomaly detection off

62

Hybrid IDS

(cont)


The ultimate hybrid IDS would
incorporate logic from vulnerability
scanners*


Build maps of existing vulnerabilities into
its logic of where to watch for attacks


Backfeed statistical information into
misuse detection via a user interface

* Presumably, a clueful network

admin would just fix the vulnerabilty

63

Books


Internet Security and Firewalls:
Repelling the Wily Hacker
, by Bill
Cheswick and Steve Bellovin, from
Addison Wesley


Internet Firewalls,
by Brent Chapman
and Elizabeth Zwicky


64

URLs


Hacker sites: the fringe


http://www.2600.com


http://www.digicrime.com


http://www.zone
-
h.org/defaced/2003/01/30/www.defensivet
hinking.com/hacked.html


http://www.website.to/hacker

65

Addresses


CERT


cert@cert.org


Firewalls mailing list


firewall
-
wizards
-
request@honor.icsalabs.com:
subscribe
firewalls


Web security mailing list


listserv@lehigh.edu:
subscribe www
-
security