Ph.D. Confirmation Report

dimerusticΔίκτυα και Επικοινωνίες

23 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

208 εμφανίσεις


1


Ph.D. Confirmation Report




Implement Novel Techniques for
I
ntrusion

Detection in Honeynets, for
Automated IDS Signature Engineering





Fahim Abbasi


Supervisor: Prof. Richard Harris


School of Engineering & Advanced Technology (SEAT)

Massey
University





March 16, 2010


© Copyright by Fahim Abbasi 2010


All Rights Reserved





2

Table of Contents




CHAPTER 1: INTRODUC
TION
................................
................................
................................
....

5

1.1. INTRODUCTION

................................
................................
................................
.....................

5

1.2. DEFINIT
IONS

................................
................................
................................
..........................

7

1.2.1.

I
NFORMATION
S
ECURITY

................................
................................
................................
........

7

1.2.2.

C
OMPUTER
S
ECURITY

................................
................................
................................
.............

7

1.2.3.

N
ETWORK
S
ECURITY

................................
................................
................................
..............

7

1.2.4.

S
ECURITY
S
TANDARDS AND
D
OCUMENTS

................................
................................
................

8

1.3.

B
LACK
H
ATS

................................
................................
................................
..............................

9

1.4.

W
HITE
H
ATS

................................
................................
................................
..............................

9

1.5. ATTACKS AND ATT
ACK CLASSIFICATION

................................
................................
....

10

1.5.1.

A
CTIVE
A
TTACKS
................................
................................
................................
..................

10

1.5.2.

P
ASSIVE
A
TTACKS

................................
................................
................................
.................

10

1.6.

T
AXONOMY OF ATTACKS
................................
................................
................................
..........

11

CHAPTER 2: PROBLEM S
TATEMENT

................................
................................
.....................

12

2. SECURITY PROBLEM
................................
................................
................................
.............

12

2.1.

C
URRENT
S
CENARIO

................................
................................
................................
................

12

2.2.

C
OST
................................
................................
................................
................................
........

13

2.3.

W
HAT
P
EOPLE
S
AY
A
BOUT
S
ECURITY

................................
................................
.....................

14

2.4.

N
EEDS

................................
................................
................................
................................
......

15

CHAPTER 3: MOTIVATIO
N AND RESEARCH CHALL
ENGES

................................
..............

15

3.1.

M
OTIVATION

................................
................................
................................
...........................

15

3.2.

O
BJECTS THAT DEMAND S
ECURITY

................................
................................
..........................

16

3.3.

W
HO IS TO
B
LAME
?

................................
................................
................................
.................

17

3.4.

A

F
EW
D
OCUMENTED
A
TTACKS
................................
................................
...............................

17

3.5.

M
OVING
T
OWARDS A
S
OLUTION

................................
................................
..............................

18

3.6. HONEYPOTS AND H
ONEYNETS

................................
................................
........................

19

3.6.1.

W
HO
.

W
HAT
.

W
HERE
,

WHY AND HOW
?

................................
................................
.................

19

3.6.2. HONEYPOTS

................................
................................
................................
......................

19


3

3.6.2.1.

M
OT
IVATION AND
C
ONCEPT
................................
................................
...............................

20

3.6.2.2.

C
LASSIC
E
XAMPLES

................................
................................
................................
................

20

3.6.2.3.

D
ISCUSSING
E
XPLOITS
................................
................................
................................
........

20

3.6.2.4.

E
XAMPLE
:

L
EAVES
W
ORM

................................
................................
................................
.

21

3.6.2.5.

E
XAMPLE
:

C
ODE
R
ED
II

W
ORM

................................
................................
.........................

21

3.6.2.6.

E
XAMPLE
:

S
OLARIS
DTSCD

EXPLOIT

................................
................................
................

21

3.6.3. HONEYNETS

................................
................................
................................
......................

21

3.6.3.1.

D
ATA
C
ONTROL

................................
................................
................................
.................

22

3.6.3.2.

D
ATA
C
APTURE

................................
................................
................................
..................

22

3.6.3.3.

D
ATA
C
OLLECTION

................................
................................
................................
............

22

3.6.3.4.

H
ONEYNET
A
RCHITECTURES

................................
................................
..............................

23

3.6.3.4.1.

G
ENERATION
I

A
RCHITECTURE

................................
................................
...........................

23

3.6.3.4.2.

G
ENERATION
II

AND
III

A
RCHITECTURE
:

................................
................................
.............

23

3.6.3.5.

V
IRTUAL
H
ONEYNET
................................
................................
................................
.............

24

3.7. RESEARCH CHALLE
NGE
# 1

................................
................................
..............................

24

3.7.1.

A
RCHITECTURE AND
D
ESIGN
C
ONSIDERATIONS IN
V
IRTUAL
H
ONEYNETS

.............................

24

3.7.2.

I
NTRODUCTION
................................
................................
................................
........................

24

3.8. RES
EARCH CHALLENGE # 2

................................
................................
..............................

25

3.8.1.

I
NTRUSION
D
ETECTION

................................
................................
................................
.........

25

3.8.2.

I
NTRUSION
D
ETECTION
P
ROBLEM

................................
................................
.........................

25

3.8.3.

I
NTRUSION
D
ETECTION
S
IGNATURES

................................
................................
....................

26

3.8.4.

A
UTOMATED
S
IGNATURE
E
NGINEERING

................................
................................
...............

26

CHAPTER 4: OVERVIEW
OF RELATED WORKS

................................
................................
...

26

4.1.

H
ONEYPOTS AS ATTACK D
ETECTION AND LEARNIN
G TOOLS

................................
.....................

26

4.2.

A
UTOMATED
S
IGNATURE
E
NGINEERING USING
H
ONEYPOTS

................................
....................

27

4.3.

A
NOMALY
D
ETECTION

................................
................................
................................
.............

29

4.4.

N
ETWORK
B
EHAVIOURAL
A
NALYSIS
(NBA)
................................
................................
.............

30

CHAPTER 5: RESEARCH
QUESTIONS

................................
................................
.....................

30

CHAPTER 6: METHODOLO
GY REVIEW

................................
................................
.................

31

6.1. PROPOSED SYSTEM

FOR VIRTUAL HONEYNET

ARCHITECTURE PROBLEM

.........

31

6.1.2.

M
ETHODOLOGY AND
D
ISCUSSION

................................
................................
.........................

31

6.1.3.

U
BUNTU AS
H
ONEYPOT
................................
................................
................................
............

33

6.1.4.

VMW
ARE AS
V
IRTUALIZATION
S
OFTWARE

................................
................................
...............

34

6.1.5.

H
ONEYWALL
R
OO

................................
................................
................................
...................

34

6.1.6.

S
EBEK AS DATA CAPTURE

TOOL

................................
................................
................................

35

6.2. PROPOSED SYSTEM

FOR AUTOMATED SIGNAT
URE ENGINEERING

........................

36


4

6.2.1.

D
ISCUSSION

................................
................................
................................
...........................

36

6.2.2.

M
ETHODOLOGY

................................
................................
................................
....................

37

6.2.2.1.

A
NALYSIS OF
S
YSTEM
E
VENTS
................................
................................
............................

37

6.2.2.2.

A
NALYSIS OF
N
ETWORK
E
VENTS

................................
................................
........................

37

6.2.2.3.

H
ASHING
A
LGORITHM FOR
P
AYLOAD
H
ASHING

................................
................................
.

38

6.2.2.4.

C
LUSTERING
B
Y
C
OMPRESSION
................................
................................
..........................

38

6.3.

R
ESULTS AND
D
ISCUSSION

................................
................................
................................
.......

40

CHAPTER 7: RESULTS

................................
................................
................................
...............

43

7.1.

S
UMMARY

................................
................................
................................
................................

43

7.2.

A
TTACK
S
TATISTICS

................................
................................
................................
................

44

7.2.1.

A
TTACKED
P
ORTS AND
S
ERVICES

................................
................................
.............................

44

7.2.2.

A
TTACKER
IP'
S

................................
................................
................................
........................

44

7.2.3.

A
TTACKER

S
C
OUNTRY OF
O
RIGIN

................................
................................
...........................

45

7.3.

F
ORENSIC
A
NALYSIS

................................
................................
................................
................

46

7.3.1.

F
IRST
H
ACK

................................
................................
................................
............................

46

7.3.2.

B
RUTE
F
ORCE AND
B
OTNETS
................................
................................
................................
....

46

7.3.3.

M
ORE
B
OTNETS
................................
................................
................................
.......................

46

7.3.4.

C
OORDINATED
A
TTACKS

................................
................................
................................
.........

47

7.3.5.

L
OCAL
P
RIVILEGE
E
SCALATION ATTEMPT

................................
................................
.................

47

7.3.6.

F
ORENSICS OF AN
E
NCRYPTED
B
OTNET
................................
................................
.....................

47

7.3.6.3.

F
ORENSICS OF A
H
ACKER

S
IRC

SESSION
................................
................................
................

48

8. ACHIEVEMENTS

................................
................................
................................
.....................

48

9. RESEARCH PLAN
................................
................................
................................
....................

49

REFERENCES:
................................
................................
................................
.............................

50

APPENDIX

-

A

................................
................................
................................
..............................

55

S
EBEK
L
OGS

................................
................................
................................
................................
.....

55

SSH

L
OGS

................................
................................
................................
................................
........

56



List of Figures


F
IGURE
1:

A
TTACK
C
ONSEQUENCES VS
L
IKELIHOOD
[84]
................................
................................
................................
.......

14

F
IGURE
2:

I
NTRUDER KNOWLEDGE VS

SOPHISTICATION OF AT
TACK
[42]
................................
................................
..............

16

F
IGURE
3:

I
NCIDENTS REPORTED TI
LL
2003

[37,

43]
................................
................................
................................
.................

17

F
IGURE
4:

T
HREAT CATEGORIES OVE
R TIME BY PER
CENT OF BREACHES
[50]
................................
................................
.......

18

F
IGURE
5:

G
EN
I

H
ONEYNET
A
RCHITECTURE
[12]

................................
................................
................................
....................

23

F
IGURE
6:

G
ENERATION
III

H
ONEYNET
A
RCHITECTURE
[12]

................................
................................
................................
.

24

F
IGURE
7:

P
ROPOSED
V
IRTUAL
H
ONEYNET
A
RCHITECTURE
................................
................................
................................
...

32

F
IGURE
8:

R
OO
L
OGICAL
D
ESIGN

................................
................................
................................
................................
................

35

F
IGURE
9:

B
EHAVIOURAL PROFILE F
OR
W32
-
B
AGLE
-
Q WORM
[94]
................................
................................
.......................

37

F
IGURE
10:

C
LUSTERING BY
C
OMPRESSION AND HASHI
NG

................................
................................
................................
.....

42

F
IGURE
11:

H
ONEYNET
D
ATA
G
RAPHICAL VIEW
(
IP
-
PORT
)
................................
................................
................................
.....

43

F
IGURE
12:

P
ROBED
P
ORTS

................................
................................
................................
................................
...........................

44

F
IGURE
14:

P
ROBED PORTS
(
EXCLUDING
SSH)

................................
................................
................................
.............................

1

F
IGURE
15:

T
OP
50

A
TTACKS BY
C
OUNTRY
................................
................................
................................
...............................

45



5

List of Tables


T
ABLE
1:

W
HAT
P
EOPLE SAY ABOUT SECU
RITY
?

................................
................................
................................
......................

15

T
ABLE
2:

H
ONEYPOT
:

C
LASSIC
E
XAMPLES

................................
................................
................................
................................

20

T
ABL
E
3:

H
ONEYPOT
:

D
ISCUSSING
E
XPLOITS
................................
................................
................................
............................

21

T
ABLE
4:

H
ONEYPOT
:

L
EAVES
W
ORM

................................
................................
................................
................................
........

21

T
ABLE
5:

H
ONEYPOT
:

C
ODE
R
ED
II

W
ORM
................................
................................
................................
................................

21

T
ABLE
6:

H
ONEYPOT
:

S
OLARIS
DTSCD

EXPLOIT

................................
................................
................................
.....................

21

T
ABLE
7:

SSH

PATCH FOR THE
H
ONEYPOT

................................
................................
................................
................................

33

T
ABLE
8:

SSH

L
OGS
................................
................................
................................
................................
................................
.......

34

T
ABLE
9:

C
OMPARISON
OF
MD5

AND
F
UZZY
H
ASHING
................................
................................
................................
...........

38

T
ABLE
10:

P
ROPOSED
H
ASHED
T
ECHNIQUE
................................
................................
................................
...............................

40

T
ABLE
11:

O
LD
T
ECHNIQUE
(NCD

ONLY
)

................................
................................
................................
................................
.

41

T
ABLE
12:

F
ORENSICS
:

H
ACK
................................
................................
................................
................................
.......................

46

T
ABLE
13:

F
ORENSICS
:

B
RUTE
F
ORCE AND
B
OTNETS

................................
................................
................................
..............

46

T
ABLE
14:

F
ORENSICS
:

M
ORE
B
OTNETS

................................
................................
................................
................................
.....

47

T
ABLE
15:

F
ORENSICS
:

C
OORDINATED
A
TTACKS

................................
................................
................................
.....................

47

T
ABLE
16:

F
ORENSICS
:

L
OCAL
P
RIVILEGE
E
SCALATION ATTEMPT

................................
................................
........................

47

T
ABLE
17:

F
ORENSICS OF
E
NCRYPTED
B
OTNET

................................
................................
................................
........................

48

T
ABLE
18:

F
ORENSIC OF A HACKERS

IRC

SESSION
................................
................................
................................
....................

48














Chapter 1: Introduction

1.1. INTRODUCTION


The revolution in Information Technology has provided a flood of assets in the form of
applications and services. Enterprises have based their entire business models on top
of these assets. Networks have evolved from low speed half duplex links to full dup
lex,
multi
-
homed, self convergent, gigabyte streams, controlled by advanced protocols. The
security of the available applications and services accessible over these networks
currently represents a major challenge to the IT industry. Each day, exploits, wor
ms,
viruses and buffer overflows severely threaten the IT infrastructure and associated
business assets along with mission critical systems. By learning the tactics and
techniques used by malicious
black hats,
crackers, we can secure our
d
ata assets and
in
frastructure. This demands learning from both system wide and network wide
resources.



6

Security is not an out of the box solution. It requires careful analysis of the environment
at hand before being able to propose a solution. It is a layered process and
demands a
great deal of thorough understanding of the system and its constraints. No system is
100 percent secure, the security of a system is as strong as its weakest point [28].
Security designs based on eggshell security models have proven to be most vu
lnerable.
"This can be viewed as an 'eggshell' security model: hard outer shell, soft in the center."
[29]. Therefore, security should be implemented in layers based on a defence in depth
model
[30]

rather than an eggshell model. This considerably increases the difficulty for
an attacker to penetrate through the system, as he might have gained access to part or
a component of the entire system. It will give the system administrators eno
ugh time to
address the problem by patching or configuring his resources. Each day we witness
hundreds of thousands of vulnerabilities coming out in our everyday use software.
These vulnerabilities when exploited cause compromise to systems. Crackers write

special customized software to target these vulnerabilities. These are called worms.
Worms spread like an epidemic over the internet capable to self propagate and infect
systems at very high rates. Soon they consume millions of systems, by taking over ful
l
control and awaiting further instructions. Many such worms install special client software
on their victims by virtue of which they chain them to their existing network of zombies.
Result is a highly distributed network of machines that on receiving a si
ngle instruction
from their owner may cause all sorts of havoc. Examples can be data and information
theft, including credit card, online bank accounts, email and other social networking
credentials. This information is a valuable asset in the underground
economy, where it
is sold for a good amount of money. Available security tools where provide a good set of
static defences, cannot cope with the dynamic nature of the threats. Most network
security tools are passive in nature; like, firewalls and Intrusion

Detection Systems
(IDS). They operate on available rules and signatures in their database. Anomaly
detection is thus limited only to
a

set of available rules. Any activity not in alignment with
such

rules goes unnoticed and undetected.
F
or analysing the t
ools that they use to
obtain this access

we need to set

up a vulnerable environment that poses as a valid
resource to any attacker, but is heavily logged. Honeypots
,

by design
,

allow you to take
the initiative by turning the tables on malicious black hats.

The Honeypot system has no
production value and has no authorized activity. Thus any interaction with the Honeypot
is most likely the result of malicious intent. Honeypots do not solve the security problem
but provide data and knowledge that aids the syst
em administrator in enhancing the
overall security of their network. This knowledge can be used as input to any early
warning systems. Over the years, researchers have successfully isolated and identified
worms and exploits using Honeypots placed in specia
lized architectures called
Honeynets. These are then used for signature and rule development. Honeynets are
capable of logging far more information than any other available security tools. They
give insight into attacks and attackers, their skill level, th
eir organization as groups or
individuals, their motives and tactics; and thus, almost every aspect is logged and can
be made auditable. This information will be analysed to develop a system for automated
attack classification and signature generation.


We

start
the proposal

by
defining
Computer and Network security
terminology as
background to the research work to be undertaken.
This is followed by a brief
description of attackers and attacks. In Chapter 2 we describe the security problem as
the problem st
atement for this research, along with a brief background of its evolution.
In Chapter 3 We describe the motivation for studying this domain and detail some of the
problems that are associated with it. In Section 3.5 we propose a solution to the

7

problem. In

Section 3.6 we present the technology required to underpin the research
and discuss current implementa
tions and standards. In Section

3.7 we identify the first
research challenge and discuss our
experiences with the technology;

here we find that
current implementation
s

lack some vital functionality which is solved by our proposed
technique. In Section 3.8 we identify the second research challenge and discuss
problems with current technology. In Chapter 4 we give an overview of e
xisting related
research activity
. In Chapter 5 We
provide
a summary of the key research questions

relevant to this proposal
. In Chapter 6 we propose solutions to address the problem. In
Chapter 7 we discuss results obtained so far by the technology
that w
e have
used.
Finally
,

we shall detail the progress we have made

and
the resources

that have been
developed so far. We also list
publications

and

talks
that have been delivered, together
with an indication of proposed
future direction
s

with
their associated

milestones.

1.2. DEFINITIONS

1.2.1. Information Security

“Information security deals with those administrative policies and procedures for
identifying, controlling, and protecting information from unauthorized manipulation. This
protection encompasses how

information is processed, distributed, stored, and
destroyed” [31]

1.2.2. Computer Security

“A computer is secure if you can depend on it and its software to behave as you
expect.” [32]

Computer security is essential

[33]
:

:




To prevent theft of or
damage to the hardware



To prevent theft of or damage to the information



To prevent disruption of service



1.2.3. Network Security

“Network security refers to all hardware and software functions, characteristics,
features, operational procedures, accountab
ility measures, access controls, and
administrative and management policy required to provide an acceptable level of
protection for hardware, software, and information in a network”. [31]


Network security is the art of securing preventing and protecting n
etwork resources and
assets such as routers, servers, hosts and any device connected with the organizations
network from unauthorized and unwanted access that may cause threats,
vulnerabilities, and denial of service, modification, destruction or disclosur
e of
information against these network assets.



8

Network security is a term that resides under information security and demands
securing all information assets connected to a network as well as securing all
information passing through the network.


1.2.4.
Security Standards and Documents


The ITU
-
T Security Architecture for Open System Interconnection (OSI) document
X.800 and RFC 2828 are the standard documentation defining security services. X.800
divide the security services into 5 categories and 14 speci
fic services which can be
summarized as:



1. AUTHENTICATION:

The assurance that the communicating entity is the one that it claims to be.

It includes:

Peer Entity Authentication

Data Origin Authentication


2. ACCESS CONTROL:

The prevention of unauthorized

use of a resource (i.e., this service controls who can have
access to a resource, under what conditions access can occur, and what those accessing the
resource are allowed to do).


3. DATA CONFIDENTIALITY:

The protection of data from unauthorized disclosu
re.

It includes:

Connection Confidentiality

Connectionless Confidentiality

Selective
-
Field Confidentiality

Traffic Flow Confidentiality


4. DATA INTEGRITY:

The assurance that data received are exactly as sent by an authorized entity (i.e., contain no
modif
ication, insertion, deletion, or replay).

It includes:

Connection Integrity with Recovery

Connection Integrity without Recovery

Selective
-
Field Connection Integrity

Connectionless Integrity

Selective
-
Field Connectionless Integrity


5. NONREPUDIATION:

Provi
des protection against denial by one of the entities involved in a communication of having
participated in all or part of the communication.

It includes:

Nonrepudiation, Origin:

Nonrepudiation, Destination:



9


[8], [9], [1]


1.3. Black Hats


Black hats are

highly skilled hackers or computer professionals who use their skill and
knowledge to gain illegitimate access to computer and information systems. They are
often socially, economically, financially or politically (
hactivist
) motivated in their cause.
Oft
en they are driven by their zeal and curiosity to learn about computer systems and
their secrets. Their goal is to exploit flaws or vulnerabilities in systems and use them for
their gain. These can be exploiting computer systems or humans


social engineer
ing
.
Black hats use technology for identity theft, vandalism, credit card fraud, phishing,
intellectual property theft (piracy) and many other types of sophisticated crimes. In
general terms this can lead to illegal control of remote computing resources vi
a a
network, having illegal access to software by
cracking
, collect victims information using
spyware
, scan their victims for exploits or
enumeration

using various
scanners
,
writing software that self
-
replicates and exploits all network accessible systems such as
worms

and
viruses
, infecting their victims with
backdoors, rootkits

and
trojans

for
remote access, creating an army of such remotely controlled zombie systems usual
ly
over irc


botnets,

and finally launching
Denial of Service(DOS)

and
Distributed
Denial of Service (DDOS)

attacks to knock their targets offline or cease their service
temporarily. These attackers can be 13 year old novice users playing around with
powe
rful hacking tools


scriptkiddies
. Or very sophisticated and elite system and
network administrators


1337
(a term used by the more sophisticated or elite hackers).


Black hat hackers are the biggest threat both internal and external to the IT
infrastruc
ture of any organization, as they are consistently challenging the security of
applications and services. Black hats are called “blackhats” in correspondence to colour
of their hat representing their intent as shown in many western movies and throughout
me
dia representing outlaws and bad guys; however
,

some computer geeks find the
black colour more appealing.


1.4. White Hats


White hats are ethically opposed to the blackhats. White hat hackers utilize their skill
and knowledge in securing, protecting and
preventing attackers from accessing
information and computer systems illegally. They study all the blackhat threats and
devise mechanisms for identification, protection and prevention in the form of security
policies and tools. They are constantly checking

and correcting systems for
vulnerabilities and exploits and have devised mechanisms for quick update and
distribution of their research and knowledge amongst the community to secure systems.


White hat hackers are considered as the white knights or the go
od guys and protectors.
They are the defenders of the cyber frontier that is always under attack by the black
hats. Attacking or defending, hackers have played a major role in evolving today's
technology and services. No system is 100% secure, thus a princ
ipal requirement is to

10

strengthen the mechanisms used to study the black hats and defend our information
assets.


1.5. Attacks and Attack Classification


Generally attacks are categorized under 2 major categories:

1.

Active Attacks

2.

Passive Attacks


1.5.1.
Active Attacks
:


Active attacks involve the attacker taking the offensive and directing malicious packets
towards its victims in order to gain illegitimate access of the target machine such as by
performing exhaustive user password combinations as in
brute
-
force attacks.
Or by
exploiting
remote and local vulnerabilities
in services and applications that are
termed as 'holes'. Other types of attacks include:

Masquerading attack
when attacker masquerades or pretends to be a different entity,

Replay attack

in
which attacker captures data and retransmits it to produce an
unauthorized effect.
Modification attack
in which a message or file is modified by the
attacker to achieve his malicious goals. and finally when the attackers try knock a
machine or resource off
line to disrupt or delay a service it is termed as a
denial of
service (DOS) attack
. TCP and ICMP
scanning

is also a form of active attacks in
which the attackers exploit the way protocols are designed to respond. E.g. ping of
death, syn attacks etc.


In a
ll types of active attacks the attacker creates noise over the network and transmits
packets making it possible to detect and trace the attacker. Depending on the skill level,
it has been observed that the skill full attackers usually attack their victims
from proxy
destinations that they have victimized earlier.


1.5.2. Passive Attacks

Passive attacks involve the attacker being able to intercept, collect and monitor any
transmission sent by their victims.
In the process,

they can
eavesdrop on their victim
and

they are

able to listen in to their victim

s or target

s communications. Passive
attacks are very specialized types of attacks which are aimed at obtaining information
that is being transmitted over secure and insecure channels. Since the attacker does

not create any noise
,

or minimal noise
,

on the network
,

it is very difficult to detect and
identify them.


Passive attacks can be divided into 2 main types, the
release of message content

and
traffic analysis.

Release of message content

involves protectin
g message content from getting in
hands of unauthorized users during transmission. This can be as basic as a message
delivered via a telephone conversation, instant messenger chat, email or a file.


11

Traffic analysis

involves techniques used by attackers to
retrieve the actual message
from encrypted intercepted messages of their victims. Encryption provides a means to
mask the contents of a message using mathematical formulas and thus make them
unreadable. The original message can only be retrieved by a rever
se process called
decryption. This cryptographic system is often based on a key or a password as input
from the user. With traffic analysis the attacker can passively observe patterns, trends,
frequencies and lengths of messages to guess the key or retriev
e the original message
by various cryptanalysis systems


1.6. Taxonomy of attacks


Attack classification has always been an interesting area for security researchers. As a
first step, Computer Incident Response Teams (CIRT), are required to classify the
at
tacks at hand in their reports. This classification should be complete enough to give
an in
-
depth view of the attack, the attacker, the target and the vulnerability exploited.
Based on this classification
,

a mitigation plan is proposed. Many classification

techniques have been proposed and adopted and later replaced by better techniques
over the years. Based on taxonomical work conducted by Hansman et.al on
characterization and dimensioning of computer and network attacks we can classify
attacks as [37]:




V
irus: self
-
replicating program that propagates through some form of infected
files



Worms: self
-
replicating program that propagates through network services on
computers or through email.



Trojans: a program made to appear benign that serves some malicious purpose



Buffer overflows: a process that gains control or crashes another process by
overflowing the other process’s buffer



Denial of service attacks: an attack which prevents legitimate
users from
accessing or using a host or network



Network attacks: attacks focused on attacking a network or the users on the
network by manipulating network protocols, ranging from the data
-
link layer to
the application layer



Physical attacks: attacks bas
ed on damaging physical components of a network
or computer



Password attacks: attacks aimed at gaining a password



Information gathering attacks: attacks in which no physical or digital damage is
carried out and no subversion occurs, but in which importan
t information is
gained by the attacker, possibly to be used in a further attack



12

Chapter 2: Problem Statement

2. SECURITY PROBLEM

2.1. Current Scenario


Cyber crime has taken off from being a vague report of a victim’s Yahoo or Hotmail
account being hacke
d, or a student changing his grades in the school database, to an
entire underground industry with its own underground economy. Data being the raw
material for this industry is continually being ripped out and harvested from globally
distributed computers
at an industrial scale. Malicious hackers and crackers act as the
workforce and enablers of this industry. The industry makes revenue by selling their
finished products (Credit Card details, authentication credentials, malware etc) and
services (customized

malware and support to use it) to the general public. All is
available in this market, sophisticated customized malware scripts, to a network of a
hundred thousand node botnet for hire.


Entire industry of cyber criminals is creating non
-
stop sophisticate
d malware causing
data breaches from network connected computers all across the globe. Cyber crimes
are easy to commit due to lack of policies or their implementation within a state or
across borders. The Internet crime industry is getting highly lucrative
. [38]


Malware getting more targeted, harder to detect, harder to remove. Security arena has
witnessed a huge change in the threat landscape, by emergence of mobile devices and
virtualization. Threats now are getting mobile and pervasive over the cloud.
VM sprawl
big security concern as VM’s sprouting out like mushrooms from the ground, often miss
configured to the dismay of the security engineer. All these events contributing to
weakened grip on security as the “Protect, Detect, and React paradigm” being

harder to
implement
.
[30]


Solera networks have suggested a threat classification in their whitepaper [84]
.

T
hese
network threats are classified into four categories:

1.

Threats coming
in

2.

Threats invited in

3.

Threats already in

4.

Threats going out.


Network perimeter attacks, such as XSS and SQL injections exploit vulnerabilities in
pubic web portals to steal sensitive proprietary data from backend databases including
sensitive user
information. Such attacks incorporate for incoming threats. [84]


Social and technological attacks, from emails phishing for information or inviting users
to be victimized by drive
-
by downloads, and online social interactions which innocently
request perso
nal or confidential information are credited [84]

for the threats invited in.


Threats already inside the network claim to be the most dangerous threats. This can be
due to compromised systems or
a
renegade employee. If left unattended
,

the damage

13

potenti
al of these threats can quickly escalate. Being inside

the network,

an

attacker
can do nearly anything th
at they desire
. [84]


The threat leading to the exodus of sensitive data from a business enterprise is critical.
This can result in jeopardizing confid
ential trade secrets, customer information like
social security numbers or credit cards, or classified national security information like
security plans for the head of state or parliamentarians’. The attackers may also turn the
business network to meet th
eir personal underground business needs like active spam
-
bots that can push out bulk emails. [84]


Many organizations implement their security limited only to perimeter security. An
enterprise faces constant threat from “things coming in” due to perimeter
penetration. It
is also termed as “walking through the front door”. This can be due to technical
vulnerabilities such as SQL injection, browser, flash, media player or can be a social
vulnerability. Once an attacker has managed to bring down that wall then

very bad
things can happen. Since traditional perimeter defences are facing the outside network
they are blind to the inside. This can lead to emergence of flows comprising of traffic
from bots, spambots, content distribution nodes and other sensitive dat
a leakage from
inside the network to the outside world. A survey conducted by the FBI and Computer
Security Institute, reported that over 70 percent of the loss of confidential information
comes from inside the organization. The security model must be laye
red, where internal
assets are secured, partitioned, and monitored. [39] Need for a defence in depth
strategy is ever so felt now.


In the realm of security, response time is critical and saves money. There are many
threats that an organization is prone to
, with a very small subset of them marked as
known threats. The only way to respond to breaches quickly and effectively is by doing
root cause analysis.


Surveillance is vital to security. We all expect a breach but our existing tools don’t help
us when it

happens. It’s synonymous to the situation in
the
real world where we have
security cameras everywhere. They monitor everything but don’t respond. It’s the
vigilance of the surveillance expert to identify event of interest and report it. There is a
d
ire ne
ed to look out for events of interest.


Cohen et al [40]

established the security problem as:

“Our society is so reliant on information that the loss or corru
ption of the United States’

i
nformation infrastructure would create a situation where the national

banking system,
electric power grid, transportation systems, food and water supplies, communication
systems, medical systems, emergency services, and most businesses [could not]
survive.”


“Organizations that value their internal informati
on realize that

information is
a strategic
and competitive tool [41].

2.2. Cost

As attacks are ongoing they tend to get more and more expensive. Direct costs include:


14

d
owntime, IT resources, stolen data or IP. Indirect costs include follow on incidents,
impact to brand r
emediation for maximum scope. The faster we may be able to find the
source and scope of the breach, the less expensive it will be for us.


Figure
1
: Attack Consequences vs Likelihood [84]


2.3. What People Say About Security



Symantec predicts: In 2010, 'antivirus is not enough'

December 10th, 2009

"...the industry is quickly realizing that traditional approaches to antivirus, both
file signatures and heuristic/behavioural capabilities, are not enough to protect
against
today's threats."



Network Solutions Warns Merchants After Hack

600,000 credit card numbers stolen from Ecommerce Hosting merchants

Robert McMillan



July 7, 2009


NY Times Website Infected With Fake Antivirus

September 15th, 2009

"It's a fake page
for a nonexistent antivirus app, which is actually malware...It's a
multimillion dollar business"



Annual Threat Assessment of the US Intelligence Community for the Senate
Select Committee on Intelligence

February 2nd, 2010

"Sensitive information is st
olen daily from both government and private sector
networks... We often find persistent, unauthorized, and at times, unattributable
presences on exploited networks... We cannot be certain that our cyberspace
infrastructure will remain available..."


Hackers are defeating tough authentication, Gartner warns

January 18th, 2010

"Cybercriminals are using increasingly sophisticated tactics to outmaneuver
security systems so they can steal customers' log
-
in credentials and pillage their

15

bank accounts, acco
rding to a Gartner analyst"


Google Hack Attack Was Ultra Sophisticated

January 14th, 2010

"Hackers...used unprecedented tactics that combined encryption, stealth
programming and an unknown hole in Internet Explorer"


More Victims Of Chinese Hacking A
ttacks Come Forward

January 14th, 2010

"This attack involved very advanced methods, with several pieces of malware
working in concert to give the attackers full control of the infected system, at the
same time it attempts to disguise itself as a common
connection to a secure
website"


U.S. Army Website Hacked

January 12th, 2010

"Every organization has these problems...They may not realize it, but they're just
waiting for a smart kid to come along and copy off every critical piece of
information they h
ave"

Table
1
: What People say about security?

2.4. Needs

Lessons learned from history point towards a need of re
-
evaluation of current
techniques. These can be summarized as the needs:




Need to stop and remediate events quickly.



Need to do more and find root cause of breaches.



Need better Forensic analysis and tools



Need for techniques to gleam information from the data.



Need for consistent policies across borders



Need for stronger passwords or keys.



Need to secure application lev
el vulnerabilities.



Need for automation in security industry.



Need for more dynamic security technology



Need to get information out of systems intelligently, logging in depth, better log
management, better log analysis and better management roles



Need to k
now what to protect and how to protect



Need to know the threat you face, know your enemy



Requirement to be vigilant and responsive.


Chapter 3: Motivation and Research Challenges

3.1. Motivation

Security is a collective effort and demands thorough planning
. Unfortunately in the past
it has always been overlooked and never considered a real problem. The need for
securing data and information assets really got felt publically after the commercialization
of the internet in the late 1980’s. Paula [41]

has corre
lated this with emergence of the

16

first virus in 1988 “Therefore, in the fall of 1988 the world saw evidence of the true
threats that existed to network security. The Internet Virus was launched at that time
and all of the 60,000 computers on the Internet w
ere crippled for two entire days”


Historical study [41]

reveals that the first ever published document on security became

Trusted Computer Security Evaluation Criteria
” which was a host hardening manual
ignoring the network security aspects.



There
were no real threats felt as the early internet was shared between very few
organizations mostly to conduct, collaborate and share research. As Paula [41]

states
:

“Before this, more emphasis was laid down on running, maintaining and expanding the
Arpanet.
“People who used the ARPAnet were scholars and government employees
who were at the time more concerned with discovery than with destruction”



Over the years we have observed a sharp increase in the intricacy, sophistication and
overall frequency of atta
cks. Availability of user friendly hack tools has claimed a great
share of these attacks which do not demand a great deal of understanding from their
users. Lipson (2002) has studied and correlated this trend graphically as:



Figure
2
: Intruder knowledge vs sophistication of attack [42]


3.2. Objects that demand security




Operating Systems not designed with much security in mind. (win9x, winnt, xp,
linux)



Applications not designed with security in mind. (office applications, web
b
rowsers)



Services not designed with security in mind.(ftp, telnet, http, r
-
services)



Miss configured folder permissions, let ordinary system users access sensitive
system files.


17



Miss
-
configured networks, exposing disk shares and other information resources

to the outside world with full permissions.


3.3. Who
is
to Blame?



Why everything is considered secure by default till exploited?



Blame the Coders?



Blame the architects/designers?



What about users which keep weak or easily guessable passwords? Blame the
human?


Time has proven that security is a collective effort. We can only blame ourselves for not
thinking about security while coding, designing, testing or implementing software and
hardware. It wasn’t till organizations were ripped off of data till they

realized the
magnitude of the problem and started work to devise a solution for it.


3.4. A Few Documented Attacks


Since 1999 there has been a tremendous increase in the number of incidents reported
as statistics from the Computer Emergency Response Team

Coordination Center
(CERT/CC) (CERT, 2003)



Figure
3
: Incidents reported till 2003 [37, 43]


A f
ew
n
otable incidents are documented here:




FBI statistics state that up to five billion dollars is lost each year due to
information

theft through computer crimes
>



285 million records were compromised in 2008
.




In 2009, 10 million USD were stolen worldwide using ATM cards in less than 24
hours. These thefts were conducted by a well
-
organized band of bank robbers.
[38]



“US
-
CERT is aware

of public reports indicating a widespread infection of the
Conficker/Downa
n
dup worm, which can infect a Microsoft Windows system from
a thumb drive, a network share, or directly across a corporate network, if the

18

network servers are not patched with the M
S08
-
067 patch from Microsoft.

Researchers have discovered a new variant of the Conficker Worm on April 9,
2009.” [49]



Increase in web based and application hacks as per Verizon report. [50]



Verizon data breach report of 2009 reveals that behind data
breaches : [50]



74% resulted from external sources, 20% were caused by insiders, 32%
implicated business partners, and 39% involved multiple parties (+ 9%). [50]



The scales for breaches were:



67% were aided by significant errors, 64% resulted from hacking,

38% utilized
malware,



22% involved privilege misuse (+7%), 9% occurred via physical attacks. [50]



85% organizations had a major network incident in the past 3 years or expect a
major incident in next 3 years. [50]



Figure
4
:
Threat categories over time by percent of breaches [50]


3.5. Moving Towards a Solution

Security tools themselves cannot save us from the onslaught of the malicious black hat
crackers. These tools require intelligent use and configuration before being effe
ctive
enough. Stephen Northcutt and Judy Novak have established this in their book as,


“Intrusion detection is not a specific tool but a capability, a blending of tools and
techniques” [51]


Flawed assumptions made by security tools lead to fake sense o
f security. E.g. what
use is antivirus software if it is not updated frequently? What use is a firewall if the user
does not know how to configure it and relies on default policies every time? Same goes
for IDS and IPS. Networks are becoming more scalable
and rapidly evolving. It’s a world
of dynamic services and dynamic networks, attracting dynamic threats.



19

Available static defences like AV systems, Firewalls and IDS are not sufficient enough.
They involve too much manual input from humans. They require h
ours of analysis till
new rules, signatures can be produced, meanwhile the threat is running out in the wild
infecting and claiming more and more resources. Most network security tools are
passive in nature; like, firewalls and Intrusion Detection Systems
(IDS). They operate on
available rules and signatures in their database. Anomaly detection is thus limited only
to these set of available rules. Any activity not in alignment with those rules goes
undetected. Research remains the most effective way to unde
rstand vulnerabilities, how
they are identified and how they are exploited
.

Hacker tools used to exploit these
vulnerabilities and the tactics involved. By learning the tactics and techniques used by
the malicious black hats we can secure our IT assets and

infrastructure. Honeypots
provide a means to study black
-
hat techniques and tactics by which they gain
illegitimate access to system resources along with methods to analyse the tools they
use. This is achieved by setting up a vulnerable environment that p
oses as a valid
resource to any attacker, but is heavily logged.


The most ideal solution to meet the security challenges of today is a comprehensive
vulnerability management program that detects all sorts of intrusions, threats and
exploits, analyses them
, correlates the events that occurred and generates automated
proactive responses to the newly identified weaknesses. This thesis will aim to achieve
some or part of this idea. Our research will focus on Intrusion Detection and creation of
an automated sig
nature engineering system, as an active response for mitigation.


We have divided the research into 2 main phases:


1.

Deployment of Honeypot sensors in Honeynets to collect real
-
time data on
intrusions and attacks.

2.

Automated analysis of attack data to
identify, classify and cluster attacks to serve
as input for signature generation.


3.6. Honeypots and Honeynets

3.6.1. Who. What. Where, why and how?


The f
irst step towards achieving my research goals involved setting up Honeypot sensor
nodes. These sens
ors will aid us in understanding who the attackers are
.

What
methods

and tools
do
they use to attack? Where do they get the knowledge and tools from? Why
do they attack us? How do they organize and gain access to so many victim machines
simultaneously?

3.
6.2. Honeypots


A Honeypot i s generally defi ned as a network securi ty resource whose value li es i n i t
bei ng scanned, attacked, compromi sed, controlled and mi sused by an attacker to
achi eve hi s mali ci ous goals.


20


Lance Spitzn
er
[1] defines Honeypots as “A Ho
neypot is an information system
resource whose value lies in unauthorized or illicit use of that resource”


3.6.2.1. Motivation and Concept


Mostly network security tools are passive in nature for example Firewalls and IDS. They
operate on available rules
and signatures in their database. That is why anomaly
detection is limited only to the set of available rules. Any activity not in alignment with
those rules goes under the radar and is thus undetected. Honeypots by design allow
you to take the initiative;

they turn the tables on the bad guys. This system has no
production value, with no authorized activity. Any interaction with the Honeypot is most
likely malicious in intent. Honeypots do not solve the security problem but provide data
and knowledge that a
ids the system administrator to enhance the overall security of his
network. This knowledge can be used as input for any early warning systems. Over the
years researchers have successfully isolated and identified worms and exploits using
Honeypots. These a
re then used for signature and rule development.


Honeypots are capable
of

log
ging

far more information than any other available security
tools. They give us an insight
into

attacks and attackers, their skill level, their
organization as groups or individ
uals,
and
their motives and tactics
.

T
hus
,

almost every
aspect is logged and can be made auditable. Honeypots effectively empower us to
study malicious hackers under a microscope. This can be
demonstrated

with
a
few
examples:

3.6.2.2. Classic Examples

:j@c
k :hehe come with yure ip i`ll add u to the new 40 bots

:j@ck :
i owned and trojaned 40 servers of linux in 3 hours

:j@ck ::)))))


:j1ll :heh

:j1ll :damn

:j@ck :heh

:j1ll :107 bots now

:j@ck:yup

[1]

Table
2
: Honeypot: Classic
Examples

3.6.2.3. Discussing Exploits

:_pen :do u have the syntax

:_pen :for

:D1ck :yeah

:_pen :sadmind exploit

:_pen :?

:D1ck :lol

:D1ck :yes

:_pen :what is it

:D1ck :.
/sparc
-
h hostname
-
c command
-
s sp [
-
o offset] [
-
a alignment] [
-
p]

:_pen : what do i
do for
-
c

:D1ck :heh

:D1ck :u dont know?

:_pen :no

:D1ck :"echo 'ingreslock stream tcp nowait root /bin/sh


21


sh
-
i' >> /tmp/bob ; /usr/sbin/inetd
-
s /tmp/bob“

[1]

Table
3
: Honeypot: Discussing Exploits

3.6.2.4. Example: Leaves

Worm



On June 19, 2001 a sudden rise of scans for the
Sub7

Trojan was detected. (port
27374)




An Infected emulated Windows Honeypot revealed a worm was pretending to be a
Sub7 client and attempting to infect systems.



Matt Fearnow and the Incidents.org tea
m identified it as the W32/Leaves worm

National Infrastructure Protection Center (NIPC) was informed. CERT advisory July 3,
2001[1]

Table
4
: Honeypot: Leaves Worm

3.6.2.5. Example: Code Red II Worm



Ryan Russel at SecurityFocus.com
for analysis of the CodeRed II worm (MS IIS
indexing exploit)




A typical signature of the Code Red II worm would appear in a web server log as:


GET /default.ida?XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX



XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801


%u9090%u6858%ucbd3%u7801%u9090%u9090%u8190%
u00c3


%u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a HTTP/1.0

This worm tried to infect other computers at random, along with machines on the same
subnet as the infected machine.[1]

Table
5
: Honeypot: Code Red II Worm

3.6.2.6.
Example: Solaris DTSCD exploit



A Solaris Honeypot captured a dtspcd exploit, an attack never seen before.



On November 12, 2001, the CERT Coordination Center had released an advisory for
the CDE Subprocess Control Service or, more specifically, dtspcd



Explo
it code was isolated and attack was detected. This was the first incident a
Honeypot was used to identify and document an unknown attack.

[1]

Table
6
: Honeypot: Solaris DTSCD exploit

3.6.3. Honeynets

A Honeynet is a special kind of high
-
interaction Honeypot.
Honeynets extend the concept of a single Honeypot to a highly
controlled network of Honeypots. A Honeynet is a specialized
network architecture configured in a way to achieve
Data


22

Control, Data Cap
ture and Data Collection
. This architecture creates a highly
controlled network, in which one can control and monitor all kind
s

of system and
network activity. Honeypots are then placed within this network. A basic Honeynet
comprises of Honeypots placed behind a transparent gateway


the Honeywall. Acting
as a transparent gateway the Honeywall is undetectable by attackers and serv
es its
purpose
by

logging all network activity going in or out of the Honeypots
.

3.6.3.1. Data Control


Data control is the containment of activity within the Honeynet. It determines the means
through which the attacker's activity can be restricted in a wa
y to avoid
damaging/abusing other systems/resources through the Honeynet. This demands a
great deal of planning as we require to give the attacker freedom in order to learn from
his moves and at the same time not let our resources (Honeypot

+

bandwidth) to

be
used to attack, damage and abuse other hosts on the same or different subnets. Careful
measures are taken by the administrators of the Honeynet to study and formulate a
policy on the attacker’s freedom versus containment and implement this in a way to
achieve maximum data control and yet not be discovered or identified by the attacker as
a Honeypot. Various mechanisms to achieve data control are available such as firewall,
counting outbound connections, intrusion detection systems, intrusion prevention
systems and bandwidth restriction etc. Depending on our requirements and risk
thresholds defined, we implement data control mechanisms accordingly.

3.6.3.2. Data Capture


Data Capture involves the capturing, monitoring and logging of
all threats and attac
ker activities within the Honeynet. Analysis of
this captured data provides an insight on the tools, tactics,
techniques and motives of the attackers. The concept is to
achieve maximum logging capability at all nodes and hence log
any kind of attacker's in
teraction without the attacker knowing it.
This type of stealthy logging is achieved by setting up tools and mechanisms on the
Honeypots to log all system activity and have network logging capability at the
Honeywall. Every bit of information is crucial in

studying the attacker whether it’s a TCP
port scan, remote and local exploit attempt, brute force attack, attack tool download by
the hacker, various local commands run, any type of communication carried out over
encrypted and unencrypted channels (mostly

IRC) and any outbound connection
attempt made by the attacker. All of this should be logged successfully and sent over to
a remote location to avoid any loss of data due to risk of system damage caused by
attackers, such as data wipe out on disk etc. In o
rder to avoid detection of this kind of
activity from the attacker, data masking techniques such as encryption should be used.


3.6.3.3. Data Collection



23

Once data is captured, it is securely forwarded to a centralized data collection point.
This allows da
ta captured from numerous Honeynet sensors to be centrally collected for
analysis and archiving. Implementations may vary depending on the requirements of the
organization, however latest implementations incorporate data collection at the
Honeywall gateway
.

3.6.3.4. Honeynet Architectures


There are 3 Honeynet architectures namely:



Generation I



Generation II



Generation III

3.6.3.4.1. Generation I Architecture

Gen I Honeynet
s

were developed in 1999 by the Honeynet Project. Its purpose was to
capture attacker
’s activity and give them feel of a real network. The architecture is
simple with a firewall aided by an IDS
placed
at the front and Honeypots placed behind
it.
Unfortunately, t
his makes it detectable by attackers.


Figure
5
: Gen
I Honeynet Architecture [12]

3.6.3.4.2. Generation II and III Architecture:


Gen II Honeynets were first introduced in 2001 and Gen III Honeynets was released in
the end of 2004. Gen II Honeynets were made in order to address the issues of Gen I
Honeynets.

Gen II and Gen III Honeynets have the same architecture. The only
difference being
, that

there have been significant
improvements in
the
deployment and
management
of

Gen III Honeynets along with the addition of
a
Sebek server built

in
to

the Honeywall.

A
radical change in architecture was brought about by the introduction of a single device
that handles the data control and data capture mechanisms of the Honeynet called the
IDS Gateway or

to use the

marketing

-

terminology:

T
he Honeywall. By making the

24

arc
hitecture more “stealthy”, attackers are kept longer and thus more data is captured.
There was also a major thrust in improving Honeypot layer of data capture with the
introduction of a new UNIX and windows based data


Figure
6
:
Generation III Honeynet Architecture [12]

3.6.3.5. Virtual Honeynet

Virtualization is a technology that allows running multiple
virtual machines on a single physical machine. Each virtual
machine can be an independent Operating system
installation. This
is achieved by sharing the physical
machines resources such as CPU, Memory, Storage and
peripherals through specialized software across multiple
environments. Thus multiple virtual Operating systems can
run concurrently on a single physical machine.

A virt
ual Honeynet is a solution that facilitates to run a
Honeynet on a single computer. We use the term virtual
because all the different operating systems placed in the
Honeynet have the 'appearance' to be running on their
own, independent computer.

3.7. Rese
arch Challenge # 1

3.7.1. Architecture and Design Considerations in Virtual Honeynets

3.7.2. Introduction

The Honeynet project provides documentation on deploying Generation 3 virtual
Honeynets, this documentation was developed by the Pakistan Honeynet Pro
ject
Chapter. This document was a step
-
by
-

step How
-
To for deploying virtual Honeynets
using VMware. This served as a standard template for anyone who wants to deploy a

25

virtual Honeynet using VMware and Honeywall Roo and has thus become a de facto
document
:

http://www.Honeynet.pk/Honeywall/roo/page2b.htm.

During literature review it was decided to use this document as the standard template
for our project's implementation. Generation 3 ar
chitecture demands 3 interfaces on the
Honeywall, in which one is used as management interface while other two are used as
bridged interfaces. Using VMware, a bridged interface like vmnet0 has direct access to
the physical interface and thus 2 such interfa
ces will cause the bridging between the
same LAN segments, whereas
a
requirement was to bridge between two LAN
segments i.e. the external network segment pointing to the router and the internal
network segment on which the Honeypots will be placed. It was
observed that the
Honeynet design suggested by the website had configured both eth0 and eth1
interfaces as a VMware bridge interface and eth2 as a VMware host
-

only interface.
This was causing
a
loop in the Honeywall and the Honeypot LAN segment was being
avoided. This problem was extended to the Pakistan Honeynet Project, who
then
accepted and updated the design on their website.


3.8. Research Challenge # 2

3.8.1. Intrusion Detection


Intrusion detection is the art of detecti
ng

malicious activity in a co
mputer related system
[76].
M
alicious activities and
intrusion techniques

are interesting from a computer
security perspective. Analysis of traffic and events reveals that intrusion is different from
the normal behaviour of system usage, and hence anomaly
detection techniques are
applicable in
the
intrusion detection domain. Denning [74] classifie
d

intrusion detection
systems into
1)
host based and
2)
network based intrusion detection systems. K.
Scarfone et al. [80] classif
ied

Intrusion detection systems b
y their detection
methodology (signature matching, anomaly detection or stateful protocol analysis) and
location (on a host, a wired network, or a wireless network), or capability (simple
detection or active attack prevention) [80]


3.8.2. Intrusion
Detection Problem


Conventional intrusion detection and prevention system solutions defend a network's
perimeter by using packet inspection, signature detection and real
-
time blocking.
Although these techniques are effective as a static defence, they fail
to cope with the
dynamic nature of threats faced today.


Signature matching techniques are used to identify attacks by comparing the contents
of packets with a set of signatures or rules that describe the known attack. These
techniques can become unreliabl
e against ciphered traffic and self modifying malware
or other evasion techniques. [81]


26


Stateful protocol analysis techniques involve matching of each connection with an
existing template that acts as a profile for a given protocol. Any deviations from th
is
profile are immediately reported. The effectiveness of this technique can be seen in
areas such as horizontal network scanning or host behaviour profiling. On the contrary
attacks conforming to normal protocol behaviour tend to go unnoticed. [81]


3.8.3
. Intrusion Detection Signatures

A signature is a pattern or characteristic used for identification

and i
t is used to
“describe the characteristic elements of an attack” [52]
.

Intrusion detection systems
identify attacks based on signature matches. These s
ignatures are created after
analysing attack traffic data. In the absence of signature writing standards, it has been
observed that signatures vary from implementation to implementation [52, 17].


A signature is considered effective based on its ability t
o narrow down the attack
characteristics and be elastic enough to detect any kind of variations in the attack [52,
17]. Examples of some well known signature
-
based intrusion detection systems include
Bro and Snort [17].


3.8.4. Automated Signature Engineer
ing


Signature generation is a laborious process. It may require hours of analysis until a final
effective signature can be produced. This analysis is based on some unique
characteristics visible within the traffic. Automating this process will be ideal in

saving an
enterprise from an imminent attack. A requirement is that a system should intelligently
perform traffic analysis to identify unique characteristics that can serve as a key in
generating signatures for intrusion detection systems.


Chapter 4: Ove
rview of Related Works

4.1. Honeypots as attack detection and learning tools

Honeypots began as an idea to study and isolate black hat hackers. The requirement to
learn and profile the enemy has always been an interesting area for security
researchers. The

concept has been around for some time in different forms and
implementations until it recently evolved into a well defined and documented solution.
This was followed by the development of various commercial products. It is, as yet, not
clear as to who cam
e up with the word “Honeypot” for such projects; however the core
concept remained the same. Many experts believe that the most primitive set of
documents available on the concept of Honeypots were Clifford Stoll's “The Cuckoo's
Egg” [2] and Bill Cheswick'
s "An Evening with Berferd in Which a Cracker Is Lured,
Endured, and Studied" [3]. In both papers the researchers had a chance to come face to
face with an attacker who gained access to their system and were then presented with

27

various types of data to stu
dy the attacker’s responses. This was essentially a proof of
concept that it was possible to learn from an attacker in such a way that the community
can benefit from it. This led to an effort to have better logging mechanisms and tools for
studying attacke
r tactics.

In 1999, Lance Spitzner the founder of The Honeynet Project [4] started work in the
area of Honeypots. In a very short span of time the Honeynet Project contributed a
series of publications focused on definition, development, architecture and or
ganization
of Honeypots. Researchers in the Honeynet Project have published their findings and
experiences with their Honeypots over a number of years. The most notable book in this
regard is “Honeypots, Tracking Hackers” [1]. This book gives
us a
deep ins
ight into
Honeypots and is the first compilation of Honeypot based books. This was followed by
“Know Your Enemy: Learning about Security Threats” published by the Honeynet
Project in 2004.

The era of virtualization had its impact on security and Honeypots.

The community
responded, marked by the fine efforts of Niels Provos (founder of honeyd) and Thorsten
Holz for their excellent book “Virtual Honeypots: From Botnet Tracking to Intrusion
Detection” in 2007 [6]. Papers on Virtual Honeynets were published by
the Honeynet
Project in early 2003, whilst the year 2004 marked the start of a new type of Honeypot
known as the client Honeypot. Kathy Wang's “honeyClient” became the first publically
available Client Honeypot tool. Generation III Honeynets also emerged i
n 2004
-
2005
and Honeywall CDROM version 2 “Roo [22]” became the first publicly available tool
based on Generation III technology. The road onwards has seen many improvements
and enhancements to the functional components of a Honeynet, especially with respe
ct
to the tools for data analysis. There has been a significant shift of focus from Honeynets
to client Honeypots and then towards virtual Honeynets. A significant amount of work is
being carried for client Honeypot based developments and to enhance the ca
pabilities
of existing Honeynet technologies.

Our system will incorporate existing Honeynet technology and will
be
set up in a virtual
environment
using

VMware ESX server. This will give us another dimension of valuable
data on the state of the Honeypot as

it is under attack.

4.2. Automated Signature Engineering using Honeypots


The existence of complex self
-
similar patterns in internet traffic was first revealed in
work done by Leland
et al.
[73 ]Multiple invariant substrings must often be present in all
variants of worm payload [54]. The substrings correspond to return addresses, protocol
framing, and poorly obfuscated code [53]. Generation of a short single substring
signature for all worm i
nstances can result in high false positive rates [54]. Systems
based on pattern
-
based analysis extract common byte patterns across suspicious flows,
to generate signature for novel internet worms. Examples of such systems include
EarlyBird [56], Honeycomb
[53], and Autograph [55]. A single signature is used to match
all worm instances based on unique substrings in the payload. These substrings are
considered invariant across worm connections [54]. Such systems may suffer from a
relatively high false positiv
e and high false negative rate [54]


Classification of signatures for polymorphic worms can be done under two main
categories [53, 54, 55, and 56]:


28


1.

Content
-
based:

Detect similarity in different instances of byte sequences to characterize a given
worm
.

2.

Behaviour based:

Characterization by perceiving the semantics of byte sequences.


We would like to incorporate both approaches in our research.


Honeypots provide us with insight information for intrusion

and attack analysis. Pouget
et
al [65] analyse
d traffic in Honeypots to identify root causes of frequent processes.
Observed traffic was organized based on the port sequence. This data was then
clustered using association rules mining [64]. “Phrase distance” was then implemented
on the result. Levin e
t al. explained the use of Honeypots to extract particulars of a
worm that can be analysed to generate signatures [57]. Honeycomb [52] was one of the
first implementations of an automated signature generator. It was implemented as a
Honeyd [58] plug
-
in. Ho
neycomb incorporated the longest common substring (LCS)
algorithm on connection pairs to determine common byte sequences. It generates
signatures consisting of a single, long substring of a worm’s payload. This inhibits its