PPT - Monkey.org

standguideNetworking and Communications

Oct 26, 2013 (3 years and 9 months ago)

81 views

Sniffing network traffic in Python

Jose Nazario, Ph.D. <jose@monkey.org>

Why Python?


Interpreted language


Bound to be slower than C



Rapid development


Easy data structure use


Fewer LoC per tool


Easy to manipulate strings


http://www.python.org/

Marrying Python and Sniffing


Librares in C


Often SWIGged, exported to Python


pcap, dnet, nids …


Modules


pypcap/pcappy


pcap for python


dpkt


packet deconstruction library


libdnet


packet construction library (has python
bindings in the distribution)



pynids


connection reassembly tool

libnids



reassemble IP streams

NIDS “E” box (event generation box)

Userland TCP/IP stack

Based on Linux 2.0.36 IP stack

Uses libpcap, libnet internally

IP fragment reassembly

Kernel

IP stack

Userland

Kernel

IP stack

Userland

IP stack

Libnids

libnids Basics


Initialize


nids_init()


Register callbacks


nids_register_tcp()


nids_regster_ip()


nids_regiser_udp()


Run!


nids_run()


React


nids_kill_tcp()

nids_run()

UDP callback

TCP callback

IP callback

TCP stream object:


-

TCP state


-

client data


-

server data


-

source IP, port


-

dest IP, port


-

seq, ack, etc …

UDP packet:


-

source IP, port


-

dest IP, port


-

UDP payload

IP packet


-

struct IP packet


-

contains upper


layers

libnids TCP states


NIDS_JUST_ESTABLISHED


New TCP connected state (3WHS)


Must set
stream
-
>{client,server}.collect=1

to get stream payload collected


NIDS_DATA


Data within a known, established TCP connection


NIDS_RESET, NIDS_CLOSE,
NIDS_TIMED_OUT


TCP connection is reset, closed gracefully, or was
lost

libnids doesn’t expose SYN_SENT, FIN_WAIT, etc …

pynids Basics


Event driven interface (nids_run(),
nids_next())


TCP stream reassembly


TCP state exposure


Creates a TCP object


Holds addresses, data, etc


UDP and IP packet reassembly

Basic pynids Steps


Initialize


nids_init()


Establish parameters


nids.param(“
attribute
”, value)


Register callbacks


nids.register_tcp(handleTcp)


def handleTcp(tcp): …


Go!


nids_run()


while 1: nids_next()

pynids Order of Operations


Packets come in



TCP?


State exist? Create state or reuse state


Append data


Process based on state in callback


UDP or IP?


Use handler, pass packet in


You process in callback

Code Example (Python)

import nids

<handleTcpStream>


def main():


nids.param("scan_num_hosts", 0)


if not nids.init():


print "error
-
", nids.errbuf()


sys.exit(1)


nids.register_tcp(handleTcpStream)


try: nids.run() # loop forever


except KeyboardInterrupt:


sys.exit(1)

Code Example (Python) cont

def handleTcpStream(tcp):


if tcp.nids_state == nids.NIDS_JUST_EST:


if dport in (80, 8000, 8080):


tcp.client.collect = 1


tcp.server.collect = 1


elif tcp.nids_state == nids.NIDS_DATA:


tcp.discard(0)


elif tcp.nids_state in end_states:


print "addr:", tcp.addr


# may be binary


print "To server:“, tcp.server.data


print "To client:“, tcp.client.data

Code Example (C)

int main(int argv, char *argv[])

{


if (nids_init() == 0)


err(1, “error, %s”, nids_errbuf);


nids_register_tcp(handleTcp);


nids_run();


exit(0);

}

Code Example (C), cont

int handleTcp(struct tcp_stream *tcp)

{


switch (tcp
-
>nids_state) {


case ‘NIDS_JUST_EST’:


if ((tcp
-
>addr.dest == 80) ||


(tcp
-
>addr.dest == 8000) ||


(tcp
-
>addr.dest == 8080) {


tcp.server.collect = 1;


tcp.client.collect = 1;


}


break;


case ‘NIDS_DATA’:


nids_discard(tcp, 0);


break;


case ‘NIDS_CLOSE’:


case ‘NIDS_RESET’:


case ‘NIDS_TIMED_OUT’:


printf(“((%s, %d), (%s, %d))
\
n”, inet_ntoa(tcp
-
>saddr), tcp.srce,


inet_ntoa(tcp
-
>daddr), tcp.dest);


printf(“%s
\
n”, tcp
-
>server.data);


printf(“%s
\
n”, tcp
-
>client.data);


break;


}

}

About the same LoC, until we start string manipulation

VersionDetect


Small python tool


Reports on headers


Fully passive


Support for: SSH (client, server), WWW
(client, server), and SMTP clients


Motivation: coordinate data collection with
TCP stack fingerprinting

63.236.16.161 SymbianOS 6048 (on Nokia 7650?) www 80/tcp


63.236.16.161: 80: Microsoft
-
IIS/6.0

VersionDetect Output


192.168.1.7: 22: SSH
-
2.0
-
OpenSSH_3.5


192.168.1.101:http: Mozilla/5.0 (X11; U; OpenBSD i386; en
-



US; rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1


168.75.65.85: 80: Microsoft
-
IIS/5.0


165.1.76.60: 80: Netscape
-
Enterprise/3.6 SP2


168.75.65.69: 80: Microsoft
-
IIS/5.0


168.75.65.87: 80: Microsoft
-
IIS/5.0


69.28.159.7: 80: ZEDO 3G


198.65.148.234: 80: Apache/1.3.29 (Unix) PHP/4.3.3


216.150.209.231: 80: Apache/1.3.31 (Unix)


212.187.153.30: 80: Apache/1.3.31 (Unix)


212.187.153.37: 80: Apache/1.3.31 (Unix)


212.187.153.32: 80: thttpd/2.25b 29dec2003


64.209.232.207: 80: Apache/1.3.27 (Unix)


mod_perl/1.27


216.239.39.99: 80: CAFE/1.0

http
-
graph


Small, passive python tool


Examines HTTP request header:


GET /blog/styles
-
site.css HTTP/1.1

Host: www.jackcheng.com

User
-
Agent: Mozilla/5.0 (X11; U; OpenBSD i386; en
-
US;
rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1Accept:
text/css,*/*;q=0.1

Referer:
http://www.jackcheng.com/blog/archives/2004/12/ipod_rumo
rs.html

http
-
graph


Directed graph history of browsing


Reconstructs graph from referrer and URL
in the header:



Referrer Request



Lets you view your history as you took it


Shows natural “hubs” of information


See also:
http://www.uiweb.com.nyud.net:8090/issues/issue37.htm

Displaying http
-
graph Output


Writes a small “dot” file


“dot” part of “graphviz” tool


Use “neato” to graph


Output formats: SVG, PS, PDF, image map


Can make fully interactive!

Example http
-
graph Output

Grabbing Data with pynids


tcp.{server, client}.data and just strings


Any string operations will work


Searching

if “HTTP/1.0” in tcp.client.data:


Regular Expression searches

if re.search(“HTTP/1.[10]”, tcp.client.data):


Rewriting

string.replace(req, “GET HTTP/1.0”, “”, 1)

More Fun!


Privacy invasion


Snarf mail


Log conversations


IRC, AIM, etc …


Steal files


FTP, P2P apps, HTTP downloads …


Disrupt sessions

tcp.kill()

New dsniff is written in Python …

flowgrep


Marries sniffing with regular expressions


A lot like ngrep, tcpkill, and dsniff


Logs the whole connection, not just a packet


Look for data in streams using regular
expressions


Log or kill selected streams


Dirt cheap IDS or IPS


Under 400 lines of code


Resources


http://www.tcpdump.org/


http://www.packetfactory.net/projects/libnids/


http://monkey.org/~provos/libevent/


http://monkey.org/~dugsong/{dpkt, pycap}


http://oss.coresecurity.com/projects/pcapy.html


http://monkey.org/~jose/software/flowgrep/


http://pilcrow.madison.wi.us/pynids/


Additional Resources


Stevens, TCP/IP Illustrated vols 1 and 2


Schiffman, Building Open Source Network
Security Tools


RFCs from the IETF