Packet Mangling for Fun & Profit

volleyballbeginnerNetworking and Communications

Oct 27, 2013 (3 years and 7 months ago)

98 views

Packet Mangling for Fun & Profit

A Brief Intro to Netfilter, User
Mode Linux, and the Linux TCP/IP
Stack

Topics


SplitTCP Summary (Context)


Netfilter Introduction


One buffer to rule them all: sk_buffs


User Mode Linux


Current Status

SplitTCP In a Nutshell


Work started by Michalis, Srikanth, and
Swastik.


The idea is to add transport layer proxies to
long (in terms of hop count) TCP connections.


If a link fails (due to mobility, etc.) the packet
can be re
-
transmitted by the proxy closest to
the failure.


A “demo
-
quality” implementation was done
which made many simplifying assumptions.

So What?


My current task is to take that demo
implementation and turn it into a
general implementation.


This presentation discusses the lessons
learned up to this point in that process.

Design Goals


Any and all modifications made to the
behavior of TCP should be backwards
compatible.


I.e. nodes should interoperate regardless
of weather or not they’re “split tcp
enabled”.)


If possible, no changes should be made
to the kernel proper. It’s just better for
everyone that way…

Enter Netfilter


Fortunately, Linux has a plugin API,
called Netfilter, which allows kernel
modules to hook into strategic spots in
the networking stack.


Unfortunately, in the grand tradition of
open source, documentation takes a
back seat to implementation


so
there’s a non
-
trivial learning curve.

Sideline: Plugin APIs


An example of the Delegation design
pattern (GoF)


a work unit is passed
through a series of cooperating steps to
produce the final result.


A well
-
designed plugin architecture can
facilitate otherwise impossible tasks and
simply possible ones.

Sideline: Kernel Modules


Most of the linux kernel can be built as
a module


a bit of code that’s loaded
into (and unloaded from) the kernel
dynamically.


Device drivers, file systems, even
networking protocols (IPX, Appletalk)
are candidates for modularization.

Writing a Module


Just define a few preprocessor macros
(__KERNEL__, MODULE)


Include a few header files
(linux/module.h, linux/version.h,
linux/config.h)


And honor a small interface
(module_init(), module_exit())


You’ll have a no
-
op module.

Netfilter Overview


Initial design by Paul “Rusty” Russell


Netfilter is a series of callback functions
within the network stack. The API is
non
-
portable and appeared in linux
2.3.x


Each protocol has it’s own set of
callback points. We care about IPv4.

Netfilter Concepts


A module expresses interest in being
invoked at an arbitrary subset of the
available callback points


specifying
the function and the (global) priority in
which it should be called.


That function is passed (among other
things) a pointer to a pointer to a
packet buffer ( sk_buff ** ).

Return Values


The netfilter function has five possible
return values:


NF_ACCEPT: continue callback chain


NF_DROP: drop the packet and stop the
chain


NF_STOLEN: stop the chain


NF_QUEUE: send the packet to userspace


NF_REPEAT: call the hook again

Netfilter Hooks in IPv

Routing Engine

Local Sockets

1

In

Out

2

3

4

5

Say that Again?


1: NF_IP_PREROUTING


any received packet which checksums OK.


2: NF_IP_LOCAL_IN


packets destined for local sockets


3: NF_IP_FORWARD


foreign packets being forwarded


4: NF_IP_POST_ROUTING


any outbound packet


5: NF_IP_LOCAL_OUT


packets originating from local sockets

Routing Engine

Local Sockets

1

In

Out

2

4

5

An sk_what??


Linux uses a structure called an sk_buff
to store packet data internally.


It contains a handful of pointers to
other structures as well as a packet
data region.

Data

Sk_buff’s and you


The data area is like a stack, only you
can insert at the head and the tail
(deque?).


The kernel provides a handful of helper
functions to manage sk_buff’s and their
data areas.


The various header pointers point into
the data area


which can be thought of
as a serialized packet.

Why do We Care?


An sk_buff is built as a packet travels
down the stack


each layer (TCP, IP,
Ethernet) adds their own special sauce.


This means that each header is
“squashed” in against the next


so
while modifying existing data is
relatively easy, adding new header data
is a bit trickier.

Don’t leave me in suspense…


Basically, you make a copy of the
sk_buff, and ask it to “grow” a bit
during the copy.


Once you have the copy


you “slide”
the IP and TCP headers backwards a
bit, insert the new option bytes, and re
-
checksum the packet.

A tale of “n” checksum’s


Sounds easy, right? Remember that
these sk_buffs are built one layer at a
time?


There is no nice friendly function which
will take a TCP sk_buff and compute all
the needed checksums.


Funny thing about checksums


almost
isn’t good enough.

Ok, insmod and <BOOM>


Remember developing on a system
without memory protection and having
to reboot ?


Kernel modules execute in kernel space


so no one’s watching your back. If
you goof, it’s time to reboot.


User Mode Linux to the rescue

User Mode Linux (UML)


A kernel patch that allows running the
linux kernel as a user
-
mode process on
a linux machine.


If you crash the user
-
mode kernel, you
just restart the process, no reboot
required.

Where do I sign up?


Setup is (in principal) fairly easy


only it
turns out that the standard distribution
doesn’t have netfilter enabled.


So I re
-
built with the appropriate options and
placed the binaries in ~swift/user
-
mode
-
linux/bin


There’s a README there


some support
executables must be installed as root on your
workstation.

That’s It?


Not quite


you also need a file system to
boot this kernel off.


Good News: you can download a file system
image


you don’t have to make one.


Bad News: it’s really big. (up to 700MB)


Good News: you can share one among many
people


Bad News: you have to read the HOWTO.

Building Modules Under UML


Building a kernel module is the same
under UML as under “KML”


it is
important that you build it against the
source used to build the target kernel.


For “our” UML build


that source is in
~swift/user
-
mode
-
linux/src

Current Status


A skeleton of a splittcp module (tcpproxy.c)
exists.


It can inspect locally generated packets and
add our newly defined PROXY option,
rechecksum the packet, and send it on it’s
way.


It can inspect arriving packets, check for the
option, and decide if that packet should be
proxied.

So What’s Left?


It doesn’t (yet) actually proxy the
packet.


Nor does it send an acknowledgement
of receipt to the upstream proxy.


There are also issues around ICMP error
messages, and what should be done
about them.

Conclusion


Once the code is “alpha” quality, I’ll
commit it to the swift CVS repository for
your collective viewing pleasure.


Until then, if you have questions or
suggestions, see me.