REDUNDANCY ELIMINATION AS A NETWORK-WIDE SERVICE

enginestagΔίκτυα και Επικοινωνίες

26 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

78 εμφανίσεις

REDUNDANCY ELIMINATION AS A
NETWORK
-
WIDE SERVICE

Aditya Akella

UW
-
Madison




Shuchi

Chawla


Ashok
Anand


Chitra

Muthukrishnan

UW
-
Madison


Srinivasan

Seshan

Vyas

Sekar

CMU


Ram
Ramjee

MSR
-
India

Scott
Shenker

UC
-
Berkeley

Growing traffic vs. network performance

2


Network traffic volumes growing
rapidly


Annual growth: overall (45%),
enterprise (50%), mobile (125%)*



Growing strain on installed
capacity everywhere


Core (Asian ISPs


80
-
90%
core utilization), enterprise
access, data center, cellular,
wireless…



How to sustain robust network
performance?

* Interview with Cisco CEO, Aug 2007, Network world






Enterprises








Mobile

users

Home users

Video

Data

centers

Web content

Other
svcs

(backup)

ISP

core

Strain on installed

link capacities






Enterprises

Scale link capacities by suppressing
duplicates

3


A key idea:

suppress duplicates


Popular objects, partial content
matches, backups, app headers


Effective capacity improves ~ 2X


Many approaches


Application
-
layer caches


Protocol
-
independent schemes


Below app
-
layer


WAN accelerators, de
-
duplication


Content distribution


CDNs

like
Akamai
, CORAL


Bittorrent


Point solutions


apply to
specific link, protocol, or app

Mobile

users

Home users

Video

Data

centers

Web content

Other
svcs

(backup)

Wan

Opt

Wan

Opt

Dedup
/

archival

Dedup
/

archival

ISP HTTP

cache

CDN

Universal need to scale capacities

4





Wan

Opt

Wan

Opt

Dedup
/

archival

Dedup
/

archival

ISP HTTP

cache

Network Redundancy

Elimination Service

Point solutions
inadequate

RE: A primitive operation
supported inherently in

the network


o

Applies to all links,
flows (long/short), apps,
unicast
/multicast


o

Transparent network
service; optional end
-
point modifications


o

How? Implications?


Architectural support to
address universal need to
scale capacities? Implications?

Bittorrent



Point solutions:

Little or no benefit

in the core



Point solutions:

Other links must

re
-
implement specific

RE mechanisms



Point solutions:


Only benefit

system/app

attached

How? Ideas from WAN optimization

5

5

Cache

Cache

WAN link

Data center

Enterprise








Network must examine byte streams, remove duplicates, reinsert


Building blocks from WAN optimizers: RE agnostic to application, ports or
flow semantics


Upstream cache = content table + fingerprint index


Fingerprint index: content
-
based names for chunks of bytes in payload


Fingerprints computed for content, looked up to identify redundant byte
-
strings


Downstream cache: content table











Internet2

Packet cache

at every
router

Network RE service:
apply protocol
-
indep

RE at the packet
-
level
on network links



IP
-
layer RE service

From WAN acceleration to

router packet
caches

6

Wisconsin

Berkeley

CMU

Router upstream removes
redundant
bytes


Router downstream
reconstructs
full
packet

(Hop
-
by
-
hop works for

slow links


Alternate approaches to
scale to faster links…)

Implications overview: Performance and
architectural benefits

7


Improved performance everywhere even if partially enabled


Generalizes point deployments

and app
-
specific approaches


Benefits all network end
-
points, applications


Crucially,
benefits ISPs


Improved switching capacity,
responsiveness to sudden overload



Architectural benefits


Enables

new protocols and apps


Min
-
entropy routing, RE
-
aware traffic engineering (intra
-

and inter
-
domain)


Anomaly detection, in
-
network filtering of unwanted traffic


Simplifies/improves apps
: need not worry about using network
efficiently


Application control messages & headers can be verbose


better diagnostics


Controlling duplicate transmission in app
-
layer multicast is a non
-
issue












Internet2

Implications example: Performance benefits

8

Network RE



12
pkts


(ignoring tiny
packets)

Without RE



18
pkts

33% lower

Wisconsin

Berkeley

CMU

Generalizes point

deployments

Benefits ISPs:
improve effective
switching capacity

6

2
packets

3

2
packets

3

2
packets

Wisconsin









Internet2

Implications example: New protocols

9

RE + routing


10
pkts

Simple RE



12
pkts

Berkeley

CMU

9



Redundancy
-
based anomaly


detectors



Network
-
assisted spam filtering



New content distribution


mechanisms



Minimum
-
entropy routing



New, flexible traffic engineering


mechanisms



Inter
-
domain protocols

Network RE service: Quantitative results

10


Analysis of 12 enterprises:
traffic 15
-
60% redundant
[SIGMETRICS 09]


~1GB of cache sufficient to identify redundancies


DRAM or PCM (PRAM) on routers



Network RE
benefits both ISPs and end
-
networks
[SIGCOMM 08]


Upto

15
-
50% better
util
, responsive TE, control inter
-
domain traffic impact


Centralized algorithm for min
-
entropy routing (using “redundancy profiles”)


Reduces utilization by a further 10
-
25% in intra
-
domain case


Inter
-
domain min
-
entropy routing: gains much more significant (50
-
80%)



Is network RE
viable at high speeds
?

Not in its current form…


Compression is slow: limits hop
-
by
-
hop speed at each hop to 2.5Gbps


Acceptable for access, wireless, cellular links, not for the core


Also, wastes memory on multiple routers


limits effectiveness

SmartRE
: Concerted network
-
wide RE

11


Toss out link
-
by
-
link view; treat RE as a network
-
wide problem per ISP
[Current work]


Memory usage:
each packet compressed/un
-
compressed once


Throughput:
allow reconstruction multiple hops away from compression


Stand
-
alone reconstruction much faster when freed from dependence on
compression immediately upstream


Reconstructor

can reconstruct a lot more, from multiple different compression
agents


Resource
-
awareness:
carefully account for network and device resources,
and traffic


Compression/reconstruction/caching locations decided based on memory capacity
and memory operations


Also consider global TE objectives


Just 4% from ideal RE (no memory or processing constraints)



Summary and future directions

12


RE service to scale link capacities everywhere


Architectural niceties and performance benefits


High speed router RE seems feasible


Future directions


End
-
host participation


Role of different memory technologies


DRAM, flash and PCM


Theoretical issues


pricing and economics, routing policy, network
design


Network coding as an alternative to compression