On error correction for networks and
deadlines
Tracey
Ho
Caltech
INC, 8/5/12
Introduction
Network error correction
[Yeung & Cai 06]
s
t
•
Errors in some bits, locations unknown
→ Code across bits
•
Errors in some links/packets, locations unknown
→ Code across links/packets
s2
unknown
erroneous
links
t
1
t
2
network
s1
Classical error correction
Problem statement
•
Given a network and error model
−
What communication rates are feasible?
(info theory)
−
How to achieve with practical codes?
(coding theory)
s2
unknown
erroneous
links
t
1
t
2
network
s1
5
1
4
1
r
2
r
1
Background
–
network error correction
•
Rich literature on single

source multicast with uniform
errors
−
All sinks demand the same information
−
Equal capacity network links/packets, any
z
can be
erroneous
−
Various capacity

achieving codes, e.g.
[Cai & Yeung 06,
Jaggi et al. 08,
Koetter & Kschischang 08]
and results on code properties, e.g. [Yang & Yeung 07,
Balli
, Yan & Zhang 07, Prasad &
Sundar
Rajan 09]
This talk
•
Generalizations
−
Non

multicast demands, multiple sources,
rateless
codes,
non

uniform link capacities
−
New capacity bounding and coding techniques
•
Applications
−
Streaming (non

multicast nested network)
−
Distribution of keys/decentralized data (multi

source
network)
−
Computationally limited networks (
rateless
codes)
Outline
•
Non

multicast nested networks, streaming communication
•
Multiple

source multicast, key distribution
•
Rateless
codes, computationally limited networks
•
Non

uniform link capacities
Outline
•
Non

multicast nested networks, streaming communication
O. Tekin, S. Vyetrenko, T. Ho and H. Yao, "Erasure correction for nested
receivers,"
Allerton
2011.
O. Tekin, T. Ho, H. Yao and S. Jaggi, “On erasure correction coding for
streaming,” ITA 2012.
D. Leong and T. Ho, “Erasure coding for real

time streaming,” ISIT 2012.
•
Multiple

source multicast, key distribution
•
Rateless
codes, computationally limited networks
•
Non

uniform link capacities
www.mobileapptesting.com/testing

the

alien

hunting

android

app/2011/03/et

phone

home
Background

non

multicast
•
Not all sinks demand the same information
•
Capacity even without errors is
an open problem
−
May need to code across different sinks’ data (inter

session coding)
−
Not known in general when intra

session coding suffices
•
Non

multicast network error correction
−
Capacity bounds from analyzing three

layer networks
(Vyetrenko, Ho & Dikaliotis 10)
−
We build on this work to analyze coding for streaming of
stored and online content
Streaming stored content
m
1
I
1
m
2
I
2
m
3
I
3
Demanded information
x
x
Initial play

out delay
Decoding deadlines
Forward error correction
Source
Packet erasures
packet erasure link
(unit size packets)
Nested network model
I
1
, I
2
, I
3
I
1
I
1
, I
2
t
1
t
2
t
3
m
1
I
1
m
2
I
2
m
3
I
3
Deadlines
Sinks
Demanded info
x
x
Spatial
network
problem
Temporal
coding
problem
•
Each sink sees a subset of
the info received by the next
(nested structure)
•
Non

multicast demands
Unit
capacity
links
Unit size
packets
source
Nested network model
I
1
, I
2
, I
3
I
1
I
1
, I
2
t
1
t
2
t
3
m
1
I
1
m
2
I
2
m
3
I
3
x
x
Packet error/erasure correction
streaming code
Capacity outer bound
Finite
blocklength
network error/
erasure correction code
Capacity outer bound
source
Problem and results
Problem
•
Given an erasure model and deadlines
m
1
, m
2
, …, what rate
vectors
u
1
, u
2
, … are achievable?
Results
•
We find the capacity and a simple optimal coding scheme for
a uniform erasure model
−
At most
z
erasures, locations unknown
a priori
•
We show this scheme achieves at least a guaranteed fraction
of the capacity region for a sliding window erasure model
−
Constraints on number of erasures in sliding windows of
certain length
−
Exact optimal scheme is sensitive to model parameters
Problem and results
Problem
•
Given an erasure model and deadlines
m
1
, m
2
, …, what rate
vectors
u
1
, u
2
, … are achievable?
Results
•
We find the capacity and a simple optimal coding scheme for
a uniform erasure model
−
At most
z
erasures, locations unknown
a priori
•
We show this scheme achieves at least a guaranteed fraction
of the capacity region for a sliding window erasure model
−
Constraints on number of erasures in sliding windows of
certain length
−
Exact optimal scheme is sensitive to model parameters
z
erasures
–
upper bounding
capacity
•
Want to find the capacity region of
achievable rates
u
1
,u
2
,…,u
n
•
We can write a cut

set bound for
each sink:
u
1
≤ m
1
̶ z
u
1
+u
2
≤ m
2
̶ z
…
u
1
+u
2
+…+u
n
≤
m
n
̶ z
•
Can we combine bounds for multiple
erasure patterns and sinks to obtain
tighter bounds?
I
1
I
1
, I
2
I
1
, I
2
, I
3
t
1
t
2
t
3
Cut

set combining procedure
•
Obtain bounds involving progressively
more links and rates
u
i
, by iteratively applying steps:
Extend:
H(X

I
1
i

1
)+ Y ≥ H(X,Y

I
1
i

1
)= H(X,Y

I
1
i
)+
u
i
where
X,Y
is a decoding
set for
I
i
Combine:
)

,
(
1


)

,
(
1


,
:
1
i
k
A
X
A
A
i
I
X
Z
H
k
X
I
A
Z
H
Example:
m
1
=3,m
2
=5, m
3
=7, m
4
=11, z=1
12
13
23
1245
1345
2
345
123
4
123
5
124567
134567
2
34567
123467
123567
123
45
6
123
45
7
12345
6
12345
7
12345
67
12345
67
123
4567
123
4567
1234567
u
1
+
H(X
1
X
2

I
1
) ≤2
u
1
+
H(X
1
X
3

I
1
) ≤2
u
1
+
H(X
2
X
3

I
1
) ≤2
Upper bound derivation graph
•
Different choices of links at each step give different upper bounds
•
Exponentially large number of bounds
•
Only some are tight
–
how to find them?
•
We use an achievable scheme as a guide and show a matching
upper bound
Example:
m
1
=3,m
2
=5, m
3
=7, m
4
=11, z=1
u
1
≤2
3u
1
+2u
2
≤8
3u
1
+2u
2
+u
3
≤9
6u
1
+5u
2
+4u
3
≤24
6u
1
+4u
2
+2u
3
+u
4
≤20
9u
1
+6u
2
+4u
3
+3u
4
≤36
6u
1
+5u
2
+4u
3
+2u
4
≤28
6u
1
+4.5u
2
+4u
3
+3u
4
≤30
9u
1
+7.5u
2
+7u
3
+6u
4
≤54
Capacity region:
12
13
23
1245
1345
2
345
123
4
123
5
124567
134567
2
34567
123467
123567
123
45
6
123
45
7
12345
6
12345
7
12345
67
12345
67
123
4567
123
4567
1234567
Intra

session Coding
•
A rate vector
(u
1
,u
2
,…,u
n
)
is achieved if and only if for every
unerased
set
P
:
1
2
…
m
n
Σ
P
I
1
y
1,1
y
1,2
…
y
1,m_n
≥ u
1
I
2
y
2,1
y
2,2
…
y
2,m_n
≥ u
2
…
…
…
…
…
I
n
y
n,1
y
n,1
…
y
n,m_n
≥ u
n
Σ
≤
1
≤
1
≤
1
•
Separate erasure coding over
each sink’s data
•
Code design → capacity allocation problem
•
y
j,k
: capacity on
k
th
link allocated to
j
th
sink’s data
•
We may assume
y
j,k
=
0
for
k>
m
j
1,2,…,9,10
11,12,13,14
15,16,17,18
19,20,21,22
I
1
I
2
I
3
I
4
0.2
0.4
0.4
0.4
0
“As uniform as possible” intra

session
coding scheme
0.75
0.1875
0.1875
0.25
0.1875
0.2
0.2
0.2
0.5
0.5
0
0.5
0.25
0.5
0.25
0
m
1
= 10, m
2
= 14, m
3
= 18, m
4
= 22,
u
1
= 6, u
2
= 3, u
3
= 3, u
4
= 4, z=2
75
.
0
2
10
6
1
1
z
m
u
25
.
0
2
14
3
2
2
z
m
u
1875
.
0
2
18
3
3
3
z
m
u
5
.
0
2
10
18
3
1
3
3
z
m
m
u
2
.
0
2
22
4
4
4
z
m
u
4
.
0
2
10
22
4
1
4
4
z
m
m
u
5
.
0
2
14
22
3
25
.
0
*
4
2
4
4
z
m
m
u
•
For a given rate vector, fill each row as uniformly as possible
subject to constraints from previous rows
•
Example:
Can we do better?
Capacity region
Theorem: The
z

erasure (or error) correction capacity region is
achieved by the “as uniform as possible”
coding scheme.
•
Characterization of the capacity region in a form that is simple
to specify and calculate
•
Intra

session coding is also relatively simple to implement
Proof Idea
•
Consider any given rate vector
(u
1
,u
2
,…,u
n
)
and let
T
i,j
denote
its corresponding “as uniform as possible” allocation:
•
Show inductively: the conditional entropy of any set of
unerased
links given messages
I
1
,…,
I
k
matches the residual
capacity from the table
•
Use
T
i,j
values to find the appropriate path through upper
bound derivation graph
1,2,…,m
1
m
1
+1,…,m
2
m
2
+1,…,m
3
…
m
n

1
+1,…,
m
n
I
1
T
1,1
I
2
T
2,1
T
2,2
…
…
…
…
…
I
n
T
n,1
T
n,2
T
n,3
…
T
n,n
Streaming online content
•
Messages arrive every
c
time steps at the source, and must be
decoded within
d
time steps
packet erasure link
(unit size packets)
Message
decoding
deadlines (
d
=8)
Message
creation
times (
c
=3)
Problem and results
Problem
•
Given an erasure model and parameters
c
and
d
, what is the
maximum size of independent uniform messages?
Results
•
We find the capacity and a simple coding scheme that is
asymptotically optimal for the following erasure models:
−
#1: Limited number of erasures per sliding window
−
#2: Erasure bursts and guard intervals of certain lengths
•
For other values of burst length and guard interval, optimal
inter

session
convolutional
code constructions [
Martinian
&
Trott
07, Leong & Ho 12]
Problem and results
Problem
•
Given an erasure model and parameters
c
and
d
, what is the
maximum size of independent uniform messages?
Results
•
We find the capacity and a simple coding scheme that is
asymptotically optimal for the following erasure models:
−
#1: Limited number of erasures per sliding window
−
#2: Erasure bursts and guard intervals of certain lengths
•
For other values of burst length and guard interval, optimal
inter

session
convolutional
code constructions [
Martinian
&
Trott
07, Leong & Ho 12]
Code construction
•
Divide each packet evenly among current messages
•
Intra

session coding within each message
when
d
is a multiple of
c
…
messages 2, 3, 4 are current at
t
= 12
constant number of
current messages at
each time step
Code construction
•
Divide each packet evenly among current messages
•
Intra

session coding within each message
variable number of
current messages at
each time step
messages 3, 4, 5 are current at
t
= 13
messages 3, 4 are current at
t
= 12
when
d
is not a multiple of
c
…
Capacity result
•
Like the previous case, converse obtained by
−
combining bounds for multiple erasure patterns and sinks
(deadlines)
−
inductively obtaining upper bounds on the entropy of sets
of
unerased
packets, conditioned on previous messages
•
The converse bound coincides with the rate achieved by our
coding scheme asymptotically in the number of messages
n
•
Gap for small
n
corresponds to underutilization of capacity at
the start and end by the time

invariant coding scheme
Outline
•
Non

multicast nested networks, streaming communication
•
Multiple

source multicast, key distribution
T. Dikaliotis, T. Ho, S. Jaggi, S. Vyetrenko, H. Yao, M. Effros, J. Kliewer and E.
Erez, "Multiple

access Network Information

flow and Correction Codes,"
IT Transactions
2011
.
H. Yao, T. Ho and C. Nita

Rotaru, "Key Agreement for Wireless Networks in
the Presence of Active
Adversaries,"Asilomar
2011
.
•
Rateless
codes, computationally limited networks
•
Non

uniform link capacities
www.fanpop.com/spots/the

usual

suspects
Multiple

source multicast, uniform
z
errors
•
Coherent (known topology) and
noncoherent
(unknown topology)
cases
s
2
t
s1
•
Sources with independent information
We could partition network capacity among different sources…
But could rate be improved by coding across different sources? To
what extent can different sources share network capacity?
Challenge: owing to the need for coding across sources in the
network and independent encoding at sources, straightforward
extensions of single

source codes are suboptimal
Related work: code construction in (Siavoshani, Fragouli & Diggavi
08) achieves capacity for
C1+C2=C
Multiple

source multicast, uniform
z
errors
•
Sources with independent information
•
We could partition network capacity among different sources…
But could rate be improved by coding across different sources? To
what extent can different sources share network capacity?
Challenge: owing to the need for coding across sources in the
network and independent encoding at sources, straightforward
extensions of single

source codes are suboptimal
Related work: code construction in (
Siavoshani
, Fragouli & Diggavi
08) achieves capacity for
C1+C2=C
s2
t
s1
•
Coherent (known topology) and
noncoherent (unknown topology)
cases
Multiple

source multicast, uniform
z
errors
•
Sources with independent information
•
We could partition network capacity among different sources…
•
But could rate be improved by coding across different sources?
To what extent can different sources share network capacity?
•
Challenge: owing to the need for coding across sources in the
network and independent encoding at sources, straightforward
extensions of single

source codes are suboptimal
•
Related work: code construction in (
Jafari
, Fragouli & Diggavi 08)
achieves capacity for
C1+C2=C
s2
t
s1
•
Coherent (known topology) and
noncoherent (unknown topology)
cases
Capacity region
•
Theorem
: The coherent and non

coherent capacity region
under any
z
link errors is given by the cut set bounds
−
U
= set of source nodes
−
m
S
= min cut capacity between sources in subset
S
of
U
and
each sink
−
r
i
= rate from the
i
th
source
•
Redundant capacity can be fully shared via coding
U
S
z
m
r
S
i
S
i
,
2
Capacity

achieving non

coherent
code
constructions
1.
Probabilistic construction
−
Joint decoding of sources, using injection distance metric
−
Subspace distance metric used in single

source case
is
insufficient in multi

source case
2.
Lifted
Gabidulin
rank metric codes over nested fields
−
Successive decoding of sources
−
Linear transformation to separate out other sources’
interference increases the field size of errors
−
Sources
encode over
nested extension fields
An application: key distribution
•
Robust distribution of keys from a pool (or other decentralized
data)
•
Nodes hold subsets of keys, some pre

distributed
•
Further exchange of keys among nodes
•
Want to protect against some number of corrupted nodes
•
Questions:
−
How many redundant transmissions are needed?
−
Can coding help?
V1
V2
V
3
V4
V5
V6
V7
V8
V9
k
1, k2
k
1
, k
2
k
1, k2
k
1, k3
k
1, k3
k
1, k3
k2, k3
k2, k3
k2, k3
R
wants k1, k2, k3
An application: key distribution
•
Problem is equivalent to multi

source network error correction
•
Coding across keys strictly outperforms forwarding in general
S2
V1
V2
V3
V
4
V5
V6
V7
V8
V9
V1
V2
V
3
V4
V5
V6
V7
V8
V9
k
1, k2
k
1
, k
2
k
1, k2
k
1, k3
k
1, k3
k
1, k3
k2, k3
k2, k3
k2, k3
R
S1
S3
R
wants k1, k2, k3
Outline
•
Non

multicast nested networks, streaming communication
•
Multiple

source multicast, key distribution
•
Rateless
codes, computationally limited networks
S. Vyetrenko, A. Khosla & T. Ho, “On combining information

theoretic and
cryptographic approaches to network coding security against the pollution
attack,”
Asilomar
2009.
W. Huang, T. Ho, H. Yao & S. Jaggi, “
Rateless
resilient network coding
against Byzantine adversaries,” 2012.
•
Non

uniform link capacities
Background
–
adversarial errors in
multicast
•
Information theoretic network error correction
−
Prior codes designed for a
given
mincut
and
max no. of
errors
z
U
−
Achieve
mincut

2
z
U
, e.g. [
Cai
and Yeung
06
, Jaggi et al.
08
,
Koetter &
Kschischang
08
]
−
No computational assumptions on adversaries
−
Use network diversity and redundant capacity as resources
•
Cryptographic signatures with
rateless
network codes
−
Signatures for checking network coded packets, e.g.
[Charles et al.
06
, Zhao et al.
07
,
Boneh
et al.
09
]
−
Achieve realized value of
mincut
after erasures
−
Use computation, key infrastructure as resources
Motivation
Cryptographic approach
+
Does not require a priori estimates of network capacity and
errors (
rateless
)
+
Achieves higher rate
−
Performing signature checks requires significant computation;
checking all packets at all nodes can limit throughput if nodes
are computationally weak,
e.g. low

power wireless nodes
Questions:
•
Can we achieve the
rateless
benefits without the
computational drawback?
•
Can we use both network diversity as well as computation as
resources, to do better than with each separately?
Rateless
network error correction
codes
•
Incrementally send redundancy until decoding succeeds
•
Without an a priori bound on the number of errors, need a means
to verify decoding
•
We give code constructions using respectively:
1.
Shared secret randomness between source and sink (small
compared to message size)
2.
Cryptographic signatures
•
These constructions are asymptotically optimal:
−
Decoding succeeds
w.h.p
. once received information/errors
satisfy cut set bound
−
Overhead becomes negligible with increasing packet length
Rateless
code using shared secret
•
Shared secret is random and independent of the message
•
Non

rateless
case [
Nutman
and Langberg 08]
−
Redundancy
Y
added to message
W
so as to satisfy a matrix
hash equation
[
Y W I
]
V= H
defined by shared secret
(V, H)
−
Hash is used to extract
[
Y W I
]
from received subspace
•
Challenges in the
rateless
case:
1.
Calculate redundancy incrementally such that it is
cumulatively useful for decoding
2.
Send redundancy incrementally
•
Growth in dimension of subspace to be recovered in turn
necessitates more redundancy
•
Each adversarial error packet can correspond to an addition (of
erroneous information) and/or an erasure (of good information)
•
Code structure:
•
y
k
=
w
V
(k)
+
h
k
, where
w
is the
vectorized
message,
V
ij
(k)
=
a
k
ij
,
and
h
k
and
a
k
are shared secrets
Rateless
code using shared secret
y
3
Message
W
B
I
d
31
y
1
+
0 0 1
d
11
d
32
y
2+
d
33
y
3
y
2
d
31
d
32
d
33
d
22
y
2
d
11
y
1
d
21
y
1
+
d
21
d
22
y
1
1
0 1
Linearly
dep
redundancy for
erasures
Long packets
C
2
C
2
W
C
3
C
3
W
C
4
C
4
W
C
1
C
1
W
Linearly
indep
redundancy for
additions
Short packets
Rateless
code using signatures
•
Each adversarial error packet can correspond to an addition (of
erroneous information) and/or an erasure (of good information)
•
Code structure:
•
y
i
=
w
S
i
, where
w
is the
vectorized
message and
S
i
is a generic
known matrix
B
I
Message
W
Y
1
B
I
I
0
0
0
C
2
W+D
21
Y
1
+D
22
Y
2
Linearly dependent
redundancy for
erasures
Y
2
C
2
Linearly independent
redundancy for
additions
D
22
C
1
C
1
W+D
11
Y
1
D
11
D
21
Example: simple hybrid strategy on
wireless butterfly network
D
source
1
sink
2
sink
Node
D
has limited computation
and outgoing capacity
→ Probabilistically checks/codes a
fraction of packets
−
Proportion of packets
checked/coded chosen to
maximize expected information
rate subject to computational
budget
Example: simple hybrid strategy on
wireless butterfly network
20
,
200
mincut
,
40
coding
of
cost
checking
of
cost
z
Outline
•
Non

multicast nested networks, streaming communication
•
Multiple

source multicast, key distribution
•
Rateless
codes, computationally limited networks
•
Non

uniform link capacities
S. Kim, T. Ho, M. Effros and S. Avestimehr, "Network error correction with
unequal link capacities," IT Transactions 2011.
T. Ho, S. Kim, Y. Yang, M. Effros and A. S. Avestimehr, "On network error
correction with limited feedback capacity," ITA 2011.
http://www.geekosystem.com/tag/starbucks/
Uniform and non

uniform links
•
Adversarial errors on any
z
fixed but unknown links
•
Uniform links:
−
Multicast error correction capacity = min cut
–
2z
−
Worst

case
errors occur on the min cut
•
Non

uniform links:
−
Not obvious what are worst

case errors
•
Cut size versus link capacities
•
Feedback across cuts matters (can provide information
about errors on upstream links)
−
Related work: Adversarial nodes (Kosut, Tong & Tse 09)
Tighter cut set bounding approach
•
The classical cut
set
bound
is equivalent to adding reliable,
infinite

capacity bidirectional links between each pair of
nodes on each side of the cut
•
Tighter bounds can be obtained
by taking into account which
forward links affect or are affected by which feedback links
•
Equivalent to adding a link (
i,j
) only if there is a directed path
from node
i
to node j on that does not cross the cut
46
Zigzag network
New cut

set bound
For any cut
Q
,
•
adversary can erase a set of
k
≤
z
forward links
•
adversary then chooses two sets
Z
1
,Z
2
of
z

k
links
s.t
. decoder
cannot distinguish which set is adversarial:
−
no feedback links downstream of
Z
1
,Z
2
,
−
downstream feedback links are included in
Z
i
, or
−
downstream feedback links
W
i
that are not in
Z
i
have
relatively small capacity
s.t
. distinct
codewords
have the
same feedback link values
•
sum of capacities of remaining forward links + capacities of
links in
W
1
,W
2
is an upper bound
Bound is tight on some families of zigzag networks
z=
1
Achieve rate 3
using new code
construction
Achievability

example
•
For
z=
1
, upper bound
= 5
•
Without feedback link, capacity
= 2
•
Can we use feedback link to achieve
rate
5
?
∞
∞
z=
1
Achieve rate 3
using new code
construction
b
r
2
a
e
•
Some network capacity is allocated to
redundancy enabling partial error
detection at intermediate nodes
•
Nodes that detect errors forward
additional information allowing the sink to
locate errors
•
Use feedback capacity to increase the
number of symbols transmitted with error
detection
•
Remaining network capacity carries an
MDS error correction code over all
information symbols
r
1
c
+c
a
d
b
“Detect and forward”
coding strategy
z=
1
capacity
= 5
∞
∞
Conclusion
•
Network error correction
−
New coding and outer bounding techniques for non

multicast demands, multiple sources, non

uniform errors
−
A model for analysis and code design in various
applications, e.g. robust streaming, key distribution
−
Rateless
and hybrid codes for computationally limited
networks with adversaries
Thank you
Comments 0
Log in to post a comment