Yonina Eldar
Technion
–
Israel Institute of Technology
http://www.ee.technion.ac.il/people/YoninaEldar
yonina@ee.technion.ac.il
Beyond Nyquist: Compressed
Sensing of Analog Signals
Dagstuhl Seminar
December, 2008
2
Sampling: “Analog Girl in a Digital World…”
Judy Gorman 99
Digital world
Analog world
Reconstruction
D
2
A
Sampling
A2D
Signal processing
Denoising
Image analysis…
(Interpolation)
3
Compression
Original 2500 KB
100%
Compressed
950
KB
38
%
Compressed 392 KB
15%
Compressed
148
KB
6
%
“Can we not just
directly measure
the part that will not end up being
thrown away ?
”
Donoho
4
Outline
Compressed sensing
–
background
From discrete to analog
Goals
Part I : Blind multi

band reconstruction
Part II : Analog CS framework
Implementations
Uncertainty relations
Can break the Shannon

Nyquist barrier
by exploiting signal structure
5
K
non

zero entries
at least
2
K
measurements
CS Setup
y A x
Recovery: brute

force, convex optimization, greedy algorithms, …
6
is uniquely determined by
is random
with high probability
Donoho,
2006
and Candès et. al.,
2006
NP

hard
Convex and tractable
Greedy algorithms: OMP, FOCUSS, etc.
Donoho,
2006
and Candès et. al.,
2006
Tropp, Elad, Cotter et. al,. Chen et. al., and many others
Brief Introduction to CS
Donoho and Elad,
2003
Uniqueness:
Recovery:
7
Naïve Extension to Analog Domain
Standard CS
Discrete Framework
Analog Domain
Sparsity prior
what is a sparse analog signal ?
Generalized sampling
Finite
dimensional elements
Infinite
sequence
Continuous
signal
Operator
Random
is stable w.h.p
Stability
Randomness
Infinitely many
Need structure
for efficient implementation
Finite program,
well

studied
Undefined program
over a continuous signal
Reconstruction
8
Random
is stable w.h.p
Naïve Extension to Analog Domain
Sparsity prior
what is a sparse analog signal ?
Generalized sampling
Finite
dimensional elements
Infinite
sequence
Continuous
signal
Operator
Stability
Randomness
Infinitely many
Need structure
for efficient implementation
Finite program,
well

studied
Undefined program
over a continuous signal
Reconstruction
Questions:
1.
What is the definition of analog sparsity ?
2.
How to select a sampling operator ?
3.
Can we introduce stucture in sampling and still
preserve stability ?
4.
How to solve infinite dimensional recovery
problems ?
Standard CS
Discrete Framework
Analog Domain
9
Goals
1.
Concrete analog sparsity model
2.
Reduce sampling rate (to minimal)
3.
Simple recovery algorithms
4.
Practical implementation in hardware
10
Analog Compressed Sensing
A signal with a multiband structure in some basis
no more than
N
bands, max width
B,
bandlimited to
(Mishali and Eldar
2007
)
1.
Each band has an uncountable
number of non

zero elements
2.
Band locations lie on an infinite grid
3.
Band locations are unknown in advance
What is the definition of analog sparsity ?
More generally
only
sequences
are non

zero
(Eldar 2008)
11
Multiband “Sensing”
bands
Sampling
Reconstruction
Goal: Perfect reconstruction
Next steps:
1.
What is the minimal rate requirement ?
2.
A fully

blind system design
Analog
Infinite
Analog
Known band locations (subspace prior):
Minimal

rate sampling and reconstruction
(NB)
with known band
locations (Lin and
Vaidyanathan
98
)
Half blind system (
Herley
and Wong
99
,
Venkataramani
and
Bresler
00
)
We are interested in unknown spectral support (a union of subspace prior)
(Mishali and Eldar 2007)
12
Rate Requirements
Average sampling rate
Theorem (non

blind recovery)
Landau (
1967
)
Theorem (blind recovery)
Mishali and Eldar (2007)
1.
The minimal rate is doubled
2.
For , the rate requirement is samples/sec (on average)
13
Sampling
Analog signal
In each block of samples, only are kept, as described by
Point

wise samples
0
2
3
0
0
2
2
3
3
Multi

Coset: Periodic Non

uniform on the Nyquist grid
14
The Sampler
DTFT
of sampling
sequences
Constant
matrix
known
in vector form
unknowns
Length
.
known
Problems:
1.
Undetermined system
–
non unique solution
2.
Continuous set of linear systems
is jointly sparse and unique under appropriate
parameter selection ( )
is sparse
Observation:
15
Paradigm
Solve finite
problem
Reconstruct
0
1
2
3
4
5
6
S
= non

zero rows
16
CTF block
Solve finite
problem
Reconstruct
MMV
Continuous to Finite
Continuous
Finite
span a finite space
Any basis preserves the sparsity
17
2

Words on Solving MMV
Variety of methods based on optimizing mixed column

row norms
We prove equivalence results by extending RIP and coherence to allow for
structured sparsity
(Mishali and Eldar, Eldar and Bolcskei)
New approach: ReMBo
–
Reduce MMV and Boost
Main idea: Merge columns of
V
to obtain a single vector problem
y=Aa
Sparsity pattern of a is equal to that of
U
Can boost performance by repeating the merging with different coeff.
Find a matrix
U
that has as few non

zero rows as possible
18
Algorithm
CTF
Continuous

to

finite block: Compressed sensing for analog signals
Perfect reconstruction at minimal rate
Blind system: band locations are unknown
Can be applied to CS of general analog signals
Works with other sampling techniques
19
Framework: Analog Compressed Sensing
Sampling signals from a union of shift

invariant spaces (SI)
generator
s
Subspace
(Eldar
2008
)
20
Framework: Analog Compressed Sensing
What happen if only K<<N sequences are not zero ?
There is no prior knowledge on the exact indices in the sum
Not a subspace !
Only k sequences are non

zero
21
Framework: Analog Compressed Sensing
Only k sequences are non

zero
CTF
Step
1
: Compress the sampling sequences
Step
2
: “Push” all operators to analog domain
System A
High sampling rate = m/T
Post

compression
22
Framework: Analog Compressed Sensing
CTF
Eldar (
2008
)
Theorem
System B
Low sampling rate = p/T
Pre

compression
23
Simulations
0
5
10
15
20
25
1
2
3
4
5
x 10
4
Time (nano secs)
50
0
50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
n
0
5
10
15
20
25
1
2
3
4
5
x 10
4
Time (nano secs)
Signal
Reconstruction
filter
Output
Time (nSecs)
Time (nSecs)
Amplitude
Amplitude
24
Simulations
Brute

Force
M

OMP
5
10
15
0
0.2
0.4
0.6
0.8
1
r
Empirical success rate
SBR4
SBR2
5
10
15
0
0.2
0.4
0.6
0.8
1
r
Empirical success rate
SBR4
SBR2
Sampling rate
Sampling rate
Minimal rate
Minimal rate
25
Simulations
SBR
4
SBR
2
r
SNR
5
10
15
10
15
20
0.2
0.4
0.6
0.8
1
r
SNR
5
10
15
10
15
20
0.2
0.4
0.6
0.8
1
r
SNR
5
10
15
10
15
20
50
100
150
200
250
0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
0.8
1
Empirical recovery rate
Sampling rate
Sampling rate
0
% Recovery
100
% Recovery
0
% Recovery
100
% Recovery
Noise

free
26
Multi

Coset Limitations
0
2
3
0
0
2
2
3
3
Analog signal
Point

wise samples
ADC
@ rate
Delay
1.
Impossible to match rate for wideband RF signals
(Nyquist rate >
200
MHz)
2.
Resource waste for IF signals
3
. Requires accurate
time delays
27
Efficient Sampling
Use CTF
Efficient
implementation
(Mishali, Eldar, Tropp
2008
)
28
Hardware Implementation
A few first steps…
29
Pairs Of Bases
Until now: sparsity in a single basis
Can we have a sparse representation in two bases?
Motivation: A combination of bases can sometimes better represent
the signal
Both and are small!
30
Uncertainty Relations
How sparse can be in each basis?
Finite setting: vector in
Elad and Brukstein
2002
Uncertainty
relation
Different bases
31
Analog Uncertainty Principle
Eldar (
2008
)
Theorem
Eldar (
2008
)
Theorem
32
Bases With Minimal Coherence
In the DFT domain
Spikes
Fourier
What are the analog counterparts ?
Constant magnitude
Modulation
“Single” component
Shifts
33
Analog Setting: Bandlimited Signals
Minimal coherence:
Tightness:
34
Finding Sparse Representations
Given a dictionary ,
expand using as few elements as possible:
minimize
Solution is possible using CTF if is small enough
Basic idea:
Sample with basis
Obtain an IMV model:
maximal value
Apply CTF to recover
Can establish equivalence with as long as is small enough
35
Conclusion
Extend the basic results of CS to the analog setting

CTF
Sample analog signals at rates much lower than Nyquist
Can find a sparse analog representation
Can be implemented efficiently in hardware
Questions:
Other models of analog sparsity?
Other sampling devices?
Compressed Sensing of Analog Signals
36
Some Things Should Remain At The
Nyquist Rate
Thank you
Thank you
High

rate
37
References
M. Mishali and Y. C. Eldar, "Blind Multi

Band Signal Reconstruction:
Compressed Sensing for Analog Signals,“ to appear in IEEE Trans. on
Signal Processing.
M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary
Sets of Jointly Sparse Vectors",
IEEE Trans. on Signal Processing, vol.
56
, no.
10
, pp.
4692

4702
, Oct.
2008
.
Y. C. Eldar , "Compressed Sensing of Analog Signals", submitted to
IEEE Trans. on Signal Processing.
Y. C. Eldar and M. Mishali, "Robust Recovery of Signals from a
Union of Subspaces’’, submitted to IEEE Trans. on Inform. Theory.
Y. C. Eldar, "Uncertainty Relations for Analog Signals",
submitted to
IEEE Trans. Inform. Theory.
Y. C. Eldar and T. Michaeli, "Beyond Bandlimited Sampling:
Nonlinearities, Smoothness and Sparsity", to appear in IEEE Signal
Proc. Magazine.
Comments 0
Log in to post a comment