Sensing of Analog Signals

concretecakeUrban and Civil

Nov 29, 2013 (3 years and 11 months ago)

114 views

Yonina Eldar


Technion


Israel Institute of Technology


http://www.ee.technion.ac.il/people/YoninaEldar



yonina@ee.technion.ac.il


Beyond Nyquist: Compressed
Sensing of Analog Signals

Dagstuhl Seminar

December, 2008

2

Sampling: “Analog Girl in a Digital World…”


Judy Gorman 99

Digital world

Analog world

Reconstruction

D
2
A

Sampling

A2D

Signal processing

Denoising

Image analysis…

(Interpolation)

3

Compression

Original 2500 KB

100%

Compressed
950
KB

38
%

Compressed 392 KB

15%

Compressed
148
KB

6
%


“Can we not just
directly measure

the part that will not end up being
thrown away ?



Donoho

4

Outline

Compressed sensing


background


From discrete to analog

Goals

Part I : Blind multi
-
band reconstruction

Part II : Analog CS framework

Implementations

Uncertainty relations




Can break the Shannon
-
Nyquist barrier

by exploiting signal structure

5


K
non
-
zero entries



at least
2
K

measurements

CS Setup


y A x

Recovery: brute
-
force, convex optimization, greedy algorithms, …

6

is uniquely determined by


is random

with high probability

Donoho,
2006
and Candès et. al.,
2006

NP
-
hard

Convex and tractable

Greedy algorithms: OMP, FOCUSS, etc.

Donoho,
2006
and Candès et. al.,
2006

Tropp, Elad, Cotter et. al,. Chen et. al., and many others

Brief Introduction to CS

Donoho and Elad,
2003


Uniqueness:


Recovery:

7

Naïve Extension to Analog Domain

Standard CS

Discrete Framework

Analog Domain

Sparsity prior

what is a sparse analog signal ?

Generalized sampling

Finite

dimensional elements

Infinite

sequence

Continuous

signal

Operator


Random

is stable w.h.p

Stability

Randomness


Infinitely many

Need structure

for efficient implementation

Finite program,
well
-
studied

Undefined program

over a continuous signal

Reconstruction

8

Random

is stable w.h.p

Naïve Extension to Analog Domain

Sparsity prior

what is a sparse analog signal ?

Generalized sampling

Finite

dimensional elements

Infinite

sequence

Continuous

signal

Operator


Stability

Randomness


Infinitely many

Need structure

for efficient implementation

Finite program,
well
-
studied

Undefined program

over a continuous signal

Reconstruction

Questions:


1.
What is the definition of analog sparsity ?


2.
How to select a sampling operator ?


3.
Can we introduce stucture in sampling and still
preserve stability ?


4.
How to solve infinite dimensional recovery
problems ?


Standard CS

Discrete Framework

Analog Domain

9

Goals

1.
Concrete analog sparsity model


2.
Reduce sampling rate (to minimal)


3.
Simple recovery algorithms


4.
Practical implementation in hardware


10

Analog Compressed Sensing


A signal with a multiband structure in some basis

no more than
N
bands, max width
B,
bandlimited to

(Mishali and Eldar
2007
)

1.
Each band has an uncountable
number of non
-
zero elements

2.
Band locations lie on an infinite grid

3.
Band locations are unknown in advance

What is the definition of analog sparsity ?

More generally


only

sequences

are non
-
zero
(Eldar 2008)

11

Multiband “Sensing”

bands

Sampling

Reconstruction

Goal: Perfect reconstruction

Next steps:

1.
What is the minimal rate requirement ?

2.
A fully
-
blind system design

Analog

Infinite

Analog

Known band locations (subspace prior):

Minimal
-
rate sampling and reconstruction
(NB)

with known band
locations (Lin and
Vaidyanathan

98
)


Half blind system (
Herley

and Wong
99
,
Venkataramani

and
Bresler

00
)

We are interested in unknown spectral support (a union of subspace prior)

(Mishali and Eldar 2007)

12

Rate Requirements

Average sampling rate

Theorem (non
-
blind recovery)

Landau (
1967
)

Theorem (blind recovery)

Mishali and Eldar (2007)

1.
The minimal rate is doubled

2.
For , the rate requirement is samples/sec (on average)

13

Sampling

Analog signal

In each block of samples, only are kept, as described by

Point
-
wise samples

0

2

3

0

0

2

2

3

3

Multi
-
Coset: Periodic Non
-
uniform on the Nyquist grid

14

The Sampler

DTFT

of sampling
sequences

Constant


matrix

known


in vector form


unknowns


 
 
 
 
 
 
 
 
 
 
 
Length
.

known

Problems:


1.
Undetermined system


non unique solution



2.
Continuous set of linear systems



is jointly sparse and unique under appropriate

parameter selection ( )


is sparse

Observation:


15

Paradigm

Solve finite

problem

Reconstruct


 
 
 
 
 
 
 
 
 
 
 
0

1

2

3

4

5

6

S
= non
-
zero rows

16

CTF block

Solve finite

problem

Reconstruct

MMV

Continuous to Finite

Continuous

Finite


span a finite space

Any basis preserves the sparsity

17

2
-
Words on Solving MMV

Variety of methods based on optimizing mixed column
-
row norms

We prove equivalence results by extending RIP and coherence to allow for
structured sparsity
(Mishali and Eldar, Eldar and Bolcskei)

New approach: ReMBo


Reduce MMV and Boost

Main idea: Merge columns of
V

to obtain a single vector problem
y=Aa

Sparsity pattern of a is equal to that of
U

Can boost performance by repeating the merging with different coeff.

Find a matrix
U

that has as few non
-
zero rows as possible

18

Algorithm






CTF

Continuous
-
to
-
finite block: Compressed sensing for analog signals

Perfect reconstruction at minimal rate

Blind system: band locations are unknown

Can be applied to CS of general analog signals

Works with other sampling techniques

19

Framework: Analog Compressed Sensing

Sampling signals from a union of shift
-
invariant spaces (SI)

generator
s

Subspace

(Eldar
2008
)

20

Framework: Analog Compressed Sensing

What happen if only K<<N sequences are not zero ?

There is no prior knowledge on the exact indices in the sum

Not a subspace !

Only k sequences are non
-
zero

21

Framework: Analog Compressed Sensing

Only k sequences are non
-
zero

CTF

Step
1
: Compress the sampling sequences

Step
2
: “Push” all operators to analog domain

System A

High sampling rate = m/T

Post
-
compression

22

Framework: Analog Compressed Sensing

CTF

Eldar (
2008
)

Theorem

System B

Low sampling rate = p/T

Pre
-
compression

23

Simulations

0
5
10
15
20
25
1
2
3
4
5
x 10
4
Time (nano secs)
















-50
0
50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
n







0
5
10
15
20
25
1
2
3
4
5
x 10
4
Time (nano secs)

















Signal

Reconstruction
filter

Output

Time (nSecs)

Time (nSecs)

Amplitude

Amplitude

24

Simulations

Brute
-
Force

M
-
OMP

5
10
15
0
0.2
0.4
0.6
0.8
1
r
Empirical success rate


SBR4
SBR2
5
10
15
0
0.2
0.4
0.6
0.8
1
r
Empirical success rate


SBR4
SBR2
Sampling rate

Sampling rate

Minimal rate

Minimal rate

25

Simulations

SBR
4

SBR
2

r
SNR


5
10
15
10
15
20
0.2
0.4
0.6
0.8
1
r
SNR


5
10
15
10
15
20
0.2
0.4
0.6
0.8
1
r
SNR


5
10
15
10
15
20
50
100
150
200
250
0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
0.8
1
Empirical recovery rate

Sampling rate

Sampling rate

0
% Recovery

100
% Recovery

0
% Recovery

100
% Recovery

Noise
-
free

26

Multi
-
Coset Limitations

0

2

3

0

0

2

2

3

3

Analog signal

Point
-
wise samples

ADC

@ rate

Delay

1.
Impossible to match rate for wideband RF signals

(Nyquist rate >
200
MHz)

2.
Resource waste for IF signals

3
. Requires accurate

time delays

27

Efficient Sampling

Use CTF

Efficient

implementation

(Mishali, Eldar, Tropp
2008
)

28

Hardware Implementation

A few first steps…

29

Pairs Of Bases

Until now: sparsity in a single basis

Can we have a sparse representation in two bases?

Motivation: A combination of bases can sometimes better represent
the signal





Both and are small!

30

Uncertainty Relations

How sparse can be in each basis?






Finite setting: vector in








Elad and Brukstein
2002



Uncertainty


relation

Different bases

31

Analog Uncertainty Principle

Eldar (
2008
)

Theorem

Eldar (
2008
)

Theorem

32

Bases With Minimal Coherence

In the DFT domain

Spikes

Fourier

What are the analog counterparts ?


Constant magnitude


Modulation


“Single” component


Shifts

33

Analog Setting: Bandlimited Signals


Minimal coherence:








Tightness:

34

Finding Sparse Representations

Given a dictionary ,


expand using as few elements as possible:



minimize


Solution is possible using CTF if is small enough


Basic idea:




Sample with basis


Obtain an IMV model:



maximal value



Apply CTF to recover


Can establish equivalence with as long as is small enough

35

Conclusion

Extend the basic results of CS to the analog setting
-

CTF

Sample analog signals at rates much lower than Nyquist

Can find a sparse analog representation

Can be implemented efficiently in hardware

Questions:



Other models of analog sparsity?


Other sampling devices?










Compressed Sensing of Analog Signals


36

Some Things Should Remain At The
Nyquist Rate

Thank you

Thank you

High
-
rate

37

References

M. Mishali and Y. C. Eldar, "Blind Multi
-
Band Signal Reconstruction:
Compressed Sensing for Analog Signals,“ to appear in IEEE Trans. on
Signal Processing.

M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary
Sets of Jointly Sparse Vectors",
IEEE Trans. on Signal Processing, vol.
56
, no.
10
, pp.
4692
-
4702
, Oct.
2008
.

Y. C. Eldar , "Compressed Sensing of Analog Signals", submitted to
IEEE Trans. on Signal Processing.

Y. C. Eldar and M. Mishali, "Robust Recovery of Signals from a
Union of Subspaces’’, submitted to IEEE Trans. on Inform. Theory.

Y. C. Eldar, "Uncertainty Relations for Analog Signals",


submitted to
IEEE Trans. Inform. Theory.


Y. C. Eldar and T. Michaeli, "Beyond Bandlimited Sampling:
Nonlinearities, Smoothness and Sparsity", to appear in IEEE Signal
Proc. Magazine.