Adaptive Signal Processing

pancakesbootΤεχνίτη Νοημοσύνη και Ρομποτική

24 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

63 εμφανίσεις

AGC

DSP

Professor A G Constantinides©

1

Adaptive Signal Processing


Problem
: Equalise through a FIR filter the distorting
effect of a communication channel that may be
changing with time.


If the channel were
fixed

then a possible solution
could be based on the
Wiener filter

approach


We need to know in such case the
correlation matrix

of the transmitted signal and the
cross correlation

vector between the input and desired response.


When the the filter is operating in an unknown
environment these required quantities need to be
found from the
accumulated data
.

AGC

DSP

Professor A G Constantinides©

2

Adaptive Signal Processing


The problem is particularly acute when not
only the
environment is changing

but also the
data involved are
non
-
stationary


In such cases we need temporally
to follow

the behaviour of the signals, and
adapt

the
correlation parameters as the environment is
changing.


This would essentially produce a
temporally
adaptive filter
.

AGC

DSP

Professor A G Constantinides©

3

Adaptive Signal Processing


A possible framework is:


]
[
n
d
]
[
ˆ
n
d
]}
[
{
n
x
]
[
n
e
w
:
Filter
Adaptive
Algorithm

AGC

DSP

Professor A G Constantinides©

4

Adaptive Signal Processing


Applications are many


Digital Communications


Channel Equalisation


Adaptive noise cancellation


Adaptive echo cancellation


System identification


Smart antenna systems


Blind system equalisation


And many, many others

AGC

DSP

Professor A G Constantinides©

5

Applications



AGC

DSP

Professor A G Constantinides©

6

Adaptive Signal Processing


Echo Cancellers in Local Loops


-

+

-

+

Rx1

Rx2

Tx1

Rx2

Echo canceller

Echo canceller

Adaptive Algorithm

Adaptive Algorithm

Hybrid

Hybrid

Local Loop

AGC

DSP

Professor A G Constantinides©

7

Adaptive Signal Processing


Adaptive Noise Canceller


Noise

Signal +Noise

-

+

FIR filter

Adaptive Algorithm

PRIMARY SIGNAL

REFERENCE SIGNAL

AGC

DSP

Professor A G Constantinides©

8

Adaptive Signal Processing


System Identification

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm

AGC

DSP

Professor A G Constantinides©

9

Adaptive Signal Processing


System Equalisation

Unknown System

Signal

-

+

FIR filter

Adaptive Algorithm

Delay

AGC

DSP

Professor A G Constantinides©

10

Adaptive Signal Processing


Adaptive Predictors

Signal

-

+

FIR filter

Adaptive Algorithm

Delay

AGC

DSP

Professor A G Constantinides©

11

Adaptive Signal Processing


Adaptive Arrays

Linear Combiner

Interference

AGC

DSP

Professor A G Constantinides©

12

Adaptive Signal Processing


Basic principles:


1) Form an objective function (performance
criterion)


2) Find gradient of objective function with
respect to FIR filter weights


3) There are several different approaches
that can be used at this point


3) Form a differential/difference equation
from the gradient.


AGC

DSP

Professor A G Constantinides©

13

Adaptive Signal Processing


Let the desired signal be


The input signal


The output


Now form the vectors





So that




]
[
n
d
]
[
n
x
]
[
n
y


T
m
n
x
n
x
n
x
n
]
1
[
.
]
1
[
]
[
]
[




x


T
m
h
h
h
]
1
[
.
]
1
[
]
0
[


h
h
x
T
n
n
y
]
[
]
[

AGC

DSP

Professor A G Constantinides©

14

Adaptive Signal Processing


The form the objective function





where




}
]
[
]
[
{
)
(
2
n
y
n
d
E
J


w
Rh
h
p
h
h
p
w
T
T
T
d
J




2
)
(

}
]
[
]
[
{
T
n
n
E
x
x
R

]}
[
]
[
{
n
d
n
E
x
p

AGC

DSP

Professor A G Constantinides©

15

Adaptive Signal Processing


We wish to minimise this function at the
instant
n


Using
Steepest Descent

we write




But

]
[
])
[
(
2
1
]
[
]
1
[
n
n
J
n
n
h
h
h
h






Rh
p
h
h
2
2
)
(





J
AGC

DSP

Professor A G Constantinides©

16

Adaptive Signal Processing


So that the
“weights update equation”




Since the objective function is quadratic this
expression will converge in
m
steps


The equation is not practical


If we knew and a priori we could find
the required solution (Wiener) as

])
[
(
]
[
]
1
[
n
n
n
Rh
p
h
h





p
R
p
R
h
1


opt
AGC

DSP

Professor A G Constantinides©

17

Adaptive Signal Processing


However these matrices are not known


Approximate expressions are obtained by
ignoring the expectations in the earlier
complete forms




This is very crude. However, because the
update equation accumulates such quantities,
progressive we expect the crude form to
improve


T
n
n
n
]
[
]
[
]
[
ˆ
x
x
R

]
[
]
[
]
[
ˆ
n
d
n
n
x
p

AGC

DSP

Professor A G Constantinides©

18

The LMS Algorithm


Thus we have



Where the error is



And hence can write



This is sometimes called
the stochastic
gradient

descent

])
[
]
[
]
[
](
[
]
[
]
1
[
n
n
n
d
n
n
n
T
h
x
x
h
h





])
[
]
[
(
])
[
]
[
]
[
(
]
[
n
y
n
d
n
n
n
d
n
e
T




h
x
]
[
]
[
]
[
]
1
[
n
e
n
n
n
x
h
h




AGC

DSP

Professor A G Constantinides©

19

Convergence

The parameter is the
step size
, and it
should be selected carefully


If too small it takes too long to
converge, if too large it can lead to
instability


Write the autocorrelation matrix in the
eigen factorisation form



ΛQ
Q
R
T

AGC

DSP

Professor A G Constantinides©

20

Convergence


Where is orthogonal and is
diagonal containing the eigenvalues


The error in the weights with respect to
their optimal values is given by (using
the Wiener solution for



We obtain

Q
Λ
])
[
(
]
[
]
1
[
n
n
n
opt
opt
opt
Rh
Rh
h
h
h
h







p
]
[
]
[
]
1
[
n
n
n
h
h
h
Re
e
e




AGC

DSP

Professor A G Constantinides©

21

Convergence


Or equivalently



I.e.




Thus we have




Form a new variable

]
[
)
1
(
]
1
[
n
n
h
h
e
ΛQ
Q
e
T




]
[
)
(
]
[
)
1
(
]
1
[
n
n
n
h
h
h
e
ΛQ
QQ
Q
e
ΛQ
Q
Q
Qe
T
T







]
[
)
1
(
]
1
[
n
n
h
h
Qe
Λ
Qe




]
[
]
[
n
n
h
Qe
v

AGC

DSP

Professor A G Constantinides©

22

Convergence


So that



Thus each element of this new variable is
dependent on the previous value of it via a
scaling constant


The equation will therefore have an
exponential form in the time domain, and the
largest coefficient in the right hand side will
dominate

]
[
)
1
(
]
1
[
n
n
v
Λ
v




AGC

DSP

Professor A G Constantinides©

23

Convergence


We require that



Or



In practice we take a much smaller
value than this


1
1
max



max
2
0




AGC

DSP

Professor A G Constantinides©

24

Estimates


Then it can be seen that as the
weight update equation yields





And on taking expectations of both sides of it
we have



Or




n
]}
[
{
]}
1
[
{
n
E
n
E
h
h


])}
[
]
[
]
[
](
[
{
]}
[
{
]}
1
[
{
n
n
n
d
n
E
n
E
n
E
T
h
x
x
h
h





])}
[
]
[
]
[
]
[
]
[
{(
0
n
n
n
n
d
n
E
T
h
x
x
x



AGC

DSP

Professor A G Constantinides©

25

Limiting forms


This indicates that the solution
ultimately tends to the Wiener form


I.e. the estimate is unbiased

AGC

DSP

Professor A G Constantinides©

26

Misadjustment


The excess mean square error in the
objective function due to gradient noise


Assume uncorrelatedness set



Where is the variance of desired
response and is zero when uncorrelated.


Then misadjustment is defined as


opt
T
d
J
h
p


2
min

2
d

opt
h
min
min
/
)
)
(
(
J
J
J
J
LMS
XS



AGC

DSP

Professor A G Constantinides©

27

Misadjustment


It can be shown that the misadjustment
is given by






m
i
i
i
XS
J
J
1
min
1
/


AGC

DSP

Professor A G Constantinides©

28

Normalised LMS


To make the step size respond to the
signal needs




In this case



And misadjustment is proportional to
the step size.

]
[
]
[
]
[
1
2
]
[
]
1
[
2
n
e
n
n
n
n
x
x
h
h





1
0



AGC

DSP

Professor A G Constantinides©

29

Transform based LMS

]
[
n
d
]
[
ˆ
n
d
]}
[
{
n
x
]
[
n
e
w
:
Filter
Adaptive
Algorithm

Transform

Inverse Transform

AGC

DSP

Professor A G Constantinides©

30

Least Squares Adaptive



with




We have the Least Squares solution



However, this is computationally very
intensive to implement.


Alternative forms make use of recursive
estimates of the matrices involved.




n
i
T
i
i
n
1
]
[
]
[
]
[
x
x
R



n
i
n
d
n
n
1
]
[
]
[
]
[
x
p
]
[
]
[
]
[
1
n
n
n
p
R
h


AGC

DSP

Professor A G Constantinides©

31

Recursive Least Squares


Firstly we note that





We now use the Inversion Lemma (or the
Sherman
-
Morrison formula)


Let


]
[
]
[
]
1
[
]
[
n
d
n
n
n
x
p
p



T
n
n
n
n
]
[
]
[
]
1
[
]
[
x
x
R
R



AGC

DSP

Professor A G Constantinides©

32

Recursive Least Squares (RLS)


Let




Then




The quantity is known as the
Kalman
gain

]
[
]
1
[
]
[
1
]
[
]
1
[
]
[
1
1
n
n
n
n
n
n
T
x
R
x
x
R
k






1
]
[
]
[


n
n
R
P
]
1
[
]
[
]
[
]
1
[
]
[




n
n
n
n
n
T
P
x
k
R
P
]
[
n
k
AGC

DSP

Professor A G Constantinides©

33

Recursive Least Squares


Now use in the computation of
the filter weights




From the earlier expression for updates we
have




And hence





]
[
]
[
]
[
n
n
n
x
P
k

])
[
]
[
]
1
[
](
[
]
[
]
[
]
[
n
d
n
n
n
n
n
n
x
p
P
p
P
h




]
[
n
P
]
1
[
]
1
[
]
[
]
[
]
1
[
]
1
[
]
1
[
]
[







n
n
n
n
n
n
n
n
T
p
P
x
k
p
P
p
P
])
1
[
]
[
]
[
](
[
]
1
[
]
[





n
n
n
d
n
n
n
T
h
x
k
h
h
AGC

DSP

Professor A G Constantinides©

34

Kalman Filters


Kalman filter is a sequential estimation
problem normally derived from



The Bayes approach


The Innovations approach


Essentially they lead to the same equations
as RLS, but underlying assumptions are
different

AGC

DSP

Professor A G Constantinides©

35

Kalman Filters


The problem is normally stated as:


Given a sequence of noisy observations to
estimate the sequence of state vectors of a linear
system driven by noise.


Standard formulation


]
[
]
[
]
1
[
n
n
n
w
Ax
x



]
[
]
[
]
[
]
[
n
n
n
n
ν
x
C
y


AGC

DSP

Professor A G Constantinides©

36

Kalman Filters


Kalman filters may be seen as RLS with the
following correspondence





Sate space


RLS


Sate
-
Update matrix




Sate
-
noise variance




Observation matrix






Observations





State estimate



]
[
n
A
]
[
n
x
T
n
]
[
x
]
[
n
C
]
[
n
y
]
[
n
h
}
]
[
]
[
{
]
[
T
n
n
E
n
w
w
Q

I
0
]
[
n
d
AGC

DSP

Professor A G Constantinides©

37

Cholesky Factorisation


In situations where storage and to some
extend computational demand is at a
premium one can use the Cholesky
factorisation tecchnique for a positive definite
matrix


Express , where is lower
triangular


There are many techniques for determining
the factorisation


T
LL
R

L