AGC
DSP
Professor A G Constantinides©
1
Adaptive Signal Processing
Problem
: Equalise through a FIR filter the distorting
effect of a communication channel that may be
changing with time.
If the channel were
fixed
then a possible solution
could be based on the
Wiener filter
approach
We need to know in such case the
correlation matrix
of the transmitted signal and the
cross correlation
vector between the input and desired response.
When the the filter is operating in an unknown
environment these required quantities need to be
found from the
accumulated data
.
AGC
DSP
Professor A G Constantinides©
2
Adaptive Signal Processing
The problem is particularly acute when not
only the
environment is changing
but also the
data involved are
non

stationary
In such cases we need temporally
to follow
the behaviour of the signals, and
adapt
the
correlation parameters as the environment is
changing.
This would essentially produce a
temporally
adaptive filter
.
AGC
DSP
Professor A G Constantinides©
3
Adaptive Signal Processing
A possible framework is:
]
[
n
d
]
[
ˆ
n
d
]}
[
{
n
x
]
[
n
e
w
:
Filter
Adaptive
Algorithm
AGC
DSP
Professor A G Constantinides©
4
Adaptive Signal Processing
Applications are many
Digital Communications
Channel Equalisation
Adaptive noise cancellation
Adaptive echo cancellation
System identification
Smart antenna systems
Blind system equalisation
And many, many others
AGC
DSP
Professor A G Constantinides©
5
Applications
AGC
DSP
Professor A G Constantinides©
6
Adaptive Signal Processing
Echo Cancellers in Local Loops

+

+
Rx1
Rx2
Tx1
Rx2
Echo canceller
Echo canceller
Adaptive Algorithm
Adaptive Algorithm
Hybrid
Hybrid
Local Loop
AGC
DSP
Professor A G Constantinides©
7
Adaptive Signal Processing
Adaptive Noise Canceller
Noise
Signal +Noise

+
FIR filter
Adaptive Algorithm
PRIMARY SIGNAL
REFERENCE SIGNAL
AGC
DSP
Professor A G Constantinides©
8
Adaptive Signal Processing
System Identification
Unknown System
Signal

+
FIR filter
Adaptive Algorithm
AGC
DSP
Professor A G Constantinides©
9
Adaptive Signal Processing
System Equalisation
Unknown System
Signal

+
FIR filter
Adaptive Algorithm
Delay
AGC
DSP
Professor A G Constantinides©
10
Adaptive Signal Processing
Adaptive Predictors
Signal

+
FIR filter
Adaptive Algorithm
Delay
AGC
DSP
Professor A G Constantinides©
11
Adaptive Signal Processing
Adaptive Arrays
Linear Combiner
Interference
AGC
DSP
Professor A G Constantinides©
12
Adaptive Signal Processing
Basic principles:
1) Form an objective function (performance
criterion)
2) Find gradient of objective function with
respect to FIR filter weights
3) There are several different approaches
that can be used at this point
3) Form a differential/difference equation
from the gradient.
AGC
DSP
Professor A G Constantinides©
13
Adaptive Signal Processing
Let the desired signal be
The input signal
The output
Now form the vectors
So that
]
[
n
d
]
[
n
x
]
[
n
y
T
m
n
x
n
x
n
x
n
]
1
[
.
]
1
[
]
[
]
[
x
T
m
h
h
h
]
1
[
.
]
1
[
]
0
[
h
h
x
T
n
n
y
]
[
]
[
AGC
DSP
Professor A G Constantinides©
14
Adaptive Signal Processing
The form the objective function
where
}
]
[
]
[
{
)
(
2
n
y
n
d
E
J
w
Rh
h
p
h
h
p
w
T
T
T
d
J
2
)
(
}
]
[
]
[
{
T
n
n
E
x
x
R
]}
[
]
[
{
n
d
n
E
x
p
AGC
DSP
Professor A G Constantinides©
15
Adaptive Signal Processing
We wish to minimise this function at the
instant
n
Using
Steepest Descent
we write
But
]
[
])
[
(
2
1
]
[
]
1
[
n
n
J
n
n
h
h
h
h
Rh
p
h
h
2
2
)
(
J
AGC
DSP
Professor A G Constantinides©
16
Adaptive Signal Processing
So that the
“weights update equation”
Since the objective function is quadratic this
expression will converge in
m
steps
The equation is not practical
If we knew and a priori we could find
the required solution (Wiener) as
])
[
(
]
[
]
1
[
n
n
n
Rh
p
h
h
p
R
p
R
h
1
opt
AGC
DSP
Professor A G Constantinides©
17
Adaptive Signal Processing
However these matrices are not known
Approximate expressions are obtained by
ignoring the expectations in the earlier
complete forms
This is very crude. However, because the
update equation accumulates such quantities,
progressive we expect the crude form to
improve
T
n
n
n
]
[
]
[
]
[
ˆ
x
x
R
]
[
]
[
]
[
ˆ
n
d
n
n
x
p
AGC
DSP
Professor A G Constantinides©
18
The LMS Algorithm
Thus we have
Where the error is
And hence can write
This is sometimes called
the stochastic
gradient
descent
])
[
]
[
]
[
](
[
]
[
]
1
[
n
n
n
d
n
n
n
T
h
x
x
h
h
])
[
]
[
(
])
[
]
[
]
[
(
]
[
n
y
n
d
n
n
n
d
n
e
T
h
x
]
[
]
[
]
[
]
1
[
n
e
n
n
n
x
h
h
AGC
DSP
Professor A G Constantinides©
19
Convergence
The parameter is the
step size
, and it
should be selected carefully
If too small it takes too long to
converge, if too large it can lead to
instability
Write the autocorrelation matrix in the
eigen factorisation form
ΛQ
Q
R
T
AGC
DSP
Professor A G Constantinides©
20
Convergence
Where is orthogonal and is
diagonal containing the eigenvalues
The error in the weights with respect to
their optimal values is given by (using
the Wiener solution for
We obtain
Q
Λ
])
[
(
]
[
]
1
[
n
n
n
opt
opt
opt
Rh
Rh
h
h
h
h
p
]
[
]
[
]
1
[
n
n
n
h
h
h
Re
e
e
AGC
DSP
Professor A G Constantinides©
21
Convergence
Or equivalently
I.e.
Thus we have
Form a new variable
]
[
)
1
(
]
1
[
n
n
h
h
e
ΛQ
Q
e
T
]
[
)
(
]
[
)
1
(
]
1
[
n
n
n
h
h
h
e
ΛQ
QQ
Q
e
ΛQ
Q
Q
Qe
T
T
]
[
)
1
(
]
1
[
n
n
h
h
Qe
Λ
Qe
]
[
]
[
n
n
h
Qe
v
AGC
DSP
Professor A G Constantinides©
22
Convergence
So that
Thus each element of this new variable is
dependent on the previous value of it via a
scaling constant
The equation will therefore have an
exponential form in the time domain, and the
largest coefficient in the right hand side will
dominate
]
[
)
1
(
]
1
[
n
n
v
Λ
v
AGC
DSP
Professor A G Constantinides©
23
Convergence
We require that
Or
In practice we take a much smaller
value than this
1
1
max
max
2
0
AGC
DSP
Professor A G Constantinides©
24
Estimates
Then it can be seen that as the
weight update equation yields
And on taking expectations of both sides of it
we have
Or
n
]}
[
{
]}
1
[
{
n
E
n
E
h
h
])}
[
]
[
]
[
](
[
{
]}
[
{
]}
1
[
{
n
n
n
d
n
E
n
E
n
E
T
h
x
x
h
h
])}
[
]
[
]
[
]
[
]
[
{(
0
n
n
n
n
d
n
E
T
h
x
x
x
AGC
DSP
Professor A G Constantinides©
25
Limiting forms
This indicates that the solution
ultimately tends to the Wiener form
I.e. the estimate is unbiased
AGC
DSP
Professor A G Constantinides©
26
Misadjustment
The excess mean square error in the
objective function due to gradient noise
Assume uncorrelatedness set
Where is the variance of desired
response and is zero when uncorrelated.
Then misadjustment is defined as
opt
T
d
J
h
p
2
min
2
d
opt
h
min
min
/
)
)
(
(
J
J
J
J
LMS
XS
AGC
DSP
Professor A G Constantinides©
27
Misadjustment
It can be shown that the misadjustment
is given by
m
i
i
i
XS
J
J
1
min
1
/
AGC
DSP
Professor A G Constantinides©
28
Normalised LMS
To make the step size respond to the
signal needs
In this case
And misadjustment is proportional to
the step size.
]
[
]
[
]
[
1
2
]
[
]
1
[
2
n
e
n
n
n
n
x
x
h
h
1
0
AGC
DSP
Professor A G Constantinides©
29
Transform based LMS
]
[
n
d
]
[
ˆ
n
d
]}
[
{
n
x
]
[
n
e
w
:
Filter
Adaptive
Algorithm
Transform
Inverse Transform
AGC
DSP
Professor A G Constantinides©
30
Least Squares Adaptive
with
We have the Least Squares solution
However, this is computationally very
intensive to implement.
Alternative forms make use of recursive
estimates of the matrices involved.
n
i
T
i
i
n
1
]
[
]
[
]
[
x
x
R
n
i
n
d
n
n
1
]
[
]
[
]
[
x
p
]
[
]
[
]
[
1
n
n
n
p
R
h
AGC
DSP
Professor A G Constantinides©
31
Recursive Least Squares
Firstly we note that
We now use the Inversion Lemma (or the
Sherman

Morrison formula)
Let
]
[
]
[
]
1
[
]
[
n
d
n
n
n
x
p
p
T
n
n
n
n
]
[
]
[
]
1
[
]
[
x
x
R
R
AGC
DSP
Professor A G Constantinides©
32
Recursive Least Squares (RLS)
Let
Then
The quantity is known as the
Kalman
gain
]
[
]
1
[
]
[
1
]
[
]
1
[
]
[
1
1
n
n
n
n
n
n
T
x
R
x
x
R
k
1
]
[
]
[
n
n
R
P
]
1
[
]
[
]
[
]
1
[
]
[
n
n
n
n
n
T
P
x
k
R
P
]
[
n
k
AGC
DSP
Professor A G Constantinides©
33
Recursive Least Squares
Now use in the computation of
the filter weights
From the earlier expression for updates we
have
And hence
]
[
]
[
]
[
n
n
n
x
P
k
])
[
]
[
]
1
[
](
[
]
[
]
[
]
[
n
d
n
n
n
n
n
n
x
p
P
p
P
h
]
[
n
P
]
1
[
]
1
[
]
[
]
[
]
1
[
]
1
[
]
1
[
]
[
n
n
n
n
n
n
n
n
T
p
P
x
k
p
P
p
P
])
1
[
]
[
]
[
](
[
]
1
[
]
[
n
n
n
d
n
n
n
T
h
x
k
h
h
AGC
DSP
Professor A G Constantinides©
34
Kalman Filters
Kalman filter is a sequential estimation
problem normally derived from
The Bayes approach
The Innovations approach
Essentially they lead to the same equations
as RLS, but underlying assumptions are
different
AGC
DSP
Professor A G Constantinides©
35
Kalman Filters
The problem is normally stated as:
Given a sequence of noisy observations to
estimate the sequence of state vectors of a linear
system driven by noise.
Standard formulation
]
[
]
[
]
1
[
n
n
n
w
Ax
x
]
[
]
[
]
[
]
[
n
n
n
n
ν
x
C
y
AGC
DSP
Professor A G Constantinides©
36
Kalman Filters
Kalman filters may be seen as RLS with the
following correspondence
Sate space
RLS
Sate

Update matrix
Sate

noise variance
Observation matrix
Observations
State estimate
]
[
n
A
]
[
n
x
T
n
]
[
x
]
[
n
C
]
[
n
y
]
[
n
h
}
]
[
]
[
{
]
[
T
n
n
E
n
w
w
Q
I
0
]
[
n
d
AGC
DSP
Professor A G Constantinides©
37
Cholesky Factorisation
In situations where storage and to some
extend computational demand is at a
premium one can use the Cholesky
factorisation tecchnique for a positive definite
matrix
Express , where is lower
triangular
There are many techniques for determining
the factorisation
T
LL
R
L
Comments 0
Log in to post a comment