1
EEL 6502 Adaptive Signal Processing
Project 1: Adaptive
noise cancellation
By
Meena Ramani
2
This report
analyzes the design and performance of an adap
tive noise canceller using nlms.
1)
Abstract
2)
Introduction
a.
Adaptive noise cancellation
b.
Wiener Filter
c.
LMS
d.
NLMS
3)
Analysis of
input
data
Primary and Secondary Inputs
a.
Time domain
b.
Frequency domain
4)
Results for a 2

tap filter
a.
Performance surface contours
b.
Weight Tracks
c.
Learning Curve
d.
Frequency Response
e.
Cleaned output
&
SNR improvement
5)
Analysis o
f
optimal step size and filter order
a.
Variation of step size (fixed order)
b.
Variation of order (fixed step size)
c.
Misadjustment vs. step size
6)
Results for a higher order
filters
a.
Weight Tracks
b.
Frequency Response
c.
Speed and Step S
ize
d.
Misadjustment
e.
Cleaned output &
SNR improvement
7)
Conclusion
3
Abstract
Adaptive systems are different fundamentally from other signal processing systems in that they require a
minimum
a priori
knowledge of the signals or systems under analysis. The
objective of any linear adaptive
system is to obtain, using some iterative process, an optimum linear estimator of a system.
There are four main types of adaptive filter configurations: Adaptive system identification, Adaptive noise
cancellation,
A
daptive
linear prediction and
A
daptive inverse systems. While they implement the same
algorithm
,
their system configuration differs.
Adaptive noise cancellation (ANC) is often used to extract the
desired speech from the given noisy speech. ANC plays an important
role in digital voice communication
systems For Eg: Cell phones require adaptive noise cancellation to reduce further degradation of vocoded
speech.
We will perform ANC using nlms in order to clean the noisy speech signal. The effects on stability,
conver
gence, speed and computation on choosing the different parameters for nlms is studied here and in
the end we will decide on a system which has the best tradeoffs.
4
2)
Introduction
2.
a)
Adaptive noise cancellation
ANC deals with the enhancement of
noise corrupted signals. Its biggest strength when compared to other
enhancement techniques is that no a priori information about either the signal or noise is required. This
advantage comes at the cost that we need in addition to the primary signal
d(n)
which is Signal corrupted by
noise a reference/
secondary signal
x(n)
which contains noise
1
( )
v n
correlated with the noise component
0
( )
v n
in the primary signal.
The principle here is to adaptively filter the ref
erence signal to get a replica of
the noise in the primary and subtract it from the
p
rimary.
Here the system output is the error signal.
Now we can see that:
0
( ) ( ) ( ) ( )
e n s n v n y n
Squaring and taking the expectation op
eration on both sides gives us:
2
2 2
0 0
{ ( )} { ( )} {( ( ) ( )) } 2 { ( )( ( ) ( ))}
E e n E s n E v n y n E s n v n y n
Assuming that s(n) is uncorrelated with
0
( )
v n
and
1
( )
v n
we get that
s(n)
is also uncorrelated with
y(n)
since
y(n)=
x(n)*w(n).
Thus the last term above b
ecomes zero and we get:
2 2 2
0
{ ( )} { ( )} {( ( ) ( )) }
E e n E s n E v n y n
Now to minimize the error we cannot reduce
s(n)
and hence we get the MSE minimized when
0
( ) ( )
y n v n
This implies
that
( ) ( )
e n s n
, the desired result.
Also if we assume
0 1
( ) ( )* ( )
v n v n h n
, where the transmission path between
0
( )
v n
and
1
( )
v n
has been
modeled by
h(n)
, we can conclude that the adaptive filter models the unknown transmission path
h(n)
.
0
( ) ( ) ( )
d n s n v n
1
( ) ( )
x n v n
( )
y n
( )
e n
w
+
5
Limitations of ANC:
We
assume the noise components are correlated if they are uncorrelated then the speech will further be
degraded as there is another additive noise component.
Also there might be leakage or cross

talk and the reference signal may contain a correlated componen
t of
Speech. This will result in the adaptive filter canceling speech and will result in distortion.
2.
b)
Wiener Filter
The solution of the
Normal equations
:
1
0
( ) ( ) ( )
L
k
P i R i k w k
i
=
0,1,…
L

1
Where
L
is the order of the filter yields the optimal
solution
.
The Wiener

Ho
p
f filter is an iterative solution
for
stationary normal equations
. It
is a linear filter which minimizes the mean

square estimation error. For an
FIR filter of order M the output
y(n)
is given in terms of its inputs and
weights (im
pulse response) as
1
0
( ) ( ) ( )
L
k
y n x n k w k
.
Let
d(n)
be the desired signal. The Wiener

Hopf equations minimize the MSE
J given by:
2
2
{ ( )} ( ) ( )
J E e n E d n y n
a
nd
provide
s
us with M optimum values of
w
which minimize the MSE
i.e.
1
*
w p
R
,
where
w
is the weight vector (
L
x 1)
the superscript * is to indicate that it is the optimal solution.
R
is the auto

correlation matrix (
L
x
L
)
p
is a vector of cross correlations between the desired signal & the input (
L
x 1)
For stationary signals we can use time average estimates (biased/unbiased) for the correlation coefficients
(
R
and
p
).
The error
vector J is a second order function of the weights. That is J is a ‘bowl shaped’ function of the
weights. J is also known as the error performance surface. For an L order filter the bowl is actually L

dimensional

a hyperparaboloid.
The minimum value
J
mi
n
occurs at the bottom of this bowl where
w

>w*
6
*
min
2
t t t
J J w w w p w p
R
Where
*
2
min
t
d
J w p
2 2
,{ ( )}
d
E d n
From the above we can observe that
w=w*
is a minimum of J since any change in
w
from
w*
c
an only
increase
J
. Also we have that
min
J
is a monotonically non

increasing function of filter order i.e.
1
min min
L L
J J
2.
c
)
Least Mean Squares

LMS
The problem
s
with the Weiner filter
are as follows:
Calculation of
R
and
p
is very computational
expensive
3
( )
O L
We have to use time average estimates to compute
R
and
p
and this creates problems for non

stationary signals.
We can assume local

statio
narity and then approach the problem using a short
term time average estimate but a better approach would be to use a filter which would track the
changes in the optimal solution.
The Least Mean Squares (LMS) filter does just this.
De
signed
by Bernard Wi
drow and M. Hoff in 1959, this simple algorithm now represents the cornerstone
for the design of adaptive filters
.
The LMS is based on the steepest descent approach where at each iteration
we move, in weight space, in a direction opposite to the gradient o
f the MSE
(
( )
J n
)
and by a distance
proportional to the magnitude of the gradient.
* ( )
( 1) ( )
2
J n
w n w n
, w
here
is a constant of proportionality which we call the
step size
.
In the LMS,
we estimat
e the gradient of the MSE using the gradient of the instantaneous value of squared
error
(
( )
J n
)
. The iterative update equation
thus
becomes
:
( 1) ( ) ( ) ( )
w n w n x n e n
,
where
( )
w n
is
the weights
vector (Lx1)
at time
n
This simple update equation reduces the computational complexity to O(L).
Now by using the
instantaneous estimate for the gradient
( )
J n
instead of
( )
J n
we cannot expect the filter to converge to
the
optimal solution
w*
obtained using the Wiener filter
.
This approximation leads to a “gradient noise”
which
makes the weights computed by LMS to have a random motion about
w*
.
Because of t
he feedback involved in the LMS,
stability and convergence are imp
ortant issues.
The algorithm is proven to converge if the
adaptation step
satisfies:
7
max
2
0
,
where
max
is the biggest eigenvalue of input correlation
matrix
. In practice we
use
2
0
*
L InputPower
When the value of
is in the range above, the
LMS behaves like a low pass filter and filters out the
gradient noise by computing the we
ights slowly.
Thus
for smaller
,
the MSE on an average is smaller.
The
time constant (/speed) for convergence
or the average time constant
is given by:
min
1
2
,
where
min
is the
smal
lest
eigenvalue of input correlation matrix
.
Misadjustment
is defined as t
he ratio of
min
min
( )
J J
M
J
, where
( ) ( )
J n J
as
n
For small
we can write
* [ ]
2
tr R
M
This is a measure of how far the LMS steady state solution is from the Wiener solution
and i
s the price we
pay by using LMS instead of the Wiener filt
er.
Smaller M is from unity the better.
Having a small
reduces Misadjustment as there is less deviation from
w*
.
Now the trace of R depends of the filter
order (
L)
and hence M increases as L increases
. It is also obvious that M
and
are inversely proportional.
We can make the following observations from
the
above
equations
:
The Convergence and stability of the LMS depend upon the input signal and the filter order.
If
( )
x n
is lar
ge then the LMS filter amplifies the gradient noise
T
he speed of convergence
gets very slow if there is a big spread among the eigenvalues of
(a small
eigenvalue
leads to a small correction in the filter weights so that there should be a lot of iterations
to reach the optimal point in that dimension).
8
2.
d)
Normalized
Least Mean Squares

N
LMS
As we saw
above
the Convergence, stability and the steady state prope
rties of the LMS filter depend upon
the input signal and the filter order. To remove this dependence the most obvious change would be to
normalize the step size by these factors:
'
2
* { ( )}
p
L E x n T
In practice we calculate the power
T
p
using a tim
e average
( ) ( )
t
x n x n
Hence we get:
( ) ( )
( 1) ( )
( ) ( )
t
x n e n
w n w n
x n x n
This is the Normalized LMS (NLMS) algorithm. This normalization carries a computational overhead but is
required especially in cases where the input signals fluctuate widely.
The
in the denominator is
a small
positive constant used to avoid dividing by zero.
A point to observe is that the NLMS has a higher and faster rate of convergence when compared to LMS
3)
Analysis of
input
data
T
he
Secondary/No
ise
Input
signal
time and frequency domain characteristics:
As the sampling frequency is 16
Khz the highest frequency signal recoverable is 8000Hz
.
9
The Spectrogram of the noise signal shows the distribution of energy over time and frequency. Ther
e is a
strong noise signal at frequency of 2.8Khz existing throughout the sentence duration.
T
he
Primary/Speech + Noise Input signal
time and frequency domain characteristics:
The frequency domain picture for speech+noise shows how noise has corrupted
the speech
. The spectrum
of primary and the secondary look almost identical!
The spectrogram below shows the presence of some speech signal among the noisy signal
10
4)
Results for a 2

tap filter
We are going to observe the performance of a 2

tap nlms fi
lter.
4.a)
Performance surface contour
As mentioned before, t
he quadratic MSE (
J
) is a
concave
upward hyperparaboloid. Contours of constant
J
or MSE are elliptical as can be seen below. The point at the
bottom of the bowl is the point of minimum
MSE and the weights which result in this are the wiener optimal weights
w*
.
Using the wiener filter the optimal weights got for a filter order 2 w0=

0.3465 and w1=1.2205. Also the
MSE here is:
0.0611
.
Note the pe
rformance of the Wiener filter
(
Excess
MSE)
for non

stationary signals is
worse than for NLMS.
2 dimensional Performance Surface
11
The following is a plot for a filter of order 2. Note that after about
=0.1 the MSE incr
eases with increase
in step size.
For values below 0.1
you can see that as the
is very small
and as the filter order is very low,
no of iterations required for the weights to converge is more than what we have provided (which is
the
length of the signal.) Thus the MSE computed for
<0.1 does not follow the trend of increasing with
increase in step size.
For a step size:
=0.
001
MSE
=
0.0160
For step size
=
0.01
MSE
=
0.0105
12
For a step size:
=0.
1
Using an NLMS filter of order 2 we get an MSE of
0.0105
. Note the weights w0=

0.35
and w1=1.2
W
e can
also
notice that larger the step size
less the stability
or more the movement
a
bout
the
optimal
weights.
At values of
>1 we can notice that the weights are very unstable.
For a step size:
=0.
5
MSE=
0.0106
For step size
=
2
MSE=
0.0113
13
4.b) Weight Tracks
We can notice that as
increases the
variation
of the weights
about the optimal
(w0=

0.35
and w1=1.2
)
increases.
Also notice that when
=
0.
001 the weights haven’t got any time to settle at.
For a step size:
=0.
001
For step size
=
0.
1
For a step size:
=0.
5
For step size
=
2
14
4.c)
Learning Curve
The learning curve which is normally plotted is for the ensemble averaged square estimation error. But here
since we have only one signal the plot is o
f
the time averaged
MSE. In a noise cancellation case the error
s
ignal is Speech so when we plot the learning curve what we get is as follows:
We can plot the residual noise as follows:
10
1
2
0
1
2
0
( ) 10log ( )/( )
( ) ( )
( ) ( )
E X
Q
E
j
Q
X
j
R k P k P k
P k e k j
P k x k j
, w
here Q=512
e(n) is the error signal (Speech)
x(n) is the Primary/de
sired signal (Speech+Noise)
The Residual error has been plotted below for both
Wiener filter and the nlms for step size 0.1 and order 2.
We can see that the Wiener Filter has worse performance which proves that nlms works better than wiener
for non

stati
onary data.
15
16
4.d)
Frequency Response
The
transfer function for the system with input as the primary
d(n)
and output the error
e(n)
is as follows:
( )
( )
( )
E z
H z
D z
Where H(Z) is the transfer fu
nction i.e. the response in Z domain for an unit impulse input at the secondary
x(n)
Thus
for an unit impulse
1
0
( ) ( ) ( )
L
j
j
y k w k x k j
reduces to:
1
1
1
1
.1
2 (.1)
..
2
2 ( )
(1 2 ) 2
k k k
k k k
k k k
k k k k
k k k
y w w
w w e
i e
y y e
y y d y
y y d
Now taking the Z transform we get:
0
( ) ( ) ( )
d n s n v n
1
( ) ( )
x n v n
( )
y n
( )
e n
w
+
Output
Input
17
1
( ) (1 2 ) ( ) 2 ( )
( ( ) ( )) (1 2 ) 2 ( )
( ) 1 2 2 ( ) (1 2 )
( ) ( 1)
,( )
( ) (1 2 )
Y z z Y z D z
D z E z z D z
D z z E z z
E z z
Thus H z
D Z z
Thus we ha
ve a zero on the unit circle and a pole at the (1

2
) value. For
=0.1 the pole is at 0.8 and
hence we have the following pole zero plot:
Also we have the transfer function estimate plot as: We can ob
serve that the filter appears to behave like a
HPF.
18
4.e)
SNR improvement
2
2
( )
10log
( )
d n
SNR
e n
d
(n)
is the desired and
e(n)
is the error signal. This is the reverse of the residual noise measure. A high value
of SNR is not really a good ind
ication of the cleaned speech quality as the speech might be attenuated too.
For order 2 step size=0.1 we have SNR=
3.6522
dB
The following plot shows the variation of SNR with step size. The last
10
00 samples of
d(n)
and
e(n)
were
used here. This is done
to give enough time for most of the step sizes to settle down.
SNR nlms:
7.0550
dB
SNR Wiener:

0.7545
dB
Let us now see the time, frequency and the spectrogram plot of the cleaned output. Here an order 2 step
size 0.1 nlms system was used.
19
5)
Analysis on optimal step size and filter order
The filter order and the step size affect the stability, the convergence and the speed of convergence of the
nlms filter. Hence we have to carefully select these and have a tradeoff between accuracy and c
omputation.
5.a)
Variation of step size (fixed order)
For a Fixed filter order of 2 variation in step size
Increasing the Filter order also increases the Misadjustment. This can be seen from the above plots which
show variation of step sizes for th
ree different filter orders.
In general as the Step size increases the
MSE increases because of increase in
misadjustment. The initial decrease in
MSE is for those small step sizes which
require longer iteration peri
ods than can
be provided.
As mentioned before the
step size of 0.1 gives the best results.
20
5.
b
) Variation of order (fixed step size)
For a Fixed step size of 0.1 effe
ct of variation of order on MSE is as follows:
J
is a monotonically non

increasing function of filter order i.e.
1
L L
J J
.
We can see that the error reduces and remains almost constant at about 80. But the value of MSE=0.006 is
also acceptable hence lets compare the performance of Order=2,25,80 and choose the best among the
three.
5.
c
) Misadjustmen
t vs. step size
Misadjustment
is defined as the ratio of
min
min
( )
J J
M
J
, where
( ) ( )
J n J
as
n
For small
we can write
* [ ]
2
tr R
M
This is a measure
of how far the LMS steady state solution is from the Wiener solution
and is the price we
pay by using LMS instead of the Wiener filter. Smaller M is from unity the better. Having a small
reduces Misadjustment as there is less
deviation from
w*
.
It is also obvious that M and
, the speed of
convergence , are inversely proportional.
21
Let’s first
calculate the autocorrelation of the Input R
XX
using the unbiased estimator for a window of 200.
Then
let’s
ca
lculate M using the above formula
6)
Results for a higher order filters
6.a)
Weight Tracks
25 order filter:
80 order filter:
22
6.b
) Frequency Response
6.c
)
Step Size
The step size,
, determines th
e speed of convergence and the stability of the algorithm.
As
increases
the speed also increases. Thus is short we can say:
Small
implies
small misadjustment in steady

state
slow adaptation/tracking
Large
implies
large misadjustment in steady

state
fast adaptation/tracking
The
time constant (/speed) for convergence
or the average time constant is given by:
min
1
2
,
where
min
is
the
smallest
eigenvalue of input correlation matrix
.
The convergence rate determines the rate at which
the filter settles down at its optimal weights. A faster
convergence is a good feature for an adaptive filter. But Speed of convergence and misadjustment are
inversely proportional. Thus while a large step size will cause large misadjustment it will cause the speed of
convergence to increase but the stability wi
ll decrease if it exceeds
2
0
*
L InputPower
And then the system is more likely to diverge as was seen in the previous plots.
23
6.c
) SNR improvement
2
2
( )
10log
( )
d n
SNR
e n
The following plot shows the variation of order and SNR improvemen
t
. This is because increase in order
also increases Misadjustment as the Size of R increases
The following table has the values of SNR for the 3 orders under consideration
Order
SNR(dB)
2
7.0550
25
13.2881
80
13.0065
6.d
)
Misadjustment
vs Order
The following plot shows the variation in M with L.
The MSE decreases with order but the M increases
with order increase. T
his is because
min
J
is a monotonically non

increasing function of filter order i.e.
1
min min
L L
J J
24
The following table shows the value of M for the three orders under consideration.
Order
M
2
0.0042
25
0.0527
80
0.1687
6.
e
)
Output signal
ORDER 2
25
ORDER 25
MSE:
8.8032e

004
MSE:
0.0105
26
Order 80
We can notice that for a con
stant step size of 0.1 as the order increases the MSE decreases as shown. At
order 25 and 80 the speech is much clearer than before and though slightly blurred sounds like:
“He writes at home every other night”
Comments on Convergence:
NLMS performs be
tter for stationary data compared to non

stationary data as can be seen from the plots
and tables above. And it performs better than the Wiener filter with non

stationary data as shown below.
This is because the Wiener filter assumes stationarity which is
not always a valid assumption especially in the
case of speech.
MSE:
3.0102e

004
27
Conclusion
NLMS is recommended for use in noisy environments
. Increasing the filter order increased the performance
of the system. Also the step size was fixed at a point which gav
e reasonable tradeoff between the
Misadjustment and the convergence.
A point to be noted was that the system
worked even
when the inputs
were switched. This could be because the unknown system was IIR
.
Comments 0
Log in to post a comment