DSP

CIS
Chapter

6: Filter Implementation
Marc Moonen
Dept. E.E./ESAT, KU Leuven
marc.moonen@esat.kuleuven.be
www.esat.kuleuven.be
/
scd
/
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
2
Filter Design/Realization
•
Ste
p

1
: define filter specs
(pass

band, stop

band, optimization criterion,…)
•
Ste
p

2
: derive optimal transfer function
FIR or IIR design
•
Ste
p

3
: filter realization
(block scheme/flow graph)
direct form realizations, lattice realizations,…
•
Ste
p

4
: filter implementation
(software/hardware)
finite word

length
issues, …
question: implemented filter = designed filter ?
‘You can’t always get what you want’

Jagger/Richards (?)
Chapter

4
Chapter

5
Chapter

6
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
3
Chapter

6 : Filter Implementation
•
Introduction
Filter implementation & finite
wordlength
problem
•
Coefficient Quantization
•
Arithmetic Operations
Scaling
Quantization noise
Limit Cycles
•
Orthogonal Filters
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
4
Introduction
Filter implementation & finite word

length problem :
•
So far have assumed that signals/coefficients/arithmetic
operations are represented/performed with
infinite
precision.
•
In practice, numbers can be represented only to a
finite
precision, and hence signals/coefficients/arithmetic
operations are subject to quantization
(truncation/rounding/...) errors.
•
Investigate impact of…

quantization of filter coefficients

quantization (& overflow) in arithmetic operations
•
PS
: Chapter partly adopted from
`A course in digital signal processing
’
, B.Porat, Wiley 1997
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
5
Introduction
Filter implementation & finite word

length problem :
•
We consider
fixed

point filter implementations
, with a
`short
’
word

length.
In hardware design, with tight speed requirements, finite
word

length problem is a relevant problem.
•
In signal processors with a `sufficiently long
’
word

length,
e.g. with 16 bits (=4 decimal digits) or 24 bits (=7 decimal
digits) precision, or with
floating

point
representations and
arithmetic, finite word

length issues are less relevant.
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
6
Q:Why
bother
about many different realizations
for one and the same filter?
Introduction
Back to Chapter

5…
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
7
Introduction: Example
Transfer function
•
%
IIR Elliptic
Lowpass
filter
designed using
•
%
ELLIP
function.
•
% All frequency values are in Hz.
•
Fs
= 48000;
% Sampling
Frequency
•
L
= 8;
%
Order
•
Fpass
= 9600; %
Passband
Frequency
•
Apass
= 60;
%
Passband
Ripple (dB)
•
Astop
= 160;
%
Stopband
Attenuation (dB)
•
Poles & zeros
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
8
Introduction: Example
Filter outputs…
Direct form realization
@ infinite precision…
Lattice

ladder realization
@ infinite precision…
Difference…
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
9
Introduction: Example
Filter outputs…
Direct form realization
@ infinite precision…
Direct form realization
@ 10

bit precision…
Difference…
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
10
Introduction: Example
Filter outputs…
Direct form realization
@ infinite precision…
Direct form realization
@ 8

bit precision…
Difference…
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
11
Introduction: Example
Filter outputs…
Direct form realization
@ infinite precision…
Lattice

ladder realization
@ 8

bit precision…
Difference…
Better select a good realization
!
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
12
Coefficient Quantization
The coefficient quantization problem
:
•
Filter design in Matlab (e.g.) provides filter coefficients to
15 decimal digits (such that filter meets specifications)
•
For implementation, have to quantize coefficients to the
word

length used for the implementation.
•
As a result, implemented filter may fail to meet
specifications… ??
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
13
Coefficient Quantization
Coefficient quantization effect on
pole locations
:
•
example : 2nd

order system
(e.g. for cascade/direct form realization)
`triangle of stability
’
: denominator polynomial is stable (i.e.
roots inside unit circle) iff coefficients lie inside triangle…
Proof: Apply Schur

Cohn stability test (see Chapter

5).
2
1
2
1
.
.
1
.
.
1
)
(
z
z
z
z
z
H
i
i
i
i
i
i
i

1
1

2
2
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
14
Coefficient Quantization
•
example (continued) :
with 5 bits per coefficient, all possible `quantized
’
pole positions are...
Low density of `quantized
’
pole locations at z=1, z=

1,
hence problem for narrow

band LP and HP filters (see Chapter

4).
1.5
1
0.5
0
0.5
1
1.5
1.5
1
0.5
0
0.5
1
1.5
end
end
)
plot(poles
1
:
0625
.
0
:
1
for
2
:
1250
.
0
:
2
for
stable)
(if
i
i
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
15
Coefficient Quantization
•
example (continued) :
possible remedy: `coupled realization
’
poles are where are realized/quantized
hence
‘
quantized
’
pole locations are (5 bits)
1.5
1
0.5
0
0.5
1
1.5
1.5
1
0.5
0
0.5
1
1.5
.
j
1
,
1
+
+
+

y[k]
u[k]
coefficient precision = pole precision
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
16
Coefficient Quantization
Coefficient quantization effect on
pole locations
:
•
example : higher

order systems (first

order analysis)
tightly spaced poles (e.g. for narrow band filters) imply
high sensitivity of pole locations to coefficient quantization
hence preference for low

order systems (e.g. in parallel/cascade)
p
o
l
y
n
o
m
i
a
l
:
1
a
1
.
z
1
a
2
.
z
2
.
.
.
a
L
.
z
L
r
o
o
t
s
a
r
e
:
p
1
,
p
2
,
.
.
.
,
p
L
`
q
u
a
n
t
i
z
e
d
'
p
o
l
y
n
o
m
i
a
l
:
1
ˆ
a
1
.
z
1
ˆ
a
2
.
z
2
.
.
.
ˆ
a
L
.
z
L
`
q
u
a
n
t
i
z
e
d
'
r
o
o
t
s
a
r
e
:
ˆ
p
1
,
ˆ
p
2
,
.
.
.
,
ˆ
p
L
ˆ
p
l
p
l
p
l
L
i
(
p
l
p
j
)
j
l
.
(
ˆ
a
i
a
i
)
i
1
L
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
17
Coefficient Quantization
PS
: Coefficient quantization in lossless lattice realizations
In lossless lattice, all coefficients are sines and cosines, hence all
values between
–
1 and +1…, i.e. `dynamic range
’
and coefficient
quantization error well under control.
o = original transfer function
+ = transfer function after 8

bit
truncation
of lossless lattice
filter coefficients

= transfer function after 8

bit
truncation
of direct

form
coefficients (bi
’
s)
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
18
Arithmetic Operations
Finite word

length effects in arithmetic operations :
•
In
linear filters, have to consider additions & multiplications
•
Addition:
if, two B

bit numbers are added, the result has (B+1) bits.
•
Multiplication:
if a B1

bit number is multiplied by a B2

bit number, the
result has (B1+B2

1) bits.
For instance, two B

bit numbers yield a (2B

1)

bit product
•
Typically (especially so in an IIR (feedback) filter), the result of an
addition/multiplication has to be represented again as a B
’

bit number
(e.g. B
’
=B). Hence have to remove most significant bits and/or least
significant bits…
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
19
Arithmetic Operations
•
Option

1:
Most significant bits
(MSBs)
If the result is known to be
(almost)
always upper bounded such that 1 or
more MSBs are
(almost)
always redundant, these MSBs can be dropped
without loss of accuracy
(mostly)
. Dropping MSBs then leads to better
usage of available word

length, hence better SNR.
This implies we have to monitor potential
overflow
(=dropping MSBs
that are non

redundant), and possibly introduce additional
scaling
to
avoid overflow.
•
Option

2 :
Least significant bits
(LSBs)
Rounding/truncation/… to B
’
bits introduces
quantization noise
.
The effect of quantization noise is usually analyzed in a
statistical
manner.
Quantization, however, is a
deterministic non

linear
effect, which may
give rise to
limit cycle oscillations
.
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
20
Quantization Noise
Quantization mechanisms:
Rounding Truncation Magnitude Truncation
mean=0 mean=(

0.5)LSB (biased!) mean=0
variance=(1/12)LSB^2 variance=(1/12)LSB^2 variance=(1/6)LSB^2
input
probability
error
output
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
21
Quantization Noise
Statistical analysis is based on the following
assumptions
:

each quantization error is random, with uniform probability distribution
function (see previous slide)

quantization errors at the output of a given multiplier are
uncorrelated/independent (=white noise assumption)

quantization errors at the outputs of different multipliers are
uncorrelated/independent (=independent sources assumption)
One noise source is inserted
for each
multiplier(/adder)
Since the filter is a
linear filter
the output
noise generated by each noise source can be
computed and added to the output signal…
y[k]
u[k]
+
x

.99
+
e[k]
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
22
Limit Cycles
Statistical analysis is simple/convenient, but quantization is
truly a
non

linear
effect, and should be analyzed as a
deterministic
process.
Though very difficult, such analysis may reveal odd behavior:
Example:
y[k] =

0.625.y[k

1]+u[k]
4

bit rounding arithmetic
input u[k]=0, y[0]=3/8
output y[k] = 3/8,

1/4, 1/8,

1/8, 1/8,

1/8, 1/8,

1/8, 1/8,..
Oscillations in the absence of input (u[k]=0) are called
`
zero

input limit cycle oscillations
’
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
23
Limit Cycles
Example:
y[k] =

0.625.y[k

1]+u[k]
4

bit truncation (instead of rounding)
input u[k]=0, y[0]=3/8
output y[k] = 3/8,

1/4, 1/8, 0, 0, 0,.. (no limit cycle!)
Example:
y[k] = 0.625.y[k

1]+u[k]
4

bit rounding
input u[k]=0, y[0]=3/8
output y[k] = 3/8, 1/4, 1/8, 1/8, 1/8, 1/8,..
Example:
y[k] = 0.625.y[k

1]+u[k]
4

bit truncation
input u[k]=0, y[0]=

3/8
output y[k] =

3/8,

1/4,

1/8,

1/8,

1/8,

1/8,..
Conclusion: weird, weird, weird,… !
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
24
Limit Cycles
Limit cycle oscillations are clearly
unwanted
(e.g. may be
audible in speech/audio applications)
Limit cycle oscillations can only appear if the filter has
feedback. Hence
FIR filters
cannot have limit cycle
oscillations.
Mathematical analysis is very difficult
Truncation often helps to avoid limit cycles (e.g.
magnitude
truncation
, where absolute value of quantizer output is
never larger than absolute value of quantizer input
(=`passive quantizer
’
)).
Some filter realizations can be made limit cycle free, e.g.
coupled realization,
orthogonal filters
(see below).
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
25
Orthogonal Filters
Orthogonal filter
=
state

space realization with
orthogonal realization matrix:
PS
: lattice filters and lattice

part of lattice

ladder ….
Strictly speaking, these are not state

space realizations (cfr. supra), but
orthogonal
R is realized as a product of matrices, each of which is
again
orthogonal
, such that useful properties of orthogonal states

space realizations indeed carry over (see p. 38)
Why should we be so fond of orthogonal filters ?
I
R
R
R
R
D
C
B
A
R
T
T
.
.
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
26
Orthogonal Filters
Scaling
:

in a state

space realization, have to monitor overflow in internal states

transfer functions from input to internal states (`
scaling functions
’
) are
defined by impulse response sequence (see p. 22)

for an orthogonal filter (R
’
.R=I), it is proven (next slide) that
which implies that all scaling functions (rows of Delta) have L2

norm=1
(=diagonal elements of )
conclusion: orthogonal filters are `
scaled in L2

sense
’
]
[
.
]
[
.
]
1
[
k
u
B
k
x
A
k
x
...
.
2
B
A
AB
B
I
T
T
.
...
0
3
2
B
A
B
A
AB
B
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
27
Orthogonal Filters
Proof
:
First :
Then:
I
B
B
A
A
I
R
R
T
T
T
.
.
.
...
0
0
0
...
...
)
.
.
(
)
.
.
(
)
(
...
.
.
.
.
.
.
.
T
T
T
T
T
T
T
T
A
A
I
T
T
A
A
I
T
A
A
I
T
T
A
A
AA
A
A
AA
A
A
A
A
I
I
A
A
B
B
AA
A
B
B
A
B
B
I
I
T
T
T
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
28
Orthogonal Filters
Quantization noise
:

in a state

space realization, quantization noise in internal states may be
represented by equivalent noise source (vector) Ex[k]

transfer functions from noise sources to output (`
noise transfer
functions
’
) are defined by impulse response sequence

for an orthogonal filter (R
’
.R=I), it is proven that (…try it)
which implies that all noise gains are =1 (all noise transfer functions
have L2

norm = 1 = diagonal elements of ).

This is then proven to correspond to
minimum possible output noise
variance
for given transfer function, under L2 scaled conditions
(i.e.
when states are scaled such that scaling functions have L2

norm=1)
!!
(details omitted)
]
[
]
[
.
]
[
.
]
1
[
k
E
k
u
B
k
x
A
k
x
x
...
)
(
)
(
.
2
T
T
T
T
T
CA
CA
C
I
.
T
...
)
(
)
(
2
T
T
T
CA
CA
C
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
29
Orthogonal Filters
Limit cycle oscillations
:
if
magnitude truncation
is used (=`passive quantization
’
),
orthogonal filters are
guaranteed to be free of limit cycles
(details omitted)
intuition:
quantization consumes energy/power, orthogonal
filter does not generate power to feed limit cycle.
DSP

CIS / Chapter

6: Filter Implementation / Version 2012

2013
p.
30
Orthogonal Filters
Orthogonal filters
=
L2

scaled, minimum output noise,
limit cycle oscillation free filters !
It can be shown that these statements also hold for
:
* lossless lattice realizations of a general IIR filter
* lattice

part of a general IIR filter in lattice

ladder realization
Comments 0
Log in to post a comment