Digital Signal Processing II Lecture 4: Filter Realization ...

photohomoeopathAI and Robotics

Nov 24, 2013 (3 years and 6 months ago)

49 views

DSP
-
CIS


Chapter
-
6: Filter Implementation

Marc Moonen

Dept. E.E./ESAT, KU Leuven

marc.moonen@esat.kuleuven.be

www.esat.kuleuven.be
/
scd
/


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
2

Filter Design/Realization


Ste
p
-
1

: define filter specs


(pass
-
band, stop
-
band, optimization criterion,…)


Ste
p
-
2

: derive optimal transfer function


FIR or IIR design



Ste
p
-
3

: filter realization
(block scheme/flow graph)


direct form realizations, lattice realizations,…


Ste
p
-
4

: filter implementation
(software/hardware)


finite word
-
length

issues, …


question: implemented filter = designed filter ?


‘You can’t always get what you want’
-
Jagger/Richards (?)

Chapter
-
4

Chapter
-
5

Chapter
-
6

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
3

Chapter
-
6 : Filter Implementation




Introduction

Filter implementation & finite
wordlength

problem


Coefficient Quantization


Arithmetic Operations

Scaling

Quantization noise

Limit Cycles


Orthogonal Filters



DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
4

Introduction


Filter implementation & finite word
-
length problem :


So far have assumed that signals/coefficients/arithmetic
operations are represented/performed with
infinite

precision.


In practice, numbers can be represented only to a
finite

precision, and hence signals/coefficients/arithmetic
operations are subject to quantization
(truncation/rounding/...) errors.


Investigate impact of…


-

quantization of filter coefficients


-

quantization (& overflow) in arithmetic operations


PS
: Chapter partly adopted from


`A course in digital signal processing

, B.Porat, Wiley 1997


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
5

Introduction


Filter implementation & finite word
-
length problem :



We consider
fixed
-
point filter implementations
, with a
`short


word
-
length.


In hardware design, with tight speed requirements, finite
word
-
length problem is a relevant problem.



In signal processors with a `sufficiently long


word
-
length,
e.g. with 16 bits (=4 decimal digits) or 24 bits (=7 decimal
digits) precision, or with
floating
-
point

representations and
arithmetic, finite word
-
length issues are less relevant.


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
6

Q:Why

bother


about many different realizations




for one and the same filter?



Introduction

Back to Chapter
-
5…

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
7

Introduction: Example

Transfer function


%
IIR Elliptic
Lowpass

filter
designed using


%
ELLIP
function.


% All frequency values are in Hz.


Fs

= 48000;

% Sampling
Frequency


L
= 8;

%
Order


Fpass

= 9600; %
Passband

Frequency


Apass

= 60;

%
Passband

Ripple (dB)


Astop

= 160;

%
Stopband

Attenuation (dB)




Poles & zeros

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
8

Introduction: Example

Filter outputs…

Direct form realization


@ infinite precision…

Lattice
-
ladder realization


@ infinite precision…

Difference…

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
9

Introduction: Example

Filter outputs…

Direct form realization


@ infinite precision…

Direct form realization


@ 10
-
bit precision…

Difference…

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
10

Introduction: Example

Filter outputs…

Direct form realization


@ infinite precision…

Direct form realization


@ 8
-
bit precision…

Difference…

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
11

Introduction: Example

Filter outputs…

Direct form realization


@ infinite precision…

Lattice
-
ladder realization


@ 8
-
bit precision…

Difference…

Better select a good realization
!

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
12

Coefficient Quantization

The coefficient quantization problem

:



Filter design in Matlab (e.g.) provides filter coefficients to
15 decimal digits (such that filter meets specifications)


For implementation, have to quantize coefficients to the
word
-
length used for the implementation.


As a result, implemented filter may fail to meet
specifications… ??


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
13

Coefficient Quantization

Coefficient quantization effect on
pole locations

:


example : 2nd
-
order system
(e.g. for cascade/direct form realization)





`triangle of stability


: denominator polynomial is stable (i.e.


roots inside unit circle) iff coefficients lie inside triangle…






Proof: Apply Schur
-
Cohn stability test (see Chapter
-
5).

2
1
2
1
.
.
1
.
.
1
)
(









z
z
z
z
z
H
i
i
i
i
i




i

i

-
1

1

-
2

2

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
14

Coefficient Quantization


example (continued) :


with 5 bits per coefficient, all possible `quantized


pole positions are...











Low density of `quantized


pole locations at z=1, z=
-
1,
hence problem for narrow
-
band LP and HP filters (see Chapter
-
4).

-1.5
-1
-0.5
0
0.5
1
1.5
-1.5
-1
-0.5
0
0.5
1
1.5
end
end


)
plot(poles

1
:
0625
.
0
:
1
for

2
:
1250
.
0
:
2
for
stable)

(if




i
i


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
15

Coefficient Quantization


example (continued) :


possible remedy: `coupled realization



poles are where are realized/quantized


hence

quantized


pole locations are (5 bits)








-1.5
-1
-0.5
0
0.5
1
1.5
-1.5
-1
-0.5
0
0.5
1
1.5


.
j

1
,
1







+

+

+







-

y[k]

u[k]

coefficient precision = pole precision

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
16

Coefficient Quantization

Coefficient quantization effect on
pole locations

:


example : higher
-
order systems (first
-
order analysis)












tightly spaced poles (e.g. for narrow band filters) imply


high sensitivity of pole locations to coefficient quantization




hence preference for low
-
order systems (e.g. in parallel/cascade)


p
o
l
y
n
o
m
i
a
l

:


1

a
1
.
z

1

a
2
.
z

2

.
.
.

a
L
.
z

L
r
o
o
t
s

a
r
e

:

p
1
,
p
2
,
.
.
.
,
p
L
`
q
u
a
n
t
i
z
e
d
'

p
o
l
y
n
o
m
i
a
l
:


1

ˆ
a
1
.
z

1

ˆ
a
2
.
z

2

.
.
.

ˆ
a
L
.
z

L
`
q
u
a
n
t
i
z
e
d
'

r
o
o
t
s

a
r
e
:


ˆ
p
1
,
ˆ
p
2
,
.
.
.
,
ˆ
p
L
ˆ
p
l

p
l


p
l
L

i
(
p
l

p
j
)
j

l

.
(
ˆ
a
i

a
i
)
i

1
L

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
17

Coefficient Quantization

PS
: Coefficient quantization in lossless lattice realizations












In lossless lattice, all coefficients are sines and cosines, hence all


values between

1 and +1…, i.e. `dynamic range


and coefficient


quantization error well under control.

o = original transfer function


+ = transfer function after 8
-
bit


truncation

of lossless lattice

filter coefficients


-

= transfer function after 8
-
bit


truncation

of direct
-
form


coefficients (bi

s)


DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
18

Arithmetic Operations

Finite word
-
length effects in arithmetic operations :



In

linear filters, have to consider additions & multiplications


Addition:



if, two B
-
bit numbers are added, the result has (B+1) bits.


Multiplication:


if a B1
-
bit number is multiplied by a B2
-
bit number, the


result has (B1+B2
-
1) bits.


For instance, two B
-
bit numbers yield a (2B
-
1)
-
bit product


Typically (especially so in an IIR (feedback) filter), the result of an
addition/multiplication has to be represented again as a B

-
bit number
(e.g. B

=B). Hence have to remove most significant bits and/or least
significant bits…

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
19

Arithmetic Operations


Option
-
1:
Most significant bits

(MSBs)


If the result is known to be
(almost)
always upper bounded such that 1 or
more MSBs are
(almost)
always redundant, these MSBs can be dropped
without loss of accuracy
(mostly)
. Dropping MSBs then leads to better
usage of available word
-
length, hence better SNR.

This implies we have to monitor potential
overflow
(=dropping MSBs
that are non
-
redundant), and possibly introduce additional
scaling

to
avoid overflow.


Option
-
2 :
Least significant bits

(LSBs)


Rounding/truncation/… to B


bits introduces
quantization noise
.


The effect of quantization noise is usually analyzed in a
statistical

manner.


Quantization, however, is a
deterministic non
-
linear
effect, which may
give rise to
limit cycle oscillations
.

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
20

Quantization Noise

Quantization mechanisms:


Rounding Truncation Magnitude Truncation










mean=0 mean=(
-
0.5)LSB (biased!) mean=0


variance=(1/12)LSB^2 variance=(1/12)LSB^2 variance=(1/6)LSB^2

input

probability

error

output

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
21

Quantization Noise

Statistical analysis is based on the following
assumptions

:


-

each quantization error is random, with uniform probability distribution
function (see previous slide)


-

quantization errors at the output of a given multiplier are
uncorrelated/independent (=white noise assumption)


-

quantization errors at the outputs of different multipliers are
uncorrelated/independent (=independent sources assumption)


One noise source is inserted

for each
multiplier(/adder)




Since the filter is a
linear filter

the output

noise generated by each noise source can be

computed and added to the output signal…


y[k]

u[k]

+

x

-
.99

+

e[k]

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
22

Limit Cycles

Statistical analysis is simple/convenient, but quantization is
truly a
non
-
linear

effect, and should be analyzed as a
deterministic

process.

Though very difficult, such analysis may reveal odd behavior:



Example:

y[k] =
-
0.625.y[k
-
1]+u[k]


4
-
bit rounding arithmetic


input u[k]=0, y[0]=3/8


output y[k] = 3/8,
-
1/4, 1/8,
-
1/8, 1/8,
-
1/8, 1/8,
-
1/8, 1/8,..



Oscillations in the absence of input (u[k]=0) are called


`
zero
-
input limit cycle oscillations




DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
23

Limit Cycles

Example:

y[k] =
-
0.625.y[k
-
1]+u[k]


4
-
bit truncation (instead of rounding)


input u[k]=0, y[0]=3/8


output y[k] = 3/8,
-
1/4, 1/8, 0, 0, 0,.. (no limit cycle!)

Example:

y[k] = 0.625.y[k
-
1]+u[k]


4
-
bit rounding


input u[k]=0, y[0]=3/8


output y[k] = 3/8, 1/4, 1/8, 1/8, 1/8, 1/8,..

Example:

y[k] = 0.625.y[k
-
1]+u[k]


4
-
bit truncation


input u[k]=0, y[0]=
-
3/8


output y[k] =
-
3/8,
-
1/4,
-
1/8,
-
1/8,
-
1/8,
-
1/8,..

Conclusion: weird, weird, weird,… !

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
24

Limit Cycles

Limit cycle oscillations are clearly
unwanted

(e.g. may be
audible in speech/audio applications)

Limit cycle oscillations can only appear if the filter has
feedback. Hence
FIR filters

cannot have limit cycle
oscillations.

Mathematical analysis is very difficult


Truncation often helps to avoid limit cycles (e.g.
magnitude
truncation
, where absolute value of quantizer output is
never larger than absolute value of quantizer input
(=`passive quantizer

)).

Some filter realizations can be made limit cycle free, e.g.
coupled realization,
orthogonal filters

(see below).

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
25

Orthogonal Filters



Orthogonal filter

=

state
-
space realization with


orthogonal realization matrix:




PS

: lattice filters and lattice
-
part of lattice
-
ladder ….



Strictly speaking, these are not state
-
space realizations (cfr. supra), but
orthogonal

R is realized as a product of matrices, each of which is
again
orthogonal
, such that useful properties of orthogonal states
-
space realizations indeed carry over (see p. 38)


Why should we be so fond of orthogonal filters ?



I
R
R
R
R
D
C
B
A
R
T
T









.
.
DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
26

Orthogonal Filters




Scaling

:


-

in a state
-
space realization, have to monitor overflow in internal states



-

transfer functions from input to internal states (`
scaling functions

) are


defined by impulse response sequence (see p. 22)



-

for an orthogonal filter (R

.R=I), it is proven (next slide) that




which implies that all scaling functions (rows of Delta) have L2
-
norm=1


(=diagonal elements of )


conclusion: orthogonal filters are `
scaled in L2
-
sense



]
[
.
]
[
.
]
1
[
k
u
B
k
x
A
k
x





...

.
2
B
A
AB
B
I
T





T


.


...
0
3
2
B
A
B
A
AB
B
DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
27

Orthogonal Filters

Proof

:


First :




Then:






I
B
B
A
A
I
R
R
T
T
T




.
.
.



...
0
0
0


...
...
)
.

.
(
)
.
.
(
)
(




...
.

.
.
.
.
.
.






















T
T
T
T
T
T
T
T
A
A
I
T
T
A
A
I
T
A
A
I
T
T
A
A
AA
A
A
AA
A
A
A
A
I
I
A
A
B
B
AA
A
B
B
A
B
B
I
I
T
T
T
DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
28

Orthogonal Filters

Quantization noise

:



-

in a state
-
space realization, quantization noise in internal states may be


represented by equivalent noise source (vector) Ex[k]



-

transfer functions from noise sources to output (`
noise transfer


functions

) are defined by impulse response sequence



-

for an orthogonal filter (R

.R=I), it is proven that (…try it)





which implies that all noise gains are =1 (all noise transfer functions


have L2
-
norm = 1 = diagonal elements of ).


-

This is then proven to correspond to
minimum possible output noise


variance

for given transfer function, under L2 scaled conditions
(i.e.


when states are scaled such that scaling functions have L2
-
norm=1)

!!
(details omitted)


]
[
]
[
.
]
[
.
]
1
[
k
E
k
u
B
k
x
A
k
x
x






...
)
(
)
(

.
2
T
T
T
T
T
CA
CA
C
I







.
T


...
)
(
)
(
2
T
T
T
CA
CA
C
DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
29

Orthogonal Filters

Limit cycle oscillations

:



if
magnitude truncation

is used (=`passive quantization

),


orthogonal filters are
guaranteed to be free of limit cycles



(details omitted)





intuition:

quantization consumes energy/power, orthogonal
filter does not generate power to feed limit cycle.

DSP
-
CIS / Chapter
-
6: Filter Implementation / Version 2012
-
2013


p.
30

Orthogonal Filters




Orthogonal filters


=
L2
-
scaled, minimum output noise,
limit cycle oscillation free filters !




It can be shown that these statements also hold for

:


* lossless lattice realizations of a general IIR filter


* lattice
-
part of a general IIR filter in lattice
-
ladder realization