# DISP-2003: Introduction to Digital Signal Processing - Duet

AI and Robotics

Nov 24, 2013 (4 years and 4 months ago)

72 views

FROM ANALOG

TO DIGITAL DOMAIN

Dr.M.A.Kashem

Asst. Professor, CSE,DUET

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
2

/
30

TOPICS

1.

Analog vs. digital: why, what & how

2.

Digital system example

3.

Sampling & aliasing

4.

5.

Digital data formats

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
3

/
30

-0.2
-0.1
0
0.1
0.2
0.3
0
2
4
6
8
10
sampling time, t
k
[ms]
Voltage [V]
t
s
-0.2
-0.1
0
0.1
0.2
0.3
0
2
4
6
8
10
sampling time, t
k
[ms]
Voltage [V]
t
s
Analog & digital signals

Continuous function

V of
continuous

variable t (time,
space etc) : V(t).

Analog

Discrete function

V
k

of
discrete

sampling variable t
k
,
with k = integer: V
k =
V(t
k
).

Digital

-0.2
-0.1
0
0.1
0.2
0.3
0
2
4
6
8
10
time [ms]
Voltage [V]
Uniform (periodic) sampling.
Sampling frequency f
S

= 1/ t
S

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
4

/
30

Digital
vs

analog proc’ing

Digital Signal Processing (DSPing)

More flexible.

Data easily stored.

Better control over accuracy
requirements.

Reproducibility.

A/D & signal processors speed:
wide
-
band signals still difficult to
treat (real
-
time systems).

Finite word
-
length effect.

Obsolescence (analog
electronics has it, too!).

Limitations

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
5

/
30

DSPing: aim & tools

Software

Programming languages: Pascal, C / C++ ...

“High level” languages: Matlab, Mathcad, Mathematica…

Dedicated tools (ex: filter design s/w packages).

Applications

Predicting a system’s output.

Studying a certain signal.

General purpose processors (GPP),

-
controllers.

Digital Signal Processors (DSP).

Programmable logic ( PLD, FPGA ).

Hardware

real
-
time
DSPing

Fast

Faster

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
6

/
30

Digital system example

ms

V

ANALOG
DOMAIN

ms

V

Filter

Antialiasing

k

A

DIGITAL
DOMAIN

A/D

k

A

Digital
Processing

ms

V

ANALOG
DOMAIN

D/A

ms

V

Filter

Reconstruction

Sometimes steps missing

-

Filter + A/D

(ex: economics);

-

D/A + filter

(ex: digital output wanted).

General scheme

Topics of this
lecture.

Digital
Processing

Filter

Antialiasing

A/D

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
7

/
30

Digital system implementation

Sampling rate.

• Pass / stop bands.

KEY DECISION POINTS:

Analysis bandwidth, Dynamic range

• No. of bits. Parameters.

1

2

3

Digital
Processing

A/D

Antialiasing
Filter

DIGITAL OUTPUT

• Digital format.

What to use for processing?

See slide “DSPing aim & tools”

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
8

/
30

Sampling

How fast must we sample a continuous
signal to preserve its info content?

Ex: train wheels in a movie.

25 frames (=samples) per second.

Frequency misidentification due to low sampling frequency.

Train starts wheels ‘go’ clockwise.

Train accelerates wheels ‘go’ counter
-
clockwise.

1

Why?

*

Sampling
: independent variable (ex: time) continuous

††
Q畡湴楳慴楯i
㨠摥灥摥琠癡v楡扬攠⡥㨠癯l瑡t攩⁣潮楮畯畳i

††
Here we’ll talk about uniform sampling
.

*

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
9

/
30

Sampling
-

2

__

s(t) = sin(2

f
0
t)

-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
t

s(t) @ f
S

f
0

= 1 Hz, f
S

= 3 Hz

-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
t
__

s
1
(t) = sin(8

f
0
t)

-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
t
__

s
2
(t) = sin(14

f
0
t)

-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
t
s
k

(t) = sin( 2

(f
0

+ k f
S
) t ) ,

k

s(t) @ f
S
represents exactly all sine
-
waves s
k
(t) defined by:

1

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
10

/
30

The sampling theorem

A signal s(t) with maximum frequency f
MAX

can be
recovered if sampled at frequency
f
S

> 2 f
MAX

.

Condition on f
S
?

f
S

> 300 Hz

t)
cos(100
π
t)
π
sin(300
10
t)
π
cos(50
3
s(t)

F
1
=25 Hz, F
2

= 150 Hz, F
3

= 50 Hz

F
1

F
2

F
3

f
MAX

Example

1

Theo
*

*

Multiple proposers: Whittaker(s), Nyquist, Shannon, Kotel’nikov.

Nyquist frequency (rate) f
N

= 2 f
MAX

or

f
MAX

or

f
S,MIN

or

f
S,MIN
/2

Naming gets

confusing !

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
11

/
30

Frequency domain
(hints)

Time & frequency
: two complementary signal descriptions.
Signals seen as “projected’ onto time or frequency domains.

Warning
: formal description makes use
of “negative” frequencies !

1

Bandwidth
: indicates rate of change of a signal.
High bandwidth

signal changes fast.

Ear

+ brain act as frequency analyser: audio spectrum
split into many narrow bands low
-
power sounds
detected out of loud background.

Example

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
12

/
30

Sampling low
-
pass signals

-
B 0 B f

Continuous spectrum

(a)

Band
-
limited signal:

frequencies in [
-
B, B] (f
MAX

= B).

(a)

-
B 0

B f
S
/2 f

Discrete spectrum

No aliasing

(b)

Time sampling frequency
repetition.

f
S

> 2 B no aliasing.

(b)

1

0 f
S
/2 f

Discrete spectrum

Aliasing & corruption

(c)

(c)

f
S

2 B
aliasing !

Aliasing: signal ambiguity
in frequency domain

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
13

/
30

Antialiasing filter

-
B 0 B f

Signal of interest

Out of band
noise

Out of band
noise

-
B 0 B f
S
/2
f

(a),(b)

Out
-
of
-
band

noise can aliase
into band of interest.
Filter it before!

(a)

(b)

(c)

Passband
: depends on bandwidth of
interest.

Attenuation A
MIN

: depends on

ADC resolution ( number of bits N).

A
MIN, dB

~ 6.02 N + 1.76

Out
-
of
-
band noise magnitude.

Other parameters: ripple, stopband
frequency...

(c)

Antialiasing filter

1

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
14

/
30

Under
-
sampling
(hints)

1

Using spectral replications to reduce
sampling frequency f
S

req’ments.

m
B
C
f
2
S
f
1
m
B
C
f
2

m

, selected so that f
S

> 2B

B

0

f
C

f

Bandpass signal
centered on f
C

-
f
S
0 f
S
2f
S

f

f
C

needed.

Simpler antialiasing filters.

f
C
= 20 MHz, B = 5MHz

Without under
-
sampling

f
S

> 40 MHz.

With under
-
sampling

f
S

= 22.5 MHz (m=1);

= 17.5 MHz (m=2); = 11.66 MHz (m=3).

Example

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
15

/
30

Over
-
sampling
(hints)

1

f
OS

= over
-
sampling frequency,

f
OS

= 4
w

f
S

-
sampling by a factor of four.

It works for:

-
white noise

with amplitude sufficient to change the input
signal randomly from sample to sample by at least LSB.

-
Input that can take all values between two ADC bits.

Caveat

Oversampling : sampling at frequencies f
S

>> 2 f
MAX

.

Over
-
sampling & averaging may improve ADC resolution

( i.e. SNR, see )

2

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
16

/
30

1.
Number of bits N (
~resolution
)

2.
Data
throughput (
~speed
)

3.
Signal
-
to
-
noise ratio (
SNR
)

4.
Signal
-
to
-
noise
-
&
-
distortion rate (
)

5.
Effective Number of Bits (
ENOB
)

6.
Spurious
-
free dynamic range (
SFDR
)

7.
Integral non
-
linearity (
INL
)

8.
Differential non
-
linearity (
DNL
)

9.

NB: Definitions may be slightly manufacturer
-
dependent!

Different applications
have different needs.

2

Static distortion

Dynamic distortion

Imaging / video

Communication

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
17

/
30

-

Number of bits N

Continuous input signal digitized into
2
N

levels
.

-4
-3
-2
-1
0
1
2
3
-4
-3
-2
-1
0
1
2
3
4
000
001
111
010
V
V
FSR
Uniform, bipolar transfer function (N=3)

Quantisation step

q =

V
FSR

2
N

Ex: V
FSR

= 1V , N = 12 q = 244.1

V

LSB

Voltage ( = q)

Scale factor (= 1 / 2
N
)

Percentage (= 100 / 2
N
)

-1
-0.5
0
0.5
1
-4
-3
-2
-1
0
1
2
3
4
- q / 2
q / 2
Quantisation error

2

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
18

/
30

-

Quantisation error

2

Quantisation Error e
q

in

[
-
0.5 q, +0.5 q].

e
q

limits ability to resolve
small signal.

Higher resolution means
lower e
q
.

-0.2
-0.1
0
0.1
0.2
0.3
0
2
4
6
8
10
time [ms]
Voltage [V]
QE for

N = 12

V
FS

= 1

0
2
4
6
8
10
Sampling time, t
k
|e
q

| [V]
10
-4
2 10
-4
M. E. Angoletta
-

DISP2003
-

From analog to digital domain
19

/
30

2

)
q
RMS(e
input
RMS
10
log
20
ideal
SNR
(1)

Also called SQNR

(signal
-
to
-
quantisation
-
noise ratio)

e
q

(
p(e)

constant, no stuck bits…)

e
q

uncorrelated with signal.

Assumptions

2
2
FSR
V
T
0
dt
2
ωt
sin
2
FSR
V
T
1
input
RMS

Input(t) = ½ V
FSR

sin(

t).

12
N
2
FSR
V
12
q
q/2
q/2
-
q
de
q
e
p
2
q
e
)
q
RMS(e

e
e
q
q

Error value

p
p
(
(
e
e
)
)

quantisation error probability density

1

q

q

2

q

2

(sampling frequency f
S

= 2 f
MAX
)

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
20

/
30

-

2

[dB]
1.76
N
6.02
SNR
ideal

(2)

Substituting in (1) :

One additional bit SNR increased by 6 dB

2

Actually

(2) needs correction factor depending on ratio between sampling freq
& Nyquist freq. Processing gain due to oversampling.

-

Real signals have noise.

-

Forcing input to full scale unwise.

-

-
linearities etc).

Real SNR lower because:

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
21

/
30

SNR
:

(
sine_in
RMS
RMS
), with

out_noise = output
-

(DC + first 5 input harmonics) output components.

:

(
sine_in
RMS
RMS
), with

out_noise_2 = output
-

(DC output component).

12
-

~ 11
-

Example

2

ENOB

:

N from
(2) when setting SNR
ideal

1.76 dB) / 6.02.

Actual number of bit available to an equivalent ideal ADC

SNR

and

often confused in specs.

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
22

/
30

Speed & resolution:

2

High resolution (bit #)

-

Higher cost & dissipation.

-

Tailored onto DSP word width.

High speed

-

Large amount of data to store/analyse.

-

Lower accuracy & input impedance.

*

*

DIFFICULT

area moves down
& right every year. Rule of thumb:
1 bit improvement every 3 years.

may increase SNR
.

2

Oversampling & averaging

(see ).

Dithering

random noise before quantisation).

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
23

/
30

Digital data formats

Important bases:
10 (decimal)
,
2 (binary)

Positional number system with base

b
:

[ .. a
2

a
1

a
0

.

a
-
1

a
-
2

..
]
b

= .. + a
2
b

2
+ a
1
b

1
+ a
0
b

0
+ a
-
1
b

-
1
+ a
-
2
b

-
2
+ ..

Integer
part

Fractional
part

3

Early computers (ex: ENIAC) mainly base
-
10 machines. Mostly turned
binary in the ’50s.

a) less complex arithmetic h/w;

Benefits

b) less storage space needed;

c) simpler error analysis.

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
24

/
30

Decimal arithmetic

BUT

Increasing number of applications requires decimal arithmetic
.

Ex: Banking, Financial Analysis.

IEEE 754,1985
:

binary floating point arithmetic standard specified

IEEE 854,1987:

standard expanded to include decimal arithmetic.

Common decimal fractional numbers only
approximated

by binary
numbers. Ex: 0.1 infinite recurring binary fraction.

Non
-
integer decimal arithmetic software emulation available
but

often too slow.

3

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
25

/
30

Fixed
-
point binary

Represent integer or
fractional binary numbers.

NB: Constant gap

between numbers.

Binary representation

Fractional point (DSPs)

15 14
...

0

MSB LSB

Sign bit

-
4

100

-
3

111

-
3

000

0

000

-
3

101

-
2

110

-
2

001

1

001

-
2

110

-
1

101

-
1

010

2

010

-
1

111

0

100

0

011

3

011

0

000

0

000

1

100

4

100

1

001

1

001

2

101

5

101

2

010

2

010

3

110

6

110

3

011

3

011

4

111

7

111

Two’s
complement

Sign
-
Magnitude

Offset
-
Binary

Unsigned
integer

Ex: 3
-
bit formats

3

Decimal equivalent

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
26

/
30

Floating
-
point binary

Formats & methods for binary floating
-
point arithmetic.

IEEE 754 standard

Definition of IEEE 754 standard between 1977 and 1985.
De facto

standard before 1985 !

Note:
NOT

the easiest h/w choice!

Wide variety of floating point hardware in ‘60s and ‘70s,

different ranges, precision and rounded arithmetic.

William Kahan:
“Reliable portable software was becoming more expensive
to develop than anyone but AT&T and the Pentagon could afford”
.

3

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
27

/
30

Floating
-
point binary
-

2

IEEE 754 standard

NB: Variable gap between numbers.

Large numbers large gaps; small numbers small gaps.

31 30 23 22 0

f

e

s

MSB LSB

e

= exponent, offset binary,
-
126 < e < 127

s

= sign, 0 = pos, 1 = neg

f

= fractional part, sign
-
magnitude + hidden bit

Single (32 bits)

Double (64 bits)

Double
-
extended ( 80 bits)

Precision

Coded number x = (
-
1)
s

∙ 2
e

∙ 1
.
f

Single precision range

Max = 3.4

∙ 10
38

Min = 1.175
∙ 10
-
38

3

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
28

/
30

Finite word
-
length effects

Dynamic range
dB

= 20 log
10

largest value

smallest value

Fixed point ~ 180 dB

Floating point ~1500 dB

High dynamic range wide data set representation with no overflow.

NB:
Different applications have different needs.

Ex: telecomms: 50 dB; HiFi audio: 90 dB.

3

Overflow

: arises when arithmetic operation result has one too

many bits to be represented in a certain format.

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
29

/
30

Finite word
-
length effects
-

2

Round
-
off error estimate:

Relative error

= (floating
-

actual value)/actual value (depends on base).

The smaller the base, the tighter the error estimate.

3

For integers

within
±
16.8 million range: single
-
precision floating point gives
no round
-
off error.

Outside

that range, integers are missing: gaps between consecutive floating
point numbers are larger than integers.

Round
-
off
: error caused by rounding math calculation result to

nearest quantisation level.

Big concern for

real numbers
.

0.1 not exactly represented (falls
between two floating point numbers).

Example

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
30

/
30

References

1.
On bandwidth
, David Slepian, IEEE Proceedings, Vol. 64, No 3, pp 291
-

300.

2.
The Shannon sampling theorem
-

Its various extensions and applications: a tutorial
review
, A. J. Jerri, IEEE Proceedings, Vol. 65, no 11, pp 1565

1598.

3.
What every computer scientist should know about floating
-
point arithmetic
, David
Goldberg.

4.
-
independent floating
-
point arithmetic
, ANSI/IEEE Std 854
-
1987.

Papers

1.
Understanding digital signal processing
-
Wesley Publishing, 1996.

2.
The scientist and engineer’s guide to digital signal processing
, S. W. Smith, at
http://www.dspguide.com
.

3.
Discrete
-
time signal processing
, A. V. Oppeheim & R. W. Schafer, Prentice Hall, 1999.

Books

M. E. Angoletta
-

DISP2003
-

From analog to digital domain
31

/
30

COFFEE BREAK

Be back in ~15 minutes

Coffee in room #13