Using Hardware Neural Networks

muscleblouseΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 10 μήνες)

80 εμφανίσεις

Speech/Music Classification
Using Hardware Neural Networks

Puneet Mulchandani

Peter O’ Connor


ECSE487


Computer Architecture Lab: Project

December 4, 2009

Project Overview


Problem: Speech/Music classification


Proposed Solution:


Hardware neural network (NN) classifiers


Software training, pre
-
processing and post
-
processing

12/3/2009

2

ECSE487
-

Computer Architecture Lab

Feed
-
forward (FF)
Perceptron Neural Network

Network Training

Feature Extraction

Sound Samples

Training Dataset

Testing Dataset

weights

feature vector

classification

Legend

Data

Software

Hardware

Hardware System Components


Package File: Global definitions


Testbed
: Top
-
level entity, state machine to operate FF
perceptron

network


[1] Feed
-
forward Network: the classifier network


[
l

] Feed
-
forward Layers:
perceptron

collections


[
n
k
]
Perceptrons
: basic computational unit


[1] Activation Function:
perceptron

transfer function


12/3/2009

ECSE487
-

Computer Architecture Lab

3

n
0

= 4

n
1

= 3

n
2

= 2

n
3

= 3

n
4

= 4

Example:


Network with
l =

5

Image Source:
http://cse.stanford.edu/class/sophomore
-
college/projects
-
00/neural
-
networks/Architecture/images/feedforward.jpg


Network Construction


Hardware Generation using VHDL generics


Strength: varying topologies with minimal changes


Improvement: could be extended for file I/O


Example: Topology
n
= [4 2 2 2]

12/3/2009

ECSE487
-

Computer Architecture Lab

4

Snippet from Package Header file
declaring network topology

Generic parameters defining
testbed

settings

Network Construction:

Synthesis [4
-
2
-
2
-
2]

12/3/2009

ECSE487
-

Computer Architecture Lab

5

Perceptron Layer Synthesis

Testbed

State Machine


Manages control signals to input ROMs and FF net


State Transition Chart:


12/3/2009

ECSE487
-

Computer Architecture Lab

6

INIT

STANDBY

READ

WEIGHTS

LOAD

WEIGHTS

OUTPUT

DONE

aclr

!EN

EN

then

Repeat until no
more weights

READ

INPUTS

LOAD

INPUTS

when
ready

when
output
valid

Repeat until

no more inputs

then

Reset all

control signals

Reset all

control signals

-

Store loaded

w,
θ

in buffer

-

Increment memory
counters until EOF

Set
LD_w

Set
LD_i

-

Store loaded
i

in buffer

-

Increment memory
counters until EOF

-

Set output buffer

-

Set output valid

Flag end of simulation

Important Control Signals

aclr
: system reset (asynchronous)

EN: begin simulation

LD_i
: network load input

LD_w
: network load weights & offsets


Text Legend

Text in black: state transition conditions

Text in green: state actions/outputs


Network Initialization


Testbed

loads weights and offset from ROM


Convention used:


1
st

in ROM [
std_logic_vector
]
-
> 1
st

in buffer [
W_INT_ARRAY
]


Example: Reading weights/offsets from ROM…


12/3/2009

ECSE487
-

Computer Architecture Lab

7










Test Bed Circuit: state
READ_WEIGHTS




weights.mif

offsets.mif

FSM

ffn_weights

ffn_offsets

FF Net

offset_address

weight_address

weight_index

offset_index

w

θ

VHDL Entity

Memory Buffer

ROM Input File

data

control

Rising edge:

state set to
READ_WEIGHT


ffn_weights

content is:

[ 0, 0 …,
-
400,
-
500 ]


ffn_offsets

content is:

[ 0, 0, 0, 0, 65739,
-
126213 ]

Generic delay state

Value read from address 2

Counters incremented to 3


ffn_weights

content is:

[ 0, 0 …,
-
300,
-
400,
-
500 ]


ffn_offsets

content is:

[…,
-
34587, 65739,
-
126213 ]

Network Initialization

12/3/2009

ECSE487
-

Computer Architecture Lab

8










Test Bed Circuit: state
LOAD_WEIGHTS




weights.mif

offsets.mif

FSM

ffn_weights

ffn_offsets

FF Net

LD_w

w[ ]

θ
[ ]

VHDL Entity

Memory Buffer

ROM Input File

data bus

control


Testbed

loads weights and offset from ROM


Convention used:


1
st

in ROM [
std_logic_vector
]
-
> 1
st

in buffer [
W_INT_ARRAY
]


Example: Loading weights/offsets into FF Net…


State:
LOAD_WEIGHT


Loads
ff_net

with values in
ffn_weights

and
ffn_offsets

Last value of
ffn_weights

set


ffn_weights

content is:

[ 1000, 900 …,
-
400,
-
500 ]


ffn_offsets

content is:

[ 0, 0, 0, 0, 65739,
-
126213 ]

LD_w

Network Usage


Testbed

loads input vectors from ROM


Convention used:


1st in ROM [
std_logic_vector
]
-
> 1st in buffer [
IO_INT_ARRAY
]


Example: Reading input from ROM…

12/3/2009

ECSE487
-

Computer Architecture Lab

9










Test Bed Circuit: state
READ_INPUT




inputs.mif

FSM

FF Net

input_address

i

data

control

VHDL Entity

Memory Buffer

ROM Input File

ffn_input

Rising edge:

state set to
READ_INPUT

Value read from address 3

Counters incremented to 4


ffn_inputs

content is:

[ 132, 232, 176, 192 ]

Network Usage


Testbed

loads input vectors from ROM


Convention used:


1st in ROM [
std_logic_vector
]
-
> 1st in buffer [
IO_INT_ARRAY
]


Example: Loading input into FF Net…

12/3/2009

ECSE487
-

Computer Architecture Lab

10










Test Bed Circuit: state
LOAD_INPUT




inputs.mif

FSM

FF Net

data bus

control

VHDL Entity

Memory Buffer

ROM Input File

LD_i

i

ffn_input

Rising edge:

state set to
LOAD_INPUT

-

Output [247, 253] ready

-

Valid signal set

-

State in
OUTPUT
for 1CC

Sound Classification: Test Data Set

12/3/2009

ECSE487
-

Computer Architecture Lab

11

Music

Voice

Feature Vectors


Audio segmented into 125ms segments.


Each segment treated as a sample


Spectral Analysis done on samples,
features
extracted:



Zero crossings


Brightness


Bandwidth


Mel
-
Frequency
cepstrum

12/3/2009

ECSE487
-

Computer Architecture Lab

12

Loading to the network


MIF files serve as the interface between
hardware and software.


Network, inputs first converted to integer
format, then loaded into
mif

format.


Network reads from there.


12/3/2009

ECSE487
-

Computer Architecture Lab

13

Extract
Features

Train
Software
Net

Scale
trained net
values

Save to
MIF

Load to
hardware

Error Source: Truncation

12/3/2009

ECSE487
-

Computer Architecture Lab

14

Balancing act:



Must be large enough that
truncation error do not
significantly hurt classification
accuracy



Smaller value means less bus
width, hardware usage.

Error Source: Approximation

12/3/2009

ECSE487
-

Computer Architecture Lab

15

Test: Single
Perceptron

12/3/2009

ECSE487
-

Computer Architecture Lab

16

1: Weights, offsets loaded

2: Inputs loaded

3: Output stabilizes

Conclusion

What we’ve designed:


Template for building a hardware neural
network of arbitrary size.


Supporting software system for extracting
feature vectors from raw audio, training the
neural network.

Next logical steps:


Design hardware implementations of feature
-
vector extractor.

12/3/2009

ECSE487
-

Computer Architecture Lab

17

References


need to update?


Carey et al.
A Comparison of Feature Vectors for
Speech, Music Discrimination.

Ensigma

Ltd.


Bugatti

et al.
Audio Classification in Speech and Music:
A Comparison between a Statistical and Neural
Approach.
EURASIP
jornal

on Applied Signal
Processing, 2002:4,pp.372
-
378


Lu,L
, Zhang H,
Li,S
.
Content based audio classification
and segmentation by using support vector machines
.
Multimedia Systems. v. 8, pp 482
-
492 (2003)


Slaney,
Malcom
.
Mel
-
frequency
cepstrum

coefficient
analysis

(
Matlab

code). Interval Research Analysis
(1998).


Wikimedia Commons

(Public Media database).
Acquired from commons.wikimedia.org.




12/3/2009

ECSE487
-

Computer Architecture Lab

18