Janusz Starzyk, Yongtao Guo and Zhineng Zhu

canolaokahumpkaΗλεκτρονική - Συσκευές

2 Νοε 2013 (πριν από 3 χρόνια και 9 μήνες)

76 εμφανίσεις

SOLAR @ 2003

1


Janusz Starzyk, Yongtao Guo and Zhineng Zhu

Ohio University, Athens, OH 45701, U.S.A.


April 27
th
, 2003

SOLAR @ 2003

2

OUTLINE


INTRODUCTION


Biological Neural Networks


Traditional Hardware Implementation


BACKGROUND


Advantages & Algorithm


Self
-
Organizing Principle, Process and Simulation


Hardware Architecture


NEURON ARCHITECTURE


ROUTING


Scheme



Structure



SIMULATION


3D SOLAR


SUMMARY


SOLAR @ 2003

3



Biological Neural Networks


Cell
body

From IFC’s
webpage

Dowling, 1998, p. 17

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

4

Traditional ANN Hardware


Quadratic relationship between the routing
and the number of neurons makes traditional
ANNs wire dominated


Limited hardware resource, especially routing
resource limits the growth of VLSI NN chip
complexity


Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

5

Self
-
Organizing Learning
ARray (SOLAR )


Advantages:


Self
-
organizing;


Re
-
configurable;


Expandable to
multiple chips;


Linear relationship
between routing and
number of neurons


Algorithm:


Entropy
-
based
neural network
learning algorithm

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

6

Self
-
organizing Principle

The information index and subspace
information deficiencies are related as

With subsequent divisions the
accumulated information deficiency is
expressed as a product of information
deficiencies in each subsequent
partition.

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

7

Neuron’s Simulation
Structure

Neuron Inputs


System clock


Data input


Threshold control input (TCI)


Input information deficiency (ID)

Other

Neurons

This neuron


System clock

Nearest


neighbor

neuron

Remote

neurons

TCI

ID

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

8

Cont’d

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

9

Single Neuron’s Structure


Pico
-
Blaze 8
-
bit micro
-
controller from Xilinx


Use dual
-
port 256x16
-
bit
memory as instruction
storage


Dynamical reconfiguration


Full connections
implementation mimics the
dense connection scheme in
the neighborhood neurons


instruction

in_port

clk

int

reset

address

out_port

port_id

read_strobe

write_strobe

x

30 inputs

Dual port
memory

R

R

Neural

Controller

(use

KCPSM)

address

addr_bus

data_bus

clk

enable

instruction

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

10

Neuron Array Structure

Array neurons’ organization

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


Identical neuron
architecture


The programming
contents can be
dynamically updated
via the configuration
bus


3D expansion of these
chips represents the
sparse connections
between remote
neurons

SOLAR @ 2003

11

Routing Structure


Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


Resources of a single sub
-
network reused to implement
multiple sub
-
networks
sequentially


Many resources are saved.


Cost of additional processing
time is not a problem


New dynamically
reconfigurable routing and
addressing scheme is proposed

SOLAR @ 2003

12

Cont’d


Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


Two basic elements: the
routing channel and the
neuron cell


Shift register


Data addressing and clock
cycle counting


Connections information is
stored in individual neurons


Column reordering

SOLAR @ 2003

13

Routing Scheme

Hardware reuse to

implement 2 sub
-
networks.

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


Network system implemented by a
single sub
-
network.


Two groups of addresses: the first
group of 4 addresses storing the
number of clock cycles
corresponding to the 4 inputs in the
first sub
-
network, and the second
group of 5 addresses storing the
number of clock cycles
corresponding to the 5 inputs in the
second network.

SOLAR @ 2003

14

Routing Structure

Routing channel with a micro controller

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


PicoBlaze


CLB Based


8 parallel shift registers
corresponding to
PicoBlaze bus width


Self
-
organizing

Process Matlab Simulation

Initial interconnection

Learning process

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

16

Simulation Results

Training

Data

SOLAR & other

Algorithms

Credit card data

(
ftp:cs.uci.edu
)


SOLAR & other

Classifier

(Model)

Method


Miss
Detection
Probability


Method


Miss
Detection
Probability


CAL5


.131


Naivebay


.151


DIPOL92


.141


CASTLE


.148


Logdisc


.141


ALLOC80


.201


SMART


.158


CART


.145


C4.5


.155


NewID


.181


IndCART


.152


CN2


.204


Bprop


.154


LVQ


.197


RBF


.145


Quadisc


.207


Baytree


.171


Default


.440


ITule


.137


k
-
NN


.181


AC2


.181


SOLAR


.135


Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

17

Behavioral VHDL Model

All functions and signal variables
in the packages are shared, and
program execution is functionally
interleaved.


lower level package describes
system input and output, and
updates the network memory.


The higher level packages
encapsulate new system functions
based on the functions of the
lower level packages.


The highest level design function
implements the system
organization and management.

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

18

Software

Model

In

Behavioural

VHDL


Hardware


Model

In

Structural

VHDL


SW/HW VHDL Cosimulation


A software process


Written in behavioral
VHDL which is not
synthesizable


A hardware process


Written in
synthesizable RTL
VHDL


HW/SW communication


FSM and FIFOs

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

19


The initial connections shown in solid lines.


Every neuron adds two inputs


For example neuron 3 adds input 2 and the
output of neuron 1; neuron 5 adds the outputs of
neuron 2 and 3, etc.


After dynamical reconfiguration


neuron 3 has inputs from the outputs of
neuron 1 and 2

NN Example

1

3

5

2

4

6

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

and neuron 5 has inputs from the
outputs of neuron 3 and 4

(shown by the dotted lines. )


The results read out from the chip
via PCI bus are shown in the
Matlab console


“initial” values show primary
input values (6 and 2) and neuron
outputs for two rows of neurons


“updated” values show inputs and
neuron outputs after dynamical
reconfiguration step

SOLAR @ 2003

20

“Enable_bus” selects a neuron to be configured.

Once the configuration process for all neurons is over, the outputs from neurons
are ready to be read out.

Updating any neuron’s configuration do not affect the other neurons.

NN Example

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

In this example, we update neurons 3 and 4 connections represented by
“Enable_bus” content 4 and 5.

The simulation results after updating correspond to the real experiments read
back from hardware as the “updated” values in previous slide.

SOLAR @ 2003

21

Neurons Prototyping

Problem:

Memory usage for every neuron needs to be optimized to avoid
bottleneck of resource utilization.

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

22

SW/HW codesign of SOLAR



JTAG

Programming

Software

run in PC

PCI Bus

Hardware


Board

Virtex XCV800FPGA
dynamic configuration

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR


FPGAs make a
scalable fabric


FPGAs communicate
and execute in
parallel


FPGAs operate at the
bit and byte levels




In 1990 FPGA had


400
-
800 4
-
LUTs



In 2003 FPGA has


100,000 4
-
LUTs


Multiple Megabit
memory


SOLAR @ 2003

23

PCB Design

Single SOLAR PCB contains 2x2 VIRTEX XCV1000 chips

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

24


Two PCB Boards

Interface Board

SOLAR Board

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

25



Expandable


Parallel


Local interconnect


Power management


Broadcast
configuration


Dynamic Self
-
Reconfiguration

SOLAR Board

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

26


3D SOLAR system,
containing 5x5 SOLAR
racks with close to 400
million gates



It will have a 24000
computing cells
(neurons) working in
parallel to process the
incoming data



Can be used for other
massively parallel
computations (e.g.
silicon wind tunnel)


RACK and CUBE SOLAR

RACK and CUBE SOLAR

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

27

SOLAR Development

Rack

(4 boards,1x4)

1 Million
gates

4 Million
gates

16 Million gates

System

(25 cabinets,
5X5)

Single Chip

Solar Board
+Interface Board

(4 chips,2x2)+(2 chips)

400 million gates

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

28

Conclusion

SOLAR is different from traditional
neural networks …



Learning and organization is based on local
information


Hardware
-
oriented expandable parallel architecture


Dynamically reconfigurable hardware structure


Interconnection number grows linearly with the
number of neurons


Data
-
driven self
-
organizing learning hardware

Introduction

Background

Neuron

Architecture

Routing

Summary

Simulation

3D SOLAR

SOLAR @ 2003

29

Questions?