Concurrently-Processed Adaptive Chip

cracklegulleyAI and Robotics

Oct 19, 2013 (3 years and 8 months ago)

80 views

A Dynamic Analog
Concurrently
-
Processed
Adaptive Chip

Malcolm Stagg

Grade 11

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Purpose:


To design a reconfigurable analog neural
network on a chip

A chip that can learn over time

For any application neural networks are used


Improvement over previous designs:

High density

High routability

On
-
chip learning

Multiple learning algorithms

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Neuron and synapse circuits are created


For TSMC 0.35
µm CMOS (Complementary
Metal Oxide Semiconductor) implementation


Dense layout of transistors for analog
arithmetic circuits

High Density


Neurons and Synapses are as small as
possible without compromising performance


Supports both Backpropagation and Hebbian
learning on each cell

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Neural networks:


Have been used for years in many applications, from
business and finance to science and engineering


Ideally implemented as arrays of electronic cells


Many learning algorithms have been designed

The first electronic neural network was made in
1951, by Marvin Minsky (called “SNARC”)

Carver Mead pioneered chip implementation of
neural cells, designing a retina
-
like neural circuit

Intel created a popular analog neural network
chip in 1989 (called “ETANN”)

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Backpropagation:


Feedback
-
based learning algorithm (supervised
learning)


Requires calculation of the correct output for an input
pattern


Will learn to represent the correct output over time

Hebbian:


Non
-
feedback
-
based learning algorithm
(unsupervised learning)


Calculates synapse weights based on correlation
between input patterns


Will learn to identify and classify input patterns

Hopfield:


Advanced memory recall algorithm, using Hebbian
learning

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Neuron Block Diagram

in

out

BPin

Hebb

BP

BPout

Hebb

+

-

BP

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Neuron Cell Diagram

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

The neuron design:


Supports Backpropagation and Hebbian

Uses an SRAM cell to enable/disable algorithm
-
dependent circuits


Uses tanh sigmoid circuit for forwards
-
propagation (range: [
-
1, 1])


Derivative as sech
2

for backwards
propagation (range: [0, 1])


A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Synapse Block Diagram

in

out

BPin

BP

BPout

Hebb

BP

Weight Update
Parameters


A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Synapse Cell Diagram

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

The synapse design:


Supports Backpropagation and Hebbian

Uses an SRAM cell to enable/disable algorithm
-
dependent circuits


Uses the Gilbert multiplier cell for forwards
-

and backwards
-

propagation

All inputs are ensured to be symmetrical (c+x, c
-
x)


Differential pair
-
based circuits are used

Input range is small for good linearity


A Dynamic Analog Concurrently
-
Processed Adaptive Chip

High routability


Multiple routing pathways between neurons
and synapses

Using programmable wires and connections


Enables use of advanced learning algorithms

To evolve/change the layout of the network over
time

This allows the network to become more dense
and more efficient, while maintaining a low
synapse count


A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Programmable Wire

Programmable Connection


SRAM

Analog
Switch


SRAM

Analog
Switch

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Routing Cell Diagram



Neuron



Synapse



Synapse




Synapse


A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Circuit Design


Created and modified analog and mixed
-
signal circuits for each learning/propagation
task


Most circuits are implemented using a small
number of MOSFET transistors

Transistors are relatively large (5
µm length) and
interdigitated for good matching

Simulations have been completed for all cells


CMOS layout has been completed for some
of the cells

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Current Summation

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Tanh and Sech
2

(Sigmoid and Derivative)

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Differential Pairs (for multiplier input)

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Gilbert Multiplier Cell

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Layout:


Completed for tanh/sech
2

First a stick diagram was planned

Initial layout planned in Microsoft Visio

Drawn in Cadence Virtuoso using unmatched
transistors

Drawn in Cadence Virtuoso using matched
(interdigitated) transistors


Planned for other cells, but not yet complete

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Initial Virtuoso Layout of tanh (no matching)

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Virtuoso Layout of tanh with interdigitated matching

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Initial Visio layout of tanh (no matching)

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Simulation Graphs

Tanh Sigmoid

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Sigmoid:


Input Common Mode Range: 0.5V


1.5V


Good accuracy compared to ideal function of
tanh

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Simulation Graphs

Sech
2

Sigmoid

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Sigmoid Derivative:


Input Common Mode Range: 0.5V


1.5V


Good accuracy compared to ideal function of
sech
2

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Simulation Graphs

Multiplier

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Multiplier:


Input Common Mode Range: 2V


3V


Symmetrical inputs around center point


Good linearity for input swing of
±
150mV


Performance is reduced as input range is
extended

Pre
-
processing differential pair is required

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Results:


A neural network is simulated in C++

OCR problem, of 5 training sets of randomly
ordered numerical characters


10x10 pixels


5% noise


Position offset of
±
1 pixel horizontally and vertically

Comparing:


100%
-
connected network


20%
-
connected network


Simulation of a dynamically re
-
routed network by
removing 10% of the synapses over 1000 cycles

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Results

0 to 1000 training cycles

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Results

200 to 1000 training cycles

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Results

800 to 1000 training cycles

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Conclusions:


The dynamic routing algorithm is the ideal
method of optimizing performance in a
hardware neural network

Reducing the number of synapses to increase
density


The cell circuits perform very well with
system
-
made restrictions on input ranges

Restrictions increase accuracy and linearity


The arrangement of routing cells between
neurons and synapses allows the routing
algorithm to perform very efficiently

A Dynamic Analog Concurrently
-
Processed Adaptive Chip

Special Thanks To:


Victoria Stagg

My mother, for all of her help and support


Andrew Stagg

My brother, for all of his help and support


Dr. Jim Haslett, University of Calgary

For use of the ATIPS laboratory, to use Cadence Virtuoso


John Carney, Cadence Design

For the demo license of Cadence Orcad Layout


Dr. Vance Tyree, MOSIS Corporation

For agreeing to accept a fabrication proposal under the
MOSIS education program


David Wells, Auton Engineering, Ltd.

For printing my trifold