progress

glibdoadingAI and Robotics

Oct 20, 2013 (3 years and 11 months ago)

92 views




Final
Report


Final Year Project

Evolving
Spiking
Neural Networks
















Submitted By:
Yusuf Cinar













06107729



Introduction


My Final Year Project is a contribution to an ongoing project
on
evolving

Spiking Neural
Networks

pur
sued by
Patrick Rocke, John Maher,
and
Fearghal Morgan
. As it is currently
an ongoing project, in order to have a good understanding of
project details,
I started
with
reading on the theory behind
the
materials used in the project like Switching Technology
,
FPAA technology, Spiking Neural Networks (SNN), PAMA paper and Patrick Rocke’s
paper,
Investigation of Reconfigurable ANN’s on an FPAA
. I highly benefited from
Patrick’s

paper in this report

as well as other related documents
.





As scheduled, I
have co
mpleted the followings to date;




Reading PAMA paper




Reading on Switching Technology



Reading on neural networks



Understanding the design of Single Ended to Differential Converter



Understanding the design of Spiking Neural Networks in Anadigm Designer.



In
itial Web Page has been designed and uploaded



Figuring out the connections on the Base board of the MUX Boards



Figuring out the connections MUX Boards



Figuring out the connections on the connectors



Programming IO Cards

-
Ampicon PCI 272

-
NIDAQ 6200



Making
a plan to test the Multiplexers to see if they work correctly



Figuring out the connections of the PCI 272 connectors and solving the
connection conflicts



Connecting Ampicon PCI 272 and NiDAQ device to a PC and programming
some scenarios to write some input

voltages to Multiplexers and sending select
bits
via Ampicon PCI 272
and reading the output voltages via NIDAQ device.




The next stage of the project, which is predominantly
ultimate
aim of the project, is to
connect the IO Cards and Multiplexers to th
e
FPAA
system
. At the first stage
,
FPAA
devices will be configured as buffers
, and

then the artificial neural network will be
implemented.


Afterwards, w
riting a Genetic Algorithm to automate the system
, which is the core of the
project, will take place.
The project has come to

a point that system works through the
static connections in between the neurons. Proposed system will be running through
dynamic connections between neurons through a completed Genetic Algorithm

by the
end of the project
.




Objec
tives


The concept of this project is to develop

novel applications and methods for electronic
circuits
and
platform
s

for implementing evolved Spiking Neural Networks (SNN)

based
on
the
ideas and concepts found in nature.



The objectives of this project a
re to examine the possibility of intrinsically evolving
analogue neural networks on a Field Programmable Analogue Array (FPAA)

and
possible techniques to allow dynamic reconfiguration of network connectivity and the
plausibility of applying
Genetic
Algorit
hms to automate the discovery of neural network
solutions.


The inspiration point of the area of this project is the parallel architecture of the neurons
in the brain.
Having accepted that new generation computers are fast enough to process
the given or pr
ogrammed code, it is a fact that they are far away from the adaptation
property of brain to new environment. Also
as far as
processing under noisy data

is
concerned
, biological neurons are superior over computers.


In iteration, pattern recognition, graph
ics and image processing software based artificial
neural networks are spectacular, but they may be slow to process large networks as they
run on sequential computers. At this point the necessity of hardware based artificial
neural networks becomes obvious

due to its fast
er

processing speeds over its alternative
solutions mentioned above
.



Figure
1

Artificial Neural Network


Figure 1 shows the typical architecture of artificial neural networks. The mat
hematical
expression
for
Figure 1 is;






This equation is also the top level expression for an artificial neural network

consisting of
n inputs
.



Circuit Model of Spiking Neuron



Figure
2

Circuit Model of Spiking Neuron



The figure above shows the single input spiking neuron

which is implemented on an

FPAA
.
Here is a SumDiff to process the sum of weighted inputs. Output of SumDiff
feeds the integrator
to integrate neural input. Feedback from integrator to SumDiff
provides
the decay mechanism for the internal potential
.



Figure
3

SNN, designed in Anadigm Designer 2


The figure above shows the circuit schematic drawn in Anadigm

Designer 2.


Mechanism
of the circuit
works through following steps;



A voltage is set as predefined voltage for comparator.



Once internal potential reaches that voltage, comparator go
es

high.



Internal potential is reset.



Output is reset.


Si
mulation of
the circuit clearly shows these steps;




top


input




middle


internal potential





bottom


output


Figure
4

Simulation of the SNN




FPAA Interconnectivity


Neurons
explained above
are implemented on FPAA devices and FPAA
devices are
connected each other. There is no difficulty in connecting FPAA devices together but
generating a dynamic connectivity in between FPAA devices using a Generic Algorithm
is non
-
trivial. Currently the FPAA devices, that is neurons, are connected
each other
statically, which means
system of neurons works through a single network. But the aim is
to build the most appropriate network for various conditions.




Figure
5

Static connections between neurons


The figure below sh
ows the static connectivity in between the
neurons with different
weights. To establish such connectivity is not a big deal, but in many cases in order
to
have more accurate and effective outputs it is
needed to
have dynamic

connections
between neurons

whi
ch can be configured depending on the inputs or external factors
.

The resulting dynamic configuration is expected to be as follows;



Figure
6

Dynamic
connections between neurons


This dynamic connectivity will be realised by usin
g Multiplexers running between
neurons. Select bits of the multiplexers are obtained via a Genetic Algorithm.









Building a Dynamic Reconfiguration Mechanism



Figure
7

Base Board used for Multipl
exer Boards


A base board, which is used to implement the dynamic configuration, consists of
following components.




4 MUX Boards

exist on the board,

each
of which
consists of 4 Multiplexers. In
total 16 Multiplexers are used

in the whole system
.



Connector

A

provides WR (Write) lines to the Multiplexers. WR lines are used to
turn on the desired one of the 16 Multiplexer. It also provides
the select
/address

bits which are shared by
all of the 16 Multiplexers. According to the Genetic
Algorithm, these select
bits will address one of the inputs to the output.
And this
output will be transmitted to one of the FPAA devices accordingly.
(By probing
it
is confirmed that
the Table

1

are the pin description for the connector A.
)



Connector

B
shows the power, VSS and g
round of the system.



Connector

C
and
Connector

D
shows the system inputs.



Connector

E,
Connector

F,
Connector

G,

and
Connector

H
show the connections
to the FPAA devices.


Pin No.

Pin Description

Pin No.

Pin Description

1

Addr_
3

21

Board_2_Mux_2_WR

2

A
ddr
_
2

22

Board_2_Mux_3_WR

3

Addr_1

23

GND

4

Addr_0

24

GND

5

GND

25

Board_3_Mux_0_WR

6

GND

26

Board_3_Mux_1_WR

7

Board_0_Mux_0_WR

27

Board_3_Mux_2_WR

8

Board_0_Mux_1_WR

28

Board_3_Mux_3_WR

9

Board_0_Mux_2_WR

29

NC

10

Board_0_Mux_3_WR

30

NC

11

GND

3
1

NC

12

GND

32

NC

13

Board_1_Mux_0_WR

33

NC

14

Board_1_Mux_1_WR

34

NC

15

Board_1_Mux_2_WR

35

NC

16

Board_1_Mux_3_WR

36

NC

17

GND

37

NC

18

GND

38

NC

19

Board_2_Mux_0_WR

39

NC

20

Board_2_Mux_1_WR

40

NC


Table 1:
Connector A

(see
Figure 2)

Pin

Out



Figure below shows the Top View of one of 4 MUX Boards.





A single MUX chip can be operated in one of two modes, ADG732 or ADG726. That is
either
a

32 single ended input to 1 single ended output
multiplexer

or 16 differe
ntial
inputs to 1 differential output
multiplexer respectively
.



The grey areas show the 2 connectors, J1 and J2, to the MUX Board.



Mux Board Side View (Connector Pin Out)


By probing the connections, the following
tables a
re

confirmed to be the pin outs for the
J1 and J2 connectors.


Pin No.

Pin Name

Pin No.

Pin Name

1

GND

21

A2

2

EN (Mux 3)

22

A1

3

WR (Mux 3)

23

A0

4

CS (Mux 3)

24

GND

5

GND

25

Vss

6

EN (Mux 2)

26

Vdd

7

WR (Mux 2)

27

GND

8

CS (Mux 2)

28

GND

9

GND

2
9

BusA_0

10

EN (Mux 1)

30

BusB_0

11

WR (Mux 1)

31

BusA_1

12

CS (Mux 1)

32

BusB_1

13

GND

33

BusA_2

14

EN (Mux 0)

34

BusB_2

15

WR (Mux 0)

35

GND

16

CS (Mux 0)

36

GND

17

GND

37

BusA_3

18

GND

38

BusB_3

19

A4

39

BusA_4

20

A3

40

BusB_4


Table
2
:

J1 C
onnector Pin Descriptions



Pin No.

Pin Name

Pin No.

Pin Name

1

BusA_5

21

BusA_12

2

BusB_5

22

BusB_12

3

GND

23

BusA_13

4

GND

24

BusB_13

5

BusA_6

25

BusA_14

6

BusB_6

26

BusB_14

7

BusA_7

27

GND

8

BusB_7

28

GND

9

BusA_8

29

BusA_15

10

BusB_8

30

BusB_
15

11

GND

31

DA (Mux 0)

12

GND

32

DB (Mux 0)

13

BusA_9

33

DA (Mux 1)

14

BusB_9

34

DB (Mux 1)

15

BusA_10

35

GND

16

BusB_10

36

GND

17

BusA_11

37

DA (Mux 2)

18

BusB_11

38

DB (Mux 2)

19

GND

39

DA (Mux 3)

20

GND

40

DB (Mux 3)


Table 3
:

J2 Connector P
in Descriptions




To operate in ADG726 mode (i.e. 16 to 1 dual ended multiplexer) the pin number 19 in
Table 2 is used as CSA while pins number 4, 8, 12 and 16 from Table 2 are used as CSB.


In order to test the operability of the Base Board and MUX Board
s, a test plan was made.
According to this plan, a PC and two IO cards are
employed in the test procedure.

These
IO cards are NIDAQ
62
00 and PCI 272 Digital IO Card.
Following diagram shows the
test procedure for the MUX Boards.








Send the select bit
s via PCI 272


Read output via
NIDAQ













System Inputs







Select bits


Output of the mux






As can be seen from the diagram, the initial test purpose is to check the operability of the
MUX Boards if they work correctly. Basic
ally;



sending some inputs to the system,



sending select bits to the MUX Boards using a Digital IO Card



reading the output vial NIDAQ device which has both digital and analog input
and output ports.


To realize these purposes, quite good time has been spen
t on to understand how to control
a digital IO Card, i.e. Ampicon PCI 272, and a NIDAQ device. Firstly, dealing with
digital IO Card is rather straight forward as we don’t have
an
analog signal.
That is, a

NIDAQ 6200


PCI 272



MUX BOARD

signal exists or not, if it does exist, it is 5V,
if it does not exist, it is 0V. But while dealing
with NIDAQ device, because it is sometimes needed to read or generate analog signals, it
can be tricky compared to Ampicon PCI 272 device.


IO Cards are programmed in Visual Basic 6.0. National Instruments
’ and Ampicon’ s
own libraries are utilized. Programming an IO Card essentially consists of two parts;
writing an voltage and reading a voltage. Let’s take an example of writing an analog
voltage to a port of NIDAQ device.
The algorithm developed for this
particular example
is 3 steps;


The interface prepared for this purpose is;


Create a Task

Gene
rate taskHandle

Add an Analog Output
Channel to the Task

Write voltage to the
desired channel


Stop Task



When button “Write” clicked, 3V will be written to Port1, Analog Output 0.

The starting point of the code running behind the interface is creating
a task, which is
essentially common in every National Instruments Algorithms.



' Create the DAQmx task.


DAQmxCreateTask("", taskHandle)


DAQmxCreateTask is function located in “
nidaqmx.tlb
” library. This library is written by
National Inst
ruments
placed under C:
\
system32
and access to this library is not granted
,
but the functions in this library and how to call these functions in Visual Basic 6.0 is open
to public knowledge.

When a task created an task
Handle variable is generated as Long.

This variable is used throughout the code for the same task.


After creating a task, desired voltage is written to the desired channel in two steps, firstly
add an analog output channel to the task and next write the voltage to the channel;



' A
dd an analog output channel to the task


DAQmxCreateAOVoltageChan(taskHandle,
physicalChannelTextBox.Text, "aoChannel",
minValueTextBox.Text, maxValueTextBox.Text,
DAQmx_Val_VoltageUnits2_Volts, "")


data(0) = Voltage.Text




'
Write to the desired channel(s).


DAQmxWriteAnalo
gF64(taskHandle, 1, True,
10#,
DAQmx_Val_GroupByScanNumber, data(0),
sampsPerChanWritten, ByVal 0&)





StopTask


Again
,

functions “DAQmxCreateAOVoltageChan” and “DAQmxWriteAnalogF64”

are
located in

nidaqmx.tlb
” library.
When these three steps are completed, 3V is written to
Analog Output
0 which is confirmed by voltmeter.


Reading a voltage from an input port is achieved using the required functions in

nidaqmx.tlb
” library.
The algorithm developed
for reading the data consists of following
steps;



Same as generating a voltage, first thing to do is to generate a taskHandle variable.


DAQmxCreateTask("", taskHandle)


'Add an analog input channel to the task.

DAQmxCreateAI
VoltageChan(taskHandle,
physicalChannelTextBox.Text, "", _

Create a Task

Generate taskHandle

Add an Analog Input
Channel to the Task

C
onfigure task for
finite sample
acquisitio
n


Read in data


Stop Task


DAQmx_Val_InputTermCfg_RSE,
minValueTextBox.Text, maxValueTextBox.Text, _


DAQmx_Val_VoltageUnits1_Volts, "")


'Configure task for finite sample acquisition a
nd read in
'
data


DAQmxCfgSampClkTiming(taskHandle, "OnboardClock",
frequencyTextBox.Text, DAQmx_Val_Rising, _


DAQmx_Val_AcquisitionType_FiniteSamps,
CLng(samplesPerChannelTextBox.Text))




DAQmxGetTaskNumChans(taskHandle, numChannel
s)


DAQmxReadAnalogF64(taskHandle, numSampsPerChannel, 10#, _


fillMode, data(0), arraySizeInSamps,
sampsPerChanRead, ByVal 0&)



StopTask



Up to this point, I have completed to test routines for multiplexers, i.e. reading and
writin
g voltages. Next stage is to implement the multiplexer system to the FPAA devices.
A genetic algorithm will automate the system to establish the dynamic connections
between the neurons.


Implementing the multiplexers to FPAA devices is expected to be comp
leted in a month.
In the next month, the GA will be up and running to automate the connections. So by the
end of February project is expected to reach almost end.




















References


Investigation of Reconfigurable ANN’s on an FPAA
,
Patrick Rock
e, John Maher,
Fearghal Morgan

(This paper was written project members and I benefited from it with
writing my report.)