An Introduction to Neural Networks

cartcletchΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

99 εμφανίσεις

Presented by Scott Lichtor


An Introduction to Neural Networks

Motivation I found for Neural
Networks


Pavlov’s dog


Simple
-
>Complex


Learning

Overview


Basics of the Nervous System


Neurons


Synapses


Action Potentials


Neural Networks


Abstract Neurons


More Complicated Neurons


Learning


Supervised


Unsupervised


Reinforcement


Conclusion


Basics of the Nervous System


The nervous system coordinates the actions of an animal


Body parts send messages to the brain


Brain sends messages to body parts


The basic unit of the nervous system is the neuron

Neurons


Receive messages at the dendrites


Message is sent quickly down the axon using electrical impulses


What happens when the signal reaches the end of the axon?


Image taken from img460.imageshack.us

Synapses


Chemical Synapses


Slow


Strong


Can be transmitted over long distances

Image taken from http://www.airlinesafety.com/editorials

Synapses


Electrical Synapses


Very fast


Fade quickly

Image taken from wikipedia.org

Action Potentials


Action potentials are shocks to a particular neuron


The shock travels along the affected neuron


Then, the action potential is transmitted from the affected
neuron to the neurons connected to it


The shock is transmitted to its destination in the same fashion

Abstract Neurons


So biological neurons can be used to send modified messages
from place to place


Can be used to accomplish very complex tasks using
relatively simple parts


Can neurons represent other things/be used for other
objectives?

Abstract Neurons


Neurons can represent neuron
-
like things


Inputs
-
> Processes
-
> Outputs


Image taken from http://3.bp.blogspot.com/

Abstract Neurons


Can “train” the neurons


Neurons fire (output 1) under certain patterns


Don’t fire (output 0) under other patterns


Firing rule: if an outcome doesn’t fit in either pattern, it fires if
it has more in common with the first set, and doesn’t fire if it
has more in common with the second set.


If there’s a tie, the neuron may fire, or it may not

Abstract Neurons


Example


A neuron takes three inputs (X1, X2, X3)


The neuron is trained to output 1 if the inputs are 111 or 101


Trained to output 0 if the inputs are 000 or 001


Before firing rule:






After firing rule:



X1

0

0

0

0

1

1

1

1

X2

0

0

1

1

0

0

1

1

X3

0

1

0

1

0

1

0

1

Out

0

0

0/1

0/1

0/1

1

0/1

1

X1

0

0

0

0

1

1

1

1

X2

0

0

1

1

0

0

1

1

X3

0

1

0

1

0

1

0

1

Out

0

0

0

0/1

0/1

1

1

1

Abstract Neurons


The abstract neuron model can be used for pattern
recognition


Example: determine whether a ‘T’ or ‘H’ is displayed


Can we model more complicated processes with neurons?

More Complicated Neurons


McCulloch and Pitts model


Difference from previous model: inputs are weighted.


Add weighted inputs together: if the sum is greater than a
threshold, then the neuron fires

Image taken from
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11
/report.html


More Complicated Neurons


Mathematically: neuron fires if


X
1
W
1

+ X
2
W
2

+ X
3
W
3

+ ... > T

Examples

AND Gate

XOR Gate


Image taken from http://www.heatonresearch.com



More Complicated Neurons


New model is very adaptable/powerful


Input weights and threshold can be changed so the neuron
responds differently/more accurately to a situation


Pavlov’s dog


Various algorithms adapt neurons and neural networks to
situations


Delta rule (feed
-
forward networks)


Back
-
error projection (feedback networks)

Learning


For the network to adapt, it must learn.


There are three types of learning used with neural networks:


Supervised learning


Unsupervised learning


Reinforcement learning

Supervised Learning


In supervised learning, the system learns using test data given
from an external teacher


The test data tells the system what outputs result from certain
inputs


The system tries to match the response of the test data, i.e.
minimize the error between the neural network outputs and the
test data outputs given the same inputs

Image taken from http://www.learnartificialneuralnetworks.com

Unsupervised Learning


In unsupervised learning, the network is given no output data


Instead, the network is given just input data


The goal of the network, then, is to group the input data


Example: mortgage requests


The network is given credit ratings, size of mortgage, interest
rate, etc.


The network groups the data; probably into accept and deny

Reinforcement Learning


Network performs actions on the input data


The environment grades the network (good or bad)


The network makes adjustments accordingly


Middle ground between supervised and unsupervised
learning

Conclusion


The learning aspect of neural networks makes their
applications astounding


For computers, one has to know how to solve a particular
problem


Neural networks can solve problems that one doesn’t know
how to solve

Conclusion


Just some of the uses: sales forecasting, stock market
prediction, customer research, modeling and diagnosing the
cardiovascular system, “Instant Physician”, interpretation of
multi
-
meaning Chinese words, facial recognition, etc. etc.
etc.


Something I found interesting: the interconnectedness of
different subjects



Sources


http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11
/report.html


http://www.learnartificialneuralnetworks.com/


http://www.ryerson.ca/~dgrimsha/courses/cps721/unsupervis
ed.html


http://www.willamette.edu/~gorr/classes/cs449/intro.html


http://www.statsoft.com/textbook/stneunet.html


http://www.wikipedia.org