Supervised Learning: Perceptrons and Backpropagation

appliancepartΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

86 εμφανίσεις

Supervised Learning:
Perceptrons

and
Backpropagation


Connectionist /ism==


Parallel Distributed Processing (PDP)


Intelligence is emergent


Used to train multilayer
feedforward

networks


Used to train multilayer
feedforward

networks


Assumes a continuous activation function


Used to train multilayer
feedforward

networks


Assumes a continuous activation function


Delta rule


Perceptron

update rule was:




Backprop

update rule is:



Error of an output node:





Error of a hidden node:





















demo


Encoding / Feature Extraction


# neurons used


# layers used


Output mapping


Classification


Classification


Pattern Recognition


Classification


Pattern Recognition


Content Addressable Memory


Classification


Pattern Recognition


Content Addressable Memory


Prediction


Classification


Pattern Recognition


Content Addressable Memory


Prediction


Optimization


Classification


Pattern Recognition


Content Addressable Memory


Prediction


Optimization


Filtering


Degrade gracefully


Degrade gracefully


Solve ill
-
defined problems


Degrade gracefully


Solve ill
-
defined problems


Flexible


Degrade gracefully


Solve ill
-
defined problems


Flexible


Generalization


Time & Memory


Time & Memory


Black box


Time & Memory


Black box


Trial and Error


If you can draw a flow chart or formula


If you can draw a flow chart or formula


If a piece of hardware or software already
exists that does what you want


If you can draw a flow chart or formula


If a piece of hardware or software already
exists that does what you want


If you want to functionality to evolve


If you can draw a flow chart or formula


If a piece of hardware or software already
exists that does what you want


If you want to functionality to evolve


Precise answers are required


If you can draw a flow chart or formula


If a piece of hardware or software already
exists that does what you want


If you want to functionality to evolve


Precise answers are required


The problem could be described in a lookup
table


You can define a correct answer


You can define a correct answer


You have a lot of training data with examples
of right and wrong answers


You can define a correct answer


You have a lot of training data with examples
of right and wrong answers


You have lots of data but can’t figure how to
map it to output


You can define a correct answer


You have a lot of training data with examples
of right and wrong answers


You have lots of data but can’t figure how to
map it to output


The problem is complex but solvable


You can define a correct answer


You have a lot of training data with examples
of right and wrong answers


You have lots of data but can’t figure how to
map it to output


The problem is complex but solvable


The solution is fuzzy or might change slightly


2007
Rechnender

Raum’s

Inverted Machine


Jonathan McCabe’s


Nervous States

2006


Each pixel is the

Output state of a

Neural network given

Different inputs


2007 Phillip Stearns

AANN: Artificial Analog Neural Network


Ted?