Artificial Neural Networks

chickenchairwomanAI and Robotics

Oct 19, 2013 (3 years and 7 months ago)


Artificial Neural Networks


Artificial Neural Networks

First proposed in 1940s as an attempt to
simulate the human brain’s cognitive learning

They have ability to model complex, yet
poorly understood problems.

ANNs are simple computer
based programs
whose function is to model a problem space
based on trial and error.

Fundamentals of Neural Computing

The basic processing element in the human
nervous system is the neuron. Networks of
these interconnected cells receive information
from sensors in the eye, ear, etc.

Information received by a neuron will either
excite it (and it will pass a message along the
network) or will inhibit it (suppressing
information flow).

Sensitivity can change with passing of time or
gaining of experience.

Biological Neural Networks


is a cell

Neural Network

network of neurodes

An ANN is composed of three basic layers:

The input layer receives the data

The internal or hidden layer processes the data.

The output layer relays the final result of the net.

See’s Artificial Neural Networks article

Inside the Neurode

The neurode usually has multiple inputs,
each input with its own weight or importance.

A bias input can be used to amplify the

The state function consolidates the weights of
the various inputs into a single value
(weighted sum).

The transfer (also called activation) function
processes this state value and makes the
output (sigmoid).

The neurode (simulation of a neuron)


is called an activation function

Typically a
sigmoid function

Sigmoid Activation Functions

Activation function normalizes outputs to between 0 and 1

1 and 1), and also fosters the ability to handle nonlinear


Types of Neural Networks


No hidden layers

Can only learn
linearly separable


Cannot learn nonlinear functions

Multilayer feed
forward networks

Includes at least one hidden layer

Uses backpropagation to modify neuron

Can learn nonlinear functions

Types of learning


learning paradigms, the ANN
receives input data but not any feedback about
desired results. It develops clusters of the
training records based on data similarities.

In a

learning paradigm, the ANN
gets to compare its guess to feedback
containing the desired results. The most
common of these is
, which
does the comparison with squared errors.

Learning From Experience

The process is:

A piece of data is presented to a neural
net. The ANN “guesses” an output.

The prediction is compared with the
actual or correct value. If the guess was
correct, no action is taken.

An incorrect guess causes the ANN to
examine itself to determine which
parameters to adjust.

Another piece of data is presented and
the process is repeated.

Training the Artificial Neural Network
via Backpropagation


Starting with (random? best guess?) weights…

Compute outputs

Calculate errors

Correct weights

Benefits Associated with Neural

Avoidance of explicit programming

Reduced need for experts

ANNs are adaptable to changed inputs

No need for refined knowledge base

ANNs are dynamic and improve with use

Able to process erroneous or incomplete data

Allows for generalization from specific info

Allows inclusion of common sense into the
solving domain

Limitations Associated with

Neural Computing

ANNs cannot “explain” their inference

The “black box” nature makes accountability
and reliability issues difficult

Repetitive training process is time consuming

Highly skilled machine learning analysts and
designers are still a scarce resource

ANN technology pushes the limits of current

ANN require “faith” be imparted to the output

Microsoft’s Neural Network Algorithm

Input layer

One input neuron for each state (possible value) of discrete
input attributes in the model plus one for missing (null) value

Two input neurons for continuous input attributes (one for
the continuous value and one for a missing value)

Uses a single hidden layer

You can control how many nodes are in this layer

Output layer

One neuron for each state of the predictable attribute