a brief intro
artificial neural networks
commonly referred to as
human brain computes in an entirely different
manner from a conventional computer.
Neural networks are an attempt to
mimic the human
brain’s nonlinear and parallel processing capability
applications such pattern recognition, classification,
data mining, speech recognition, etc.
Similar to the human brain, a neural network acquires
knowledge through a
. The inter
neuron connection strengths known as synaptic weights
are used to store the knowledge.
Neural networks model problems for which an explicit
mathematical relationship is not known.
Neural networks learn relationships
human brain. But
neural networks certainly do NOT
Neither will they ever be a
Neural networks have been used to solve numerous
disciplines (Computer Science,
Mathematics, Physics, Medicine, Marketing, Electronics,
How they work
Neural networks are composed of simple elements
nspired by biological nervous systems. As in nature, the
is determined by the connections between
train a neural network to perform a particular
adjusting the values
of the connections
) between elements.
eural networks are
, so that a particular input
leads to a
specific target output.
because they learn by example.
from their training data to other 'new'
That is, the ability to interpolate from a previous
e.g. broomstick example.
That is, they can produce a reasonably
correct output from noisy and incomplete data, whereas
conventional computers usually require correct data.
because many interconnected processing units work in
Again modeled on the human brain, which may
contain a 100 billion neurons.
Note that most of these are
not replaced as you get older!
, but computationally intensive to
problems whose solution is
and difficult to specify, but which provides an
abundance of data from which a response can be learnt.
Can be trained to generate non
work on real world problems!
e.g predicting the weather.
magically create information
that is not
contained in the training data
In the above figure, t
network is adjusted, based on a comparison
of the output and the target, until
the network output matches the
any such input/target pairs
are used, in this
train a network.
Stock Value Prediction
When should Neural Networks be used
The solution to the problem
cannot be explicitly described
algorithm, a set of equations, or a set of rules
There is some evidence that an
output relationship exists
between a set of input variables
and corresponding output data
There should be a
large amount of data
available to train the
In practice, NNs are especially useful for classification and
function approximation problems
Problems which can lead to poor performance
data do not represent
the network will encounter in practice.
The main factors are not present in the available data.
E.g. trying to
the loan application without
having knowledge of the applicant's salaries.
The network is required to implement a very complex
The two main kinds of learning algorithms are supervised
, the correct results (
and are given to the NN during
training so that the NN can adjust its weights to try match
its outputs to the target values. After training, the NN is
tested by giving it only input values, not target values, and
seeing how close it comes to outputing the correct target
, the correct results
during training. Unsupervised NNs usually perform
some kind of data compression, such as dimensionality
reduction or clustering.
Two major kinds of network topology are feedforward and
, the connections between units do
not form cycles. Feedforward NNs usually produce a
response to an input quickly. Most feedforward NNs can be
trained using a wide variety of
numerical methods in addition to algorithms invented by
feedback or recurrent NN
, there are cycles in the
connections. In some feedback NNs, each time an input is
presented, the NN must iterate for a potentially long time
before it produces a response. Feedback NNs are usually
more difficult to train than feedforward NNs.
NNs also differ in the kinds of data they accept. Two major kinds of
data are categorical and quantitative.
take only a finite number of possible values, and
there are usually several or more cases falling into each category.
Categorical variables may have symbolic values (
e.g., "male" and
"female", or "red", "green" and "blue"
) that must be encoded into
numbers before being given to the network
are numerical measurements of some attribute,
in meters. The measurements must be made in such a
way that at least some arithmetic relations among the measurements
reflect analogous relations among the attributes of the objects that
Stages of solving problems
using Neural Networks
a neural network solution are:
: The collection, preparation and analysis of
the training data
etworks very rarely
operate on the raw data. An initial pre
stage is essential
: The design, training and testing of the
The quality of the neural network will therefore critically depend on
the quality and quantity of the training data.
Neural networks are great for data
fusion i.e. combining different
types of data from different sources. For example the state of a motor
could be determined by a network trained on measurements of sound,
temperature, vibrations and flow rate of the lubricant.
applications, require target data and
Preprocessing of the raw data is a very important stage
in the data set. An outlier is an unusually large or
sufficient number of training examples to ensure that
the neural network is trained to recognize and respond to the full
of Neural Networks
The aim is to perform a specific mapping between input
and output. In order to do this we need to:
number of layers
in the network
number of neurons
in each layer.
for each layer.
Train the network
to give the required mapping
(training means calculate the weights and biases that
will do that).
Check that the
in Neural Networks
In training, we calculate the weights of the network by using a number of
input and output pairs (this means that for specific inputs we know
what the output should be and we provide this information to the
network so that we will make it “learn”).
The network however, must be able to
training (otherwise the training is not correct).
in Neural Networks
In order to achieve good generalisation, the training set (=the pairs of
known inputs and outputs) must be well designed.
In a well designed training set
, the training inputs must:
(=cover) the whole range of likely inputs to the network
The training inputs are sufficiently
over the range of
possible inputs to allow accurate interpolation (=calculation of
intermediate unknown values)
Can extend definition of ‘feature set’ to
allow classification of ‘face’ objects by
gender, age, race, mood, ...
‘Poverty map’ based on 39 indicators from World Bank statistics (1992)
Software to use
Software to use
Software to use
The Evolutionary Cycle
The evolutionary cycle
(CHARACTERISTICS OF EAs)
EAs can quickly locate areas of high quality solutions when the
domain is very large or complex (GAS can quickly explore huge
search spaces and find those regions that are more likely to
contain the solution)
Can scan through the global space simultaneously instead of
restricting themselves to localized regions of gradient shifts
Require little knowledge of the problem to be solved
Slow in LOCAL FINE TUNING in comparison to
Hybrid training can speed up convergence
Advantages & Disadvantages of GAs
GAs are good at optimisation
Evolutionary Computation & ANNs
VERY ATTRACTIVE IDEA
architecture) can be evolved
with little human interaction
, because they combine the learning
power of Nnets and the adaptive capabilities of evolutionary processes.
Thank You for Your Attention