page
1
M. Muraszkiewicz
Intelligent Information
Systems
Prof. M. Muraszkiewicz
Institute
of Information
and Book Studies
Warsaw University
mietek@n

s.pl
page
2
M. Muraszkiewicz
Neural Nets
Module 10
page
3
M. Muraszkiewicz
1.
Background
2.
Historical Note
3.
Definition
4.
Properties and Applications
Table of Contents
page
4
M. Muraszkiewicz
Background
page
5
M. Muraszkiewicz
Two Tracks in AI
Analytical, symbolic
Invented by researchers
(
inspired by logics and math
–
J. von Neumann)
.
„
Naturalistic
”
Based on solutions worked
out by “mother nature”
through evolution
(
inspired by psychology, neurology, biology,
evolution
–
K. Darwin, ...)
.
page
6
M. Muraszkiewicz
About the Human Brain
“
If the human brain
were so simple that
we could understand
it, we would be so
simple that we
couldn’t.
”
Emmerson M. Pough
page
7
M. Muraszkiewicz
Parameters
volume
:
~
1400 cm3,
weight
:
~
1,5 kG,
surface
:
~
2000 cm2
(the surface of a sphere of
the same volume is
~
600 cm2)
,
~
10
10
neurons
,
10
12
glia cells
,
number of connections

~
10
15
average length from
0,01 mm
to
1m.
Neurons receive and send impulses whose frequency is
1

100 Hz,
duration
1

2 ms,
voltage
100 mV
and
speed of
propagation
1

100 m/s.
Speed of brain
–
10
18
operations
/s (
parallel processing
).
Informational capacity of senses’ channels
:

vision

100 Mb/s,

touch

1 Mb/s,

audition

15 Kb/s,

smell

1 Kb/s,

taste

100 b/s. (
source
R. Tadeusiewicz, „Sieci neuronowe”).
page
8
M. Muraszkiewicz
Historical Note
page
9
M. Muraszkiewicz
Difficult History
W. McCulloch, W. Pitts
–
first mathematical model
of a neuron
(1943),
D. Hebb
–
the rule that
determines the change in
the weight connection,
F. Rosenblatt
’s
Perceptron
(1957)
, a two

layer
network,
for recognizing alphanumerical
characters
,
B. Widrow, M. Hoff
–
ADALINE
M. Minsky (1969)
–
proved limits of simple neural
nets which weakened research in the 70’ies
,
J. Hopfield
’s Net
with a feedback
(1982),
Works by
J. Andersona (1988)
–
neural nets’
“
comeback
".
Warren McCulloch
1898

1969
page
10
M. Muraszkiewicz
Definition
page
11
M. Muraszkiewicz
Intuitive
Definition
“A neural network is a set of simple
processors (“neurons”) connected in a certain
way
.
A neuron can have many inputs
(synapses) with which weights can be
associated.
The value of weights can be changed during
the operation of a network to produce the
desired data flow within it what makes the
network and adaptive device.
Topology of the network and the values of
weights determine the program executed on
the network
.
page
12
M. Muraszkiewicz
Definition from Wikipedia
“
An artificial neural network (ANN),
often just called a "neural network"
(NN), is an interconnected group of
artificial neurons that uses a
mathematical model or computational
model for information processing
based on a connectionist approach to
computation
”
http://en.wikipedia.org/wiki/Artificial_neural_network
page
13
M. Muraszkiewicz
Types of Nets
The neurons learn in an iterative
way.
By adding an error detector and a
feature to change weights simple
nets become to new models such
as
ADALINE
(ADAptive LINear
Element).
F
eedforward
Linear
One

layer
other
With a
feedback
(Hopfield)
Non

linear
Multi

layer
page
14
M. Muraszkiewicz
Properties
and Applications
page
15
M. Muraszkiewicz
Main Properties
Advantages
adaptiveness and self

organization
parallel processing
,
learning (supervised and unsupervised)
fault tolerance
Disadvantages
non

explicability
slow
page
16
M. Muraszkiewicz
Type of Applications
prediction
optimization
classification
pattern and sequence recognition
data analysis and association
,
filtering
...
page
17
M. Muraszkiewicz
Examples of Applications
Diagnostics of electronic
devices
Psychiatric research
Stock exchange predictions
Sales predictions
Search for oil fields
Interpretation of biological
research
Prices prediction
Analysis of medical data
Planning of machines
maintenance
Speech analysis
Planning of learning progress
Analysis of production problems
Trade activities optimization
Spectral analysis
Optimization of wastes utilization
Selection of row materials
Forensic support
Staff recruitment support
Industrial processes control
...
page
18
M. Muraszkiewicz
Readings
Haykin S., “
Neural Networks: A Comprehensive
Foundation
”
(3rd Edition)
,
Prentice Hall
, 2007.
Lawrence, J
., “
Introduction to Neural Networks
”
,
California Scientific Software Press
, 1994.
Royas R., “
Neural Networks: A Systematic Introduction
”,
Springer, 1996.
http://en.wikipedia.org/wiki/
Neural_networks
http://en.wikipedia.org/wiki/Artificial_neural_network
page
19
M. Muraszkiewicz
Comments 0
Log in to post a comment