www.spatialanalysisonline.com
Chapter 8
Geocomputation Part B:
Artificial Neural Networks (ANNs) &
Genetic Algorithms (GAs)
3
rd
edition
www.spatialanalysisonline.com
2
Geocomputation: ANNs
In this presentation on geocomputation:
ANNs discussed include
Multi

level perceptrons (MLPs)
Radial basis function neural networks (RBFNNs)
Self organising feature maps (SOFMs)
ANNs are particularly concerned with
Function approximation and interpolation
Image analysis and classification
Spatial interaction modelling
3
rd
edition
www.spatialanalysisonline.com
3
Geocomputation: Evolutionary computing
In this presentation on geocomputation:
EC elements discussed include
Genetic algorithms (GAs)
Genetic programming (GP)
EC is particularly concerned with
Complex problem solving using GAs
Model design using GP methods
3
rd
edition
www.spatialanalysisonline.com
4
Geocomputation
Artificial Neural Networks (ANNs)
A computational model based on emulating
biological neural networks
A form of non

linear modelling tool
Often a 3

layer network structure is used:
input, hidden, output
The output layer of such structures are typically
modified weighted sums of intermediate layers,
which are modified weighted sums of the input
layer
3
rd
edition
www.spatialanalysisonline.com
5
Artificial Neural Networks
Hence at each output node (hidden or
final) a two

step process takes place:
3
rd
edition
www.spatialanalysisonline.com
6
Artificial Neural Networks
Simple 3

layer feedforward ANN
Fully inter

connected; each connection
is given a weight, w
Known as a Multi

level perceptron
(MLP)
In this case: 3 input nodes, 5 hidden
nodes, 2 output nodes and 2 bias
nodes (bias, B, is similar to the
constant term in regression models)
At hidden node 1 we have:
where the
w
ij
are weights to be
determined,
b
1
=1
,
and the
x
i
are the
observed input values
3
rd
edition
www.spatialanalysisonline.com
7
Artificial Neural Networks
is simply a linear weighted
sum of the inputs. To generate
a non

linear output it must be
modified by some (well
behaved) non

linear function,
g(), e.g. the logistic function:
i.e.
Sample activation functions
3
rd
edition
www.spatialanalysisonline.com
8
Artificial Neural Networks
We can now compute the output layer values as the
weighted sum
Suppose we have known input values x
1
=1, x
2
=

3, x
3
=5,
and known outputs of 0 and 1. Can we select the weights
to ensure the inputs generate the known outputs?
Suggestion: <build your own worked example & program
here>
3
rd
edition
www.spatialanalysisonline.com
9
Artificial Neural Networks
Learning
Supervised learning
Split training/test data sets (control data)
Known inputs and output (target) values for training data
(Network output

Target output) = Error signal,
e
Systematically adjust weights to minimise sum of
e
2
Adjustment typically based on backpropagation and gradient
descent
Used in many classification/pattern recognition applications
and in function approximation
Unsupervised learning
No training data
Must create clusters by analysing dataset for patterns/clusters
3
rd
edition
www.spatialanalysisonline.com
10
Artificial Neural Networks
Some basic issues:
local vs global minimisation
Initialisation and selection
Data normalisation and coding
Momentum
Model design and over

fitting
Overtraining
Interpolation vs Extrapolation/Forecasting
3
rd
edition
www.spatialanalysisonline.com
11
Artificial Neural Networks
MLP: Example 1 function approximation
source data
fitted solution curve
RMSE vs epochs
3
rd
edition
www.spatialanalysisonline.com
12
Artificial Neural Networks
MLP Example 2: LCM
3
rd
edition
www.spatialanalysisonline.com
13
Artificial Neural Networks
MLP Example 2: LCM
3
rd
edition
www.spatialanalysisonline.com
14
Artificial Neural Networks
MLP Example 2: LCM
3
rd
edition
www.spatialanalysisonline.com
15
Artificial Neural Networks
MLP Example 2: LCM
weights matrix
3
rd
edition
www.spatialanalysisonline.com
16
Artificial Neural Networks
MLP Example 3: Spatial interaction model
Generalised model:
T
ij
=
f
(
O
i
,
D
j
,
d
ij
)
Sample data format
(log transformed):
3
rd
edition
www.spatialanalysisonline.com
17
Artificial Neural Networks
MLP Example 3: Spatial interaction model
3
rd
edition
www.spatialanalysisonline.com
18
Artificial Neural Networks
Radial Basis Function Networks
Basic functional form:
Gaussian RBF:
3
rd
edition
www.spatialanalysisonline.com
19
Artificial Neural Networks
Self organising function maps
SOM as an output space
Neighbourhood relations
Grid size, form and topology
3
rd
edition
www.spatialanalysisonline.com
20
Artificial Neural Networks
Self organising function maps
Dimensional reductions
Mapped output
–
similar vectors (units) are
close to each other
Typically an unsupervised procedure
Spatial mapping of SOM can follow using
simple assignment to best matching unit (BMU)
3
rd
edition
www.spatialanalysisonline.com
21
Artificial Neural Networks
Self organising function maps
Choose a grid size, form and topology
Train the network
Identify the best matching units
Modify the BMU and its neighbours (spatially biased
learning rule)
Map the trained network
3
rd
edition
www.spatialanalysisonline.com
22
Artificial Neural Networks
Self organising function maps
–
some issues
Initialisation
Pre

processing
Normalisation
Missing data
Masking and weighting
Learning and tuning
Distance metrics
Neighbourhood functions (kernels)
Learning rate functions
3
rd
edition
www.spatialanalysisonline.com
23
Artificial Neural Networks
Self organising function maps
–
Idrisi
3
rd
edition
www.spatialanalysisonline.com
24
Artificial Neural Networks
Self organising function maps
–
Idrisi
3
rd
edition
www.spatialanalysisonline.com
25
Genetic Algorithms
Solutions are represented as individuals
Individuals are modelled as
chromosomes
Chromosomes are comprised of
genes
Genes have values known as
alleles
Chromosomes have a measurable
fitness
New chromosomes (children) are created by
reproduction
and
mutation
processes
The fittest individuals survive
The creation process is then iterated
3
rd
edition
www.spatialanalysisonline.com
26
Genetic Algorithms
GAs: Example 1

TSP
chromosome
genes
allele=12 (ID of town in TSP problem set)
Each chromosome contains complete list of towns
•
create a set of m randomly permuted strings and compute lengths, d
•
evaluate the fitness of each string (e.g. 1/d)
•
select random pairs of tours (biased by fitness)
•
combine pairs by crossover operation
•
evaluate fitness of offspring
•
apply replacement rule (fittest retained) and iterate till stable
3
rd
edition
www.spatialanalysisonline.com
27
Genetic Algorithms
GA components
Encoding or representation
–
binary, list, tree etc
Fitness function selection
–
use of rank transforms
Population initialisation
Selection: roulette, tournament, uniform random
Reproduction
Crossover e.g.
A =
[a b c d e f g h]
B =
[1 2 3 4 5 6 7 8]
and the crossover point is 3, the following children are generated:
child 1 =
[a b c 4 5 6 7 8]
child 2=
[1 2 3 d e f g h]
Mutation
Local search
Termination
3
rd
edition
www.spatialanalysisonline.com
28
Genetic Algorithms
GAs: application areas
TSP (as above)
Clustering
Map labelling
Optimum location with capacity constraints
Concept can be extended to alleles that are
expressions or program elements rather than
numerical values
Genetic programming
Comments 0
Log in to post a comment