chapter8B

apricotpigletΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

41 εμφανίσεις

www.spatialanalysisonline.com

Chapter 8

Geocomputation Part B:

Artificial Neural Networks (ANNs) &
Genetic Algorithms (GAs)

3
rd

edition

www.spatialanalysisonline.com

2

Geocomputation: ANNs

In this presentation on geocomputation:

ANNs discussed include


Multi
-
level perceptrons (MLPs)


Radial basis function neural networks (RBFNNs)


Self organising feature maps (SOFMs)

ANNs are particularly concerned with


Function approximation and interpolation


Image analysis and classification


Spatial interaction modelling

3
rd

edition

www.spatialanalysisonline.com

3

Geocomputation: Evolutionary computing

In this presentation on geocomputation:

EC elements discussed include


Genetic algorithms (GAs)


Genetic programming (GP)

EC is particularly concerned with


Complex problem solving using GAs


Model design using GP methods

3
rd

edition

www.spatialanalysisonline.com

4

Geocomputation



Artificial Neural Networks (ANNs)


A computational model based on emulating
biological neural networks


A form of non
-
linear modelling tool


Often a 3
-
layer network structure is used:

input, hidden, output


The output layer of such structures are typically
modified weighted sums of intermediate layers,
which are modified weighted sums of the input
layer


3
rd

edition

www.spatialanalysisonline.com

5

Artificial Neural Networks


Hence at each output node (hidden or
final) a two
-
step process takes place:

3
rd

edition

www.spatialanalysisonline.com

6

Artificial Neural Networks


Simple 3
-
layer feedforward ANN


Fully inter
-
connected; each connection
is given a weight, w


Known as a Multi
-
level perceptron
(MLP)


In this case: 3 input nodes, 5 hidden
nodes, 2 output nodes and 2 bias
nodes (bias, B, is similar to the
constant term in regression models)


At hidden node 1 we have:




where the
w
ij

are weights to be
determined,
b
1
=1
,

and the
x
i

are the
observed input values


3
rd

edition

www.spatialanalysisonline.com

7

Artificial Neural Networks


is simply a linear weighted
sum of the inputs. To generate
a non
-
linear output it must be
modified by some (well
behaved) non
-
linear function,
g(), e.g. the logistic function:




i.e.


Sample activation functions

3
rd

edition

www.spatialanalysisonline.com

8

Artificial Neural Networks


We can now compute the output layer values as the
weighted sum





Suppose we have known input values x
1
=1, x
2
=
-
3, x
3
=5,
and known outputs of 0 and 1. Can we select the weights
to ensure the inputs generate the known outputs?




Suggestion: <build your own worked example & program
here>


3
rd

edition

www.spatialanalysisonline.com

9

Artificial Neural Networks



Learning


Supervised learning


Split training/test data sets (control data)


Known inputs and output (target) values for training data


(Network output
-
Target output) = Error signal,
e


Systematically adjust weights to minimise sum of
e
2


Adjustment typically based on backpropagation and gradient
descent


Used in many classification/pattern recognition applications
and in function approximation


Unsupervised learning


No training data


Must create clusters by analysing dataset for patterns/clusters

3
rd

edition

www.spatialanalysisonline.com

10

Artificial Neural Networks


Some basic issues:


local vs global minimisation


Initialisation and selection


Data normalisation and coding


Momentum


Model design and over
-
fitting


Overtraining


Interpolation vs Extrapolation/Forecasting


3
rd

edition

www.spatialanalysisonline.com

11

Artificial Neural Networks


MLP: Example 1 function approximation

source data

fitted solution curve

RMSE vs epochs

3
rd

edition

www.spatialanalysisonline.com

12

Artificial Neural Networks



MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

13

Artificial Neural Networks



MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

14

Artificial Neural Networks



MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

15

Artificial Neural Networks



MLP Example 2: LCM

weights matrix

3
rd

edition

www.spatialanalysisonline.com

16

Artificial Neural Networks



MLP Example 3: Spatial interaction model


Generalised model:
T
ij
=
f
(
O
i
,
D
j
,
d
ij
)


Sample data format

(log transformed):

3
rd

edition

www.spatialanalysisonline.com

17

Artificial Neural Networks



MLP Example 3: Spatial interaction model

3
rd

edition

www.spatialanalysisonline.com

18

Artificial Neural Networks



Radial Basis Function Networks

Basic functional form:



Gaussian RBF:

3
rd

edition

www.spatialanalysisonline.com

19

Artificial Neural Networks



Self organising function maps


SOM as an output space


Neighbourhood relations


Grid size, form and topology

3
rd

edition

www.spatialanalysisonline.com

20

Artificial Neural Networks



Self organising function maps


Dimensional reductions


Mapped output


similar vectors (units) are
close to each other


Typically an unsupervised procedure


Spatial mapping of SOM can follow using
simple assignment to best matching unit (BMU)

3
rd

edition

www.spatialanalysisonline.com

21

Artificial Neural Networks



Self organising function maps


Choose a grid size, form and topology


Train the network


Identify the best matching units


Modify the BMU and its neighbours (spatially biased
learning rule)


Map the trained network

3
rd

edition

www.spatialanalysisonline.com

22

Artificial Neural Networks



Self organising function maps


some issues


Initialisation


Pre
-
processing


Normalisation


Missing data


Masking and weighting


Learning and tuning


Distance metrics


Neighbourhood functions (kernels)


Learning rate functions

3
rd

edition

www.spatialanalysisonline.com

23

Artificial Neural Networks



Self organising function maps


Idrisi

3
rd

edition

www.spatialanalysisonline.com

24

Artificial Neural Networks



Self organising function maps


Idrisi

3
rd

edition

www.spatialanalysisonline.com

25

Genetic Algorithms



Solutions are represented as individuals


Individuals are modelled as
chromosomes


Chromosomes are comprised of
genes


Genes have values known as
alleles


Chromosomes have a measurable
fitness


New chromosomes (children) are created by
reproduction

and
mutation

processes


The fittest individuals survive


The creation process is then iterated

3
rd

edition

www.spatialanalysisonline.com

26

Genetic Algorithms


GAs: Example 1
-

TSP

chromosome

genes

allele=12 (ID of town in TSP problem set)

Each chromosome contains complete list of towns



create a set of m randomly permuted strings and compute lengths, d



evaluate the fitness of each string (e.g. 1/d)



select random pairs of tours (biased by fitness)



combine pairs by crossover operation



evaluate fitness of offspring



apply replacement rule (fittest retained) and iterate till stable

3
rd

edition

www.spatialanalysisonline.com

27

Genetic Algorithms



GA components


Encoding or representation


binary, list, tree etc


Fitness function selection


use of rank transforms


Population initialisation


Selection: roulette, tournament, uniform random


Reproduction


Crossover e.g.
A =
[a b c d e f g h]

B =
[1 2 3 4 5 6 7 8]



and the crossover point is 3, the following children are generated:



child 1 =
[a b c 4 5 6 7 8]

child 2=
[1 2 3 d e f g h]



Mutation


Local search


Termination

3
rd

edition

www.spatialanalysisonline.com

28

Genetic Algorithms



GAs: application areas


TSP (as above)


Clustering


Map labelling


Optimum location with capacity constraints


Concept can be extended to alleles that are
expressions or program elements rather than
numerical values


Genetic programming