# chapter8B

AI and Robotics

Oct 19, 2013 (4 years and 8 months ago)

84 views

www.spatialanalysisonline.com

Chapter 8

Geocomputation Part B:

Artificial Neural Networks (ANNs) &
Genetic Algorithms (GAs)

3
rd

edition

www.spatialanalysisonline.com

2

Geocomputation: ANNs

In this presentation on geocomputation:

ANNs discussed include

Multi
-
level perceptrons (MLPs)

Radial basis function neural networks (RBFNNs)

Self organising feature maps (SOFMs)

ANNs are particularly concerned with

Function approximation and interpolation

Image analysis and classification

Spatial interaction modelling

3
rd

edition

www.spatialanalysisonline.com

3

Geocomputation: Evolutionary computing

In this presentation on geocomputation:

EC elements discussed include

Genetic algorithms (GAs)

Genetic programming (GP)

EC is particularly concerned with

Complex problem solving using GAs

Model design using GP methods

3
rd

edition

www.spatialanalysisonline.com

4

Geocomputation

Artificial Neural Networks (ANNs)

A computational model based on emulating
biological neural networks

A form of non
-
linear modelling tool

Often a 3
-
layer network structure is used:

input, hidden, output

The output layer of such structures are typically
modified weighted sums of intermediate layers,
which are modified weighted sums of the input
layer

3
rd

edition

www.spatialanalysisonline.com

5

Artificial Neural Networks

Hence at each output node (hidden or
final) a two
-
step process takes place:

3
rd

edition

www.spatialanalysisonline.com

6

Artificial Neural Networks

Simple 3
-
layer feedforward ANN

Fully inter
-
connected; each connection
is given a weight, w

Known as a Multi
-
level perceptron
(MLP)

In this case: 3 input nodes, 5 hidden
nodes, 2 output nodes and 2 bias
nodes (bias, B, is similar to the
constant term in regression models)

At hidden node 1 we have:

where the
w
ij

are weights to be
determined,
b
1
=1
,

and the
x
i

are the
observed input values

3
rd

edition

www.spatialanalysisonline.com

7

Artificial Neural Networks

is simply a linear weighted
sum of the inputs. To generate
a non
-
linear output it must be
modified by some (well
behaved) non
-
linear function,
g(), e.g. the logistic function:

i.e.

Sample activation functions

3
rd

edition

www.spatialanalysisonline.com

8

Artificial Neural Networks

We can now compute the output layer values as the
weighted sum

Suppose we have known input values x
1
=1, x
2
=
-
3, x
3
=5,
and known outputs of 0 and 1. Can we select the weights
to ensure the inputs generate the known outputs?

Suggestion: <build your own worked example & program
here>

3
rd

edition

www.spatialanalysisonline.com

9

Artificial Neural Networks

Learning

Supervised learning

Split training/test data sets (control data)

Known inputs and output (target) values for training data

(Network output
-
Target output) = Error signal,
e

Systematically adjust weights to minimise sum of
e
2

descent

Used in many classification/pattern recognition applications
and in function approximation

Unsupervised learning

No training data

Must create clusters by analysing dataset for patterns/clusters

3
rd

edition

www.spatialanalysisonline.com

10

Artificial Neural Networks

Some basic issues:

local vs global minimisation

Initialisation and selection

Data normalisation and coding

Momentum

Model design and over
-
fitting

Overtraining

Interpolation vs Extrapolation/Forecasting

3
rd

edition

www.spatialanalysisonline.com

11

Artificial Neural Networks

MLP: Example 1 function approximation

source data

fitted solution curve

RMSE vs epochs

3
rd

edition

www.spatialanalysisonline.com

12

Artificial Neural Networks

MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

13

Artificial Neural Networks

MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

14

Artificial Neural Networks

MLP Example 2: LCM

3
rd

edition

www.spatialanalysisonline.com

15

Artificial Neural Networks

MLP Example 2: LCM

weights matrix

3
rd

edition

www.spatialanalysisonline.com

16

Artificial Neural Networks

MLP Example 3: Spatial interaction model

Generalised model:
T
ij
=
f
(
O
i
,
D
j
,
d
ij
)

Sample data format

(log transformed):

3
rd

edition

www.spatialanalysisonline.com

17

Artificial Neural Networks

MLP Example 3: Spatial interaction model

3
rd

edition

www.spatialanalysisonline.com

18

Artificial Neural Networks

Basic functional form:

Gaussian RBF:

3
rd

edition

www.spatialanalysisonline.com

19

Artificial Neural Networks

Self organising function maps

SOM as an output space

Neighbourhood relations

Grid size, form and topology

3
rd

edition

www.spatialanalysisonline.com

20

Artificial Neural Networks

Self organising function maps

Dimensional reductions

Mapped output

similar vectors (units) are
close to each other

Typically an unsupervised procedure

Spatial mapping of SOM can follow using
simple assignment to best matching unit (BMU)

3
rd

edition

www.spatialanalysisonline.com

21

Artificial Neural Networks

Self organising function maps

Choose a grid size, form and topology

Train the network

Identify the best matching units

Modify the BMU and its neighbours (spatially biased
learning rule)

Map the trained network

3
rd

edition

www.spatialanalysisonline.com

22

Artificial Neural Networks

Self organising function maps

some issues

Initialisation

Pre
-
processing

Normalisation

Missing data

Learning and tuning

Distance metrics

Neighbourhood functions (kernels)

Learning rate functions

3
rd

edition

www.spatialanalysisonline.com

23

Artificial Neural Networks

Self organising function maps

Idrisi

3
rd

edition

www.spatialanalysisonline.com

24

Artificial Neural Networks

Self organising function maps

Idrisi

3
rd

edition

www.spatialanalysisonline.com

25

Genetic Algorithms

Solutions are represented as individuals

Individuals are modelled as
chromosomes

Chromosomes are comprised of
genes

Genes have values known as
alleles

Chromosomes have a measurable
fitness

New chromosomes (children) are created by
reproduction

and
mutation

processes

The fittest individuals survive

The creation process is then iterated

3
rd

edition

www.spatialanalysisonline.com

26

Genetic Algorithms

GAs: Example 1
-

TSP

chromosome

genes

allele=12 (ID of town in TSP problem set)

Each chromosome contains complete list of towns

create a set of m randomly permuted strings and compute lengths, d

evaluate the fitness of each string (e.g. 1/d)

select random pairs of tours (biased by fitness)

combine pairs by crossover operation

evaluate fitness of offspring

apply replacement rule (fittest retained) and iterate till stable

3
rd

edition

www.spatialanalysisonline.com

27

Genetic Algorithms

GA components

Encoding or representation

binary, list, tree etc

Fitness function selection

use of rank transforms

Population initialisation

Selection: roulette, tournament, uniform random

Reproduction

Crossover e.g.
A =
[a b c d e f g h]

B =
[1 2 3 4 5 6 7 8]

and the crossover point is 3, the following children are generated:

child 1 =
[a b c 4 5 6 7 8]

child 2=
[1 2 3 d e f g h]

Mutation

Local search

Termination

3
rd

edition

www.spatialanalysisonline.com

28

Genetic Algorithms

GAs: application areas

TSP (as above)

Clustering

Map labelling

Optimum location with capacity constraints

Concept can be extended to alleles that are
expressions or program elements rather than
numerical values

Genetic programming