# Neural Network Design

AI and Robotics

Oct 19, 2013 (4 years and 8 months ago)

129 views

1
Class #29 Updated 11/2/2001 2:21 PM page 1
B. M. Wilamowski - Neural Network Design
Neural Network Design
EE-578 ME-578 class # 29
• Problems with Kohonen WTA network
• Mountain Clustering
• Forming clusters as needed
Class #29 Updated 11/2/2001 2:21 PM page 2
B. M. Wilamowski - Neural Network Design
Kohonen Network Problems
1.Important information about length of the vector is
lost during the normalization process
2.Clustering depends from:
a) Order patterns are applied
b) Number of initial neurons
c) Initial weights
2
Class #29 Updated 11/2/2001 2:21 PM page 3
B. M. Wilamowski - Neural Network Design
Kohonen Network Problems
Important information about length of the vector is lost during the normalization process
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
Class #29 Updated 11/2/2001 2:21 PM page 4
B. M. Wilamowski - Neural Network Design
Kohonen Network Problems
Important information about length of the vector is
lost during the normalization process
The problem can be solved by increasing a dimension by one and usage
of vector angles as variables. Lengths are the same.
This approach (used by Kohonen) leads to complex trigonometric
computations
Other way to approach the problem is to project patterns into
hypersphere of higher dimensionality. This way all patterns have the
same length but important information is not lost.
3
Class #29 Updated 11/2/2001 2:21 PM page 5
B. M. Wilamowski - Neural Network Design
Input pattern transformation on a hypersphere
xc3
xc1
xc2
xe1
xe2
xe3
-10
-5
0
5
10
-10
-5
0
5
10
-1
-0.5
0
0.5
1
R
2
2
− x
x
1
x
2
x
n
z
1
z
2
z
n+1
z
n
.
.
.
Class #29 Updated 11/2/2001 2:21 PM page 6
B. M. Wilamowski - Neural Network Design
ART
Copied form Jacek Zurada “Artificial Neural Systems” West Publishing Company 1992
4
Class #29 Updated 11/2/2001 2:21 PM page 7
B. M. Wilamowski - Neural Network Design
ART
Copied form Jacek Zurada “Artificial Neural Systems” West Publishing Company 1992
mess!
Class #29 Updated 11/2/2001 2:21 PM page 8
B. M. Wilamowski - Neural Network Design
Mountain clustering
Superposition of Gaussian hills
1.For each pattern a small hill of a bell
(Gaussian) shape is formed.
2.These hills are forming mountains.
3.The highest mountains are the clusters
This is very computationally intensive process
5
Class #29 Updated 11/2/2001 2:21 PM page 9
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
much simpler and more efficient then ART
1. First pattern is applied and the cluster is formed
2. Next pattern is applied and then:
a) If distance form all existing clusters is larger then threshold
then a new cluster is formed
b) Else weights of the closest cluster are updated
1+
+
=
m
m
k
k
XW
W
α
where m is the number of previous patterns of a given set which
were used to update this particular neuron and α is the learning
constant
Class #29 Updated 11/2/2001 2:21 PM page 10
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
First pattern applied and the first cluster is formed
5.9630 0.7258
4.1168 2.9694
1.8184 6.0148
6.2139 2.4288
6.1290 1.3876
1.0562 5.8288
4.3185 2.3792
2.6108 5.4870
1.5999 4.1317
1.1046 4.1969
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
5.9630 0.7258
6
Class #29 Updated 11/2/2001 2:21 PM page 11
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
Second pattern applied and the first cluster is moved
5.9630 0.7258
4.1168 2.9694
1.8184 6.0148
6.2139 2.4288
6.1290 1.3876
1.0562 5.8288
4.3185 2.3792
2.6108 5.4870
1.5999 4.1317
1.1046 4.1969
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
ED=2.9056
5.3476 1.4737 5.9630 0.7258

4.1168 2.9694
Class #29 Updated 11/2/2001 2:21 PM page 12
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
Third pattern applied and a new cluster is formed
5.9630 0.7258
4.1168 2.9694
1.8184 6.0148
6.2139 2.4288
6.1290 1.3876
1.0562 5.8288
4.3185 2.3792
2.6108 5.4870
1.5999 4.1317
1.1046 4.1969
ED=5.7513
5.3476 1.4737
1.8184 6.0148
5.3476 1.4737

0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
1.8184 6.0148
7
Class #29 Updated 11/2/2001 2:21 PM page 13
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
Fourth pattern applied and the first cluster is moved
5.9630 0.7258
4.1168 2.9694
1.8184 6.0148
6.2139 2.4288
6.1290 1.3876
1.0562 5.8288
4.3185 2.3792
2.6108 5.4870
1.5999 4.1317
1.1046 4.1969
ED=1.2895 5.6727
5.3476 1.4737
1.8184 6.0148
5.5642 1.7125
1.8184 6.0148

0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
6.2139 2.4288
Class #29 Updated 11/2/2001 2:21 PM page 14
B. M. Wilamowski - Neural Network Design
Forming clusters as needed using
minimum distance concept
After all 10 patterns
5.9630 0.7258
4.1168 2.9694
1.8184 6.0148
6.2139 2.4288
6.1290 1.3876
1.0562 5.8288
4.3185 2.3792
2.6108 5.4870
1.5999 4.1317
1.1046 4.1969
5.4507 1.7694
1.6380 5.1318

0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10