Fuzzy Clustering Algorithms
SSIE 617 2
nd
Presentation
Benjamin James Bush
05/02/2012
What is Clustering?
Crisp & Fuzzy Clustering
CRISP
Each point belongs
to exactly one cluster.
C

Means Clustering
FUZZY
Cluster membership is a matter of degree.
Fuzzy C

Means Clustering (FCM)
Fuzzy Min

Max Clustering Neural Network
C

Means Clustering
Fixed number of
clusters. One
per cluster.
Each data point belongs
to the cluster
corresponding to the
closest centroid
.
Animation by
Andrey
A.
Shabalin
, Ph.D
.
Figure
cost function
# of clusters
cost of the
i
th
cluster
data points belonging
to the
i
th
group
distance between data
point and cluster center
C

Means Clustering
C

Means Clustering
A
nimation by
Andrey
A.
Shabalin
, Ph.D
.
pick
c
centroids at
random
a
ssign each data
point to the cluster
corresponding to the
nearest centroid.
move each centroid to the
mean value of its cluster’s
data points.
Fuzzy C

Means Clustering (FCM)
Fuzzy C

Means Clustering
Fixed number of clusters.
One
.
Clusters are fuzzy sets.
Membership degree of a
point can be any number
between 0 and 1.
Sum of all degrees for a
point must add up to 1.
Animation
by
Matteo
Matteucci
, Ph.D
.
Figure
Fuzzy C

Means Clustering (FCM)
Fuzzy C

Means Clustering
C

Means
Fuzzy
C

Means
(FCM)
summing over
all data points
membership
degree
f
uzziness
exponent
Fuzzy C

Means Clustering
pick
c
centroids at
random
a
ssign membership degrees
according to:
move each centroid to the
following position:
Note: formulas are result of
the method
of
Lagrange multipliers as applied to aforementioned
cost function. Proof left as exercise.
Crisp & Fuzzy Clustering
CRISP
Each point belongs
to exactly one cluster.
C

Means Clustering
FUZZY
Cluster membership is a matter of degree.
Fuzzy C

Means Clustering (FCM)
Fuzzy Min

Max Clustering Neural Network
How Many Clusters?
?
Fuzzy Min

Max Clustering NN
Variable
number of
clusters. Each cluster has
a
Hyperbox
Fuzzy Set
.
Degrees inside the box
are 1. Degrees outside
the
hyperbox
decrease
linearly with distance
from the box.
Total degrees for a point
need not add up to 1.
Boxes may not overlap.
Hyperbox
Fuzzy Sets
Start
Mathematica
...
Hyperbox
Fuzzy Sets
Easy to implement as ANNs.
Potential to take advantage of
m
assive parallel processing.
Initialize population of 250 randomly chosen individuals, each with a
random
# of boxes. For each box, choose min point and max point at random.
Create an child individual from each
member of the population. When creating
a child, add a
Gaussean
r.v
. to each
component of the min and max point, and
change the # of boxes with probability 0.5.
Evaluate the fitness of each
individual based on its Minimum
Description Length (MDL)
g
oodness of fit
Penalty for #
of clusters.
Eliminate half of the
individuals via round

robin
tournament competition.
Applications
Applications of Fuzzy C

Means
Applications of Fuzzy C

Means
Applications of Min

Max Clustering NN
Applications of Min

Max Clustering NN
Bibliography
Ch. 15
Videolectures.net:
MDL Tutorial
http
://videolectures.net/icml08_grunwald_mld/
Ch. 1
Comments 0
Log in to post a comment