Fuzzy Clustering Algorithms

trainerhungarianAI and Robotics

Oct 20, 2013 (3 years and 7 months ago)

115 views

Fuzzy Clustering Algorithms

SSIE 617 2
nd

Presentation

Benjamin James Bush

05/02/2012

What is Clustering?

Crisp & Fuzzy Clustering

CRISP

Each point belongs

to exactly one cluster.


C
-
Means Clustering


FUZZY

Cluster membership is a matter of degree.


Fuzzy C
-
Means Clustering (FCM)


Fuzzy Min
-
Max Clustering Neural Network


C
-
Means Clustering

Fixed number of
clusters. One

per cluster.


Each data point belongs
to the cluster
corresponding to the
closest centroid
.

Animation by
Andrey

A.
Shabalin
, Ph.D
.



Figure

cost function

# of clusters

cost of the

i
th

cluster

data points belonging

to the
i
th

group

distance between data
point and cluster center

C
-
Means Clustering

C
-
Means Clustering

A
nimation by
Andrey

A.
Shabalin
, Ph.D
.



pick
c

centroids at
random

a
ssign each data
point to the cluster
corresponding to the
nearest centroid.

move each centroid to the
mean value of its cluster’s
data points.

Fuzzy C
-
Means Clustering (FCM)

Fuzzy C
-
Means Clustering

Fixed number of clusters.

One

.


Clusters are fuzzy sets.


Membership degree of a
point can be any number
between 0 and 1.


Sum of all degrees for a
point must add up to 1.

Animation
by
Matteo

Matteucci
, Ph.D
.



Figure

Fuzzy C
-
Means Clustering (FCM)

Fuzzy C
-
Means Clustering

C
-
Means

Fuzzy

C
-
Means

(FCM)

summing over

all data points

membership

degree

f
uzziness

exponent

Fuzzy C
-
Means Clustering

pick
c

centroids at
random

a
ssign membership degrees
according to:

move each centroid to the
following position:

Note: formulas are result of
the method
of

Lagrange multipliers as applied to aforementioned

cost function. Proof left as exercise.

Crisp & Fuzzy Clustering

CRISP

Each point belongs

to exactly one cluster.


C
-
Means Clustering


FUZZY

Cluster membership is a matter of degree.


Fuzzy C
-
Means Clustering (FCM)


Fuzzy Min
-
Max Clustering Neural Network


How Many Clusters?

?

Fuzzy Min
-
Max Clustering NN

Variable

number of
clusters. Each cluster has
a
Hyperbox

Fuzzy Set
.


Degrees inside the box
are 1. Degrees outside
the
hyperbox

decrease
linearly with distance
from the box.


Total degrees for a point
need not add up to 1.


Boxes may not overlap.

Hyperbox

Fuzzy Sets

Start
Mathematica
...

Hyperbox

Fuzzy Sets

Easy to implement as ANNs.

Potential to take advantage of

m
assive parallel processing.

Initialize population of 250 randomly chosen individuals, each with a

random
# of boxes. For each box, choose min point and max point at random.

Create an child individual from each
member of the population. When creating
a child, add a
Gaussean

r.v
. to each
component of the min and max point, and
change the # of boxes with probability 0.5.

Evaluate the fitness of each
individual based on its Minimum
Description Length (MDL)

g
oodness of fit

Penalty for #

of clusters.

Eliminate half of the
individuals via round
-
robin
tournament competition.

Applications

Applications of Fuzzy C
-
Means

Applications of Fuzzy C
-
Means

Applications of Min
-
Max Clustering NN

Applications of Min
-
Max Clustering NN

Bibliography

Ch. 15

Videolectures.net:
MDL Tutorial

http
://videolectures.net/icml08_grunwald_mld/

Ch. 1