Lecture 2 - UEF-Wiki

wonderfuldistinctAI and Robotics

Oct 16, 2013 (3 years and 10 months ago)

92 views

Teuvo

Kohonen


Dr. Eng., Emeritus Professor of the Academy
of Finland; Academician

Since the 1960s, Professor
Kohonen

has introduced several new concepts to neural computing: fundamental theories
of distributed associative memory and optimal associative mappings, the learning subspace method, the self
-
organizing feature maps (SOMs), the learning vector quantization (LVQ), novel algorithms for symbol processing like
the redundant hash addressing, dynamically expanding context and a special SOM for symbolic data, and a SOM called
the Adaptive
-
Subspace SOM (ASSOM) in which invariant
-
feature filters emergence. A new SOM architecture
WEBSOM

has been developed in his laboratory for exploratory textual data mining. In the largest WEBSOM implemented so far,
about seven million documents have been organized in a one
-
million neuron network: for smaller WEBSOMs, see the
demo at
http://websom.hut.fi/websom/

.

Gender

detection

The classification or description scheme is usually based on the
availability of a set of patterns that have already been classified or
described. This set of patterns is termed the
training set
, and the
resulting learning strategy is characterized as
supervised learning
.


Learning can also be
unsupervised
, in the sense that the system is
not given an
a priori

labeling of patterns, instead it itself establishes
the classes based on the statistical regularities of the patterns.

The classification or description scheme usually uses one of the following
approaches:

statistical

(or decision theoretic) or

syntactic

(or structural).


Statistical pattern recognition is based on statistical characterizations of
patterns, assuming that the patterns are generated by a
probabilistic

system.


Syntactical (or structural) pattern recognition is based on the structural
interrelationships of features. A wide range of algorithms can be applied
for pattern recognition, from simple
naive
Bayes

classifiers

and
neural
networks

to the powerful
KNN

decision rules.

Pattern recognition is more complex when templates are used to
generate variants. For example, in English, sentences often follow the
"N
-
VP" (noun
-

verb phrase) pattern, but some knowledge of the
English language is required to detect the pattern.


Pattern recognition is studied in many fields, including
psychology
,
ethology
,
cognitive science

and
computer science
.


Holographic associative memory

is another type of pattern matching
where a large set of learned patterns based on cognitive meta
-
weight
is searched for a small set of target patterns.

What is a Pattern?


“A pattern is the
opposite of a chaos
; it is an
entity vaguely defined, that could be given a
name.” (Watanabe)

Recognition


Identification of a pattern as a member of a
category we already know, or we are familiar with


Classification

(known categories)


Clustering

(learning categories)

Category “A”

Category “B”

Classification

Clustering

Handwritten Digit Recognition


Cat vs. Dog

Supervised Classification

Training samples are labeled

Unsupervised Classification

Training samples are unlabeled

Segmentation

Pattern Recognition


Given an input pattern,
make a decision
about the “category” or “class” of the pattern


Pattern recognition is needed in designing
almost
all automated systems


Other related disciplines: data mining,
machine learning, computer vision, neural
networks, statistical decision theory


This course will present various techniques
to solve P.R. problems and discuss their
relative strengths and weaknesses

How

do

we

design
similarity
?

I
ntra
-
class Variability

The letter “T” in different typefaces

Same face under different expression, pose, illumination

Inter
-
class Similarity

Identical twins

Characters that look similar

Difficulties of Representation



How do you instruct someone (or some computer)
to
recognize caricatures
in a magazine, let alone
find a human figure in a misshapen piece of work?”


“A program that could
distinguish between male
and female faces
in a random snapshot would
probably earn its author a Ph.D. in computer
science.” (Penzias 1989)


A representation could consist of a vector of real
-
valued numbers, ordered list of attributes, parts
and their relations….


Difficulties of Representation

John P. Frisby,
Seeing
.
Illusion
,
Brian and Mind
, Oxford University Press, 1980

How should we
model a face to
account for the
large intra
-
class
variability?

Pattern Class Model



A mathematical or statistical description for each pattern
class (
population
); it is this class description that is
learned
from samples




Given a pattern, choose the
best
-
fitting model
for it;
assign the pattern to the class associated with the best
-
fitting model



Pattern Recognition System


Domain
-
specific knowledge


Acquisition, representation


Data acquisition


camera, ultrasound, MRI,….


Preprocessing


Image enhancement, segmentation


Representation


Features: color, shape, texture,…


Decision making


Statistical (geometric) pattern recognition


Syntactic (structural) pattern recognition


Artificial neural networks


Post
-
processing; use of context