1. Introduction to
Pattern Recognition and
Prof. A.L. Yuille.
Dept. Statistics. UCLA.
Stat 231. Fall 2004.
Examples of Patterns.
Discriminate/Decisions about Patterns.
Schools of Pattern Recognition.
What are Patterns?
Laws of Physics & Chemistry generate patterns.
Patterns in Astronomy.
Humans tend to see patterns everywhere.
Patterns in Biology.
Applications: Biometrics, Computational
Anatomy, Brain Mapping.
Patterns of Brain Activity.
Relations between brain activity, emotion,
cognition, and behaviour.
Variations of Patterns.
Patterns vary with
Goal of Pattern Recognition.
Recognize Patterns. Make decisions about
is this person happy or
did the speaker say
“Yes” or “No”?
is this an atom or a
Applications of Pattern Recognition.
Handwritten digit/letter recognition
Biometrics: voice, iris, fingerprint, face, and gait recognition
Smell recognition (e
nose, sensor networks)
Defect detection in chip manufacturing
Interpreting DNA sequences
Credit Fraud Detection
Two Extreme Approaches
Determine models of how patterns are formed.
Use these models to perform discrimination.
Pattern Theory. Grenander.
Don’t model pattern formation.
Instead extract features from patterns and make
decision using these features.
Example: Salmon versus Sea Bass.
attempt to model the
full appearance of
Salmon and Sea
features sufficient to
make the decision
(e.g. length and
Fish Features. Length.
Salmon are usually shorter than Sea Bass.
Fish Features. Lightness.
Sea Bass are usually brighter than Salmon.
Classify fish as
Salmon or Sea Bass
based on a decision
boundary in feature
Generative Models for Speech.
Stochastic Grammars for Speech & Natural
Language. (Manning & Schutze).
Bayes Decision Theory
Bayes Decision Theory
gives a framework for
Generative and Discriminative approaches.
(i) Discriminative methods are simpler,
computationally faster, and easier to apply.
(ii) Generative methods are needed for most
Hybrid methods are increasingly popular.
Stat 231 concentrates on Discriminative Methods
and simple Generative Models.
Other courses by Prof.s Zhu & Yuille deal with
complex Generative Models.
Both Generative and Discriminative
methods require training data to learn the
Machine Learning concentrates on
learning discrimination rules.
Key Issue: do we have enough training
data to learn?
Bayes Decision Theory as theoretical
Simple discriminative and generative
Advanced Discriminative Methods.