Department of Computer Science, University of Calgary

runmidgeAI and Robotics

Oct 20, 2013 (3 years and 10 months ago)

78 views


Presented by: Kushan Ahmadian


Department of Computer Science, University of Calgary

kahmadia@ucalgary.ca

1

Outline


Introduction


Research Contributions


Motivations


Background Research


Neural Network


Dimensionality Reduction


Biometrics


Proposed Methodology


Subspace Clustering


Chaotic Associative Memory


Overall System Architecture


Preliminary Experimental Results


Conclusion and Future Work



2

Research Goal

The purpose of my research is to develop a
novel methodology based on the
subspace
clustering dimension reduction technique

and
chaotic neural network

to improve the
performance

and
circumvention

of multi
-
modal biometric system.

3

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

My Research Contributions


A novel correlation clustering approach
accounting for the feature relevance and/or
feature correlation problem in multi
-
modal
biometric system



Design and utilization of
a chaotic associative
neural memory with original noise injection
policy to learn the patterns of biometric
features



Designing and evaluating
the performance of
the system comparing the results to the post
-
classification (decision level) fusion results

4

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Motivation


Alleviate problems of current dimensionality
reduction methods
such as “curse of
dimensionality” and “locality” by proposing a
new subspace clustering based dimensionality
reduction
for biometric data.


Reducing the
FAR (False Acceptance Rate)
and
FRR (False Rejection Rate)
by minimizing the
effect of noise, template aging and other errors
using
a novel feature selection method
.


Utilizing a
brain
-
like associative memory
(chaotic neural network)
for the first time in
biometric

to enhance the ability of pattern
-
based data retrieval from memory.




5

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Biometrics

6

Source: http://360biometrics.com/

Biometrics comprises
methods for uniquely
recognizing humans
based upon one or more
intrinsic physical or
behavioral traits
.



1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Multi
-
modal biometric

7

Matchers Fused

Authors

Level of Fusion

Fusion methodology

Hand

Kumar et al (2003)

Feature, Match score

Feature concatenation/
sum rule

Pulmprint (geometry,
local texture)

You, et al (2004)

Decision

Hierarchical matching

Fingerprint(2
impressions)

Jain and Ross (2002)

Sensor, feature

Mosaicing of
templates

Fingerprint

Wilson et al (2004)

Match score

Sum rule

Face (global and local
features)

Ferrez et al (2005)

Feature level

Feature concatenation

Voice

Cheung et al(2004)

Match score

Zero sum fusion

Face, Iris and
Signature

Gavrilova and
Monwar (2009)

Rank Level

Markov model

Examples of fusion methods.

8

Matchers Fused

Authors

Level of Fusion

Fusion methodology

Hand

Kumar et al (2003)

Feature, Match score

Feature concatenation/
sum rule

Pulmprint (geometry,
local texture)

You, et al (2004)

Decision

Hierarchical matching

Fingerprint(2
impressions)

Jain and Ross (2002)

Sensor, feature

Mosaicing of
templates

Fingerprint

Wilson et al (2004)

Match score

Sum rule

Face (global and local
features)

Ferrez et al (2005)

Feature level

Feature concatenation

Voice

Cheung et al(2004)

Match score

Zero sum fusion

Face, Iris and
Signature

Gavrilova and
Monwar (2009)

Rank Level

Markov model

Feature Space and Dimensionality Reduction


Transform the data in the high
-
dimensional space to a space
of fewer dimensions.



9

Subspace obtained by PCA and ideal resulted subspace

projected clustering (Han and
Kamber
, 2001)

DBSCAN (Ester et.al. 1996)

Specifications of clustering methods (Achtert and Böhm,
2007).

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Reducing Dimensionality by Subspace Analysis


The principle for subspace analysis is based on a
generalized description of spherical coordinates.



A point in data space is represented by a
sinusoidal curve in parameter space P.



A point in parameter space corresponds
to a (d − 1)
-
dimensional hyperplane in data space.

10

Neural network


Chaotic Neural Networks un pattern Rec.(Wang, 2006)


CSA (Chen and Aihara, 1997)


Applications of Optimization (Wang, 1998)

11

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Traditional System Architecture

12

Traditional multimodal architecture

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Biometric
Database

Eigenfaces
vectors

PCA
-
based
dimensionality reduction

User
samples

Yes/No

Learner 1

Learner 1

Learner 1

Aggregation method

Proposed System Architecture

13

Proposed biometric recognition system

Biometric
Database

User samples

Mean faces

Novel representation
of Feature Vector

Train neural
networks

Testing
neural
network

Yes/No

Verified
?

Train
?

N

Y

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Subspace Clustering Step 1

14

Mean image
for each class

For each person (class) compute the mean image

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Input Data

Eigenface images

15


The eigenvectors are sorted in order of descending eigenvalues and the greatest
eigenvectors are chosen to represent face space.


This reduces the dimensionality of the image space, yet maintains a high level
of variance between face images throughout the image subspace.


Any face image can then be represented as a vector of coefficients,
corresponding to the

contribution


of each eigenface.

Each eigenvector can be displayed as an image and due to the likeness to faces (FERET database)

Subspace Clustering Step 2

16

Number of dimensions: m (number of mean images)

Number of points in the high dimensional space: x*y

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

17


Three points p
1
, p
2
, p
3

on a plane (b) Corresponding parameterization functions.

Reducing Dimensionality by Subspace Analysis

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

18

Reducing Dimensionality by Subspace Analysis


Find the clusters within an error range of
ε
.


Use the mean vector as the candidate for
the members of a cluster and create the
new vector space. The number of points of
the new space is:


M << x*y


Next, we try to learn the pattern using a learner (Chaotic
Neural Network)

19

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Associative Memory

20


The neuron signals comprise an output pattern.



The neuron signals are initially set equal to
some input pattern.



The network converges to the nearest stored
pattern

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Chaotic Associative Memory

21


Chaotic and period
doubling noise
injection
policies




To overcome the
drawback of non
-
autonomous methods
is their
blind

noise
-
injecting
strategy




Proposing the
adjacency matrix to
evaluate the chances of
a neuron to receive
chaotic noise

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Fingerprint Neural Based Method


Case
Study

22

The general goal is to train the network using the
Delaunay triangulation of minutiae points
.



1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

23

DT based Matching
-
Experimental Results

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Multimodal Training Phase

24

Analyzing and obtaining the best set of feature
vectors


Data acquisition

Training the chaotic associative
memory with the obtained vectors

User
1

Biometric1


Biometric2

Biometric3

User
2

User
N

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

25

Dimensionality reduction


New feature space


Data acquisition

Biometric1

Biometric2

Biometric3

User’s obtained feature
space vector

Feeding the new vector space into
the associative memory

Network
Convergence

(Matching)

Yes/ No

User

Testing

1
Introduction

2
Background

3
Methodology

4
Experiments

5. Conclusions

Experimental Results


My method: Subspace Clustering (SC) and Chaotic Noise Neural
Network (CNNN)


Compared methods:


Simple
-
Sum (SS), Min
-
Score (MIS), Max
-
Score (MAS), Matcher
Weighting (MW), User Weighting (UW)









Min
-
Max Score (MM), Zero Score (ZS),
Tanh

(TH), Quadratic
Line Quadric (QLQ), Subspace Clustering (SC)

26

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Experimental Results

27

EER rate, SC with different fusion techniques

EER rate, CNN with different normalization techniques

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

28

EER rate, Combination of different fusion and normalization techniques

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Experimental Results

Conclusions


The contributions are :


Introducing method for selecting a proper set
of input features to reduce the dimensionality
of biometric data and consequently
enhancing the performance of the system.


Introducing chaotic associative memories in
biometric system, which have significant
advantages over conventional memories in
terms of capacity of the memory


Implementing and enhancing performance of
the biometric multimodal verification system.



29

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Future Work


Continuing research on the axis
-
parallel subspace
clustering.


Comparing to a newly proposed system where the
data analysis is run over the vectors of each biometric
separately. The benefit of such a system would being
more tolerable over the absence of each biometric.


Enhancing the capacity of the associative memories
which is the current drawback of associative based
memories.


Finding a better candidate vector for subspace
clustered data to improve the quality of data
reduction method.


Continuing research on subspace clustering methods
to further decrease FAR and FRR rates

30

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Project Timeline

31

Phase

Task Name

Start

End

Duration

2008

2009

2010

2011

2012

1
st

Required

course taken. Literature review
on biometrics, feature selection techniques
and associative memories. Problem
statement formulation
.

Sept.
2008

Aug.
2009

12
months

2
nd

Required course taken. Prototype system
development with various associative
neural memory models including chaotic
neural network and send results to different
peer reviewed journals and conferences for
reviewer's feedback.

Sept.
2009

Aug.
2010

12
months

3
rd

Complete system development with
different feature selection policies.
Comments from various reviewers are
highly considered during the complete
system development.

Sept.
2010

Aug.
2011

12
months

4
th

Validation of developed system.
Performing performance analysis of the
proposed system against different
biometric databases. The results will be
communicated through appropriate venues
and consequently modify the system
according to the feedback

Sept.
2011

Jan.
2012

5 months

5
th

The thesis will be prepared as part of the
PhD degree requirements.

Feb.
2012

Jul.
2012

6 months

1
Introduction

2
Background

3
Methodology


4
Experiments

5. Conclusions

Key References


Bohm C,
Kailing

K,
Kriegel

H.P, Kroger P, (2004) Density Connected Clustering with
Local Subspace Preferences,
Proceedings of the Fourth IEEE International Conference on
Data Mining
, p.27
-
34,


Jain A. K., Ross A., and
Prabhakar

A. (2004) An Introduction to Biometric Recognition.
IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Image
-

and Video
-
Based Biometrics
, 14(1):4

20.


Kriegel

H. P,
Kröger

P,
Zimek

Z. (2009) Clustering High Dimensional Data: A Survey on
Subspace Clustering, Pattern
-
based Clustering, and Correlation Clustering


ACM Transactions on Knowledge Discovery from Data pp.1
-
58,


Wang L
,
and Shi H (2006) A gradual noisy chaotic neural network for solving the
broadcast scheduling problem in packet radio networks.
IEEE Transactions on neural
networks,
vol

17, no. 4:989
-

1001


Zhao L. and Yang Y. H., (1999) “Theoretical Analysis of Illumination in PCA
-
Based Vision
Systems,”
Pattern Recognition
, Vol. 32, No. 4, pp.547
-
564.


Belhumeur

P. N,
Hespanha

J. P, and
Kriegman

D. J. (1997) Eigenfaces vs.
Fisherfaces
:
recognition using class specific linear projection,
IEEE Trans. Pattern Analysis and
Machine Intelligence
, Vol. 19, No. 7, pp.711
-
720.


32

Publications


K.Ahmadian

and M. Gavrilova, “Transiently Chaotic Associative Network for
Fingerprint Image Analysis”, Special Issue on Intelligent Computing for
Multimedia Assurance in the


International Journal of Neural Network World,
A. Abraham editor, 2009
-
2010, 21 pages, in print (accepted in May 2009)


K.

Ahmadian and M. Gavrilova “On
-
Demand Chaotic Neural Network for
Broadcast Scheduling Problem”, Journal of Supercomputing, 18 pages, Springer
( accepted with minor revisions in May 2010)
.


K. Ahmadian, and M. Gavrilova, “Multi
-
objective Evolutionary Approach for
Biometric Fusion,” IEEE International Conference on Biometrics and
Kansei

Engineering, pp. 12
-
17, June 25
-
28, Poland, 2009.


K. Ahmadian, M. Gavrilova and D.
Taniar
,
“Multi
-
criteria Optimization in GIS:
Continuous K
-
Nearest
Neighbor

search in mobile navigation,” ICCSA, pp.574
-
589, March 2010, Japan.

33