ER: An Intuitive Similarity

utterlypanoramicSecurity

Nov 30, 2013 (3 years and 6 months ago)

66 views

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

1

ER
2
: An Intuitive Similarity
Measure for On
-
Line Signature
Verification

Hansheng Lei, Srinivas Palla, Venu Govindaraju


CUBS, Center for Unified Biometrics and Sensors

Univ. at Buffalo, the State Univ. of New York

Amherst, NY USA 14260

{hlei, spalla2, govind}@cse.buffalo.edu


CUBS

Center for Unified Biometrics and
Sensors, NY, USA

2

ER
2
: An Intuitive Similarity Measure for On
-
Line
Signature Verification

1.
Introduction
-

On
-
line signature verification


2. ER
2
: Intuitive Similarity Measure

3. Experimental Results

4. Demo


CUBS
signature verification system

5. Conclusion

6. References

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

3

Introduction


Handwritten signatures are commonly used for financial
transactions and documents.


Verification is usually done by visual inspection.


Unlike
iris, retina, fingerprint, face, signature does not
require any expensive hardware, thus it is already widely
accepted by general public.


Two kinds of signatures: off
-
line and
on
-
line
.

Fig.1 An on
-
line signature sensor.
The
X
-
Y

coordinates and Pressure of
signing are captured. With more
sophisticated devices, Altitude and
Azimuth are also recorded.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

4

Introduction


Ideal Goals of On
-
line Verification


1. High accuracy
(current accuracy is about 97%
depending on test datasets)


2. Eliminating fraud.


3. Cheap implementation.


4. Substituting PIN or password.

On
-
line signature verification is attracting
increasing interests, academic and industrial.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

5

Introduction


Challenges


1). Intra
-
class variation



We are unaware of whether an individual

s signature is unique.
The variation of a person

s signature can be large.



2). Forgery


Easier to be forged than other biometric attributes such as
fingerprint, iris, etc.



3). Very limited signatures for training



Usually we can not expect more than 6 genuine signatures for
training for each individual. This is unlike handwriting
recognition.


4). Decide the consistent features


There are possibly over 100 features for signature[2], such as
Width, Height, Duration, Orientation, X positions, Y positions,
Speed, Curvature, Pressure, so on.
Which of them are reliable?



CUBS

Center for Unified Biometrics and
Sensors, NY, USA

6

Introduction

Basic Procedure for Signature Verification



Raw data Preprocessing


Make signature invariant to
scaling, translation & rotation
.


Template generation from given signature



The generated template include: 1)what kinds of feature are
chosen, 2)the
features
,3)
distance measures
, 4) the
threshold

for decision.


Verification according to the template



1). Preprocess the raw data of the given signature.


2). Extract features and compare distances with the those in
the template.


3). Make decision according to the
threshold

specified in the
template.


CUBS

Center for Unified Biometrics and
Sensors, NY, USA

7

Introduction
-

Raw data Preprocessing


Invariant to scaling and translation


Suppose Sig=[X Y], both X and Y are sequences. To
make it invariant to scaling and translation by mean
-
standard deviation normalization:

)
(
X
X
X
X



)
(
Y
Y
Y
Y




Invariant to rotation


Method

A. Represent sig=[X Y] in polar space. (x
i
, y
i
) =>


(r
i
,

θ
i
).


Method B. Determine the orientation of the mass of



signature and rotate it.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

8

Introduction
-

Raw data Preprocessing


Arc
-
length Normalization


Given signature is considered as a 2D curve. It is believed
that it is necessary to normalize its length and resample
the points by equal arc
-
length.



Smoothing the curve


Smoothing is to discard the noises. Basically two choices:


1). Gaussian filters. Convolute the curve with a Gaussian
mask.


2). FFT transform. FFT makes energy concentrated on
the first few coefficients. We can extract these coefficients
and reversely FFT back to reconstruct the sequences.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

9

Introduction
-

Raw data Preprocessing

mean
-
std
norm.

-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
-3
-2
-1
0
1
2
3
4
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-3
-2
-1
0
1
2
3
20
40
60
80
100
120
140
160
-170
-165
-160
-155
-150
-145
-140
-135
-130
-125
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
-3
-2
-1
0
1
2
3
4
Resampling

Smoothing

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

10

Introduction
-
Template generation



Feature Extraction/Selection


Because of limited training samples of signatures (say, 6)
and no forgeries,
features can not be extracted
statistically
. We think statistics
-
based methods are quite
difficult.


Distance Measures


Distance measures are associated with features. For
scalar features, Euclidean norm is a proper measure; for
sequential features, Dynamic Time Warping (DTW) is
good measure.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

11

Introduction
-
Template generation

Features


Global features
:


#Width, Height, #Duration, #Orientation


Local features
:


#X
-
coordinates, #Y
-
coordinates , #Curvature


Dynamic features
:


#Velocity, #Acceleration, #Pressure, #Pressure changing


Other features
:


# Number of segments, #Critical points, etc.




CUBS

Center for Unified Biometrics and
Sensors, NY, USA

12

Introduction
-
Template generation

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

13

Introduction
-
Template generation


Coordinate sequences


X, Y , [X,Y ] are the most straightforward features. They
are
featureless features
.


Speed sequences.


Speed
V
, speed of
X
-
coordinate
V
x

and speed of
Y
-

coordinate
V
y

can be derived from sequence [X,Y ]
directly by subtracting neighboring points. From the
speed, acceleration
V
a

can be further derived.


CUBS

Center for Unified Biometrics and
Sensors, NY, USA

14

Introduction
-
Template generation


Pressure, Altitude, Azimuth


Pressure is one of the most common dynamic


information of on
-
line signature. Some devices can capture
additional information, such as Azimuth (the clockwise
rotation of cursor about the z
-
axis) and altitude (the angle
upward toward the positive z
-
axis).



CUBS

Center for Unified Biometrics and
Sensors, NY, USA

15

Introduction
-
Template generation


Center of Mass, Torque, Curvature
-
ellipse S
1

and S
2


The five features were defined by
Vishvjit S. Nalwa [6].


Torque measures the area swept by the vector of pen
position. S
1

and S
2

measure the curvature ellipse based on
moments. The distance measure used here is cross
-
correlation (Pearson's r) weighted by the consistency of
points.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

16

Introduction
-
Template generation


Average, average positive speed on
X
-
axis ,average positive speed on
Y
-
axis, total
signing duration.


Lee et al. [3]

lists two sets of scalar features (over 100
features). These four features have the highest
preference in the first set. The distance measure is
Euclidean norm.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

17

Introduction
-
Template generation


Cos(a), sin(a), Curvature





a

is the angle between the speed vector and the X
-
axis.
The three features are proposed by
Jain et al. [10].

It also
proposes coordinate sequence differences.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

18

Introduction
-
Template generation


Features (examples)


0
20
40
60
80
100
120
140
160
180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
0
20
40
60
80
100
120
140
160
180
-3
-2
-1
0
1
2
3
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-3
-2
-1
0
1
2
3
A Signature
sample

X

coordinates

Y

coordinates

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

19

Introduction
-
Template generation


Features (examples)

0
20
40
60
80
100
120
140
160
180
-3
-2
-1
0
1
2
3
4
5
Torques

S
1

of Curvature ellipse

-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-3
-2
-1
0
1
2
3
A Signature
sample

0
20
40
60
80
100
120
140
160
180
-1
-0.9
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
CUBS

Center for Unified Biometrics and
Sensors, NY, USA

20

Introduction
-
Template generation


Feature comparison

0
20
40
60
80
100
120
140
160
180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
0
20
40
60
80
100
120
140
160
180
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-3
-2
-1
0
1
2
3
X
-
coordinates (genuine)

X
-
coordinates (forgery)

Genuine sig.

Forgery sig.

Only
X
-
coordinates can not distinguish them!

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

21

Introduction
-
Template generation

We have following experience:


1). One of the most reliable features is the
shape

of the
signature. Shape is described by the combination of X, Y
-
coordinates [X,Y].



2). The second reliable feature is the
speed

of writing.




To represent shape and speed, each signature is a 3
-
D
sequence: Sig
i
=[X
i
, Y
i
, V
i
], where V is the sequence of
speed magnitude. Then we use
ER
-
Squared

to match
two signatures and return a
Confidence

of similarity
(0%
-
100%). The details will be given later in section
2..

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

22

Introduction
-
Distance measures

Most commonly
-
used measures


Euclidean norm




Weighted Correlation






Where f(l), h(l) are functions of two signatures and w(l) is the
consistency function.



Dynamic Time Warping (DTW)


Elastic sequence matching. Very good for on
-
line signatures.





N
i
x
x
X
X
D
1
2
2
1
2
1
)
(
)
,
(




dl
l
h
l
w
dl
l
f
l
w
dl
l
h
l
f
l
w
r
)
(
)
(
)
(
)
(
)
(
)
(
)
(
2
2
2
2
2
CUBS

Center for Unified Biometrics and
Sensors, NY, USA

23

Introduction
-
Distance measures


DTW



S1
S2
S1
S2
One
-
One alignment

Dynamic alignment

Both Euclidean norm and
correlation assume one
-
one
alignment. Easy but brittle!

Elastic alignment is more robust
for sequences, at the cost of
computational resources.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

24

Introduction
-
Distance measures


DTW




)}
1
,
(
),
1
,
(
),
,
1
(
min{
)
(
)
,
(
2







j
i
D
j
i
D
j
i
D
x
x
j
i
D
j
i
Current cost

Recursive cumulative cost

The calculation of matrix D. The DTW
warping path in the matrix D is the
path which has minimum average
cumulative cost. The unmarked area
is the constraint imposed by |i
-
j|<w (w
is the width of the allowed margin).

Subject to optional constrain: |i
-
j|<w

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

25

Introduction
-

some remarks


Remarks on some research directions in on
-
line signature
verification


Segmentation?



Signature is an art of drawing, not limited to some kind language.
A Segmentation method by Perceptually Important Points was
proposed by Jean
-
Jules Brault et al [7]. Many works have been
done to apply segmentation to signature verification. Problems:
1)The consistency of segmentations? 2)If DTW is used as measure,
Segmentation is of little necessity, because those Perceptually
Important Points can be aligned accurately by DTW.

-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
CUBS

Center for Unified Biometrics and
Sensors, NY, USA

26

Introduction
-

some remarks


User
-
dependent distance threshold?



Distance (Euclidean, DTW, etc.) for dissimilarity measure is
not

intuitive. In real applications, users tends to ask:
how similar is the
two signatures? Or, what is the confidence that this signature is
genuine?
It is intuitive to answer: their similarity confidence is 90%!
(instead of saying their distance of dissimilarity is 5.8).



It is hard to obtain a user
-
dependent threshold, because of limited
genuine samples. Though it is a choice to use the genuine samples
from other users as forgeries, it won

t help much on determining
the threshold.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

27

Introduction
-

some remarks


Statistics based methods?


Again because we can not expect many signature
samples, statistics based methods, such as Markov
Model, is hard to achieve high performance.



Artificially generate genuine signatures? Using random
forgeries or use the signatures from other users?
Possible ways.



CUBS

Center for Unified Biometrics and
Sensors, NY, USA

28

ER
2
: An Intuitive Similarity Measure for On
-
Line
Signature Verification

1.
Introduction
-

On
-
line signature verification


2
.
ER
2
: Intuitive Similarity Measure

3. Experimental Results

4. Demo


CUBS
signature verification system

5. Conclusion

6. References

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

29

ER
2
: Intuitive Similarity Measure

Similarity measures must satisfy:


The similarity of intra
-
class is very high. (so that
we can accept genuine signature)


The similarity of inter
-
class is very low. (so that
we can reject forgery).


An intuitive score range, like 0
-

1.




CUBS

Center for Unified Biometrics and
Sensors, NY, USA

30

ER
2
: Intuitive Similarity Measure

Traditional Linear Regression

-2000
-1500
-1000
-500
0
500
1000
1500
2000
0
50
100
150
200
250
300
350
400
-2000
-1000
0
1000
2000
3000
4000
0
50
100
150
200
250
300
350
400
-2000
-1000
0
1000
2000
3000
4000
-2000
-1500
-1000
-500
0
500
1000
1500
2000
-500
0
500
1000
1500
2000
2500
3000
3500
4000
0
50
100
150
200
250
300
350
400
-500
0
500
1000
1500
2000
2500
3000
3500
0
50
100
150
200
250
300
350
400
-500
0
500
1000
1500
2000
2500
3000
3500
4000
-500
0
500
1000
1500
2000
2500
3000
3500
Similarity:
91%

Similarity:
31%

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

31

ER
2
: Intuitive Similarity Measure

Linear Regression


Given two sequences
X=(x
1
,x
2
,

, x
n
)
,
Y=(y
1
,y
2
,

,
y
n
)
, then the similarity by R
2

of
X

and
Y

is:












n
i
i
n
i
i
n
i
i
i
Y
y
X
x
Y
y
X
x
R
1
2
1
2
1
2
2
)
(
)
(
)]
)(
(
[
R
2
named R
-
squared because R
2

= (r)
2
, where r is
Pearson

s correlation r.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

32

ER
2
: Intuitive Similarity Measure

Extended Regression


Traditional regression handles two
1
-
dimentional
sequences. We extend it to multi
-
dimensional
sequences as follows:


















M
j
n
i
j
ij
M
j
n
i
j
ij
M
j
n
i
j
ij
j
ij
Y
y
X
x
Y
y
X
x
ER
1
1
2
1
1
2
1
1
2
2
)
(
)
(
))]
)(
(
(
[
We name it ER
2

since is an extension from 1
-
D to multi
-
D

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

33

ER
2
: Intuitive Similarity Measure

The intuition of ER
2

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

34

ER
2
: Intuitive Similarity Measure

Remarks on Linear Regression


Advantages: Invariant to scale and translation; Similarity
(Goodness
-
of
-
fit) makes sense.


Disadvantages: One
-
one alignment, brittle.

S1
S2
S1
S2
One
-
One alignment

Dynamic alignment

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

35

ER
2
: Intuitive Similarity Measure

We couple ER
2
with DTW
-
based Curve Matching

Dynamic Alignment by DTW.
However, we found direct
DTW on two signatures is not
very robust.

We use Curve Matching, which is
to calculate the total cost of
changing one curve to fit another
curve. The dynamic programming
of DTW is used to realize the
calculation.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

36

ER
2
: Intuitive Similarity Measure


DTW
-
based Curve
Matching

Suppose we have two curves
C

and
C

. Curve matching is
actually:

))
'
(
),
(
(
)
'
,
(
_
C
speed
C
speed
DTW
C
C
Match
C

Where speed(
C
)
i
=
C
i+1
-
C
i.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

37

ER
2
: Intuitive Similarity Measure

ER
2

coupled with Curve Matching

The DTW warping path in the matrix is the path
which has minimum average cumulative cost.
The unmarked area is the constraint that path is
allowed to go.

]
,...
,
,
[
2
2
1
m
y
y
y
y
Y

]
,...
,
,
[
3
2
1
n
x
x
x
x
X

)
,
(
2
X
Y
ER
Similarity

( y
2
is

matched x
2
, x
3,
so
we extend it to be two
points in Y sequence.)

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

38

Experimental Results

1.
Introduction
-

On
-
line signature verification


2. ER
2
: Intuitive Similarity Measure


3.
Experimental Results

4. Demo


CUBS
signature verification system

5. Conclusion

6. References

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

39

Experimental Results


Signature database


The released signature datasets by SVC( First
International Signature Verification Competition). SVC
released the signatures of 80 individuals, 20 genuine and
20 skilled forgeries each.


Methods comparison


ER
2
coupled with Curve Matching




Vs.


Curve Matching without ER
2


CUBS

Center for Unified Biometrics and
Sensors, NY, USA

40

Experimental Results


Enrollment


Enroll 6 genuine signatures from each
individual.


Preprocessing


Only X,Y
-
coordinates are used. Other information, such
as
Pressure, Altitude, Azimuth are not used in the
experiments
. 1) Smooth the raw sequence by Gaussian



filter. 2) Rotate if necessary. 3) Normalize


each signature by:



)
(
X
X
X
X



)
(
Y
Y
Y
Y



CUBS

Center for Unified Biometrics and
Sensors, NY, USA

41

Experimental Results

a) FRR and FAR of ER
2
(coupled with Curve Matching). b) FRR and
FAR of Curve Matching (without ER
2
). Both using universal
threshold.

0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
EER=20.9%
FRR
FAR
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
EER=7.2%
FRR
FAR
a)
b)
Threshold (%)
Threshold (%)
Error Rate (%)
Error Rate (%)
CUBS

Center for Unified Biometrics and
Sensors, NY, USA

42

Experimental Results

Table 2.
EERs with universal or user
-
dependent

threshold. Skilled forgeries are provided by
the dataset, while random forgery means the
forgeries are selected from the signatures of
different individuals.

*The results of SVC are available at
http://www.cs.ust.hk/svc2004/results.html
. We are team 14.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

43

Experimental Results


A project regarding on
-
line signature

Recently, we have a Multimodal
Biometrics project supported by
US Army Laboratory. It requires
to test signatures from 1000
individuals, each 2 as enrollment
and 3 as queries. We collected
330 individuals so far. The
preliminary ROC based on ER
2

is:

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

44

Experimental Results

1.
Introduction
-
On
-
line signature verification



2. ER
2
: Intuitive Similarity Measure


3. Experimental Results


4.
Demo


CUBS
signature verification system

5. Conclusion

6. References

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

45

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

46

Demo

CUBS Sign. System

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

47

Conclusion




We propose ER
2

as a similarity measure for multi
-
dimensional sequence matching. Signature verification
system can use ER
2

coupled with curve matching for
intuitive similarity output and higher performance as
well. The experimental results are encouraging,
although we have to notice that further evaluation on
large and real databases is necessary.



Our future work will explore the feasibility of ER
2

on


dynamic features like pressure, speed, etc.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

48

References

[1] Rejean Plamondon, Guy Lorette. Automatic Signature Verification and
Writer identification
-
the state of the art. Pattern Recognition, Vol.22,
No.2, pp.107
-
131, 1989.

[2] F. Leclerc and R. Plamondon. Automatic signature verification: the
state of the art 1989
-
1993. International Journal of Pattern
Recognition and Artificial Intelligence, 8(3):643
-
660, 1994.

[3] Luan L. Lee, Toby Berger, Erez Aviczer. Reliable On
-
line Human
Signature Verifications Systems. IEEE trans. On Pattern Analysis and
Machine Intelligence, Vol. 18, No.6, June 1996.

[4] R. Plamondon. The Design of On
-
line Signature Verification System:
From Theory to Practice. Int

l J. Pattern Recognition and Artificial
Intelligence, vol. 8, no. 3, pp. 795
-
811, 1994.

[5] Mario E. Munich, Pietro Perona. Visual Identification by Signature
Tracking. IEEE Trans. On Pattern Analysis and Machine Intelligence,
Vol. 25, No. 2, pp. 200
-
216, February 2003.


CUBS

Center for Unified Biometrics and
Sensors, NY, USA

49

References

[6] Vishvjit S. Nalwa. Automatic On
-
line Signature Verification.
Proceedings of the IEEE, Vol. 85, No. 2, pp. 215
-
239, February
1997.

[7] Jean
-
Jules Brault and Rejean Plamondon. Segmenting Hanwritten
Signatures at Their Perceptually Important Points. IEEE Trans. On
Pattern Analysis and Machine Intelligence, Vol, 15, No. 9, pp. 953
-
957, September 1993.

[8] Taik H. Rhee, Sung J. Cho, Jin H. Kim. On
-
line Signature
Verification Using Model
-
Guided Segmentation and Discriminative
Feature Selection for Skilled Forgeries. Sixth International
Conference on Document Analysis and


Recognition (ICDAR '01), September, Seattle, Washington, 2001.

[9] Thomas B. Sebastian, Philip N. Klein, Bejamin B. Kimia. On Aligning
Curves. IEEE Trans. On Pattern Analysis and Machine Intelligence,
Vol. 25, No. 1, January 2003.

CUBS

Center for Unified Biometrics and
Sensors, NY, USA

50

References

[10] A.K. Jain, Friederike D. Griess and Scott D. Connell. On
-
line
Signature Verification.
Pattern Recognition
, vol. 35, no. 12, pp.
2963
--
2972, Dec 2002.


[11] K. Huang and H. Yan,

On
-
Line Signature Verification Based on
Dynamic segmentation and Global and Local Matching,


Optical Eng.,
vol. 34, no. 12, pp. 3480
-
3487, 1995.

[12] G. Lorette and R. Plamondon,

Dynamic Approaches to Hand
-
written Signature Verification,


Computer Processing of Hand
-
writing,
pp. 21
-
47, 1990.

[13] R. Martens and L. Claesen,

On
-
Line Signature Verification by
Dynamic Time
-
Warping,


Proc. 13th Int

l Conf. Pattern Recognition,
pp. 38
-
42, 1996.

[14] B. Wirtz,

Stroke
-
Based Time Warping for Signature Verification,


Proc. Int

l Conf. Document Analysis and Recognition, pp. 179
-
182,
1995.