CUBS
Center for Unified Biometrics and
Sensors, NY, USA
1
ER
2
: An Intuitive Similarity
Measure for On

Line Signature
Verification
Hansheng Lei, Srinivas Palla, Venu Govindaraju
CUBS, Center for Unified Biometrics and Sensors
Univ. at Buffalo, the State Univ. of New York
Amherst, NY USA 14260
{hlei, spalla2, govind}@cse.buffalo.edu
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
2
ER
2
: An Intuitive Similarity Measure for On

Line
Signature Verification
1.
Introduction

On

line signature verification
2. ER
2
: Intuitive Similarity Measure
3. Experimental Results
4. Demo
–
CUBS
signature verification system
5. Conclusion
6. References
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
3
Introduction
Handwritten signatures are commonly used for financial
transactions and documents.
Verification is usually done by visual inspection.
Unlike
iris, retina, fingerprint, face, signature does not
require any expensive hardware, thus it is already widely
accepted by general public.
Two kinds of signatures: off

line and
on

line
.
Fig.1 An on

line signature sensor.
The
X

Y
coordinates and Pressure of
signing are captured. With more
sophisticated devices, Altitude and
Azimuth are also recorded.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
4
Introduction
Ideal Goals of On

line Verification
1. High accuracy
(current accuracy is about 97%
depending on test datasets)
2. Eliminating fraud.
3. Cheap implementation.
4. Substituting PIN or password.
On

line signature verification is attracting
increasing interests, academic and industrial.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
5
Introduction
Challenges
1). Intra

class variation
We are unaware of whether an individual
’
s signature is unique.
The variation of a person
’
s signature can be large.
2). Forgery
Easier to be forged than other biometric attributes such as
fingerprint, iris, etc.
3). Very limited signatures for training
Usually we can not expect more than 6 genuine signatures for
training for each individual. This is unlike handwriting
recognition.
4). Decide the consistent features
There are possibly over 100 features for signature[2], such as
Width, Height, Duration, Orientation, X positions, Y positions,
Speed, Curvature, Pressure, so on.
Which of them are reliable?
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
6
Introduction
Basic Procedure for Signature Verification
Raw data Preprocessing
Make signature invariant to
scaling, translation & rotation
.
Template generation from given signature
The generated template include: 1)what kinds of feature are
chosen, 2)the
features
,3)
distance measures
, 4) the
threshold
for decision.
Verification according to the template
1). Preprocess the raw data of the given signature.
2). Extract features and compare distances with the those in
the template.
3). Make decision according to the
threshold
specified in the
template.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
7
Introduction

Raw data Preprocessing
Invariant to scaling and translation
Suppose Sig=[X Y], both X and Y are sequences. To
make it invariant to scaling and translation by mean

standard deviation normalization:
)
(
X
X
X
X
)
(
Y
Y
Y
Y
Invariant to rotation
Method
A. Represent sig=[X Y] in polar space. (x
i
, y
i
) =>
(r
i
,
θ
i
).
Method B. Determine the orientation of the mass of
signature and rotate it.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
8
Introduction

Raw data Preprocessing
Arc

length Normalization
Given signature is considered as a 2D curve. It is believed
that it is necessary to normalize its length and resample
the points by equal arc

length.
Smoothing the curve
Smoothing is to discard the noises. Basically two choices:
1). Gaussian filters. Convolute the curve with a Gaussian
mask.
2). FFT transform. FFT makes energy concentrated on
the first few coefficients. We can extract these coefficients
and reversely FFT back to reconstruct the sequences.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
9
Introduction

Raw data Preprocessing
mean

std
norm.
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
3.5
3
2
1
0
1
2
3
4
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
2
1
0
1
2
3
20
40
60
80
100
120
140
160
170
165
160
155
150
145
140
135
130
125
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
3.5
3
2
1
0
1
2
3
4
Resampling
Smoothing
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
10
Introduction

Template generation
Feature Extraction/Selection
Because of limited training samples of signatures (say, 6)
and no forgeries,
features can not be extracted
statistically
. We think statistics

based methods are quite
difficult.
Distance Measures
Distance measures are associated with features. For
scalar features, Euclidean norm is a proper measure; for
sequential features, Dynamic Time Warping (DTW) is
good measure.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
11
Introduction

Template generation
Features
Global features
:
#Width, Height, #Duration, #Orientation
Local features
:
#X

coordinates, #Y

coordinates , #Curvature
Dynamic features
:
#Velocity, #Acceleration, #Pressure, #Pressure changing
Other features
:
# Number of segments, #Critical points, etc.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
12
Introduction

Template generation
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
13
Introduction

Template generation
Coordinate sequences
X, Y , [X,Y ] are the most straightforward features. They
are
featureless features
.
Speed sequences.
Speed
V
, speed of
X

coordinate
V
x
and speed of
Y

coordinate
V
y
can be derived from sequence [X,Y ]
directly by subtracting neighboring points. From the
speed, acceleration
V
a
can be further derived.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
14
Introduction

Template generation
Pressure, Altitude, Azimuth
Pressure is one of the most common dynamic
information of on

line signature. Some devices can capture
additional information, such as Azimuth (the clockwise
rotation of cursor about the z

axis) and altitude (the angle
upward toward the positive z

axis).
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
15
Introduction

Template generation
Center of Mass, Torque, Curvature

ellipse S
1
and S
2
The five features were defined by
Vishvjit S. Nalwa [6].
Torque measures the area swept by the vector of pen
position. S
1
and S
2
measure the curvature ellipse based on
moments. The distance measure used here is cross

correlation (Pearson's r) weighted by the consistency of
points.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
16
Introduction

Template generation
Average, average positive speed on
X

axis ,average positive speed on
Y

axis, total
signing duration.
Lee et al. [3]
lists two sets of scalar features (over 100
features). These four features have the highest
preference in the first set. The distance measure is
Euclidean norm.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
17
Introduction

Template generation
Cos(a), sin(a), Curvature
a
is the angle between the speed vector and the X

axis.
The three features are proposed by
Jain et al. [10].
It also
proposes coordinate sequence differences.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
18
Introduction

Template generation
Features (examples)
0
20
40
60
80
100
120
140
160
180
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
0
20
40
60
80
100
120
140
160
180
3
2
1
0
1
2
3
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
2
1
0
1
2
3
A Signature
sample
X
coordinates
Y
coordinates
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
19
Introduction

Template generation
Features (examples)
0
20
40
60
80
100
120
140
160
180
3
2
1
0
1
2
3
4
5
Torques
S
1
of Curvature ellipse
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
2
1
0
1
2
3
A Signature
sample
0
20
40
60
80
100
120
140
160
180
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
20
Introduction

Template generation
Feature comparison
0
20
40
60
80
100
120
140
160
180
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
0
20
40
60
80
100
120
140
160
180
2
1.5
1
0.5
0
0.5
1
1.5
2
2
1.5
1
0.5
0
0.5
1
1.5
2
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
2
1.5
1
0.5
0
0.5
1
1.5
2
2.5
3
2
1
0
1
2
3
X

coordinates (genuine)
X

coordinates (forgery)
Genuine sig.
Forgery sig.
Only
X

coordinates can not distinguish them!
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
21
Introduction

Template generation
We have following experience:
1). One of the most reliable features is the
shape
of the
signature. Shape is described by the combination of X, Y

coordinates [X,Y].
2). The second reliable feature is the
speed
of writing.
To represent shape and speed, each signature is a 3

D
sequence: Sig
i
=[X
i
, Y
i
, V
i
], where V is the sequence of
speed magnitude. Then we use
ER

Squared
to match
two signatures and return a
Confidence
of similarity
(0%

100%). The details will be given later in section
2..
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
22
Introduction

Distance measures
Most commonly

used measures
Euclidean norm
Weighted Correlation
Where f(l), h(l) are functions of two signatures and w(l) is the
consistency function.
Dynamic Time Warping (DTW)
Elastic sequence matching. Very good for on

line signatures.
N
i
x
x
X
X
D
1
2
2
1
2
1
)
(
)
,
(
dl
l
h
l
w
dl
l
f
l
w
dl
l
h
l
f
l
w
r
)
(
)
(
)
(
)
(
)
(
)
(
)
(
2
2
2
2
2
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
23
Introduction

Distance measures
DTW
S1
S2
S1
S2
One

One alignment
Dynamic alignment
Both Euclidean norm and
correlation assume one

one
alignment. Easy but brittle!
Elastic alignment is more robust
for sequences, at the cost of
computational resources.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
24
Introduction

Distance measures
DTW
)}
1
,
(
),
1
,
(
),
,
1
(
min{
)
(
)
,
(
2
j
i
D
j
i
D
j
i
D
x
x
j
i
D
j
i
Current cost
Recursive cumulative cost
The calculation of matrix D. The DTW
warping path in the matrix D is the
path which has minimum average
cumulative cost. The unmarked area
is the constraint imposed by i

j<w (w
is the width of the allowed margin).
Subject to optional constrain: i

j<w
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
25
Introduction

some remarks
Remarks on some research directions in on

line signature
verification
Segmentation?
Signature is an art of drawing, not limited to some kind language.
A Segmentation method by Perceptually Important Points was
proposed by Jean

Jules Brault et al [7]. Many works have been
done to apply segmentation to signature verification. Problems:
1)The consistency of segmentations? 2)If DTW is used as measure,
Segmentation is of little necessity, because those Perceptually
Important Points can be aligned accurately by DTW.
2
1.5
1
0.5
0
0.5
1
1.5
2
3
2.5
2
1.5
1
0.5
0
0.5
1
1.5
2
2
1.5
1
0.5
0
0.5
1
1.5
2
3
2.5
2
1.5
1
0.5
0
0.5
1
1.5
2
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
26
Introduction

some remarks
User

dependent distance threshold?
Distance (Euclidean, DTW, etc.) for dissimilarity measure is
not
intuitive. In real applications, users tends to ask:
how similar is the
two signatures? Or, what is the confidence that this signature is
genuine?
It is intuitive to answer: their similarity confidence is 90%!
(instead of saying their distance of dissimilarity is 5.8).
It is hard to obtain a user

dependent threshold, because of limited
genuine samples. Though it is a choice to use the genuine samples
from other users as forgeries, it won
’
t help much on determining
the threshold.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
27
Introduction

some remarks
Statistics based methods?
Again because we can not expect many signature
samples, statistics based methods, such as Markov
Model, is hard to achieve high performance.
Artificially generate genuine signatures? Using random
forgeries or use the signatures from other users?
Possible ways.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
28
ER
2
: An Intuitive Similarity Measure for On

Line
Signature Verification
1.
Introduction

On

line signature verification
√
2
.
ER
2
: Intuitive Similarity Measure
3. Experimental Results
4. Demo
–
CUBS
signature verification system
5. Conclusion
6. References
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
29
ER
2
: Intuitive Similarity Measure
Similarity measures must satisfy:
The similarity of intra

class is very high. (so that
we can accept genuine signature)
The similarity of inter

class is very low. (so that
we can reject forgery).
An intuitive score range, like 0

1.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
30
ER
2
: Intuitive Similarity Measure
Traditional Linear Regression
2000
1500
1000
500
0
500
1000
1500
2000
0
50
100
150
200
250
300
350
400
2000
1000
0
1000
2000
3000
4000
0
50
100
150
200
250
300
350
400
2000
1000
0
1000
2000
3000
4000
2000
1500
1000
500
0
500
1000
1500
2000
500
0
500
1000
1500
2000
2500
3000
3500
4000
0
50
100
150
200
250
300
350
400
500
0
500
1000
1500
2000
2500
3000
3500
0
50
100
150
200
250
300
350
400
500
0
500
1000
1500
2000
2500
3000
3500
4000
500
0
500
1000
1500
2000
2500
3000
3500
Similarity:
91%
Similarity:
31%
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
31
ER
2
: Intuitive Similarity Measure
Linear Regression
Given two sequences
X=(x
1
,x
2
,
…
, x
n
)
,
Y=(y
1
,y
2
,
…
,
y
n
)
, then the similarity by R
2
of
X
and
Y
is:
n
i
i
n
i
i
n
i
i
i
Y
y
X
x
Y
y
X
x
R
1
2
1
2
1
2
2
)
(
)
(
)]
)(
(
[
R
2
named R

squared because R
2
= (r)
2
, where r is
Pearson
’
s correlation r.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
32
ER
2
: Intuitive Similarity Measure
Extended Regression
Traditional regression handles two
1

dimentional
sequences. We extend it to multi

dimensional
sequences as follows:
M
j
n
i
j
ij
M
j
n
i
j
ij
M
j
n
i
j
ij
j
ij
Y
y
X
x
Y
y
X
x
ER
1
1
2
1
1
2
1
1
2
2
)
(
)
(
))]
)(
(
(
[
We name it ER
2
since is an extension from 1

D to multi

D
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
33
ER
2
: Intuitive Similarity Measure
The intuition of ER
2
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
34
ER
2
: Intuitive Similarity Measure
Remarks on Linear Regression
Advantages: Invariant to scale and translation; Similarity
(Goodness

of

fit) makes sense.
Disadvantages: One

one alignment, brittle.
S1
S2
S1
S2
One

One alignment
Dynamic alignment
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
35
ER
2
: Intuitive Similarity Measure
We couple ER
2
with DTW

based Curve Matching
Dynamic Alignment by DTW.
However, we found direct
DTW on two signatures is not
very robust.
We use Curve Matching, which is
to calculate the total cost of
changing one curve to fit another
curve. The dynamic programming
of DTW is used to realize the
calculation.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
36
ER
2
: Intuitive Similarity Measure
DTW

based Curve
Matching
Suppose we have two curves
C
and
C
’
. Curve matching is
actually:
))
'
(
),
(
(
)
'
,
(
_
C
speed
C
speed
DTW
C
C
Match
C
Where speed(
C
)
i
=
C
i+1

C
i.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
37
ER
2
: Intuitive Similarity Measure
ER
2
coupled with Curve Matching
The DTW warping path in the matrix is the path
which has minimum average cumulative cost.
The unmarked area is the constraint that path is
allowed to go.
]
,...
,
,
[
2
2
1
m
y
y
y
y
Y
]
,...
,
,
[
3
2
1
n
x
x
x
x
X
)
,
(
2
X
Y
ER
Similarity
( y
2
is
matched x
2
, x
3,
so
we extend it to be two
points in Y sequence.)
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
38
Experimental Results
1.
Introduction

On

line signature verification
√
2. ER
2
: Intuitive Similarity Measure
√
3.
Experimental Results
4. Demo
–
CUBS
signature verification system
5. Conclusion
6. References
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
39
Experimental Results
Signature database
The released signature datasets by SVC( First
International Signature Verification Competition). SVC
released the signatures of 80 individuals, 20 genuine and
20 skilled forgeries each.
Methods comparison
ER
2
coupled with Curve Matching
Vs.
Curve Matching without ER
2
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
40
Experimental Results
Enrollment
Enroll 6 genuine signatures from each
individual.
Preprocessing
Only X,Y

coordinates are used. Other information, such
as
Pressure, Altitude, Azimuth are not used in the
experiments
. 1) Smooth the raw sequence by Gaussian
filter. 2) Rotate if necessary. 3) Normalize
each signature by:
)
(
X
X
X
X
)
(
Y
Y
Y
Y
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
41
Experimental Results
a) FRR and FAR of ER
2
(coupled with Curve Matching). b) FRR and
FAR of Curve Matching (without ER
2
). Both using universal
threshold.
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
EER=20.9%
FRR
FAR
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
EER=7.2%
FRR
FAR
a)
b)
Threshold (%)
Threshold (%)
Error Rate (%)
Error Rate (%)
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
42
Experimental Results
Table 2.
EERs with universal or user

dependent
threshold. Skilled forgeries are provided by
the dataset, while random forgery means the
forgeries are selected from the signatures of
different individuals.
*The results of SVC are available at
http://www.cs.ust.hk/svc2004/results.html
. We are team 14.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
43
Experimental Results
A project regarding on

line signature
Recently, we have a Multimodal
Biometrics project supported by
US Army Laboratory. It requires
to test signatures from 1000
individuals, each 2 as enrollment
and 3 as queries. We collected
330 individuals so far. The
preliminary ROC based on ER
2
is:
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
44
Experimental Results
1.
Introduction

On

line signature verification
√
2. ER
2
: Intuitive Similarity Measure
√
3. Experimental Results
√
4.
Demo
–
CUBS
signature verification system
5. Conclusion
6. References
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
45
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
46
Demo
–
CUBS Sign. System
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
47
Conclusion
We propose ER
2
as a similarity measure for multi

dimensional sequence matching. Signature verification
system can use ER
2
coupled with curve matching for
intuitive similarity output and higher performance as
well. The experimental results are encouraging,
although we have to notice that further evaluation on
large and real databases is necessary.
Our future work will explore the feasibility of ER
2
on
dynamic features like pressure, speed, etc.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
48
References
[1] Rejean Plamondon, Guy Lorette. Automatic Signature Verification and
Writer identification

the state of the art. Pattern Recognition, Vol.22,
No.2, pp.107

131, 1989.
[2] F. Leclerc and R. Plamondon. Automatic signature verification: the
state of the art 1989

1993. International Journal of Pattern
Recognition and Artificial Intelligence, 8(3):643

660, 1994.
[3] Luan L. Lee, Toby Berger, Erez Aviczer. Reliable On

line Human
Signature Verifications Systems. IEEE trans. On Pattern Analysis and
Machine Intelligence, Vol. 18, No.6, June 1996.
[4] R. Plamondon. The Design of On

line Signature Verification System:
From Theory to Practice. Int
’
l J. Pattern Recognition and Artificial
Intelligence, vol. 8, no. 3, pp. 795

811, 1994.
[5] Mario E. Munich, Pietro Perona. Visual Identification by Signature
Tracking. IEEE Trans. On Pattern Analysis and Machine Intelligence,
Vol. 25, No. 2, pp. 200

216, February 2003.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
49
References
[6] Vishvjit S. Nalwa. Automatic On

line Signature Verification.
Proceedings of the IEEE, Vol. 85, No. 2, pp. 215

239, February
1997.
[7] Jean

Jules Brault and Rejean Plamondon. Segmenting Hanwritten
Signatures at Their Perceptually Important Points. IEEE Trans. On
Pattern Analysis and Machine Intelligence, Vol, 15, No. 9, pp. 953

957, September 1993.
[8] Taik H. Rhee, Sung J. Cho, Jin H. Kim. On

line Signature
Verification Using Model

Guided Segmentation and Discriminative
Feature Selection for Skilled Forgeries. Sixth International
Conference on Document Analysis and
Recognition (ICDAR '01), September, Seattle, Washington, 2001.
[9] Thomas B. Sebastian, Philip N. Klein, Bejamin B. Kimia. On Aligning
Curves. IEEE Trans. On Pattern Analysis and Machine Intelligence,
Vol. 25, No. 1, January 2003.
CUBS
Center for Unified Biometrics and
Sensors, NY, USA
50
References
[10] A.K. Jain, Friederike D. Griess and Scott D. Connell. On

line
Signature Verification.
Pattern Recognition
, vol. 35, no. 12, pp.
2963

2972, Dec 2002.
[11] K. Huang and H. Yan,
“
On

Line Signature Verification Based on
Dynamic segmentation and Global and Local Matching,
”
Optical Eng.,
vol. 34, no. 12, pp. 3480

3487, 1995.
[12] G. Lorette and R. Plamondon,
“
Dynamic Approaches to Hand

written Signature Verification,
”
Computer Processing of Hand

writing,
pp. 21

47, 1990.
[13] R. Martens and L. Claesen,
“
On

Line Signature Verification by
Dynamic Time

Warping,
”
Proc. 13th Int
’
l Conf. Pattern Recognition,
pp. 38

42, 1996.
[14] B. Wirtz,
“
Stroke

Based Time Warping for Signature Verification,
”
Proc. Int
’
l Conf. Document Analysis and Recognition, pp. 179

182,
1995.
Comments 0
Log in to post a comment