UIDAI Role of Biometric Technology in Aadhaar Authentication

licoricebedsΑσφάλεια

22 Φεβ 2014 (πριν από 3 χρόνια και 6 μήνες)

267 εμφανίσεις

h








UIDAI


Unique Identification

Authority of India

Planning C
ommission, Govt. of India (GoI)
,

3rd Floor, Tower II,

Jeevan Bharati Building
,

Connaught Circus,

New Delhi 110001



Role of Biometric
Technology in
Aadhaar
Authentication

IRIS
Authentication Accuracy


PoC
Report

Date:
14

September 2012


Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
2

of
36

Executive Summary

UIDAI
has collected iris images of residents during enrollment, and
successfully leveraged iris modality for
de
-
duplication
.
A Proof of
Concept (PoC) study was carried out to assess the feasibility of
using
iris for on
-
line biometric authentication in the Indian context
.
The
findings of this study are presented in this report.

This report covers the design framework, field implementation, data
collection and analysis
.
It interprets empirical results to assess
effectiveness of iris technology, authentication processes and devices
.
It
concludes with

a se
t of
observations and findings

for

iris
authentication process and device ecosystem.

Assessment of iris feasibility can be further divided into

1.

Coverage: Can residents use iris efficiently and conveniently for Aadhaar authentication?

2.

Accuracy: How accurate
ly can iris authenticate a resident?

3.

Device readiness: Are there commercially available devices capable of performing
authentication?

4.

System readiness: Is the entire authentication system ready for performing iris
authentication?

Study

Process
: PoC was conducted in a manner similar to
the
previous study
[
UIDAI
,
FP, 2012
]
.
Residents with Aadhaar number were asked to participate in the test
.
In
the
first part,

scenario test,
each resident was authenticated on
-
line on one of the eight cameras
.
In the second part, technology
test, the resident’s iris images were captured on all eight cameras
.
In both parts, extensive logs
were generated for off
-
line analysis
.
During the analysis, the data was first reviewed and cleansed of
Feasibility of Iris bas
ed on
-
line Authentication

The empirical results clearly demonstrate iris authentication to be viable in
Indian context.

Findings

% of Residents

Single
-
eye cameras

Dual
-
eye cameras

Authenticated in first try

95.89

99.29

Authenticated in multiple tries

99.21

99.40

Failed authentication
(FRR+FTC)

0.79

0.60


Device Readiness
:

Six

different devices with a variety of form and function are
available to form competitive vendor eco
-
system.

System Readiness
:

M
edian time
for end
-
to
-
end authentication
was less than
one
minute

over GPRS mobile network in semi
-
rural setting.

Figure
1

Young resident at POC

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
3

of
36

exceptions and anomalies
.

Set
-
up:
The

P
o
C was conducted in semi urban setting at
centers in Nanjagud taluk in Mysore district of Karnataka
between May 27
th

and July 30
th

2012
.
The PoC

centers
were designed to resemble expected ground reality
.
No
attempt was made to create an ideal situation
.
In on
-
line (scenario) testing, 17,990 online authentication
transactions were performed by 5,747 residents
.
The
online authentications were per
formed on a GPRS based
mobile network
.
In the off
-
line (technology) study,
5,833 residents went through 40,148 capture sessions
on 8 authentication cameras
. All the cameras used in
the POC studies were

obtained through an Expression of
Interest (EOI)
.
In

all, 215,342 iris images were capture
d

during this process.

Iris PoC was carried out using the Aadhaar authentication system described in an earlier report
[UIDAI, FP, 2012]
.

Results:

The
PoC was conducted to obtain sample that demographically represents
Indian population
.
At the
end, the sample used has

a

slight over representation of women and seniors (over 65 years) and
under
-
representation of children (below 15 years) as compared to Aadhaar enrollment data.

Coverage
:
The coverage achieved by iris auth
entication was high, demonstrating the potential of iris
to be an inclusive modality
.
Out of 5,833 residents, only one resident was not able to use
any of
the

eight cameras, and 18 additional residents
(
0.3
1
%)

could not use four or more cameras
.
Thus,
there is no inherent technical or physiological limitation to using iris for authentication
.
It
should be possible for 99.
6
7% of population to conveniently
attempt
iris

authentication
.
In
other words, failure to capture
(FTC)
rate is in the range of 0.3
3% with current capture software
.
The study

has

identified the root cause and suggested methods to improve vendor’s capture
algorithm to achieve even higher coverage

through this report
.

Accuracy:

Table
1

shows the True
A
ccept Rate (TAR) for different authentication modes with two
attempts averaged over six “good” cameras
.
The data shows that high accuracy can be obtained
using both single and dual eye cameras
.
In
summary, False Reject Rate (FRR) of less than 0.5% is
achievable whenever two irises are matched on single or dual eye cameras.

Authentication Mode

Single eye camera

Dual eye camera

One iris authentication

96.21%

Not Applicable

Two irises authentication

99.54%

99.73%

Table
1
:

True Accept Rate at FAR = 1e
-
6

Accuracy insight and conclusions:

a)

Manual inspection of iris images provided insight into
the
root cause of failure to capture
and false rejects cases
.
Over two
-
third residents

in question had eye
deformity

arising from
certain types of surgeries or eye conditions

and hence

s
ome
cameras were unable to
capture the image which resulted in FTC
.
Others cameras were more lenient in capturing
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
4

of
36

the image but authentication failed
.
These residents
with
special eye condition

will
not be
able to authenticate using irises because either the cameras will be unable to capture or
even when captured, they may fail to authenticate successfully as the

current capture and
matcher software are
unable to process them
.
With
this
situation clearly identified

and
communication with vendors in progress
, future devices and matching algorithms
could

be
enhanced to accommodate some of these condition further increasing iris accuracy
.
Additional study
may be also needed to more accurately estimate and analyse eye surgery
cases

(particularly, cataract surgeries)

across India.

b)

Fingerprints of these individuals with
special

eye conditions were reviewed
.
For most of the
residents
,
i
t was estimated that the
ir fingerprints are of sufficient quality to enable
fingerprint based biometric authentication
.

c)

It was observed that DET curve for iris is flat implying litt
le deterioration in FRR rate with
stringent FAR
.
This binary behaviour


either a good match or no
t a match


could
be
uniquely useful in situations requiring high level of assurance such as money transfer or
secure access control
.
Conversely, running iris authentication at less stringent FAR
(such as 1
in 10,000
) will not improve the FRR
.

d)

Two iris a
u
thentication
shows material improvement over one iris
authentication
.
It is
estimated that additional 3% residents could be reliably authenticated by expanding
authentication to include two irises
.
For all age categories, accuracy continues to be higher
than 98.94
%
.
Among those children have performed best, followed by adults
.
Some of the
seniors could not be authenticated due to special eye conditions
.
This indicates the need to
tune the capture and matcher algorithms specifically for seniors to accom
modate for special
eye conditions
.

Device readiness:
Six

of the iris
cameras
participating in this study
perform
ed

at required
accuracy
level.
This
will assure competitive vendor eco
-
system

to the service providers willing to use iris
authentication
.
Key

variables determining effectiveness of a camera are

a.

Single eye and
d
ual eye capture: Four cameras each of single eye and dual eye were tested
.
B
oth single eye and dual eye devices were found suitable for iris authentication.

b.

Capture aid: Various physical

and optical aids were utilized by different cameras
.
Cameras

with visual indications

on the camera

for capture process, eye alignment
, actionable
feedback

and capture completion were
materially easier

to use
.

c.

High Capture quality


In addition to quality

algorithms being in the device to capture high
quality images, vendors could incorporate specific suggestions made in this document to
further enhance image quality.

All participating camera vendors were asked to supply IRIS segmentation SDK which produce
s
image in formats KIND 1, 2, 3 and 7 (also known as Uncropped, VGA, Cropped and Cropped &
Masked respectively) compliant to ISO 19794
-
6

standard

[ISO 19794
-
6
, 20
11
]
.
The camera
vendors provided predominantly only two formats, KIND 1 and KIND 2
.
The imag
e size is a
dominant factor in network latency for mobile networks
.
While this PoC used KIND 2
compressed image
of size 15 KB, use

of KIND 7 would have reduced it to 2 KB resulting in
speedier authentication.

As a result of factors mentioned above, there
was a difference in performance of different
cameras
.
In each category of single and dual eye cameras, one camera was an outlier compared
to median performance of the group
.
For example, one camera demonstrated a
failure to
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
5

of
36

capture

(FTC) rate of
about
10
% when the median of peer cameras was below 0.5%
.
Some of
the outliers
were also judged to be more difficult to use in the field and were
not considered

in

accuracy analysis
.
Suggestions

made
in this report

would improve performance
of
cameras
,

especiall
y the
se

outlier

cameras
.

System readiness:
The overall system behaved reliably and conveniently for both resident and
operators
.
A f
ew cases of system failure were observed
.
Two important metrics measure user
friendliness of the iris based
authentication.

a.

Authentication time
.
Using a GPRS based semi
-
rural network, median time
across
all devices
was less than one minute
.
Authentication time is defined
as
time
taken

to complete full
transaction including capture of two irises, local image qu
ality check, operator review,
transmission, backend processing and authentication
response
.

b.

Round trip time
.
Less than a quarter of the authentication cycle was transmission and
backend matching time
.
Median round trip was less than 10 seconds,

while onl
y
7.8% of
transactions

took more than 15

sec
onds.

Previous PoC demonstrated that
the
current setup is capable of supporting sustained fingerprint
authentication speed
s

of one million per hour
.
It has been demonstrated repeatedly in the
literature that iri
s matcher is more efficient than fingerprint matcher
.
Therefore no independent
tests were performed using iris matcher
.
Based on published data, it is safe to state that Aadhaar is
capable of sustaining iris authentication ra
te of over one million per ho
ur.

However UIDAI will do
benchmarking of t
its
backend system to validate the peak load scenarios for iris authentication also.

Findings

and
N
ext
S
teps

The empirical results clearly demonstrate iris authentication to be viable in Indian context
.
With
current level of device readiness for iris capture, it is
cap
able
of

provid
ing

coverage for
99.
6
7%

of
population with authentication accuracy of above
99.5%

using two irises and up to two attempts
.
Suggestions made in this document for the vendors
,

o
nce implemented, will improve the rates
further
.
The overall systems


network and software
-

have shown to meet desired requirements in
real life condition
.
Finally, six different devices with variety of form and function are available to
provide compet
itive vendor eco
-
system.

The PoC also provided a number of insights that could further improve performance of device and
system
.
Three key improvements are

a.

Improvement in matcher algorithm and device capture software to handle residents with
special eye c
onditions.

b.

Support for KIND 7 image format by device vendor to help reduce image size during
transmission to about 2 KB from
the
current 15KB.

Several ergonomic improvements in device design to aid in easier and more accurate
capture
.
These cover visual a
ids for proper placement of camera and appropriate visible
light source inside certain types of camera.

UIDAI would be working with the biometric ecosystem to implement the findings mentioned in this
document and take up further field studies to fine
-
tune
iris authentication technology. These studies
would also lead to formulation of iris device specifications which will be used for certification by
STQC.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
6

of
36

Abbreviations

API

Application Programming Interface

ASA

Authentication Service Agency

AUA

Authentication User Agency

EOI

Expression of Interest

CIDR

Central
Identity

Data Repository

DET

Detection Error Tradeoff

NIST

National Institute of Standards and Technologies

IREX

Iris Exchange

FAR

False Accept Rate

FRR

False Reject Rate

FTC

Failure To

Capture

IEC

Information
,

Education and Communication

KYC

Know Your Customer

PDS

Public Distribution System

PoC

Proof of Concept

SDK

Software Development Kit

STQC

Standardisation

Testing and Quality Certification Directorate

TAR

True Accept Rate

(1


FAR)

UIDAI

Unique Identification Authority of India

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
7

of
36

Contents

Executive Summary

................................
................................
................................
................................
.

2

Abbreviations

................................
................................
................................
................................
..........

6

1

Aadhaar Authentication

................................
................................
................................
................

10

2

Objective

................................
................................
................................
................................
.......

11

3

Design and Field Implementation

................................
................................
................................
.

12

3.1

Introduction to PoC studies

................................
................................
................................
..

12

3.2

Iris Authentication Concepts

................................
................................
................................
.

12

3.3

Test Methodology

................................
................................
................................
.................

13

3.4

Resident and operator communication and training
................................
............................

17

3.5

Data Quality & Data Analysis

................................
................................
................................

18

4

Feasibility of using iris for authentication

................................
................................
.....................

19

4.1

Demographic

Profile

................................
................................
................................
.............

19

4.2

Coverage

................................
................................
................................
...............................

20

4.3

Accuracy results from technology test

................................
................................
.................

21

4.4

System findings

................................
................................
................................
.....................

24

4.5

Devices Findings

................................
................................
................................
....................

25

4.6

Other findings

................................
................................
................................
.......................

27

5

Observations & Findings

................................
................................
................................
...............

28

5.1

Coverage

................................
................................
................................
...............................

28

5.2

Accuracy

................................
................................
................................
................................

28

5.3

Device Readiness

................................
................................
................................
..................

28

5.4

System Readiness

................................
................................
................................
..................

29

5.5

Findings

................................
................................
................................
................................
.

29

5.6

Way Forward

................................
................................
................................
.........................

30

6

References

................................
................................
................................
................................
....

31

7

Appendix


Camera related findings

................................
................................
.............................

32

7.1

Capture distance and Capture Volumes

................................
................................
...............

32

7.2

Image Capture and Output

................................
................................
................................
...

32

7.3

Capture aids

................................
................................
................................
..........................

33

7.4

Detection and Capture of iris

................................
................................
................................

34

7.5

Dual eye vs. single eye cameras

................................
................................
............................

35

7.6

Capture Distance

................................
................................
................................
...................

36

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
8

of
36


Figures


Figure 1 Young resident at POC

................................
................................
................................
..............

2

Figure 2 POC Center at Nanjangud

................................
................................
................................
.......

12

Figure 3 Residents waiting for their turn

................................
................................
..............................

12

Figure 4 Resident at POC site

................................
................................
................................
................

15

Figure 5 Screen shot of Client UI

................................
................................
................................
...........

16

Figure 7: IEC material used during POC

................................
................................
................................

17

Figure 6: Announcer

................................
................................
................................
..............................

17

Figure 8: Age distribution of participants in study compared to Aadhaar enrolment

.........................

19

Figure 9: Gender distribution of participants in study compared to Aadhaar enrolment

...................

20

Figure 10: ICCE Cataract surgery

................................
................................
................................
...........

20

Figure 11: Occlusion

................................
................................
................................
..............................

21

Figure 13 DET Curve
-

Single and Dual Iris Authentication

................................
................................
...

23

Figure 12 FRR by matchers
................................
................................
................................
....................

23

Figure 14: Resident authentication time

................................
................................
..............................

24

Figure 15: Network Latency histogram

................................
................................
................................
.

24

Figure 16: Image Formats

................................
................................
................................
.....................

25

Figure 17: Sample images of residents providing samples during POC

................................
................

26

Figure 18: Age wise DET chart

................................
................................
................................
..............

27

Figure 19 Resident with special condition Eyes

................................
................................
....................

35

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
9

of
36

List of
Tables


Table 1: True Accept Rate at FAR = 1e
-
6

................................
................................
................................
..

3

Table 2: Vendor Participation

................................
................................
................................
...............

16

Table 3: Failure to Capture

................................
................................
................................
...................

20

Tabl
e 4: Camera
-
wise FTC

................................
................................
................................
.....................

21

Table 5: Resident wise Accuracy

................................
................................
................................
...........

22

Table 6: Camera wise Reject Rates (2 irises with 2 attempts)

................................
..............................

22

Table 7: Single
Eye Camera
-

Good and all single eye camera TAR

................................
......................

22

Table 8: Dual eye camera
-

Good and all Dual eye camera TAR

................................
...........................

23

Table 9: True Accept Rate at FAR = 1e
-
6.

................................
................................
.............................

28

Table 10 Camera Capture distance and capture volume information

................................
.................

32


Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
10

of
36

1

Aadhaar Authentication

The Unique Identification Authority of India (
UIDAI) has been created with the mandate of providing
a Unique Identity (Aadhaar) to all residents of India
.
The CIDR processes these enrolments by de
-
duplicating them to ensure uniqueness and then issues Aadhaar numbers
.
As of August 2012, nearly
200
million residents have been enrolled in Aadhaar

and over 195 million of them have been issued
with Aadhaar numbers

One of the mandates given to UIDAI is to define usages and applicability of Aadhaar for delivery of
benefit services
.
The Aadhaar number, wh
ich uniquely identifies a resident, will give individuals
means to clearly establish their identity to public and private agencies across the country for service
delivery
.
UIDAI provides online authentication using the resident’s demographic and biometric

information to support Aadhaar
-
enabled delivery of services.

Aadhaar

Authentication

is the process wherein, Aadhaar number along with the Aadhaar holder’s
personal identity data is securely submitted to the CIDR for matching, following which the CIDR
veri
fies the correctness thereof on the basis of the match with the Aadhaar holder’s identity
information available with it
.
To protect resident’s privacy, Aadhaar Authentication service responds
only with a “Yes/No” and no Personal Identity Information (PII)

is returned as part of the response.

Aadhaar Authentication supports several different types of authentication through a combination of
demographic fields, biometric fields as well as other methods such as one
-
time
-
password (OTP)
.
In
case of iris, the CI
DR server supports authentication of iris images compliant with ISO standard that
ensures vendor neutral, open format.

For further information on Aadhaar authentication system, please refer to

www.uidai.gov.in
.

This study analyzes the feasibility of using
iris as a modality for authentication
.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
11

of
36

2

Objective

The objective of the PoC was to determine the feasibility of using iris modality for on
-
line
authentication.

UIDAI has successfully leveraged iris modality in enrollment and has collected iris images of the
resident during enrollment
.
Iris is used for biometric de
-
duplication
.
This study was aimed at
assessing its use for biometric authentication in the Indian context
.

Iris as a biometric modality promises certain unique characteristics in Indian environmen
t
.
Iris
technology literature lists several benefits.

1.

The iris does not get worn out with age, or with use
.
In addition, iris authentication is not
impacted by changes in the weather.

2.

Iris image capture
does not require physical contact
.
Capturing iris
image is physically similar to
familiar practice of taking photographs.

3.

Iris capture requires simple instructions such as ‘look at the camera, keep your eyes wide open’.

4.

Fake iris is difficult to synthesize making it harder to impersonate
.

5.

Iris image canno
t be captured without resident cooperation.

6.

Cost and manufacturing of iris cameras have been positively affected by the spread
of
low cost
consumer cameras
.

Few global initiatives have empirically published results on iris based online authentication in a
context similar to Aadhaar
.
Therefore the goal for the study was
also
to

Characterize & propose optimal authentication setups (server & device level) for online
biometric authentication of residents using iris authentication device.

To achieve the goal, it is necessary to conduct
rigorous
field tests on sample population and to
measure feasibility in four

areas
:

A.

Coverage

indicates the percentage of the population that is able to conveniently and efficiently
utilize iris authentication
.
For any system to be utilized for large scale benefit delivery, ease of
use and speed of verification are also key measure of the overall coverage.

B.

Accuracy

measure percentage of time system is able to authenticate genuine resident while
rejecting attem
pt by imposter.

C.

System readiness

covers overall response time with particular emphasis on network latency,
reliability, stability and e
ase of
deployment

D.

Device

readiness

measures individual device’s ability to perform above three factors reliably
.
Particu
lar attention was given to understand different device characteristics and develop
recommendation for enhancing device performance.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
12

of
36

3

Design and Field Implementation

3.1

Introduction to PoC studies

There have been few studies regarding biometric
authentication in the Indian field context and it was
important to conduct rigorous PoC studies in order to
design and configure the UIDAI authentication system
.
A
number of such PoC studies have been carrie
d out, each
building on the learning from the previous ones
.
This
chapter describes the design and setup for this study
.

Continuing the methodology used in
the previous study
[UIDAI, FP, 2012]
, the iris PoC tests various parameters
using
[ISO 19795
-
2, 2007]
, which
provides the following definitions:

Scenario evaluation is an online evaluation of end
-
to
-
end system performance in a prototype or
simulated application
.
The utility of scenario test
ing stems from the inclusion of human
-
sensor
acquisition interaction in conjunction with the enrolment and recognition processes, whose
benefits include the following:



Ability to gauge impact of additional attempts and transactions.



Ability to collect
throughput results for resident authentication.

and

Technology evaluation is the offline evaluation of one or more devices and algorithms for the
same biometric modality using a corpus of samples
.
The utility of technology testing stems from
its separatio
n of the human
-
sensor acquisition interaction and the recognition process
.
It allows
for varying different parameters
to re
-
run the same sample image

[ISO 19795
-
2, 2007]
.

The tests in the iris PoC have been instrumented to collect sufficient data to condu
ct both a specific
scenario
evaluation
and follow on technology
evaluation
.

3.2

Iris A
uthentication

Concepts

Prior to presenting
the
authentication accuracy results,
t
erms used in this document are defined
below.

1.

Session:
V
isit
.
This PoC

uses only one session.

2.

Sample:
U
ser’s
biometric measures as output by the data capture subsystem
.
For e.g.
,
face
image
,

f
ingerprint
image and iris image are samples.

3.

Presentation:
S
ubmission
of a single biometric sample on the part of a user
.
It may
al
so
be
referred to as impression
.
On single
eye camera
, each
presentation

may generate one image
.
One presentation of two eye iris camera generates two images.

4.

Attempt:
S
ubmission
of
one (or a sequence of) biometric
samples to the system after it has passed quality check
.
NOTE
:

An attempt results in a template and can be used
to generate a matching score
.
There is a difference
between
the

definition of attempt
by ISO
and
t
his

PoC
att
empt
.
This
PoC attempt is after quality check
.
A
failure to acquire due to missing fingers/iris or inability to
Figure
2

POC
Center

at Nanjangud

Figure
3

Residents

waiting for their turn

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
13

of
36

even register image on scanner does not result in an attempt
.

5.

False reject rate (FRR):
P
roportion of verification transactions with truthful c
laims of identity that
are incorrectly denied
.


All data collected from the field included an operator
-
set flag for the
expected response (the Ground Truth)
.
Based on this, all ‘true’ authentication transactions were
rerun in the lab and the matching scor
es recorded
.
This allowed capture of the false reject rate
for a particular threshold.

6.

False accept rate (FAR):
P
roportion of verification transactions with wrongful claims of identity
that are incorrectly confirmed
.
Expected false requests were created
by pairing biometric data
collected from the field with an Aadhaar number other than the original Aadhaar holder
(imposter requests)
.
A large number of these requests were rerun and matching scores
recorded
.
This allowed capture of the false accept rat
e
for several thresholds
.
As many false
requests as necessary were created to provide statistically significant results.

7.

Failure
-
to
-
capture

(FT
C
) include
s:



A
ttempts where the biometric characteristic cannot be captured;



A
ttempts in which the extracted
features do not meet the quality control thresholds
;



A
ttempts for which the segmentation or feature extraction fail; and



Attempts where the raw sample was not acquired or did not meet quality thresholds are
not processed by the matching algorithm, and do n
ot generate matching scores.

8.

Transaction: Entire on
-
line process of capturing biometric sample, packaging it for on
-
line
submission, communicating to the backend server (CIDR), backend server matching and
receiving of the response
.
It began when the resid
ent provided
his/her
Aadhaar number and
ended when the response was received
.

9.

True Accept Rate (
TAR): TAR = 1


FRR
.

3.3

Test Methodology

Since collection of biometric data from the residents is an expensive process, the multi
-
modal
authentication PoC cover
ed

both scenario and technology testing in one session
.
The complete
process that included several more steps took about 15 minutes per resident and consisted of

1.

One Best Finger Detection procedure

(UIDAI, 2011)

(UIDAI, 2012)

2.

Two online fingerprint authentications (scenario test)

3.

One face picture

4.

One online iris authentication on one of the iris authentication camera
s

(scenario test)

5.

2 “best” images on
each of
8 iris authentication came
ras (technology test)

6.

2 “best” images on
each of
6 iris enrolment camera (technology test)

This report covers results of iris authentication only (steps 4 and 5).


T
he session setup consist
ed

of two parts
.
The resident
went

through both two parts
.
Part
1 cover
ed

scenario testing and part 2 cover
ed

technology testing.


Part 1: Scenario Testing
.
The scenario testing
utilized

Aadhaar’s
production

environment
.
It was performed on
-
line and in real
time
.
The

authentication frontend was connected with the CIDR
through mobile network
.
The resident
was

authenticated
in real
time against the enrolment data
.

Figure 4 Resident at POC site

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
14

of
36


Part 2: Technology Testing
.
The test collect
ed

a number of presentations (images) of iris for use in
o
ff
-
line analysis
.


Iris authentication devices come in a variety of
forms

and design
s
.
Available models
were

tested to
determine readiness of iris authentication including suitability,
usability,
interoperability and
performance of the camera
.


In scenar
io testing, it is important to measure process parameters such as end to end
transaction

time and number of attempts
.
Accuracy results were obtained from technology test
.
Technology
test also provided device level performance
.
Technology tests were done

using offline data
collection
.
Raw uncompressed images were captured
.
Images were compressed using JPEG 2000
lossy compression
during

off line analysis.

3.3.1

Iris Capture attempts

under scenario testing

Iris
images were

captured using maximum of 3 attempts of up to
3

presentations
.
All images were
captured automatically by the device, without operator forcing the capture
.

1.

Each successful presentation result
ed

in image capture
.
Unsuccessful p
resentation, where
the camer
a was unable to capture image was

still counted as presentation
.

2.

Each captured presentation
was

checked for quality using SDK’s quality algorithm
.
If the
quality
was

below acceptable threshold, the image
was

captured again (
i.e.,

second
presentation)
.
Th
e final image
was

either the first image meeting quality threshold or best
image among the three presentations, whichever comes first
.
Final image
was

sent t
o the
back end for
authentication
using the chosen mode of communication.

3.

If the
authentication
fa
iled
, the resident
was

asked to undergo step 1

& 2

again

up to two
additional
attempts
.

4.

FTC condition would occur if no image was captured after three attempts

as mentioned in
step 1
.

Single eye
camera
:

Each presentation
was

defined as image of one eye
.
Second attempt used
different eye from the first
.
I
n other words, the recommended multiple attempt sequence
was

right
eye, left eye, right eye
.
The switching of eyes effectively provides “best finger” stra
t
egy for two
eyes.

Dual eye

camera
:

Each present
ation
was

defined as one image of each eye (two iris images)
.
While
other

aspect of process
was

same (3 attempts of
3

presentations

each), the
maximum

number of
images
captured would be

twice that of single eye camera
.
Dual eye camera is effectively equi
valent
to using
two fingers for authentication in
previous studies
.

3.3.2

Iris Capture attempts

under technology testing

Maximum of two images of each eye were

captured using up to
5

attempts
.
All images were
captured automatically by the device, without
operator forcing the capture.



Each successful presentation
would
result in image capture
.
Unsuccessful p
resentation,
where the camera was

unable to capture image
,

was

skipped
.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
15

of
36



Each captured presentation
was

checked for quality using SDK’s
quality algorit
hm
.
If the quality is below acceptable threshold,
the image
was

captu
red again
.
The process continued

until two
quality images are captured

or maximum of five attempts,
whichever comes first
.
If
the process did not generate two
quality images
, two best
quality images of each eye
were

selected from available images.



In case of single eye camera, above steps
were

repeated for
each eye
.
In case of dual eye camera, even when two eyes are
captured, the above process
was
repeated until two images of
each eye is collected
.
In other words, regardless of camera
type, two “best” quality images of each eye
are

collected.



FTC condition was record
ed if no image could be captured in five attempts
.

3.3.3

Iris Stations

for scenario testing

The resident
was
authenticated at only
one
of the iris station
s
.
In other words,
with

eight

iris
camera stations, each station receive
d

approximately 1/
8
th

of the reside
nts
.
The distribution of
residents to the iris station
was
not controlled
.

1.

Iris authentication

during scenario testing
.

a.

All the cameras were
handled

by the operator during the test.

b.

Attempts: Section
3.3.1

details process to be used for multiple attempts.

c.

Quality check
:
UIDAI’s quality check SDK was used whenever the device vendor did not
supply it
.

d.

Image format
:
V
endor supplied software

to generate ISO image (VGA) was used
.
Images
were compressed using JPEG 2000 lossy compression
.

e.

Image selection
:
In case of one eye camera, best image was the first image that passed
quality check or the best image of the attempt
.
In case of two eye cam
era the images
within an attempt would be buffered and best image of each eye was sent.

2.

The operator
recorded eye position (left or right) using provided UI.

3.

Matching: The resident is authenticated when match occurs in any attempt
.
No further
biometric ca
pture is done at the station for the resident.

4.

If a resident is unable to authenticate at
the

station, the resident is not excluded from going to
technology testing section
.

3.3.4

Iris Stations

for technology testing

1.

The resident authenticated

using iris

through

the following steps
.

a.

All cameras
were
handled

by the operator.

b.

Auto c
apture function captured

iris image(s)
.
If
one
-
eye camera

was used
, one image
was

captured
.
If two
-
eye camera

was used
, two images
were

captured
.

c.

Eye: The default first attempt
was

righ
t eye unless operator determined

that the
resident’s right eye
was
not ideal
.
This
was

not relevant for two eye camera
s
.

d.

The operator
was

required to note the eye position in the software for single eye
camera

using the interface provided
.

e.

Attempts:

See Section
3.3.2
.

f.

Quality check
-

SDK quality check used in enrolment
was

used.

Figure
4

Resident at POC site

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
16

of
36

g.

On
-
line authentication
.
It
was
not necessary to perform on
-
line authentication
.
A
ll iris
stations
were off
-
line
.

h.

Raw images
were
stored.

2.

If a resident was

unable to provide iris image

at one station, the resident was

not excluded from
going to subsequent stations
.
For such residents,
FTC (Failure

to capture) event is recorded.

3.3.5

PoC Client Application and Data Collection

As part of the PoC client application, to measure different variables and compare/fine
-
tune
performance of various SDKs, detailed logs were captured
for further analysis
.
At
the
UIDAI
Technology Centre, all logs were analyzed and authentication requests were replayed on multiple
SDKs

to understand the impact of various biometric matching algorithmic changes and other
backend interventions that could b
e used to improve the authentication accuracy
.
POC application
prominently displayed quality parameters along with captured iris
.

3.3.6

Vendor Participation

In this study, Iris camera vendors were invited to participate based on an EOI

(EOI No. K
-
1102/16/2012
-
U
IDAI dated 7
th

March, 2012)
.
Following an initial evaluation, vendors selected for
the study, as summarized below:

Measure

Number

EOI responses received

11

Number of models ready with H/W and S/W by due date

8

Number of models participated in PoC

8

Number of OEM participated

6

Number of devices which were made ready post POC

and
hence could not participate in PoC

1

Table
2
:

Vendor Participation

Figure
5

Screen shot of Client UI

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
17

of
36

3.3.7

Field Setup

Authentication PoC

centers were designed to resemble expected ground reality
.
No attempt was
made to create an ideal situation
.
Some key considerations for setting up and managing the
authentication stations were as follows:



UIDAI procured
all
the components and standardi
zed them across all authentication PoC
stations


such as laptops, network connectivity, power

backup,

etc
.
EOI participants
provided cameras and capture software
.
UIDAI performed final software integration with
PoC UI
.



C
ommercially available telecom dat
a card (GPRS) used for network connection in order to
send authentication requests to CIDR
.



Regular electrical supply available at the PoC location, backed up with UPS/ generators was
used for the PoC exercise
.
Disruptions in the main power supply were no
ticed during the
course of the PoC
.



Sensors were deployed across various stations to provide equal opportunity to all
participating device technologies / OEMs, constrained by USB port availability.



The key consideration was to get each resident to authenti
cate using all the authentication
devices.



For resident mobilization, residents were requested to participate in
the Proof of
C
oncept studies through announcements by local
administration
.
Announcer was made to move into surrounding
villages and to draw t
he resident population for the studies.

During the field operation, some of the deployed devices intermittently
failed causing system
hang
-
ups

or freeze
.
Considering the larger PoC
objectives & crowd management issues, the devices had to be disconnected
t
emporarily
.
As a result not every resident went through every device
.

3.4

Resident and operator communication and training

Detailed communication material was developed to educate the operators
and residents to

conduct the process as well do’s and don

ts
.
Banners

and

Standees

were placed in and around the P
o
C site
.
Laminated sheets containing these instructions
were handed over to residents as they entered the P
o
C site in order for them to acquaint
themselves
with the process to follow during the capture
.
Operators were also trained using the
same instructions.


Figure
7
:

IEC material used during POC


Figure
6
:

Announcer

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
18

of
36

3.5

Data Quality & Data Analysis

Data analysis consisted of the following steps:



Analyzing the logs from field



Replaying the images captured from the field against the authentication server



Visually reviewing images to eliminate

errors
.
Several types of errors were identified and
rectified

o

Labeling errors
.
Wrong demographical
information, wrong label of right or left eye

o

Enrollment errors
.
Residents
whose authentication images could not be matched
because
the corresponding
enrolment data
consisted of
extremely low quality
images,

etc
.
These residents (8 residents)
were identi
fied and not considered in
the
accuracy analysis.


Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
19

of
36

4

Feasibility of using iris for authentication

This section summarizes the following areas from the iris study:



Demographics of the study participants

(scenario test)



Coverage
-

R
esident
population
who
were

able to have their iris captured

for authentication

(technology test)



Accuracy
-

Matching accuracy using various matchers

(technology test)



System readiness
-

Capture time, Network time related characteristics

(scenario test)



Devices readiness and related
findings

(technology test)

In scenario testing, 17,990 online authentication transactions were performed by 5,747 residents on
available

GPRS mobile network
.
The 5,833 residents participated in the iris technology test resulting
in 40,148 capture session
s

on 8 authentication camera
s
.
In all, 215,342 iris images were capture
during this process
.

4.1

Demographic
Profile

The demographic profile of participants in the iris study was compared with the profile of residents
enrolled in Aadhaar


both nationally as
well as within Mysore district
.
The comparison was done on
the basis of participant age bands, as well as gender.

It was found that children in 5
-
15 years range were under represented in the sample compared to
overall Aadhaar enrolment statistics, while s
enior residents (66+ years) were over represented
.
Similarly, females were over represented in the study.

The following charts show the age and gender distribution of all Aadhaar holders nationwide, in
Mysore district and participants in the iris PoC.


F
igure
8
:

Age distribution of participants in study compared to Aadhaar enrolment

0%

20%

40%

60%

80%

100%

India

Mysore

Poc

18

18

9

78

77

83

4

5

8

6
-
15 years

16
-
65 years

66 and above

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
20

of
36


Figure
9
:

Gender distribution of participants in stud
y compared to Aadhaar enrolment

4.2

Coverage

Coverage is primarily measur
ed by failure to capture (FTC) rate
.
Since devices with different
characteristics and quality were utilized, FTC rate over all devices does not accurately measure
maturity of iris technology
.
A new metric that measures inherent technology limitation in c
apturing
iris images in Indian context was also introduced
.
This measured the number of residents who could
not use any device and could not use several devices.

4.2.1

Resident coverage
across
multiple devices

Table
3
:

Failure to Capture

Two conclusions can be drawn from the statistics:



The coverage achieved by iris authentication was high, demonstrating the potential of iris

to be
an inclusive modality
.
There is no inherent technical or physiological limitation to using iris for
authentication.



The observed FTC rate of 0.33% (based on failure across 4 or more cameras) indicates that i
t
should be possible for
most
of
the
resi
dents to conveniently authenticate using iris
.

Since the resident’s iris image capture was attempted on different cameras in one session, it was
possible to manually review cases where image capture failed on one camera but succeeded on
another
.
While
the FTC rate
was
very small, the most frequent cause for those failed
was
due to eye
conditions such as deformed pupil/iris (mostly
result of
surgeries), antiquated cataract surgery or
other types of eye abnormalities
.
Majority of the cases were from
non
-
circular pupil s
hape

caused
due to cataract operations using Intra Capsular Cataract Extraction (ICCE) cryo
-
surgery technique
used prior to
1990
.



0%

20%

40%

60%

80%

100%

India

Mysore

Poc

52

50

45

48

50

55

Male

Female

Resident coverage across 8 cameras

Number of
residents

% failure

Total
no. of
Residents who participated

5833


Neither eye could be captured in all cameras

1

0.017%

O
nly left eye was captured (
right eye not captured)

3

0.051%

O
nly right eye was captured

(
left eye not captured)

9

0.154%

F
ailed iris capture
(either eye)
on 4 or more cameras

1
8

0.308%

Figure
10
:

ICCE Cataract surgery

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
21

of
36

Figure
11
:

Occlusion

Other
causes observed were from
:



Residents with squint eyes often suffered from FTC
especially
on dual eye camera
.
Single eye
cameras are able to capture one eye
with less difficul
ty.



Some residents could not open their eyes sufficiently leading to FTC or false rejects

(
Figure
11
).



Some residents suffered from involuntary movement
of the eye
-
ball
(
Nystagmus
)
and hence
had
an unstable iris resulting in FTC.





In the analysis, failure due to segmentation, if any, was included in FTC. Failure in template
generation, if any, was included in false reject statistics.

4.2.2

Failure to Capture (FTC) by Camera

FTC rates are computed by
device and can provide another perspective on the capture failure.

Single Eye Camera

FTC

Dual Eye Camera

FTC

A

0.12%

E

0.61%

B

2.01%

F

2.13%

C

0.30%

G

9.84%

D

0.10%

H

0.31%

Table
4
:

Camera
-
wise FTC

The failure to capture (FTC)
of individual iris cameras varies widely by camera as shown in

Table
4
.
When compared to median performance of
peer

devices, camera “G” performed poorly and

was

judged to be an outlier.

Improvements in capture software and ergonomics can enhance camera performance
.
These issues
are discussed in detailed
as part of
device recommendations under Section
4.5

and in Annexure

4.3

Accuracy results from
technology

test

The most common measure of accuracy is True Accept Rate (TAR) and False Accept Rate (FAR)
.
Just
like coverage it is instructive to use additional measure, called resi
dent accuracy
,

to understand
behavior of iris technology across cameras
.

All biometric matcher
s

require FAR threshold as input to calculate matching score which is used to
determine FRR
.
Accuracy calculations shown below are done
at

FAR of
1e
-
6

or 1 in 1
,
0
00,000
.

4.3.1

Resident Accuracy

Of the 5833 residents who participated,
5793

resident
s

were able to successfully authenticate using
at least one authentication camera using either their left or righ
t eye
.
Of remaining residents, 8

could not be authenticated due to incorrect enrolments (i.e
.
issues with enrollment gallery), while
13

residents (0.
22
%) could not be successfully authenticated using either the left or right iris on all
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
22

of
36

cameras
.
It
is indicative of

issue
s inherent to th
e image

and not
capture or
matching algorithm
.
19
residents were

FTC
in at least four

cameras.

Table
5
:

Resident wise Accuracy

4.3.2

Device Accuracy

The accuracy of individual iris cameras varie
d

widely
.
The FRR achieved using up to 2 attempts of
both left and rig
ht iris (FAR of
1e
-
6
) is shown in
Table
6

for single eye and dual eye camera
.
Compared to median performance of peer groups, two cameras


C and G


were considered to be
outlier
.
Others are referred

to as “good” cameras
.
Further analysis on the cameras can be
found in
the Annexure
.
Remaining six cameras are used for all further analysis unless
noted otherwise
.

Single Eye Camera

FRR

Dual Eye Camera

FRR

A

0.5
2
%

E

0.
21
%

B

0.4
3
%

F

0.
31
%

C

0
.
9
1%

G

0.8
5
%

D

0.41%

H

0.2
8
%

Table
6
:

Camera wise Reject Rates

(2 irises with 2 attempts)

4.3.3

Impact of using two irises and multiple attempts

Single eye camera captures one iris at time
.
The 3 “good” single eye cameras achieved a true accept
rate (TAR)
96.21%
in one attempt
.
This means that
96.21%
resident
s

can be authenticated using a
single (left) iris capture
.
When the other (right) iris is added in matching the percentage of resident
successfully authenticating improves to
99.23%
, again in single attempt
.
Capturing both the left and
rig
ht iris a second time (second attempt) improves the TAR to
99.54%
.

There is a significant improvement in the TAR rate, between a single (left) iris capture and both iris
capture
.

Single
Eye

Camera

Single Iris TAR

Both Iris

1st Attempt TAR

Both Iris

2
Attempts TAR

All Cameras (4)

93.77%

98.99%

99.43%

Good Cameras (3)

96.21%

99.23%

99.54%

Table
7
:

Single Eye Camera
-

Good and all single eye camera TAR

Dual eye camera captures both irises at the same time
.
The “3” good dual
iris cameras achieved a
TAR of 99.
62
% in the first attempt
.
A second attempt of the dual iris capture improved the TAR to
99.
73
%.



Accuracy centric statistics

Number of
residents

% of residents

Resident successfully with either their left or right eye (on
at least one camera)

5793

99.45%

Resident did NOT authenticate successfully on any eye(right
or left)

13

0.22%

Resident did NOT authenticate due to enrolment

issues

8

0.14%

FTC (image capture failed on four or more cameras)

19

0.33%

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
23

of
36

Dual
Eye

Camera

Both Iris 1st Attempt TAR

Both Iris 2 Attempts TAR

All Cameras (4)

99.
46
%

99.
62
%

Good Cameras (3)

99.
62
%

99.
73
%

Table
8
:

Dual eye camera
-

Good and all Dual eye camera TAR

The data shows that high accuracy can be obtained using both single
eye

and dual
eye
cameras
.

Performing a second attempt of capturing the iris slight
ly

improves the TAR for both single and dual
iris camera
.
It is estimated that third attempt
may
not further improve accuracy
.
Hence i
t is safe to
specify maximum attempts
at

two in standard operating procedure.

4.3.4

Matcher Performance

The images captured wer
e compared using two different backend
matchers (SDK)
.
The graph below illustrates the session wise (up to 2
attempts of both iris
es
) FRR achieved using the two S
DK
.
Matcher 2
had an FRR of 0.43
% wh
ile matcher 1 had an FRR of 1.11
%
.
Matcher 2
has been u
sed for all other comparisons in this report
.

When falsely rejected images were manually reviewed, it was noticed
that Matcher 2 was able to match off axis (gaze)
, dilated, out of focus
and motion blur
images

better than Matcher 1
.
This was a significant
co
ntributing factor in
superior

performance of Matcher
2
.

4.3.5

Iris DET Curve

Iris authentication has a flat DET curve, i.e
.,

there is only a slight
degradation in FRR for significant reduction in FAR
.
As s
hown

in

the

DET

curve

in
Figure
13
,
the
single

eye

camera

FRR (2 iris, 2 attempts)

increases

from

0.
31
%

for

an

FAR

of

1e
-
3

to

an

FRR

of

0.
46
%

for

an

FAR

of
1e
-
6
.


Figure
13

DET Curve
-

Single and Dual Iris Authentication

As shown

in

the

DET

curve

in
Figure
13

the

dual

iris

camera

FRR (2 iris, 2 attempts)

increases

from

0.
21
%

for

an

FAR

of

1e
-
3

to

an

FRR

of

0.
27
%

for

an

FAR

of
1e
-
6
.

Hence, iris
can be a

particularly attractive option for high secur
ity applications such as financial
transactions

or secures access control
.
Iris authentication can operate at TAR > 99% with FAR
=

1e
-
6.

0.46%

0.34%

0.33%

0.31%

0.27%

0.23%

0.22%

0.21%

0.00%

0.20%

0.40%

0.60%

0.80%

1.00%

1.00E
-
06

1.00E
-
05

1.00E
-
04

1.00E
-
03

FRR

FAR

Iris DET (2 Iris, 2 Attempts)

Single IRIS Camera

Dual IRIS Camera

Figure

12

FRR

by

matchers

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
24

of
36

0.4%

65.4%

31.0%

3.3%

24.0%

60.4%

13.6%

2.0%

0%

10%

20%

30%

40%

50%

60%

70%

< 30 sec

<1 min

<2min

>2mins

Single Eye device

Dual Eye Device

4.4

System findings

Besides accuracy and coverage, the third most important factor for feasibility of iris based
authentic
ation is system level factors
.
These factors consist of



Response time



Image s
ize used in transmission
.



Matcher speed

The response is measured with two metrics: Resident authentication time and round trip time
.
Resident authentication time includes time
to select the device, enter the Aadhaar number, guiding
the resident for correct capture procedure, completing the capture of both eyes, submitting the

transaction and receiving response
.
Round trip time measures the last two parameters of the
authenticat
ion time during which the resident is waiting for system to respond
.

4.4.1

Resident Authentication Time

Figure
14

is a histogram of authentication time
separated by single

and dual eye cameras
.
Median time in both cases is below one minute.

In case of single eye devices, one
iris

is captured
at a time whereas in case of dual eye device, both
iris
es

are captured at a time
.
Therefore, median
time of dual eye camera is lower

than that of
single eye
.

This time distribution also includes the time
taken for repeated impressions due to failed
quality or failure to capture during the first
attempt.

4.4.2

Round

trip
time

The histogram of round trip time under these
conditions is shown
in
Figure
15
.
Median
is

below

ten seconds
.
Over
62% of transactions returned in
less than 10

s
ec
onds, while only
7.8% of
transactions

took more than 15

sec
onds.

The network performance can be further improved
by smaller image sizes as

discussed in the next
section.

4.4.3

Image size

Both sce
nario and technology testing used KIND 2
image compressed to about 15KB using JPEG 2000

lossy
.
Nearly all device vendors were unable to
provide usable compact formats


KIND 3 and KIND 7 (also called CROPPED and CROPPED AND
MASKED)
.
NIST has shown

[Groth
er, P, 2009]

that authentication can be achieved using KIND 7
without any degradation in accuracy.

Figure
15
:

Network Latency histogram

Figure
14
: Resident authentication time

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
25

of
36

The diagram below shows compression used in this PoC

and the roadmap to get the device vendors
to support kind 7 images which will reduce the iris image to 2KB
.
Refer
[
ISO 19794
-
6
-
2011
]

for
further definitions of image types cited above.


Figure
1
6
:

Image Formats

4.4.4

Matcher Speed

Pre
vious PoC demonstrated that current setup is capable of supporting sustained fingerprint
authentication speed of one million per hour
.
It has been demonstrated repeatedly in the literature
that iris matcher is more efficient than fingerprint matcher
.
The
refore no independent tests were
performed using iris matcher
.
Based on published data, it is safe to state

that Aadhaar is capable of
sustaining iri
s authentication
rate of over one million per hour
.
Further tests could verify this
assertion.

4.5

Devices Fi
ndings

During the
PoC
, four cameras were single eye cameras
and four cameras were dual eye cameras
.

The camera
s

varied in form and function
.
Satisfactory performance of six cameras supplied by
different OEMs indicates that cameras will be competitively
available in different form and in desired
quantities.

All devices provided documentation on mode of capture and
were
easy to learn and operate from
operator and resident point of view
.
Similarly 8 of 11 device vendors provided software in
compliance to A
PI specifications published for the purpose of POC
.
One vendor achieved
compliance after POC process was completed and could not therefore be tested.

All the devices used during the test, were supplied with rugged casing
.
No breakages and
malfunctioning
due to packaging were reported during the POC.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
26

of
36

Devices supported auto capture feature
.
This enabled devices to automatically capture good quality
iris pictures when operator held the device at the specified capture distance from the resident iris.

Single
eye devices captured single iris at a time
.
Dual eye devices captured both
irises

at the same
time
.
There was one device which was capable of dual iris capture, but was configured to capture
both
irises
, in a sequence, one iris at a time
.
When presented with residents who had one eye in
good condition, single
eye

devices fared better than dual eye camera which looked for two good
irises
.
Dual iris cameras could improve the capture algorithms in order to overcome this issue.










Figure
17
:

Sample images of residents providing samples during POC

From usage model point of view, capture distance and capture volume of iris devices used varied
widely
.
Capture
distance (distance between edge of device and

eye)
of

less than10
mm to
over
300mm were tested during the POC
.
All but one

device

were expected to be handled by the
operator
and
last
device was mounted using tripod
.
As the capture volume supported by the device
plays important role in ease of capture, devices
with
larger capture distance
and
small capture
volume required
addi
tional
training for both operators and residents.

Similarly, actionable feedback and capture a
ids supported by the devices played key role in ensuring
high quality and easier iris captures
.
Most devices were providing actionable feedback to the
operator and capture was supported through various aids
.
Devices that provided no feedback and
capture
aid
s

took more number of
presentations
to achieve high quality of iris capture
.

VGA images provided by devices were compressed and analyzed during the analysis
.
Iris images
were compressed to 15KB for the analysis
.
However, devices supporting cropped/mas
ked (Kind 7)
images would help reduce the size of the iris further to order of 2KB
.
This reduction will help during
iris online transactions
.
It has been shown by NIST report
[Grother, P, 2009]

that such reduction
does not have accuracy related
implicati
ons
.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
2
7

of
36

The a
nnexure
provides
further information related to device readiness and areas of further
improvements.

4.6

Other findings

4.6.1

Impact of Resident Age


The impact of age on authentication accuracy was studied
.
The computed DET curves compare the
accuracy of
matching for people in the age group 5
-
15, 15
-
60 and 60+ years
.


Figure
18
:

Age wise DET chart

For all age categories, overall accuracy continues to be higher than
98.
94
%

at FAR of 1e
-
6
.
Among,
those children have performed best

(0.19% reject rate, statistically not significant), followed by
adults (
0.
29
%)

and seniors (1.
06
%)
.
Some of the seniors could not be authenticated due to special
eye conditions
.

0.19%

0.14%

0.14%

0.14%

0.29%

0.22%

0.21%

0.20%

1.06%

0.87%

0.84%

0.81%

0.00%

0.20%

0.40%

0.60%

0.80%

1.00%

1.20%

1.00E
-
06

1.00E
-
05

1.00E
-
04

1.00E
-
03

FRR

FAR

Age wise DET Curves (2 Iris, 2 Attempts)

< 15

15
-
60

>60

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
28

of
36

5


Observations &

Findings


In conclusion, a set of observations
and findings
d
erived from the analysis of the PoC
data

is
presented

below
.

5.1

Coverage

It is possible to achieve over 99.5% population coverage for on
-
line
iris
authentication
.
Most of
those
who
se

iris images could not be captured

on
four

or more cameras
suffered from
special

eye
condition, mostly due to older generation of cataract or eye surgery that resulted in highly irregular
shaped pupil or iris
.
With current devices and matcher algorithm, they cannot avail themselves to
iris authentication
.
Altern
ate method suc
h as fingerprint

or

OTP will be required
.
They could be
easily identified
using enrollment data or during first attempt to authenticate.

Fingerprints of these individuals with
special

eye conditions were reviewed
.
It was
found through
best finger detecti
on (
BFD
)

and authentication

that their fingerprints
were
of sufficient quality to
enable fingerprint based biometric authentication
.

5.2

Accuracy

It is possible to attain true
accept

rate of over 99% on
both single and

dual eye cameras with
stringent FAR of

1e
-
6
.

Authentication Mode

Single eye camera

Dual eye camera

One eye authentication

96.
21
%

Not Applicable

Two eye authentication

99.5
4
%

99.
73
%

Table
9
:


True

Accept Rate at FAR =

1
e
-
6.

Operating at false accept rate of one in million,
Iris technology also lends itself for highly secure
authentication
against imposter
attacks
.
Table
9

shows
how high accuracy authentication may be
achieved using various strategies.

Use of
second

iris for authentication improves accuracy by 3%

over single iris
.
However, second
attempt
of the same iris
only marginally improves accuracy
.
Due to
diminishing return, it is unlikely
that third attempt would increase accuracy any further.

For all
age
categories,
overall accuracy continues to be higher than
98.
94
%
.
Among those children
have performed best, followed by adults
.
Some of the seniors coul
d not be authenticated due to
special eye conditions
.
This indicates the need to tune the capture and matcher algorithms
specifically for seniors to accommodate for special eye conditions
.

A smaller percentage of people (0.
14
%, not statistically
valid
) co
uld not be authenticated because of
enrollment data quality issues
.
They would be able to use iris authentication
once their biometric
data is updated.

Anecdotally
, blindness by itself was not perceived to be an issue and blind people without
special

eye
condition
s

authenticated successfully.

5.3

Device Readiness

There is a variation in authentication accuracy cameras based on capture algorithms
,

ergonomics
and related factors
.
In the study, it was observed that while most cameras performed well, there
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
29

of
36

were
outliers with high FTC and FRR rates
.
Operator’s difficulty in using them correlated with their
poor FTC and FRR rates
.
For example, one camera
had

FTC rate of
about
10% while most others
were operating below 1%
.
It is critical that
only
good
cameras

are selected for deployment
in the
field through
a rigorous testing and
certification
process
.

Single eye camera was judged better with people with squint eye
.
Residents found dual eye camera
easier to use resulting in slightly faster capture time
.
Dual
eye camera that captures one eye at a
time seems to offer best of both

the

worlds.

Devices providing capture aids and actionable feedback help operator with faster and better quality
capture
.
The devices which provided effective capture aids and feedback
to operator regarding
capture process and image quality was seen to achieve very high degree of accuracy.

It is important to emphasize that
the
quality of image plays
an
important role
in
ensur
ing

authentication accuracy
.
Devices
have
incorporated quality

measurements within their capture
algorithms
to

enable auto

capture
.
In
addition to the existing parameters,

to further improve the
quality of capture

and minimize number of attempts
, algorithms could check for gaze correctness,
rotated device, upside do
wn eyes, occlusion,

focus, motion blur

and special eye conditions
.

Vendor supplied software provided iris image in KIND 1 and KIND 2 formats
.
Resulting compressed
image was 15 KB in size
.
KIND 3 and KIND 7 could bring this size down to 2 KB which would b
e
beneficial in reducing network latency and improving network reliability.

5.4

System Readiness

The median time to perform end
-
to
-
end online authentication is less than one minute out of which
about a quarter is for round trip transmission (& back end process
ing which is sub second)
.
From
resident’s view, the wait time after capture has occurred is less than 15 seconds which was judged
acceptable.

Faster network or smaller image size will reduce resident wait time
.
Device improvement
recommendation mentioned

in this report will also improve transaction time.

5.5

Findings

1.

One eye vs
.
two eyes:
Two
irises

authentication provide
s

significant improvement in accuracy
and coverage over one
iris.
In case of single eye camera, it is
recommended
to authenticate first
wit
h one eye
, and only in
case of failure,
the
second eye be used

for authentication
.

2.

Attempts: Two attempts are recommended
.
While the second attempt only marginally
improved accuracy, it compensates for other forms of human error should they occur
.

3.

Device
Certification: Stringent authentication device testing and certification is recommended to
ensure high quality authentication devices are deployed for Aadhaar authentication.

4.

Capture aids:
Device
vendors
should provide better capture aid for operator and r
esidents
.
The
study categories three types of aid: actionable feedback, visual aid and
“appropriate light source
to improve quality of image and finally measures to block ambient light reflections
.
These
improvements detailed in Annexure will reduce capt
ure time, improve accuracy and offer better
user experience.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
30

of
36

5.

Capture quality
:

It is important to emphasize that quality of image plays important role to ensure
authentication accuracy
.
Devices should make attempt to incorporate specific areas of quality
e
xplained in the document along with what is being already being done within the devices to
ensure quality.

6.

KI
ND 7 image formats
: Device vendors
need to
implement KIND 7 format

which is more efficient
than the current standard used
.

7.

Device

Enhancement: The
study has clearly identified
a
small number of residents (< 0.3%) with
special

eye conditions
.
Vendors
would

be provided with the images
(suitably
anonymized
)

and

UIDAI would work with them

to improve device capture and iris matching algorithms to handle
such cases.

8.

Matcher SDKs: Since there was a variance in the performance of two matcher SDKs tested, the
matcher SDK vendors need to enhance their algorithms to provide
higher tolerance to problems
arising out of incorrect gaze,
dilation,
focus
,

motion blur

etc
.

9.

Matcher Setting:
Since FRR does not materially change when FAR is changed from 1e
-
4 to 1e
-
6,
iris lends itself well to be operated at much lower FAR thus providing robust security against
imposter attacks.

5.6

Way Forward


UIDAI would be working with the

biometric ecosystem to implement the findings mentioned in this
document and take up further field studies to fine
-
tune iris authentication technology. These studies
would also lead to formulation of iris device specifications which will be used for certi
fication by
STQC.


Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
31

of
36

6

References

[Grother, P, 2009]:
IREX Summary
.

By
P. Grother,
E. Tabassi, G. W.
Quinn, W. Salamon
:
NIST
.

[ISO 19795
-
2, 2007]:
Biometric performance testing and reporting

Testing methodologies for technology and
scenario evaluation
. By
International Standards Organization.

[ISO 19794
-
6, 2011]:

Information technology
--

Biometric data interchange formats
--

Part 6: Iris image data
.
By International Standards Organization.

[UIDAI, FP, 2012]:
Role of Biometric Technology in Aadhaar Authent
iation
-

Detailed Report

http://uidai.gov.in/images/role_of_biometric_technology_in_aadhaar_authentication_0
20412.pdf


Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
32

of
36

7

Appendix


Camera related
findings

This section is concerned with devices used in the field exercise
.
The section goes into detailed
findings and observations including future areas of improvements to improve capture efficiency and
ergonomic improvements.

7.1

Capture distance and Capt
ure Volumes

During the proof of concept, four cameras were single eye cameras
and four cameras were dual eye
cameras
.
Most of the single cameras had capture distance of less than 3cm
.
One dual iris camera
had a capture distance of about 33 centimeters
.
Camera distance and volume of cameras used in
the POC is listed below.

Camera

Single or Dual Iris

Capture Distance

Capture Volume

( WxHxD)

Camera 1

Dual

22.5 cm to 25.0 cm

15 cm x 11 cm x 4 cm depth
.

Camera 2

Dual

0 to 10 mm from forehead

42 x
24 x 30 mm for each eye

Camera 3

Single

120mm

39x25x20mm

Camera 4

Dual

315 to 345 mm

60 x 45 x 30 mm for each eye

Camera 5

Single

50mm(from sensor)
-

0
-
10mm
from the eye guard

33x24.5x8mm(D)

Camera 6

Dual

>58mm

20x15x12mm

Camera 7

Single

>58mm

20X15X12mm

Camera 8

Single

0 to 10 mm from the eye
guard

32 x 24 x 10 mm

Table
10

Camera Capture distance and capture volume information

Note: Cameras have been listed above in alphabetical order and are not to be correlated with

other
camera listing tables.

7.2

Image
Capture and O
utput


All participating camera vendors were requested IRIS segmentation SDK which produces image in
format Type 1, 2, 3 and 7 (also known as
U
ncropped, VGA, Cropped and Cropped
&
Masked
respectively) compli
ant to ISO 19794
-
6 standard
.
It was observed the camera vendors support for
Uncropped and VGA formats (Kind1 and 2 respectively as per ISO 19794
-
6 standard)

(ISO, 2011)

was
complete and support for Kind3 (cropped) and kind7 (masked) was not complete
.

Support for Kind7 (Cropped

&

Masked
) is key for iris authentication to be viable in the field due to
size of the image
.
Crop/masked images allow for close to 2kb images whi
ch tends to be relatively
easier on transmission during online authentication.

All devices supported auto capture
.
Forced capture was not implemented across devices uniformly
and hence was not used
.

Auto capture algorithms on few cameras failed to capture

in instances where special eye conditions
were presented (see section 8.4)
.
It is very important for the auto capture algorithms be tuned to
capture resident iris with special eye conditions in order to improve the coverage of iris
authentication
.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
33

of
36

Camera
s that seek to
block

ambient light provide visible light source inside the camera
.
Visible light
source serves two purposes
.
It prevents dilation of pupil that would otherwise occur in a dark
environment
.
It helps resident to maintain proper gaze
.
Resi
dents complained about some light
sources to be too bright to cause stress
.
When light source was not present,
the residents had to be
instructed to look straigh
t
.
Providing resident focus points such as a mirror so resident can see
his/her own eyes insi
de will also help retain gaze.

Two single cameras supported eye flaps to help position the device close to eyes softly
.
These flaps
could rotate 180 degrees to capture either of the eyes
.
Although this feature was innovative and
helped operator and resid
ent to capture easily, room for error due to
non
-
rotation

of flaps for each

capture

may cause bad quality image
s

as flaps tend to obstruct movement of device close to eyes
and nose, when not rotated
.

All the devices were connected to the PC through USB int
erface
.
Few cameras required independent
power source while most drew power through USB
.
This makes certain cameras useful for mobile
field requirements with handheld usage while others which require continuous power suitable for
desk usage with continuo
us power availability.

7.3

Capture aids

7 Devices out of 8 implemented one or many capture aids during the capture process to provide
support and actionable feedback to operator
.
Actionable Feedback provided to operator falls in one
or more of the following
categories.

1.

LCD display for operator to visually see the camera output
.
By moving device closer/farther
the operator was able to get improve the focus and quality
.

2.

Using small lights behind the camera to indicate capture start and end process
.
This helpe
d
operator to position the device at the right distance for the capture process to begin and
also know when the process completed
.

3.

Changing color of the light to indicate when the resident is too far or too close or at the right
distance
.
For example, lig
ht color would change to blue when the resident moved too close
or and turned to red when the resident moved too far and turn green when the resident is in
the right range.

4.

Devices using
visible light source

also provided feedback in that when the source was turned
on, by watching the eye socket being lit, operator could infer that capture process having
started and similarly when it was turned off, infer that capture process was completed
.
However, if the r
oom ambient light
was
high, then operator
might

not be able to notice the
light source from the camera
.
Excess light source was seen to result in constriction (reduced
pupil) affecting the ability of the matchers to segment iris.

5.

Provide operator feedback

related to capture process on the PC
.
Messages such as resident
being far, close and at the right distance etc
.
were provided to operator
.
However it
required that the operator watch both PC and camera at the same time
.
Operator feedback
should be inc
orporated in the camera.

6.

Alignment aids in terms of view finder for operator to align the camera to the resident eyes
.
This helped operator to align the device to the eye level.


It was observed that providing capture aids and actionable feedback to the o
perator
improves the capture process and enhances the quality of capture
.
Actionable feedback
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
34

of
36

becomes essential when the capture distance is high and capture volume is not big enough
for quick and high quality capture
.
Lack of capture aids/effective oper
ator feedback was
seen to make the capture process harder.


There is scope for providing further capture aids and more actionable feedback for the
operator
.
These areas include

1.

Feedback related to quality of the image
-

devices

could provide image quality

feedback
to the operator to enable him to take another impression
.
Quality areas such as focus,
gaze, occlusion, presence of reflected objects covering iris etc
.
could be highlighted
during the capture process by device (as actionable feedback)
.
This w
ill help operator to
capture high quality images.

2.

Iris rotation


Device vendors could alert the operators when device is not held straight
.
Images are likely to rotate by certain angle when the devices are not held straight which
may result in rejects du
ring the matching process
.
Feedback to operator in this regard is
likely to help the operator to hold the device straight
.
Similarly, devices could provide
feedback when the device is held upside down.

3.

Cameras with mechanical aids
-

Alignment aid



Presence of alignment aids helps operator to position the device for capture.



Alignment aid
helps
to fix the camera on
capture area to

help position the camera
correctly
.
Alignment aid will also help in placing the device at the right capture distance.



In

the absence of alignment aids, display of live stream from camera will help operator
to align the device.



Alignment aid design should take safety of eyes into account



Alignment aid / Enclosure
help

reduce ambient reflections
.
This may help accuracy
.
Cameras with Enclosure help reduce ambient reflections
.
T
his will
also
help when the
capture is taking place in open areas
or
areas
with

excess ambient lighting conditions
.

In summary,
Cameras with visual indications for capture process, eye alignment and

capture
completion were easiest to use
.
Cameras that offered backlit LCD display to the operator helped
operator to capture very quickly
.
Conversely
,

cameras that offered no feedback to operator were
relatively more difficult to use
.
Dual eye cameras w
ould benefit from alignment visual indication for
operator
.
Capture aids help operator and resident during the capture process
.

7.4

Detection and Capture of iris

It was seen during the POC

that certain cameras were able to detect and capture resident iris mo
re
effectively than others
.
It was observed that certain special eye conditions where resident had
undergone possibly surgery (for cataract in most cases) due to which the pupil shape has changed
had resulted in capture failures in certain authentication
cameras whereas certain other
authentication cameras were able to capture the same
.
Some of the resident’s iris images which
were successfully captured on few cameras but failed to capture on others are shown below.

Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
35

of
36

Improvements to detection and capture a
lgorithms after taking note of the nature of iris images
below is expected to improve the capture efficiency of failing devices
.


Figure
19

Resident with special condition Eyes

7.5

Dual eye
vs
.
single eye cameras

During POC, of 8 cameras deployed, 4 cameras were dual iris and 4 were single
eye

cameras
.
Of the
four dual iris cameras deployed, one
camera implemented
d
ual simultaneous capture, with left/right
single
-
eye c
apture in succession
.


Dual iris cameras were
seen to complete the dual eye capture overall faster than single eye cameras
where the capture for each iris had to be initiated one eye at a time
.
Since single eye cameras had
to capture one eye at a time, and dual eye cameras capture both eyes together
.

Consequently dual
eye capture time is less for dual eye cameras.


However when capturing iris of residents with special eyes, it is possible that one of the iris is good
and can be captured with high quality
.
Dual iris devices which look for pair of iris in good conditions
Iris
Authentication Accuracy
-

P
o
C
Report

© UIDAI, 2009
-
2012


Page
36

of
36

are likely to fail in these situations
.
Single eye devices have an advantage in these situations where
resident’s only eye can be captured due to special eye conditions
.
Dual iris cameras c
an improve
capture algorithms
to capture resident iris when presented with resident with special eye
conditions.


In Summary, both single eye and dual eye devices were found suitable for iris authentication.

7.6

Capture Distance

Capture distance refers to
distance at which the camera should be held in order to enable the
capture of iris
.
As noted in the table above, in the field trial, cameras had capture distance ranging
from 10mm to over 300mm.

Higher capture distance in general has convenience
and hygie
ne
value as device need not be taken
very close to eyes
during the capture
.
As capture distance increase, capture depth has to be
correspondingly increased to ensure ease of capture
.