Pattern Recognition and Image Analysis Group (RFAI) Document (Image) Analysis related work

bijoufriesAI and Robotics

Oct 19, 2013 (3 years and 11 months ago)

183 views

Pattern Recognition and Image Analysis Group (RFAI)

Document (Image) Analysis related work


Laboratory of Computer Science (LI)

François Rabelais University

Tours city, France

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition

6.7. Graph matching and embedding


2

Tours city

3

-

137

046 people (2009)

-

204 km southwest of Paris

-

Region «


centre
-

Indre

et
loire

»

-

1h20 from
Paris by high speed train

-

Direct train connection to Charles de
Gaulle
-
Orly

airport in 2h00

Paris

Tours

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition

6.7. Graph matching and embedding


4

François
-
Rabelais University,

les
deux

lions /
Portalis

5

François Rabelais

i.e. a famous French
writer of XV
°

Century

François Rabelais University

Faculties

Art & human

sciences,

economy, business &

manage
-
ment
, health, information and
technology

Students

21 207 (2 500 foreign students)

Teachers

1 300

Support staff

1000

Laboratories

40

Place


5

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition

6.7. Graph matching and embedding


6

School of Engineering
Polytech

7

-

12
schools in France
(Grenoble, Lille, Marseille,
Montpellier, Nantes, Nice
-
Sophia, Paris
-
UPMC, Paris
ORSAY, Savoie, Orléans,
Tours, Clermont
-
Ferrand)


-

12
000
students

Urban Planning

-
720 students

-

5
departments (with Labs)

Mechanics

Electronics

Computer Science

LI

LMP

CITERES

LMR

Embedded computing

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


8

Pattern Recognition
and Image Analysis

Data Bases and Natural
Language Processing

Scheduling and Control

Handicap and New Technologies

Visual Data Mining and
Biomimetic Algorithms

77 people, 5 research groups (2009)

Laboratory of Computer Science

9

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


10

Pattern Recognition and Image Analysis (RFAI) (1)

Medical Imaging




-

Image segmentation (ultrasound, MRI)


-

Video analysis, 3D reconstruction



Document Image Analysis




-

Layout analysis & document recognition


-

OCR, word spotting & signature verification


-

Symbol recognition & spotting


-

Content based Image Retrieval


-

Camera based Recognition


-

Graph matching and embedding



Machine learning for time series prediction

11

Hubert

Cardot

Jean
-
Yves

Ramel

Romuald

Boné

Nicolas

Ragot

Thierry

Brouard

Mathieu

Delalandre

Pascal

Makris

Gilles

Verley

Julien


Olivier

Sabine

Barrat

Professors

PhD

Partha

Roy

Pattern Recognition and Image Analysis (RFAI) (2)

Romain


Raveaux

Nicolas

Sidere

Muzzamil


Luqman

Alireza

Alaei

12

Pattern Recognition and Image Analysis (RFAI) (3)

Aymen

Cherif

Fareed

Ahmed

Cyrille

Faucheux

PhD
Students & engineers

The
Anh


Pham

Ahmed Ben
Salah

Anh Khoi

Ngo ho

Frédéric


Rayar

13

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


14

Projects

& partners outline (1)

15

International projects

National

projects

Local government
projects

Scholarships

Partnership

contracts

Madonne

2003
-
2006



Navidomass

2006
-
2009



EPEIRES

2004
-
2007



BVH

2004
-
today



ATOS

2005
-
today



PIVOAN

2008
-
2009



HEC

2005
-
2011



SNECMA

2008
-
2011



AAP

2010
-
2011



VIED

2010
-
2013



Bnf

2010
-
2013



Digidoc

2011
-
2014



Google

2011
-
2012



ISRC2011

2011
-
2012



DOD

2011
-
2015



IndoFrench

2012
-
2015



SPD

2012
-
2015



0
4
8
12
16
20
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
DIA people plot

Projects & partners outline (2)

16

People

Institutes

Length

Funding

ACI MADONNE

2003
-
2006

55

8

2 years

110 k


ANR
Navidomass

2006
-
2009

40

7

3 years

443 k


Technovision

ÉPEIRES

2004
-
2007

30

7

3 years

100 k


ANR Digidoc

2011
-
2014

18

7

3 years

866 k


National projects

Centre de Recherche en

Informatique de Paris 5

(Paris)

Institut de Recherche en

Informatique et Systèmes

Aléatoires (Rennes)

Laboratoire Informatique

(Tours)

Laboratoire d'InfoRmatique

en Image et Systèmes

d'information (Lyon)

Laboratoire Lorrain de

Recherche en Informatique

et ses Applications (Nancy)

Laboratoire d'Informatique

de Traitement

de l'Information (Rouen)

Laboratoire d’informatique


image et interaction

(La Rochelle)

Centre d’Etude Supérieures de la
Renaissance (Tours)

Laboratoire Bordelais de

Recherche en Informatique

(Bordeaux)

Projects & partners outline (3)

17

PhD Scholarships

R.J.
Qureshi


Higher Education Commission
(HEC) of

Pakistan

M. Luqman

T.H.

Pham

Vietnam International Education
Development (VIED)

Local government projects (i.e.
projets

region centre)

People

Institutes

Length

Funding

PIVOAN

3

1

1 year

33 k


AAP

3

1

1 year

38 k


Partnership contracts

Centre des études supérieures de
la renaissance


bibliothèque
virtuelle humaniste

international high
-
technology
group in aerospace, defense and
security

Bibliothèque Nationale de
France
-

portail
Gallica

Digitalisation

company

capturing, automatically
processing, and managing all
company’s incoming
documents

Atos

Origin is a leading
international IT services
provider for business solutions

So famous !

Maison des Sciences de l'Homme

Bilateral program

People

Institutes

Length

Funding

IndoFrench

3

2

3 year

70 k


Projects & partners outline (4)

18

Computer Vision Center
Document Analysis Group

Barcelona
-

Spain

“J.
Llados
, E.
Valveny


Indian Statistical Institute

Kolkata
-

India

“U. Pal”

Dept. of Computer Science and IS

Osaka Prefecture University

Osaka
-

Japan

“K.
Kise


Computational Intelligence Laboratory

Athens
-

Greece

“B. Gatos”

Projects &

partners outline (5)

19

Layout analysis &
document

recognition

OCR,
word spotting &
signature verification

Symbol

recognition

&
spotting

Graph matching &
embedding

CBIR

Camera based

Recognition

Madonne

2003
-
2006



Navidomass

2006
-
2009





EPEIRES

2004
-
2007



BVH

2005
-
today





ATOS

2005
-
2009



PIVOAN

2008
-
2009





HEC

2005
-
2011





SNECMA

2008
-
2011



AAP

2010
-
2011



VIED

2010
-
2013





Bnf

2010
-
2013



Digidoc

2011
-
2014



Google

2011
-
2012



ISRC2011

2011
-
2012



DOD

2011
-
2015





IndoFrench

2012
-
2015



SPD

2012
-
2015



Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


20

Layout analysis & document recognition

“AGORA (1)”

(1) Text/graphics separation

Foreground map: adaptive
binarization

[Saul2000] with connected component
labeling, text/graphics separation is done
in terms of size of connected components

1. Background map: statistical
distribution of white and black pixel
on horizontal and vertical
scanline

2. Fusion: word segmentation (i.e.
connected components grouping) is
done in terms of thresholding on the
background map.

People

Jean
-
Yves
Ramel

Funding

CESR partnership,

Madonne
,

PIVOAN

Starting

2005

Ref

J.Y.
Ramel

and al. User
-
driven Page
Layout Analysis of historical printed
Books. IJDAR, 2007.

(2) Line/word segmentation

Layout analysis & document recognition

“AGORA (2)”

(3) Interactive system (i.e. user driven analysis)

Vertical position

average

= 0,46

std

deviation

= 0,41

Horizontal position

average

= 0,51

std

deviation

= 0,07

People

Jean
-
Yves
Ramel

Funding

CESR partnership,

Madonne
,

PIVOAN

Starting

2005

Ref

J.Y.
Ramel

and al. User
-
driven Page
Layout Analysis of historical printed
Books. IJDAR, 2007.

(4) Results, since 2005: 300 books (50 000 pages)

http://www.bvh.univ
-
tours.fr/


22

Layout analysis & document recognition

“Document image characterization (1)”

23

Directional

rose

(1)

main direction

(2)

isotropy

(3)

standard deviation

Spatial

(4)

ink/paper transition

(5)

white spaces separating
the collateral

elements

(1) Descriptor based on five features:

People

Nicholas

Journet

Funding

CESR partnership &
Madonne

project

Starting

2006

Ref

N.
Journet

and al. Document Image
Characterization Using a
Multiresolution

Analysis of the
Texture: Application to Old
Documents. IJDAR, 2008.

Layout analysis & document recognition

“Document image characterization (2)”

24

(2) Segmentation:

-
features are extracted at four different resolution (4

5 = 20 features)

-

features are then processed with the clustering algorithm CLARA
(Clustering
LARge

Applications) [Kaufman1990] to achieve automatic
segmentation in text/graphics/background

People

Nicholas

Journet

Funding

CESR partnership &
Madonne

project

Starting

2006

Ref

N.
Journet

and al. Document Image
Characterization Using a
Multiresolution

Analysis of the
Texture: Application to Old
Documents. IJDAR, 2008.

Layout analysis & document recognition

“Document image characterization (3)”

25

(3) Indexing applied on two different problems

-
Layout retrieval, distance is based on a contingency table [Younes2004]

-
Graphics retrieval, distance based on a dissimilarity function

People

Nicholas

Journet

Funding

CESR partnership &
Madonne

project

Starting

2006

Ref

N.
Journet

and al. Document Image
Characterization Using a
Multiresolution

Analysis of the
Texture: Application to Old
Documents. IJDAR, 2008.

Handmade dataset I (400 images)

Handmade dataset II (400 images)

Layout analysis & document recognition

“Cognitive digitalization”

26

Topic:

Incremental and interactive learning for document image, application for
intelligent cognitive scanning of old documents.


Problematic:

-

Estimate

the scan parameters

according to

usage,

past experience.

-

Improve

the scan parameters for a

document during the scanning.

-

Detect the default settings for a document, a collection, a work.

People

A.K. Ngo ho, N.
Ragot
,
J.Y.
Ramel


Funding

Digidoc

project

Starting

10/2011

Ref

Na

Layout analysis & document recognition

“Document classification”

27

Form

Publicity

Free letter

Acknowledge reply

Topic:
Recognition of administrative
forms for companies


Problematic:

-

high variability “600 to 800 classes”

-

binary images at 300 dpi

-

time constraint:


to 1,5 s per image

-

commercial systems can’t outperform


a 60% recognition rate


Goals:


1. To gain in robustness (set of adapted


and robust specialists)


2. To gain in flexibility (self


learning, content adaptation)

People

Mathieu Delalandre

Funding

DOD project

Starting

12/2011

Ref

Na

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


28

OCR, word spotting and signature verification


“Robust OCR
-
I (1)”

29

People

Kamel

Ait
-
Mohand

Funding

Navidomass

project

Starting

2006

Ref

K.

Ait
-
Mohand

and al.

Structure
Adaptation of HMM
Applied

to OCR.
ICPR, 2010.

Key idea : improving OCR robustness by using similar technics as those used
for handwriting recognition:


-

Hidden Markov Models without explicit segmentation


-

Adapting a polyfont OCR to specificities of pages (fonts/noise)

Sliding window

(1) Feature extraction is based on a sliding window and HoG features (no
word/character segmentation)

(2) HMM classification and training

-
HMM characters models are learnt on a synthetic dataset (numerous
fonts, degradations possible, no limits in the number of samples per
character) = >
polyfont

OCR system

-
Each character model can be adapted to a specific font/book using only
few lines of transcriptions. The HMM model is adapted at the structure
level (number of states) and at the parameter level (Gaussian MAP
adaptation).

initial model

training

Structure
adaptation

New model

OCR, word spotting and signature verification


“Robust OCR
-
I (2)”

30

People

Kamel

Ait
-
Mohand

Funding

Navidomass

project

Starting

2006

Ref

K.

Ait
-
Mohand

and al.

Structure
Adaptation of HMM
Applied

to OCR.
ICPR, 2010.

Experiments done using 100 fonts with the degradation model of Baird

Training

Testing

Degradation models

Baird

Font size

12

Resolution

300 dpi

Fonts

70

30

Image

sets (lines)

10000

15000

blurred

thresholding

sparse pixels

Commerc
ial OCR

Polyfont

Adapted

Average (30 fonts)

88.72%

91.70%

97.37%

69.21%

98.33%

98.68%

44.78%

85.46%

98.09%

52.73%

39.73%

74.89%

OCR, word spotting and signature verification


“Robust OCR
-
II”

31

Topic:
digitalization and indexing of a military document database for retired pay


Problematic:

-

large amount of data (800 000 applications every 3 years)

-

large heterogeneity:

from XIX
°

century “middle” to today,

handwritten and typographic documents,

different languages,

no common layout,

different colors,

etc.

People

J.Y. Ramel, N.
Ragot
,

S.
Barrat

Funding

SDP

Starting

10/20012

Ref

Na

OCR, word spotting and signature verification



Performances prediction/control of OCR


32

Problematic: control and cost reduction of the digitization service to know which
collection/document/part of document is OCRisable and at which quality



Select only adequate documents to be sent to the private service provider in
charge of the digitization and OCRisation


Studies of relationships between meta
-
data information (date, format, …)
and OCR results => difficult without deep analysis of the pages


Characterization of image content with SIFT+LBP; regression towards
OCR results













Control of OCR quality assessed by the service provider


Detection of text zones forgotten by OCR using correct detection performed
(contextual information) => in progress


Verification of OCR result by matching with image (=> in a near futur)

People

Ahmed Ben Salah, K.
Ait
-
Mohand

Funding

BNF

Starting

10/2010

Ref

Na


Real OCR
Result




健牦潭慮捥

數灥捴敤



0
-
㜰7


-
㠰8


-
㠵8


-
㤰


-
㄰〥

Good (≥ 88%)

0%

0%

6,1%

23,11%

77,12%

Undecidable

(82%
-
88%)

5,5%

16%

53,43%

59,11%

20,34%

Bad (<82%)

94,55%

84%

40,47%

17,78%

2,54%

OCR, word spotting and signature verification

“Semi
-
automatic transcription (1)”

33

(1) Segmentation process based on
Agora



Standardized output (
e.g.

Alto)





(2) Clustering process


Finer description of shapes


Features extraction and selection

People

Jean
-
Yves Ramel, Frédéric
Rayar

Funding

CESR partnership, PIVOAN, Google

Starting

2008

Ref

S.
Hocquet

and al. Analyse de classes
de formes pour la transcription de
textes imprimés anciens. CIFED,

2010

(3) Transcription & Typography studies


Topic:
user driven transcription of character in historical books

OCR, word spotting and signature verification

“Semi
-
automatic transcription (2)”

34

People

Jean
-
Yves Ramel, Frédéric
Rayar

Funding

CESR partnership, PIVOAN, Google

Starting

2008

Ref

S.
Hocquet

and al. Analyse de classes
de formes pour la transcription de
textes imprimés anciens. CIFED,

2010

Pages

connected
components

Custers

(i.e. Classes)

150

1

062 081

40 000

10%

90%

> 10 occurrences

< 10 occurrences

93% of the text

7% of the text

0.5% (top 200)

89.5%

85 % of text

8% of text

Experiments, The
Vésales

book

Reasons is noise

spot

character
on verso

touching
characters

split
character

OCR, word spotting and signature verification

“Word Spotting (1)”

35

Manuscripts

AGORA

Text Extraction

Transcription

by

codebook

Codebook of Primitives

Query word

Sequence of primitives

PrimitiveString

Matching

Word Detection

Topic:

Word Retrieval in Historical Documents

People

P.P. Roy, J.Y.
Ramel
, F.
Rayar

Funding

AAP ,
Renon

Starting

10/2010

Ref

P. P. Roy

and al
. An efficient coarse
-
to
-
fine
indexing technique for fast text retrieval in
historical documents", DAS 2012
.

OCR, word spotting and signature verification

“Word Spotting (2)”

36

Topic:

Word Retrieval in Historical Documents

People

P.P. Roy, J.Y.
Ramel
,

F.
Rayar

Funding

AAP ,
Renon

Starting

10/2010

Ref

P. P. Roy,
F.Rayar

and
J.Y.Ramel
. An efficient
coarse
-
to
-
fine indexing
technique for fast text
retrieval in historical
documents", DAS 2012
.

The codebook is created using a clustering
algorithm by template matching of
similarity

Overcoming of segmentation problems are
solved by the Water reservoir method.

Query word is thus converted into a string
of primitives. Approximate string
matching algorithm is used for string
matching

Tests done from

-

24 pages

-
corresponding to 57324 primitives

-
clustered in 183 representative primitives

-
P/R computed with 20 query word images

OCR, word spotting and signature verification



Multlingual

Word Spotting


37

People

N.
Ragot
, J. Y.
Ramel
, U.
Pal

Funding

IFCPAR

Starting

04/2012

Ref

Na

Topic
:

Robust

multilingual

word

spotting
:



Problematic
:

-
Query

by
text
/image

-
Partial
matching

allowed

(for occlusion,
special

characters
)

-
Matching

in
two

steps

: global (
shape

context
) / local (HMM)

OCR, word spotting and signature verification


“Online signature verification (1)”

38

People

Nicolas
Ragot

Funding

ATOS project

Starting

2005

Ref

N. Ragot and al.
Study of
Temporal Variability in
On
-
Line Signature
Verification. ICPR, 2008

Problematic: to evaluate impact of temporality (i.e. time evolution) on
signature, for performance evaluation of signature verification algorithms.

(1) Database acquisition

signers

18

sessions

12

signatures/

session

10

mean time

interval

2 weeks

total
signatures

2160

total
duration

25
months

Training

Enrollment

“5 signatures”

Final
acquisition

“5 signatures”

loop if rough
differences (based on
length, duration,
speed)

OCR, word spotting and signature verification


“Online signature verification (2)”

39

People

Nicolas
Ragot

Funding

ATOS project

Starting

2005

Ref

N. Ragot and al.
Study of
Temporal Variability in
On
-
Line Signature
Verification. ICPR, 2008

(2) Statistical analysis

variation

correlation

Speed

variation

yes

Length

variation

no

Duration

stable

Total length per signer/ session

(2.1) Global

i.e. without temporal variability


(2.1) With temporal variability

(3) Performance evaluation

Authentication (i.e. recognition) algorithm

based on a Coarse to fine approach


-

Coarse step on “basic” features


(length, duration)


-

Fine step based on DTW

Total duration per
signer/ session

Dataset without
temporality

Proposed dataset

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


40

Symbol recognition & spotting

“Vectorization and
GbR

(1)”

(1) Contour detection, chaining and
polygonalisation

[Wall1984]





(2) Quadrilateral building

(2.1.) Matching

(2.2.) Sorting

(2.3.) Merging

People

Jean
-
Yves
Ramel

Funding

Na

Starting

Ref

J.Y.
Ramel
. A Structural
Representation for
Understanding Line
-
Drawing
Images. IJDAR, 2000.

41

Symbol recognition & spotting

“Vectorization and
GbR

(2)”

(3) Graph based representation

(3) Pros and cons

Cons
-

lost of connectivity

Cons
-

parasite quadrilaterals

Pro
-

better representation of filled

& crossed areas

People

Jean
-
Yves
Ramel

Funding

Na

Starting

Ref

J.Y.
Ramel
. A Structural
Representation for
Understanding Line
-
Drawing
Images. IJDAR, 2000.

Symbol recognition & spotting

“Generation of synthetic documents (1)”

People

Mathieu Delalandre

Funding

Na

Starting

Ref

M. Delalandre

and
. Generation
of Synthetic Documents for
Performance Evaluation of
Symbol Recognition & Spotting
Systems. IJDAR, 2010.

Key idea

Graphical documents are
composed of two layers

To use a same
background layer with
different symbol layers

(1) Constraint model

c
2

c
1

M
1

M
2

M
3

M
4

C
1

C
2

C
3

C
4

L
1

θ
1

p
1

L
2

θ
2

p
2

p

L



bounding box and
control point

alignment

symbol
model

loaded symbol

43

Symbol recognition & spotting

“Generation of synthetic documents (2)”

People

Mathieu Delalandre

Funding

Na

Starting

Ref

M. Delalandre

and
. Generation
of Synthetic Documents for
Performance Evaluation of
Symbol Recognition & Spotting
Systems. IJDAR, 2010.

(2) Building engine and user interaction

Symbol

Models

Building

Engine

(2) run

(3) display

(1) edit

Background
Image

44

Symbol recognition & spotting

“Generation of synthetic documents (3)”

People

Mathieu Delalandre

Funding

Na

Starting

Ref

M. Delalandre

and
. Generation
of Synthetic Documents for
Performance Evaluation of
Symbol Recognition & Spotting
Systems. IJDAR, 2010.

(3) Datasets

(4) Performance evaluation


-

Goal is to evaluate variability impact of


produced datasets on spotting system(s)


-

Experiments have been done from


the spotting system of R.
Qureshi

Background sets

Mean localization results

Symbol recognition & spotting

“Graph scoring for symbol spotting (1)”

People

Rashid
Qureshi

Funding

HEC scholarship

Starting

2005

Ref

R.
Qureshi

and al
. Spotting
Symbols in Line Drawing Images
Using Graph Representations.
GREC, 2008.

(1) Graph based representation: based
on the Jean
-
Yves
Ramel’s

work

(2) Seeds detection in graph: a set of scoring functions is
computed from all nodes and edges

Scoring

functions

Edges

PE1

parallel segments

PE2



橵湣瑩潮j

偅P

捯c灡牡扬攠汥湧瑨l獥杭敮瑳

乯摥N

偎P

2
-
㌠捯湮散瑩潮

偎P

獨潲琠汥湧瑨l獥杭敮瑳

(3) Score propagation: based on a shortest path algorithm, a
global score is normalized from individual score of edge/node

46

Symbol recognition & spotting


“Graph scoring for symbol spotting (2)”

People

Rashid
Qureshi

Funding

HEC scholarship

Starting

2005

Ref

R.
Qureshi

and al
. Spotting
Symbols in Line Drawing Images
Using Graph Representations.
GREC, 2008.

(4) Results &

performance evaluation

0

1

Precision

Recall

SESYD dataset

47

Symbol recognition & spotting

“Bayesian based system for symbol spotting (1)”

(1) Representation phase: used the
graph based representation of Jean
-
Yves
Ramel

(2) Description phase: approach based on
attributes (of nodes and edges)

(3) Learning and classification phases base on Bayesian network

People

Muzzamil

Luqman

Funding

HEC scholarship

Starting

2008

Ref

M.M.
Luqman
. A Content Spotting
System for Line Drawing Graphic
Document Images. ICPR, 2010.

(3.1.) Discretization step: based on the
Akaike

Information Criterion

(3.2.) Learning step:


-

network topology is done from a genetic algorithm


-

parameters conditional probabilities is done from a


maximum likelihood estimation

(3.3.) Classification step:

48

Symbol recognition & spotting

“Bayesian based system for symbol spotting (2)”

(4) Performance evaluation at recognition level

People

Muzzamil

Luqman

Funding

HEC scholarship

Starting

2008

Ref

M.M.
Luqman
. A Content
Spotting System for Line
Drawing Graphic Document
Images. ICPR, 2010.

clean

Rotation

Scaling

Scalability

Clean

yes

Yes

100%

100%

100%

100%

Hand
drawn

level1

no

no

99%

96%

93%

92%

level2

no

no

98%

94%

92%

91%

level3

no

no

91%

77%

71%

69%

Binary degrade

no

no

98%

95%

93%

92%

Binary
degrade

Hand
drawn

(5) Improvements of Rashid
Qureshi’s

results

ISRC 2003 dataset

Precision

Recall

SESYD dataset

49

Symbol recognition & spotting

“Graph Embedding”

50

People

N.
Sidere
, J.Y.
Ramel

Funding

Navidomass

Starting

2007

Ref

N.
Sidère

et al. Vector Representation of
Graphs : Application to the

Classification
of Symbols and

Letters. ICDAR 2009.

Topic:

Topological Graph Embedding

(1) A
lexicon

is

generated

from

the network of
non
-
isomorphic

graphs

(2)
The embedding is based on occurrences of the patterns

Symbol recognition & spotting

“International Symbol Recognition Contest 2011” (1)

id

domain

models

symbols

distortion

Training
#1
-
#7

Technical

36
-
150

16650

Rotation, Scaling,
Kanungo
, Context

Final

#1
-
#4

Technical

36
-
150

16800

Rotation, Scaling,

Kanungo
, Context

id

domain

models

images

symbols

distortion

Training

#8
-
#15

Electrical

Architectural

16
-
21

40

835

None,
Kanungo

Final

#5
-
#12

Electrical

Architectural

16
-
21

120

3463

None,
Kanungo

http://iapr
-
tc10.univ
-
lr.fr/index.php/symbol
-
contest
-
2011

Workshop

GREC 2011

Contest starting

March 2011

T
raining datasets

6
th

of April 2011

Call of participation

2sd of May 2011

Final datasets

25
th

of July

Contest slot

25
th

July
-

1
st

August 2011

People

M. Delalandre, R.
Raveaux

Funding

Support of TC10

Starting

2011

Ref

E.

Valveny

and al. Report on
the Symbol Recognition and
Spotting Contest. GRECLNCS
2012.

Recognition

Tests

Localization

Tests

Symbol recognition & spotting

“International Symbol Recognition Contest 2011” (2)

People

M. Delalandre, R.
Raveaux

Funding

Support of TC10

Starting

2011

Ref

E.

Valveny

and al. Report on
the Symbol Recognition and
Spotting Contest. GRECLNCS
2012.

Connected
components
filtering

Contour
detection and
sampling

Geometric
matching

groundtruth

results

set name

models

noise

recognition rate

Final #1

50

Kanungo

94.76%

Final #3

150

Kanungo

85.88%

Final #4

36

Context

96.22%

set name

domain

models

noise

precision / recall

Final #5

Architectural

16

None

0.62 / 0.99

Final #6

Architectural

16

Kanungo

0.64 / 0.98

Final #9

Electrical

21

None

0.37 / 0.56

Final #10

Electrical

21

Kanungo

0.44 / 0.63

Recognition

Tests

Localization

Tests

intersection

The participant
:
Nayef
, N. &
Breuel
, T. On the Use of Geometric
Matching for Both: Isolated Symbol Recognition and Symbol Spotting
Workshop on Graphics Recognition (GREC), 2011

52

Symbol recognition & spotting

“Performance characterization of symbol localization (1)”

People

Mathieu Delalandre

Funding

Na

Starting

2008

Ref

M. Delalandre

and al
. A
Performance Characterization
Algorithm for Symbol
Localization. GREC, 2010.

probability error

detection rate

p
1

p
3

p
2

Groundtruth, gravity
centers, contours

Result points

Highest probabilities

Lowest probabilities

how
to make the difference
between segmentation errors of
background with segmentation
errors of objects

The characterization method must do some rejection, ways
to solve
are...

1. To define and apply manual threshold (
bad ...)

2. To propose a method for
adaptative

tthresholding, how to do ?

Open problem

Key idea, characterization method based on context

53

Symbol recognition & spotting

“Performance characterization of symbol localization (2)”

People

Mathieu Delalandre

Funding

Na

Starting

2008

Ref

M. Delalandre

and al
. A
Performance Characterization
Algorithm for Symbol
Localization. GREC, 2010.

3.1 Localization
comparison

3.2 Probability
scores

3.3 Matching
algorithm

Groundtruth

Results

(1) The method

(1.1) Localization comparison moves results
from Euclidean to a scale space, to deal with the
scale invariance

(1.2) Probability scores are computed from a groundtruth
point
gi

and the result point r, considering the neighboring
groundtruth point
gj
. Final result is computed considering
all the groundtruth using a probability score function

(1.3) Matching algorithm looks for statistical distribution of
single, miss, merge and split cases, in an decreasing order of
precision using a bipartite list.

3.4 Transform
function

probability error

detection rate

detection rates

0

1

0

1

score error

ε

single (T
s
)

alarm (T
f
)

multiple (T
m
)

(1.4) Transform function make results context independent,
making difference with self
-
matching of groundtruth, to
achieve coherent comparison of methods on different datasets

single detections T
s

0

1

0

1

score error (
ε
)

ε

g

s
i

54

Symbol recognition & spotting

“Performance characterization of symbol localization (3)”

People

Mathieu Delalandre

Funding

Na

Starting

2008

Ref

M. Delalandre

and al
. A
Performance Characterization
Algorithm for Symbol
Localization. GREC, 2010.

(2) Results obtained from the
Rashid
Qureshi’s

system


Drawing level

Symbol level

Setting

backgrounds

5

models

16

Dataset

images

100

symbols

2521

Setting

backgrounds

5

models

17

Dataset

images

100

symbols

1340

floorplans

diagrams

-0.20
0.00
0.20
0.40
0.60
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
electrical diagrams

floorplans

1,00

0.00
0.10
0.20
0.30
0.40
0.50
0.00
0.10
electrical diagrams

floorplans


i(1) = 0.496


i(1) = 0.529

score error (
ε
)

score error (
ε
)


i
(
ε
)


i
(
ε
)

SESYD dataset

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition


56

Content based Image Retrieval

“Robust key points detection for document image retrieval”

People

The
Anh

Pham

Funding

VEID scholarship

Starting

10/2010

Ref

Na

The work focuses on robustness
-
keypoint

detection, robustness in DIA
including: 2D and illumination, noises, artifacts, blurred.


Keypoints

detection takes part of the general features extraction,

some typical features used as key points and their characteristics are:

Patterns

Regions

Edges, contours

Corners, salient points

Like
-
blob points

Level of detail
(primitive)

Robustness

Semantic

So, some ideas of new robust
keypoint

detection may be:


-

Detecting robust features first, then extracting salient points, or


-

Using robust methods (
i.e

Machine learning, Model
-



based, Parametric
-
based) to extract salient points, or


-

Combination of above methods

57

Content based Image Retrieval

“Logo recognition and spotting”

58

Problematic

-

document classification can be supported by logo recognition

-

a meta engine will manage the decision rules

-

spotting will depend of a previous stage of page segmentation







-

binary images at 300 dpi

-

time constraint:


to 1,5 s per image

People

Mathieu Delalandre

Funding

DOD project

Starting

04/2012

Ref

Na

Document
classification

Page
segmentation

Logo spotting

Logo
recognition

Meta engine

High variability and
scalability of logos

Logo are rich graphical parts, 300
dpi and
binarization

could result
in a high level of degradation

Some logo recognition case looks
like OCR only

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition

6.7. Graph matching and embedding


59

Camera based recognition

“Robust OCR for video text recognition”

60

People

Thierry
brouard

Funding

SNECMA project

Starting

2008

Ref

International patent

Problematic


-
Automatic routing of input letters by digitization and


OCR of input documents received from customers



-
Response time < 3 s, Recognition rate > 80%, Precision


Equal to 100%, Java environment on mutualized servers



Approach based on cognitive vision and knowledge based systems
(blackboard & mathematical theory of evidence), to achieve robust
segmentation & OCR


Camera based recognition

“Real time logos recognition in urban environment”

61

People

Mathieu Delalandre

Funding

Na (JSPS grant)

Starting

2010

Ref

Na

Problematic


-

Logo detection from video capture using some


handled interactions, to display context based


information (tourist check points, bus


stop, meal, etc.).


-

Hard points are the real time constraints and


the complexity of the recognition task.

video

a set of images

First goal of the project is to support the real time recognition. We start from the
hypothesis than logo appear in a static way in video. We propose to achieve an
automatic control/selection of image capture to reduce the amount of data to
process.

Frame
selection

Video
capture

Frame

stack

Gyro
access

Workflow
management

3D
Mapping

frame

tag/remove

stored

Detection/
Recognition

processing
charge

frequency, resolution
parameters

size

motion data

frame

overlapping

Talk workplan

1.
Tours city

2.
François
-
Rabelais University, les
deux

lions /
Portalis

3.
School of Engineering
Polytech’Tours


4.

Laboratory of Computer Science

5.

RFAI group

6.

DIA related work

6.1. Projects & partners outline

6.2. Layout analysis and document recognition

6.3. OCR, word spotting and signature verification

6.4. Symbol recognition & spotting

6.5. Content Based Image Retrieval

6.6. Camera based recognition

62