XH.2012Oct26.OpenCVx - Picb.ac.cn

soilflippantΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

79 εμφανίσεις

OpenCV

Introduction

Hang Xiao

Oct 26, 2012

History


1999 Jan :
lanched

by Intel, real time machine vision library for UI, optimized code for
intel


2000

Jun :
OpenCV

alpha
3



2000 Dec :
OpenCV

beta
1

for
linux


2006 : the
first 1.0
version supports Mac OS


2008 mid : obtain
corporate support
from Willow Garage


2009

Sep :
OpenCV

1.2

beta2.0


2009

Oct

: Version 2.0 released



2010

Dec

:
OpenCV

2.2



2011

Aug :
OpenCV

2.3



2012

Apr :
OpenCV

2.4.

Overview


Goals


Develop a universal toolbox for research and development in the field
of Computer Vision


Algorithms


More than
350

algorithms,
500

API


Programming
language



C/C++, C#,
Ch

, Python, Ruby,
Matlab
, and Java (using
JavaCV
)


OS
support



Windows, Android,
Maemo
, FreeBSD,
OpenBSD
,
iOS
, Linux and Mac
OS.


Licence


BSDlisence
, free for commercial and non
-
commmercial

Overview
-

Applications


2D and 3D feature toolkits


Egomotion

estimation


Facial recognition system


Gesture recognition


Human

computer interaction (HCI)


Mobile robotics


Motion understanding


Object identification


Segmentation and Recognition


Stereopsis Stereo vision: depth perception from 2 cameras


Structure from motion (SFM)Motion tracking

Overview
-

A statistical machine
learning library


Boosting (meta
-
algorithm)


Decision tree learning


Gradient boosting trees


Expectation
-
maximization algorithm


k
-
nearest neighbor algorithm


Naive Bayes classifier


Artificial neural networks


Random forest


Support vector machine (SVM)

Outline


Image
Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

Outline


Image
Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

Image Analysis


Thresholds


Statistics


Pyramids


Morphology


Distance transform


Flood fill


Feature detection


Contours retrieving

Image Thresholding


Fixed threshold;


Adaptive
threshold;


Image Thresholding Examples

Source picture

Fixed threshold

Adaptive threshold

Statistics


min, max, mean value, standard
deviation over the image


Norms C, L1, L2


Multidimensional histograms


Spatial moments up to order 3 (central,
normalized, Hu)



Multidimensional Histograms


Histogram operations


calculation, normalization, comparison, back project


Histograms types:


Dense histograms


Signatures (balanced tree)



EMD algorithm


The EMD computes the distance between two distributions, which
are represented by signatures.


The signatures are sets of weighted features that capture the
distributions. The features can be of any type and in any number of
dimensions, and are defined by the user.


The EMD is defined as the minimum amount of work needed to
change one signature into the other


EMD


a method for the
histograms comparison

.
)
,
(
,
,
)
,
(
)
,
(
,
1
,
,
1
,
,
,
j
i
j
i
ij
j
i
ij
j
i
j
i
ij
j
i
q
and
p
elements
the
between
distance
the
q
p
d
ts
coefficien
weight
f
f
q
p
d
f
Q
P
EMD
historams
two
Q
j
Q
q
P
i
P
p













Image Pyramids


Gaussian and
Laplacian

pyramids


Image
segmentation by
pyramids

Image Pyramids


Gaussian and Laplacian

Pyramid
-
based color
segmentation

On still pictures

And on movies

Morphological Operations


Two basic morphology operations using structuring element:



erosion



dilation


More complex morphology operations:


opening :
erosion + dilation


c
losing :
dilation + erosion


morphological gradient :
the difference between the dilation and the erosion of an
image


top hat :
the
difference between an input image and its
opening


black hat :
the
difference between the closing and its input image

Ope
n
in
g
Clo
sing
M
o
r
p
h
o
l
o
gi
c
a
l

G
r
a
d
i
e
n
t
To
p

H
a
t
Bla
ck
Ha
t
Morphological Operations Examples


Morphology
-

applying Min
-
Max
.

Filters and its combinations

Opening IoB= (I

B)

B

Dilatation I

B

Erosion I

B

Image

I

Closing I•B= (I

B)

B

TopHat(I)= I
-

(I

B)

BlackHat(I)= (I

B)
-

I

Grad(I)= (I

B)
-
(I

B)

Distance Transform


Calculate the distance for all non
-
feature points to
the closest feature point


Two
-
pass algorithm, 3x3 and 5x5 masks, various
metrics predefined

Flood Filling


Simple


Gradient

Feature Detection


Fixed filters (
Sobel

operator,
Laplacian
);


Optimal filter kernels with floating point
coefficients (first, second derivatives,
Laplacian
)


Special feature detection (corners)


Canny operator


Hough transform (find lines and line segments)


Gradient runs


Canny Edge Detector

Hough Transform

Detects lines in a binary image


Probabilistic Hough
Transform


Standard Hough
Transform

Another Sample of the
Hough Transform Using

Source picture

Result

Contour Retrieving


The contour representation:


Chain code (Freeman code)


Polygonal representation



Initial Point

Chain code for the curve:
34445670007654443

Contour representation

Hierarchical representation of
contours

Image Boundary

(W1)

(W2)

(W3)

(B2)

(B3)

(B4)

(W5)

(W6)

Contours Examples

Source Picture

(300x600 = 180000 pts total)

Retrieved Contours

(<1800 pts total)

After Approximation

(<180 pts total)

And it is rather fast: ~70 FPS for 640x480 on complex scenes

Outline


Image Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

Structural Analysis


Contours processing


Approximation


Hierarchical representation


Shape characteristics


Matching


Geometry


Contour properties


Fitting with primitives


PGH:

pair
-
wise geometrical histogram for the
contour
.

Contour Processing


Approximation:


RLE algorithm

(
chain code)


Teh
-
Chin approximation (polygonal)


Douglas
-
Peucker

approximation (polygonal);


Contour moments (central and normalized up to order 3)


Hierarchical representation of contours


Matching of contours


Hierarchical Representation
of Contours


A contour is represented with a binary tree


Given the binary tree, the contour can be retrieved with arbitrary precision


The binary tree is quasi invariant to translations, rotations and scaling

Contours matching


Matching based on hierarchical
representation of contours

Geometry


Properties of contours: (perimeter, area,
convex hull, convexity defects, rectangle of
minimum area)


Fitting: (2D line, 3D line, circle, ellipse)


Pair
-
wise geometrical histogram

Pair
-
wise geometrical
histogram (PGH)

)
,
(
j
i
p
.
)
,
(
/
)
,
(
)
(
,
)
,
(
/
)
,
(
)
(
,
)]
(
),
2
(
),
1
(
),
(
),
2
(
),
1
(
[









i
i
c
j
j
r
T
c
c
c
r
r
r
PGH
j
i
p
j
i
p
i
j
E
j
i
p
j
i
p
j
i
E
M
E
E
E
N
E
E
E
f


Outline


Image Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

Object Recognition


Eigen objects


Hidden Markov Models

Eigen Objects

.
.
}
,
,
,
{
,
,
1
},
,
,
,
{
),
(
/
1
,
/
1
),
(
)
(
,
cov
}
{
,
,
1
,
}
,
,
{
2
1
1
1
1
1
2
1
vectors
eigen
and
values
eigen
and
basis
eigen
m
m
i
e
e
e
e
u
u
v
e
u
m
u
u
u
u
u
c
matrix
ariance
c
C
n
m
m
i
R
u
u
u
u
i
m
i
i
i
i
i
n
i
i
i
l
k
k
l
i
k
i
i
l
m
k
k
l
l
l
j
l
l
l
i
l
ij
ij
n
i
n
i
i
i



































Eigen objects
(continued)

Hidden Markov Model

Definitions


}
,...,
,
{
2
1
N
s
s
s
S

}
,...,
,
{
2
1
M
o
o
o
O

O
q
t

)}
|
(
\
{
1
i
t
j
t
ij
ij
s
q
s
q
P
a
a
A





)}
|
(
\
{
j
i
ij
ij
s
o
P
b
b
B


)}
(
\
{
0
i
i
i
s
q
P






-

The set of states

-

The set of measurements

-

The state at time

t


-

The transition probability matrix

-

The conditional probability
matrix

-

The starting states distribution

Embedded
HMM for Face
Recognition

Model
-

-

Face ROI partition

Face recognition

using Hidden Markov Models


One person


one HMM


Stage 1


Train every HMM





Stage 2


Recognition



P
i
-

probability


Choose
max(P
i
)




1

n

i

Outline


Image Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

Motion Analysis and
Object Tracking


Background subtraction


Motion templates


Optical flow


Active contours


Estimators

Background Subtraction


Background model (normal distribution)


Background statistics functions:


Average


Standard deviation


Running average



Motion Templates


Object silhouette


Motion history images


Motion history gradients


Motion segmentation algorithm

silhouette

MHI

MHG

Motion Segmentation
Algorithm


Two
-
pass algorithm labeling all motion segments

Motion Templates Example



Motion

templates

allow
to retrieve the dynamic
characteristics of the
moving object

Optical Flow


Block matching technique


Horn &
Schunck

technique


Lucas &
Kanade

technique


Pyramidal LK algorithm


6DOF (6 degree of freedom) algorithm









































y
x
t
y
y
x
y
x
x
I
I
I
b
I
I
I
I
I
I
G
y
x
X
b
X
G
dt
dy
y
I
dt
dx
x
I
t
I
t
y
x
I
dt
t
dy
y
dx
x
I
,
,
,
),
,
(
,
);
/
(
/
)
/
(
/
/
);
,
,
(
)
,
,
(
2
2
Optical flow equations:

Pyramidal Implementation of the
optical flow algorithm

J image

I image

Image Pyramid
Representation

Iterative Lucas


Kanade
Scheme

Generic Image

(L
-
1)
-
th Level

L
-
th Level

Location of point u on image
u
L
=u/2
L

Spatial gradient matrix

Standard Lucas


Kanade scheme for
optical flow computation at level L
d
L

Guess for next pyramid level L


1


Finally,




Image pyramid building

Optical flow computation











2
2
,
,
y
y
x
y
x
x
I
I
I
I
I
I
G
)
(
2
1
L
L
L
d
g
g



0
0
g
d
d


d
U
V


6DOF Algorithm
























N
i
ROI
T
i
t
N
i
ROI
i
T
i
I
I
ds
I
I
s
X
I
s
I
I
i
1
1
/
/
/

).
(
s
X


Parametrical optical flow equations:

Active Contours


Snake energy:


Internal energy:


External energy:


Two external energy types:

ext
E
E
E


int
curv
cont
E
E
E


int
con
img
ext
E
E
E


min
,
)
(
,











img
curv
cont
img
img
E
E
E
E
I
grad
E
I
E



Estimators


Kalman

filter


ConDensation

filter

Kalman
object

tracker

Outline


Image Analysis


Structural Analysis


Object Recognition


Motion Analysis and Object Tracking


3D
Reconstruction

3D reconstruction


Camera Calibration


View Morphing


POSIT

Camera Calibration


Define intrinsic and extrinsic camera parameters.


Define Distortion parameters

]
,
[
],
,
,
[
,
,
,
1
0
0
0
0
,
]
[
3
2
1
33
32
31
23
22
21
13
12
11
v
u
p
Z
Y
X
P
t
t
t
T
r
r
r
r
r
r
r
r
r
R
c
f
c
f
A
P
RT
A
p
y
y
x
x




































.
)],
2
/
(
2
[
)
(
~
)],
2
/
(
2
[
)
(
~
2
2
2
2
1
2
4
2
2
1
2
2
1
4
2
2
1
y
x
r
y
y
r
p
x
p
r
k
r
k
c
v
v
v
x
x
r
p
y
p
r
k
r
k
c
u
u
u
y
x






















Camera Calibration

Now, camera calibration can be done by holding
checkerboard in front of the camera for a few seconds
.

And after that you

ll get:

3D view of etalon

Un
-
distorted image

View Morphing

POSIT Algorithm


Perspective projection:


Weak
-
perspective projection:

i
i
i
i
i
i
Y
Z
f
y
X
Z
f
x




)
/
(
,
)
/
(
.
/
,
,
Z
f
s
Y
s
y
X
s
x
i
i
i
i





OpenCV

Websites


http://
opencv.org


OpenCV

official webpage.


http://opencvlibrary.sourceforge.net/



OpenCV

documentation and FAQs.


OpenCV

Examples


adaptiveskindetector

: detect skin area


fback_c

: dense
Franeback

optical flow


c
ontours : calculate contours on different levels


d
elaunay

:
delaunay

triangle


find_obj

: SURF Detector and Descriptor
using either
FLANN
or
brute force
matching on
planar
objects


m
orphology : open/close, erode/dilate


m
otempl


motion templates


m
ser_sample

: Maximal
Extremal

Region interest point
detector


polar_transforms

: illustrates
Linear
-
Polar and Log
-
Polar image
transforms


pyramid_segmentation

:
color pyramid segmentation


Thanks !!!