OpenCVTutorialx

bijoufriesΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

167 εμφανίσεις

An Introduction



Ouline


Introduction


GUI commands


Primitive data structures


Working with images


Working with videos


What is
OpenCV


Open source computer vision library in C language.


Generic image/video acquisition and manipulation


Optimized for Intel and intended for real
-
time
applications


Intel Image Processing Library (IPL)



Supported OS:


Linux, Mac OS, Windows, and Android.


Language supports:


C/C++, and Python


Third party wrappers


C# (
Emgu

CV)
http://www.emgu.com/wiki/index.php/Main_Page


Java (
JavaCV
)

http://code.google.com/p/javacv/

OS and Language Supports

OpenCV

Library


opencv_highgui.lib: High
-
level GUI


opencv_core.lib: Core Library


opencv_imgproc.lib: Image Processing


opencv_video.lib: Video Analysis


opencv_gpu.lib: GPU
-
accelerated Computer Vision


opencv_ml.lib: Machine Learning


And many more…

Supported Sources


Supported image formats:


BMP, DIB, JPEG, JPG, JPE, PNG, PBM, PGM, PPM, SR,
RAS, TIFF, TIF


Supported video formats:


AVI file


Web Camera


Live Stream


Naming Conventions


Function naming conventions:

cvActionTargetMod
(...)


E.g.:
cvLoadImage
,
cvReleaseImage



Matrix data types:

CV_<
bit_depth
>(S|U|F)C<
number_of_channels
>


S = Signed integer


U = Unsigned integer


F = Float


E.g.: CV_8UC1, CV_32FC2

Naming Conventions (Cont’d)


Image data types:

IPL_DEPTH_<
bit_depth
>(S|U|F)


E.g.: IPL_DEPTH_8U, IPL_DEPTH_32F



Header files:


#include <
cv.h
>


#include <
cvaux.h
>


#include <
highgui.h
>

Window management

Input handling

Window Management


Load an image:

IplImage
*
img
=0;

img
=
cvLoadImage
(
fileName
);

if(!
img
)
printf
("Could not load image file: %s
\
n",fileName
);




Create and position a window:

cvNamedWindow
("win1", CV_WINDOW_AUTOSIZE);

cvMoveWindow
("win1", 100, 100);



Window Management


Display an image:

cvShowImage
("win1",img);



Close a window:

cvDestroyWindow
("win1");



Resize a window:

cvResizeWindow
("win1",100,100);

Event Handling


Mouse Events


Keyboard Events


Trackbar

Events

(1) Mouse Events: 2 Steps

1.
Define a mouse handler:

void
mouseHandler
(
int

event,
int

x,
int

y,
int

flags, void*
param
) {



switch(event){



case CV_EVENT_LBUTTONDOWN:




if(flags & CV_EVENT_FLAG_CTRLKEY)




printf
("Left button down with CTRL pressed
\
n");




break;



case CV_EVENT_LBUTTONUP:




printf
("Left button up
\
n");




break;


}

}

2.
Register the handler:

mouseParam
=5;

cvSetMouseCallback
("win1",mouseHandler, &mouseParam);

(2) Keyboard Events


Get keyboard input without blocking:

int

key;

key=
cvWaitKey
(10); // wait 10ms for input


Get keyboard input with blocking:

int

key;

key=
cvWaitKey
(0); // wait indefinitely for input


When used in a loop:

while(1){


key=
cvWaitKey
(10);


if(key==27) break;


switch(key){



case 'h': ... break;



case '
i
': ... break;



}

}

(3)
Trackbar

Events: 2 Steps

1.
Define a
trackbar

handler:


void
trackbarHandler
(
int

pos) {



printf
("
Trackbar

position: %d
\
n",pos
);


}

2.
Register the handler:


int

trackbarVal
=25,
maxVal
=100;

cvCreateTrackbar
("bar1", "win1", &
trackbarVal

,
maxVal

,
trackbarHandler
);

CvPoint
,
CvSize
,
CvRect
,
CvScalar

CvArr
,
CvMat
,
IplImage


Primitive Data Structures

Data Structures

Contains

Represents

CvPoint

int

x, y

Point in an image

CvPoint2D32f

float x, y

Point in

2

CvPoint3D32f

float x, y, z

Point in

3

CvSize

int

width, height

Size of image

CvRect

int

x, y

Portion of image

CvScalar

double
val
[4]

RGBA value

CvArr

Abstract class

CvMat

Matrix
struct

IplImage

Image
struct

CvPoint

-

A Point

0

1

2

3

4

5

6

0

1

2

3

4

CvPoint

p =
cvPoint
(4,2)

CvSize



Image Size

0

1

2

3

4

5

6

0

1

2

3

4

CvSize

r =
cvSize
(5,7)

CvRect



A Portion of Image

0

1

2

3

4

5

6

0

1

2

3

4

CvRect

r =
cvRect
(2,1,4,3)

CvScalar

0

1

2

3

4

5

6

0

1

2

3

4

CvScalar

s =
cvScalar
(150,57,111,0)

B

G

R

A

CvScalar



|
--

double
val
[4]; //A container of 4
-
tuple (ch1,ch2,ch3,ch4) of doubles

Primitive Data Type (Cont’d)

CvArr

<<Abstract>>

CvMat

IplImage

2D Matrix

CvMat


|
--

int

type;

// elements type (
uchar,short,int,float,double
) and flags


|
--

int

step;

// full row length in bytes


|
--

int

rows, cols;

// dimensions


|
--

int

height, width;

// alternative dimensions reference


|
--

union data;


|
--

uchar
*

ptr
;

// data pointer for an unsigned char matrix


|
--

short*

s;

// data pointer for a short matrix


|
--

int
*

i
;

// data pointer for an integer matrix


|
--

float*

fl;

// data pointer for a float matrix


|
--

double* db;

// data pointer for a double matrix


ND
-
Matrix

CvMatND


|
--

int

type;

// elements type (
uchar,short,int,float,double
) and flags


|
--

int

dims;

// number of array dimensions


|
--

union data;


| |
--

uchar
*

ptr
;

// data pointer for an unsigned char matrix


| |
--

short*

s;

// data pointer for a short matrix


| |
--

int
*

i
;

// data pointer for an integer matrix


| |
--

float*

fl;

// data pointer for a float matrix


| |
--

double*db;

// data pointer for a double matrix


|


|
--

struct

dim[];

// information for each dimension


|
--

size;

// number of elements in a given dimension


|
--

step;

// distance between elements in a given dimension

Things you can do with
CvMat


Add


Sub


Div


CrossProduct


DotProduct


Sum


Trace


Transpose


Det


CalcCovarMatrix


EigenVV


InRange


InvSqrt


Invert


IsInf


IsNaN


Mahalonobis


Merge


Solve


SolveCubic


Sqrt


SVD

IplImage


|
--

int

nChannels
;

// Number of color channels (1,2,3,4)


|
--

int

depth
;

// Pixel depth in bits:


|


// IPL_DEPTH_8U, IPL_DEPTH_8S,


|


// IPL_DEPTH_16U,IPL_DEPTH_16S,


|


// IPL_DEPTH_32S,IPL_DEPTH_32F,


|


// IPL_DEPTH_64F


|
--

int

width
;

// image width in pixels


|
--

int

height
;

// image height in pixels


|
--

char*
imageData
;// pointer to aligned image data. Color images are stored in
BGR

order


|
--

int

dataOrder
;

//
cvCreateImage

can only create interleaved images


|


// 0
-

interleaved color channels,


|


// 1
-

separate color channels


|
--

int

origin
;

// 0
-

top
-
left origin,


|


// 1
-

bottom
-
left origin (Windows bitmaps style, flipped)


|
--

int

widthStep
;

// size of aligned image row in bytes


|
--

int

imageSize
;

// image data size in bytes = height*
widthStep


|
--

struct

_
IplROI

*
roi
;

// image ROI. when not NULL specifies image


|


// region to be processed.


|
--

char *
imageDataOrigin
;

// pointer to the unaligned origin of image data


|


// (needed for correct image
deallocation
)


|
--

int

align;

// Alignment of image rows: 4 or 8 byte alignment


|


//
OpenCV

ignores this and uses
widthStep

instead


|
--

char
colorModel
[4]; // Color model
-

ignored by
OpenCV

Interleaved Color Channels

0

1

2

3

4

5

6

0

1

2

3

4

IplImage


:


|
--

int

dataOrder
; //
cvCreateImage
() only creates interleaved images



// 0
-

interleaved color channels,



// 1
-

separate color channels

[0]

[1]

[2]

3
-
channel image (interleaved)

IplImage


:


|
--

char*
imageData
; // pointer to
aligned

image data


|
--

int

widthStep
; // size of aligned image row in bytes


|
--

int

imageSize
;

// image data size in bytes = height*
widthStep

Aligned Image Data

0

1

2

3

4

5

6

0

1

2

3

4

0

1

2

3

4

5

6

7

8

9

10

11

0,0

0,1

0,2



x

1,0

1,1

1,2



x

2,0

2,1

widthStep

widthStep

...

1
st

row

2
nd

row

.

.

.

.

.

.

.

.

.

.


imageSize
-
1

X

X

X

X



imageData

Accessing Image Elements

IplImage
*
src
=
cvLoadImage
(“lena.jpg”);

uchar

*
aPixelIn

= (
uchar

*)
src
-
>
imageData
;

for (
int

iRow

= 0;
iRow

<
src
-
>height
;
iRow
--

) {


for (
int

iCol

= 0;
iCol

<
src
-
>width
;
iCol
++ ) {


int

R, G, B;


B =
aPixelIn
[
iRow
*
src
-
>
widthStep

+
iCol
*
src
-
>
nChannels

+ 0 ];


G =
aPixelIn
[
iRow
*
src
-
>
widthStep

+
iCol
*
src
-
>
nChannels

+ 1 ];


R =
aPixelIn
[
iRow
*
src
-
>
widthStep

+
iCol
*
src
-
>
nChannels

+ 2 ];


}

}


0

1

2

3

4

5

6

7

8

9

10

11

0,0

0,1

x

1,0

1,1

x

widthStep

widthStep

...

1
st

row

2
nd

row

.

.

.

.

.

.

.

.

.

.


imageSize
-
1

aPixelIn

[0]

[1]

[2]

Allocating and releasing images

Reading and writing images

Image conversion

Drawing commands



Allocating and releasing images


Reading and writing images


Image conversion & annotation

Allocate an Image

Usage:

IplImage
*
cvCreateImage
(
CvSize

size,
int

depth,
int

channels);


Examples:

// Allocate a 1
-
channel byte image

IplImage
* img1=
cvCreateImage
(
cvSize
(640,480),IPL_DEPTH_8U,1);

// Allocate a 3
-
channel float image

IplImage
* img2=
cvCreateImage
(
cvSize
(640,480),IPL_DEPTH_32F,3);


Note:


It is developer’s responsibility to de
-
allocate the memory

Clone an Image

Usage:

IplImage
*
cvCloneImage
(const
IplImage
* image)


Example:

IplImage
* img1=
cvCreateImage
(
cvSize
(640,480), IPL_DEPTH_8U,1);

IplImage
* img2=0;

img2=
cvCloneImage
(img1);
// need to free memory later


Note:


It is developer’s responsibility to de
-
allocate the memory

Release an Image

Usage:

void
cvReleaseImage
(
IplImage
** image)


Example:

IplImage
*
img
=
cvCreateImage
(
cvSize
(640,480),IPL_DEPTH_8U,1);

cvReleaseImage
(&
img
);


Note:


You MUST release memory for
cvCreateXX
,
cvCloneXX

by calling
cvReleaseXX
. Or
memory leak
occurs.

Region of Interest of an Image

Usages:

void
cvSetImageROI
(
IplImage
* image,
CvRect

rect
);

void
cvResetImageROI
(
IplImage
* image);

CvRect

cvGetImageROI
(const
IplImage
* image);


Example:

If (!
cvGetImageROI
(
src
))
cvResetImageROI
(
src
);

CvRect

mask =
cvRect
(0,0,320,120);

cvSetImageROI
(
src
, mask);


NOTE:


The majority of
OpenCV

functions support ROI.


Allocating and releasing images


Reading and writing images


Image conversion & annotation

Reading an Image

Usage:

IplImage
*
cvLoadImage
(const char* filename,

int

iscolor
=CV_LOAD_IMAGE_COLOR)


Example:

IplImage
*
img
=
cvLoadImage
(“lena.jpg”);

if(!
img
)
printf
(“Failed to load lena.jpg
\
n”);


Note:


Can be color or grayscale byte/float
-
image


Byte image: 0
-
255; Float image: 0..1


A color image is assumed to have data in BGR order


Saving an Image

Usage:

int

cvSaveImage
(
const

char*
filename
,
const

CvArr
* image)


Example
:

if(!
cvSaveImage
(“out.jpg”,
img
))
printf
("Could not save out.jpg.
\
n”);


Note:


The output image format is determined based on the file
extension.


Allocating and releasing images


Reading and writing images


Image conversion & annotation

Color Model Conversion

Usage:

void
cvCvtColor
(const
CvArr
*
src
,
CvArr
*
dst
,
int

code)


Color code: CV_BGR2GRAY, CV_BGR2HSV, CV_BGR2Lab


Example:

IplImage

*
dst
=
cvCreateImage
(
cvGetSize
(
src
),
src
-
>depth, 1);

cvCvtColor
(
src

,
dst
, CV_RGB2GRAY);


Note:


Color spaces: RGB, BGR, GRAY, HSV,
YCrCb
, XYZ, Lab,
Luv
, HLS



R

G

B


L

a

b

Ex. RGB


Lab

RGB Image

Lab Image

Image Conversion (8
-
bit

16
-
bit)

Usage
:

void
cvConvertScale
(const
CvArr
*
src
,
CvArr
*
dst
, double scale=1, double
shift=0)

Example
:

//
IplImage

*
src

is a 8
-
bit unsigned color image

IplImage

*img32=0;

img32=
cvCreateImage
(
cvGetSize
(
src
),IPL_DEPTH_32F,
src
-
>
nChannels
);

cvConvertScale
(
src
, img32);


Image Conversion (Flip)

Usage
:

void
cvConvertImage
( const
CvArr
*
src
,
CvArr
*
dst
,
int

flags=0 );


CV_CVTIMG_FLIP
-

flip the image vertically


CV_CVTIMG_SWAP_RB
-

swap red and blue channels

Example
:

IplImage

*
img
=0;

img
=
cvCreateImage
(
cvGetSize
(
src
),
src
-
>depth,
src
-
>
nChannels
);

cvConvertImage
(
src
,
img
,

CV_CVTIMG_FLIP
);


Resize an Image

Usage
:

void
cvResize
(const
CvArr
*

src
,
CvArr
*

dst
,

int

interpolation=CV_INTER_LINEAR
)


Example
:

// resize to half of original size

IplImage

*
dst

=
cvCreateImage
(

cvSize
((
int
)(
src
-
>width
*0.5
),(
int
)(
src
-
>height
*0.5
)),

src
-
>depth,
src
-
>
nChannels
);

cvResize
(
src,dst
);


Annotate an Image

Usage:

void
cvCircle
(
CvArr
*

img
,
CvPoint

center
,
int

radius
,
CvScalar

color
,
int

thickness=1
,
int

lineType
=8
,
int

shift=0
)

void
cvLine
(
CvArr
*

img
,
CvPoint

pt1
,
CvPoint

pt2
,
CvScalar

color
,
int

thickness=1
,
int

lineType
=8
,
int

shift=0
)

void
cvPutText
(
CvArr
*
img
, const char* text,
CvPoint

org, const
CvFont
* font,
CvScalar

color)


Example:

cvCircle
(
img
,
cvPoint
(100,100), 20,
cvScalar
(0,255,0), 1);

Retrieve an image from video

Access video information

Write a video file

Open from a Video


From Camera:

CvCapture
* capture =
cvCaptureFromCAM
(0);



From an AVI File:

CvCapture
* capture =
cvCaptureFromAVI
("infile.avi");



From a Live Stream:

CvCapture
* capture =
cvCaptureFromFile

(“rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov");

If
OpenCV

failed

to connect to the source,
cvCaptureFromXX
(..) returns NULL.

Grab an Image

// make sure the connection succeeded

If (capture ==0) {


printf
(“Failed to open video source.
\
n”);


return;

}

IplImage
*
img

=
cvQueryFrame
( capture );

if (
img
==0) {


printf
(“Failed to grab image.
\
n”);


return;

}

cvShowImage
(“
myWindow
”,
img
);




Whenever you grab images from
CvCapture

object,
there is NO need to call
cvReleaseImage
().


It is
CvCapture’s

responsibility to free the memory!

Release the Capture Source

Example

cvReleaseCapture
(&capture);


Note
:


Images captured by the device is allocated/released by
the capture function.


There is no need to release it explicitly.


Access Video Information

Usage:

double
cvGetCaptureProperty
(
CvCapture
* capture,
int

property_id
)

int

cvSetCaptureProperty
(
CvCapture
* capture,
int

property_id
)


Example
:

int

height = (
int
)
cvGetCaptureProperty
(capture, CV_CAP_PROP_FRAME_HEIGHT);

int

width= (
int
)
cvGetCaptureProperty
(capture, CV_CAP_PROP_FRAME_WIDTH);

int

fps = (
int
)
cvGetCaptureProperty
(capture, CV_CAP_PROP_FPS);

int

count = (
int
)
cvGetCaptureProperty
(capture, CV_CAP_PROP_FRAME_COUNT);

Write to a Video File

CvVideoWriter

*writer = 0;

int

isColor

= 1;

int

fps = 30;

int

frameW

= 640;

int

frameH

= 480;

writer=
cvCreateVideoWriter
("
out.avi",CV_FOURCC
('P','I','M','1'),
fps,cvSize
(
frameW,frameH
),
isColor
);

For (
int

i
=0;
i
<100;
i
++)


cvWriteFrame
(writer,
img
[
i
]);

cvReleaseVideoWriter
(&writer);

Resources


Introduction to programming with
OpenCV


http://www.cs.iit.edu/~agam/cs512/lect
-
notes/opencv
-
intro/opencv
-
intro.html


OpenCV

Tutorial (with examples)


http://www.pages.drexel.edu/~nk752/tutorials.html


The
OpenCV

Tutorials


[
OpencvDir
]
\
doc
\
opencv_tutorials.pdf


The
OpenCV

Reference Manual


[
OpencvDir
]
\
doc
\
opencv2refman.pdf


OpenCV

Home


http://opencv.willowgarage.com/wiki/

Tutorial 7:
cvThreshold

double
cvThreshold
(const
CvArr
*

src
,
CvArr
*

dst
, double

threshold
,
double

maxValue
,
int

thresholdType
)




int

thresholdType
:


CV_THRESH_BINARY



CV_THRESH_BINARY_INV



CV_THRESH_TRUNC



CV_THRESH_TOZERO

Value & Threshold

Tutorial 10: Histogram


Histogram is about Data Partition & Counting

How to Create Histogram


CvHistogram

can be N
-
dimensional


Steps to create a
CvHistogram

1.
Define the number of bins for

each histogram

2.
Define the range for the bin

3.
Define the input array data

4.
Create

the histogram object and

calculate

the histogram

1.
cvCreateHist
() // histogram specification

2.
cvCalcHist
()

// process the input data using above spec.

# of bins

range

Things to do with
CvHistogram


cvCompareHist


cvGetMinMaxHistValue


cvThreshHist


cvQueryHistValue
*D


1D, 2D, 3D, ND