Win8 on Intel Programming Course Desktop : Perceptual Computing

minedesertSoftware and s/w Development

Oct 31, 2013 (4 years and 7 days ago)

96 views

Win8 on Intel Programming Course

Desktop : Perceptual Computing

Cédric Andreolli

www.Intel
-
Software
-
Academic
-
Program.com

paul.guermonprez@intel.com

Intel Software

2013
-
03
-
20

Agenda

Agenda

Introduction to
Perceptual

Computing

What is the Perceptual Computing?


The Intel
Perceptual

Computing

SDK

What can the SDK do? What is it for?


Set up the
programming

environment

Setting up the Visual Studio C++ environment will be explained here


Using

the SDK

Some of the SDK functionalities will be explained here

Introduction to
Perceptual

Computing

Introduction

What

is

Perceptual

Computing
?

Perceptual computing offers a new way to interact with
your
computer


What

do I
need
?

It
works with:

A camera created by Creative*

A SDK created by Intel*

Introduction

The
Creative

camera

3D
depth

sensor

720p camera

Dual
-
array
microphone

Introduction

The Intel*
Perceptual

SDK

Provides a lot of services that use the camera as input

Gesture
recognition

Skeleton tracking

Face recognition

Voice recognition

Voice synthesis

Image
capture


The camera associated with the SDK provides

a

new way to interact with your
Ultrabook


The Intel*
Perceptual

SDK

The Intel* Perceptual SDK

Architecture

The SDK provides libraries and header files that you must
add to your projects

It can be used in C++ but also in C#

The Intel* Perceptual SDK

High and
low

levels

programming

The SDK is flexible

You
can use Utility classes that provide an easy way to access gesture
recognition and high level process

You can also work directly on the raw images and implement your own
algorithms


More information

You can download the SDK for free
http://software.intel.com/en
-
us/vcsource/tools/perceptual
-
computing
-
sdk



The Intel* Perceptual SDK

Starting

with

the SDK

Download the SDK and install it

Be sure to remember the installation directory




The Intel* Perceptual SDK

SDK directories

The
bin

folder contains a lot of examples that give you an
idea of what you can do with the SDK

The
doc

folder contains the documentation in PDF format

The
include

folder contains the main headers

The
lib

folder contains the SDK library

libpcx.lib

The
sample

folder contains the source code of the
programs in the bin folder




The Intel* Perceptual SDK

The
Sample

folder

in
details

This folder also contains important tools in the
common

folder




The Intel* Perceptual SDK

The
Sample
/
common

folder

in
details

This folder contains


Important headers in the
include

directory


Important library in the
lib

directory




The Intel* Perceptual SDK

The
Sample
/
common

folder

in
details

In the lib folder, you can find 2 libraries

libpxcutils.lib

for the release

libpxcutils_d.lib

for the debug



Headers can be found in the $(SDK)/include directory

but also in the $(SDK)/sample/common/include.


Libraries can be found in the
$(SDK
)/lib directory

But also in the
$(SDK)/
sample/common/lib.


Set up the
environment

Set up the environment

Using

the SDK in C++

The SDK has been programmed in C++

This is an efficient language


Using

the SDK in C++

Open Visual Studio and create a C++ win32 console
application project



Set up the environment

What

do
we

have to do?

Visual
Studio’s

compiler must know
where

to
find


Header files


Library directories


Libraries

to
link


Open
your

project

properties

Right click on your project’s name and select
Properties

In
C/C++
-
>
Code Generation
-
>
Runtime Library

Select /
MTd

(we are working in Debug mode)


Set up the environment

Change the
Runtime

Library

Set up the environment

Add

include

directories

The SDK installation added some macro to your Visual
Studio environment

In your project properties, on the left, select
VC++
Directories

In
Include Directories
, select
<edit>
and add

$(PCSDK_DIR)/include

$(PCSDK_DIR)/sample/common/include


Set up the environment

Add

include

directories








Setting up the include Directories allows

Visual Studio to find the headers

Set up the environment

Add

library

directories

We also need to indicate Visual Studio where to find the libraries

In your project properties, on the left, select
VC++ Directories

In
Library Directories
, select
<edit…>
and add

$(PCSDK_DIR)/lib/$(Platform)

$(PCSDK_DIR)/sample/common/lib/$(
PlatformName
)/$(
PlatformToolset
)








Set up the environment

Add

library

directories








Setting up the
library Directories
allows

Visual Studio to
look for the libraries

in the good directories


Set up the environment

Link the
libraries

Setting up the
library

directories
is

not
enough
,
we

must
also

tell the compiler
which

are the
libraries

that

our

project

will

use

In the project properties, on the left, select
Linker
, then
Input

In the field
Additional Dependencies
, click on
<Edit…>









Set up the environment

Link the
libraries

Add the following libraries:

libpxc_d.lib

libpxcutils_d.lib

We are working in Debug mode, if you work in release
mode, remove the
“_d”









Set up the environment

Link the
libraries



Linking the libraries tells the compiler which are

The libraries that contain the implementation of the

Functions that you are using









Using

the SDK
-

HelloWorld

Using the SDK
-

HelloWorld

Programming

a Hello World

Programming an application that displays the images
acquired by the camera is really simple with the SDK



You
only

need

1 C++ file
and few
lines

of code








Using the SDK
-

HelloWorld

1
-

Add

the
includes





2
-

Request

access

to the
depth

image





Using the
SDK
-

HelloWorld

3
-

Create

the main
loop









Using the SDK
-

HelloWorld

Full code











Using the SDK
-

HelloWorld

See

the
result

You
can

run

the program,
you

will

see

windows

displaying

the
depth

map







Using the SDK to display an image that comes from the

Camera is really simple. You can also try to retrieve

the color image.

Using

the SDK
-

Gestures

Using the SDK
-

Gestures

Create

your

own

class

A good practice
when

using

the SDK
is

to
create

your

own

class
that

inherits

from

the
UtilPipeline

Create

and set a new
project

named

SDKTest

(win32 application)

Create

also

a new class
(
PipelineTest
)
that

inherits

from

UtilPipeline


Right click on the
project’s

name

and select
Add

and
Class









Using the SDK
-

Gestures

Create

your

own

class

In the
popup
, click on
Add





Set the
name

for
your

class and click on
Finish









Using the SDK
-

Gestures

Inherit

from

UtilPipeline

Open the header
(
PipelineTest.h
)
and change the content to












Using the SDK
-

Gestures

Step

1
-

The
OnGesture

method

When

a
gesture

is

recognized

by the SDK, the
method

OnGesture

is

called












Using the SDK
-

Gestures

Step

2
-

The
OnAlert

method

The
OnAlert

method

is

called

when

an
unusual

event

append
(
your

hands go out of the camera range, etc.)












Using the SDK
-

Gestures

Step

3

-

The
OnNewFrame

method

The
OnNewFrame

method

is

called

when

the SDK
is

able to
provide

a new frame












Using the SDK
-

Gestures

Bonus


Add

a
function

to
debug

later

This
example

will

also

be

used

to
work

on
skeleton

tracking

so

add

a
function

to display logs












Using the SDK
-

Gestures

It’s

time to code!

Go in the
.
cpp

file and
enable

the
gesture

pipeline in the
constructor

This
step

is

mandatory

when

you

want

to use the
gesture

recognition but
also

when

you

want

to
track

the
skeleton

nodes












Using the SDK
-

Gestures

Implement

the
OnGesture

The SDK
informs

your

application
when

a
gesture

is

identified

by
calling

the
function

OnGesture



The variable
data

provides

information
that

let
you

know
which

gesture

has been
recognized













Using the SDK
-

Gestures

The full
OnGesture

function











Using the SDK
-

Gestures

The full
OnAlert

function











Using the SDK
-

Gestures

Work

with

nodes

Using

the
skeleton

nodes

can

be

intereseting

to
implement

new
controls

in
your

application

To
print

out the
nodes

positions,
fill

the
printNodes

function


Here

is

the Full code









Using the SDK
-

Gestures

Full code
explained



Using the SDK
-

Gestures

The label
definitions







More information

You
can

get

more information in the documentation




Using the SDK
-

Gestures

Gestures

and
alerts



As
you

can

see
,
manipulating

the
gestures

or the
alerts

is

done


in a
similar

way
. You
can

let the SDK
inform

you

when


such

an
event

has been
detected
.



Using the SDK
-

Gestures

Lot of
features

The
UtilPipeline

class
provides

4 main
features

in
realtion

with

gestures

recognition,
voice

recognition, face
recognition and
access

to
raw

images








Using the SDK
-

Gestures

Access
raw

images

Manipulating

raw

images
allows

you

to
implement

your

own

algorithms

The
UtilPipeline

allows

you

to
access

the
raw

images
through

the
UtilPipelineRaw

class


Create

a new
project

Configure a new C++ win32 console application
that

works

with

the SDK

Name
it


«

RawImages

»






Using the SDK
-

Gestures

New
project

creation





Using the SDK
-

Gestures

Inherit

from

UtilPipeline

The
UtilPipeline

inherits

from

UtilPipelineRaw

so

everything

needed

to
manipulate

raw

images
is

also

included

in the
UtilPipeline

class




Using the SDK
-

Gestures

OnNewFrame

implementation

Using the SDK
-

Gestures

How to
get

image pixels?

From

the
previous

code,
you

can

directly

manipulate

the
images

Using the SDK
-

Gestures

Working

with

planes

The planes
hold

different

kind

of informations


For the
d
epth

map

plane[0]
holds

the
depth

map

plane[1]
holds

the confidence
map

plane[2]
holds

the UV
map

Using the SDK
-

Gestures

Write

your

filters

Manipulating

raw

images
allows

to
write

your

own

filters

The
lab

will

show
you

how to
retrieve

the first 20
centimeters

starting

from

the
closest

object

in front of the camera







Using the raw images provides a full control

on the events you want to work on.

Using the SDK
-

Gestures

Try

the
lab

A
lab

is

available

with

this

course!

You
will

learn

to use the
Perceptual

SDK in


C++


C#


Unity

You
will

also

learn

how to
create

a simple
filter

and how to
control a
character

in a 3D
game

License
Creative

Commons


By 3.0

You are free:


to Share



to copy, distribute and transmit the work


to Remix



to adapt the work


to make commercial use of the work

Under the following conditions
:


Attribution



You must attribute the work in the manner specified by the author or licensor (but
not in any way that suggests that they endorse you or your use of the work
).

With the understanding that:


Waiver



Any of the above conditions can be waived if you get permission from the copyright
holder.


Public Domain



Where the work or any of its elements is in the public domain under applicable
law, that status is in no way affected by the license.


Other Rights



In no way are any of the following rights affected by the license:


Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations;


The author's moral rights;


Rights other persons may have either in the work itself or in how the work is used, such as publicity or
privacy rights.


Notice



For any reuse or distribution, you must make clear to others the license terms of this
work. The best way to do this is with a link to this web page.


http://creativecommons.org/licenses/by/3.0
/