Gait Analysis for Human Identification (GAHI)

brewerobstructionΤεχνίτη Νοημοσύνη και Ρομποτική

7 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

285 εμφανίσεις



2009

Faculty of Computer and
Information Sciences

1/1/2009

Gait Analysis for Human
Identification (GAHI)

Faculty of Computer and Information
Sciences


Supervisors
:



Prof
.

Dr
.

Ahmed Mohammed Hamad
.



Dr. Eman Shaaban.

Teaching Assistant:



Mohammad Hamdy
.

Team Member
s
:



Agwad Hammad El
-
Sayed

(CSys)



Sherif Saad El
-
Din

(IS)



Salah El
-
Din Emam

(IS)



Saed Ezz El
-
Arab

(IS)


2009

Gait Analysis for Human Identification (GAHI)



2

Acknowledgment



irs
t of all, we would like to direct o
ur unlimited gratitude to
ALLAH

who, by his will directed us to what is best and gave us faith in
ourselves and our work.


We would like to thank
Professor
Dr.

Ahmed
Mohammed

Hamad

for his continuous support and encouragement.

Also
w
e would like to thank
Dr.
Eman Shaaban

for h
er

continuous support and encouragement.



A special thank to
Teaching Assistan
t

Mohamm
a
d Hamdy

for his
e
ffort,
time
and support.


Thank You














F

2009

Gait Analysis for Human Identification (GAHI)



3

Table of Contents

Acknowledgment

................................
................................
................................
................................
................................
..

2

Table of Contents

................................
................................
................................
................................
................................
..

3

List of Figures and Tables

................................
................................
................................
................................
.................

4

Abstract

................................
................................
................................
................................
................................
......................

5

Chapter 1:
Introd
u
ction

................................
................................
................................
................................
.....................

6

1.1
Overview

................................
................................
................................
................................
................................
..........

7

1.2
Biometrics

................................
................................
................................
................................
................................
.......

8

1.
3
Different types of
identification

................................
................................
................................
.............................

9

1.4
Project Motivation

................................
................................
................................
................................
.....................

15

1.
5
Project Scenario

................................
................................
.............................

Error! Bookmark not defined.

1.
6
Project Phases

................................
................................
................................

Error!
Bookmark not defined.

1.
7
System
Architecture

................................
................................
.....................

Error! Bookmark not defined.

1.
8
System Use Case

................................
................................
.........................

2
Error! Bookmark not defined.

1.
9
Tools Used

................................
................................
................................
................................
................................
.....

22

1.
10
Min
imu
m Requirments

................................
................................
............

Error! Bookmark not defined.

1.
11
S
umary of each Chapter

................................
................................
................................
................................
........

24

Chapter 2:
Human Detection and Tracking

................................
................................
................................
..........

25

2.1
Overview

................................
................................
................................
...........

Error! Bookmark not defined.

2.2
Human Detection and Tracking Block diagram

................................

Error! Bookmark not defined.

2.3
Moving Object Detection

................................
................................
................................
................................
.........

28

2.3.1
types of Beckground Model

................................
................................
................................
..........................

28

2.3.2Shadow Removal Based on

Gaussain Model

................................
................................
..........................

34

2.3.3
Connected Components Labeling

................................
................................
................................
...............

36

2.3.4Class diagram

................................
................................
................................
................................
......................

39

2.4
Object Tracking

................................
................................
................................
................................
...........................

40

2.4.1
Human Tracking
Types

................................
................................
...

Error! Bookmark not defined.
1

2.4.2Human Tracking Limitations
................................
........................

4
Error! Bookmark not defined.

2.4.3Class diagram

................................
................................
......................

Error! Bookmark not defined.
5

2.5
Object Classification

................................
................................
................................
................................
..................

4
6

2.5.1Object Classification Types

................................
...........................

Error! Book
mark not defined.
7

2.5.2Clssification Algorithm based on CodeBook

Theory

................................
................................
..........

4
9

2.5.3Class diagram

................................
................................
................................
................................
......................

5
3

Chapter 3:
Feature Extraction

................................
................................
................................
................................
......

5
4

3.1
Overview

................................
................................
................................
................................
................................
........

5
5

3.2
Feature Extraction Types
................................
................................
................................
................................
........

5
6

3.3
Gait Energy Image (GEI) Feature Extraction

................................
................................
................................
..

5
7

3.4Model
-
Based Feature Extraction

................................
................................
................................
.........................

6
2

3.5
Gait Energy Image (GEI)
Class diagram

................................
................................
................................
............

7
0

Chapter 4:
Training and Classification

................................
................................
................................
....................

71

4.1
Overview

................................
................................
................................
........

Error! Bookmark not defined.
2

4.2Priciple Component Analysis (PCA)

................................
...................

Error! Bookmark not defined.
5

4.3Multiple Discriminant Analysis (MDA)

................................
................................
................................
.............

81

4.4Training and Classification Class Diagram

................................
................................
................................
......

85

Chapter
5
: Conclusion and future work

................................
................................
................................
.................

88

5
.1
Conclusion

................................
................................
................................
................................
................................
.....

89

5
.
2Future Work

................................
................................
................................
................................
................................
.

89

Appendix:

Mathematical Background

................................
................................
................................
.....................

90

Bibliography

................................
................................
................................
................................
................................
.......

102

2009

Gait Analysis for Human Identification (GAHI)



4

List of Figures

F
IGURE
1.1
:
F
INGER PRINT CLASSIFI
CATION

................................
................................
................................
...............

9

F
IGURE
1.2
:
D
NA
S
TRUCTURE

................................
................................
................................
..............................

10

F
IGURE
1.3
:
SURVEILLANCE CAMERA

USED BY
F
ACIAL
R
ECOGNITION SYSTEM

................................
................................
.

1
1

F
IGURE

1
.4
:
P
ROJECT
S
CENARIO

................................
................................
................................
...........................

16

F
IGURE

1
.5
:
S
YSTEM
P
HASES

................................
................................
................................
...............................

1
7

F
IGURE

1
.6
:
S
YSTEM
A
RCHITECTURE

................................
................................
................................
......................

1
9

F
IGURE

1
.7
:
S
YSTEM
U
SE
C
ASE

................................
................................
................................
............................

21

F
IGURE
2.
1
:
H
UMAN
D
ETECTION AND TRACKIN
G
B
LOCK DIAGRAM
.

................................
................................
.............

27

F
IGURE
2.
2
:
A
DAPTAVIE
B
ACKGROUND
S
UBTRACTION
S
AMPLE

................................
................................
..................

30

F
IGURE
2.
3
:
A
DAPTAVIE
B
ACKGROUND
S
UBTRACTION
S
AMPLE

................................
................................
..................

32

F
IGURE
2.
4
:
C
ONNECTIVITY
T
YPES

................................
................................
................................
........................

3
6

F
IGURE

2.
5
:
O
UTPUT OF
C
ONNECTED
C
OMPONENT
A
LGORITHM

................................
................................
................

3
8

F
IGURE
2.
6
:
M
OVING
O
BJECT

D
ETECTION
C
LASS DIAGRAM

................................
................................
.......................

39

F
IGURE

2.
7
:
T
RACKING
S
EQUENCES

................................
................................
................................
.......................

43

F
IGURE
2.
8
:
H
UMAN
T
RACKING
C
LASS DIAGRAM

................................
................................
................................
.....

45

F
IGURE

2
.
9
:
N
ORMALIZED
I
MAGE

................................
................................
................................
.........................

4
9

F
IGURE

2.
10
:
B
OUNDARY OF
H
UMAN
B
ODY

................................
................................
................................
...........

50

F
IGURE
2.
11
:
O
BJECT
C
LASSIFICATION
C
LASS DIAGRAM

................................
................................
............................

53

F
IGURE

3
.
1
:
G
AIT
E
NERGY
I
MAGE
(GEI)

B
LOCK DIAGRAM

................................
................................
.........................

57

F
IGURE
3.2
:G
AIT
E
NERGY
I
MAGE
(GEI)

OF PERSON UNDER DIFF
ERENT
C
O
N
DITIONS

................................
......................

58

F
IGURE

3.3
:E
XAMPLE OF
F
EATURE
S
ELECTION

................................
................................
................................
........

60

F
IGURE

3.4
:C
ORNER PRXIMITY
M
EASURE
RE
SULT

................................
................................
................................
...

66

F
IGURE

3
.
5
:
H
ELL STRIKE EXTRACTIO
N USING PROXIMITY ME
ASURE

................................
................................
..............

67

F
IGURE

3
.
6
:
M
OTION ANALYSIS OF TH
E JOINTS

................................
................................
................................
........

68

F
IGURE

3
.
7
:
G
AIT
E
NERGY IMAGE
C
LASS DIAGRAM

................................
................................
................................
...

70

F
IGURE

4
.
1
:
T
RAINING
B
LOCK DIAGRAM

................................
................................
................................
................

73

F
IGURE

4
.
2
:R
ECOGNITION
B
LOCK DIAGRAM

................................
................................
................................
...........

74

F
IGURE

4.3
:PCA

EXAMPLE DATA
................................
................................
................................
..........................

77

F
IGURE

4
.
4
:A

PLOT OF
N
ORMALIZED DATA

................................
................................
................................
............

79

F
IGURE

4.5
:T
RAINING AND
C
LASSIFICATION
C
LASS DIAGRAM

................................
................................
.....................

85


List of Tables

T
ABLE

A
PPENDIX
.
1
:
C
ALCULATION OF STANDA
RD DEVIATION

................................
................................
.....................

93

T
ABLE

A
PPENDIX
.2
:
2
-
DIMENSIONAL DATA SET

AND CALCULATION OF

C
OVARIANCE

................................
.......................

97









2009

Gait Analysis for Human Identification (GAHI)



5

Abstract



Gait
-

A particular way or manner of
moving on foot.


Gait analysis for human
identification

(GAHI) is the process of
identifying
individual
s

by the
manner

in which they walk. This is a
marker less unobtrusive biometric, which offers the possibility to
identify people at a distance, without any interaction or co
-
operation
with

the subject; this is the property which makes it as attractive as a
method of
identification.


This project aims to develop a system capable of automatic gait
recognition. A person’s gait signature is created using

model
-
based
or model
-
free approaches
,
then

can be used to identify the
subject in
subsequent video sequences.













2009

Gait Analysis for Human Identification (GAHI)



6

Chapter 1: Introduction


This
Ch
apter is a brief review of GAHI system
.






Overview



Biometrics



Different type
s

of human identification methods



Project
Motivation



Project Scenario



Project
Phases



System Architecture



System Use Case



Tools



Minimum Requirements



Summary Of Each Chapter








Chapter

1

2009

Gait Analysis for Human Identification (GAHI)



7

1.1: Overview

With the increasing threat of terrorist, the advanced video
surveillance system has to be put into use. The advanced video
surveillance system needs to analyze the behaviors of people in order
to prevent the occurrence of the potential dangerous case.


One of
these surveillance systems

is
GAHI

system
;

the
system
proposed

to identify an individual efficiently and accurately is an
important
need
. Controlled environments such as banks, military
installations and even airports need to be able to
quickly detect
threats and provide differing levels of access to different user groups.


Gait as a biometric has many advantages as stated above which make
it an attractive proposition as a method of identification. Gait

main
advantage, unobtrusive identi
fication at a distance, makes it a very
attractive biometric.


The ability to identify a possible threat from a distance, gives the user
a time frame in which to react before the suspect becomes a possible
threat.


Another motivation is that video footag
e of suspects are readily
available, as surveillance cameras are relatively low cost and
installed in most buildings or locations requiring a security presence,
the video just needs to be checked against that of the suspect.









2009

Gait Analysis for Human Identification (GAHI)



8




1.
2
:
Biometrics

Biometrics
is

used in a wide array of applications, which makes
a precise definition difficult to establish. The most general definition
of a biometric is:


“A physiological or behavioral characteristic, which can be used to
identify and verify the identit
y of an individual”


There are numerous biometric measures which can be used to help
derive
individual's

identity. They can be classified into two distinct
categories:


a
-

Physiological


these are biometrics which
is

derived from a direct
measurement of a part of a human body. The most prominent and
successful of these types of measures to date are fingerprints, face
recognition, iris
-
scans and hand scans.

b
-

Behavioral


extract characteristics based on an action perfor
med by
an individual, they are an indirect measure of the characteristic of the
human form. The main feature of a behavioral biometric is the use of
time as a metric. Established measures include keystroke
-
scan and
speech patterns.


Biometric identificatio
n should be an automated process.
Manual feature extraction would be both undesirable and time
consuming, due to the large amount of data that must be acquired
and processed in order to produce a biometric signature. Inability to
automatically extract the
desired characteristics which would render
the process infeasible on realistic size data sets, in a real
-
world
application.


With a biometric a unique signature for an individual does not exist,
each time the data from an individual is acquired it will generate a
slightly different signature, there is simply no such thing as a 100%
match. This does not mean that the systems are
inherently insecure,
as very high rates of recognition have been achieved. The recognition
2009

Gait Analysis for Human Identification (GAHI)



9

is done through a process of correlation and thresholding, but
systems offering 100% recognition should be greeted with a pinch of
salt.



1.
3
:
Different type
s

of id
entification methods


1.
3
.1:
Finger
print identification or palm
print
identification

Is

the process of comparing questioned and known friction skin
ridge impressions from fingers or palms or even toes to determine
whether

the impressions are from the same finger or palm. The
flexibility of friction ridge skin means that no two finger or palm
prints are ever exactly alike (never identical in every detail), even
two impressions recorded immediately after each other.


Fingerp
rint identification (also referred to as individualization)
occurs when an
expert determines

that two friction ridge
impressions originated from the same finger or palm (or toe, sole) to
the exclusion of all others
.


Classifying fingerprints
:






(a)


(b)

2009

Gait Analysis for Human Identification (GAHI)



10






(c)


(d)


Fig1
.1
:
Fingerprint

Classification

(a)
Arch, (
b
) Loop (right loop), (
c
)
Whorl, (
d
) Arch (tented arch)



1.
3
.
2
:
DNA Profiling

Identification of individuals on the basis of their

respective
DNA profiles. DNA profiles are basically just sets of numbers that can
be used as
an

identifier.


The number set can be encrypted to a DNA identification number.
DNA profiling should thus not be confused with full genome
sequencing.




2009

Gait Analysis for Human Identification (GAHI)



11


(a)





Fig
1.
2: (a) DNA Structure
.










1.
3
.3
:
A facial recognition system

Is

a computer application for automatically identifying or
verifying a person from a digital image or a video frame from a video
source
.


One of the ways to do this is by comparing
selected facial features
from the image and a facial database.

Some facial recognition algorithms identify faces by extracting
landmarks, or features, from an image of the subject's face. For
example, an algorithm may analyze the relative position, size, a
nd/or
2009

Gait Analysis for Human Identification (GAHI)



12

shape of the eyes, nose, cheekbones, and jaw.

These features are then
used to search for other images with matching features
.

Other algorithms normalize a gallery of face images and then
compress the face data, only saving the data in the image that
is
useful for face detection. A probe image is then compared with the
face data
.


One of the earliest, successful systems

is based on template matching
techniques applied to a set of salient facial features, providing a sort
of compressed face representati
on.






(a)


Fig1.
3: (a)
surveillance camera

used by the system
.



1.
3
.
4
:
Gait Analysis

Is

a behavioral biometric source that can be acquired at a
distance.


Gait recognition is the term typically used in the computer
community to refer to the automatic extraction of visual cues that
characterize the motion of a walking person in
a
video and is used for
identification purposes in surveillance systems
.



Why
Gait Analysis for Human Identification
?


The
Gait is defined as:

2009

Gait Analysis for Human Identification (GAHI)



13

“A particular way or manner of moving on foot”


Using gait as a biometric is a relatively new area of study, within the
realms of computer vision.

It has been receiving growing interest within the computer vision
community and a number of gait metrics have been developed.


Early psychological studies into gait by Murray suggested that gait
was a unique personal characteristic, with cadence and was c
yclic in
nature.

Johansson carried out studies by attaching moving lights onto human
subjects on all the major body parts and showed these moving
patterns to human observers.



The observers could recognize the biological patterns of gait from the
moving
light displays (MLD’s), even when some of the markers were
removed, once again indicating gait as a potential candidate as a
prominent biometric.



We use the term gait recognition to signify the identification of an
individual from a video sequence of the

subject walking.


This does not mean that gait is limited to walking, it can also be
applied to running or any means of movement on foot.

Gait as a biometric can be seen as advantageous over other forms of
biometric identification techniques for the fol
lowing reasons:

a.


Unobtrusive


the gait of a person walking can be extracted
without the user knowing they are being analyzed and without
any cooperation
with

the user in the information gathering
stage unlike fingerprinting or retina scans.

b.

Distance
recognition


the gait of an individual can be
captured at a distance unlike other biometrics such as
fingerprint recognition.

c.

Reduced detail


gait recognition does not require images that
have been captured to be of a very high quality unlike other
biome
trics such as face recognition, which can be easily
affected by low resolution images.

d.


Difficult to conceal


the gait of an individual is difficult to
disguise, by trying to do so the individual will probably appear
more suspicious. With other biometric
techniques such as face
2009

Gait Analysis for Human Identification (GAHI)



14

recognition, the individuals face can easily be altered or
hidden.


Being a biometric, an individual’s biometric signature will be affected
by certain factors such as:

a.

Stimulants


drugs and alcohol will affect the way in which a
person walks.

b.

Physical changes


a person during pregnancy, after an
accident/disease affecting the leg, or after severe weight gain

/
loss can all affect the movement characteristic of an
individual.

c.


Psychological
changes


a
person's

mood can also affect an
individual's

gait signature.

d.


Clothing


the same person wearing different clothing may
cause an automatic signature extraction method to create a
widely varying signature for an individual.


Although these disadvantages are inhere
nt in a gait biometric
signature, other biometric measures can easily be disguised and
altered by individuals, in order to attempt to evade recognition.


The process of automatic gait feature extraction is made more
difficult from the external factors such

as lighting conditions, self
occlusion of feature points when the subjects’ legs cross over and
different types of clothing, which can all affect the data acquisition
process.


Results from previous work on automatic gait recognition look

promising with r
esult ranging up to 100% successful identification
rates on small database samples. Subsequent work has to be carried
out on larger databases to ascertain how effective an identification
method this will be with a large population size, but initial reports

are promising.








2009

Gait Analysis for Human Identification (GAHI)



15


1.
4
:
Project Motivation

The ability to identify an individual efficiently and accurately is
an important
need
. Controlled environments such as banks, military
installations and even airports need to be able to quickly detect
threats and provide differing levels of access to different user groups.


Gait as a biometric has many advantages as stated above which make
it an attractive proposition as a method of identification.

Gait

main advantage, unobtrusive identification at a
distance, makes
it a very attractive biometric.

The ability to identify a possible threat from a distance, gives the user
a time frame in which to react before the suspect becomes a possible
threat.


Another motivation is that video footage of suspects a
re readily
available, as surveillance cameras are relatively low cost and
installed in most buildings or locations requiring a security presence,
the video just needs to be checked against that of the suspect.


As well as the inherent advantages of gait, t
he increase in processor
power, along with the fall in price of high speed memory and data
storage devices have all contributed to the increased availability and
applicability of computer vision and video processing techniques.


Real time video
processing,

which is req
uired for gait recognition is a
feasible possibility

on current home PC technology,
making this
technology a viable security application.












2009

Gait Analysis for Human Identification (GAHI)



1
6

1.
5
:
Project Scenario



Fig
1
.
4
:

GAHI in Action


GAHI

system

can be used in a number of different scenarios. One
example would be to analyze the video stream from surveillance
cameras. If an individual walks by the camera whose gait has been
previously recorded and they are a known threat, then the system
will
iden
tify

the
m

and the appropriate authorities can be
automatically alerted and the person can be dealt with before they
are allowed to become a threat. The threat has been successfully
detected from a distance, creating a time buffer for authorities to take
an

action.


Such systems have a large amount of potential application domains,
such as airports, banks and general high security area.












2009

Gait Analysis for Human Identification (GAHI)



17

1.
6
:
Project
Phases


Fig
1.5
: Project Phases

Human Detection and Tracking
:

Human Detection and Tracking

process

means

to extract a
good quality Human(s) silhouette image and track this silhouette(s),
from a video frame. This was needed in order to perform the
extraction of the gait feature from the walking sequence.


Feature Extraction
:



Feature Extraction means that
the process in which an initial
measurement pattern or some subsequence of measurement
patterns is transformed to a new pattern feature. In image pattern
recognition, features often contain information relative to gray shade,
texture, shape or context.



Training or Classification
:


Training or classification process means

How to train (store)
feature of the gait
of specific person
and how to classify specific gait
feature correspondence to the training data.


2009

Gait Analysis for Human Identification (GAHI)



18




1.
7
:
System
Architecture


(put this in a separate part titled "
Objective
")

The main objective of this project is to:

1.


Develop a
system

that
is
capable of performing recognition of
individuals derived from a video sequence of a person walking. The
program should be
able to store the derived gait signature for
comparison at a later stage.

2.

Automatic extraction of relevant gait feature points should be
available from a video sequence in order to automate the
classification process.


System Architecture

2009

Gait Analysis for Human Identification (GAHI)



19






1.

Input
s

:

a.

Video sequences:

frames that captured by camera.

b.

Bin
-
data store:

contains the gait features of individuals.

2.

Processes:

a.

Human detection and tracking:

extract the people
images and track each one.

b.

Gait feature extraction:

extract the gait signatures of
individuals.

c.

Human training:

use Principle of component analysis
for train the extracted features.

d.

Human recognition:

identify human(s).

e.

File handler:

act as an interface between bin
-
data store
(input) and human recognition and human training.

Bin
-
Data
Store



Video
Sequences

Human
Detection

and Tracking

Gait Feature
Extraction

File
Handler

Human

Recognition


User Interface

NO

Human(s) Blob(s)

Feature data


Receive and
Store data


Recognition
Result(s)


Input
s

Process
es

Output

Stored data


Recognitio
n
?

Human
Training

YES

Frames

Store Data

Fig
1.
6:

System Architecture

2009

Gait Analysis for Human Identification (GAHI)



20

3.

Output
s
:

a.

User interface:

that not
ify if the human is identified or
not and display information about the identified ones










1.
8
:
System Use Case

2009

Gait Analysis for Human Identification (GAHI)



21







1.
9
:
Tools

Capture Video

Detect Moving Object

Track
Object

Classify
Object

Extract
Feature
s


Train

Feature
s

Classify

Feature

System

Fig
1.
7:

System
Use Case


2009

Gait Analysis for Human Identification (GAHI)



22



C#

Programming Language

(Visual Studio.Net
)



OpenCV

(
Library
)


1.
10
:
Minimum Requirements

GAHI user

must have the following requirements:



A
Camera

device
with

following specification
:



3X zoom



Resolution 10 Mega Pixel



Install
GAHI

application

on the
PC
.



Processor 3.0 G
Hz

Core 2 duo



High VGA Card
.



Large space on Hard Disk to store data.



4

GB memory


1.
11
:
Summary

for

Each Chapter
.

In the subsequent sections of this
Documentation
, all the project
development phases will be introduced in details.

Chapter 1:

This chapter is a brief review of
GAHI

System

which
includes the p
roject description, motivation
, and architecture and

system specification
.

Chapter 2:

This chapter shows how implementation process goes,
and describes
Human Detection

and Tracking

phase
.

Chapter 3:

This chapter shows how implementation process goes,
and describes
Feature Extraction

phase
.

2009

Gait Analysis for Human Identification (GAHI)



23

Chapter
4
:

This chapter shows how implementation process goes,
and describes
Training

and Classification

phase
.

Chapter
5
:

This Chapter is a description
for system result and future
work.

Appendix
:

This
Appendix

Containing Mathematical review

for
Training and Classification part
.


2009

Gait Analysis for Human Identification (GAHI)



24


Chapter
2
:
Human Detection and Tracking


This
Ch
apter is a
description of Human Detection and
Tracking Phase
.




Overview



Human Detection and Tracking

Block d
iagram



Moving Object Detection



Object Tracking



Object Classification







Chapter

2

2009

Gait Analysis for Human Identification (GAHI)



25

2.
1:
Overview

The advanced video surveillance system needs to analyze the
behaviors of people in order to prevent the occurrence of the

potential dangerous case. The analysis of behaviors of people
requires the human detection and tracking system.


The main aim of Human Detection and Tracking in this project
was to extract a good quality Human(s) silhouette image and track
this silhouette
(s), from a video frame. This was needed in order to
perform the extraction of the gait feature from the walking sequence.





















2009

Gait Analysis for Human Identification (GAHI)



26

2.
2
:
Human

Detection and Tracking

Block
Diagram




















Video Sequence

Object Tracking


Foreground Detection
and Extraction


Foreground
Blobs

Labeled
Object

Blobs


Object Classification



Labeled
Human Blobs

Feature Extraction


Fig 2.1: Human
Detection

and Tracking

Block
Diagram

2009

Gait Analysis for Human Identification (GAHI)



27

2
.
3
:
Moving Object Detection

The main aim of image segmentation in this project was to
extract a good quality binary foreground image, from a video frame
.


2.
3
.1
:
Types of Background Model
s

In recent years, the development of human detection and
tracking system has been going forwards for several years; many real
time systems have been developed. However, there are still some
challenging technologies need more researches: foreground
segmentat
ion and false alarm elimination. From the view of the state
of arts, there is no perfect algorithm for foreground segmentation to
be adaptive to tough situations such as heavy shadow, sudden light
change, tree shaking
and
so on. Most systems of human detec
tion and
tracking work fine in the environment with gradual light change,
however, they fail to deal with the situation with sudden light change,
tree shaking and moving background

there
are
types of background
model
.




Background
Subtraction Model

Background subtraction method initializes a reference
background with the first few frames of video input. Then it subtracts
the intensity value of each pixel in the current image from the
corresponding value in the reference background image. The
differen
ce is filtered with an adaptive threshold per pixel

to account
for frequently changing noisy pixels. The reference background
image and the threshold values are updated with an IIR filter to adapt
to dynamic scene changes.


Let


(

)

represent the gray
-
level intensity value at pixel
position (x) and at time instance n of video image sequence I which is
in the range [0, 255]. Let


(

)

is

the corresponding background
intensity value for pixel position (x) estimated

over time from vide
o
images




through




. As the generic background subtraction
scheme suggests, a pixel at position (x) in the current video image
belongs to foreground if it satisfies:


|


(

)




(

)
|





(

)











(



)

2009

Gait Analysis for Human Identification (GAHI)



28


Where



(

)

an adaptive threshold value

is

estimated using the
image sequence




through




. The Equation 2
.1 is used to generate
the foreground pixel map

which represents the

foreground regions
as a binary array where a 1 corresponds

to a foreground pixel and a 0
stands for a background
pixel.


The reference background


(

)

is initialized with the first
video
image


,




=


, and

the threshold image is initialized with
some pre
-
determined value

(e.g. 15).


Since our system will be used in outdoor environments as well
as indoor
environments,

the background model needs to adapt itself
to the dynamic changes

such as global illumination change (day night
transition) and long term background

update (parking a car in front
of a building). Therefore the reference

background and thresho
ld
images are dynamically updated with incoming images.

The update
scheme is

different for pixel positions which are detected as

belonging to
foreground
(



)

and which are detected as part of the
background
(



)
:






(

)

{


(

)

(



)


(

)






(

)

(



)


(

)















(



)






(

)

{


(

)

(



)
(





(


(

)



(

)
)








(

)






(



)



Where

α
,
β

(


[0.0, 1.0]) are learning constants which specify
how much information

from the incoming image is
put to the
background and threshold images.

In other words, if each
background pixel is considered as a time series, the background

image is a weighted local temporal average of the incoming image
sequence

and the threshold image is a weighted local tempor
al
average
of times

the difference

of incoming images and the
background. The values for
α, β

and

are

experimentally determined
by examining several indoor and outdoor video clips.


The subtle point in this update is choosing the correct value for
β
. If it

is too

small, foreground objects will be merged to the reference
background soon and

it will lead to inaccurate segmentation in later
2009

Gait Analysis for Human Identification (GAHI)



29

frames. Also, detecting stopped

objects will not be possible. If it is too
big, objects may never be diffused into the

ba
ckground image, thus
background model would not adapt to long
-
term scene

changes. In
the extreme case where
β

= 1.
0
.


Sample

foreground region de
tection is shown in Figure 2.2
The
first image

is the estimated reference background of the monitored
site. The
second image is

captured at a later step and contains two
foreground objects (two people). The

third image shows the detected
foreground pixel map using background subtraction.






Temporal

Model

Temporal differencing makes use of the pixel
-
wise difference
between two or three consecutive frames in video imagery to extract
moving regions. It is a highly adaptive appr
oach to dynamic scene
changes; however, it fails in extracting all relevant pixels of a
foreground object especially when the object has uniform texture or
moves slowly. When a

foreground object stops moving, temporal
differencing method fails in detecting

a change between consecutive
frames and loses

the object. Special supportive algorithms are
required to detect stopped objects


Let



(

)

represents

the gray
-
level intensity value at pixel
position (
x
) and at time instance
n
of video image sequence
I
which is
in the range [0, 255]. The two frame
s

temporal differencing scheme
suggests that a pixel is moving if it satisfies the following:


|


(

)




(

)
|





(

)











(



)


Hence, if an object has uniform colored regions, the Equation
2.4 fails

to detect some of the pixels inside these regions even if the
object moves. The per
-
pixel threshold,
T
, is initially set to a pre
-
determined value and later updated as follows:

(a)

(b)

(c)

Fig 2.2:

Adaptive Bac
kground Subtraction sample.

(a)
Estimated background
,

(b) Current image
,

(c) Detected region
.

2009

Gait Analysis for Human Identification (GAHI)



30






(

)

{


(

)

(



)
(





(


(

)



(

)
)








(

)






(



)


The implementation of two
-
frame
s

differencing can be
accomplished by exploiting the background subtraction method’s
model update parameters shown in Equation
2.5
. If
α

and
β

are set to
zero, the background holds the image




and background
subtraction scheme becomes identical to two
-
frame
s

differencing.




Gaussian Model

In this
method
, we use the running

average as background

model, and use the equation 2.
6

and 2.
7

to update in order to adapt to
gradual light

change.












(



)












(



)











(



)
|










|






(



)

Where



is a running average,



is a standard deviation,


is
pixel
value, α

is updating rate in the
nth
frame.



In order to filter out some noises caused by camera movement,
water wave and tree leaf shaking,

we have to figure out the new way
to create
the difference image between the current image and the
background image because the traditional way is not successf
ul for
dealing with such situations.
With

the following steps, we will get the
robust difference image:


We assume


is a

pixel in background image,




,




,




,




are its neighbors on the vertical and horizontal directions.



Is

the
responding pixel of




in current image,
and



,





are its neighbors
on the vertical direction. Firstly, we compute the intensity histogram
of pixels

in the window
r
×
r
centered
by


,

and then find the
intensity value



with the maximum possibility in the window
r
×
r.
In our system,
r
=
7.
Secondly, we calculate the median value


̂

of
intensity values
of




,




,





; and calculate the mean value


̅
̅
̅

of
intensity values of




,




,




,




.


Finally, the difference value



can be computed according to
the equation

2.
8

based on assumption that water wave and tree
shaking are the movement of the part of background.

2009

Gait Analysis for Human Identification (GAHI)



31








̂



|

|


̂



̅
̅
̅
|

|


̂




|
)




(



)


Where |
a
| is the function of computing the absolute value of
a,





is the intensity value of


. In order to create the foreground mask
image, we assume



is the value of corresponding pixel of




in
the foreground mask image.


And




are threshold values we
used. We define



is greater than



.


In order to filter out the shadow that is cast by people moving
in the scene, we use the following

rule to threshold the difference
image and get the foreground mask image:


If



<




, then




; if



>=



, then




; if



is between



and



, we will check whether



is shadow or
not.


If



is shadow,





, else



=1. The selection of



is the key for successful threshold of the difference image. If



is
too low, some background are labeled into foreground.

If



is too high, some foreground are detected as

background.


The traditional way is to select



as



based on the
assumption that illumination gradually changes. However when light
suddenly changes, this assumption will be violated. Instead of the
traditional way, we have to automatically select



according to the
level of light change.


Firstly, we

search the median value
MID
of difference ima
ge.
After that, the equation 2.
9

is used to compute



for each pixel.
















(



)


Where


is initial threshold, normally




. In our
system, we select
TD
=5. The




can be selected as







where
Gat
is a gate. In our system,
Gat
is 50. In order to adapt the
sudden light change, we set

different α to update the background for
different level of light change. The rule for the selection of updating
rate α is:


2009

Gait Analysis for Human Identification (GAHI)



32



{





















































































Where


<




.In our system, we use







,





;






,




;






.





















In
GAHI

System we implement all of the pervious algorithms
finally we select
Gaussian model
to
be our method for background
model in moving object detection as this method can handle most of
tough situations such as sudden light change, heavy shadow, the
objects in background remove etc.


2.
3.
2
:
Shadow Removal based on Gaussian

Model

(
2.10
)



A)
Origin
Image

C)
Origin
Image

B) Segmented

Image

D) Segmented

Image

Sudden Illumination

Fig 2.3:
Object Segmentation

Block
Diagram

2009

Gait Analysis for Human Identification (GAHI)



33

The shadow
affects the performance of foreground

detection.
The regions of shadow will be detected as

foreground. Recognizing
shadow is hard work, and

now this subject is still under research.
There is no

perfect algorithm for shadow detection.


The assumption for th
e shadow detection is that the regions of
shadow are semi
-
transparent. An area cast into shadow often results
in a significant change in intensity without much change in
chromaticity. S.J. McKenna proposed a method to detect shadow
based
on RGB color space
. In order to utilize the color invariant
feature of shadow, we use the normalized color components in YUV
color space, which are defined as





=
U
|
Y,



=
V|
Y.

Our shadow detection algo
rithm is described as following:


Step1: Compute the color
difference.


We
assume
b




,
b




is

the normalized color

components of





,





,






is

the normalized

color components of





. The color difference is

defined as equation
.





|









|

|










|



(



)



Step2:
Compute the texture difference.


We assume





is the intensity value of



in

background
image,







,






,








,








are the

intensity values of pixels of its
neighbors






,






,






,






,

on the vertical and
horizontal

direction
s
.





is the intensity value of



pixel in current image,

and
are the intensity values of

pixels of its neighbors
.



,

.



,

.



,





on
the

vertical and horizontal direction. The texture

diffe
rence is defined
as
equation
.





|


(
|







|
)

(
|







|
)
|























(



)


Where
Th
(
Val
) is a

function defined as equation
.




(

)
{





























































(



)



2009

Gait Analysis for Human Identification (GAHI)



34

Step 3: Shadow
decision


If





and






and






, then




is shadow,
otherwise




is not shadow.

Where
(

)
is the color
threshold. The
assumption

for







is that the region of shadow is always

dark
than background.



2.
3
.
3
:
Connected Component Labeling

Connected component labeling is an algorithmic application of
graph theory, where subsets of
connected components

are uniquely
labeled

based on a given heuristic. Connected component labeling is
not to be confused with segmentation.

Connected component labeling is used in computer vision to
detect unconnected regions in binary digital images, although color
images and data with higher
-
di
mensionality can also be processed.
When integrated into an image recognition system or human
-
computer interaction interface, connected component labeling can
operate on a variety of information.

A graph, containing vertices and connecting edges, is
constr
ucted from relevant input data. The vertices contain
information required by the comparison heuristic, while the edges
indicate connected 'neighbors'. An algorithm traverses the graph,
labeling the vertices based on the connectivity and relative values of
their neighbors. Connectivity is determined by the medium; image
graphs, for example, can be 4
-
connected or 8
-
connected. Following
the labeling stage, the graph may be partitioned into subsets, after
which the original information can be recovered and proc
essed.

2009

Gait Analysis for Human Identification (GAHI)



35







The algorithms discussed can be generalized to arbitrary
dimensions,

(?)
with increased time and space complexity.



Two

Pass

Algorithm

Relatively simple
to implement and understand, the two
-
pass
algorithm iterates through 2
-
dimensional, binary data. The algorithm
makes two passes over the image: one pass to record equivalences
and assign temporary labels and the second to replace each
temporary label by th
e label of its equivalence class.

The input data can be modified in

(?)
(which carries the risk of
data corruption), or labeling information can be maintained in an
additional data structure.

1.

On the first pass



Iterate through each element of the data by
column, then by row



If the element is not the background

o

Get the neighboring elements of the
current element

o

If there are no neighbors, uniquely
label the current element and continue

(a)

(
b
)

Fig 2.
4
:

Connectivity types

(a) 4
-
connected, (b) 8
-
connected.

2009

Gait Analysis for Human Identification (GAHI)



36

o

Otherwise, find the neigh
bor with the
smallest label and assign it to the
current element

o

Store the equivalence between
neighboring labels


2.

On the second pass



Iterate through each element of the data
by column, then by row



If the element is not the background

o

Reliable the
element with the lowest
equivalent label

Here, the
background

is a classification, specific to the data,
used to distinguish salient elements from the
foreground
. If the
background variable is omitted, then the two
-
pass algorithm will
treat the background
as another region.







Fig 2.
5:
Sample graphical output from running the two
-
pass algorithm on a binary
image. The first image is unprocessed; while the last one has been recolor with label
information. Darker hues indicate the neighbors of the pixel be
ing processed.

2009

Gait Analysis for Human Identification (GAHI)



37


2.
3.4
:
Moving Object
Detection
Class Diagram





MovingObjectdetection Class:



Member:



_mea
nImage




_medianImage




_intensityHistogram



_diffImage



_alpha



double

_TD



double

_Gat


Fig 2.6: Moving Object

Detection

Class

diagram

2009

Gait Analysis for Human Identification (GAHI)



38




double _MID




_segma




_mu




_YUVbackground




_YUVcurrentImage




_grayBackground




_grayCurrent



Functions



GenerateBinaryImage: segment moving object from
back
ground current frame



GetMn
:

intensity value Mn



GetBnMean
:

get mean of each pixel in background
image



GetPnMedian
:

get median value for each pixel in
current image



GetMask forground mask



GetRunningAvg
:

running
average



GetStandardDeviation
:

Standard Deviation



GetTH1
:

get value of threshold 1



GetTH2
:

get
value of threshold 2



GetAlpha
:

get value of Alpha



GetMID
:

median value MID of difference image



Shadow_Remove
:
check

if this pixel is shadow or not



Threshold1
:

shadow threshold



Connectedcomponent class:



Members



_nextLabel



components



Functions



Find_ConnectedComponent: find all connected
component
s

in image



ExtractImage: copy part of Gray image base on top,
lower,right left points and part label



FirstPath: Do first path of connected component algo.
start from point (0,0)



SecondPath: Do second pa
th of connected component
algo. start from point (n,n)



Find8Neighbours: find the label of neighbour pixel

2009

Gait Analysis for Human Identification (GAHI)



39



FilleachObjectPar: add connected com
p
onent to
component
list that contain info about each object not
add silhou
e
tte in this fn



Componentdata class:

This class is
a data class which contains

all required
information of each object blob
.

2
.
4
: Object Tracking

In order to

add facility to identify multi human at the same
time (move at same time in the video
) we

need to

track the
individuals and

the human model has to be created for each
individual.


The good human model should be invariant to translation and
changes in scale, and should be robust to partial occlusion,
deformation and light change
.








2.
4
.
1
:
Human
Tracking Types




Appearance
-
based Tracking

In this method use

the color histogram
,

the number of pixels
and size as the human model to describe the humans. In order to
decrease the computation cost, we define the

color of pixel as
equation
.



























(



)


Where



,




,




are the Y,U,V value
s

of pixel in

current image, and

i
n

is the color value we use to

compute the histogram. We
assume



and




are

the current histogram and reference histogram

respectively
, and

then the comparison rule of histogram is defined as

equation
.


2009

Gait Analysis for Human Identification (GAHI)



40







(


(

)



(

)






(







)





(



)





T
racking based on direction and position

In
GAHI

system we use this approach
because the
appearance
-
based
tracking has a lot of problems when all
objects have colors similar to each other.

As the following we will describe how to track the objects:



for all objects in the image

o

compare each object center with all references

o

then get the minimum one

o

if the
minimum greater than 10 pixel
s
,

then create
new object

o

else the object get the label of this reference



if new object created far from the image boundary

o

then it means occlusion happened

o

then the nearest two references in different
direction will swapped

2009

Gait Analysis for Human Identification (GAHI)



41











X1

/X1

0

1

X1

X2

/X2

/X1

Ref 0

Ref 1

0

1

Compare with
references

2009

Gait Analysis for Human Identification (GAHI)



42


Fig 2.
7
: Tracking Sequences











X1

X4

/X4

/X1

Ref 0

Ref 1

1

0

Occlusion finished

X1

X3

/X3

/X1

Ref 1

Ref 0

Occlusion

Update References as show when occlusion happened

2009

Gait Analysis for Human Identification (GAHI)



43

2.
4
.2
:
Human

Tracking Limitations




No more
than

one occlusion happen

at
the
same time



minimum distance between two occlusion
s

mus
t

be more than
one

meter































2009

Gait Analysis for Human Identification (GAHI)



44

2.
4.
3
:
Human Tracking
Class Diagram





Tracking Class:



Members



_reference




_referenceBlackList




_blackRightArea



_blackleftArea



_labels



functions



Track: Give each object in
Image Special Label



CopyImage: copy part of image base on mask



AddNewObject: Add new Ref object to refs

This class contains

the implantation of two approaches



MaskData Class:

Contain all information of new object in the image

Fig 2.8: Human Tracking Class diagram

2009

Gait Analysis for Human Identification (GAHI)



45



ReferenceHuman Class:

Contain all information about the references






2
.
5
: Object
Classification

The aim of different smart visual surveillance applications is to
extract semantics from video to be used in higher level activity
analysis tasks. Categorizing the type of a dete
cted video object is a
crucial step in achieving this goal.


With the help of object type information, more specific and
accurate methods can be developed to recognize higher level actions
of video objects. Hence, we developed video object classification
m
ethod based on object shape similarity as part of our visual
surveillance system.


Typical video scenes may contain a variety of objects such as
people, vehicles, animals, natural phenomenon (e.g. rain, snow),
plants and clutter. However, main target of interest in surveillance
applications are generally humans only.


Also, real time na
ture and operating environments of visual
surveillance applications require a classification scheme which is
computationally inexpensive, reasonably effective on small targets
and invariant to lighting conditions.


We have satisfied most of these
requirements by implementing
a classification scheme which is able to categorize detected video
objects into humans by using image
-
based object features.






2009

Gait Analysis for Human Identification (GAHI)



46











2.
5
.1
:
Object

Classification Types


Moving regions detected in video may correspond to
different
objects in real
-
world such as pedestrians, vehicles, clutter, etc. It is
very important to recognize the type of a detected object in order to
track it reliably and analyze its activities correctly. Currently, there
are two major approaches towar
ds moving object classification
which are shape
-
based and motion
-
based methods. Shape
-
based
methods make use of the object’s 2D spatial information whereas
motion
-
based methods use temporally tracked features of objects for
the classification solution.




Shape
-
based Classification


Saptharishi et al
. propose a classification scheme which uses a
logistic linear neural network trained with Differential Learning to
recognize two classes: vehicle and people [3].

Papageorgiou et al
. presents a method that make
s use of the
Support Vector Machine classification trained by wavelet
transformed object features (edges) in video images from a sample
pedestrian database [4]. This method is used to recognize moving
regions that correspond to humans.


Another classificat
ion method proposed by
Brodsky et al
. [5]
uses a Radial Basis Function (RBF) classifier which has a similar
architecture like a three
-
layer back
-
propagation network. The input
to the classifier is the normalized gradient image of the detected
object region
s.




Motion
-
based Classification


2009

Gait Analysis for Human Identification (GAHI)



47

Some of the methods in the literature use only temporal motion
features of objects in order to recognize their classes [6, 7,
and 8
]. In
general, they are used to distinguish non
-
rigid objects (e.g. human)
from rigid
objects (e.g. vehicles).


The method proposed in [6] is based on the temporal self
-
similarity of a moving object. As an object that exhibits periodic
motion evolves, its self
-
similarity measure also shows a periodic
motion. The method exploits this clue to

categorize moving objects
using periodicity.


Optical flow analysis is also useful to distinguish rigid and non
-
rigid objects.
A. J. Lipton

proposed a method that makes use of the
local optical flow analysis of the detected object regions [8]. It is
expected that non
-
rigid objects such as humans will present high
average residual flow whereas rigid objects such as vehicles will
present little

residual flow. Also, the residual flow generated by
human motion will have a periodicity. By using this cue, human
motion, thus humans, can be distinguished from other objects such as
vehicles.



In
GAHI

system we use Shape
-
Based Approach to i
mplement
obj
ect classification
u
s
ing
Jianpeng Zhou and Jack Hoang

Algorithm's
based on
codebook
theory

which
will be explained in the next
section.

















2009

Gait Analysis for Human Identification (GAHI)



48









2.
5
.2
:
Classification
Algorithm based on
Codebook

Theory


First step
, we normalize
the size of object to 20 by 40, and then
extract the shape of object as the features.




(a)



(b)

Fig
2.
9
:

Normalize
Image

(a) Current image (b) Normalized image (20 by 40).


Second step
, we match the feature vector with the code vectors
of codebook.


The

match

process

is to find a code vector in codebook with
the minimum
distortion
to the feature vector of object. If the
minimum distortion is less than a threshold, this object is human.


In order to describe the procedure of classification based on
codebook. We assume



is the




code vector in code book.
X

is
feature vector of object.
N

is the number of code vectors in code book.
The dimension of code vector is
M
. So the distortion between



and
X

is computed as
equation (
2.16
):














|






|





(
2.
16
)

The minimum distortion between
X
and the code vectors in the code
book is defined as equation (2
.17
):

2009

Gait Analysis for Human Identification (GAHI)



49





(


)











(
2.1
7
)

If
diss
is less than a threshold, the object with the feature vector
X

is
human;

otherwise, it is not human.





(a)



(b)


F
ig 2
.
10
:
(a) Mask image of human
,

(b) Boundary of human body.


In order to create the shape vector of object, the mask image
and boundary of human body are created by the previous steps

(
Fig

2
.
10
). We use the distance from the boundary of human body to
the left side of bounding box as
a
feature vector.