Intelligent Systems and Robotics Research Group (ISR) Creative Technologies University of Portsmouth

odecrackAI and Robotics

Oct 29, 2013 (3 years and 8 months ago)

84 views

Geoffrey Samuel


PhD Researcher

Intelligent Systems and Robotics Research Group (ISR)

Creative Technologies

University of Portsmouth

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Intro


Background subtraction is a important
and vital step for computers to
understand and interpreter a real
-
world
scene



It allows a computer to ignore a
background so to concentrate on a
foreground object

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Hypothesis


Each background subtraction algorithm
will have its advantages and
disadvantages, and that looking and
comparing these with a real
-
world
situation, it would be possible to pick
one algorithm or a method of combining
algorithms to produce a algorithm
capable of balancing speed with quality.


Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

The Goal


Test and evaluate the quality and speed
of existing background subtraction
algorithms on a complex background
with different everyday motions, and to
compare the results with those of the
extracted “Ground Truth”


Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Complex Background

Static Background:
-

Background does not contain any
secondary “unwanted” motion.
Controlled environment.

Complex Background:
-

Background contains secondary
“unwanted” motion such as the winds
effect on trees or blinds.

Real
-
world data.

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Synthetic Test Data

Advantages:


Automatically got the “Ground Truth”.


More control over each test clip.



Disadvantages:


Manual frame by frame “Ground Truth”
extraction.


Added artefacts from the Chroma keying
and compositing.

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

The Experiment


To Create a set of synthetic data with
the “Ground Truth”



To test different motions with each
background subtraction algorithm



To Compare the results of each
algorithm with that of the “Ground Truth”

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

The Motions


7 everyday motions were chosen:


Drinking


Jogging


Picking up wallet


Scratching head


Sitting down


Standing up


Walking



Each motion started on the left of the
screen and concluded on the right.

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Creating the test scenarios

Green Screen

Back Ground

Green Screen with actor

Final Composite

“Ground Truth”

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Back Plate Difference



│frame
i



backplate│>T
s

The Algorithms

50

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Frame Difference



│frame
i



frame
i
-
1
│>T
s

The Algorithms

50

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Approximate median



(x = (
frame
i

-

frame
i
-
1



frame
i
-
2
. . .
frame
i
-
n
) > T
s
)

→ {background += 1}

→ {background
-
= 1}

The Algorithms

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Mixture of Gaussians




frame
(i
t
=
μ
) =
Σ
i=1

ω
i,t
.
ț(
μ
,o)

The Algorithms

k

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Measuring the Quality

Compare the Per
-
Pixel value of

each frame with the “Ground Truth”

(0,0)

(768,0)

(768,576)

(0,576)

(0,0)

(768,0)

(768,576)

(0,576)

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Results
-

Quality

Test Motions

Backplate

Difference

Frame Difference

Approximate Median

Mixture of Gaussian

% of image

#

of pixels

% of image

#

of pixels

% of image

#

of pixels

% of image

#

of pixels

Drinking

90.78%

401577.3019

82.12%

363282.5031

89.52%

396024.7107

83.78%

370625.2327

Jogging

88.24%

390349.3529

88.88%

393194.9412

92.14%

407602.3824

88.20%

390146.7941

Picking up Wallet

91.26%

403717.114

88.22%

390256.9035

83.40%

368940.5088

90.19%

398979.9737

Scratch head

88.18%

390065.7255

84.87%

375422.2549

90.56%

400599.9216

86.15%

381117.049

Sitting down

88.51%

391528.6796

80.07%

354204.932

82.28%

363994.2039

81.68%

361327.3981

Standing up

89.40%

395491.6311

83.82%

370787.165

80.99%

358290.4563

83.78%

370631.6893

Walking

88.47%

391373.5094

89.81%

397309.3396

94.22%

416820.1321

90.01%

398195.3396

Most correct pixels

Most incorrect pixels

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Results
-

Quality

70.00%
75.00%
80.00%
85.00%
90.00%
95.00%
100.00%
Drinking
Jogging
Picking up
Wallet
Scratch head
Sitting down
Standing up
Walking
Average Percent of
correctly
identified pixels per frame

Test Motions

Percent of correctly identified pixels

Backplate Difference
Frame Difference
Approximate Median
Mixture of Gaussian
Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Results
-

Speed

Test Motions

Backplate

Difference

(Average of 100 times)

Frame Difference

(Average of 100 times)

Approximate
Median

(Average of 100 times)

Mixture of Gaussian

Drinking

0.0507

0.0004

0.3301

10.6954

Jogging

0.0507

0.0025

0.0691

10.8219

Picking up Wallet

0.0492

0.0819

0.0730

12.2895

Scratch head

0.0450

0.0850

0.0718

10.6132

Sitting down

0.0420

0.0692

0.0662

10.8503

Standing up

0.0416

0.0747

0.0529

12.7196

Walking

0.0319

0.0129

0.0541

10.5202

“Fastest” Algorithm

“Slowest “Algorithm

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Results
-

Speed

Drinking
Jogging
Picking up
Wallet
Scratch head
Sitting down
Standing up
Walking
0.0000
2.0000
4.0000
6.0000
8.0000
10.0000
12.0000
14.0000
Test Motions

Average processing time per
frame in Seconds
(run 100 times)

Average time to process per frame

Backplate Difference
Frame Difference
Approximate Median
Mixture of Gaussian
Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Results
-

Speed

Drinking
Jogging
Picking up
Wallet
Scratch head
Sitting down
Standing up
Walking
0.0000
0.0500
0.1000
0.1500
0.2000
0.2500
0.3000
0.3500
Test Motions

Average processing time per
frame in Seconds (run
100 times)

Average time to process per frame

Backplate Difference
Frame Difference
Approximate Median
...now ignoring the Mixture of Gaussian speed results

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Conclusion


Backplate

difference was the fastest and
produce the highest results in 4 out of 7
tests.



Frame difference was the ONLY
algorithm to correctly remove the
complex background, but couldn't
correctly identify the foreground
element.

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Conclusion

Frame Difference :
-


Correctly Removed Complex Background


Incorrectly Removed inside of Subject

Backplate

Difference :
-


Correctly Identified Subject


Incorrectly kept Complex Background

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Taking it further

A new method that incorporated both the

speed of updating to remove the

background and yet the knowledge of the

background to properly extract the wanted

foreground element.

Theory Framework
idea
:

Frame Difference

Backplate

Difference

ƒ

Complex background removed

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Where can this lead?


Application of this technology could be
used in:



Games


Surveillance


Mesh reconstruction and silhouette
extraction


Various computer vision tasks


Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Any Questions?


Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Acknowledgments



UK Engineering and Physical Science
Research Council




Seth Benton for his Matlab code

Geoffrey Samuel

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Thank you for your time




Geoffrey.Samuel@Port.ac.uk



www.GeoffSamuel.com