for Lossy Compression Impairment

companyscourgeAI and Robotics

Oct 19, 2013 (3 years and 8 months ago)

58 views

Maria Grazia Albanesi, Riccardo Amadeo

University of
Pavia, Faculty
of
Engineering
,
Computer
Department

Impact of Fixation Time on Subjective
Video Quality Metric: a New Proposal
for Lossy Compression Impairment
Assessment


ICMVIPPA 2011 : International Conference on Machine
Vision, Image Processing, and Pattern
Analysis

Venezia (Mestre),
November

28, 2011



The addressed problem:


subjective video quality assessment for lossy compression
impairment


The tools and the
experiments


eye tracking and subjective experiments


The goals


Comparison to literature


The results and their interpretation



A possible application: a new protocol for no
-
reference video quality assessment


Future developments


Outline

2


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011

How I can
measure

the
loss

of
quality

due to
compression
?


Field of applications: TV, video services on Internet,
video for mobile applications, test of emerging
compression algorithms…..

Evaluation of multimedia quality user experience


Two approaches: objective and
subjective

metrics



Our goal: find
objective

parameters coming form
subjective experiments which reflect the
subjective

video quality, as perceived by a human observer
.

The
problem

3


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011


Eye tracker: it records the point and the duration of
fixation of the eye, when the observer looks at a
monitor.












Data are subsequently analyzed from a statistical point
of view (mean, std.
dev
….)

The tools: eye tracker and subjective QA
experiments

4


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011


The set of
videos

5


A set of 19 videos downloaded
from available online
public libraries


http
://trace.eas.asu.edu/yuv/ (Video trace library of Arizona
State University)


ftp://ftp.tnt.uni
-
hannover.de/pub/svc/testsequences/
(Hannover
Liebnitz

University video library)


http://media.xiph.org/video/derf/



The
original
files: YUV
sequences, 4:2:0, in CIF
resolution (352x288) at 30
fps are converted in
avi

sequences and compressed by a H.264 at 2 bitrates:
450 bps and 150 bps


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011

Examples
:

6


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011

Visual
behavior

and
impairment

7

Visual
path

for
original

«best»
video

Visual
path

for
compressed

(br150 bps) video


Protocol

ACR5
-
HR (
absolutely

category

ranking


hidden

reference


MOS scale with
five

levels
:


Only

one

observation

for
each

video


The
observer

has

no information
about

the
unimpaired

version

of the video.


The
subjects: 8
females and 10 males, of age varying
from 22 to 27 years old
.



Their vision was normal or corrected
-
to
-
normal


T
hey
had no experience in subjective video quality
assessment.


They had
normal or good experience in using IT interfaces to
watch videos both online and offline.


Methodology

8


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011


Our parameter are not related to fixation points, but to
the duration of the fixation.



Videos are classified according to color content
relevance and
and

movement relevance to create
semantic filters



Parameters:


Duration of fixation time


MOS, five point scale


Subjective Color Score,
three point scale


Subjective Movement Score (SCS
e
SMS)
, three point scale.



Removal of «Memory effect» due to the conditioning of ocular
motion activity by the visual attention of preceding scenes.




Novelties

and
comparison

to
literature

9

Starting

point
:
O. Le
Meur
, A.
Ninassi
, P. Le
Callet
, D.
Barba
, Overt visual attention for free
-
viewing and quality
assessment tasks: Impact of the regions of interest on a video quality metric, Signal Processing Image
Communication, 2010, vo. 25,
pp
-

547
-
548.



18 tester, 6 in
each

playlist


Each

video
has

three

version
:
reference

br450 br150 (57
videos
)


Each

observer

looks

at

only

one

version

of
each

video. No
repetitions

are
allowed

in
each

playlists
.



Numero video utilizzati 19

N.

ID

Playlist A

Playlist B

Playlist C



1

Foreman

ref

Br150

Br450

2

Silent

Br150

ref

Br450

3

Flower

Br450

Br150

ref

4

Bus

Br450

ref

Br150

5

Tempete

ref

Br450

Br150

6

Bridge_close

ref

Br150

Br450

7

Ice

Br150

ref

Br450

8

Coastguard

Br450

ref

Br150

9

Mother_daug

Br150

Br450

ref

10

Football

ref

Br150

Br450

11

Crew

Br450

ref

Br150

12

Paris

Br450

Br150

ref

13

Container

ref

Br450

Br150

14

Highway

Br150

Br450

ref

15

Waterfall

Br150

ref

Br450

16

Hall

Br450

Br150

ref

17

Stefan

ref

Br450

Br150

18

News

ref

Br150

Br450

19

Mobile

Br150

Br450

ref











tot:

ref

7

6

6

Br450

6

6

7

Br150

6

7

6

Playlists

to
remove

memory

effect

10


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011

The MOS
really

reflect

the progressive
loss

of
quality

due to
compression
.









Mean Opinion Score

0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
4.50
5.00
Foreman
Silent
Flower
Bus
Tempete
Bridge_close
Ice
Coastguard
Mother_daughter
Football
Crew
Paris
Container
Highway
Waterfall
Hall
Stefan
News
Mobile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
MOS

ACR5
-
HR

MOS

Ref
Br 450
Br 150
11


Color and
movement

are
considered

relevant

if

the score
is

> 2



«Highly
animated

video»: 2
, 4, 7, 8, 10, 12,
14, 17,
19


«Highly coloured
video» »:
3, 5, 7, 10, 15,
19

N.

Playlist

SCS

SMS









1

Foreman

1,67

1,50

2

Silent

1,28

2,28

3

Flower

2,33

1,61

4

Bus

1,67

2,33

5

Tempete

2,50

1,89

6

Bridge_close

1,44

1,50

7

Ice

2,11

2,33

8

Coastguard

1,72

2,33

9

Mother_daughter

1,78

1,28

10

Football

2,39

2,78

11

Crew

1,83

1,94

12

Paris

1,94

2,11

13

Container

1,83

1,67

14

Highway

1,39

2,56

15

Waterfall

2,72

1,72

16

Hall

1,72

1,72

17

Stefan

1,67

2,50

18

News

1,78

1,83

19

Mobile

2,56

2,22

SCS

e
SMS

12


The
mean

fixation

time
does

not

seem

to be
related

to the video
quality
!

Analysis of
Mean

fixation

time (MFT)

200
250
300
350
400
450
500
550
600
Foreman
Silent
Flower
Bus
Tempete
Bridge_close
Ice
Coastguard
Mother_daughter
Football
Crew
Paris
Container
Highway
Waterfall
Hall
Stefan
News
Mobile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ms

Durata media
fissazioni,
MFT

Ref
Br450
Br150
13

MFT
,
semantic

filtering

200
250
300
350
400
450
500
550
600
Silent
Bus
Ice
Coastguard
Football
Paris
Highway
Stefan
Mobile
2
4
7
8
10
12
14
17
19
ms

MFT
,
SMS
>2

Ref
Br 450
Br 150
200
250
300
350
400
450
500
550
600
Flower
Tempete
Ice
Football
Waterfall
Mobile
3
5
7
10
15
19
ms

MFT
,
SCS
>2

Ref
Br 450
Br 150

Even

by
filtering

by
movement

or
colour
,
there

is

not

a
clear

relation
between

MFT

and
MOS

14


Even

the standard
deviation

of
fixation

time
does

not

seem

to be
related

to the video
quality
!


Analysis of Standard
deviation

of
FT

0
50
100
150
200
250
300
350
400
450
Foreman
Silent
Flower
Bus
Tempete
Bridge_close
Ice
Coastguard
Mother_daughter
Football
Crew
Paris
Container
Highway
Waterfall
Hall
Stefan
News
Mobile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ms

Deviazione Standard,
SDoFT

Ref
Br450
Br150
15



Ref

Br 450

Br 150

Average

SDoFT
, SMS>2

155,4752

170,6233

225,5709

Standard
deviation

of
SDoFT
, SMS>2

69,7361

85,7656

100,3062

Ref

Br 450

Br 150

Average

SDoFT

149,3888

174,2289

173,8378

Standard
deviation

of
SDoFT

60,6719

88,2987

90,7023

The
solution
:
third

order

statistics
!



Ref

Br 450

Br 150

Average

SDoFT
, SCS>2

145,4335

169,0900

178,9579

Standard
deviation

of
SDoFT
, SCS>2

69,4838

122,8213

87,1634


Standard
deviation

of FT
is

on the
average

less

for
videos

of high
quality



The
semantic

filtering

shows
that

this

behaviour

is

stressened

for
highly

animated

videos
.

16


The

duration

of

fixation

time

seems

to

have

a

more

predictable

behavior

when

the

observer

watches

to

a

high

quality

video
.


If

we

compute

the

third

order

statistics

on

the

fixation

time,

we

can

guess

a

rank

of

a

collection

of

video

which

reflects

the

perceptive

visual

quality


The

experiments

confirm

this

behavior

for

degradation

due

to

lossy

compression
.


The

rank

according

third

order

statistic

reflect

the

loss

of

quality

and

subjective

MOS

especially

for

highly

animated

videos
.


Future

researches
:



Test

on

a

greater

level

of

quality

impairments


Test

on

other

kinds

of

quality

impairments


Finding

a

more

efficient

semantic

filtering

about

color

or

other

criteria
.

Conclusions and future researches

17


ICMVIPPA
2011
-

Venezia (Mestre)
-

November

28, 2011