Social Signal Processing Understanding Nonverbal ... - ICMS

bunkietalentedΤεχνίτη Νοημοσύνη και Ρομποτική

24 Νοε 2013 (πριν από 3 χρόνια και 4 μήνες)

68 εμφανίσεις

Social  Signal  Processing
Understanding  Nonverbal  Behavior
in  Human-‐‑‒Human  Interactions
A.Vinciarelli
University  of  Glasgow  and  Idiap  Research  Institute
http://www.dcs.gla.ac.uk/~∼vincia
e-‐‑‒mail:  
vincia@dcs.gla.ac.uk
1
Outline

“Socialism”
 and  Big  Data

Social  Signal  Processing

The  Conflict  Example

Conclusions
2
2
Outline

“Socialism”
 and  Big  Data

Social  Signal  Processing

The  Conflict  Example

Conclusions
3
3
4
“Socialism”  and  Big  Data
0
22.5
45
67.5
90
2004
2005
2006
2007
2008
2009
2010
2011
2012
Events showing the word “
social
” in their title (from
dbworld
, a multimedia retrieval mailing list)
4
4
“Socialism”  and  Big  Data
0
22.5
45
67.5
90
2004
2005
2006
2007
2008
2009
2010
2011
2012
Events showing the word “
social
” in their title (from
dbworld
, a multimedia retrieval mailing list)
4
Outline

“Socialism”
 and  Big  Data

Social  Signal  Processing

The  Conflict  Example

Conclusions
5
5
6
Social  Signals
Vinciarelli, Pantic and Bourlard, “
Social Signal Processing: Survey of an Emerging
Domain
”, Journal of Image and Vision Computing, 27(12):1743-1759, 2009
6
6
Social  Signals
Height
Forward
Posture
Forward
Posture
Gestures
Interpersonal
Distance
Nonverbal
Behavioral
Cues
Vocal
Behavior
Mutual
Gaze
Vinciarelli, Pantic and Bourlard, “
Social Signal Processing: Survey of an Emerging
Domain
”, Journal of Image and Vision Computing, 27(12):1743-1759, 2009
6
6
Social  Signals
Social
Signals
Height
Forward
Posture
Forward
Posture
Gestures
Interpersonal
Distance
Nonverbal
Behavioral
Cues
Vocal
Behavior
Mutual
Gaze
Vinciarelli, Pantic and Bourlard, “
Social Signal Processing: Survey of an Emerging
Domain
”, Journal of Image and Vision Computing, 27(12):1743-1759, 2009
6
Outline

“Socialism”
 and  Big  Data

Social  Signal  Processing

The  Conflict  Example

Conclusions
7
7
8
Conflict

[Conflict is a] mode of interaction [where]
the attainment of the goal by one party
precludes its attainment by the others.


C.M. Judd, “
Cognitive Effects of Attitude Conflict Resolution
”, Journal of
Conflict Resolution, 22(3):483-498, 1978.
8
9
Data
Samples
1430 clips from Canal9
Samples Length
30 seconds (11h:55m)
Subjects
138 (5 moderators)
Subjects per Sample
At least 2
Assessors
551 (10 per sample via MTurk)
Questionnaire
15 items
Total Items
214,500

Vinciarelli, Kim, Valente and Salamin, “
Automatic Detection of Conflicts in
SpokenConversations
”, Proc. of IEEE International Conference on Audio,
Speech and Signal Processing, pp. 1-4, 2012.
9
10
Measuring  Conflict

The atmosphere is relaxed

People wait for their turn before speaking

One or more people talk fast

One or more people fidget

People argue

One or more people raise their voice

One or more people shake their heads and nod

People show mutual respect

People interrupt one another

One or more people gesture with their hands

One or more people are aggressive

The ambience is tense

One or more people compete to talk

People are actively engaged

One or more people frown
10
11
Measuring  Conflict

The atmosphere is relaxed

People wait for their turn before speaking

One or more people talk fast

One or more people fidget

People argue

One or more people raise their voice

One or more people shake their heads and nod

People show mutual respect

People interrupt one another

One or more people gesture with their hands

One or more people are aggressive

The ambience is tense

One or more people compete to talk

People are actively engaged

One or more people frown


11
11
Measuring  Conflict

The atmosphere is relaxed

People wait for their turn before speaking

One or more people talk fast

One or more people fidget

People argue

One or more people raise their voice

One or more people shake their heads and nod

People show mutual respect

People interrupt one another

One or more people gesture with their hands

One or more people are aggressive

The ambience is tense

One or more people compete to talk

People are actively engaged

One or more people frown


11
11
Measuring  Conflict

The atmosphere is relaxed

People wait for their turn before speaking

One or more people talk fast

One or more people fidget

People argue

One or more people raise their voice

One or more people shake their heads and nod

People show mutual respect

People interrupt one another

One or more people gesture with their hands

One or more people are aggressive

The ambience is tense

One or more people compete to talk

People are actively engaged

One or more people frown


11
Measuring  Conflict

Kim et al.,“
Predicting the Conflict Level in Television Political Debates: an
Approach Based on Crowdsourcing, Nonverbal Communication and Gaussian
Processes

, Proc. of ACM MM, pp. 793-796, 2012
12
12
13
Speech  Analysis

Clip-based statistics (pitch and intensity)
13
14
Speech  Analysis

Clip-based statistics (pitch and intensity)

Turn-based statistics (pitch and intensity)
14
15
Speech  Analysis

Clip-based statistics (pitch and intensity)

Turn-based statistics (pitch and intensity)

Speaker activity statistics
15
16
Speech  Analysis

Clip-based statistics (prosody)

Turn-based statistics (prosody)

Speaker activity statistics (time budget)

Overlapping speech statistics (prosody and length)
16
17
Results
0.65
0.7
0.75
0.8
0.85
BLR
GPR RBF
GPR ARD
SVR LIN
SVR RBF
0.74
0.78
0.77
0.77
0.77
0.69
0.73
0.71
0.71
0.74
0.76
0.80
0.81
0.80
0.81
Correlation Actual / Predicted Conflict Level
Manual
Automatic
Automatic
w.o.s
17
Outline

“Socialism”
 and  Big  Data

Social  Signal  Processing

The  Conflict  Example

Conclusions
18
18
19
Conclusions
19
19
Conclusions

Nonverbal behaviour provides machine detectable
evidence of social and psychological phenomena
19
19
Conclusions

Nonverbal behaviour provides machine detectable
evidence of social and psychological phenomena

Big Data is an
opportunity
: SSP approaches are
data driven and can benefit from large corpora
19
19
Conclusions

Nonverbal behaviour provides machine detectable
evidence of social and psychological phenomena

Big Data is an
opportunity
: SSP approaches are
data driven and can benefit from large corpora

Big Data is a
challenge
: Psychologically oriented
methodologies need to be adapted
19
19
Conclusions

Nonverbal behaviour provides machine detectable
evidence of social and psychological phenomena

Big Data is an
opportunity
: SSP approaches are
data driven and can benefit from large corpora

Big Data is a
challenge
: Psychologically oriented
methodologies need to be adapted
http://www.sspnet.eu
19
Thank  You!
20