Comprehending Conversational Utterances ... - Holtgraves

soilflippantAI and Robotics

Nov 17, 2013 (3 years and 8 months ago)

82 views

Comprehending Conversational
Utterances: Experimental Studies of the
Comprehension of Speaker Meaning


Thomas Holtgraves

Dept. of Psychological Science

Ball State University

Muncie, IN

USA

Speaker Meaning


What a speaker intends to convey with an
utterance on a particular occasion of use
(Clark, 1985)


Often different from literal or direct meaning


Fundamental issue: How do language users
comprehend indirect speaker meaning?


What are the social, cognitive, and neural
processes that underlie comprehension?


Examples of Nonliteral Speaker
Meaning

Types of Nonliteral Meaning

Commonly studied:



Metaphor


My job is a jail
.



Ironic sarcasm


You’re a great friend (when one isn’t)



Indirect requests


Could you open the window?

Less commonly studied:


Implicit Speech acts


Indirect replies

Different Comprehension Processes Involved

Implicit Speech Acts

Speech Act Theory


John Austin and John Searle


Language use as action


Illocutionary act


specific act(s) speaker
intends the hearer to recognize


Take the form of speech act verbs (e.g., criticize,
thank, apologize, offer, etc.)


Implicit speech acts do not contain the speech
act verb (I’ll definitely do it tomorrow)

Speech Act Processing


Is speech act recognition involved in utterance
comprehension?



Necessary? (Not according to relevance theory)



Good enough processing in conversations; quick
take on speaker meaning (via speech act
recognition)










Speech Act Activation Experiments



(Holtgraves & Ashley, 2001; Holtgraves, 2008)



Jenny and Emily had been close friends since grade

school.


Now there were rooming together at college.


Emily was very forgetful.


Today, Jenny was sure Emily didn’t remember her

dentist appointment.












Jenny: Don’t forget to go to your dentist

appointment today.











+










REMIND








Sample Experimental Materials




Jenny and Emily had been close friends since grade
school
.


Now there were rooming together at college.


Emily was very forgetful.


Today, Jenny was sure Emily
didn’t remember
(had forgotten) her

dentist appointment.


Jenny:
Don’t forget

(I’ll bet you forgot) to go to your

dentist appointment today.


Probe: Remind








Recognition Probe Reaction Times (ms)

Holtgraves, 2008

720
740
760
780
800
820
840
860
880
900
920
Written
Auditory
Speech Act
Control
Implicit Speech Act Comprehension
Experiments


Lexical Decision Procedure (Word/
Nonword
):


Judge speech act words (e.g., remind) faster after
speech act utterances than control utterances


Participants vs. Observers


Conversation
Bot

(SAM):


Participants demonstrate automatic speech act
activation


SAM: Don’t take a class from Harmon, he’s terrible

»
WARN


Speech Acts and Memory

Holtgraves (2008)


Do Speech acts play a role in long
-
term
representation?


Participants read scenarios/utterances


Speech act/control versions


Rated scenarios (incidental memory)


Intervening task (recall states)


Memory test


Recognition or Recall



False Memory for Speech Act Verbs

0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
Recognition
Recall
Speech Act
Control
What are the
Neurophysiological

Underpinnings of Speech Act
Comprehension?


Does Everyone Automatically
Recognize Speech Acts?



Speech Act Recognition in
Parkinson’s Disease



Parkinson’s Disease (PD) can display cognitive
and social deficits as well as motor deficits


Are social deficits due to pragmatic deficits
(speech act recognition)?


PD (N = 28) and age matched controls (N = 32)
performed lexical decision task following
speech act/control scenarios (rewritten for PD).



-

Assess executive function (
stroop

task)



Lexical Decision Times (ms)

Holtgraves & McNamara, 2010

500
700
900
1100
1300
1500
1700
1900
Control Participants
Parkinson's Participants
Speech Act
Control
Speech Act Priming and Utterance
Production



Speech act priming correlated with under
-
informativeness

in interactions


Interviews coded for under
-
informativeness


Failure to recognize others’ intentions related to lack of
informativeness

in utterance production


Neural Underpinnings


Why speech act recognition disrupted in PD?


Our results suggest executive cognitive function:


Speech act
p
riming correlated with
stroop

interference for PD Ps(r =
-

.81) but not control Ps (r =
.05).


Action verb/motor circuit connection


Speech act deficit due to motor impairment


Comprehension of action entails some simulation of action
(embodied cognition)


Upper body impairment
-

more lower body verbs


Neural
Underpinnngs

of Speech Act
Comprehension: Lateralization


Role of right hemisphere (RH) in speaker
meaning


Evidence from RHD participants (poor at
recognizing nonliteral meanings)


RH intention recognition (imaging data)


Predict RH specialized for Speech Act
comprehension


Speech Act Comprehension materials


Lateralize targets to RVF/LH or LVF/RH





Jenny: Don’t forget to go to your dentist

appointment today.














+











REMIND








Lexical Decision Speed as a Function of
Utterance Type and Visual Field

Holtgraves, 2012








Indirect Replies

Context
-
Dependent
Speaker Meaning

Indirect Replies


Replies that violate the Relation Maxim (be
relevant)


No preferred reading out of context
(particularized implicatures)

Example:


Nick: What did you think of my
presentation?


Paul: It’s hard to give a good presentation.



Indirect Replies



How are indirect replies interpreted?


Why are they interpreted this way?


How do people make this
interpretation?


What are the neural underpinnings of
this process?

Model: Grice +
Goffman


Relevance violation is noticed/inference
generated


Inference based on perceived reason for violation


Relevance violations occur because of face
management


Recipients realize this and use it as an
interpretive frame


In general, relevance violations should be
interpreted as conveying FT information

Relevance Violation Experiments

Which inference?


Participants read scenarios, questions and
replies


Manipulate context:


Positive (presentation was good)


Negative (presentation was bad)


No information


Ask Ps to:


Interpret replies


Time reply comprehension


Percent Negative Interpretations

Indirect replies interpreted just as
negatively in no information contexts as in
negative information contexts


0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
No Information
Negative
Information
Positive Information
j
Reply Comprehension Speed (in ms) as a Function
of Context

Indirect replies interpreted just as quickly
in no information contexts as in negative
information contexts


2000
2100
2200
2300
2400
2500
2600
2700
2800
No Information
Negative
Information
Positive Information
Relevance Violation Experiments

Comprehension Processes



Manipulate context: Force literal or
indirect reading


Assess: reply comprehension speed




indirect meaning priming




literal priming

Reply Comprehension Speed (ms)


Process is time
-
consuming (replies
with indirect meanings take longer
than matched controls)


1000
1200
1400
1600
1800
2000
2200
2400
2600
2800
Indirect Reading
Literal Reading
Sentence Verification Speed (ms)

Sentence verification judgments for indirect
interpretations faster following replies with indirect
meanings relative to replies with literal meaning: Indirect
meaning activated on
-
line (at comprehension, not post
-
hoc)


1000
1100
1200
1300
1400
1500
1600
1700
1800
1900
Indirect Reading
Literal Reading
Dispreferred

Markers and Reply
Comprehension


If recognition of face management drives
interpretive process, then factors suggesting
face management is operative should facilitate
comprehension


Indirect Reply comprehension faster if
preceded by:



“Well”


Brief delay (2 s)





Neural Correlates

Basnakova

et al., 2011


fMRI while comprehending indirect replies


Face
-
saving vs. informative indirect replies

(It’s hard to give a good presentation)


Greater activation of:


Right Anterior Cingulate Cortex (empathy)


Right Superior Temporal
Gyrus

(
inferencing
)


Right Inferior Frontal Cortex (contextual integration)

Summing Up


Nonliteral

speaker meaning is pervasive in conversation


Many different types of
nonliteral

meaning


Different social, cognitive, and neural processes involved
in their comprehension


Social processes: face management


Comprehension is a mirror image of production (FM)


Cognitive processes: good enough processing


Neural processes: RH and frontal networks involved in
perspective taking and inferencing (networks different
from classic language networks)


Process models require more research on real
-
time
pragmatic processing (e.g., EEG)