INTRODUCTION TO ARTIFICIAL

soilflippantΤεχνίτη Νοημοσύνη και Ρομποτική

17 Νοε 2013 (πριν από 3 χρόνια και 10 μήνες)

62 εμφανίσεις

INTRODUCTION TO ARTIFICIAL
INTELLIGENCE

Massimo Poesio


LECTURE 8

Concepts in the brain


USING BRAIN DATA TO IDENTIFY CATEGORY
DISTINCTIONS



Studies of brain
-
damaged patients have been shown to
provide useful insights in the organization of conceptual
knowledge in the
brain


Some patients are unable to identify or name man made
objects and others may not be able to identify or name
natural kinds (like animals)


Warrington and
Shallice

1984,
Caramazza

&
Shilton

1998


fMRI

has been used to identify these distinctions in
healthy patients as well


E.g.,

Haxby

et al 2000, Martin
&
Chao 2003


See, e.g.,

Mahon &
Caramazza

2011, Martin 2007 for
review


Warrington &
Shallice

1984


Warrington and
Shallice

(1984) reported a patient called
JBR who following an acute lesion to the left temporal lobe
(as a result of herpes encephalitis) had a selective deficit
when asked to name pictures from just one semantic
category


living things
.



By contrast JBR was able to name
non
-
living objects

very
well including those with low frequency names such as
‘accordion’ that were matched for the number of letters in
the name and the visual complexity of the object.



Other patients have shown opposite pattern


Evidence from semantic category
deficits



Modality
-
specific deficits


Patients are unable to name visually presented objects, but can name
them from other modalities and can access other semantic information
about visually presented stimuli
(
Beauvois
, 1982)


Other visual processing is fine.


Category
-
specific deficits
(e.g., Warrington & McCarthy, 1983, 1987;
Warrington &
Shallice
, 1984;
Gainotti

&
Silveri
, 1996)


Patients show impairments in processing living things
vs. man
-
made objects and vice versa.


Interesting exceptions: fruits, vegetables & other
foods; musical instruments

A PET Study on categories (Nature 1996)

Study


16 adults (8M, 8F) participated in a PET (positron emission
tomography) study.


Involves injecting subject with a positron emitting radioactive
substance (dye)


Regions with more metabolic activity will absorb more of the
substance and thus emit more positrons


Positron
-
electron collisions yield gamma rays, which are detected


Increased rCBF (regional changes in cerebral blood flow) was
measured


When subjects viewed line drawings of animals and tools.

The experiment


Subjects looked at pictures of animals and tools and
named them silently.


They also looked at noise patterns (baseline 1)


And novel nonsense objects (baseline 2)


Each stimulus was presented for 180ms followed by
a fixation cross of 1820 ms.


Drawings were controlled for name frequency and
category typicality

Premotor

ACC

Left middle temporal gyrus

Calcarine Sulcus

Conclusions


Both animal and tool naming activate the ventral
temporal lobe region.


Tools differentially activate the ACC, pre
-
motor and
left middle temporal region (known to be related to
processing action words).


Naming animals differentially activated left medial
occipital lobe (early visual processing)


The object categories appear to be in a distributed
circuit that involves activating different salient
aspects of the category.

REPRESENTATION OF CONCEPTS IN
THE BRAIN: COMPETING HYPOTHESES


Unitary Content Hypothesis


Semantic information is stored in an abstract, amodal
format organized by category.



Multiple Semantics Hypothesis


Semantic information is stored in many modality
-
specific
semantic subsystems. Information in each subsystem is
stored in a modality specific format.


Our intuitive sense of information being organized by
categories is based on strong connections between related
parts of these modality
-
specific semantic systems.

Unitary Content Hypotheses
(UCH)

(Caramazza et al., 1990; Caramazza & Shelton, 1998; Riddoch et al., 1988; Pylyshyn, 1973)

Multiple Semantics Hypotheses
(MSH)

(Paivio, 1971; Beauvois, 1982; Shallice, 1987, 1988; McCarthy & Warrington, 1988)

Representation of words in semantic memory:
the Functional Web hypothesis


A word is represented in the cortex as a
functional web


Spread over a wide area of cortex


Includes perceptual information


As well as specifically conceptual information


For nominal concepts, mainly in


Angular
gyrus


(?) For some, middle temporal
gyrus


(?) For some,
supramarginal

gyrus


Plus phonological information


Example: The concept
DOG


We know what a dog looks like


Visual information, in occipital lobe


We know what its bark sounds like


Auditory information, in temporal lobe


We know what its fur feels like


Somatosensory

information, in parietal lobe


All of the above..


constitute perceptual information


are
subwebs

with many nodes each


have to be interconnected into a larger web


along with further web structure for conceptual
information

Building a model of a functional web:

First steps

V

C

Each node in this diagram

represents the cardinal node* of a
subweb of properties





For example

M

T

*to be defined in a moment!

Add phonological recognition

V

M

C


For example,
FORK



Labels for Properties:

C


Conceptual

M


Motor

P


Phonological image

T


Tactile

V


Visual

T

P

The phonological image
of the spoken form [fork]
(in Wernicke’s area)

These are all

cardinal nodes


each is supported

by a subweb

Add node in primary auditory area

V

M

C

T

P

PA

Primary Auditory
: the cortical structures in the primary
auditory cortex that are activated when the ears receive
the vibrations of the spoken form [fork]


For example,
FORK



Labels for Properties:

C


Conceptual

M


Motor

P


Phonological image

PA


Primary Auditory

T


Tactile

V


Visual

Add node for phonological production

V

M

C

T

P

PA

PP


For example,
FORK



Labels for Properties:


C


Conceptual


M


Motor


P


Phonological image


PA


Primary Auditory


PP


Phonological Production


T


Tactile


V


Visual

Arcuate fasciculus

Part of the functional web for
DOG

(showing cardinal nodes

only)

V

M

C

T

P

PA

PP

Each
node
shown
here is the
cardinal
node of a
subweb

For example, the
cardinal node of
the visual
subweb

An activated functional web

(with two subwebs partly shown)

V

PR

PA

M

C

PP

T

Visual features

C


Cardinal concept node

M


Memories

PA


Primary auditory

PP


Phonological production

PR


Phonological recognition

T


Tactile

V


Visual

FROM WORDNET TO BRAINNET


Neural evidence, unlike the evidence used to
compile dictionaries and
WordNet
, and like the
evidence one gathers from corpora and certain
behavioral experiments, is entirely objective
(although it can be subjective in the sense of
differing from subject to subject)


The objective of our research is to combine
evidence from brain data, from corpora, and from
behavioral experiments (all of which is rather
noisy) to develop a new architecture for
conceptual knowledge:
BrainNet

A CASE STUDY:

ABSTRACT CONCEPTS


Until recently, most work on concepts in CL /
neuroscience / psychology focused on concrete
concepts


But the type of conceptual knowledge that really
challenges traditional assumptions about its
organization are `abstract concepts’


or to be more
precise, the set of categories of non
-
concrete concepts


Events / actions


States



Urabstract
’ concepts: LAW, JUSTICE, ART


We are carrying out explorations of abstract
knowledge using
fMRI

Anderson et al 2012a, 2012b, 2013, submitted

THEORIES OF ABSTRACT CONCEPTS IN AI AND
COGNITIVE (NEURO)SCIENCE


In CL/AI: TAXONOMIC organization for both abstract
and concrete concepts


‘UPPER ONTOLOGIES’, e.g., DOLCE


In psychology: ‘concreteness’ scale


Best known Cognitive Neuroscience:
Paivio’s

DUAL
CODE theory (
Paivio
, 1986)


CONCRETE: verbal system & visual system


ABSTRACT: verbal system only


Schwanenflugel

& Akin 1994: CONTEXT AVAILABILITY


Barsalou’s

SCENARIO
-
BASED MODEL (
Barsalou
, 1999):


Abstract knowledge organized around SCENARIOS


The DOLCE UPPER ONTOLOGY


Q

Quality

PQ

Physical

Quality

AQ

Abstract

Quality

TQ

Temporal

Quality

PD

Perdurant

EV

Event

STV

Stative

ACH

Achievement

ACC

Accomplishment

ST

State

PRO

Process

PT

Particular

R

Region

PR

Physical

Region

AR

Abstract

Region

TR

Temporal

Region

T

Time

Interval

S

Space

Region

AB

Abstract

Set

Fact









TL

Temporal

Location

SL

Spatial

Location







ASO

Agentive
Social Object

NASO

Non
-
agentive
Social Object

SC

Society

MOB

Mental Object

SOB

Social Object

F

Feature

POB

Physical

Object

NPOB

Non
-
physical

Object

PED

Physical

Endurant

NPED

Non
-
physical

Endurant

ED

Endurant

SAG

Social Agent

APO

Agentive

Physical

Object

NAPO

Non
-
agentive
Physical

Object



AS

Arbitrary

Sum

M

Amount of

Matter









THE OBJECTIVES OF OUR EXPERIMENT


Identify the representation in the brain of a variety of
WordNet

categories exemplifying both concrete and abstract concepts
(abstract words chosen by inspecting the words rated as most
abstract in the De Rosa et al norms 2005)


Really abstract: ATTRIBUTE, COMMUNICATION, EVENT, LOCATION,
‘URABSTRACT’


A category of concrete objects: TOOLS


A complex category: SOCIAL
-
ROLE


Comparing two types of classification:


TAXONOMIC (as in
WordNet
)


DOMAIN (
cfr
.
Barsalou’s

hypothesis about abstract concepts being
‘situated’)


Two domains: LAW and MUSIC


Using
WordNet

Domain

STIMULI

CATEGORY

LAW

(English)

MUSIC

(English)

attribute

giurisdizione

jurisdiction

sonorita
'

sonority

cittadinanza

citizenship

ritmo

rhythm

impunita'

impunity

melodia

melody

legalita'

legality

tonalita'

tonality

illegalita'

illegality

intonazione

pitch

communication

divieto

prohibition

canzone

song

verdetto

verdict

pentagramma

stave

ordinanza

decree

ballata

ballad

addebito

accusation

ritornello

refrain

ingiunzione

injunction

sinfonia

symphony

STIMULI, 2: URABSTRACTS

CATEGORY

urabstracts

giustizia

justice

musica

music

liberta'

liberty

blues

blues

legge

law

jazz

jazz

corruzione

corruption

canto

singing

refurtiva

loot

punk

punk

STIMULI, 3: SOCIAL ROLES



Social
-
role

giudice

judge

musicista

musician

ladro

thief

cantante

singer

imputato

defendant

compositore

composer

testimone

witness

chitarrista

guitarist

avvocato

lawyer

tenore

tenor

THE OBJECTIVES OF OUR EXPERIMENT


Identify the representation in the brain of a variety of
WordNet

categories exemplifying both concrete and abstract concepts
(abstract words chosen by inspecting the words rated as most
abstract in the De Rosa et al norms 2005)


Really abstract: ATTRIBUTE, COMMUNICATION, EVENT, LOCATION,
‘URABSTRACT’


A category of concrete objects: TOOLS


A complex category: SOCIAL
-
ROLE


Comparing two types of classification:


TAXONOMIC (as in
WordNet
)


DOMAIN (
cfr
.
Barsalou’s

hypothesis about abstract concepts being
‘situated’)


Two domains: LAW and MUSIC


Using
WordNet

Domain

ABSTRACT CONCEPTS: DATA
COLLECTION AND ANALYSIS


7 right
-
handed native speakers of Italian


Task:


Words presented in white on grey screen for 10 sec


Cross in between, 7 sec


Subjects had to think of a situation in which the word applied


Scanner:
4T
Bruker

MedSpec

MRI scanner, EPI pulse sequence



TR=1000ms, TE=33ms, 26
°

flip angle.


Voxel

dimensions 3mm*3mm*5mm


Preprocessing: using
UCL’s

Statistical Parameter Mapping
Software


Data corrected for head motion


Classification: using a single layer NN


MAIN QUESTIONS


Can the taxonomic and domain classes be
distinguished from the
fMRI

data?


Is there a difference in classification accuracy
between taxonomy and domain?


Can the taxonomic and domain classes be
predicted across participants?

RESULTS WITHIN PARTICIPANTS

(CATEGORY DISTINCTIONS)

ALL CATEGORICAL DISTINCTIONS CAN BE
PREDICTED ABOVE CHANCE

THERE ARE SIGNIFICANT DIFFERENCES
BETWEEN CATEGORIES

RESULTS WITHIN
PARTICIPANTS(DOMAIN)

WITHIN PARTICIPANTS RESULTS
SUMMARY


Can discriminate with accuracy well above
chance both taxonomic and domain
distinctions


Easiest categories to recognize: TOOL,
ATTRIBUTE, LOCATION,


Then SOCIAL ROLE, COMMUNICATION


Main confusions: communication / event

Red: Attribute

Blue: Tool

Green: Location


R+G=Yellow

G+B=Cyan

R+B=Pink

R+G+B=White


CATEGORY

LOCALIZATION IN THE
BRAIN

Red: Social
-
role

Green: Attribute

Blue:
Urabstract


Red: Social
-
role

Green:
Communication

Blue: Event


R+G=Yellow

G+B=Cyan

R+B=Pink

R+G+B=White



Concrete categories TOOL and LOCATION can be
predicted across participant; ATTRIBUTE can also
be significantly classified; but less concrete
classes become conflated with ATTRIBUTE.


In general DOMAIN can be predicted across
participants, however domain membership is
much better classified in the most abstract
taxonomic classes (attribute, communication and
urabstract
)

CROSS PARTICIPANTS RESULTS SUMMARY


LAW





MUSIC


Attribute


giurisdizione

jurisdiction


sonorita
'


sonority


cittadinanza

citizenship


ritmo


rhythm


impunita
'

impunity


melodia


melody


legalita
'


legality



tonalita


tonality


illegalita



illegality


intonazione

pitch




communication

divieto


prohibition


canzone

song


verdetto

verdict



pentagramma

stave


ordinanza

decree



ballata


ballad


addebito

accusation


ritornello

refrain


ingiunzione

injunction


sinfonia


symphony









event


arresto


arrest



concerto

concert


processo
trial



recital


recital


reato


crime



assolo


solo


furto


theft



festival


festival


assoluzione

acquittal


spettacolo

show


social
-
role

giudice


judge



musicista

musician


ladro


thief



cantante

singer


imputato

defendant


compositore

composer


testimone

witness



chitarrista

guitarist


avvocato

lawyer



tenore


tenor










tool


manette

handcuffs


violino


violin


toga


robe



tamburo

drum


manganello

truncheon


tromba


trumpet


cappio


noose



metronomo

metronome


grimaldello

skeleton key


radio


radio




Location


tribunale

court/tribunal


palco


stage


carcere


prison



auditorium

auditorium


questura

police station


discoteca

disco


penitenziario

penitentiary


conservatorio

conservatory


patibolo

gallows



teatro


theatre





urabstracts

giustizia

justice



musica


music


liberta
'


liberty



blues


blues


legge


law



jazz


jazz


corruzione

corruption


canto


singing


refurtiva

loot



punk


punk

TAXONOMIC / DOMAIN ORGANIZATION

WHAT THE DATA SUGGESTS

READINGS


Binder & Desai 2011, The Neurobiology of
semantic memory,
Cell

(on the website)