Perception and Attention - Ohio University

brasscoffeeAI and Robotics

Nov 17, 2013 (4 years and 7 months ago)




Perception and

Janusz A. Starzyk

Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars

courses taught by

Randall O'Reilly
, University of Colorado, and

Prof. Włodzisław Duch
Uniwersytet Mikołaja Kopernika


Cognitive Architectures




Where does invariance come

A 3D image based on 2D
projections, what's remembered
is just one 3D representation
(Marr 1982).

Syntactic approach: form a
whole from pieces of a model.

Variant (Hinton 1981): look for transformations (displacement, scaling,
rotation), conform to the canonical representation in the memory.

Problem: many 2D objects can form different 3D objects; it's difficult to
match the objects because the search space to connect fragments into
a whole is too large

do we really remember 3D objects?



Gradual transformations

In the brain, rotational invariance is
strongly limited

eg. recognizing rotated

Limited invariant object recognition can be
achieved thanks to gradual hierarchical
parallel transformations, increasing
invariance and creating increasingly
complex features of distributed

Goal: not 3D, but to retain enough details to be able to recognize objects
in an invariant manner after transformation.

Map seeking circuits in visual cognition (D. W. Arathorn, 2002 )



Dorsal pathway

Recognition is a function of the ventral pathway, now let's turn to the
dorsal pathway. Functions: motion detection, localization, "where” and
how to act, but also on what to focus attention and how to shift attention
from one object to another.

Attention allows us to tie different properties of an object into one whole,
to solve the problem of cohesion of sensations in spite of distributed
processing; distributed activation => features related to each other,
referring to one object.

Mainly an attention model, an emergent process resulting from the
structure and dynamic of neural networks, mainly inhibition.

The effects of attention are universal, visible in different situations.

What to pay attention to? Is this a well posed question?

Dogs bite, but not only Spot, not only mongrels, not only black ones...



Lesion studies

Consequences of damage to early visual areas

Different visual deficits can result from neural damage at different
levels of the visual processing hierarchy.

Damage to the retina can result in monocular blindness

Damage to the LGN can lead to loss of vision in the contralateral
visual field

Damage to a small part of V1 can lead to a clearly defined scotoma.

Patients with damage to V1 area may still perform better than chance
forced choice discrimination of objects (blindsight), although they
claim they see nothing.

Although the pathway from retina to LGN to V1 provides most of
visual inputs to cortex, several alternative subcortical pathways project
to extrastriate areas (MT, V3, V4), bypassing V1. This may explain
forced choice results.



Extrastriate lesions

damage outside area V1

Motion blindness caused by a lesion to area MT: the world
appears to be a series of still snapshots.

Crossing street is dangerous since the patient cannot tell how
fast the cars are approaching.

Pouring a cap of coffee becomes a challenge since she cannot
tell how fast the liquid was rising.

Lesion studies



Cortical color blindness may be caused by a lesion to area V4:

The world appears to be drained of color, just shades of gray.

Patients can perceive the boundaries of colors but cannot name

Lesion studies



Damage to ventral
object areas

Visual Agnosia:

Patients with
visual agnosia have difficulties with
recognizing objects because of
impairments in basic perceptual
processing or higher
recognition processes

Three types of
apperceptive agnosia,
associative agnosia,


Agnosia=to lack knowledge of

Lesion studies



Patients with
apperceptive agnosia

can detect the appearance of visually
presented items, but they have
difficulty perceiving their shape and
cannot recognize or name them.

Associative agnosia

refers to the
inability to recognize objects, despite
apparently intact perception of the

Patient can copy a picture of the object but
does not recognize it.

A patient mistook his wife for a hat.

Associative agnosia results from damage
to ventral temporal cortex.

Lesion studies



Patients with
optic ataxia

can perceive visual
orientation and recognize objects but cannot
perform visually guided actions.

Optic ataxia results from damage to parietal lobe in
dorsal pathway.

Patients with

are still able to
recognize objects well, but have great difficulty
recognizing faces.

All faces look the same

Patients can recognize animals but not people

Lesion studies

Brodman area no. 37 is responsible for
face recognition

over 90% of cells in area 37 responds to
faces only.



fMRI analysis of the face recognition process.

Visible is activity in right hemisphere in lower temporal area

Face recognition is important from evolutionary perspective.

Lesion studies



Patients with

are unable to recognize

This is often a result of damage to area V4 or thalamus.

Lesion studies




refers to

characterized by a
lowered sensitivity to green light resulting in an
inability to distinguish green and purplish

It is an inherited defect in perception of red and green, or in
other words, red
green colorblindness.

Lesion studies



Dorsal pathway lesions

Lesions in the parietal cortex strongly affect mechanisms of attention
and spatial orientation, extensive lesions in one hemisphere lead to
hemispatial neglect, the inability to focus attention to the half of the
visual space which is opposite the lesion.

For small unilateral lesions, we can see a noticeable slowing of attention
switching to the damaged side. For more severe cases, switching
attention is not possible.

Bilateral lesions lead to
Balint's syndrome
, difficulties with the
coordination of hand and eye movement, simultanagnosia; differences in
attention switching times in the
Posner task

are small.

Posner contended that this is a result of attention binding, the inability to
disengage, but he didn't give the disengagement mechanism; it follows
after focusing attention elsewhere

a better model assumes normal



Damage to the posterior parietal
lobe can lead to a

, in which a patient
completely ignores or does not
respond to objects in the
contralateral hemifield.

Patients with damaged spatial
temporal recognition forget
about half the space even
though they see it

Patients with right parietal damage
may ignore the left half of the
visual field, eat half of the food
from the plate, or apply make
up to
half of the face.

Lesion studies




Unilateral Neglect

Horizontal line bisection task

Copying drawings



Bilateral lesions to parietal areas can lead to a
much more profound deficit called
, which is primarily a disruption of
spatial attention.

It can be characterized by three main deficits:

Optic ataxia

inability to point into a target

Ocular apraxia

inability to shift the gaze


inability to perceive more than one
object in the visual field

People with
Balint’s syndrome

blind since
they only focus on one object and cannot shift
attention to anything else.

Lesion studies



Linking brain activity and visual experience

Imagine you are sitting in a dark room and looking at a jacket on a chair.

Since you cannot see well, your perception is driven by your imagination

you may perceive a strange animal, a person, or a statue sitting there.

When vision is ambiguous, perception falters or alternates between
different things. This is known as
multistable perception.

There are many examples of multistable patterns or ambiguous figures
that scientists use to investigate these neural correlates of consciousness.



Binocular rivalry
what you see is what you get activated

When two very different pattern are shown, one to each eye, the brain cannot
fuse them together like it would normally do.

What happens is striking: awareness of one pattern last few seconds, then the
other pattern appears

Linking brain activity and visual experience

You can cause
binocular rivalry here
using a pair of red
green glasses



What happens in the brain during binocular rivalry?

et al.

tackled this problem by focusing on two category
areas in the ventral temporal lobes (FFA and PPA). They used the red
green filter glasses to present a face to one eye and house to the other
eye. Depending on which image was perceived, they observed
activities either in FFA (face) or PPA (house).

Linking brain activity and visual experience



Strength of activation of FFA and
PPA was the same in the
rivalry experiment as in the
case of stimulus alternation.

Another approach is to train
monkey to report which of two
patterns is dominant during
binocular rivalry and measure
activity of a single neurons in
different parts of the brain.

This experiment supports
interactive model of visual
perception where feedback
projection modulates lower

Linking brain activity and visual experience



Another way to separate physical stimulation and perceptual
awareness is a
visual detection


A subject has to detect a particular pattern.

The researcher makes the pattern harder and harder to see.

Sometimes there is no pattern at all in the picture.

Because this task gets difficult, people will get it wrong sometimes.

What is interesting, that when there is ‘false positive’ (people see pattern
even when it is not there), there is strong activity in areas V1, V2, and V3.

When the faint stimulus is not detected activities in these areas are much

So, it does not matter what was presented, but what does matter is what is
happening in the brain.

Linking brain activity and visual experience



Close your left eye, look directly at the cross with your right eye
and move the page up close to your nose, then move it slowly
away from your face, while keeping your eye fixed on the
cross. At the right distance, which should be around 12 inches
(30 cm) away from the page you should notice the red dot

Likewise, notice how the black stripes now fill
in; they become
joined and the red dot vanishes.

Brain fills
in perception of the blind spot using visual
information from around the blind spot


perceptual filling

Linking brain activity and visual experience



Optical illusions are a result of
our mind filling
in patterns
based on experience

Linking brain activity and visual experience

Adelson's motion without movement



Zoom in on the color spiral

two colors are the same shade of green.

Linking brain activity and visual experience

Two color spirals



These pictures illustrate
another type of filling
known as
neon color

(a) and


Neon color spreading were
found in V1 area.

In a similar way

that we see in a
movie theater is another
type of filling
in by neural
activities in V1 area.

Linking brain activity and visual experience



Neural correlates of object recognition

In binocular rivalry, activity in the fusiform face area and parahippocampal
place area is closely linked to the observer’s awareness of faces and houses.

Other studies deals with visually masked objects which can just barely be

Mooney face shown in figure can be recognized at right orientation, while it is
hard to recognized at different orientations.

If the objects are recognized activity in ventral temporal region is greater, while
activity in V1 region shows no difference..

Linking brain activity and visual experience



Manipulations of visual awareness

To find out causal relations
between activities in various brain
regions it is useful to directly
stimulate the selected brain


with electrical impulses.

One way is to use implants for
instance in V1 area

Another way is to use transcranial
magnetic stimulation (TMS)

TMS involves rapidly generating a
magnetic field outside of the head to
induce electrical activity on the
cortical surface.

Patients report various experiences
including ‘out of body experience’

seeing its own body from above.



Unconscious perception

We use the term
unconscious perception

when subjects
report not seeing a stimulus, but their behavior or brain
activity suggests that specific information about the
unperceived stimulus was indeed processed by the brain.

When two different stimuli are flashed in quick
succession, the visual system can no longer separate the
two stimuli.

Instead, what people perceive is a mix, or a fused blend
of the two images.

They may respond to individual images in various brain
areas without being aware of seeing them

Manipulations of visual awareness



For instance, a quick presentation of a red square followed by a green
square can be perceived as a yellow one.

Presentation of the images of the house or face in complementary colors
to different eyes has the same effect of not seeing one.

However, the brain still responds to these unseen patterns

fusiform face
area (FFA) to face and parahippocampal place area (PPA) to house.

Manipulations of visual awareness




Vision is our most important sensory modality.

We discussed the functional properties of neurons as visual signals
travel up from the retina to the primary visual cortex and onward to
higher areas in the dorsal and ventral visual pathways.

Progressing up the visual pathway, receptive fields gradually
become larger and respond to more complex stimuli, following the
hierarchical organization of the visual system.

V1 supports conscious vision, provides visual features like
orientation, motion and binocular disparity.

V4 is important for color perception.

MT is important for motion perception.

Damage to dorsal pathway leads to optic ataxia (neglect).

Damage to ventral temporal cortex leads to impairments in object or
face recognition.

In ventral temporal cortex some regions like LOC have general role
in object recognition, while others like FFA and PPA are more




Attention effects appear naturally in the model as a result of competition
between inhibition, interconnection, the necessity of compromise.

Similar effects can be seen in different cortical mechanisms.

Some psychological mechanisms (slowing attention) show themselves
to be unnecessary.

Attention effects supply specific information allowing models to be fine
tuned to comply with experiment results and allowing the use of these
models for other predictions; there is also a lot of neurophysiological
data concerning attention.

Limits of this model:

lack of effects connected with the thalamus (Wager, O’Reilly),

very simple representation of objects (one feature).



Some answers

Why does the primary visual cortex react to oriented edges?

Because correlational learning in a natural environment leads to this
type of detector.

Why does the visual system separate information into the dorsal
pathway and the ventral pathway?

Because signal transformations extract qualitatively different
information, strengthening some contrasts and weakening others.

Why does damage to the parietal cortex lead to disorders of spatial
orientation and attention (neglect)?

Because attention is an emergent property of systems with

How do we recognize objects in different locations, orientations,
distances, with different images projected on the retina?

Thanks to transformations, which create distributed representations
based on increasingly complex and spatially invariant features.