Painting an experience?

runmidgeAI and Robotics

Oct 20, 2013 (3 years and 10 months ago)

108 views

Painting an experience?

How aesthetics might assist a
neuroscience of sensory experience

Ron Chrisley

Centre for Research in Cognitive Science and

School of Informatics, University of Sussex


Neuroesthetics: Where Art and the Brain Collide

ESF Workshop, IULM, Milan 24
-
25 September 2009

5 relevant areas of my research

1.
Embodied creativity

2.
Enactive models of experience

3.
Synthetic phenomenology

4.
Interactive empiricism

5.
Art works/installations


(The title of my talk concerns area 3)

My research and neuroesthetics


Not a neuroscientist


Much of the work I am reporting on is
trying to provide a bridge from
art/aesthetics into the cognitive sciences,
which then permits a connection with
neuroscience


Re: this workshop:


How can the models of aesthetic processes I am
investigating be informed by neuroscience/aesthetics?


Can the models suggest future directions for
neuroscience? Aesthetics/art?

1. Embodied


creativity


Goal: Design a robot/environment system likely to exhibit
creative behaviour:


Novel (at least for the robot)


Of (aesthetic) value (for humans, if possible)


Engineering approach:


No direct modelling of human creativity


But exploit what is known about creativity in humans (and
animals?), when expedient


Allow for possibility that insights into the human case may accrue
anyway


Manifesto only: No implementation yet


Set of "axioms"


Assume case of musical output for examples


Principles of embodied
(aesthetic) creativity

1.
If you make your robot pleasure
-
seeking, and make creativity
pleasurable, you'll make your robot creative

2.
To be a good creator, it helps to be an appreciator

3.
Let the robot experience output in the real world, as we do

4.
We won’t like what the robot likes unless it likes what we like

5.
An important motivator is the approval or attention of others

6.
Novelty can be achieved by trying to produce outputs on the
subjective edge of chaos (that lie just beyond the robot’s ability to
explain/predict)

7.
Let dynamics play a role in appreciation

8.
Patterns in one's own states can be the objects of appreciation

9.
The best way to make outputs in the real world is to be embodied in
the real world









Underlying architecture


CNM:


Recurrent neural network


Forward model of environment


Learns to anticipate/predict the sensory input it
will receive if it performs a given action in a given
context


In conjunction with motivators can enable the
robot to select actions that carry an expectation of
"pleasure"

Underlying architecture

Key:

Recurrent Connection (Copy)

Full Inter
-
Connection Between Layers Of Units

Action

Expected
Sensations

Predicted

State

Previous
Predicted
State

(Context
Units)

D
-
map

T
-
map

Generalisation to aesthetic
experience


Perhaps the content of aesthetic
experience can be understood
using the same framework, with
two extensions:

1.
Affective expectations

2.
Aesthetic expectations

Aesthetic experience: affect


Just as a system can have expectations
concerning its actions and resulting
sensory inputs, it can also have
expectations concerning its actions and
resulting desirable or undesirable states


Thus, affect, and in particular the
affective character of aesthetic
experience, may be accommodated in
EBA

Aesthetic experience: artifice


When we experience a visual work of art we do
not just experience its visual properties (explicable
in terms of expectations to receive input x if we
move our eyes
thus)


We also experience it
as

a work of art: we possess
expectations for how the work would change if we
were to make (or the artist were to have made) this
or that brushstroke


Set of all such expectations is the content of the
aesthetic experience of the work

3. Synthetic phenomenology


A science of consciousness needs a way to refer to
or
specify

the content of conscious experiences


Standard means: e.g., "Mary is having a visual
experience of a red bike leaning against a white
fence"


Problem:
Can only specify experiences with
linguistic,
conceptual
, content


Yet several good reasons to believe that some of
content of experience is
non
-
conceptual


Do I have to draw you a picture?


An obvious alternative is to use non
-
linguistic,
non
-
symbolic

specifications


E.g., for the case of visual experiences, use
images


Can't just take a picture of the scene the subject is seeing
(literalism)


Even in the case of a robot model of experience,
can't

just use
the
raw

video camera output


For example; the current "output" of a human retina contains gaps or
blindspots

that are not part of experience.


Furthermore, our visual experience, as opposed to our retinal output, at
any given time is
stable
, encompassing
more

than the current region of
foveation, and is
coloured

to the periphery


But what
alternatives

are there?

Depictive specifications of the
content of visual experience


If the set of expectations
determines

experiential content,
then
displaying

those expectations (in the right way) will
count as a
specification

of that content


"Filled
-
in" areas specify what
input

the robot would expect
to receive if it moved its head so that it is looking in that
location


Grey areas do
not

indicate an expectation to receive grey
input; they indicate to
you

the
absence

of any expectation
for that location


"Absence of expectation is not an expectation of absence"


Alternative architectures (e.g., generalising neural networks)
would have no such undefined regions of state space

Depictions


Need not explicitly occur anywhere in the agent; are of
experience corresponding to current
expected

inputs, not
actual

inputs


Generated off
-
line by the
theorist:


Sample the action space


Feed actions into forward model


Arrange the resulting
expected sensory patterns

in a spatial array
according to the spatial relations of the sampled actions


Theorist's experience of the depiction (plus
interpretation/bracketing instructions) is the same as (or
shares crucial properties with) the experience to be
specified

Depiction of an

expectational state


Depictive specification and art


In a sense, content
specification is what (at
least some)
artists

have
been trying to do for
millenia


Thus, a clear role for


artists


graphic designers


sound engineers


directors


etc.



Depictive specification and art


E.g., creating a
film

of a car
chase in the desert


Not just a matter of
"objectively
recording"


Requires
artistic insight

into
how film will affect viewers,
e.g.:


Pacing


timing, number, kinds of cuts
and edits


what is in the frame


steadiness


Cf Picasso's
portraits

Problem: Subjectivity of Art?


Science strives to be
objective


But content of a work of
art is highly
subjective


Yes, but artisitic
insights

can be objectively
investigated
; e.g.:


Perspective


Moving images:
“persistence of vision”


Effect of colour on mood
\


Et al



Problem: Subjectivity of Art?


Is subjectivity
always

at odds
with science?


Objective science is
not

the
elimination of the scientist's
subjectivity


Rather the
negotiation

of it


Perhaps subjectivity of theorist
can be
exploited


Lessons from
interactive Art
?



A two
-
way interaction


Often, it has been thought that
the interaction was
one
-
way
:


cogntive scientists informing
the work of designers, artists,
and other "creative" types


But the
converse

interaction is
needed as well:


Artistic input into radically
different means of specifying
the contents of experience

4. Interactive empiricism


Some problems concerning (aesthetic) experience are
philosophical; require conceptual breakthroughs/progress


Such breakthroughs (e.g. new concepts) may not be
achievable by reason alone, but require experiential
activity


“…Modelling of consciousness… requires some clarifications and
refinements of our concept of consciousness. Design of,
construction of, and interaction with artificial systems can itself
assist in this conceptual development.”
(Sloman and Chrisley
2003)


Sensory augmentation (e.g., work with Froese and Spiers on how
using the “Enactive Torch” might alter concepts of perception)


Art works are also “artificial systems”: creation of and interaction
with


5. Art works/installations

5. Art works/installations

Recap

1.
Embodied (artistic) creativity

2.
Enactive models of (aesthetic) experience

3.
Synthetic phenomenology: specifying the
content of the sensory, affective and
aesthetic components of experiencing art
works

4.
Interactive empiricism: changing our
concepts of experience through creating or
interacting with art

5.
Art works/installations based on the above

5. Art works/installations


Thank you.







Comments welcome:

ronc@sussex.ac.uk