Intereaction, Reaction and Performance:

wafflejourneyAI and Robotics

Nov 14, 2013 (4 years and 8 months ago)


Intereaction, Reaction and

The human body tracking project

Sue Broadhurst

As a result of new technological advancements in performance practice, I am
arguing that
new liminal spaces exist where there is a potential for a
reconfiguration of creativity and experimentation in performance practice.

Liminality, from

(Latin: literally threshold) is a term most notably linked
to Victor Turner who writes of a no
land betwixt
between, a site of
a 'fructile chaos ... a storehouse of possibilities, not by any means a random
assemblage but a striving after new forms ' (Turner, 1990: 11
12). My own
use of the term includes certain aesthetic features described by
Turner, but
emphasizes the corporeal, technological, and chthonic (Greek: back to the
earth) or primordial (Broadhurst, 1999a: 12).

These spaces are liminal in as much as they are located on the 'threshold' of
the physical and virtual. As a result, tensio
ns exist within the spaces created
by new technological art practices, such as, motion capture and artificial
intelligence. Since no body not even a naked body escapes (re)presentation
altogether (Broadhurst, 1999a: 103), the virtual body (as any other bod
inscribes its presence and absence in the very act of its performance leaving
gaps and spaces within its wake. I suggest it is within these tension filled
spaces that opportunities arise for new experimental forms and practices.

Within my current resea
rch I explore performance practices which explode the
margins between the physical and virtual and what is seen as dominant
traditional art practices and innovative technical experimentation. This
involves collaboration with Richard Bowden, a systems engin
eer from the
Vision and Virtual Reality group at Brunel. The VVR group research into
methods to allow both humans and objects to be located and tracked
seamlessly and in real time. The applications of this technology range from
Visual Surveillance to Virtu
al Reality of the media and production industries.
For instance, constructing a smart room that can anticipate the needs of the
user and the creation of tele
virtual newsreaders.

What I was particularly interested in, initially, for my own practice was
markerless motion capture which allows the 3D pose of an individual and its
motions to be extracted without the need for sensors, calibration or
preparation. Magnetic or optical m
otion capture has been used widely in
performance and art practices for some time now. This involves the
application of sensors or markers to the performer or artist’s body. The
movement of the body is captured and the resulting skeleton has animation
ied to it. This data projected image or avatar (Hindu: manifestation of a
deity or spirit) then becomes some part of a performance or art practice. For
instance, in the most recent work of Merce Cunningham,
, an absolutely
stunning but ultimately dist
ancing performance, pre
recorded dancing avatars
were rear projected onto a translucent screen giving the effect of a direct
interface between the physical and virtual bodies.

However, the use of markers and the accompanying technology cause quite a
few r
estrictions to the performer’s movements, for instance, mainly having to
remain within certain spatial constraints. There is also the need for extra
preparation and calibration and if the physical and virtual interface is in real
time then aesthetically th
ere are some problems. So with the development of
markerless motion capture, these restrictions are removed. This particular
research is still being developed.

Our main collaborative project that we are currently working on, ‘Interaction,
Reaction and Per
formance’, involves direct interaction between an avatar and
a physical performer. The physical work was initially a text and movement
based piece which was performed at Brunel University in 2000. The text titled
Blue Bloodshot Flowers

and written by Phil
Stanier involves the remembrance
of a love affair. There is some ambiguity on whether it is between two adults
or an adult and a child. Also if the narrator is dead

the ex lover is obviously
long gone. The performer, Elodie Berland, is French and we used

a French
voiceover as a memory device in the initial performance with good effect.
When we decided to combine the piece with interactive technology we initially
wanted an avatar which would be female and perhaps a child to represent the
child of the affai
r or the inner child. However, this all seemed too literal and
when we saw Jeremiah we immediately wanted him in the performance and
decided to leave it to the audience on how they would interpret this virtual
presence. Though, of course, most people would

assume it was the image of
the departed lover.

Jeremiah is a computer generated animated head based upon Geoface
technology. He has a simple bone structure which allows him t
o express
himself and emotions with which he can become angry or sad. But most
importantly he has eyes with which he can see. He doesn’t only interact but
also reacts. In fact he possesses artificial intelligence to the degree that he
can demonstrate sever
al emotions as a reaction to visual stimulus. Jeremiah
is unique in that he embodies intelligence that is no way prescriptive.
Therefore, the performance is a direct interaction between performer or
audience and technology.

One of the most interesting aspe
cts of this project is how much the spectator
projects into the avatar. Jeremiah, as we know, is computer technology
programmed with some artificial intelligence and has the ability to track
humans or objects. However, the interaction with him is anything
objective. Most people when they first see Jeremiah find him fairly spooky.
After the initial contact people tend to treat him in the manner of a small child
or a family pet and behave accordingly. Usually trying to make him smile and
generally please
him. His face becomes so sad when he is left alone that it is
becomes quite difficult to walk away. Although he is programmed with
emotions to react to certain stimuli, he can demonstrate fairly random
behavior that can be fairly disruptive during a perfor
mance adding a further
dimension to experience of working with a virtual body.

‘Interaction, Reaction and Performance’ is a pilot scheme for future projects.
And it is a feasibility study in as much as if this is successful we would want to
develop the te
chnology even further. At the moment we are discussing the
introduction of hearing and speaking to the avatar and another head is being
constructed which can morph between male and female. Though, we are all
very reluctant to lose Jeremiah at least in this


From a technological perspective (Bowden, 2001), Jeremiah is based around
two subsystems; a graphics system which constitutes the head and a vision
system which allows him to see. There is also built in a simple emotion engine
which allows him to

respond to visual stimuli via expressions or emotions. We
are currently working on a new head model called Rachel (see above) who is
a deformable eigen head providing more lifelike appearance and motion.

The system is capable of running on a single PC but

for speed of operation
each subsystem runs on a dedicated PC connected via a network crossover
and is, therefore, self standing and also now, due to the construction of a flight
case, truly portable.

The head is based upon the Geoface articulated bone mod
el with prescripted
expressions for key emotions. It contains a simple Newtonian model of motion
with random elements of movement and random blinking and ambient motion.

The vision system is based around a per pixel Gausian mixture model of
colour distribu
tions, using expectation maximization within the Grimson
tracker framework with additional shadow suppression and noise removal
algorithms. This allows static background scenes to be learnt dynamically
providing a robust segmentation of foreground objects.

Jeremiah’s attention is
randomly distributed between these objects weighted by their size and
motion. Co
ordinates of objects within the field view are sent to the head
model for animation.

The emotion engine determines the current state of emotions from
parameters extracted from objects of interest within the visual field. This
simple set of rules allows chaotic behaviour in a similar fashion. For instance,
Jeremiah likes visual stimulus

high rates of movement make him happy. He
likes company

o stimulus makes him sad. He doesn’t like surprises

rates of change in the size of objects make him surprised. Jeremiah doesn’t
like to be ignored

if objects exist but don’t move then he assumes they are
ignoring him and hence gets angry. Also, if

Jeremiah experiences too much
pleasure due to too much of any particular stimulus, he will get bored and
reduce its influence on him.

Since this research project is a science and art collaboration, there are
marked difference in the research rationale and


For Richard,

the Turing test describes a system as artificially intelligent if a
human user cannot distinguish the system from another human in
conversation. He is attempting to test this concept of intelligence by providing
an interactive hum
an avatar with simple rules and chaotic behaviour.

believes the interactivity and human embodiment of Jeremiah is sufficient that
individuals see him as a living entity. Therefore, Richard’s foremost question
is ‘How real can

? How do we interact with A' Life?’
(Bowden and Broadhurst, 2001).

I, on the other hand, am interested in a more arts related research rationale
and questions. Since, as I have argued elsewhere, language without the body
does not ‘mean’ at all, as corporea
lity provides language with meaning under
cultural and thus temporal constraints (1999b: 17), what then are the
implications for a virtual body? Therefore my overall research question is:

Due to such developments as artificial intelligence and motio
capture becoming increasingly prominent in art practices, does
this interface between the physical and virtual body give rise to a
new aesthetics? What then are the theoretical and practical
implications of this?

My aim is to explore and analyze the eff
ect these new technologies have on
the physical body in performance. Especially in relation to the problem of re)
presenting the 'unrepresentable', that is the sublime of the physical/virtual
interface (or liminal space). Underpinning this is a series of s
pecific research

What are the effects of new technologies in the analysis of the
performing body?

What are the theoretical implications of virtual performance for the
body and space?

What are the implications of, and how do we theorise, the r
esultant de
stabilisation of identity and origin?

What is the potential for participation and interactivity, inter
and spectatorship, within this new art practice?

In conclusion, this is an ongoing project with the first public performance to b
staged at the 291 Gallery, London in August of this year and a private
performance to be presented at the Media Lab, MIT later in the year. The
performance process has proved extremely stimulating and in some ways
may prove ultimately more beneficial for

research purposes than the actual
finished product.

Finally, although much interest is directed toward such new technologies as
Jeremiah, it is my belief that technology’s most important contribution to art
practices is the enhancement and reconfiguratio
n of an aesthetic creative
potential which consists of the interaction and reaction with a physical body,
not an abandonment of that body. For, it is within these tension filled liminal
spaces of physical and virtual interface that opportunities arise for
experimental forms and practices.


Bowden, Richard. ‘The Human Tracking Project’ (Unpublished Paper), 2001

Bowden, Richard and Broadhurst, Susan.
Interaction, Reaction and
, 2001.

Broadhurst, Susan.
Liminal Acts: A Critical Overview of Contemporary
Performance and Theory,

London: Cassell/New York: Continuum, 1999a


'The (Im)mediate Body: A Transvaluation of Corporeality',
Body & Soci

5(1), March 1999b: 19


Blue Bloodshot Flowers
. Brunel University, June 2001.

Merce Cunningham Dance Company.
Barbican, London, November

Stanier, Philip. ‘Blue Bloodshot Flowers: Text for Performance’
. Body, Space,
& Technolo
gy (Vol.1 No.2), 2002

Turner, Victor 'Are there universals of performance in myth, ritual, and
drama?’, pp. 1
18 in Richard Schechner and Willa Appel (eds)
By Means of
. Cambridge: Cambridge University Press, 1990.

Blue Bl
oodshot Flowers:

Text for Performance.

Philip Stanier


I remember everything all too well. I am speaking of departure, severance. As
we were wedged together at the knees so frequently so hard. There was little
dividing us, so, severance. This is de
parture again. Another split. Doubled
over. We were broken down and drained, he more than I. We parted once he
left me laid out in the flowers and I loved him for it. This is his corpse what I
now make with my tongue, these are his remains raked through, p
loughed up,
and cried over. His breath rattled, mine he called quiet algebra. He could only
make out my face this close enough to see my lack of lines, to see the
difference between the two of us between the mushroom soft escarpments
underneath his eyes an
d the drum tight lids of my eyes, but too close to make
out the whole. He could never see me at a distance. It was always too close.
He would never know I was on the horizon to call and retrieve me, too close
or nowhere at all. This is my dismissal of him,

my move to the horizon. His
redemption. I say this in the cold. Anger has no place here. Any emotion
would be synthetic. This is nostalgia gone rotten like my insides all leather
and dust. I am speaking of a glint in his eye that was unsuitably framed. A
context that made what happened unpleasant to other eyes, the speaking of it
uncomfortable, it made peoples skin crawl. I am speaking of an attempt to
gain distance, from a fatal event. I won’t be able to repeat this enough. I had
him inside me, I had him
deep inside me, So deep in my heart, he is still really
a part of me, I had him inside me. I won’t be able to repeat this enough. It was
natural for us both, so much for childhood reasoning. He thought more
childishly than I. No style or rhetoric of though
t. Just patterns in actions. The
dispersal of flower seeds in the air, the spiral of geometry in nature. /Hold
hands he said we did. A walk he said we did. Your smile he said I did. I’m lost
he said I was. I’ve fallen he said we had. No more he whispered I

left. That’s it
all, how it began how it ended. It could have seemed so much like chance,
coincidence. But this is no eternal story, there is neither chance nor anger at
work here. Just the pedestrian fear of an airtight alien logic. I have to confess
re were more complicated requests later the arrangements of bodies and
movement with numbers and names but it was the same uncomplicated
thought. /Kiss me he said I did, lick me he said I did, suck me he said I did,
fuck me he said I did. Simple for the mu
cous mind. Thick and slow but liquid
and impossible to damage. Inevitable. We had already begun before he spoke
as much as he spoke before we began. Like the snake that eats itself tail to
head the vanishing circle chicken before egg before chicken before
before horse head over heels. He sent me spinning, reeling occasionally with
the back of his hand. I have too tell you that, it is my duty to tell you that. I did
not live too many days after that. But I can’t repeat this enough. I tried so not
to fal
l in, I told myself this could never end well, how could I resist, when I
knew so well that I had you inside me. But I can’t repeat this enough.


He marked me out, and mapped my surface. His hands moving over my skin,
fingertips rough like tiny movi
ng cancers stretching out, trying to leap his
broken down frame into mine. There was a calm arithmetic in his palm.
Strange how the body in repeating itself cell by cell, breeding, doubling,
strange how these little clones can reject the persistence of the

towards age. Set up their own little colony. I’m sorry I’ve lost you, my
metaphor was cancer as a symptom of a fear of death, that simply hastens it,
hastens death that is, the result of a paranoid body. Was he my cancer. I was
his, he was more af
raid of death. while he was with me, I reminded him of his
age and we wore each other out, but while I fed and grew there was less of
him each time. The skin on his hand hung bare like a glove, each day the
bones were thinner. I was against his skin but th
ere was no touch, the surface
was cold. His breath rattled more. We walked. Our daily average 5 miles I was
only a child, I would get tired. But those walks led to calm, great calm in the
time of sleeping or half sleep with him, when it was twinned with my

exhaustion after he wore himself out on me in the shelter of the forest. The
weather was not often kind, we walked in bent double downpours, the result
of overzealous butterflies pollinating orchids in Brazil. The pelting downpour
covered several darkenin
g equators. Windless day and night. Through fields
of a million blue bloodshot flowers. Battered shattered by the rain, broken
glass fault
lines etched in the petals. Sap and mud like blood underfoot,
always cold. Wet hair in thick coils, fingers running t
hrough. As I went down
on him. Those that went like worms through blades of grass. And the wet
ground cradled us and we were wedged together at the knees snapping
buttercup stems. Eating the flowers, despite the blankets there were too few
for sustenance.
I starved with a full belly. Now I walk my daily average 1000
miles internally. I have returned to zero so many times, I lost track the
distance covered, I deal in speed and daily averages. I don’t know when he
was ever silent or still. Always fumbling for

intimacy. Always fumbling the
anatomy of my disgrace. The other things future quiet crying and empty
returning searches could not have been known in view of time and shade.
Everything was too close to see the horizon. I won’t repeat this enough.
Cause I’v
e got you deep inside me. I won’t repeat this enough


When it happens the echo was noticeable in my skull. Ringing. When he took
me with delayed communication. When he panicked. When he shook my
head with that rock. He sometimes seems weak and distan
t, though he was
strong that time. The sweat off his body stank of his age and ran down the
crevasses in his skin. Mine still made its own random path across marble, and
smelt only of salt. But I didn’t live so many days after that. The violence of the
a had germinated the first time, but now it blossomed. Blue green blooms
on my skin, red roots from my nose, mouth and between my legs. Pretty
flowers from my skin. My body aged no more. My skin did not let go of the
muscle, the muscle did not let go of th
e bones, the bones did not let go of
each other. At least not through any living process. I did not invite decay, it
made itself a guest. The glint in my eye suitably framed by thin dry skin tight
on the near hollow socket. I did at the very least welcome
that on the inside
where time and his old body were laid bare and raw. On the inside where
neither time nor his old body hung bare. Please note that when I said he hit
me I lied at least not before he brushed my hair with earth and granite. Please
note whe
n I said we ate flowers I lied though I chewed them with my face in
the ground. Please note when I said that there were a million blue bloodshot
flowers I guessed there could have been more there could have been less.
I’ve got you inside me. I’ve lost you
inside me. I’ve left you inside me. I’ve kept
you inside me. I remember everything all too well and so I am speaking of
severance, departure. I can’t repeat this enough. My fumbling progression
through counting weeds those unwanted flowers keeps me occupie
d. I am in
the earth. I am roots and creepers. I am inside you.