Shared Representation and Joint Attention in Human-Robot ...

fencinghuddleAI and Robotics

Nov 14, 2013 (3 years and 7 months ago)

98 views



EU


FP7






AMARSi

Adaptive Modular Architectures for Rich Motor Skills


ICT
-
248311


D
1.1


October 2010 (6 months)




Comparative evaluation of notions of modularity in
human motor control and of existing algorithms for the
identification of motor
primitives




Martin Giese
1
,
Andrea d’Avella
2
,
Tamar Flash
4
,
Yuri P. Ivanenko
2
, Thomas Schack
3
,
Enrico Chiovetto
1
, Albert Mukovskiy
1
.



1
-

Section for Computational Sensomotorics, Department of
Cognitive
Neurology
,

Centre for Integrative Neuroscience
,
University Clinic Tübingen
,
T
übingen
, Germany.

2
-

Department of Neuromotor Physiology
, Santa Lucia Foundation, Rome, Italy.

3
-

Neurocognition and Action Research Group, Faculty of Psychology and Sport
Science, University of Bielefeld, Bielefeld, Germany.

4
-

Departm
ent of Computer Science and Applied Mathematics, Weizmann
Institute of Science, Rehovot, Isreal.





Due date of deliverable

1st September 2010

Actual submission date

15th October 2010

Lead Partner

Tübingen University

Revision

Final

Dissemination level

Public



Comparative evaluation of notion of modularity

in human
motor control and of existing algorithms for the
identification of motor primitives


There exists a broad range of biological approaches to movement primitives that covers different
functional
levels and different

methodological approaches. Many of the existing approaches are
represented in the AMARSI consortium. This deliverable will, on
the one hand, provide a brief
overview of the approaches towards movement primitives that have been established by the different
partners. Possible relationships that might form a basis to establish collaborations between the groups
and specifically with t
he technical groups in robo
tics will be briefly discussed.

On the other hand, in the
second part of this deliverable, some quantitative work will be presented that compares between
methodologically closely related approaches of different groups
.

Such compa
risons are possible only
between approaches that conceptualize movement primitives in similar ways, and at the same
representation level
.




1 Overview of different biological approaches towards
motor primitives


The concept of movement primitives is

quite popular in research in neuroscience. At the same time,
the use of this concept is very heterogeneous, and many different conceptualizations of motor
primitives appear in the biological literature. Some of these conceptions address phenomena at simil
ar
levels, but using different methodologies. Others signify phenomena at very different representation
levels, such as muscle synergies or cognitive action plans. The AMARSI consortium comprises groups
that have worked on different approaches for the char
acterization of movement primitives, and which
cover a large part of the existing approaches. Some of these approaches are closely related, while
others address completely different representation levels
. T
able

1

giv
es an overview of the
approaches and con
ceptualizations of motor primitives that are represented within the AMARSI
consortium.


Partner

People involved

Level

Theoretical / experimental
approaches

Section

UniBi

T. Schack,
B.
Land
, A.
Krause,
B.
Blaesing

Cognitive
representations

Volitional initiation

control strategies

1.1

WI

T. Flash, A.
Barliya, R. Fuchs,
Y. Meirovich

Kinematics, strokes /
trajectory segments

Differential geometry, invariants,
kinematic analysis

1.2

SFL

A. d’Avella, Y.
Ivanenko
, G.
Cappellini

Neuromuscular
synergies

/

kinematics

Unsupervised learning, EMG, motion
capture

1.3


UniTu

M. Giese, E.

Chiovetto, A.
Mukovskiy, N.

Taubert

Kinematics, movement
sequences

Unsupervised and supervised learning,
dynamical systems, motion capture

1.4




Table 1
.

Biological approaches for the investigation of movement primitives within the AMARSI
consortium
.


In the following, we will first sketch the definition and approaches for the identification and modelling of
motor pr
imitives
provided by the different partners that are involved in the analysis of biological data.
Finally, we will provide a comparison that highlights similarities and complementarities that might help
to establish fruitful interactions between the different part
ners.



1
.1

Neurocognitive perspective on motor p
rimitives


A very general definition of motor primitives is underlying the work by the partner Un
i
Bi
. This
conception includes a variety of levels form low to high
-
level representations, and it addresses
specifically the cognitive aspects of the representation and control of complex movements.
A particular
focus of this approach is the planning and memor
y of complex movements and changes associated
with skill learning and during development.


1.1.1

Notion of Motor Primitives (MP)


Motor Primitives

are
conceptualized as
basic building blocks in a modular motor control architecture
(cognitive architecture of human motion). These building blocks are
functionally relevant elementary
components or transitional states of (complex) movements. In our point of view it is possible

to
understand a movement as a serial and functional order of significant and goal related body postures.
Such
body postures and motion sequences are bi
-
directional linked with perceptual (movement
-
)
effects and typically stored in memory in a hierarchical

order.

A main topic of our research is the cognitive architecture of human action, showing how it is organized
over several levels and how it is built up.

Alongside
Bernstein's (1947)

approach to the construction of
action, there have been several formu
lations of the idea that movement control is constructed
hierarchically.
In contrast to most of these approaches, the model proposed here views the func
tional
construction of actions
on the basis of a reciprocal assignment of performance
-
oriented regulatio
n
levels and representational levels (see Table

2
). These levels differ with respect to their central tasks
on different

regulation and representation levels. Each level is assumed to be functionally
autonomous.


Code

Level

Main function

Subfunction

Means

IV

Mental

Control

Regulation

Volitional initiation

control strategies

Goals; goal posture
representation; strategies

III

Mental

representation

Representation

Effect
-
oriented
adjustment

Basic action concepts

II

Sensorimotor

representation

Representation

Spatial
-
temporal
adjustment

Perceptual effect

representations

I

Sensorimotor
control

Regulation

Automatization

Functional systems;

Transitional states



Table
2.

Levels of Complex Motion in Humans (Schack, 2004)
.


We identify B
asic
Actio
n Concepts (BACs)

as maj
or building blocks
a
t the

representation level.

BACs
are based on the cognitive chunking of body postures and movement events concerning common
functions in realisation of action goals. Their characteristic set of features resu
lts from the perceptive
and functional properties of action effects: They tie together their functional and sensory features.
These functional features are derived from action goals

(goal postures)
, and this connects BACs to
Level IV. However, BACs also in
tegrate sensory features of sub
-
movements, for example, through
chunking
.
As a result, they also refer to the perceptual effects of movements. This connects BACs with
Level II. All together BACs can be viewed as the mental counterparts of functionally rele
vant
elementary components or transitional states of complex movements. They are characterized by
recognizable perceptual features. They can be described verbally as well as pictorially, and can often
be labe
l
led with a linguistic marker. "Turning the head
" or "bending the knees" might be examples of
such basic action concepts in the case of, say, a complex floor exercise.
Based on our research we
learned that such movement representations might provide the basis for action control in skilled
voluntary move
ments in the form of cognitive reference structures.

With respect to

robot control
, current work
is
largely focused on a low level of abstraction that is very
close to the sensors and actuators. In contrast, human actions are heavily informed by huge amounts


of representations about the goal and the characteristics of the anticipated movements, the
encou
ntered objects, and about how to counteract the n
umerous disturbances and mistakes

that
usually occur during even moderately complex movements. Therefore one major goal of our
research
is to design
experiments and simulations
concerning the formation and c
ooperation of motion building
blocks at a cognitive level (memory) and to learn about
the integration of biomechanical and
motor
constraints into
cognitive
mot
or representations

and cognitive motor hierarchies
.



1.
1
.2

Methods and experimental evidence


1.
1.2
.1

Experiments to study the cognitive representation of MP’s (BACs) in memory


We used
different
experimental methods to study:




cognitive structures in long term memory in different types of movements



movement based chunking in short term memory



representation
s

of power
-

and precision grasps in memory



representation
s

of manual action after stroke



the overlap between kinematical and cognitive movement structures


Summarizing the expertise studies, we found

t
hat in high
-
level experts

the represen
tational
frameworks were organized in a distinctive hierar
chical tree
-
like structure, with remarkable

similar
ities
between individuals. These frameworks were well
matched with the functional and biomechanical
demands of the task. In comparison, action repr
esentations in low
-
level athletes and
non
-
athletes

were organized less hierarchically, were more variable between persons, and were less well matched
with functional and biomechanical demands.



Figure 1
.

Representation structures for two chosen tennis expertise groups
, respectively
experts

and

non
-
players

(A and B),

based

on the hierarchical cluster analysis of basic action concepts (BACs) in the
tennis serve.

The horizontally aligned numbers denote the B
ACs (for the code, see

text); the vertical numbers
specify

the Euclidean distances. The lower the numbers, the lower the distances between BACs in long term
memory.
For all

group
s

holds
n

= 11;
p

= 0.05;
d
crit

= 3.46 (
from Schack & Mechsner, 2006
).


Results from two different lines of research addressing the mental representation level showed that
not only the structure formation of representations in long
-
term memory but also chunk formation in
working memory are built up on BACs and relate systemati
cally to movement structures. Experiments
were designed to assess both the structure of mental representations in LTM (determined with the
SDA
-
M) and chunking in working memory (determined with Cognition and Movement Chronometry,
CMC). Results confirm
ed
the interaction between long
-
term memory an
d short
-
term memory,
demonstrating that cognitive systems interact to produce complex movements. Our experiments have


shown that both
,

the order formation in LTM and the chunking in working memory build on the
top
ological (spatiotemporal) structure of the movement. This provides experimental evidence that
structures in movement and memory mutually overlap.


1.
1.2
.2

Experiments to study the relationship of cognitive motor representation and
biomechanical constraints



To accomplish

a
better
understanding of the cognitive architecture of complex movements, it is not
only interesting to know whether LTM and working memory cooperate horizontally on, for example,
one level of mental representations. It is, for instance,
also crucial to know whether there is a vertical
cooperation between the level of mental representations and the level of sensorimotor control. One
could ask whether biomechanically relevant features can be found in the structure of mental
representations.

Experimental studies (
Schack,
200
4
;
Schütz et al.
,

2009
) showed that
representational frameworks were organized in a hierarchical

tree
-
like structure and revealed a good
match with the biomechanical demands of the task. After measuring kinematic parameters, we
investigated the relationship between the structure of motor representation and the kinematic
parameters of different moveme
nts. Our studies have revealed significant correlations between
kinematic parameters (time structure, angles according to the take
-
off
-
phase, tilt angle, angular
velocities, etc.) of movement and the corresponding parts of mental representations.

Other stu
dies are done to find out the relationship between anticipated goal states (end
-
state
-
comfort)
and the representation of functional grasp constellations in children (Weigelt & Schack, 2010;
Stöckel,
& Schack, 2010
).
All together our experimental results su
pport the hypothesis that voluntary
movements are executed and stored in memory directly through representations of their anticipated
perceptual effects.

We are combining experiments concerning the cognitive representation structure in
motor memory with PC
A or a new kind of spatiotemporal analysis (
Bläsing & Volchenkov, 2010)

to
identify MP’s and to learn about the relationship of cognitive and biomechanical motor constraints
(MP’s).


1.
1.2
.2

Simulations and neural network a
rchitectures


In
order to model the
learning an
d generation of

complex movements,
a hierarchical, modular
neuronal network architecture is currently under development. The architecture will be able to learn
demonstrated movements (kinaesthetic teaching) and
-

after
trainin
g

-

trigger adaptive movement
execution based on anticipated sensory data from the robot and the environment.

The idea is to have
-

on a lower level
-

a battery of recurrent networks controlling the execution of
automatically learned movement primitives i
n

a tight sensory motor loop.
The dynamics of those
network
s

is controlled through bifurcation
by
inputs (similiar to the work by Jun Tani regarding RNNPB
with parametric bias units) by a higher level network that selects
-

and interpolates between patterns

and generates long motor sequences. On the top level of the architecture, on
e or multiple hierarchical
self
-
organising maps (HSOM) represent the cognitive structures of the complex movements (BAC's,
see Schack

&

Mechsner
,

2006). The HSOMs can analyse and
cluster sensory data, detect proper
situations when to perform suitable movements and adjust task affordances

(Krause

et al.
,

2010)
.

In addition, o
ther
computational approaches
for
the
s
imulati
on of

the
s
tructure of
m
ental
r
epresentations
are being developed
(Tscherepanow

et al.
,

2010).
Such

biologically inspired model
s
of

movement

representations are

to be contrasted with other
,

les
s biologically plausible models, such
as discussed in W
P4 for motor primitives.


1.1.2

Implications for robotics


Because the production of manual actions is affected by a number of factors, such as biomechanical
constraints

(Weigelt

et al.,

2009), a line of our studies
focuses on the question of how structures of


sensory
-
motor representatio
ns are established and
chan
ged
in a stepwise manner
under
the
consideration of task constraints

(
Bläsing

et al. 2010b
;

Maycock

et al.,

2010).

In these studies it
is of
interest to learn about the relationship between the structure of
mental
representations and the
performance in manual actions,
especially in

situation
s

in which actions result in error
s
. To this end,
we combine
d

methods from cognitive psychology, biomechanics, and computer sciences (i.e.
augmented reality / virtual reality sce
narios) to assess the structure of sensory
-
motor representations,
to introduce experimental manipulations for manual actions, and to examine the resulting performance
changes (including changes in erroneous performance). Th
e insights gained in these first
experiments
will be implemented on

robotic platforms (i.e. a 7 DOF robot arm set
-
up)
with
in a longer research
perspective.

Our research concerning the cognitive architecture of human action (
Bläsing

et al.,

2010
a
;

Schack,
2004; Schack, 2010) is of
interest for the construction of robot architectures and vice versa. In different
research project
, we translate
d

our findings in
studies
of

cognitive factors in

human
motor control into
models that can guide the implementation of cognitive robot architect
ures. Focusing on the issue of
manual action control, we illustrate
d

some results in the context of grasping with a five
-
fingered
anthropomorphic robot hand

(Schack & Ritter, 2009).

A research project about
cognitive planning of action sequences and sensor
imotor adaptation

(cooperation with the CoR
-
Lab and HRI Europe)
addresses the cognitive and perceptual dimensions
of goal posture planning in humans and is related to
grasp optimization and trajectory
planning
in
robots. The aim

here
is
to integrate the research results in humans
(e.g. Weigelt & Schack, 2010)
into
a comprehensive framework, and posture based movement representation, that

allows
an
efficient

realisation of
a fluent robot grasping.

Our studies a
bout cognitive representation

of

complex motor actions (e.g. Schack & Hackfort, 2007;
Schack & Mechsner, 2006), are of relevance for a
control of full body movements in robots. We
created a
research lines from an

experimental analysis to a computational modelling of full body
movement
s

(Krause et al.
,

2010).
In this
research line
, we want to investigate complex movements of
the whole human body
,
as they

occur in sports, dance or other expert tasks

(Bläsing et al.
,

2010
a
)
, but
also in every
-
day life,
in order to

understand how these movements are controlled, learned and
reproduced under changing conditions. Our aim is to apply different biomechanical and psychological
methods to analyse the kinematics, force profiles, muscle activity, mental representations and o
ther
cognitive control structures, and to develop a computational model that integrates the results of the
different measures into a simulation of the movement. Results of th
e
s
e

stud
ies
could be helpful on
many levels for the general understanding of human

movement control, but also for movement control
in artificial agents and robots, especially if we are interested in “human
-
friendly” interaction between
humans and technical systems.

Cognitive representations are reference frames in the implementation an
d control of human motor
action (Schack & Ritter, 2009; Zentgraf et al.
,

2009). For a better understanding of the structure and
functionality of such reference frames we planned in cooperation with Rolf Pfeiffer and Rüdi Füchslin
(ai
-
Lab, Zürich) different

experiments about principles of tool use and tool manipulation

in humans. In
a next step we are going in direction of a computational modelling and a technical implementation of
functionally structured motor representations in robotic platforms. The aim i
s here to address in a
longer research perspective the “frame of reference problem” in robotics from a new point of view.


1.2

Motor primitives at the level of kinematic invariants


A

number of groups i
n the AMARSI consortium
address

the problem of movement primitives at the
level of kinematics. A very generic approach is followed by the partner WI who characterizes
movement primitives by their geometrical invariants. This approach works at a very basic level and
does not require high
er cognitive representation
levels. This conception is particularly suitable

for the
segmentation
and modelling
of action streams and
of complex movements that are modelled by
sequencing of simpler segments.



1.2
.1

Notion of Motor Primitives
(MP)


In dealing with the modularity of the motor control system the focus of the Weizmann group is on the
level of trajectory
planning,

but studies are also carried out which are dealing with motion planning at
the joint level. The te
rm motor (or motion) primit
ive

refers to elementary movements or building blocks
from which more complicated movements are constructed. In studying the issue of motor
compositionality one of the main goals is to be able to infer motion primitives from movements that are
apparently c
ontinuous. Hence
,

the main questions in investigating the nature of motion primitives and
compositionality are as follows. a) Can we identify an alphabet of motor primitives from which more
complex behaviors are constructed? b) What is the nature and int
ernal representation of such
primitives? c) What generation rules are used to generate or span an entire motor repertoire of either a
single or several motor tasks from a limited set of elementary movements
?

d) What syntactic rules are
used in joining toge
ther motor elements?

In particular in attempting to identify and characterize motor primitives our approach is based on taking
advantage of the existence of motor invariants and templates at the level of trajectory planning. These
invariants consist, for
example, of the invariance of hand paths and velocity profiles during reaching
movements or of the two
-
thirds power law observed for curved and drawing movements. These
kinematic laws of motion can also be accounted for by several optimization models, spec
ifically the
minimum jerk model (
Flash and Hogan, 1985
)
, but

also by more general maximum s
moothness
models (MSD models).
Earlier it was suggested that the observation of a piecewise segmented
power law and the change in the value of the velocity gain fa
ctor can be considered as e
vidence for
motor segmentation.

This claim, however, is problematic (Flash and Hochner, 2005) and is not
consistent with the findings reported in Richardson and Flash (2002)
who

have shown that global
optimization gives rise to a

similar piecewise constant relationship between the logarithms of speed
and curvature without any assumption about segmented control. More recent studies, however, have
suggested a somewhat alternative explanation for the power law and hence an alternati
ve approach to
segmentation, i.e., the approach based on equi
-
affine differential geometry and specifically on the
observation that the two
-
thirds power law is equivalent to moving at a constant equi
-
affine speed
(Flash and Handzel, 2007). In particular
,

w
hen this new framework was applied to the analysis of hand
trajectories it has led to a new focus on new geometric metrics and invariants which have included
the
equi
-
affine arc
-
length and
equi
-
affine curvature which remain invariant under equi
-
affine
transformations.

This approach was then extended to 3D drawing and curved movements by showing that the
hypothesis that equi
-
affine speed is kept piecewise constant also during 3D hand trajectories leads to
the formulation of a generalized power

law whereby hand speed depends both on movement
curvature and torsion (Pollick et al
.
, 2009
;

Maoz et al., 2009).

The equi
-
affine analysis has also served as a basis for developing a more general group theoretical
approach to the study of motor invariant
s and segmentation and to the suggestion of several new
ideas (Flash and Handzel, 2007
;

Polyakov et al., 2009
a, 2009b
): 1) Movement states within the motor
space might be char
acterized by their equi
-
affine

differential invariants 2) A large variety of
move
ments might be achieved by applying equi
-
affine transformations

on such limited number of
state
s and by combining them
.

This theoretical approach has also led to a series of combined behavioral, neurophysiological and
computational studies that gave some

validity t
o this idea (see below, Polyako
v et al, 2009a, 2009b).

An important regularity observed during human movement which was not accounted for by the equi
-
affine framework is

the isochrony principle, i.e.,

movements of different lengths (amplitudes
) having
nearly the same movement duration (Viviani and Flash, 1995). Also no principled rule for the selection
of the velocity gain factor was suggested. Hence more recently we have generalized the equi
-
affine
description to a new and broader theory based

on geometrical invariance. The new notion is that
movement duration and compositionality arise from cooperation among Euclidian, equi
-
affine and full


affine geometries where each geometry
possess

a canonical measure of distance along curves, an
invariant
arc
-
length parameter (Bennequin et al
.
, 2009).

The theory was mathematically formulated and its predictions were tested on three data sets:
drawings of elliptical curves, locomotion and drawing trajectories of complex figural forms. This new
theory sugg
ests new notions about motion compositionality and segmentation.






















Figure
2
. Equi
-
affine parameters for a parabolic
-
like recorded movement segment
.

a.
Equi
-
affine velocity
(dots) and curvature (asterisks) for a scribbling segment.
b.

The actual drawing made by the monkey. Since
parabolas are characterized by zero equi
-
affine curvature, the motion segment can be well
-
fitted by parabolas,
(dashed lines). Figure taken from Polyakov et al., 2009b.



1.2
.2

Methods and
experimental evidence


The following pieces of experimental evidence support the discussed theory:


a)

Earlier evidence for the existence of motion primitives at
the kinematic levels comes from

work
concerning

the principle of superposition of elementary point
-
to
-
point to point movement
s

from
a study of trajectory modification (Flash and Henis
,

1991
;

Henis and Flash, 199
5
).

b)

A co
-
articulation phenomenon was observed in a study involving continuous practice durin
g
the generation of hand
-
writing like trajectories. The kinematic properties of well
-
practiced
movements were successfully accounted for by the maximum smoothness principle and newly
formed curved primitives were shown to emerge following sufficient practi
ce (Sosnik et al.
,

2004).

c)

In relation to the above geometric approach to movement segmentation and compositionality
based on the equi
-
affine framework (
Handzel and Flash, 1999; Flash and Handzel
,

2007;

Pollick and Sapiro, 1997) we have derived a necessa
ry mathematical condition on paths for
which the predictions of both the 2/3 power
-
law and the constrained minimum
-
jerk model
coincide (Polyakov et al.
,

2009
a
). Such paths must obey the following relation




0
6



r
r

(differentiation w.r.t. the equi
-
affine arc
-
length). Using this condition and requiring the
invariance of these paths with respect to equi
-
affine transformations, we have shown that only
for parabolic shapes, the 2/3 power law and the criterion of smoothn
ess maximization are
mathematically reconciled on the geometric level

yielding trajectory elements that are invariant
to equi
-
affine transformations
. We have also demonstrated how complex piece
-
wise parabolic
trajectories can be generated from a single par
abolic template


based on equi
-
affine
geometric transformations and uniform scaling. We have then examined the validity of our
theoretical analysis by fitting free monkey scribbling movements with basic parabolic strokes

(
see Figure
2
)

and found that foll
owing practice, these drawing movements could be
decomposed into only 3
-
4 well separated clusters of parabolic segments

(
see Figure
3
)
.


Defining a

movement primitive

as an

elementary stroke that cannot be intentionally stopped after its
initiation, we also found that when the monkey's motor performance was disrupted by giving a reward
at certain locations, the monkey indeed tended to decelerate and stop their movements but
not before
the completion of parabolic
-
like path se
gments. In additional studies,
the neural activities of multiple
single
-
units, underlying scribbling movements, were recorded in parallel from M1 and PMd during 8
recording sessions and have been segmented

in an unsupervised way based on a Hidden Markov
Modeling analysis (as in Abeles et al., 1995). In many cases, the movements corresponding to the
identified states of neural activities formed clusters of similar geometric shapes; some clusters
consisting o
f parabolic
-
like segments. By applying partial cross
-
correlation analysis (Stark et al.,
2007), we have found a stronger representation in the activities of several cells of equi
-
affine speed
rather than of Euclidian speed.


1.2.3

Implications for robotics


Obviously, all the above computational problems and approaches may apply also to robotic systems
where algorithms and approaches for the selection of motion primitives at the end
-
effector and joint
levels and their blending arise also in relation to robo
t arm movements, locomotion and multi
-
effector
movements especially for robot humanoids
,

but also to more conventional manipulators or mobile
robots.
An example is work in WP4 that models human motion using both, joint
-
level and end
-
effector
levels of cont
rol resulting in more effective reaching
(see Hersch et al., 2008; Calinon et al., 2010).










Figure
3
.
Emerging parabolic clusters and dimensionality reduction.

A. Typical histograms for the fitted
parabolic segments. In the one
-
dimensional histogram (left), the
segments are counted according
to their
orientation. In the color histogram (right), they are counted in distinct bins according to the orientation and f
ocal
parameter of the parabola. B. Location of the vertex and orientation of the parabola for every 10
th

parabolic
segment for the recording sessions. Locations of the vertices of the similarly oriented parabolas are also
clustered. The clusters are marked

by ellipses and the mean orientations of the parabolas within each cluster are
depicted by arrows (Taken from Polyakov et al., 2009b).



Another important application for robotics is the segmentation of natural movements into simpler
segments, which then can be supplied to learning algorithms and as basic segments for the design of
controllers. The proposed methods have the advantage that t
hey do not require any prior knowledge
or training data, such as supervised segmentation algorithms, as the ones discussed in section 1.3.4.




0
20
-200
-100
0
100
Orientation, [deg.]
Start of practice

0
50

# of strokes

0
2
4
0
1000
16
th
practice session

0
50

# of strokes

0
100
200
300
0
10
20
-100
0
100
Orientation, [deg.]
# of strokes
0
50

p, [mm]

0
2
4
0
100
200
# of strokes
0
50

p, [mm]

0
5
10
15
0
500
# of strokes
0
50

# of strokes
p, [mm]

0
50
100
-100
-50
0
50
Vertices and orientations
Position - y, [mm]
Vertices and orientations
-100
0
100
-100
-50
0
50
100
Position - x, [mm]
Position - y, [mm]
-100
0
100
Position - x, [mm]
-100
0
100
Position - x, [mm]
Vertices and orientations
1
2
3
1
2
1
2
3
1
2
3
2
1
3
2
1
Overtrained behavior
A
B
Monkey O
Monkey U
Monkey O
Monkey U


1.3 Motor p
rimitives

at the
neuro
musc
ular

level



One of the lowest levels for the definition of movement primitives is the level of muscle synergies.
Such synergies can be identified by application of dimension reduction methods to muscle activity
patterns or to kinematic data. The partner SLF has been l
eading in the research within this field within
the last decade. For selected classes of movements this work has accurately characterized then
organization of the associated muscle activation patterns and their changes associated with different
tasks. In s
ome sense, this characterization of movement primitives in biological systems is closest to
their implementation at the level of biomechanics and the control at the muscle level.


1.3
.1

Notion of Motor Primitives
(MP)


Two notions of motor primitives have been used by the SLF group to capture the modular organization
of the muscle patterns observed during human locomotion and reaching movements. The first notion
is that of a basic temporal component in the muscle activit
y patterns
(Ivanenko et al.
,

2004, 200
6
)
. T
he
second notion is that of a time
-
varying muscle synergy
(d'Avella et al.
,

200
3
; d'Avella et al.
,

2006;
d'Avella et al.
,

200
8
)
.

Both notions are based on the assumption that the central nervous system (CNS) generates the
muscle patte
rns appropriate for performing a task by superposition of a few basic motor programs.
However, each notion emphasizes different invariant features of these motor programs.

According to the first notion, the time
-
varying muscle activation vector





observed in a specific task
condition is generated by the combination of a

few basic temporal components






each associated
with

the synchronous activation of groups of muscles through a
constant
weighting
vector



:





















)

䅳 the tas欠捯nditions 捨ange, the tempo牡l 捯mponents a牥 invariant while the weighting matr
ix is
adjusted app牯p物atel礮

䅣捯牤ing to the se捯nd notion, the same time
-
va特ing mus捬e a捴ivation ve捴o爠is gene牡ted b礠the
捯mbination of a few time
-
va特rn
g mus捬e s祮e牧ies,
i.e. time
-
va特ing mus捬e weight






,
appropriately scaled in amplitude shifted in time:






















⠲)

whe牥



is the amplitude scaling coefficient and t
i

the onset delay for the
i
-
th synergy. In this case,
the
time
-
varying synergies are invariant across task conditions and the changes in the muscle patterns are
captured by changes in the scaling and timing coefficients.


1.3
.2

Methods and experimental evidence


Experimental evidence supporting both notions of motor primitives rely on
unsupervised learning
(
decomposition algorithms
)

to identify either temporal components and synchronous muscle
weightings or time
-
varying synergies and combination coefficients from
multi
-
muscle
electromyographical recordings obtained during the performance of a motor task in many different
conditions.

Muscle patterns recorded during human walking at different speeds, with different body weight
unloading, can be reconstructed by five
basic temporal components identified with factor analysis

(
see
Figure
4

and
Ivanenko et al.
,

2004, 2006)
. Similar components are identified by other algorithms such
as non
-
negative matrix factorization or independent component analysis
(Ivanenko et al.
,

2005)
,

see
Figure
5
.

Finally, five temporal components also capture the temporal organization of running muscle


patterns with only one of these components significantly different from those for walking
(Cappellini et
al.
,

2006)
, see Figure
6
.

The spatiotemporal organization of the phasic muscle patterns for fast reaching movements in
different directions in vertical planes
(d'Avella et al.
,

2006)

can be reconstructed by a small number of
time
-
varying muscle synergies identified by an iterative optimization algorithm developed for this
purpose

(Figure
7
)
.
The reconstruction of
muscle synergies

is robust against

changes in posture and
load
,

and the amplitude coefficients show cosine directional tuning.

Moreover, modulation of phasic and tonic time
-
varying muscle synergies captures the variations in the
muscle patterns observed i
n reaching
movements in different directions and with different speeds
, as
shown in Figure 8

(d'Avella et al.
,

2008)
. These results suggest that the central nervous system might
be using a simple scaling strategy for
generating the joint torque p
r
ofiles appropriate for moving along
a given trajectory

with different speeds.
As t
he equation of motion for an articulated arm are invariant
for changes in the
movement
time scale
(r)
if the
dynamic components of the
joint torques are scaled
in amplitude
by r
2

and

anti
-
gravity
torque
component
s are

not changed
.
Thus, if a torque profile
adequate for reaching a given spatial target at one speed is known, a simple scaling rule allows to
generate the torque profiles for reaching that target, along the exact same path, at different speeds.
A
low
-
dimensional represent
ation of the muscle patterns for reaching in terms of phasic and tonic
muscle synergies would greatly simplify the implementation of such a control strategy. Indeed, the
amplitude coefficients of the phasic synergies were found to scale with speed to a rel
ationship close to
quadratic.








Figure
4
. Locomotion program as a characteristics timing of muscle activation (Ivanenko et al., 2004).

Similar five activation components, identified with factor analysis, account for about 90% of variance in the leg
EMG patterns during walking at different speeds and loads.


95%
75%
50%
35%
0%
stance
swing
1 km/h
2 km/h
3 km/h
5 km/h
averaged
comp
1
comp
2
comp
3
comp
4
comp
5
stance
swing
speed
body weight
support
Locomotion
Motor Program
Locomotion
Motor Program
comp
1
3
2
4
CPG
5
1
TD
TD
LO
time

comp
1
3
2
4
CPG
5
1
TD
TD
LO
time




Figure
5
. Basic components.

Basic temporal components identified by different
statistical

approaches (factor
analysis, independent component analysis, non
-
negative matrix factorization and fitting by
G
aussian
components) from normalized EMG p
atterns during normal walking (
Ivanenko et

al., 2005).


1.3.3

Implications for robotics


Most models of adaptive modules proposed in the robotics literature or currently being developed as
building block for the AMARS
I

architecture use a representation of kinematic variables, since robot
dynamics is usually taken care of by low
-
level feedback control loo
ps. In contrast, the notions of

motor
primitives employed by the SLF group are mainly at the muscular, thus dynamic, l
evel. While the
fundamental differences in the nature of the actuators and sensors between biological and robotic
systems might suggest that a notions of motor primitive at the dynamic level are not immediately
relevant for robot control, such statement mi
ght not apply to novel biologically inspired robot platform
with compliant actuators. Indeed, one of the features of the muscle
-
based actuator systems is that
they are naturally compliant and that they allow for adjusting the mechanical impedance by regula
ting
the amount of co
-
contraction. How the control of the impedance of multi
-
joint systems is regulated by
the nervous system and whether the representation and control of impedance relies on motor
primitives (such as, for example, muscle synergies which g
enerate zero output force/torque but control
impedance of the limb along specific directions) are novel and open questions in human motor control
that might be highly relevant for the control of robots with compliant actuators.

Along these lines, a first a
ttempt to combine estimation of the dynamics of reaching movements
through dynamical system with estimation of the impedance parameters required to adapt to
uncertainties during transport was conducted by the technical partner EPFL in
Gribovskaya et al.,
2
011
(see also
AMARS
I

Deliverable 4.1
: Ch.8,
Stable Estimator of Dynamical Systems
).


7
7
EMGs
SPLE
BIC
TRIC
DELTA
DELTP
TRAPS
TRAPI
LD
RA
OE
OI
EST1
EST9
ESL2
GM
Gmed
ILIO
TFL
ADDL
SART
BF
ST
RF
Vmed
Vlat
MG
LG
PERL
SOL
FDB
TA
STER
Factor
Analysis
comp
1
comp
2
comp
3
comp
4
comp
5
Gaussian
fitting
stance
swing
Independent
Component
Analysis
Non
-
Negative
Matrix
Factorization
EMGs
decomposition
temporal
components




N
i
i
i
residual
t
f
t
1
)
(
)
(
W
m
PV
~
90%




Figure
6
. Five basic te
mporal components for human run
ni
n
g (Cappellini et al., 2006).

The timing of the
main peak of components is similar for walking and running except for the second component that is shifted in
running.



1.4 Motor p
rimitives

defined by learned kinematic components


The partner

U
niTu

takes the viewpoint of machine learni
ng. The

work has mainly focused on the
analysis and modelling of kinematics of body movements by the application of learning methods,
which are inspired by the concept of motor primitives form neuroscience. The derived

primitive
-
based
representations have been exploited, on the one hand, for the analysis of motor behaviour and th
e
perception of body movements.

On the other hand, we have developed methods that transform such
primitive
-
based representations into algorithm
s that are suitable for the offline and online synthesis of
body movements.

In addition, we have used primitive
-
based representations for the study of
perception of body motion.




B
7 km/h
1
2
3
4
5
walking
running
6
B
7 km/h
1
2
3
4
5
walking
running
6
7 km/h
A
walking
running
STER
SPLE
BIC
TRIC
DELTA
DELTP
TRAPS
TRAPI
LD
RAS
OE
OI
EST1
EST9
ESL2
GM
ILIO
TFL
ADD
SART
BF
ST
RF
Vmed
Vlat
MG
LG
PERL
SOL
FDB
TA
Gmed
7 km/h
A
walking
running
STER
SPLE
BIC
TRIC
DELTA
DELTP
TRAPS
TRAPI
LD
RAS
OE
OI
EST1
EST9
ESL2
GM
ILIO
TFL
ADD
SART
BF
ST
RF
Vmed
Vlat
MG
LG
PERL
SOL
FDB
TA
Gmed
STER
SPLE
BIC
TRIC
DELTA
DELTP
TRAPS
TRAPI
LD
RAS
OE
OI
EST1
EST9
ESL2
GM
ILIO
TFL
ADD
SART
BF
ST
RF
Vmed
Vlat
MG
LG
PERL
SOL
FDB
TA
Gmed
walking
running















Fig
ure

7
. Reconstruction of muscle patterns for
fast
-
reaching movements by combinations of time
-
varying muscle synergies (adapted from d’Avella et al., 2006).

Five time
-
varying synergies (A) extracted from
the phasic muscle patterns recorded during point
-
to
-
point reaching movement to targets in differen
t directions on
frontal and sagittal vertical planes explain the variation in the muscle patterns across directions (B) as due to the
selection of a small number of amplitude coefficients (represented by the height of the rectangles below the EMG
traces) a
nd time
-
shifts (horizontal position of the rectangles). The amplitude coefficients (C, polar representation)
show a simple dependence to movement direction, approximated by a cosine function.



















Figure
8
. Reconstruction of muscle patterns for reaching movements in different directions and speeds
by combinations of phasic and tonic time
-
varying muscle synergies (adapted from d’Avella et al., 2008).
Thee phasic and three tonic time
-
varying synergies (A) ext
racted from the muscle patterns recorded during point
-
to
-
point reaching movement to targets in different directions on the frontal plane and with different speeds capture
the variation in the muscle patterns (B) by modulation in amplitude and timing of the

phasic synergies and in
amplitude only the tonic synergies. The cosine directional tuning of the amplitude coefficients for both synergies
(C) is modulated by speed for the phasic synergies.


1.4
.1

Notion of Motor Primitives
(MP)


In their learning
-
base
d approach UniTu

distinguish
es

temporal primitives

in the sense of individual
actions in longer sequences, and
spatial primitives

that permit the modelling of complex movements
by superposition of a small number of source components, which might include only subsets of the
available degrees of freedom.


1.4
.1.1

Temporal primitives



Temporal primitives have been investigated in the context of complex action sequences, such as
forms in martial arts. A key problem in practice is the automatic segmentation of natural action
sequences in temporal segments that correspond to meaningful sin
gle actions. This problem has been


addressed by supervised learning approaches. For this purpose, the trajectories where characterized
by a sparse set of robust trajectory features (e.g. zeros of the velocity in individual degrees of
freedom). These featur
es were optimized for segmentation. Individual temporal primitives were then
characterized by training examples and were modelled as a discrete sequence of key features that
corresponded to such vectors of such robust features. Such feature sequences form
templates which
then could be matched in a robust manner to the feature sequences extracted from the trajectories by
a special dynamic programming technique

(Figure
9
)
. This algorithm made the matching process
robust against missing or additional key features in the matched sequence compared to the template.
After the automatic segmentation the individual temporal primitives were modelled by
spatio
-
temporal
morphable mo
dels
.

This technique permits the very accurate modelling of complex movement
trajectories by interpolation in space
-
time. Inspired by image morphing techniques that are meanwhile
common in computer
g
raphics, the underlying idea is to characterize cla
sses o
f similar trajectories by
their space
-
time shifts against a reference trajectory (like the average of a set of prototypes). Such
space
-
time shifts can be computed by dynamic time warping. The correspondence shifts of a set of
prototypes relative to a refer
ence trajectory form a basis of a vector space. This permits to model
intermediate movement trajectories by linear combinations of such correspondence shifts

(Figure
10
)
.
The technique of spatio
-
temporal morphable models provides an efficient low
-
dimension
al
parameterization of movement style. It was shown to be efficient for the analysis as well as for the
synthesis of simple
movements (
Poggio and Giese,
1999
;
Giese
and Poggio
,
2000) and also for
movement parameterization form the study of movement perception in psychophysical experiments
(Giese
and

Lappe, 2002; Giese

et al.
, 2008).






Figure
9
.

Supervised segmentation of action streams. The original movement is scanned with a time
window.

The trajectory segment within the window is compared with template sequences (template primitives)
using an algorithm that is based on dynamic programming. (a)

Trajectory segment with robust features
s
i

.
The
segment can be characterized by a sequence of these key features (b) The prototypical primitive can be
characterise by a sequence of corresponding features
m
i

. Dynamic

programming makes the matching robust
against additional or missing key features. (See Ilg et al.
,

2004 for details)
.












Figure
10
.

Modelling of trajectories by linear combination in space
-
time.

(a) Spatio
-
temporal
correspondence is established between the individual prototype trajectory and a reference trajectory. Each
prototype is characterized by the functions
ζ
n
(
t
) and
τ
n
(
t
)

that characterize the spatial and the temporal shifts
against the
reference trajectories. (b) New trajectories can be generated by linear combination of the spatial and
temporal shifts, and space
-
time warping of the reference trajectories with the resulting shifts. (c) Generation of an
action sequence for a humanoid agen
t by linear combination of a neutral and a happy style. (
F
or d
etails see
Giese & Poggio, 2000; Ilg et al. 2004).


1.4
.1.2 Spatial primitives


The

approach for the extraction of spatial primitives from trajectory data

of UniTu

is closely related to
the approaches developed by SLF. The developed methods aim at an approximation of joint angle
trajectories by a minimum set of learned components or source signals that are combined using a
nonlinear mixing model, corresponding to an

anechoic mixture
. This model has been found to be
most appropriate by a detailed analysis of the approximation of sets of periodic and non
-
periodic body
movements, comparing different blind source separation algorithms (Omlor
and

Giese, 2006
a
; Giese
et al
.
,

2009). The applied model is characterized by the equation:























(
3
)

Here







denotes

the angle

trajectory,

and







are the source functions or mixture components.
These functions are interpreted as invariants over different types of movements. The mixing
weights



and the delays



are the model parameters which are movement specific and which are modified
as
part of movement planning and control. The weights



determine the amplitude (gain) of the
corresponding source and the delays



between the components highlight the importance of timing
for the coordination of joint movements. Mixture models of
the type (
3
) are called anechoic. Detailed
analyses showed that various classes of complex body movements could be approximated by these
models with high accuracy and very few source components, typically less than five (Omlor
and

Giese,
2006
a
; Giese et al
.
,

2009).

(a)

(b)

(c)



The described model mainly aims at a minimal mathematical parameterization of the trajectories. The
interpretation of model components as primitives is
,

therefore
,

less direct than in some biologically
motivated methods:





The source functions components



form a function basis that spans the trajectories. For
periodic movements these functions are similar to a Fourier basis. It remains to be clarified
how such functions are related to primitives in terms of controlled co
mponents.



Both
,

weights and

delays

are

movement specific

and

therefore are modified as
part
of
movement

control.

Approximating sets of trajectories by the model (
3
) and analyzing the
model parameters shows that typically these parameters cluster for indiv
idual movement types
and styles (e.g. Roether et al. 2009). In addition the algorithm can be modified in order to
enforce identical parameter values within individual clusters. A primitive is thus best modeled
by a combination of the source function and sp
ecific motion
-
type specific changes of the
parameters



and the
delays



. This approach has been successfully applied for the
modeling and analysis of emotional movement styles and the identification of emotion
-
specific
movement primitives (Omlor
and

Giese, 2006
b
, 2007
; Roether et al.
,
2009

see
Figure
10
a)
.




It is relatively easily possible to modify the algorithm in a way that embeds additional priors
and sparseness constraints. This might make it possible to extend the approach for the
extraction of more sources which map directly onto physiologically releva
nt movement
primitives or physiological modules. Which exact priors are required for maximum
physiological interpretability remains to be discovered.
One possibility is priors that enforce
primitives
that
are spatially localized on the moving face or skeleton of moving bodies.


In an additional set of studies we have extended the described approach for movement synthesis in
real
-
time

(
Figure
1
1
)
. The underlying idea is that solutions of nonlinear dynamica
l systems, called
dynamic primitives

in the following, are mapped onto the source signals identified by unsupervised
learning from the trajectories (Park et al.
,

2008
a
,
2008b,
2009). The resulting architecture consists of a
number of dynamic primitives, wh
ich fulfill a similar function as CPGs in biological organisms. The
coordination of the generated patterns in presence of noise can be stabilized by introduction of
dynamic couplings between the dynamic primitives. In this context it turns out that compact

representations, based on a small number of sources, show more controllable dynamical behavior
than models, e.g. based on PCA, that include a relatively large number of coupled components or
dynamic primitives (Park et al.
,

2008
a, 2008b;

Giese et al.
,

200
9). In addition, we investigated for
simple example
s

the integration of non
-
periodic and periodic movement primitives within the same
architecture.

T
he same architecture can be extended by introduction of dynamical approaches for navigation or the
synchronization between multiple interacting agents in a scene (Giese et al.
,

2009). This makes the
approach suitable for an online synthesis of the body movements of individuals and also of the
coordinated movements of groups of agents.

Due to the simpli
city of the resulting architecture, the framework is accessible for the analysis of the
dynamical stability of the resulting architecture. This has been exploited for the design of the dynamic
couplings between the individual primitives (Park et al.
,

2008
a
, 2008b
) as well as in recent work for the
analysis and the design of the stability of collective order formation scenarios realized by groups of
human agents (Park et al. 2009; Mukovskiy et al.
,

2008,
2010).
Present work focuses on the analysis
and design

of the dynamical stability properties exploiting concepts from contraction analysis (e.g.
Lohmiller and Slotine, 1998; Pham and Slotine, 2007; Mukovskiy et al., 2010).








Figure
1
1
.

Application of the extraction of
movement
primitives by blind so
urce separation.

(a) Automatic
extraction of emotion
-
specific features form gait trajectories, using a blind source decomposisiton model
combined with sparse regressions (from Roether et al.
,

2009). Colors indicate the joint angle amp
litudes
compared to normal walking for angry, fearful, happy and sad walking for the joints indicated along the vertical
axis. Each emotion is characterized by a characteristic profile of changes in joint angle amplitudes. The extracted
features closely ma
tch results from psychophysical studies (indicated by the black and white signs). (b)
Architecture for the online synthesis of complex body movements based on learned primitives. Based on the
learned mixing model that is determined by blind source decompos
ition of a set of training trajectories movements
are generated online. The source
signals are generated online by

mapping of the solutions of

nonlinear dynamical
systems (d
ynamical primitives) onto the source signals using kernel methods. (c) By coupling
of the dynamic
primitives of different agents coordinated crowd behaviour can be self
-
organized. This example shows the
autonomous formation of a crowd with agents that synchronize their s
tep p
hases. Taken fr
om Giese et al.,

2009.


1.4
.2

Methods and exper
imental evidence


A hierarchical model based on temporal primitives combined with a modelling of individual primitives
by spatio
-
temporal morphable models has been successfully applied for the modelling and imitation
learning of movements, such as writing
or martial arts techniques (Ilg et al.
,

2004; Mezger et al.
,

2005). The approach ha
s

been successful not only for movement synthesis but also for the analysis of
movement styles, such as the estimation of skill level from Karate sequences (Ilg et al.
,

2004
). The
low
-
dimensional parameterization of movement styles by spatio
-
temporal morphable models has been
shown to be very useful in psychophysical and fMRI experiments on body movements (Giese
and

Lappe, 2002; Giese et al
.,

2008; Jastorff et al., 2006
,

200
9), as well for the study of the perception of
facial movements (Knappmeyer et al.
,

2004).

Modelling of movements in terms of spatial primitives has been successfully applied for the study of
the execution and perception of emotional body movements (Roeth
er et al
.,

2008
,

2009
; Omlor and
Giese, 2006b, 2007
). The synthesis of trajectories based on dynamic primitives derived from learned
spatial primitives has been applied for the online synthesis of human locomotion and non
-
periodic
movements (Park et al.
,

2
008
a,

2008b
). This work includes integration of primitives for non
-
periodic
(a)

(b)

(c)



and periodic motion. Also this work demonstrates that the modelling of body movements in terms of
dynamic primitives is highly suitable for the modelling of the behavior of groups
of human agents for
example locomoting groups

(
Figure
1
1
C
)

or dancing (Giese et al.
,

2009). More recent work has started
to apply the framework of contraction theory (Lohmiller
and

Slotine, 1998, Wang
and

Slotine, 2005) for
the development of stability bounds for systems based on dynamic primitives (Park et al.
,

2009;
Mukovskiy et al.
,

2010). This work s
uggests that this approach migh
t be suitable for guaranteeing the
stability of systems including large n
umbers of nonlinear dynamic primitives.


1.4
.3

Implications for robotics


The concept of temporal primitives has been successfully implemented for the realization of imitation
learning on a Mitsubishi

robot arm (Ilg et al.
,

2004). The concept of learned dynamic primitives is
suitable for an online synthesis of complex full
-
body movements and the design of complex networks
of dynamic primitives. This makes it potentially interesting for the modelling of complex motor
behaviou
r on robot platforms. However, this framework at the moment lacks the integration of sensory
feedback signals. This step seems crucial for a transfer of this methodology to a variety of robot
systems

and is a central problem of present work
. In addition, i
t remains to be explored in how far
primitives learned from kinematic data can be approximated by control primitives on individual robot
platform
s.

Contraction theory as a tool for stability analysis in complex systems is very general and
has been applied
extensively in the context of robotics and nonline
ar control

(
Lohmiller and Slotine,
1998
; Lohmiller and Slotine, 2000
;
Slotine, 200
6
; Chung and Slotine, 2009
).



1.
5 Relationships between the different approaches


In order to address possible
correspondences between the

architecture of movemen
t representations
in humans and in robots, it seems

helpful to investigate

MPs using maximally similar tasks.

A very

general conception of
movement primitives (
MPs
)

is given by the approach of UniBi, that
envisions different levels of MPs, some of which overlap with the levels of description of the other
partners. W
hich levels are relevant depends on the task
. For example
, particular tasks
might primarily
address

the

higher

cognitive

levels, while others more require a treatment on the level of muscle
synergies.
In addition, the
complexity of MP
s
might change

dependent

on the level of action
organisation.
I
t seems
possible
that interactions exist
between
MPs at
a

more

biological (motor
-
driven),

perceptual
ly

driven and cognitive (intentionally driven) level
s

of acti
on organisation
.

The
general
approach of UniBi offers many opportunities to interact with the other partners. For
example, the work on MPs based on
the perce
ption and the kinematics of body postures

of UniTu can
be nicely combined with the approaches of UniBi for
the
analysis of
kinematics and the cogni
tive
representation of movement keypoints

in memory
. A combination of these different approaches might
help to
learn
more
about the
relationships between kinematic,
perce
ptual and the memory side of
MP
s.

The same question is related to the kinematic and muscle
-
based analysis of primitives by SLF.
Specific
questions of such interactive research may be how
body postures
are
represented in memory
,
or what is the
relationship between muscle
-
activation (
SLF),
body postures

and kinematics
(
as
modelled by UniTu)
,

and f
unctional
-
biomechanical
constrai
nts in the mem
ory representations of MPs.
A further set of important questions in this context is the interplay of
learning process
es and the role of
expertise
,
dependent development and structure formation of MP's in the memory

(UniBi) and at the
neuromuscular

levels (
as treated by SLF)
.

In general,
the approach of UniBi
offer
s a tool for the comparison

of
representation structures for
complex movements in the long term memory of experts and novices on the basis of different sets of
building blocks, or motor primitives,

defined via different methods
. Similar approaches can likely be
applied to
motor primitives
that are
defined via trajectory planning or movement planning on joint level
,


as analyzed by Weizmann, to m
uscle activity patterns and muscle synergies
, as analyze
d by SLF, or
kinematic primitives such as extracted by the methods of UniTu.

Another set of connections exists in the

domain of motion segmentation.

Differential geometrical
approaches by Weizmann for the determination of invariants might be combined by m
ethods for
supervised and unsupervised segmentat
ion, such as developed by UniTu

and also by UniBi. A related
interesting topic
,

presently being investigated at UniBi
,

is
how human observers spontaneously
segment “abstract”, not object
-
related,
complex move
ment sequences in

different conditions
,

and how
their decisions are influenced by learning and expertise (Bläsing

et al.
,

2010
a
).

Results of
such
experiments could be compared to results of
algorithmic methods for movement segmentation in order
to compare human and statistical methods for movement segmentation.

A further problem is the development of methods for measuring of the similarity of different movement
and cluste
ring. UniBi has comp
ared motion

segments by applying
Procrustes analysis, which is widely
used in shape matching. The resulting movement segments can

be applied to the SDA method

was
used to analyze

the
mental representations of movements in long term memory.
Other methods, e
.g.
based on spatio
-
temporal correspondence, have been proposed by UniTu. Further similarity measures
might be developed exploiting concepts from differential geometry, such as applied in the work by
Weizmann.

Close connections
also
exist between the wor
k of Weizmann and UniTu. Motion
primitives
at the level
of hand
or COM trajectories during locomotion
can be defined based on the
hypothesis that kinematic
strokes at the task level are represented in terms of non
-
Euclidian variables
,

and that motion primitives
are invariant under certain group of non
-
linear tran
sformations. E
arlier work
, partially
carried out
in
collaborat
i
on between Weizmann and UniTu
(Dayan et al.
,

2007
;
Casile et al.
,

2010)
,

has indicated
that similar invariant pro
perties of movement
apply also to motion perception.

At the same time, work
by SLF (Ivanenko et al.
,

2010) suggests that internal models of automatic postural responses might
influence perception as well as motor control.
This supports that primitives are
represented at different
levels, such as formulated in the theory of UniBi.

Work involving compositionality principles at the joint level has also been carried out by Weizmann
group in collaboration with
UniTu
(Barliya et al.
,

2009) showing that using a r
elatively simple oscillatory
model at the joint level it is possible to account for the inter
-
segmental law of coordination observed
during human locomotion (Borghese et al., 1996
;

Ivanenko et al
.,

2008).

This work is discussed in
more detail in section 2.
2.

Tight connections exist also between the work of SLF and UniTu. The central algorithms for the
unsupervised learning of primitives from EMG and trajectory data are very similar, and a presently
ongoing computational study investigates how the underlying

models are mathematically related, and
what are the advantages and disadvantages of the underlying algorithms for practical applications,
and especially for the analysis of data in motor control.

Preliminary
esults from this study are
presented in section

2.1.

Another important aspect is the investigation of the relationship
between the
neurom
uscular and the kinematic level of MPs. Ba
sic temporal components might be linked to specific
kinematic events, such as the onset of foot lift in walking (Ivanenko et

al.
,

2006). Moreover, scaling in
amplitude and duration of phasic time
-
varying muscle synergies might underlie the path invariance
observed for reaching movements with different speeds (d'Avella et al.
,

2008).
Here, interesting
connections between the app
roaches of SLF, UniBi, Weizmann and UniTu might be established.

The approach for the online synthesis of trajectories
by nonlinear dynamical systems by UniTu
is
closely related to the primitive
-
based control approaches developed at EPFL (e.g. Ijspeert et
al.
,

2002;
Bu
chli et al
.
,

2006). Challenges to be solved are to embed the proposed
learned
dynamic primitives
for
complex movements
into control architectures and to constrain them in a way that makes them
suitable for the embedding in the existing robot
platforms.


Here,

learning
-
based approaches and the
concept of stability design in modular systems modelling highly complex motor behaviour, such as
investigated by UniTu, might
form

a basis for fruitful interactions with the approaches of EPFL that are
ba
sed on dynamic primitives that are implementable on the available robot platforms.
(C.f. AMARS
I

Deliverable 4.1
: Ch.2,
Dynamical Movement Primitives
;
Ch.8
,
Stable Estimator of Dynamical


Systems
).

S
everal discussed biological models we
re based on the notion

of dynamical sy
stems in
order to

drive the
generated
motion. Notions of stability and tractability that are easily created by the
non
-
biologi
cal models such as DMP and SEDS

(
see
AMARS
I

Deliverable 4.1
: Ch.2,
Ch.8
). However,
such models

cannot be easily tr
ansferred to the anal
ysis of complex biological movements, as some of
the

one
s described in the previous sections. While this may be an impedi
ment for robotics application,
further development of biologically plausible models may nonetheless contribute to
improving the
complexity of the more mathematically oriented approaches
in robotics. Such developments would
represent a nic
e desirable outcome of the AMARSI

project.



2 Quantitative comparisons between selected approaches


Beyond the general considerations in the first part of this deliverable, some of the approaches are so
closely related that it makes sense to compare them mathematically or quantitatively based on
empirical or simulated data sets. This applies specifically

to the approaches for the identification of
primitives by unsupervised learning by SLF and UniTu, and to the characterization of invariants of
locomotion patterns as studied by WI and UniTu. In the following we give a short report about these
more quantit
ative comparisons.


2.1 Comparison between the unsupervised learning approaches by
SLF and UniTu



2.1.1

Introduction


As
discussed in the first part of this deliverable,
there are currently two main different definitions of
motor primitives at muscle level used by the SLF group: the first one is that of
basic temporal
components in the muscle activity patterns
(Ivanenko et al., 2004, 2006)
. The second notion is tha
t of
time
-
varying muscle synergies

(d'Avella et al., 2003; d'Avella et al., 2006; d'Avella et al., 2008)
.

B
oth
notions
rely on the decomposition of EMG datasets
in terms of a

linear combination
of

motor
primitives, scaled in amplitude by some scalar weights. In particular, the first
on
e
is based on the
following equation






















(
4
)


where





is a v
ector of muscle activations,







defining the

basic temporal components (scalar
functions of time)
,




being

real vectors of scaling weights
,

and

where

n

is the total number of
primitives.

In contrast, the second notion

is based on the following equation

























(
5
)


for

whi捨





and



have the same meaning
as
above, and
where the




specify

time delays relative
to the vectors


. Note that this time the



are time
-
dependent variables, whereas the

weights




are
not. Furthermore, the elements of the vectors



and all scaling coefficients



were always
constrained to

be non
-
negat
ive in both

models, given the non
-
negativity of the elements of the vector




. Interestingly, the models in (
4
) and
(
5
) can be seen as two special
case
s of the model used by
the UniTu

group to study spatial primitives (Omlor and Giese, 2007
). This model
is

described by the
equation


























(
6
)


with







represent
ing

either
a joint
-
angle

time series or, in the other case, an EMG signal at instant
t
.
It is indeed straightforward to see that each row of (
4
) can be re
-
expressed

in the form of equation (
6
)
by
equating













,









,












and by considering each coefficient



as the
i
-
th
element of the
j
-
th

vector


. Similarly, each row of (
5
) can be re
-
expressed as a specific case of
equati
on (
6) by taking












,






for








,










and










.

Note
however that, for the same dataset




, to

express eq.
(
5
)

with eq.
(
6
)

a much larger number of
source

signal



may be required

with respect of the case of expressing
eq.
(4)

with eq.
(
6
)
. In general,
each muscle (index
i
) of a time
-
varying synergy might require a different source



,
so that

















.

Therefore,
given
an EMG dataset
is given, it can be decomposed according to one of the model
s

presented above. However, the number of the parameters to be identified and their identification
methods differ for model to model. Coefficients and primitives of model (
4
) were identified by us
ing
standard methods such as Principal or Independent Components Analysis and Non
-
Negative Matrix
factorization (Ivanenko et al., 2005). Parameters of model (
5
) were identified by using
an iterative
optimization algorithm that identifies shift
-
invariant mu
ltidimensional bases using Matching Pursuits
and NMF

(see (d'Avella et al., 2003; d'Avella et al., 2006). For parameter extraction, Omlor and Giese
(2007) exploited an extraction algorithm based on a time
-
frequency transformation (Wigner
-
Ville
distributio
n).

Also the computation

time is dependent of the model and the identification method. For
all these reasons, it
is
hard to establish specific criteria
that are suitable for choosing between these
models with respect to a given application. In order t
o est
ablish a benchmark and

for a comparison
between these approaches for primitive

extraction

we chose the following proceeding:
Starting from
the generative models (
4
), (
5
) and (
6
), three different EMG
-
like datasets were generated.
Subsequently, different alg
orithms for dimensionality reduction based on the
se three models
were
applied to the generated datasets and their results were compared.
This

quantitative comparison
might
help
to reveal non
-
obvious differences between
the models and should
suggest advanta
ges and
disadvantages of the individual
identification meth
ods for synergy extraction from muscle activity
.



2.1.2

Methods


2.1.2.1

Generation of artificial data sets


Generation of source signals:

For the quantitative validation
simulated datasets were
generated

automatically
.
These data sets tried to replicate coarsely the properties of real
EMG signals
,
recorded
from a number


of muscles during
NumT

trial
s with

executions

of motor behavior. D
ifferent
generative models were
tested
(see below).
All generative models derived the data from a
set of


statistically independent EMG
-
like waveforms (
referred to
as source signals
,

or simply as sources
, in
the following). For the generation of these s
ignals we used an autoregressive moving average (ARMA
)
model
, which was

based on the following equation








































)


whe牥


is the order of the autoregressive part of the model (AR),


is the order of the moving
average part (MA),



are the coefficients of the recursive linear filter,
and where



are the coefficients
of the

non
-
recursive linear filter. The signal






signifies

white noise. The ARMA mode
l can be


interpreted as a

discrete
linear
s
ystem, with the
white noise

inpu
t






and the
output
signal





. The
AR and MA coefficients were first
estimated
(
using the MATLAB function
armax
.m

with



,



)
from real EMG data. This data was
recorded during from the biceps muscle of a subject performing an
elbow flexion. Muscle activity was
a
t 1 KHz for a period of 400 ms
. The raw signals had been
amplified, rectified, low
-
pass filter
ed with a cut
-
off frequency of

5 Hz
. In addition, it was res
ampled in
order to the

fit a




sample time window

with

time steps
)
.



output signals were then obtained
by varying the input noise of the model (
MATLAB function
si
m.m
)
. The noise input was
given by a
normally distributed random
vector of length

.
Positive
Independent component analysis

(a particular
version of independent component analysis with non
-
negativity constraints in the output)

was applied
to
the output signals of the ARMA model in order to obtain
a set of


non
-
negative
statistically
independ
ent waveforms. Finally, these signals were
normalized
by division by their
maximum values.


Generative m
odels:

The gen
e
rated



source signals were subsequently used to generate datasets
simulating the EMG activities of


muscles during hyp
othetical experimental trials.
The
simulated
EMG
-
like

datasets were

characterized by an intrinsic modular s
tructure. The goal of the analysis using
dimension reduction methods was to extract the basic primitives, and the underlying structure from this
simulated data. The
simulated datasets were generated by mixing together a set of


primitives
derived d
irectly from the source signals

assuming different mixture models that reflected the structures
of the generative models which were underlying the different unsupervised learning methods. In the
following
each algorithm will

be described along with the

mod
el that underlies the corresponding
simulated EMG datasets
.


2.1.2.2

Tested dimensionality reduction algorithms


1)
Non
-
Negative Matrix Factorization

(NMF)
:

NMF is a standard algorithm for

multivariate analysis
where a matrix


is factorized into two matrices,


and

, so that










)


The 步礠aspe捴 of this method is that all the elements of the mat物捥s


and


are imposed to be non
-
negative. Therefore, if


is thought to
be a matrix in which each row represents
an EMG signal.
A
ccording to equation (
8
) each

EMG signal

in



results from the linear combination of the rows of the
matrix


scaled by the elements of the

row
s

of the matrix

.

According to this mixing model, one simulated EMG dataset could be generated by multiplying a
matrix


of dimension


by


(in which the rows were




independent source signals) by a


by


matrix


of positive random numbers
,

dr
awn by a uniform distribution in the interval
(0,1).


different datasets were then
obtained by changing each time the elements of

. Note
th
at in this case each primitive

coincides with a source signal.


2)
Blind source separation

(or anechoic demixing, which from now on we will refer to as An) is an
unsupervised learning algorithm that approximates signals by linear superposition of components with
signal
-
specific time delays (Omlor and Giese, 2007). Given a simulated EMG matrix

, each row
signal






can be expressed as


























)


whe牥



are scalar weighting factors, each



is a time source signal and the



are time delays
between source signals



and the signals


.
Note that no non
-
negativity constraints are imposed by
this algorithm and that also in this case each primitive coincid
es with a source signal with




.



Based on the modular organization proposed by this model, simulated EMGs were generated by
co
mbining linearly


independent delayed sources weig
h
ted by


weight

coefficients

.
Delays were chosen randomly and drawn from an uniform discrete distribution in the interval





.


3)
Positive anechoic demixin
g:
The positive anechoic demixing (pAn) algorithm works exactly as An,
but additional non
-
negativity constraints are imposed to the coefficients



and the sources


.



4)
Time
-
varying synergies
:

The time
-
varying synergies extraction algorithm

(TV) is

a method for
primitive extraction that was
described in previous works

of the partner SLF

(d’Avella and Bizzi, 2005;
d’Avella et at., 2006). The me
thod is based on the following generative m
odel

























⠱0
)


whe牥





is a vector of real numbers, each component of which represents a specific simulated EMG
activation at time

. The vector signal







represents

the muscle activations for the





primitive,

and




is a time delay and



a non
-
negati
ve scaling coefficient. All
parameters of the model are
constrained to be non
-
negative. Therefore, given a matrix


of simulated EMG
signals,
the algorithm
identifies, through an iterative optimization process, all the non
-
negative parameters of the secon
d
member of equation (
10
).

In our work, to generate
simulated EMG dataset having
modular structure

of the TV model

we
considered





sources and pooled them in


primitive

matrice
s of dimension


by



. In this case
,

a primiti
ve coi
ncided with one of such matrix
. We then computed


random
numbers
that were used as
weighting coefficients



(whose values were taken form a uniform
distribution with comprised between 0 and 1) and


random integer numbers form a uni
form
discrete distribution in the interval






that specified the t
ime delays



. Finally, the primitives were
combined according to equation (
10
).


2.1.2.3

Performance measures


The
approximation quality

of the models for the data was characterized by computing the explained
variance of the data that was captured by the fitted dimension reduction model. This measure was
given by the coefficient






















̅




⠱1
)


whe牥



was

the matrix of the actual dataset,



the reconstructed values by the fitted model, and
where

̅


is a matrix with the mean values of the
data

over trials
.

T
o assess the
approximation qualities of the
algorithm
s

dependent on the
compatibility of the
algorithm with the generative model of the data, we simulated

three datasets according to equations
(
4
), (
5
) and (
6
). To each one of them we applied NMF, An, pAn and TV. We finally computed the


index for all combinations of simulate
d models and
dimensionality reduction algorithm
s
.



As a second measure, we assessed the
similarity between original and extracted primitives
.
This was
done b
y computing the maximum of the

scalar product
s

between original and recovered primitive
over
all
possible

time

delays. In detail, we considered the two sets of original and reconstructed primitives
for a spec
ific
model (NMF, An or TV). Then, given two normalized primitives



and


, we computed
the maximum of their scalar product over all possible delays


for the
second primitive, that is






















(12
)


where




indicates the second primitives delayed in time

by j time steps
. For NMF and An,



and



were already vectors (of length

). Differently from NMF and An, for TV



and



were matric
es of
dimension


by

. In this case, before computing the scalar product,



and




were rearranged
the entries in form of
vectors by concatenati
ng
them in

rows with a length of


.
By

definition,


can

only adopt

values between 0 and 1. For all possible pairs of primitives in the two groups the
corresponding


values were computed.
The pair with highest similarity was selected and the
corresponding primitives were removed from the two groups of primitives. The similarities between t
he
remaining primitives was then

computed
,

and the best matching pair
of primitives was
selected and
remove
d

from the original and

reconstructed model. This

procedure was iterated until all primitives
were matched.


2.1.3

Preliminary results and discussion


In the follo
wing, we present results obtained with the dimensionality reduction algorithms applied to
datasets with



,



, and



. Table 1 shows the R
2

values that quantify the
reconstruction accuracy for the generated datasets with the different algorithms.











Table 3. Approximation quality



for the different generative models obtained with the different
dimensionality reduction algorithms.

Rows correspond to the generative models and columns to the
algorithms.



Data simulated with the
NMF
generative model

was perfectly reconstructed by NMF. However, this
failed for the accurate reconstruction of

datasets
derived from generative models
w
ith specific
temporal structure (An and TV).
The anechoic
algorithms
(A
n and pAn) resulted in general in the
highest perfo
rmance. However, f
or the

dataset derived from an
anechoic
mixture model (second row
in
T
able
3
)
pAn

performed better than An. This makes sense since this algorithm implies
non
-
negativity constraints
for the estimation of the parameters that match the const
raint of the generating
model. The TV algorithm performed perfectly for

the reconstruction of the corresponding ge
nerated
dataset, but failed for the other datasets
.

The most interesting aspect

is whether the algorithms are
able to retrieve the original pr
imitives by applying dimensionality reduction to the simulated datasets.
This was evaluated by assessing the similarities between the primitives that were used to generate a
dataset, based on a specific modular structure (i.e NMF, An or TV), and the primit
ives extracted by the
corresponding algorithm that is based on the same generative model. One example is shown in Figure
1
2
A
, showing that NMF could perfectly retrieve the original primitives (corresponding to the very high
value of the measure
S
). A different situation arises for the An dataset, for which the sources signals,
weights
and delays where all non
-
negative.
In this case the An algorithm, which does not exploit this
positivity constraint results in reasonable approximation of the origin
al data, but a small similarity
between the original and the recovered primitives (Figure
1
2
B
). This is different for the algorithm pAn,
which

results in high approximation (Table
3
) and in addition perfectly retrieves the original source
components (Figu
re
1
2
C
). The same is true if this algorithm is applie
d to the dataset TV (Figure
1
2
D
).

While the presented results are still preliminary, they seem to represent a useful starting point for a
further validation of the different algorithms. In addition, this comparison points to interesting

NMF

An

pAn

TV

NMF

1,00

0,99

1,00

0,63

An

0,74

0,92

0,99

0,43

TV

0,84

0,97

0,98

0,99







theoretical questions, such as why

An, and in particular pAn, always perform well
for many of the
tested
dataset
s. As expected from statistical learning theo
ry,
a good capability of reconstructing the
original dataset is not always
a guarantee
a good recovery of the original primitives. A good
understanding of these issues seems essential for any analysis of biological data using such statistical
methods.

S
ome works based on the study of muscle synergies have demonstrated that scaling and temporal
parameters associated with muscle synergies can be task
-
dependent

and
differentiate between

different
behaviors (Overduin et al, 2008)
,

or healthy and impaired mot
or function

(Cheung et al,
2009). Therefore,

an thorough

understanding whether such consider
ations can be made, independent

on the

applied model for

muscle synergies
,

It seems thus to be of essential importance for
scientist in
the motor control
to assess

which claims can be derived from the parameters of such learned models,









Figure
1
2
.

Comperison between actual and extracted primitives
. A.
Primitives extracted by NMF from the
NMF generated dataset.

B
. Pri
mitives extracted by An from the dataset generated by the An model.

Remark
that all generated sdources were constreained to be
non
-
negative.
The algorithm
An extract
s malso

negative
signals becaus
e no constraints were imposed on

the the extracted sources
. C. Primitives extracted by pAn from
the An generated dataset. D.

Primitives extracted by TV from the TV the generated dataset.



2.2
Inverse kinematics and computational constraints at the
j
oint
l
evel (
j
oint work of WIS and UniTu)

Work involving compositionality principles at the joint level conducted by
the
Weizmann group in
collaboration with
UniTu
(Barliya et al.
, 2009) shows

that

based on the

us
e of
a relatively
a
simple
oscillatory model at the joint level
,

it is possible to account for the inter
-
segmental law of coordination
observed during human locomotion (Borghese et al., 1996
;

Ivanenko et al
.,

2008).

Application of the
blind source separation analysis developed by UniTu to gait trajectories resulted in
models for
locomotion that closely resemble the mathematical model developed by the group at the WI.

Earlier work shows that the redundancy in the control of walking can be efficiently parameterized if the
trajectories of the lower limb are characterized b
y elevation angles, i.e. the angles between the
segments and the cardinal axes in the external frame of reference. In this case, the trajectories of the
elevati
on angles
of
thigh
, shank and foot are lying within a two
-
dimensional plane, effectively
elimina
ting one available degree of
freedom (Borghese et al.
,

1996; see

Figure 1
3
).


The analytical model for the temporal variation of the
elevation angles is

described in detail in Barliya
et al. (2009
).
It turns out that these angles were well approximated by the simple sinusoidal time
dependence:










t
A
a
sin

(13)



In addition, it was shown that under the assumption that the natural frequencies fulfill the condition
F
S
T






(T=thigh, S=shank, F=foot),the orientation of the plane is very well approximated by
























2
1
2
1
3
1
3
1
3
2
3
2
sin
sin
sin






A
A
A
A
A
A
n

(14)


The above expression for the normal vector to the plane is a function of the amplitudes and phases of
the sinusoids that describe the
elevation angles, but not their frequencies. This predicted normal to the
plane differs from the actual normal as computed by using PCA by less th
a
n 3°.
Figure 1
4

illustrates
this

result.

Applying
the blind source separation algorithm developed by
UniTu
(O
ml
o
r and Giese, 2006
a
) it was
found that the sources used to describe joint rotations during human locomotion are similar to the
Fourier based description of joint rotations of the different leg segments applied by Barliya et al.
(2009).

A very important

question that is still open in the motor control area is: given the kinematic redundancy
that exists in the mapping from task to join
t levels, how

are elementary primitives or strokes identified
at th
e task level map into possible
sets of MP or elementary

motions at the joint level.

In
another study
currently carried out in collaboration

between Weizmann and UniTu (Barliya et al., 2010)

it was found

that the mapping from hand to joint spaces can be relatively simple and may involve the
anechoic
mixture

of
similar sources
both at the joint and task spaces
,

but only when using quite sp
ecific
kinematic representation
s of the joint coordinates

which is described in an absolute extrinsic frame of
reference
. Moreover, such similarity between task and joint relate
d sources was not found for
alternative representations.
The fact that two spaces share the same set of sources might be

exploited.

One direction that is currently explored is how this set of sources can serve as a mediator between the
task and joint space
s. The observation that both spaces share one set of sources only when the joint
space is represented in a particular manner has led us to further investigate another old question, that
of representations. The tendency for “laws” to emerge when biomechanic
al systems are represented


in an extrinsic frame of reference is consistently observed. A concrete example is the law of
intersegmental coordination which was just described above and holds only for elevation angles that







Figure
1
3
.

Elevation angles of the different leg segments during one gait cycle as function of time (left),
and as angle
-
angle plot
(right)
.

The trajectories lies in a two
-
dimensional plane, reducing the effective number
of degrees of freedom. (F
rom B
a
rliya et al.,
2009
.
)








Figure
1
4
.

Two planes illustrating the average difference (2.73°) between the plane
constrain

derived
from the original data by a PCA analysis and the plane predicted by the
analytical

model (
f
rom Barilya et
al
.,

20
10
).


are described in an external frame of reference.
This work has laid the basis for future studies
addressi
ng the question what are the re
lation
s

between motor primitives at the task and joint levels.



References:



Abeles M
.
, Bergman H
.
, Gat I
.
, Meilijson I
.
, Seidemann E
.
, Tishby

N
.
, Vaadia E
.

(1995)
Cortical activity flips
among quasi
-
stationary

states.

Proc
.

Natl
.

Acad
.

Sci
.

USA
,

92
, pp.
8616

8620


Barliya A., Omlor L., Giese M.A., Flash, T. (2009)
An analytical formulation of the law of

intersegmental
coordination during human locomotion.

Experimental Brain Research
,
193
(
3
)
,
pp.

371

385
.


Barliya A., Omlor L., Gi
e
se M.A., Flash T. (2010)
The human arm redundancy problem revisited
.

Abstract
for the Society of Neural Control of Movement, NCM 2010.


Bennequin D
.
, Fuchs R
.
, Berthoz A
.
, Flash T. (2009)
Movement Timing and Invariance

Arise from Several
Geometries.

PLoS Comput Biol
.
,

5
(7
)
.
Published online
.


Bernstein, N
.
A
.

(1947).
O postrojenii dvizenij

(Über den Aufbau der Bewegungen).
Moscow, Megdiz.


Bläsing B., Puttke M., Schack, T. (Eds) (2010
a
)
Neurocognition of dance.

Psychology Press: London.



Bläsing B., Maycock

J.
, Bockemühl

T.
, Ritter

H.
, Schack T. (2010
b
)
Motor synergies and object
representation in virtual and real grasping
. International Co
nference on Applied Bionics and Biomechanics,
ICABB, in press.


Bläsing, B. & Volchenkov (2010)
.

Spatio
-
temporal Analysis of Full
-
bo
dy

Movement in Classical Dance
.

Scientific Computing for the Cognitive

Sciences
-

SCCS10, Ruprecht
-
Karls
-
Universität,
Heidelberg.


Borghese N
.
A
.
, Bianchi L
.
, Lacquaniti F
.

(1996)
Kinematic determinants

of human locomotion.

J
.

Physiol
.

494
, pp.
863

879
.


Buchli J., Righetti L., Ijspeert A. J. (2006)
Engineering entrainment and adaptation in limit cycle systems
-

from
biological inspiration to applications in robotics.

Biological Cybernetics, 95, 6, pp. 645

664.


Cappellini G., Ivanenko Y.P., Poppele R.E., Lacquaniti F. (2006)
Motor patterns in human walking and
running.

J. Neurophysiol., 95, pp. 3426
-
3437.


Calinon S., D'halluin F., Sauser E., Caldwell D., Billard A. (2010)
Learning and reproduction of gestures by
imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression.
IEEE Robotics
and Automation Magazine, 17, pp. 44
-
54.


Casile
A., Dayan E., Caggiano

V., Hendler

T., Flash

T., Giese M.A. (2010)
Neuronal Encoding of Human
Kinematic Invariants during Action Observation.

Cerebral Cortex,
20(7), pp. 1647
-
1655.
http://cercor.oxfordjournals.org/cgi/content/full/20/7/1647


Cheung V.C.K,


Piron L.,


Agostini M., Silvoni S.,


Turolla A., Bizzi E. (2009)

Stability of muscle synergies for
volunt
ary actions after cortical stroke in humans.

Proc

Natl Acad Sci USA
106: 19563

19568.


Chung S.J., Slotine J.J.E. (2009)
Cooperative Robot Control and Concurrent Synchronization of
Lagrangian Systems.

IEEE Transactions on Robotics, 25(3),
pp. 686
-
700
.


d'Avella A., Saltiel P., Bizzi E. (2003)
Combinations of muscle synergies in the construction of a natural
motor behavior.

Nat. Neurosci., 6, pp. 300
-
308.


d'Avella A., Bizz
i

E. (2005)
Shared and specific muscle synergies in natural motor behaviors.

Proc N
atl
Acad Sci U S A, 102(8):3076

3081.


d'Avella A., Portone A., Fernandez L., Lacquaniti F. (2006)
Control of fast
-
reaching movements by muscle
synergy combinations.

J. Neurosci., 26, pp. 7791
-
7810.


d'Avella A., Fernandez L., Portone A., Lacquaniti F. (20
08)
Modulation of phasic and tonic muscle synergies
with reaching direction and speed.

J. Neurophysiol., 100, pp. 1433
-
1454.




Dayan E., Casile A., Levit
-
Binnun N., Hendler T., Giese M. A. (2007)
Neural representations of kinematic
laws of motion: Evidence
for action
-
perception coupling.

Proc. Nat. Acad. Sci
.,
104
(51), pp. 20582
-
20587.


Endres D., Christensen A., Giese M.A.

(2010)

Bayesian binning
for

motion segmentation.

Submitted to NIPS.



Flash T., Hogan N. (1985)
The Coordination of Arm Movements: An
Experimentally Confirmed
Mathematical Model.

The Journal of Neuroscience, 5(7), pp. 1688
-
1703.


Flash T., Henis E. (1991)
Arm trajectory modification during reaching towards visual targets.

Journal of
Cognitive Neuroscience,
3
, pp. 220
-
230.


Flash T., Hoch
ner B. (2005)
Motor primitives in vertebrates and invertebrates.

Current Opinion in
Neurobiology,
15
(6), pp. 660
-
666.


Flash T., Handzel A.A. (2007)
Affine differential geometry analysis of human arm movements.

Biological
Cybernetics,
96
(6), pp. 577
-
601.


Giese M.A., Poggio T. (2000)
Morphable models for the analysis and synthesis of complex motion patterns.

International Journal of Computer Vision, 38(1), pp. 59
-
73.

http://www.c
ompsens.uni
-
tuebingen.de/pub/pdf/giesepoggio00.pdf


Giese M.A., Lappe M. (2002)
Measurement of generalization fields for the recognition of biological motion.

Vision Researech, 38, p
p.

1847
-
1858.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/genfpf.pdf


Giese M.A., Thornton I.M., Edelman S. (2008)
Metrics of the perception of body movement.

Journal of
Vision, 8(9):13, p
p.
1
-
18.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/Giese
-
2008
-
jov
-
8
-
9
-
13.pdf


Giese M.A., Mukovskiy A., Park A., Omlor L., Slotine J.J.E. (2009)
Real
-
Time Synthesis of Body Move
ments
Based on Learned Primitives.

In Cremers D, Rosenhahn B, Yuille A L (eds): 'Statistical and Geometrical
Approaches to Visual Motion Analysis', Lecture Notes in Computer Science, 5604, pp. 107
-
127.
Springer
Verlag.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/LNCS5604proc2009.pdf


Gribovskaya E., Kheddar A., Billard A. (2011)
Motion Learning and Adaptive Impedance for Robot Control
during Physical Interaction with Humans.

IEEE Int. Conf. on Robotics and Automation. Submitted.


Handzel A
.
A
.
, Flash T
.

(1999)
Geometric methods in the study of human motor control.
Cogn
.

Stud
.,

6
,

pp.
309

321.


Henis E., Flash T. (1995)
Mechanisms underlying the generation of averaged modified

trajectories.

Biological Cybernetics,
72
, pp. 407
-
419.


Hersch M., Guenter F., Calinon S., Billard A. (2008)
Dynamical System Modulation for Robot Learning via
Kinesthetic Demonstrations.

IEEE Transactions on Robotics, 24, pp. 1463
-
1467.


Ijspeert A., Naka
nishi J., Schaal S. (2002)
Learning attractor landscapes for learning motor primitives.

Adv.
in Neural Inf. Process Systems, 15, pp. 1547

1554.


Ilg W., Bakir G.H., Mezger J., Giese M.A. (2004)
On the representation, learning and transfer of
spatio
-
temporal movement characteristics

International Journal of Humanoid Robotics, 1(4), p
p.

613
-
636.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/humano04_ilg.pdf


Ivanenko YP
, Poppele RE, Lacquaniti F (2004)
Five basic muscle activation patterns account for muscle
activity during human locomotion
. J Physiol (Lond) 556:267

282.


Ivanenko Y.P., Cappellini G., Dominici N., Poppele R.E., Lacquaniti F. (2005)
Coordination of locomo
tion
with voluntary movements in humans.

J. Neurosci., 25, pp. 7238
-
7253.


Ivanenko, Y.P.
,
Poppele, R.E.

&
Lacquaniti, F.

(
2006
)
Motor control programs and walking
.

Neuroscientist
,
12
,
339

348
.




Ivanenko Y
.
P
.
, d’Avella A
.
, Poppele R
.
E
.
, Lacquaniti F
.

(2008)
On the

origin of planar covariation of
elevation angles during human

locomotion.

J
.

Neurophysiol
.,

99
, pp.
1890

1898
.


Ivanenko Y.P., Dominici N., Daprati E., Nico D., Cappellini G., Lacquaniti F. (2010)
Locomotor body scheme.

Hum. Mov. Sci. Submit
ted.


Jastorff J., Kourtzi Z., Giese M.A. (2006)
Learning to discriminate complex movements: Natural vs.
artificial trajectories.

Journal of Vision, 6(8):3, p
p.

791
-
804.

http://www.compsens.uni
-
tuebingen.de/pub/pdf/Jastorff
-
2006
-
jov
-
6
-
8
-
3.pdf


Jastorff J., Kourtzi Z., Giese M.A. (2009)
Visual learning shapes the processing of complex mov
ement
stimuli in the human brain

Journal of Neuroscience, 29
(
44
)
, pp. 14026
-
38.
http://www.jneurosci.org/cgi/reprint/29/44/14026


Knappmeyer B., Bülthoff H.H., Giese M.A. (2004)
Facial motion

and the carivature effect.

In: Kerzel D,
Franz F, Gegenfurtner K (eds): Beiträge zur 46.
Tagung experimentell arbeitender Psychologen. Pabst Science
Publishers, Lengerich, Germany, 141.


Krause A.F., Dürr V., Bläsing B., Schack T. (2010)
Evolutionary Opti
mization of Echo State Networks:
multiple motor pattern learning.

6th International Workshop on Artificial Neural Networks and Intelligent
Information Processing, (peer reviewed, accepted, 10 page full paper).


Lohmiller W., Slotine J.J.
E. (1998)
On
contraction analysis for nonlinear systems.

Automatica
,

34
(
6
)
, pp.
683

696.


Lohmiller W., Slotine J.J.E. (2000)
Control System Design for Mechanical Systems Using Contraction
Theory.
IEEE Trans. Aut. Control, 45(5)
, pp. 984
-
989
.


Maoz U.
, Berthoz A., Flash T. (2009)
Complex Unconstrained Three
-
Dimensional Hand

Movement and
Constant Equi
-
Affine Speed.

Journal of Neurophysiology,
101
(2), pp. 1002
-
1015.


Maycock J., Dornbusch D., Elbrechter C., Haschke R., Schack T., Ritter H. (2010)
Approac
hing Manual
Intelligence.

Künstliche Intelligenz
, in press.


Mezger J., Ilg W., Giese M.A. (2005)
Trajectory synthesis by hierarchical spatio
-
temporal correspondence:
comparison of different methods.
ACM Symposium on Applied Perception in Graphics and V
isualization
2005, A Corona, Spain, August 26
-

28, 2005, p
p.

25
-
32. ACM press


http://www.compsens.uni
-
tuebingen.de/pub/pdf/APGV05p.pdf


Mukovskiy A., Park A., Omlor L., Slotine J.J.E., Giese M.A. (2008)
Self
-
organization of character behavior
by mixing of learned movement primitives.

Proceedings of the 13th Fall Workshop on Vision, Modeling, and
Visualization (VMV), October 8
-
10, Konstanz,

Germany, p
p.

121
-
130.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/VMV08.pdf


Mukovskiy A., Slotine J.J.E., Giese M.A. (2010)
Design of the dynamic stability properties of the collect
ive
behavior of articulated bipeds
.

In

press
:
In Proc.

of
10th IEEE
-
RAS Int. Conf. on Humanoid Robots,
Humanoids 2010
.


Omlor L., Giese M.A. (2006
a
)
Blind source separation for over
-
determined delayed mixtures.

Advances in
Neural Information Processing Systems, 19, p
p.

1074
-
1081.

http://www.compsens.uni
-
tuebingen.de/pub/pdf/OmlorGies
e_NIPS06.pdf


Omlor L., Giese M.A. (2006
b
)
Unsupervised learning of spatio
-
temporal primitives of emotional gait.

Perception and Interactive Technologies 2006, Lecture Notes in Artificial Intelligence, 4021, p
p.

188
-
192.
Springer, New York.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/PIT06p.pdf


Omlor L., Giese M.A. (2007)
Extraction of spatio
-
temporal primitives of emotional body expressions.

Neurocomputing, 70(10
-
12), pp
. 1938
-
1942.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/Neurocomputing_LO07.pdf




Overduin S.A., d'Avella A, Roh J., Bizzi E. M
odulation of muscle

synergy recruitment in primate grasping.
J Neurosci.

23;28(4):880
-
9
2.


Park A., Mukovsk
iy A., Omlor L., Giese M.A. (2008
a
)
Self organized character animation based on learned
synergies from full
-
body motion capture data.

Proceedings of the 2008 International Conference on Cognitive
Systems (CogSys), University of Karlsruhe, Karlsruhe, German
y, 2
-
4 April, Springer
-
Verlag, Berlin.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/CogSys08.pdf



Park A., Mukovskiy A., Omlor L., Giese M.A. (2008
b
)
Synthesis of character
behaviour by dynamic
interaction of synergies learned from motion capture data.

Skala V. (ed): Proceedings of the 16th
International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision
(WSCG), 4
-
7 Feb, Plzen, Czech Republic
, p
p.

9
-
16.

http://www.compsens.uni
-
tuebingen.de/pub/pdf/WSCG08.pdf


Park A., Mukovskiy A., Slotine J.J.E., Giese M.A. (
2009)
Design of dynamical stability properties in
character animation.

In: The 6th Workshop on Virtual Reality Interaction and Physical Simulation, VRIPHYS
09, Nov 5
-
6, Karlsruhe, Proc., pp.

85
-
94
.

http://www.compsens.uni
-
tuebingen.de/pub/pdf/VriPhys2009Fin.pdf


Pham Q. C., Slotine J. J. E. (2007)
Stable concurrent synchronization in dynamic system networks.

Neural
Networks, 20(3), pp. 62

77.


Poggio T., Giese M.A. (1999)
Synthesis and Recognition of Biological Motion Patterns Based on Linear
Superposition of Prototypical Motion Sequences.

In: Proceedings of the IEEE Workshop on Multi
-
View
Modeling and Analysis of Visual Scene, Fort Collins, CO, June 21
-
23, 1999, pp. 73
-
80.

http://www.compsens.uni
-
tuebingen.de/pub/pdf/cvpr99.pdf


Pollick

F
.
E
.
,
Sapiro

G.
(1997)
Constant affine velocity predicts the 1/3 power law of planar motion
perception and generation
.

Vision Res.
,
37(3)
, pp.
347
-
53
.


Pollick F
.
E
.
, Maoz U
.
, Handzel A
.
A
.
, Giblin P
.
J
.
, Sapiro G
.
, Flash T. (2009)
Three
-
dimensional arm
movement
s at constant equi
-
affine speed
.

Cortex
,
45
(3)
, pp.

325
-
339.


Polyakov F., Drori R., Ben
-
Shaul Y., Abeles M.,
Flash T. (2009
a
)
A Compact Representation of Drawing
Movements with Sequences of Parabolic Primitives.

PLoS Comput Biol.
,
5
(7).
P
ublished online
.

http://dx.plos.org/10.1371/journal.pcbi.1000427



Polyakov F., Stark E., Drori R., Abeles M., Flash T. (2009
b
)
Parabolic movement primitives and cortical
states: merging optimality with geometric invariance.

Biological Cybernetic
s,
100
(2), pp. 159
-
184.


Ri
chardson M.J.E., Flash, T. (2002)
Comparing smooth arm movements with the two
-
thirds

power law and the related segmented
-
control hypothesis.

J. of Neurosci
.,
22
, pp. 8201
-
8211
.


Roether C.L., Omlor L., Giese M.A. (2008)
Lateral asymmetry of bodily emotion
expression.

Current
Biology, 18, pp.
R329
-
330.
http://www.compsens.uni
-
tuebingen.de/pub/pdf/RoetherCB08.pdf


Roether C.L., Omlor L., Christensen A., Giese M.A. (2009)
Critical fea
tures for the perception of emotion
from gait.

Journal of Vision, 9(6)
,
15, p
p.

1
-
32. Association for Research in Vision and Ophthalmology,
Rockville.
http://www.compsens.uni
-
tebingen.de/pub/pdf/Roether
-
2009
-
jov
-
9
-
6
-
15.pdf


Schack T. (2004)
The Cognitive Architecture of Complex Movement.

International Journal of Sport and
Exercise psychology, 2(4), pp. 403
-
438.


Schack T., Mechsner F. (2006)
Representat
ion of motor skills in human long
-
term
-
memory.

Neuroscience
Letters, 391, pp. 77
-
81.


Schack T., Hackfort D. (2007)
An action theory approach to applied sport psychology.

In G. Tenenbaum &
R. C. Eklund (Eds.),
Handbook of Sport

Psychology (3rd Edition).
NJ
:Wiley
, pp. 332
-
351
.



Schack T., Ritter H. (2009).
The Cognitive Nature of Action


Functional Links between Cognitive
Psychology, Movement Science and Robotics.
Progress in Brain research: Mind and Motion
-

The
Bidirectional Link between Thought and Action.
Elsevier. pp. 231
-
252.




Schack T. (2010)
Die kognitive Architektur menschlicher Bewegungen


innovative Zugänge für
Psychologie, Sportwissenschaft und Roboti
k.

Reihe „Sportforum“. Aachen: Meyer & Meyer.

Schuetz, C., Klein
-
Soetebier, T., & Schack, T. (2009).
Modeling of

Biomechanical Parameters Based on LTM Structures. In H. Ritter, G.

Sagerer, R. Dillmann & M. Buss (Eds.),
Human Centered Robot Systems.

Cogniti
on, Interaction, Technology.

(pp. 161
-
171). Berlin Heidelberg: Springer
-
Verlag.


Slotine J.J.E. (200
6
)

Modularity, Synchronization, and What Robotics May Yet Learn from the Brain.
In:
Kawamura S., Svinin M. M., Arimoto S., Editors, "
Advances in robot
control: from everyday physics to
human
-
like movements
" Springer, Berlin, New York

, pp. 180
-
200.


Sosnik R., Hauptmann B., Karni A., Flash T. (2004)
When practice leads to co
-
articulation: the evolution of
geometrically defined movement primitives.

Exp
.

B
rain Res
.,
156
(4), pp. 422
-
438.


Stark E., Drori R., Asher I., Ben
-
Shaul Y., Abeles M. (2007)
Distinct
movement parameters are represented
by different neurons in the

motor cortex.

Eur. J. Neurosci. 26, pp. 1055

1066.


Stöckel, T. &

Schack, T. (2010, under review). Goal posture representation and end
-
state comfort planning in
children.
Journal of Experimental Child Psychology
.


Tscherepanow

M
.
, Kortkamp, Kühnel, Helbach, Schütz

C.
, Schack T. (2010)
Simulating the Structure of
Mental

Representations Using Computational Features.

NIPS, under review.


Viviani P., Flash T. (1995)
Minimum
-
jerk, two
-
thirds power law and isochrony: converging approaches to
the study of movement planning.

Journal of Experimental Psychology
:
Perception and Pe
rformance,
21, pp.
32
-
53.



Wang W., Slotine J.J.E. (2005)
On Partial Contraction Analysis for Coupled Nonlinear Oscillators.

Biological Cybernetics, 92
(
1
)
, pp. 38
-
53.


Weigelt M., Rosenbaum D.A., Hülshorst S., Schack T. (2009).
Moving and memorizing:
Motor planning
modulates the recency effect in serial and free recall.

Acta Psychologica, 132, pp. 68
-
79.


Weigelt M., Schack T. (2010)
The development of end
-
state comfort planning in preschool children.

Experimental Psychology, 6, pp. 1
-
7.


Zentgraf K., Green N., Munzert J., Schack T., Tenenbaum G., Vickers J.N., Weigelt M., Wolfensteller U.,
Heekeren H.R. (2009)
How are actions physically implemented.

Progress in Brain research: Mind and


Motion
-

The Bidirectional Link between Thought a
nd Action.
Elsevier. pp 305
-
320.