Animated Piano Player

stagetofuAI and Robotics

Oct 29, 2013 (4 years and 8 months ago)


Animated Piano Player

Verna Chang


The animation of the hands is one of the most studied branches of animation involving the
human figure, with many factors to take into consideration and several methods to choose
from. There has yet to be a
perfect method for deriving such animation however though
inverse kinematics and its many varieties provide reasonable ways to achieve animation of
limbed figures. This report documents an attempt at animating the human hands in
playing the piano us
ing the method of pseudo inverse kinematics.


In the industry of animation, one of the most heavily researched and extensive subjects is that
of the animation of the human figure. Jumping, climbing, walking, there is much to be studied
and muc
h that has need to be implemented for any variety of purposes from simulations, to
medical research, to animation, and yet, even within that expansive subject, certain parts are
still more heavily researched and studied than others. One of these such part
s is arguably, our
most important limb, the human hand, without which we could not have evolved into the
species we are today.
This all too important limb however, is the hardest to animate
of all the appendages of the human body, with even se
emingly simple tasks such as moving two
fingers simultaneously proving to be more difficult than previously assumed and the generation
of gestures a nightmare.
One might think, that with inverse kinematics and its multitude of
variations that the animatio
n of the hand would not prove an issue. But inverse kinematics is
not a perfect solution, the animation of the hand is perhaps, the strongest example of bringing
all its flaws to light. Two of these problems are documented in the paper by paper “

Animating the Human Hand
” by
George ElKoura

Karan Singh

The first is that typical

inverse kinematics (IK) algorithms are designed to deal with

constraints along a single chain, whereas a multi

limb like the human hand has constraints bet
ween joints of

different chains

The human hand, excepting the thumb, is bound by nature to be unable to move independently
of the other fingers on the hand, and even assuming that each finger could be completely
independent, there are very few applicatio
ns in which that sort of movement would be useful
or wanted, as such there is a need to create multiple chains for any calculations done to
animate the fingers.

The second shortcoming is that IK solutions

typically map a single end
effector to a single re

goal 28; 33. A sequence of reaching tasks must, therefore, be

performed in order by the same end

This second problem
creates issues when there is a need to create more specific poses with
multiple end effectors and again, brings up the pr
oblem of simultaneously occurring
tasks such
as that of the piano player who naturally uses many of his fingers simultaneously for any given
song. In this implementation of an animation of hands playing the piano, I have attempted to
use pseudo inverse ki

to animate these given motions, using vectors to store positions,
a hierarchical structure to build my models, and simple iterative animation to animate all joints
other than those that belong to the fingers.


The Model Hierarchy

Data Structures

The figure of the two armed robot was built using a hierarchical structure. In a separate header
file called pianist.cpp, a struct called bodyPart was

created for use in the model. An instance of
this struct is created for every joint inv
olved in the later calculations of inverse kinematics and
has its values called and updated for every subsequent drawing of its corresponding joint. The
struct contains

three doubles called thetaX, thetaY, and thetaZ to be used in drawing the parts
of the

model and for storing the output of every iteration of the inverse kinematics updater
method. The struct also contains a GLDouble array which holds sixteen elem
ents called matrix
which is updated with the model view matrix with every call to drawHuman()
and is primarily
used to access the position of the joint in the call to calculate inverseKinematics.

The Model

The model itself was drawn using a nested matrix format. With every step down the hierarchy,
i.e., from shoulder to upper arm to elbow, a new matrix was pushed before the previous limb’s
pop was called, effectively stacking the transformations of every su
bsequent limb drawn. The
joints were drawn using glusphere and rotated using the stored theta values assigned to each
joint struct
, while the links between them were drawn using glucylinders
All calls to drawing
the figure are stored within the function

drawHuman() which is continuously called from the
renderScene function.

The piano was built using three separate functions. The main function drawPiano, creates two
boxes built using glQuads which are nested similarly to the figure model. The nested b
contains two for loops, each of which pushes a matrix and calls a method. The methods called,
whiteKey and blackKey, contain a single piano key drawn with quads, which is then called
within the for loops, fifty two times for the white keys, and thirty
five times for the black.
Double variables are set to iterate a predetermined amount in order to separate the keys, with
extra conditions for the black keys.


Pseudo Inverse Kinematics

Two methods were used in the animation of this figure. One method was Pseudo Inverse
kinematics. Using a combination of the stored model view matrices and the matrix math library
Matrix TLC Lite, a series of rotations was calculated with the pseudo inver
se kinematics
formula. For the specified position, a vector was created for the difference between the goal
position and the end effector position. Then six values were calculated for use in the Jacobian,
each of which being a cross product between the a
xis of rotation (x or y in the case of the
fingers) and the coordinates of a joint position in the chain. After this Jacobian was formed, the
pseudo inverse was calculated using the formula J^T(JJ^T)^
1V. This was used with Euler’s to
integrate the value
s of the rotations at every joint until the goal position was either reached or
within an acceptable distance.

When goal is met, the next positions are set from several
vectors of coordinates for x, y, and z, the axis of rotation, and time steps. Each fi
nger is given its
own implementation of the pseudo inverse kinematics calculation within the function. If the
end of the goal storing vectors has been reached, the goal will be reset back to the first in the
list. A function containing all these calculat
ions is then called with every call to draw.

Secondary Animation:

Secondary animation is provided with iteration and constraints. Given rotation constraints and
variables to keep track of the iterations, a joint of the arms and elbows are rotated a certai
degree and then back, causing the arms and hands to move in a pattern over the piano keys.

an option is selected, this method of animation will also be used to animate a simple pattern of
key strikes with the figure’s fingers in lieu of pseudo invers
e kinematics being calculated.

separate animation of the arms is done to avoid the typical constraint of inverse kinematics
which involves the calculations of a single end effector to a single end root in its animations
which would cause complications

even with the separate calculations of pseudo inverse
kinematics for each finger.

Camera Control and Other Commands:

Camera controls are implemented
in two ways. Two default camera positions are saved, one
looking from the back and giving a larger vi
ew of both the figure and the piano, the other
zooming in much closer onto the hand and fingers and are accessed using key commands.
Aside from this, the user is free to control the camera in six ways, translation by x, y, and z, and
rotation by x, y, and

z using key commands to shift the position however specified. Both of
these camera control methods are implemented using rotation and translation of saved
variables on the glLookAt function. With every key command pressed for the control of the
values used in the aforementioned translations and rotations are incremented or
decremented, causing the position of the camera to change after every call. The calling of the
default cameras however, will always restore the camera control variables to the
ir original


The attempt to design the structure went through several phases, starting with a map
containing strings and structs. However, this was much too unwieldy and ultimately unneeded
as all instances of the bodyPart struct could be cr
eated in the header file and then initialized
later in the main initializer function. There was also an attempt made to link the bodyParts
together in the struct using pointers that could then be traversed as a tree, but was discarded
as I could not deter
mine any method in which to have the fingers perform simultaneously. The
current state of the overall structure is still somewhat cluttered and difficult to modify given
the large amount of nested statements and the very many variables for storage and ite
ration in
the main file and the header files.

In the attempt to animate the fingers using inverse kinematics I first attempted to use
GLDoubles and the matrix calculation functions provided by openGL before attempting several
different matrix libraries a
nd settling on TLC lite. My attempts to animate the fingers of the
structure using inverse kinematics was unsuccessful. Despite checking the calculations and the
variables the values caused the fingers to oscillate and move wildly about and refuse to adh

to a stoppage point unless the goal position specified was extremely close to the
position taken from the model view matrix, in which case it would often refuse to start at all
unless the stoppage position itself was very small. Due to
these factors I can assume that there
is something fundamental I am missing in my calculations that I was unable to discover by the
time of submission.

As a result,
one finger has
inverse kinematics
implemented, and it results in
incorrect animation

For t
he purposes of the presentation and for purposes of having some piano playing animation
done, I created a simple animation using iterating variables for the rotations of the joints.
When a constraint on the degree a joint can rotate is reached, the sign o
f the iteration changes
and animation proceeds in the opposite direction for the specified joint.
This created a very
simple looping piano playing animation would can be enabled or disabled by a key command.

The main purpose of this type of animation was

originally to implement movement of the arms
on their own, so that I would not have to calculate pseudo inverse kinematics for them, but
since the kinematics did not work, I used this method to create a very simple animation last
minute in its place.

ted Works:

There are many projects and related works that pertain to my attempts at animating a figure
using inverse kinematics.

The most prevalent rel
ated work
is that of the aforementioned
George ElKoura and Karan Singh

and their work in attempting to an
imate the human hands. Their particular implementation
was to attempt to animate hands playing the guitar, a task more arduous than those fitted
towards typing on a keyboard due to the multiple end effectors needed.

Future Work:

Though this attempt at ani
mating fingers was wholly unsuccessful, I intend to continue my work
attempting to animate complex hand and finger movements using different methods of inverse

and polygon models in the place of primitives. My end goal for this line of studies
to create a program that can take in input that corresponds to musical notes and have the
animated figure simulate it on screen.


ElKoura, George and

Singh Karan

Handrix: Animating the Human Hand
Eurographics/SIGGRAPH Symposium on
Computer Animation (2003)

Parent, Rick
Computer Animation, Algorithms and Techniques 2


2008 by Elsevier Inc.

Morten Pol Engell

Sarah Maria


iebe Morten
Bo Bonding

Inverse Kinematics With Constraints

December 14, 2007

Department of Computer Science, University of Copenhagen

Universitetsparken 1, DK
2100 Copenhagen East, Denmark