B. Comp. Dissertation
Transformer
By
Le Vu
Department of Computer Science
School of Computing
National University of Singapore
2012/21013
1
B. Comp. Dissertation
Transformer
By
Le Vu
Department of Computer Science
School of
Computing
National University of Singapore
2012/21013
Project No:
H087720
Supervisor:
Assistant Professor
Alan Cheng Ho Lun
2
Abstract
The film Transformer of director Michael Bay has been popular recently.
There have been
many products following the theme of the movie.
One of them is the Transformer toys, which
are the small versions of robots from the film.
To generalize the problem, we would like to
investigate
how
to split an object into
finitely many
components and transform them in to a
pre

determined shape
,
and to propose a solution
for
the transformation
.
The most common thing used to simulate the movement of a robot is inverse
kinematics solving. That is, the robot is treated a set of joints and li
nked components
and the
movement of robot depends on the joints parameters.
In this report, we would like to use one
of the approaches
to solve the inverse kinematic problem:
A* algorithm. A
lthough it is not
common,
it is effective in the situation of the
report, in which there are many obstacles during
the transformation.
The algorithm has been proved to work
by
achiev
ing some good
result
s
.
I
hope that this report would give a good investigation to the solution for Transformer toys as
well as it can be app
lied for other researches.
Keywords
Transformer
,
Transformation,
Rigid body
,
Degree of freedom, Blender, Autodesk Maya,
Python,
ICP,
Inverse Kinematics, Joints,
Armature,
and Constraints
.
Imple
mentation Software and Hardware
Blender 2.63
, Python 2.7
,
Autodesk
Maya 2012
3
Acknowledgement
During this project,
I always feel
the encouragement
from others. I would have not been able
to do this
without them.
The first one I would like to
show my
appreciation
is my
supervisor,
Mr.
Alan Cheng
Ho
Lun,
for his guidance.
His experienced advice is very precious
that helps me a lot
.
Besides, some suggestion and
side

story from him shows me that this field is interesting that
keeps my enthusiasm
to complete the project
.
The second one
I would also
like to thank
is
my partner, Jiang Kuo, who also play
s
an
important role in doing this project.
His
work attitude
keeps reminding
me to try harder.
Furthermore
, sharing difficulties, knowledge,
and fun
is
,
actually
,
making this project more
meaningful to me.
Finally, to my family and friends, you have been encouraging me a lot. A warmest
thank to everyone. Thank you!
4
List of Figures
Figure 1
.1
Transformer
................................
................................
................................
......
7
Figure 2
.1
Design of the
Transformer
project
................................
................................
..
1
0
Figure
4.1
Cutting methodology
................................
................................
......................
14
Figure 4
.2a
An inputted car model
................................
................................
....................
1
5
Figure 4.2b
The human skeleton
................................
................................
........................
15
Figure 4.3
UI panel of model segmentation
................................
................................
.....
16
Figure 4.4
Car components
................................
................................
..............................
1
6
Figure 5.1
A human skeleton
................................
................................
...........................
18
Figure 5.2
Side view and perspective view during
components attachment
...................
22
Figure 5.3
A bone with two possible closest states
................................
..........................
25
Figure 6.1a
Suggestion on model segmentation
................................
................................
28
Figure 6.1b
Suggestion on skeleton
................................
................................
...................
28
Figure 7.1a
A simple model
................................
................................
...............................
30
Figure 7.1b
Model segmented and attached
................................
................................
.......
30
Figure 7.1c
A
frame during animation
................................
................................
...............
30
Figure 7.1d
Another random frame
................................
................................
....................
30
Figure 7.2
Car model
................................
................................
................................
........
31
Figure 7.3
Car model after simplification and segmentation
................................
...........
31
Figure 7.4
Human skeleton and components attachment
................................
.................
3
2
Figure 7.5
Result using naive method at frame 1, 143,
289, 434
................................
.....
32
Figure 7.6
Final result
................................
................................
................................
......
33
5
Table of Contents
Title
1
Abstract
2
Acknowledgement
3
List of Figures
4
Chapter
1 Introduction
................................
................................
.............
7
1.1
Other motivation
................................
................................
................................
..
7
1.2
Goals
................................
................................
................................
......................
8
1.3
Related work
................................
................................
................................
........
8
Chapter 2
Design
................................
................................
.....................
10
2
.1
Workflow diagram
................................
................................
.............................
10
2
.2
Workflow description
................................
................................
.......................
10
2.2.1 Preprocessing
................................
................................
..............................
10
2.2.2 Model segmentation
................................
................................
...................
10
2.2.
3
Components transformation
................................
................................
.......
1
1
2.2.
4
Feedback and Suggestion
................................
................................
...........
1
1
2
.3
Design goals
................................
................................
................................
.......
11
2.4
Criteria
................................
................................
................................
................
1
2
Chapter 3
Preprocessing
................................
................................
.........
13
Chapter 4
Model segmentation
................................
...............................
1
4
4
.1
Expectations
................................
................................
................................
.......
1
4
4
.2
General view
................................
................................
................................
......
1
4
4.3
Results
................................
................................
................................
.................
15
6
Chapter 5
Components transformation
................................
..................
1
7
5
.1
Skeleton data structure
................................
................................
......................
1
7
5
.2
Inverse kinematics
–
a brief investigation
................................
......................
18
5
.2.1
Cyclic Co

ordinate Descent
................................
................................
........
1
9
5
.2.2
Jacobian Matrix
................................
................................
..........................
1
9
5
.2.3
Neural
networks
................................
................................
..........................
1
9
5
.2.4
Proposed algorithm
................................
................................
.....................
1
9
5
.2.
5
Main algorithm
–
first approach
................................
................................
.
20
5.3
Implementation of A* algorithm
................................
................................
.....
21
5
.
3
.1
Rescale
................................
................................
................................
........
21
5
.
3
.2
Attachment
................................
................................
................................
.
21
5
.
3
.3
Rotation method
................................
................................
.........................
23
5
.
3
.
4
Naive approach
................................
................................
...........................
23
5
.
3
.
5
A* algorithm with overlapping
disallowed
................................
................
2
4
5
.
3
.
6
Some discussion about algorithm
................................
...............................
26
5
.
3
.
7
Improvement on intersection
................................
................................
......
26
Chapter
6
Feedback and Suggestion
................................
.......................
28
Chapter 7
Results
................................
................................
....................
30
Chapter 8
Future work
................................
................................
...........
34
Chapter 9
Conclusion
................................
................................
..............
35
References
................................
................................
............................
36
7
C
hapter
1
:
Introduction
The
project
title
, Transformer, is i
nspired
from
the
same

name
movie
by
famous director
Michael Bay
,
where vehicles like car, truck, or plane
are able to
rear
r
ange
themselves
into
robot
s.
In t
his project
,
our objective is
to
simulate the
transformation
of any
object from
initial shape to another determined shape
.
However, d
ifferent from the movie where the
transformation process is done by
visual effect
s
,
our target is to make it more physically
possible.
1.1
Other m
otivation
s
Another motivation of the project is its application into the foldable households such as
furniture. Nowadays
an increasing number of foldable furniture is
introduced
and sold that
makes this type of products is more and more familiar to
customers.
Althoug
h most of
foldable products are
easy to use
for customers
,
to manufacturer
s
,
to design a model
of those
products is really a problem.
For example,
i
t is possible to think of several ways to
split a
box

shaped object and then, to rotate
the components into a table.
However,
it is quite a slow
process
by human
to produce different solutions,
although the goal and initial state is the
same:
transform a
box into a table.
Besides, not all solutions
are
good enough
in some criteria
such as too many rotations during the transformation, too many component, or space
consuming.
Hence, this project aims to solve the problem by suggesting some rearrangements
Figure 1.1 Transformer
8
of products components
so that they can rotate
and change the in
itial shape of object
depending on the
given
purpose.
S
olving this problem
may lead to a practical
and
manufacturing meaning
that improves the
process of designing a new foldable model.
1.2
Goals
The
first
goal of the project is
to
suggest some
ways to
split an
in
put
ted
object
into
sub

parts
linked by
joints
.
Those joints are
simulated from mechanics joints th
ose
can perform
rigid
transformation
to
objects attaching to them
.
Based on the movement
of the sub

parts, j
oints
are determined
and stored in a skeleton
or
a template
, which is pre

defined by user.
To
achieve this first goal,
following
criteria should be satisfied
:
the object should not be cut into
too many components
.
We prefer least number of cut
s
which results to
components
remain
ing
the object’s features.
The second goal is to
find a solution
to transform from initial shape to the
given
template
and to
visualize
that transformation process
in an
animation or
a
video
.
During
solving the transformation, some
analysis or suggestion is performed, and then, feedbacks to
improve the splitting process
or to improve the template itself.
Similarly
,
some criteria should
also be satisfied during achieving this goal.
As we want
a realistic solution in which
object
can
change
physically,
rigid transformation
, including translation and rotation,
must be used.
Moreover,
c
omponents are treated as rigid bodies,
which does not deform during
transformation,
so that they cannot go through each other
, or it can be said, collisio
n should
be considered.
1.3
Related work
According to the design we propose in Chapter 2, this project is divided into two primary
stages: model segmentation and components transformation.
For the splitting process,
some past works
have been done to cut the object
based
some
features.
Some features having been analyzed to perform cutting on the object
are
curvatures,
components shape
and
volume
, or
even the texture
, material
on model surface.
For example,
Sacchi
et al.
[1] introduce a
good
analysis to curvatures on surface. Cutting the model by
curvatures is a
reasonable
approach that remains the model features
, but it may not satisfy
some conditions that the template output requires.
Another significant research uses an
optimized statistical method Conditional Random Fields (CRF), which is illustrated
9
meticulously by Kalogerakis
et al.
[2].
Although
all of them have each own advantages, the
problem of
model segmentation for this projec
t
requires
a good method to split the object
into
components those have the properties the output need.
For the transformation
process
,
there have also been lots of
related
papers
. M
ost of
them
are about the i
nverse kinematics
, discussing different
specific matters
, both in
engineering and computer graphics
.
Wang and Chen [3] provide a
n
optimization method to
solve this problem, considering the physics properties, in mechanical manipulators.
Welman
[4] also briefs about this and suggests some solutio
ns for skeleton manipulation, which is
quite near from this project’s approach.
T
he
closest research
to
what we need is
using path

finding algorithm of S.G.D. Williams [5].
However, similarly to
model segmentation, they do
not completely fit all the
necessary conditions
.
10
Chapter
2
:
Design
2
.1
Workflow diagram
The whole process for the solution has been illustrated by the diagram below in Figure 2.1.
2
.
2
Workflow description
The model read by the program is passed into the
one
preprocessing stage, following by
two
primary stages:
model segmentation and components transformatio
n
.
2.2
.1
Preprocessing
Before going through main stages, the model is preprocessed by
two methods
: simplification
and
triangulation.
The preprocessing will be discussed in detail in Chapter 3.
2.2
.2
Model segmentation
After preprocessing,
the model is passed into the model segmentation stage. The target of this
stage is splitting the model into a fini
te number of components those can be used to transform
Figure 2.1 Design of the Transformer project
11
into a desired shape. There are some effective approaches that we have tried and the detail of
them will be discussed in Chapter 4.
2.2
.3
Components transformation
Following the model segmentation, components transformation is performed to find the
solution transformation to change the model into the given desired shape.
The main method is
solving the inverse kinematics applying on the components, with some known var
iables: the
components initial and final positions.
This stage will be discussed in detail in Chapter 5.
2.2.
4
Feedback and Suggestion
The program is not just about giving the solution, but it also can make suggestion to improve
the
any intermediate
result.
To be more specific, instead of determining if there is a solution,
the program has the ability to evaluate the intermediate result to suggest modification at some
stages so that they can achieve the better output.
Methods used in this section will
be
discussed in Chapter 6.
With the design of the project, we tried to implement and perform on some models.
There were some good results achieved. They are show
n
and discussed in Chapter 7.
2
.
3
Design goals
To reach for the solution effectively, there are some goals
and requirements
those must be
fulfilled
, which are
:
Read an object data or a simplified model as input.
Create, store and retrieve pre

defined templates.
o
Description: In this project, the
template is used mostly in the work skeleton.
This is the structure that we use to sketch the shape of output as well as
simulate the transformation.
Suggest cuts on model based on some regular features.
o
Description: To complete this goal, there should be
any fixed
methods.
Instead, we prefer trying it in different ways to achieve various outputs of
components.
Calculate the movement of components effectively and correctly.
12
o
Description: This goal should be done without violating any criteria mentioned
in th
e next section 2.4.
Moreover, an effective algorithm should be used to
reduce the computation time.
Give mutual feedback and suggestion.
o
Description: We want to make the program a bit more intelligent so that it can
evaluate result and correct itself if
possible.
2
.
4
Criteria
During achieving those goals, there
are
some criteria which should be met to result to realistic
solutions.
Firstly, a limited number of components should be segmented to reduce the
complexity of the transformation. It also reduces the chance that
components
overlap
each
other so there is less intersection checking to perform.
Usually, this number is equal to
the
number of
components
the skeleton needs.
Secondly,
this project is not like puzzle solving
where all pieces are separated, then put at the correct position. What we aim here is that
during the transformation, components should link together.
Besides, a
lthough both rotation
and transformation are allowed in rigid transformation
1
, the former is the main
transformation we want to focus in this project.
Thirdly, since the components and model are
treated as physical rigid body
2
, the components cannot defo
rm themselves or passing through
each other during the transformation.
1 ri gi d transformati on: the transformati on that does not change the shape of the object.
2
ri gi d body: the physi cal body that di stance between every two poi nts of the body remai ns
constant
.
13
Chapter
3
:
Preprocessing
In
preprocessing, th
is project concerns about two things:
simplification and
triangulation
, in
which
only
the latter is compulsory.
Although
the simplification is optional, it is preferred to
perform some method before
the model is given to main process
, because of
some reasons
that the simplification worth
considering.
One reason
is
from the main target of every
simplification: to
reduce
complexity
in
computation
and storage.
As the project is about
computer graphics,
the input contains lots of information and usually
consumes a large
amount of memory
for data storage
or
program
execution.
Using the complex input without
preprocessing would definitely hurt performance a lot.
Moreover,
the solution is mostly about
how to
reach the final state, so during the
transformation,
a simplified component
is
acceptable.
It helps speeding up
the process
without affecting the goal very much.
Another
reason is that t
here have been many good methods
for
surface simplificatio
n
and applying
one of them
is
not difficult
.
For example,
a fast method
using
quadric error metrics
is
introduce
d
by
Garland and Heckbert
[6]
, which is able to simplify a model with thousands
faces in less than a minute.
In contrast,
triangulation is a
mandatory
stage in preprocessing, which is a critical
condition for some work later
such as volume calculation
or surface intersection
.
It is even
essential in the simplificat
ion method we suggested above
[6]
.
The objective of
triangulation
is
to
standardi
ze all the faces of model
so that
they contain an ordered list of three vertices
.
This standard makes the model consistent in computation,
simplifies the surface complexity
as
triangle is the most basic form to represent
a face.
In spite of
the compulsory,
f
ortunately,
triangulation
has been
implemented as a built

in
function
in the used software
.
14
Chapter
4
:
Model segmentation
This stage is completed by my partner, Jiang Kuo. Here are some expectations I need from
his work.
4
.
1
Expectations
The model segmentation stage should finish at least those following requirements so that I
can make use for my work:
Model
is
segmented into finite components.
Those components should either be similar to what the skeleton expects or follow
so
me features.
Those components are simplified or contained in bounding boxes to make fast
intersection detection.
Modification on components is allowed such as chopping out
some minor region of components
or performing simplification on the whole
component.
4
.
2
General view
The model is segmented by using some plane or surface
to separate
components.
Then, using
the boolean functions to input mesh will result to components separated.
Three most used
functions, which are enough for creating any model, are
intersection (
),
union (
), and
difference (

)
.
Figure 4
.1
Cutting methodology
15
In model segmentation, the primary target is to generate reasonable cuts to give components
remaining model features.
Some approaches have been tried using
curvature
estimation
described by Sacchi
et al
.
[1] or
curvature approximation
by
Hamann [7].
Another approach
is based from the expected shape of skeleton bones.
This one tries to generate cuts with the
most similar shape desired.
Some simple past work b
ased on Iterative Closest Point algorithm,
such as
the method of Besl and McKay [
8].
In this project, as the model is not segmented using information stored in the
skeleton,
the cutting method should consider the skeleton as well.
Hence, w
e try to use some
of the
methods above remaining the advantages
, with some modification so that it can adjust the cut
depending on the
skeleton.
The
first
propert
y
we consider
in this project
is
the relational
positions of components.
The component surrounded by most components is most likely the
component in center when matching to the skeleton
. From that
component, other components
can refers to it to locate the most suitable bonds.
My partner calls this is
Neighbor oriented
matching
process.
The second property is the shape and volume of the components.
This is
stored in the skeleton and model segmentation tries to use it to give the similar shape
components.
The next section will show some result of model segmentation stage that my
partner
did.
4
.
3
Results
Starting with model of a car as in Figure 4.
2
a, the desired skeleton in Figure 4.
2
b is read to
configure the cuts on model.
b
Figure 4
.2
a) An inputted car model
b) The human skeleton
a
16
Then, based on information from
the skeleton, configures of cut is generated and
show
n
through the plug

in
UI
panel
in Figure 4.
3
.
This panel stores the cuts in X Y Z
directions with the offset of the cuts. After the model is cut, it separates into components
,
which is shown in Figure 4.
4
.
The components are passed into the Neighbor oriented matching to find the most
center or most important component.
Then, it is matched with the corresponding bone in
skeleton.
Figure 4.
3
UI panel of model segmentation
Figure 4.
4
Car components
17
Chapter
5
:
Components transformation
After
the model is segmented
into components
and matched with
the corresponding bone in
skeleton,
based on initial position (before matching)
,
final position (after matching)
and
component direction
(represented by
quaternion bone)
, the transformation
solution
is
computed
.
Before
introducing the methods used here, there are some
considerably related
matters.
5
.1
Skeleton d
ata structure
The output shape is represented by
a template, called skeleton
or armature
.
Skeleton includes
a number of bones
relating to each other by a hierarchy structure
, in which the
movement
applied for the parent bones is also applied for
the children bones.
Bones are represented by
quaternion
, which is a vector in a four

dimensional vector space.
Three dimensions are
used
in the vector space R3, which
represents the direction of the bone, while the last dimension
represents the roll degree, which is
how much the
bone rolls itself through its axis.
Moreover,
bones have other properties like head and tail, used to locate
its endings
’
positions
, or
some
constraints
like
locking in some degree of freedom.
This simulates the real situation that
some compon
ent can only
transform in some direction.
Also,
considering those constraints
would lead to
an enhance
ment
in performance due to
computation reduction.
Another
advantage of using quaternion
is to prevent the
Gimbal lock
, which may
lock
some axis and
result to unexpected rotation.
(The term
Gimbal lock
description can be found by David Hoag
[9])
.
Each skeleton has a primary bone called the root. This bone is the top parent of other
bone
s
and is not influenced by any movement of other bones.
For convenience,
it is store
d
as
the first bone in the template
so
during
template
information reading
,
it can be ret
rieved
and
created
first
, following by
its children
.
Each bone may contain more than one child
, which
creates
a branching
structure
in the skeleton.
Bones also have other properties that the
input
ted
model can make use.
Each bone contains one head and one tail, which forms the
main axis.
In addition to a de
fault bone support by the software, our bone is a bit modified by
adding
some extra
data.
The modified bone stores the shape it expects.
For example, the bone
18
of a robot head component desires a spherical component.
When model is segment
ed
,
it is
noted
that the bone need
s
which shape and adjust
s
the cut position.
In addition to shape, the
volume percentage of component is also store
d
so that it helps the segmentation stage knows
the relation in size between components.
It is
also
not compulsory that
bones are connected to each other. As can be seen in
Figure
5
.1, the legs are distanced from the lower bone in the main body.
However, using this
has
one disadvantage
.
Particularly,
as everything is allowed to rotate only with the bone’s
head being the ce
nter of rotation,
this offset distance
is a fixed vector
,
so
the starting bone
following the offset always has the head in fixed position
.
An improvement
for this limitation
is using another fake bone instead of
the offset.
This fake bone gives the offset distance
more
freedom of movement
, so the head of the starting bone is
not fixed anymore.
5
.2
Inverse Kinematics
–
a brief investigation
Inverse Kinematics (IK) is the use of kinematics
equation to solve the joint parameters when
the desired end

effector position is given.
For example,
giving a robot arm at some initial
state and a position that the hand wants to reach,
IK solves the equations and gives solution
of
how part
s
of the arm be
nd
, in both direction and degree.
There have been many ways to get
the solution
such as Cyclic Co

ordinate Descent, Jacobian Matrix or Neural Network
.
Although
t
his project IK method uses neither of them,
it would be better to have a brief look
Figure
5
.1 A human skeleton
19
into
those methods
so that we can investigate
both the advantages
and disadvantages
, based
on
the project scope and requirements.
5
.2
.1
Cyclic Co

ordinate Descent
This is an iterative method
which is only able to compute one joint at a time.
It is a simple
approach, easy to understand,
but
does not guarantee
the ability of finding result.
Even if this
method can give a result, it is not sure that the result is valid. Moreover,
as this repeatedly
tr
ies
to get
a better position so that the error
metrics is reduced as much as possible, it is quite
time consuming
to converge to the solution
.
Compared to the result
which is not always
guaranteed and valid
, it can be said that this trade

off is not good at all.
5
.2
.2
Jacobian Matrix
This is very well

known method using in a lot of
Inverse Kinematics Solver Applications.
It
also use
s
iterative method to lower down the error metrics after every small change in
angle/displacement of a joint
through the inverse matrix.
Although it gives solution
eventually,
it still has some
disadvantages.
The matrix is not always irregular, so there are
difficulties during inverse matrix calculation.
Sometimes, the solution is unstable leading to
non

linear and
non

contiguous result.
A
nother thing is
that it may not conv
erge to the best
solution. To be specific, it may st
op
at some local minimal
value of the error
metrics.
5
.2
.3
Neural networks
This is
a machine

learning method
that
may be very useful
if the dataset for training is good.
Besides,
after the data has been trained, it is quite fast to give the solution transformation to
the joint.
However, to evaluate and to train the data takes a long time.
Furthermore,
a large
and good data set should be
used to ensure that
the behavior of the skeleton is correct as
expected.
5
.2.
4
Proposed algorithm
All the above approaches have the same
disadvantage
that is they only consider the position
of the end

effector.
Here in this project,
we
prefer a method
that not only moves the end

effector to the right position but also moves all the joints in the skeleton to the correct
positions.
The reason is
that
a desire shape
, which is
determined by all joints together,
should
be reached.
The second
disadvantage is
that
they
did not c
oncern about physical obstacles
20
during the transformation.
The only possible obstacle is other components except itself
since
there is not any other model having been inputted.
Before trying proposing a new algorithm, some conditions for
the algorithm should be
determined
first:
The algorithm should be decidable. (It will stop eventually)
The solution should be a valid and good solution. If a solution is cannot be found, the
algorithm must acknowledge about this rather than trying until
it gets solution.
The algorithm should concern about the joint constraint and obstacles during
transformation.
Finally, all the bone should reach the nearest state to what we have desired.
There has been a good approach that can satisfy
all of the above co
nditions
,
and was
introduced in
[5].
It uses the path

finding
A*
algorithm.
However, it cannot be applied
directly
into this project.
The first reason is that it treats the bones as line
, so that during
transformation, the chance of intersection is very
low
.
It is also very difficult to detect
intersection.
Compared with our project which mostly transforms the 3

D components, the
chance for collision
, in the other hand,
is quite high.
The second reason is said before that it
only cares about the end

effec
tor,
compared to what we need is th
e
whole skeleton.
As a
result, the heuristic
function
, which bases on the properties a
nd requirements described in [5
]
,
is not
a valid one for this project.
5
.2.
5
Main algorithm
–
first approach
In spite
of not
fitting with the project requirements
, the algorithm described in [5] is still
considerable.
Based on that algorithm, we would like to modify and improve it,
and then
apply it into the project.
Specifically,
the main method is still A
*
path

finding algori
t
h
m.
However, the armature arm is replaced by the skeleton and its attached components so that
we have two linked things move together: the bone and the skeleton.
The skeleton is used
,
firstly, to simplify the transformation applied on components.
For example,
after any
transformation, the center of rotation of components is changed, especially to components
attached to the children bones.
Using skeleton, w
e don’t need to recalculate th
ose
center of
rotation
a
s it can be retrieved by the head of th
e corresponding bone.
Furthermore, the
skeleton bone can be used to represent the components
position and direction
that increases
21
execution speed, as process on bone is always faster than a 3

D model
.
In contrast, the
component is used to check if there i
s collision
, so they
influence the weight or score of the
possible movement.
5
.3
Implementation of A
*
algorithm
Before going through the implementation,
firstly, the component and the skeleton should be
rescaled in same size before the attachment. T
he
n,
the
components should be attached
to the
ir
corresponding
bones
.
A good attachment would lead to a more accurate
score for heuristic
later.
5
.3
.1
Rescale
In addition to the percentage of volume of
component over the total volume stored in
each
bone, the
skeleton also stores the expected total volume
from the model.
When the model and
the skeleton
are
loaded, the volume of model is calculated. Then, it is compared with the
expected volume in the skeleton
of how many times it is larger.
The skeleton is
sc
ale
d
with
that ratio.
5
.3
.2
Attachment
A component is defined by its faces with
forming vertices
.
According to that data,
the center
of a component can be computed. The most common way is to get the average of all vertices
that will give the
component
center of gravity.
This is a case that all the vertices are treated
the same or the component is uniform so we use a uniform weight approach.
However, if the
component contains parts with different materials, it is better to assign different weights to
ver
tices, or component regions to get an exact center.
Once the center
is computed, it is link with the center of the corresponding bone.
Even though those two centers are coincident,
the component and the bone are in two
different directions.
The cause is
that to match them, so far, we only use translation
that shift
s
one of them to another one.
Hence, the direction remains.
Since it is not guarantee that every
bones from template is in same direction of the components,
it results to some weird
movements of
component
during transformation.
For example, the car door
s
may not rotate
around the
ir
hinge
s
.
22
To align component and its bone directions,
after adjusting them coincidentally,
a
common method, Principal component analysis (PCA), is perform
ed
on the component.
This
analysis finds the first principal components of a data set, particularly, the set of vertices of a
component.
The result first principal components would be a line that passes through the
multidimensional mean and minimizes the sum
of squares of the distance from the bone to
the line.
Then,
the component is rotated into
the direction of that line.
Further
more, second or
third principle components can also be computed to align the component in the less important
axis.
For instance, a
fter aligning
along the first principal component, the component still has
freedom that allows it to rotate through this axis, so to perform lower principle components is
considerable
.
As can be seen from
F
igure
5
.2
,
the distance between components
is
not constant
during attachment
, although in the beginning, the distance from the components to their
corresponding is mostly the same.
The first reason is the location of component’s center.
In
side view, although components seem to be align
, their centers those are not displayed are not
Figure
5
.2
Side view and perspective view during components
attachment
23
always
like that.
The second reason
is th
at the components may need to rotate to
be
coincident
with
the first principal component direction.
5
.3
.3
Rotation
method
There are many ways of dealing with rotation in 3

D spaces.
The first one is very simple and
has been introduced for a long time: Eulers
rotation. The rotation is just a vector with three
elements representing the change in xyz

axis.
For example,
(10,

20, 0) is a rotation that
rotates 10 degrees on x

axis,

20 degrees on y

axis
.
Using this method of rotation also
requires a little bit car
efulness
, in which the order of axis in rotation is important.
For
example, rotation of (10,

20, 0) is different from (0, 10,

20).
Furthermore,
t
his
representation, in spite of taking little computer space, suffers from several problems. The
worst thing
is the
Gimble lock
, which is a mathematical problem where some axis gets stuck
or locked.
The second one is using quaternion.
This is a complex related concept, which
represents a vector and an angle. Although the mathematical it uses sometimes is quite
bizarre and difficult,
it prevents all the problems of using Eulers.
Compared to Eulers, it only
has to store one m
ore number wi
th problems solved, and also easier to interpolate.
The third one is using matrix. This is quite a nice method that stores all data of the
object such as rotation, location, scale, or shear. However, this is more than what we actually
need and the computat
ion on matrix is not really fast.
Although each method has its own advantages and disadvantages that they can
co

exist, for this project, we prefer using the quaternion method, as it is similar to the data
structure of the bone and easy to manipulate.
5
.3
.4
Naive
approach
Firstly,
we
would like to
re
state that as the structure of skeleton is hierarchy, every bone
move
ments
will
invoke
its children to move. However, the opposite way is not true: moving a
bone will not
affect
its parent
’s location
.
With the
rotation
method
described above in section
5
.3.3, we can already propose a naive method.
It is just doing the rotation based on the initial
position and destination ignoring the
collision
during transformation.
This metho
d
tries to
rotate a component to th
e desired position
without consider
ing
other
s
’
existence.
Using the
24
two endings location of bones, using interpolation will
result to
the animation of rotation.
The
naive
approach
is presented into following steps:
Traverse the skeleton
from the root
.
During the traversal,
try to rotate the bone so that it can reach the closest point to the
desired destination.
Continue until there is no more bone to rotate.
This method always provides a solution, but its
correctness
is not guaranteed
.
Because
the
colli
sion detection is ignored,
the component does not physically move.
Hence, in the
solution, there may be a chance that the interpolated
position of components overlaps
each
other,
and
it violates the condition of this project.
Although this approach
cannot solve the
problem correctly, it still has some advantages.
It is fast to perform and easy to implement.
It
also provides a good final
shape of model
after transformation if there is a possible solution.
This approach can also be used to generate the
transformation animation
i
f the model is
simple with a small number of components
that they have little chance to collide each other.
5
.3.
5
A
*
algorithm
with
overlapping disallowed
A* algorithm is a popular algorithm used in path

finding
problem.
This is a method that
keeps generating the search space and then, processes with the
thing having the least cost
path to the goal state.
This cost is computed by a heuristic cost function, which includes a
past path

cost function and a future path

cost fun
ction.
The reason we apply this algorithm
into the project is that this is quite strong in finding the way to goal given that there exist
some obstacles during moving.
In this project, as the component is rigid body that cannot
deform or change shape,
the
only possible obstacle is other components.
We initiate the algorithm by add
ing
the initial state
of every bone into open space. For
every bone,
all
possible
closest
position
s
it can
reach by rotation
are
generated.
The
possible
closest positions are
computed by rotating the bone by a small angle in some direction
s
.
In
this project, the angle chosen is 5
o
.
This value is balance, because smaller value leads to
more
neighbor
state to
process
, and larger value skips some important state.
However, not all
of the
closest position
s
are
valid
.
They should be passed to a collision detection phase
, which is at
any generated state, if there
is
any
intersection between two
or
components
, this state is not
chosen.
After
possible
closest positions
are
generated, they are added into the open space
,
25
marked the initial state is their parent state
.
Then, the initial state is deleted from open space
and added to close space so that it cannot be processed another time later.
After this
initialization
, the al
gorithm goes to another stage.
For this stage, we use a heuristic function f(x) to compute the cost

path of any state of
bone.
From the open space, the bone
B
with smallest value of f(x) is chosen, then deleted
from open space and added to close space.
From that smallest bone, all valid closest states are
computed similarly in initialization.
Then, they are added into the open space with the bon
e B
marked as their parent state.
If any of them is in the open space already, their values of
g
(x)
,
cost to re
ach to that state,
are compared. The one with smaller values is kept in open space,
while the other is ignored.
This process repeats until the final state is reached.
Because we
rotate a bone of a small value, there may be a chance that we cannot reach the
exact desired
position for all bone
s
.
As a result, the algorithm may never stop.
To solve this, a tolerance or
error function is created to calculate how different of the current state to the desired state.
If
the value from error function of any state is
smaller than the default value, it is said that it
reaches the goal and stop the algorithm.
Another condition to stop the algorithm is that there
is no solution.
In this case, the program may run infinitely
, as in 3

D space,
there are always
closest state
s to add to open space.
Hence, the program should be stop after a number of
search steps or after a period of time.
To summarize the algorithm, it is divided into following steps
:
1.
The initial state of
all
bone
s
is added to open space.
2.
For every state in the open space,
a state with S smallest value of heuristic cost

path
function is searched.
Figure
5.3
A bone with two possible closest state
s
26
3.
Then,
its closest valid states are computed.
They are added into the open space with
the state
S marked as their parent.
4.
During addition, if
the closest state to be added is in open space, the
cost g(x) to that
closest state
is compared.
The one with smaller g(x) is kept, while the other is ignored.
5.
Repeat steps
2, 3, 4
until the error function of any state is below a default tolerance
, or
a
large number of steps reached, or timeout.
Here in this project, we use a common heuristic for path

finding. g(x) is compute by
how many times of 5
o

rotation it
did to reach there.
h(x) is the
total
distance from every bones
to their final position.
5
.3.
6
Some discussion about algorithm
Because this algorithm is a modified A* algorithm, it
ensures finding out the result
if
existing.
It
is
also
a good approach to
get the way to the final position, considering
the
obstacle blocking the way.
Generating the closest position of bone is very easy
.
However, it still has some disadvantages.
Firstly, it depends on the number of bones
in skeleton.
With a 5
o
difference between states, a bone may have a maximum of 72 states.
As we have to generate the closest states of
all
bones,
an increase of one bone leads to 72
states to com
pute.
To reach the goal, a lot of number of those 72 states will be computed.
Hence
, sometimes, with large skeleton, the performance is really slow.
A bone constraint that
locks the bone from rotating in some direction may be used to reduce the number of
computations.
For example, a 3

degree

of

freedom rotation has 72 cases, compared wit
h only
2 cases in 2

degree

of

freedom rotation.
Secondly,
checking the intersection of components
also hurts performance
.
Complex components
are
slower to perform
intersection detection.
That’s why there is a simplification in preprocessing stage.
Another solution for this is
replace the component by some primitives like cube or sphere and use some fast intersect
algorithm for them.
5
.3.
7
Improvement
on intersection
In the section
5
.3.5, the intersection checking is a
b
oolean function
, which gives
only two
possible results True or False.
Now we consider a bit of the model segmentation stage.
Although we give much information
used, a perfect segmentation is hard to
get.
Because the
skeleton template is fixed, with the information provided from the s
keleton, it cannot apply to
27
all models.
Hence,
sometimes, a tiny change in the model cuts may change the result
completely.
In this section, we suggest a method that can improve the intersection
detection
so
it
can
suggest correctness to model segmentation
.
The algorithm mostly is similar to what we
have discussed in section
5
.3.5. However, the intersection checking
does
not
return
True of
False anymore. Instead,
it gives a smoother result
, by calculating the percentage of
intersection
.
If the percentage is
below
some default value, which means this intersection can
be corrected, then
the algorithm
stores the suggestion and continues
as the checking returns
True. Otherwise, it returns False.
To evaluate
if
the segmentation
i
s able to
be
corrected
, it includes many conditions.
A
condition is the position they overlap.
Side overlap is easier to correct than the case that one
component lies totally inside another one.
Another example is
that
to one component
,
the
overlapping
is so small that it is not important. However, this overlapping region is very
large, for instance, 50%, of the other one. Then,
i
t must tell the segmentation what to correct.
In this project, simply, we just use the volume percentage of the overlapping re
gion to
determine whether it is able to correct.
28
Chapter
6
:
Feedback and Suggestion
The discussion in section
5
.3.7 is an example of feedback from transformation stage to model
segmentation stage.
Besides, there are some extra feedback ability and suggestion that can be
done to improve both stages.
After the model is segmented, it is checked with the skeleton stored data.
The first
thing to check is the number of bone and the number of components.
If they are not equal,
then either of two stages must redo.
If segmentation i
s
redone, it
may perform more cuts to
get more components or
combine some to reduce the total number.
Another way is
that
the
skeleton can suggest itself to add
/
delete some bone to fit with the
missing/extra component.
Additionally,
the system checks not only the quantity.
Some components
after
splitting
have a large difference in volume percentage, compared to what is stored in the
skeleton
.
Then, it is suggested to redo the model segmentation trying another method.
It
Figure
6
.1
Suggestion on a) model segmentation
b) skeleton
29
compares not only the volume percentage but also the shape of components.
The componen
ts
are
matched with the primitive shape stored in the skeleton
. Then,
we
us
e
It
erative
Closest
Point algorithm to fit them as much as possible.
Based on the result, a measurement of how
much they fit is calculated. In this project, we use the intersection
region to evaluate. If that
region is large, they are said to be quite similar, and if it is small, model segmentation is
redone.
30
Chapter
7
:
Results
The algorithm is performed on some models and skeleton with different complexity.
The objective of the model used in Figure
7
.1 is to test if the method can run and how
it behaves.
In
F
igure
7
.1a, it is a simple model with
obvious 3 components.
Then, it is
segmented and attached in Figure
7
.1b. Figure
7
.1c, and Figure
7
.1d shows some random
frame during the transformation into a snake shape.
T
hree algorithms discussed in section
s
5
.3.4,
5
.3.5, and
5
.3.7
are
used. Similarity is
found in both the result and the performance
. All three can generate result very fast
, like
constantly,
with least overlapping during the transformation.
Figure
7
.1
a) A
simple model
b) Model segmented and attached
c) A frame during animation
d) Another random frame
31
Here below in Figure
7
.2 is another model used to test the algorithm.
Figure
7
.3 shows the car after simplification and segmentation.
Figure
7
.2
Car model
Figure
7
.3
Car model after simplification and segmentation
32
Figure
7
.4 shows the
skeleton
and car attachment
.
Figure
7
.5 shows the
final
result after performing the
naive method
.
Figure
7
.4
Human skeleton and components
attachment
Figure
7
.5
Results using naive method
at frame 1, 145, 289, 434
33
To get the
intermediate
result illustrated in Figure
7
.5,
firstly we use the naive
algorithm to sketch the shape of final state.
This one works very fast that returns solution in
seconds.
However, this one is not what we expected, because the door and the body has large
overlapping region.
This can be checked usi
ng the A* algorithm
that disallows intersection
and it keeps running very long.
After this, the door is try to be cut a little bit more to reduce
the overlapping region, using A* algorithm that allows small intersection, the result then can
be
comput
ed aga
in.
Figure
7
.6
Final results
34
Chapter
8
:
Future work
Based on the algorithm we discuss above,
there are some suggestions to continue this project.
As every A* algorithm
s,
the most important thing is the heuristic
cost

path
function.
In this
report, I have proposed a simple way to calculate the cost, which use the total distance to the
final state.
However, it is still not the best function.
Depending on the properties of the
skeleton or model, another heuristic could be used to
reduce
the execution time.
Additionally,
there are some secondary points we did not focus on during doing the
project those still can be improved.
For example,
we use the component directly to detect
intersection. However, this can be improved by replacing them with other primitives, or by
assigning simple bounding boxes around components.
This helps reducing the time for
intersection detection.
Another example i
s that
the data provided at the beginning is enough
so that they can reduce the feedbacks and suggestions.
After each feedback and suggestion
and something makes change, the whole algorithm is performed again to get the
new
solution.
In this project, we le
t the bones rotate freely
. This means that whenever it calculates
its closest states, a large list of result is returned.
This also slows the performance very much.
In fact, bones are not allowed to have a
totally free rotation. Instead, there are some positions
that the bone cannot reach. By storing rotation domain of the bone,
some disallowed states
are ignored before the computation on it starts.
We can also
suggest
some
new
structure to
skeleton
dependin
g on
what movement
of
component
we desire.
Besides,
we can eliminate some initial conditions of this project to
make it more general.
For example, the bone is not limited in rotation or translation anymore.
To simulate some realistic joints, more degree of freedom can be added. However, this may
lead to more complex
situation
and slow performance.
35
Chapter
9
:
Conclusion
Based on the
idea of Transformer, this project proposes a general solution to transform any
model from shape to shape.
The solution is decidable that always
determines
if a solution
exists. Besides, from the context of the project with lots of obstacle during the
transformation, the solution, which is based from A* algorithm is more suitable and faster
than other methods.
In addition, it
provides a realistic solution th
at considers some physical
properties of a real object.
This algorithm is also easy to modify or adjust as there are some
parameters that can be changed depending on the purpose.
For example,
if an accurate
solution is required without considering performa
nce,
the degree
value
to get closest state
s
of
bone can be adjusted to
be smaller.
This algorithm can be extended into other fields rather
than
Transformer toys
manufacture.
An application of this is foldable furniture.
From the function of products, the
algorithm can be modified so that it can generate possible folding ways to some furniture.
Hence, products can change their shape as well as their functions, or simply, they are just
folded to save space.
I believe that r
esearching on this problem would have some influence in
the industry, especially in some fields related to those I just mentioned.
36
References
[1] Sacchi, R., Poliakoff, J.F., Thomas, P.D.. Curvature Estimation for Segmentation of
Triangulated Surfaces.
[2] Kalogerakis, E., Hertzmann, A., Singh, K.. Learning 3D Mesh Segmentation and
Labeling, TOG{29}, Siggraph 2010.
[3] Wang, L.C.T
.
, Chen, C.C.. A combined optimization method for solving the inverse
kinematics problems of mechanical manipulators. IEEE Tra
nsactions on Robotics and
Automation, 7:489

499, August 1991.
[4]
Welman, C.. Inverse kinematics and geometric constraints for articulated figure
manipulation, 1989.
[5]
Williams, S.G.D.. Using the A

Star path

finding algorithm for solving general and
cons
trained inverse kinematics problems, February 10, 2008.
[6]
Garland, M., Heckbert P.S.. Surface simplification using quadric error metrics
, Carnegie
Mellon University.
[7] Hamann, B.. Curvature approximation for triangulated surfaces, Mississippi State
University, 1993.
[8]
Besl, P.J., McKay, N.D.. A Method for Regisration of 3

D Shapes, IEEE Transactions on
pattern analysis and machine intelligence, Vol 14, No 2, February 1992.
[9] Hoag, D.. Apollo Guidance and Navigation: Considerations of Apollo IMU G
imbal Lock,
April 1963.
Comments 0
Log in to post a comment