FYP report (sem 2)x - School of Computing - National University ...

kneewastefulAI and Robotics

Oct 29, 2013 (4 years and 10 days ago)

87 views





B. Comp. Dissertation




Transformer




By

Le Vu



Department of Computer Science

School of Computing

National University of Singapore


2012/21013



1






B. Comp. Dissertation




Transformer




By

Le Vu



Department of Computer Science

School of
Computing

National University of Singapore


2012/21013








Project No:
H087720

Supervisor:

Assistant Professor

Alan Cheng Ho Lun



2



Abstract

The film Transformer of director Michael Bay has been popular recently.
There have been
many products following the theme of the movie.
One of them is the Transformer toys, which
are the small versions of robots from the film.

To generalize the problem, we would like to
investigate
how
to split an object into
finitely many
components and transform them in to a
pre
-
determined shape
,

and to propose a solution
for
the transformation
.

The most common thing used to simulate the movement of a robot is inverse
kinematics solving. That is, the robot is treated a set of joints and li
nked components

and the
movement of robot depends on the joints parameters.
In this report, we would like to use one
of the approaches

to solve the inverse kinematic problem:

A* algorithm. A
lthough it is not
common,
it is effective in the situation of the
report, in which there are many obstacles during
the transformation.

The algorithm has been proved to work
by
achiev
ing some good

result
s
.

I
hope that this report would give a good investigation to the solution for Transformer toys as
well as it can be app
lied for other researches.

Keywords

Transformer
,

Transformation,
Rigid body
,
Degree of freedom, Blender, Autodesk Maya,
Python,
ICP,
Inverse Kinematics, Joints,
Armature,
and Constraints
.


Imple
mentation Software and Hardware

Blender 2.63
, Python 2.7
,
Autodesk

Maya 2012



3


Acknowledgement

During this project,
I always feel
the encouragement
from others. I would have not been able
to do this
without them.

The first one I would like to
show my
appreciation

is my

supervisor,

Mr.
Alan Cheng
Ho
Lun,
for his guidance.
His experienced advice is very precious
that helps me a lot
.
Besides, some suggestion and
side
-
story from him shows me that this field is interesting that

keeps my enthusiasm

to complete the project
.

The second one

I would also
like to thank
is
my partner, Jiang Kuo, who also play
s

an
important role in doing this project.
His
work attitude
keeps reminding

me to try harder.

Furthermore
, sharing difficulties, knowledge,
and fun

is
,

actually
,

making this project more
meaningful to me.

Finally, to my family and friends, you have been encouraging me a lot. A warmest
thank to everyone. Thank you!



4



List of Figures

Figure 1
.1

Transformer

................................
................................
................................
......

7

Figure 2
.1

Design of the
Transformer

project
................................
................................
..

1
0

Figure
4.1

Cutting methodology

................................
................................
......................

14

Figure 4
.2a

An inputted car model

................................
................................
....................

1
5

Figure 4.2b

The human skeleton

................................
................................
........................

15

Figure 4.3

UI panel of model segmentation

................................
................................
.....

16

Figure 4.4

Car components

................................
................................
..............................

1
6

Figure 5.1

A human skeleton

................................
................................
...........................

18

Figure 5.2

Side view and perspective view during
components attachment

...................

22

Figure 5.3

A bone with two possible closest states
................................
..........................

25

Figure 6.1a

Suggestion on model segmentation

................................
................................

28

Figure 6.1b

Suggestion on skeleton

................................
................................
...................

28

Figure 7.1a

A simple model

................................
................................
...............................

30

Figure 7.1b

Model segmented and attached
................................
................................
.......

30

Figure 7.1c

A
frame during animation
................................
................................
...............

30

Figure 7.1d

Another random frame
................................
................................
....................

30

Figure 7.2

Car model
................................
................................
................................
........

31

Figure 7.3

Car model after simplification and segmentation

................................
...........

31

Figure 7.4

Human skeleton and components attachment
................................
.................

3
2

Figure 7.5

Result using naive method at frame 1, 143,

289, 434
................................
.....

32

Figure 7.6

Final result

................................
................................
................................
......

33



5



Table of Contents

Title

1

Abstract

2

Acknowledgement

3

List of Figures

4


Chapter
1 Introduction

................................
................................
.............

7


1.1
Other motivation

................................
................................
................................
..

7


1.2
Goals
................................
................................
................................
......................

8


1.3
Related work

................................
................................
................................
........

8


Chapter 2

Design
................................
................................
.....................
10


2
.1
Workflow diagram
................................
................................
.............................
10


2
.2
Workflow description

................................
................................
.......................
10


2.2.1 Preprocessing
................................
................................
..............................

10


2.2.2 Model segmentation

................................
................................
...................

10


2.2.
3

Components transformation

................................
................................
.......

1
1


2.2.
4

Feedback and Suggestion

................................
................................
...........

1
1


2
.3
Design goals

................................
................................
................................
.......
11


2.4

Criteria

................................
................................
................................
................
1
2


Chapter 3

Preprocessing

................................
................................
.........
13


Chapter 4

Model segmentation

................................
...............................
1
4


4
.1
Expectations

................................
................................
................................
.......
1
4


4
.2
General view

................................
................................
................................
......
1
4


4.3

Results

................................
................................
................................
.................
15


6


Chapter 5

Components transformation

................................
..................
1
7


5
.1
Skeleton data structure

................................
................................
......................
1
7


5
.2
Inverse kinematics


a brief investigation

................................
......................
18


5
.2.1
Cyclic Co
-
ordinate Descent
................................
................................
........

1
9


5
.2.2
Jacobian Matrix

................................
................................
..........................

1
9


5
.2.3
Neural
networks
................................
................................
..........................

1
9


5
.2.4
Proposed algorithm
................................
................................
.....................

1
9


5
.2.
5

Main algorithm


first approach

................................
................................
.

20


5.3

Implementation of A* algorithm

................................
................................
.....
21


5
.
3
.1
Rescale
................................
................................
................................
........

21


5
.
3
.2
Attachment

................................
................................
................................
.

21


5
.
3
.3
Rotation method

................................
................................
.........................

23


5
.
3
.
4

Naive approach

................................
................................
...........................

23


5
.
3
.
5

A* algorithm with overlapping
disallowed

................................
................

2
4


5
.
3
.
6

Some discussion about algorithm

................................
...............................

26


5
.
3
.
7

Improvement on intersection

................................
................................
......

26


Chapter
6

Feedback and Suggestion

................................
.......................
28


Chapter 7

Results

................................
................................
....................
30


Chapter 8

Future work

................................
................................
...........
34


Chapter 9

Conclusion
................................
................................
..............
35


References

................................
................................
............................
36



7



C
hapter

1
:
Introduction

The
project

title
, Transformer, is i
nspired

from

the
same
-
name
movie
by

famous director

Michael Bay
,

where vehicles like car, truck, or plane

are able to

rear
r
ange

themselves

into
robot
s.

In t
his project
,
our objective is

to
simulate the
transformation
of any
object from
initial shape to another determined shape
.

However, d
ifferent from the movie where the
transformation process is done by
visual effect
s
,
our target is to make it more physically
possible.


1.1
Other m
otivation
s

Another motivation of the project is its application into the foldable households such as
furniture. Nowadays

an increasing number of foldable furniture is
introduced
and sold that
makes this type of products is more and more familiar to
customers.
Althoug
h most of
foldable products are
easy to use

for customers
,
to manufacturer
s
,
to design a model
of those
products is really a problem.
For example,
i
t is possible to think of several ways to
split a
box
-
shaped object and then, to rotate

the components into a table.

However,
it is quite a slow
process
by human
to produce different solutions,
although the goal and initial state is the
same:
transform a
box into a table.

Besides, not all solutions
are

good enough

in some criteria
such as too many rotations during the transformation, too many component, or space
consuming.

Hence, this project aims to solve the problem by suggesting some rearrangements
Figure 1.1 Transformer

8


of products components
so that they can rotate
and change the in
itial shape of object
depending on the
given
purpose.

S
olving this problem
may lead to a practical
and
manufacturing meaning

that improves the
process of designing a new foldable model.

1.2
Goals

The
first

goal of the project is

to
suggest some
ways to
split an
in
put
ted

object
into

sub
-
parts
linked by
joints
.
Those joints are
simulated from mechanics joints th
ose

can perform
rigid
transformation

to

objects attaching to them
.

Based on the movement
of the sub
-
parts, j
oints
are determined
and stored in a skeleton
or
a template
, which is pre
-
defined by user.

To
achieve this first goal,
following
criteria should be satisfied
:
the object should not be cut into
too many components
.
We prefer least number of cut
s

which results to

components
remain
ing

the object’s features.

The second goal is to
find a solution
to transform from initial shape to the

given

template

and to
visualize

that transformation process

in an

animation or
a
video
.

During
solving the transformation, some
analysis or suggestion is performed, and then, feedbacks to
improve the splitting process

or to improve the template itself.

Similarly
,
some criteria should
also be satisfied during achieving this goal.
As we want
a realistic solution in which
object
can
change

physically,
rigid transformation
, including translation and rotation,

must be used.

Moreover,
c
omponents are treated as rigid bodies,
which does not deform during
transformation,
so that they cannot go through each other
, or it can be said, collisio
n should
be considered.

1.3
Related work

According to the design we propose in Chapter 2, this project is divided into two primary
stages: model segmentation and components transformation.

For the splitting process,
some past works
have been done to cut the object
based

some
features.
Some features having been analyzed to perform cutting on the object
are
curvatures,
components shape

and
volume
, or
even the texture
, material

on model surface.
For example,
Sacchi
et al.

[1] introduce a

good

analysis to curvatures on surface. Cutting the model by
curvatures is a
reasonable
approach that remains the model features
, but it may not satisfy
some conditions that the template output requires.

Another significant research uses an
optimized statistical method Conditional Random Fields (CRF), which is illustrated
9


meticulously by Kalogerakis
et al.

[2].
Although
all of them have each own advantages, the
problem of
model segmentation for this projec
t

requires

a good method to split the object
into
components those have the properties the output need.

For the transformation

process
,
there have also been lots of
related
papers
. M
ost of
them
are about the i
nverse kinematics
, discussing different
specific matters
, both in
engineering and computer graphics
.
Wang and Chen [3] provide a
n

optimization method to
solve this problem, considering the physics properties, in mechanical manipulators.
Welman
[4] also briefs about this and suggests some solutio
ns for skeleton manipulation, which is
quite near from this project’s approach.
T
he
closest research

to
what we need is
using path
-
finding algorithm of S.G.D. Williams [5].
However, similarly to
model segmentation, they do
not completely fit all the
necessary conditions
.


10



Chapter
2
:
Design

2
.1
Workflow diagram

The whole process for the solution has been illustrated by the diagram below in Figure 2.1.




2
.
2

Workflow description

The model read by the program is passed into the
one
preprocessing stage, following by
two
primary stages:
model segmentation and components transformatio
n
.

2.2
.1
Preprocessing

Before going through main stages, the model is preprocessed by
two methods
: simplification
and
triangulation.

The preprocessing will be discussed in detail in Chapter 3.

2.2
.2

Model segmentation

After preprocessing,
the model is passed into the model segmentation stage. The target of this
stage is splitting the model into a fini
te number of components those can be used to transform
Figure 2.1 Design of the Transformer project

11


into a desired shape. There are some effective approaches that we have tried and the detail of
them will be discussed in Chapter 4.

2.2
.3

Components transformation

Following the model segmentation, components transformation is performed to find the
solution transformation to change the model into the given desired shape.
The main method is
solving the inverse kinematics applying on the components, with some known var
iables: the
components initial and final positions.

This stage will be discussed in detail in Chapter 5.

2.2.
4

Feedback and Suggestion

The program is not just about giving the solution, but it also can make suggestion to improve
the
any intermediate

result.
To be more specific, instead of determining if there is a solution,
the program has the ability to evaluate the intermediate result to suggest modification at some
stages so that they can achieve the better output.

Methods used in this section will

be
discussed in Chapter 6.

With the design of the project, we tried to implement and perform on some models.
There were some good results achieved. They are show
n

and discussed in Chapter 7.

2
.
3

Design goals

To reach for the solution effectively, there are some goals
and requirements
those must be
fulfilled
, which are
:



Read an object data or a simplified model as input.



Create, store and retrieve pre
-
defined templates.

o

Description: In this project, the
template is used mostly in the work skeleton.
This is the structure that we use to sketch the shape of output as well as
simulate the transformation.



Suggest cuts on model based on some regular features.

o

Description: To complete this goal, there should be
any fixed

methods.
Instead, we prefer trying it in different ways to achieve various outputs of
components.



Calculate the movement of components effectively and correctly.

12


o

Description: This goal should be done without violating any criteria mentioned
in th
e next section 2.4.
Moreover, an effective algorithm should be used to
reduce the computation time.



Give mutual feedback and suggestion.

o

Description: We want to make the program a bit more intelligent so that it can
evaluate result and correct itself if
possible.

2
.
4

Criteria

During achieving those goals, there
are
some criteria which should be met to result to realistic
solutions.
Firstly, a limited number of components should be segmented to reduce the
complexity of the transformation. It also reduces the chance that
components

overlap

each
other so there is less intersection checking to perform.
Usually, this number is equal to
the
number of

components

the skeleton needs.
Secondly,
this project is not like puzzle solving
where all pieces are separated, then put at the correct position. What we aim here is that
during the transformation, components should link together.
Besides, a
lthough both rotation
and transformation are allowed in rigid transformation

1
, the former is the main
transformation we want to focus in this project.
Thirdly, since the components and model are
treated as physical rigid body

2
, the components cannot defo
rm themselves or passing through
each other during the transformation.



1 ri gi d transformati on: the transformati on that does not change the shape of the object.

2

ri gi d body: the physi cal body that di stance between every two poi nts of the body remai ns
constant
.

13



Chapter
3
:
Preprocessing

In
preprocessing, th
is project concerns about two things:
simplification and
triangulation
, in
which
only

the latter is compulsory.
Although
the simplification is optional, it is preferred to
perform some method before

the model is given to main process
, because of
some reasons
that the simplification worth
considering.
One reason
is
from the main target of every
simplification: to
reduce
complexity
in

computation

and storage.
As the project is about
computer graphics,
the input contains lots of information and usually
consumes a large
amount of memory
for data storage

or
program
execution.
Using the complex input without
preprocessing would definitely hurt performance a lot.
Moreover,
the solution is mostly about
how to

reach the final state, so during the
transformation,
a simplified component
is
acceptable.
It helps speeding up
the process
without affecting the goal very much.

Another
reason is that t
here have been many good methods
for
surface simplificatio
n

and applying
one of them
is
not difficult
.
For example,

a fast method
using
quadric error metrics

is
introduce
d

by
Garland and Heckbert

[6]
, which is able to simplify a model with thousands
faces in less than a minute.

In contrast,
triangulation is a
mandatory

stage in preprocessing, which is a critical
condition for some work later

such as volume calculation

or surface intersection
.
It is even
essential in the simplificat
ion method we suggested above

[6]
.

The objective of
triangulation
is
to
standardi
ze all the faces of model

so that
they contain an ordered list of three vertices
.
This standard makes the model consistent in computation,
simplifies the surface complexity
as
triangle is the most basic form to represent
a face.
In spite of
the compulsory,
f
ortunately,
triangulation

has been
implemented as a built
-
in
function
in the used software
.



14



Chapter
4
:
Model segmentation

This stage is completed by my partner, Jiang Kuo. Here are some expectations I need from
his work.

4
.
1

Expectations

The model segmentation stage should finish at least those following requirements so that I
can make use for my work:



Model
is
segmented into finite components.



Those components should either be similar to what the skeleton expects or follow
so
me features.



Those components are simplified or contained in bounding boxes to make fast
intersection detection.

Modification on components is allowed such as chopping out
some minor region of components

or performing simplification on the whole
component.

4
.
2

General view

The model is segmented by using some plane or surface

to separate

components.
Then, using
the boolean functions to input mesh will result to components separated.
Three most used
functions, which are enough for creating any model, are
intersection (

),
union (

), and
difference (
-
)
.


Figure 4
.1
Cutting methodology

15


In model segmentation, the primary target is to generate reasonable cuts to give components
remaining model features.

Some approaches have been tried using
curvature
estimation

described by Sacchi
et al
.

[1] or
curvature approximation

by
Hamann [7].

Another approach
is based from the expected shape of skeleton bones.
This one tries to generate cuts with the
most similar shape desired.
Some simple past work b
ased on Iterative Closest Point algorithm,
such as

the method of Besl and McKay [
8].

In this project, as the model is not segmented using information stored in the
skeleton,
the cutting method should consider the skeleton as well.

Hence, w
e try to use some

of the
methods above remaining the advantages
, with some modification so that it can adjust the cut
depending on the
skeleton.

The
first
propert
y

we consider
in this project
is

the relational
positions of components.
The component surrounded by most components is most likely the
component in center when matching to the skeleton
. From that
component, other components
can refers to it to locate the most suitable bonds.

My partner calls this is
Neighbor oriented
matching

process.
The second property is the shape and volume of the components.
This is
stored in the skeleton and model segmentation tries to use it to give the similar shape
components.

The next section will show some result of model segmentation stage that my
partner
did.

4
.
3

Results

Starting with model of a car as in Figure 4.
2
a, the desired skeleton in Figure 4.
2
b is read to
configure the cuts on model.


b

Figure 4
.2

a) An inputted car model


b) The human skeleton

a

16


Then, based on information from
the skeleton, configures of cut is generated and
show
n

through the plug
-
in
UI
panel

in Figure 4.
3
.

This panel stores the cuts in X Y Z
directions with the offset of the cuts. After the model is cut, it separates into components
,
which is shown in Figure 4.
4
.







The components are passed into the Neighbor oriented matching to find the most
center or most important component.

Then, it is matched with the corresponding bone in
skeleton.



Figure 4.
3

UI panel of model segmentation

Figure 4.
4

Car components

17



Chapter
5
:
Components transformation

After
the model is segmented
into components

and matched with
the corresponding bone in
skeleton,
based on initial position (before matching)
,
final position (after matching)
and
component direction
(represented by
quaternion bone)
, the transformation
solution
is
computed
.

Before
introducing the methods used here, there are some
considerably related
matters.


5
.1

Skeleton d
ata structure

The output shape is represented by
a template, called skeleton

or armature
.

Skeleton includes
a number of bones
relating to each other by a hierarchy structure
, in which the
movement
applied for the parent bones is also applied for
the children bones.

Bones are represented by
quaternion
, which is a vector in a four
-
dimensional vector space.
Three dimensions are
used
in the vector space R3, which
represents the direction of the bone, while the last dimension
represents the roll degree, which is
how much the
bone rolls itself through its axis.
Moreover,
bones have other properties like head and tail, used to locate
its endings


positions
, or
some
constraints
like
locking in some degree of freedom.
This simulates the real situation that
some compon
ent can only
transform in some direction.
Also,
considering those constraints
would lead to
an enhance
ment

in performance due to
computation reduction.

Another
advantage of using quaternion
is to prevent the
Gimbal lock
, which may
lock

some axis and
result to unexpected rotation.

(The term
Gimbal lock

description can be found by David Hoag
[9])
.

Each skeleton has a primary bone called the root. This bone is the top parent of other
bone
s

and is not influenced by any movement of other bones.
For convenience,

it is store
d

as
the first bone in the template
so

during
template

information reading
,
it can be ret
rieved
and
created
first
, following by
its children
.

Each bone may contain more than one child
, which
creates
a branching

structure
in the skeleton.

Bones also have other properties that the
input
ted

model can make use.

Each bone contains one head and one tail, which forms the
main axis.
In addition to a de
fault bone support by the software, our bone is a bit modified by
adding
some extra
data.
The modified bone stores the shape it expects.
For example, the bone
18


of a robot head component desires a spherical component.
When model is segment
ed
,
it is
noted
that the bone need
s

which shape and adjust
s

the cut position.
In addition to shape, the
volume percentage of component is also store
d
so that it helps the segmentation stage knows
the relation in size between components.

It is
also
not compulsory that

bones are connected to each other. As can be seen in
Figure
5
.1, the legs are distanced from the lower bone in the main body.

However, using this
has
one disadvantage
.
Particularly,
as everything is allowed to rotate only with the bone’s
head being the ce
nter of rotation,
this offset distance
is a fixed vector
,

so
the starting bone
following the offset always has the head in fixed position
.
An improvement
for this limitation
is using another fake bone instead of
the offset.
This fake bone gives the offset distance
more
freedom of movement
, so the head of the starting bone is
not fixed anymore.



5
.2

Inverse Kinematics



a brief investigation

Inverse Kinematics (IK) is the use of kinematics
equation to solve the joint parameters when
the desired end
-
effector position is given.
For example,
giving a robot arm at some initial
state and a position that the hand wants to reach,
IK solves the equations and gives solution
of
how part
s

of the arm be
nd
, in both direction and degree.

There have been many ways to get
the solution

such as Cyclic Co
-
ordinate Descent, Jacobian Matrix or Neural Network
.

Although
t
his project IK method uses neither of them,
it would be better to have a brief look
Figure
5
.1 A human skeleton

19


into
those methods
so that we can investigate

both the advantages

and disadvantages
, based
on
the project scope and requirements.

5
.2
.1
Cyclic Co
-
ordinate Descent

This is an iterative method
which is only able to compute one joint at a time.
It is a simple
approach, easy to understand,
but

does not guarantee
the ability of finding result.
Even if this
method can give a result, it is not sure that the result is valid. Moreover,
as this repeatedly
tr
ies

to get
a better position so that the error

metrics is reduced as much as possible, it is quite
time consuming

to converge to the solution
.
Compared to the result

which is not always
guaranteed and valid
, it can be said that this trade
-
off is not good at all.

5
.2
.2
Jacobian Matrix

This is very well
-
known method using in a lot of
Inverse Kinematics Solver Applications.
It
also use
s

iterative method to lower down the error metrics after every small change in
angle/displacement of a joint

through the inverse matrix.

Although it gives solution
eventually,
it still has some
disadvantages.
The matrix is not always irregular, so there are
difficulties during inverse matrix calculation.
Sometimes, the solution is unstable leading to
non
-
linear and
non
-
contiguous result.
A
nother thing is
that it may not conv
erge to the best
solution. To be specific, it may st
op

at some local minimal
value of the error
metrics.

5
.2
.3

Neural networks

This is
a machine
-
learning method
that
may be very useful
if the dataset for training is good.
Besides,
after the data has been trained, it is quite fast to give the solution transformation to
the joint.

However, to evaluate and to train the data takes a long time.
Furthermore,
a large
and good data set should be

used to ensure that
the behavior of the skeleton is correct as
expected.

5
.2.
4

Proposed algorithm

All the above approaches have the same
disadvantage

that is they only consider the position
of the end
-
effector.
Here in this project,
we
prefer a method
that not only moves the end
-
effector to the right position but also moves all the joints in the skeleton to the correct
positions.
The reason is

that

a desire shape
, which is

determined by all joints together,
should
be reached.

The second
disadvantage is
that
they
did not c
oncern about physical obstacles
20


during the transformation.
The only possible obstacle is other components except itself

since
there is not any other model having been inputted.

Before trying proposing a new algorithm, some conditions for

the algorithm should be
determined
first:



The algorithm should be decidable. (It will stop eventually)



The solution should be a valid and good solution. If a solution is cannot be found, the
algorithm must acknowledge about this rather than trying until
it gets solution.



The algorithm should concern about the joint constraint and obstacles during
transformation.



Finally, all the bone should reach the nearest state to what we have desired.

There has been a good approach that can satisfy
all of the above co
nditions
,

and was

introduced in
[5].
It uses the path
-
finding
A*
algorithm.
However, it cannot be applied
directly
into this project.
The first reason is that it treats the bones as line
, so that during
transformation, the chance of intersection is very
low
.
It is also very difficult to detect
intersection.
Compared with our project which mostly transforms the 3
-
D components, the
chance for collision
, in the other hand,

is quite high.
The second reason is said before that it
only cares about the end
-
effec
tor,
compared to what we need is th
e
whole skeleton.
As a
result, the heuristic
function
, which bases on the properties a
nd requirements described in [5
]
,

is not
a valid one for this project.

5
.2.
5

Main algorithm


first approach

In spite
of not

fitting with the project requirements
, the algorithm described in [5] is still
considerable.
Based on that algorithm, we would like to modify and improve it,
and then

apply it into the project.
Specifically,
the main method is still A
*

path
-
finding algori
t
h
m.
However, the armature arm is replaced by the skeleton and its attached components so that
we have two linked things move together: the bone and the skeleton.
The skeleton is used
,
firstly, to simplify the transformation applied on components.

For example,
after any
transformation, the center of rotation of components is changed, especially to components
attached to the children bones.
Using skeleton, w
e don’t need to recalculate th
ose

center of
rotation
a
s it can be retrieved by the head of th
e corresponding bone.

Furthermore, the
skeleton bone can be used to represent the components
position and direction

that increases
21


execution speed, as process on bone is always faster than a 3
-
D model
.

In contrast, the
component is used to check if there i
s collision
, so they
influence the weight or score of the
possible movement.

5
.3

Implementation of A
*

algorithm

Before going through the implementation,
firstly, the component and the skeleton should be
rescaled in same size before the attachment. T
he
n,
the

components should be attached
to the
ir

corresponding

bones
.
A good attachment would lead to a more accurate
score for heuristic
later.

5
.3
.1
Rescale

In addition to the percentage of volume of
component over the total volume stored in
each
bone, the
skeleton also stores the expected total volume

from the model.
When the model and
the skeleton
are

loaded, the volume of model is calculated. Then, it is compared with the
expected volume in the skeleton

of how many times it is larger.
The skeleton is
sc
ale
d

with
that ratio.

5
.3
.2

Attachment

A component is defined by its faces with
forming vertices
.
According to that data,
the center
of a component can be computed. The most common way is to get the average of all vertices
that will give the
component
center of gravity.
This is a case that all the vertices are treated
the same or the component is uniform so we use a uniform weight approach.
However, if the
component contains parts with different materials, it is better to assign different weights to
ver
tices, or component regions to get an exact center.

Once the center
is computed, it is link with the center of the corresponding bone.
Even though those two centers are coincident,
the component and the bone are in two
different directions.
The cause is
that to match them, so far, we only use translation

that shift
s

one of them to another one.
Hence, the direction remains.
Since it is not guarantee that every
bones from template is in same direction of the components,
it results to some weird
movements of

component
during transformation.
For example, the car door
s

may not rotate
around the
ir

hinge
s
.

22


To align component and its bone directions,
after adjusting them coincidentally,
a
common method, Principal component analysis (PCA), is perform
ed

on the component.
This
analysis finds the first principal components of a data set, particularly, the set of vertices of a
component.
The result first principal components would be a line that passes through the
multidimensional mean and minimizes the sum

of squares of the distance from the bone to
the line.

Then,
the component is rotated into
the direction of that line.
Further
more, second or
third principle components can also be computed to align the component in the less important
axis.

For instance, a
fter aligning
along the first principal component, the component still has
freedom that allows it to rotate through this axis, so to perform lower principle components is
considerable
.



As can be seen from

F
igure

5
.2
,
the distance between components
is

not constant
during attachment
, although in the beginning, the distance from the components to their
corresponding is mostly the same.

The first reason is the location of component’s center.
In
side view, although components seem to be align
, their centers those are not displayed are not
Figure
5
.2

Side view and perspective view during components
attachment

23


always
like that.
The second reason
is th
at the components may need to rotate to

be

coincident
with
the first principal component direction.

5
.3
.3

Rotation

method

There are many ways of dealing with rotation in 3
-
D spaces.
The first one is very simple and
has been introduced for a long time: Eulers

rotation. The rotation is just a vector with three
elements representing the change in xyz
-
axis.
For example,
(10,
-
20, 0) is a rotation that
rotates 10 degrees on x
-
axis,
-
20 degrees on y
-
axis
.

Using this method of rotation also
requires a little bit car
efulness
, in which the order of axis in rotation is important.
For
example, rotation of (10,
-
20, 0) is different from (0, 10,
-
20).
Furthermore,
t
his
representation, in spite of taking little computer space, suffers from several problems. The
worst thing
is the
Gimble lock
, which is a mathematical problem where some axis gets stuck

or locked.

The second one is using quaternion.
This is a complex related concept, which
represents a vector and an angle. Although the mathematical it uses sometimes is quite
bizarre and difficult,
it prevents all the problems of using Eulers.
Compared to Eulers, it only
has to store one m
ore number wi
th problems solved, and also easier to interpolate.

The third one is using matrix. This is quite a nice method that stores all data of the
object such as rotation, location, scale, or shear. However, this is more than what we actually
need and the computat
ion on matrix is not really fast.

Although each method has its own advantages and disadvantages that they can

co
-
exist, for this project, we prefer using the quaternion method, as it is similar to the data
structure of the bone and easy to manipulate.

5
.3
.4

Naive
approach

Firstly,
we

would like to
re
state that as the structure of skeleton is hierarchy, every bone
move
ments

will
invoke

its children to move. However, the opposite way is not true: moving a
bone will not
affect

its parent
’s location
.
With the
rotation

method

described above in section
5
.3.3, we can already propose a naive method.
It is just doing the rotation based on the initial
position and destination ignoring the
collision

during transformation.

This metho
d

tries to

rotate a component to th
e desired position

without consider
ing

other
s


existence.

Using the
24


two endings location of bones, using interpolation will
result to

the animation of rotation.
The
naive
approach

is presented into following steps:



Traverse the skeleton

from the root
.



During the traversal,
try to rotate the bone so that it can reach the closest point to the
desired destination.



Continue until there is no more bone to rotate.

This method always provides a solution, but its
correctness

is not guaranteed
.

Because
the
colli
sion detection is ignored,
the component does not physically move.

Hence, in the
solution, there may be a chance that the interpolated

position of components overlaps

each
other,
and
it violates the condition of this project.
Although this approach
cannot solve the
problem correctly, it still has some advantages.
It is fast to perform and easy to implement.
It
also provides a good final
shape of model

after transformation if there is a possible solution.
This approach can also be used to generate the

transformation animation
i
f the model is
simple with a small number of components

that they have little chance to collide each other.

5
.3.
5

A
*

algorithm

with
overlapping disallowed

A* algorithm is a popular algorithm used in path
-
finding

problem.
This is a method that
keeps generating the search space and then, processes with the
thing having the least cost
path to the goal state.

This cost is computed by a heuristic cost function, which includes a
past path
-
cost function and a future path
-
cost fun
ction.

The reason we apply this algorithm
into the project is that this is quite strong in finding the way to goal given that there exist
some obstacles during moving.
In this project, as the component is rigid body that cannot
deform or change shape,
the
only possible obstacle is other components.

We initiate the algorithm by add
ing

the initial state
of every bone into open space. For
every bone,
all

possible
closest
position
s

it can
reach by rotation

are

generated.
The
possible
closest positions are
computed by rotating the bone by a small angle in some direction
s
.
In
this project, the angle chosen is 5
o
.
This value is balance, because smaller value leads to
more
neighbor
state to
process
, and larger value skips some important state.

However, not all
of the
closest position
s

are

valid
.

They should be passed to a collision detection phase
, which is at
any generated state, if there
is

any

intersection between two
or
components
, this state is not
chosen.

After
possible
closest positions
are

generated, they are added into the open space
,
25


marked the initial state is their parent state
.
Then, the initial state is deleted from open space
and added to close space so that it cannot be processed another time later.

After this

initialization
, the al
gorithm goes to another stage.

For this stage, we use a heuristic function f(x) to compute the cost
-
path of any state of
bone.
From the open space, the bone
B
with smallest value of f(x) is chosen, then deleted
from open space and added to close space.
From that smallest bone, all valid closest states are
computed similarly in initialization.

Then, they are added into the open space with the bon
e B
marked as their parent state.
If any of them is in the open space already, their values of
g
(x)
,
cost to re
ach to that state,

are compared. The one with smaller values is kept in open space,
while the other is ignored.

This process repeats until the final state is reached.
Because we
rotate a bone of a small value, there may be a chance that we cannot reach the

exact desired
position for all bone
s
.
As a result, the algorithm may never stop.
To solve this, a tolerance or
error function is created to calculate how different of the current state to the desired state.
If
the value from error function of any state is

smaller than the default value, it is said that it
reaches the goal and stop the algorithm.

Another condition to stop the algorithm is that there
is no solution.
In this case, the program may run infinitely
, as in 3
-
D space,
there are always
closest state
s to add to open space.

Hence, the program should be stop after a number of
search steps or after a period of time.



To summarize the algorithm, it is divided into following steps
:

1.

The initial state of
all
bone
s

is added to open space.

2.

For every state in the open space,

a state with S smallest value of heuristic cost
-
path
function is searched.

Figure
5.3

A bone with two possible closest state
s

26


3.

Then,
its closest valid states are computed.

They are added into the open space with
the state

S marked as their parent.

4.

During addition, if
the closest state to be added is in open space, the
cost g(x) to that
closest state
is compared.
The one with smaller g(x) is kept, while the other is ignored.

5.

Repeat steps

2, 3, 4

until the error function of any state is below a default tolerance
, or
a
large number of steps reached, or timeout.

Here in this project, we use a common heuristic for path
-
finding. g(x) is compute by
how many times of 5
o
-
rotation it
did to reach there.

h(x) is the

total

distance from every bones
to their final position.

5
.3.
6
Some discussion about algorithm

Because this algorithm is a modified A* algorithm, it
ensures finding out the result
if
existing.
It
is
also
a good approach to
get the way to the final position, considering
the
obstacle blocking the way.
Generating the closest position of bone is very easy
.

However, it still has some disadvantages.
Firstly, it depends on the number of bones
in skeleton.
With a 5
o

difference between states, a bone may have a maximum of 72 states.
As we have to generate the closest states of
all

bones,

an increase of one bone leads to 72
states to com
pute.

To reach the goal, a lot of number of those 72 states will be computed.
Hence
, sometimes, with large skeleton, the performance is really slow.

A bone constraint that
locks the bone from rotating in some direction may be used to reduce the number of
computations.
For example, a 3
-
degree
-
of
-
freedom rotation has 72 cases, compared wit
h only
2 cases in 2
-
degree
-
of
-
freedom rotation.

Secondly,
checking the intersection of components
also hurts performance
.
Complex components
are

slower to perform
intersection detection.

That’s why there is a simplification in preprocessing stage.

Another solution for this is
replace the component by some primitives like cube or sphere and use some fast intersect
algorithm for them.

5
.3.
7

Improvement

on intersection

In the section
5
.3.5, the intersection checking is a
b
oolean function
, which gives

only two
possible results True or False.

Now we consider a bit of the model segmentation stage.
Although we give much information
used, a perfect segmentation is hard to
get.
Because the
skeleton template is fixed, with the information provided from the s
keleton, it cannot apply to
27


all models.
Hence,
sometimes, a tiny change in the model cuts may change the result
completely.

In this section, we suggest a method that can improve the intersection
detection

so

it
can

suggest correctness to model segmentation
.

The algorithm mostly is similar to what we
have discussed in section
5
.3.5. However, the intersection checking
does

not
return

True of
False anymore. Instead,
it gives a smoother result
, by calculating the percentage of
intersection
.

If the percentage is

below
some default value, which means this intersection can
be corrected, then
the algorithm

stores the suggestion and continues
as the checking returns
True. Otherwise, it returns False.

To evaluate
if

the segmentation

i
s able to
be
corrected
, it includes many conditions.
A
condition is the position they overlap.
Side overlap is easier to correct than the case that one
component lies totally inside another one.
Another example is
that
to one component
,

the
overlapping
is so small that it is not important. However, this overlapping region is very
large, for instance, 50%, of the other one. Then,
i
t must tell the segmentation what to correct.

In this project, simply, we just use the volume percentage of the overlapping re
gion to
determine whether it is able to correct.



28



Chapter
6
:
Feedback and Suggestion

The discussion in section
5
.3.7 is an example of feedback from transformation stage to model
segmentation stage.

Besides, there are some extra feedback ability and suggestion that can be
done to improve both stages.

After the model is segmented, it is checked with the skeleton stored data.
The first
thing to check is the number of bone and the number of components.
If they are not equal,
then either of two stages must redo.
If segmentation i
s

redone, it
may perform more cuts to
get more components or
combine some to reduce the total number.
Another way is

that

the
skeleton can suggest itself to add
/
delete some bone to fit with the
missing/extra component.



Additionally,
the system checks not only the quantity.
Some components

after
splitting

have a large difference in volume percentage, compared to what is stored in the
skeleton
.

Then, it is suggested to redo the model segmentation trying another method.

It
Figure
6
.1
Suggestion on a) model segmentation

b) skeleton

29


compares not only the volume percentage but also the shape of components.
The componen
ts

are

matched with the primitive shape stored in the skeleton
. Then,
we
us
e

It
erative
Closest
Point algorithm to fit them as much as possible.
Based on the result, a measurement of how
much they fit is calculated. In this project, we use the intersection
region to evaluate. If that
region is large, they are said to be quite similar, and if it is small, model segmentation is
redone.



30



Chapter
7
:
Results

The algorithm is performed on some models and skeleton with different complexity.




The objective of the model used in Figure
7
.1 is to test if the method can run and how
it behaves.
In
F
igure
7
.1a, it is a simple model with
obvious 3 components.
Then, it is
segmented and attached in Figure
7
.1b. Figure
7
.1c, and Figure

7
.1d shows some random
frame during the transformation into a snake shape.

T
hree algorithms discussed in section
s

5
.3.4,
5
.3.5, and
5
.3.7

are

used. Similarity is
found in both the result and the performance
. All three can generate result very fast
, like
constantly,

with least overlapping during the transformation.

Figure
7
.1

a) A

simple model

b) Model segmented and attached


c) A frame during animation

d) Another random frame

31


Here below in Figure
7
.2 is another model used to test the algorithm.



Figure
7
.3 shows the car after simplification and segmentation.




Figure
7
.2

Car model

Figure
7
.3

Car model after simplification and segmentation

32


Figure
7
.4 shows the
skeleton

and car attachment
.



Figure
7
.5 shows the
final

result after performing the
naive method
.


Figure
7
.4

Human skeleton and components
attachment

Figure
7
.5

Results using naive method

at frame 1, 145, 289, 434

33



To get the

intermediate

result illustrated in Figure
7
.5,
firstly we use the naive
algorithm to sketch the shape of final state.
This one works very fast that returns solution in
seconds.
However, this one is not what we expected, because the door and the body has large
overlapping region.
This can be checked usi
ng the A* algorithm
that disallows intersection
and it keeps running very long.

After this, the door is try to be cut a little bit more to reduce
the overlapping region, using A* algorithm that allows small intersection, the result then can
be
comput
ed aga
in.





Figure
7
.6

Final results

34



Chapter
8
:
Future work

Based on the algorithm we discuss above,
there are some suggestions to continue this project.
As every A* algorithm
s,

the most important thing is the heuristic
cost
-
path
function.
In this
report, I have proposed a simple way to calculate the cost, which use the total distance to the
final state.
However, it is still not the best function.
Depending on the properties of the
skeleton or model, another heuristic could be used to
reduce

the execution time.

Additionally,
there are some secondary points we did not focus on during doing the
project those still can be improved.
For example,
we use the component directly to detect
intersection. However, this can be improved by replacing them with other primitives, or by
assigning simple bounding boxes around components.
This helps reducing the time for
intersection detection.
Another example i
s that

the data provided at the beginning is enough
so that they can reduce the feedbacks and suggestions.
After each feedback and suggestion
and something makes change, the whole algorithm is performed again to get the
new
solution.

In this project, we le
t the bones rotate freely
. This means that whenever it calculates
its closest states, a large list of result is returned.
This also slows the performance very much.
In fact, bones are not allowed to have a
totally free rotation. Instead, there are some positions
that the bone cannot reach. By storing rotation domain of the bone,
some disallowed states
are ignored before the computation on it starts.

We can also

suggest
some

new
structure to
skeleton
dependin
g on
what movement

of
component

we desire.
Besides,
we can eliminate some initial conditions of this project to
make it more general.
For example, the bone is not limited in rotation or translation anymore.
To simulate some realistic joints, more degree of freedom can be added. However, this may
lead to more complex
situation
and slow performance.



35



Chapter
9
:
Conclusion

Based on the
idea of Transformer, this project proposes a general solution to transform any
model from shape to shape.

The solution is decidable that always
determines

if a solution
exists. Besides, from the context of the project with lots of obstacle during the
transformation, the solution, which is based from A* algorithm is more suitable and faster
than other methods.

In addition, it

provides a realistic solution th
at considers some physical
properties of a real object.

This algorithm is also easy to modify or adjust as there are some
parameters that can be changed depending on the purpose.

For example,

if an accurate
solution is required without considering performa
nce,

the degree
value
to get closest state
s

of
bone can be adjusted to
be smaller.

This algorithm can be extended into other fields rather

than

Transformer toys
manufacture.
An application of this is foldable furniture.
From the function of products, the
algorithm can be modified so that it can generate possible folding ways to some furniture.
Hence, products can change their shape as well as their functions, or simply, they are just
folded to save space.

I believe that r
esearching on this problem would have some influence in
the industry, especially in some fields related to those I just mentioned.



36



References

[1] Sacchi, R., Poliakoff, J.F., Thomas, P.D.. Curvature Estimation for Segmentation of
Triangulated Surfaces.

[2] Kalogerakis, E., Hertzmann, A., Singh, K.. Learning 3D Mesh Segmentation and
Labeling, TOG{29}, Siggraph 2010.

[3] Wang, L.C.T
.
, Chen, C.C.. A combined optimization method for solving the inverse
kinematics problems of mechanical manipulators. IEEE Tra
nsactions on Robotics and
Automation, 7:489
-
499, August 1991.

[4]

Welman, C.. Inverse kinematics and geometric constraints for articulated figure
manipulation, 1989.

[5]

Williams, S.G.D.. Using the A
-
Star path
-
finding algorithm for solving general and
cons
trained inverse kinematics problems, February 10, 2008.

[6]
Garland, M., Heckbert P.S.. Surface simplification using quadric error metrics
, Carnegie
Mellon University.

[7] Hamann, B.. Curvature approximation for triangulated surfaces, Mississippi State
University, 1993.

[8]

Besl, P.J., McKay, N.D.. A Method for Regisration of 3
-
D Shapes, IEEE Transactions on
pattern analysis and machine intelligence, Vol 14, No 2, February 1992.

[9] Hoag, D.. Apollo Guidance and Navigation: Considerations of Apollo IMU G
imbal Lock,
April 1963.