Mesh-Based Inverse Kinematics - TobW.net

Computer Graphics Group


Tobias Weyand

Mesh
-
Based Inverse
Kinematics

Sumner et al 2005


presented by Tobias Weyand

Computer Graphics Group


Tobias Weyand

2

What is Inverse Kinematics?

•
Articulated body

•
Which angles for a certain
configuration?

•
Forward kinematics

•
Specify angles

•
Inverse kinematics

•
Specify limb position

a
1

a
2

a
3

a
1

a
2

a
3

Computer Graphics Group


Tobias Weyand

3

IK in Computer Graphics

© NVidia

•

Modelling

•

Meaningful deformations

•

Animation

•

Realistic movement

Problems:

•

Skinning time consuming

•

Not everything has bones

Computer Graphics Group


Tobias Weyand

4

Mesh
-
based Inverse Kinematics

Idea: Learn deformations from examples

Computer Graphics Group


Tobias Weyand

5

Overview

•
Introduction

•
Related work

•
MeshIK

•
Feature Vectors

•
Linear Feature Space

•
Nonlinear Feature Space

•
Accelerations

•
Results and Conclusion


Computer Graphics Group


Tobias Weyand

6

Related work: Deformation Transfer

Deformation Transfer for Triangle Meshes


Transfer source deformations to target

Sumner et al 2004

Computer Graphics Group


Tobias Weyand

7

Related work: Deformation Transfer

Triangle deformations:





Least squares problem:



i
i
i
s
S
s

~
i
i
i
t
T
t

~
min
||
||
|
|
1
2




M
i
F
i
i
T
S
But: Limited to example poses

Computer Graphics Group


Tobias Weyand

8

Related work: Shape Interpolation

As
-
rigid
-
as
-
possible Shape Interpolation


•
Morphing of 2D and 3D meshes

•
Considers mesh interior as rigid

Alexa et al. 2000

Computer Graphics Group


Tobias Weyand

9

Related work: Shape Interpolation

•
Triangulate source and target shapes

•
Find locally optimal triangle interpolations

But:

•

Limited to 2 meshes

•

Expensive:

•

Compatible dissection

•

Computations on interior and exterior

Better:

•

Use only surface

Computer Graphics Group


Tobias Weyand

10

Mesh
-
based Inverse Kinematics

Goal:

•
Provide a set of example meshes.

•
MeshIK learns meaningful deformations.

•
Directly move a subset of the mesh
vertices.

•
MeshIK finds a suitable deformation
according to the example meshes.


Computer Graphics Group


Tobias Weyand

11

Feature Vectors

Given: Base mesh P
0
, deformed mesh P

Deformation: set of affine mappings

}
{
j

j
j
j
t
p
T
p



)
(
Deformation gradient: Jacobian of





Discard translation

)
(
p
j

j
j
j
p
j
p
T
t
p
T
D
p
D




)
(
)
(
Computer Graphics Group


Tobias Weyand

12

Feature Vectors

Feature vector: concatentaion of
deformation gradients












3
2
1
3
2
1
3
2
1
z
j
z
j
z
j
y
j
y
j
y
j
x
j
x
j
x
j
j
t
t
t
t
t
t
t
t
t
T


3
1
3
1
3
1
3
2
1
1
1
2
1
1
1
f
z
m
z
y
m
y
x
m
x
x
x
x
t
t
t
t
t
t
t
t
t




Computer Graphics Group


Tobias Weyand

13

Feature Vectors

Calculation of f for mesh P


)
,
(
P
p
p
E
V

T
1
1
1
))
,
,
(
,
),
,
,
((
m
m
m
p
z
y
x
z
y
x
V


P
→
mesh
-
vector x:

T
n
1
n
1
n
3
2
1
)
z
z
y
y
x
x
x
x
(
x




Construct G such that:

Gx
f

Computer Graphics Group


Tobias Weyand

14

Feature Vectors

Properties of G:












g
g
g
G
-

Block
-
diagonal structure

-

Sparse

-

Only depends on P
0

Computer Graphics Group


Tobias Weyand

15

Feature Vectors

Extracting a mesh from a feature vector:

Fix one vertex in x:

•
Set corresponding rows in G to 0.

•
Add product of these rows with x.

c
G


x
~
f
Computer Graphics Group


Tobias Weyand

16

Multiple Vertex Constraints

Transform to least squares problem:

c
G


x
~
f
2
x
f
x
~
min
arg
x





c
G


Properties of x:

•

Close relation to the feature vector

•

Fulfills the vertex constraints

Computer Graphics Group


Tobias Weyand

17

Linear Feature Space

Linear combination of features:

l
l
w
w
w
w
f
f
f
f
2
2
1
1




Vector notation:

w
w
M
f



l
1
f
f
M




T
1
l
w
w
w


Mw
c
Gx
w
x
w
x



,
*
*
min
arg
,


Least squares problem



Minimize for optimal weights and mesh

Computer Graphics Group


Tobias Weyand

18

Linear Feature Space: Problems

Unnatural interpolation of rotations

Goal: Correctly capture rotations

Computer Graphics Group


Tobias Weyand

19

Transformation Interpolation

Linear Combinations of Transformations


Scalar product:



Addition:



Combination:

T

a
B
A

a
T
n
n
n
n
B
A










1
1
lim
T
e
log
a
B
A
e
log
log

n
n
A
A




a
a

1
1
n
n
A
A
e
log
log
1
1
a
a




ˆ

ˆ

ˆ

ˆ

ˆ
Alexa 2002

)
log(
a
T
e

ˆ
Computer Graphics Group


Tobias Weyand

20

Possible Nonlinear Feature Space




l
i
ij
i
T
w
j
e
T
1
log
But:

•

Practically produces singularities

•

Linear scales and shears suffice

MeshIK interpolation:

•

Scales and skews: linear

•

Rotations: above formula

Computer Graphics Group


Tobias Weyand

21

Extracting Rotations

Matrix Animation and Polar Decomposition


Method for factoring a transformation T





T=RS



New transformation combination

Shoemake and Duff 1992







l
i
ij
i
R
w
j
S
w
e
w
T
l
i
ij
i
1
)
log(
1
)
(
Computer Graphics Group


Tobias Weyand

22

Nonlinear Feature Space

)
(
x
min
arg
,
x
,
x
*
*
w
M
c
G
w
w



New nonlinear least squares problem:




Properties:

•

Fulfills vertex constraints

•

Close to nonlinear feature space

•

Natural rotation interpolation

Efficient solver required!

Computer Graphics Group


Tobias Weyand

23

Gauss
-
Newton in MeshIK

Goal: minimize

)
(
x
min
arg
,
x
,
x
*
*
w
M
c
G
w
w



Approach with Gauss
-
Newton:

•

Transform into locally linear equation

•

Solve linear least squares problem

Computer Graphics Group


Tobias Weyand

24

Gauss
-
Newton in MeshIK

Linearize M(w) with Taylor expansion



)
(
)
(
)
(
w
M
D
w
M
w
M
w



||
)
(
)
(
||
min
arg
x
,
x
,
c
w
M
w
M
D
Gx
k
k
w
k
k









Linear least squares problem











x
A
w
M
D
G
k
w
)
(
x
Reduce to one variable:

2
x
,
)
(
min
arg
x
,
c
w
M
x
A
k
k
k
















)
(
k
w
w
M
D
G
A


Computer Graphics Group


Tobias Weyand

25

Gauss
-
Newton in MeshIK

Set

)
)
(
(
2
x
2
x
'
T
T
c
w
M
A
A
A
F
k























= 0

!

)
)
(
(
x
T
T
c
w
M
A
A
A
k











Iteration steps:

•

Solve the normal equation for

•

Update








x




k
k
w
w
1
But: Far too slow! Optimization needed!

Computer Graphics Group


Tobias Weyand

26

Cholesky Factorization

Linear system:

)
)
(
(
x
T
T
c
w
M
A
A
A
k









)
)
(
(
x
T
T
c
w
M
A
C
C
k










)
)
(
(
T
T
c
w
M
A
y
C
k


y
C








x
C
C
A
A
T
T

Decompose:

Method:

y









x
Computer Graphics Group


Tobias Weyand

27

Cholesky Factorization

Further acceleration:

Exploit structure of A
T
A and C
T
C:
























3
1
T
T
3
T
3
3
T
T
3
T
T
2
T
T
1
T
T
i
i
i
J
J
g
J
g
J
J
g
g
J
g
g
g
J
g
g
g
J
g
g
A
A
























3
1
T
T
T
3
T
2
T
1
3
T
T
2
T
T
1
T
T
T
i
i
i
S
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
R
C
C
Note: R
T
R=g
T
g

•

Precompute R
T
R

•

Only calculate R
1
, R
2
, R
3
, R
S






Interactive speed

Computer Graphics Group


Tobias Weyand

28

Performance

Mesh

Tris

Examples

Preprocess

Solve

Flag

932

14

0.016

0.020

Lion

9,996

10

0.0475

0.210

Horse

16,846

4

0.610

0.160

Elephant

84,638

4

13.249

0.906

Computer Graphics Group


Tobias Weyand

29

Results

Computer Graphics Group


Tobias Weyand

30

Video

Computer Graphics Group


Tobias Weyand

31

Comparison: Shape Interpolation

Arap.

MeshIK

2 meshes

n meshes

Needs compatible
dissection of meshes

→

expensive

Needs meshes with
same topology


Operates on exterior
and interior

Operates only on
surface

Computer Graphics Group


Tobias Weyand

32

Comparison: Shape Interpolation

Computer Graphics Group


Tobias Weyand

33

Future work

•
Different mesh representation

•
like subdivision surfaces

•
multiresolution hierarchies

•
Different feature vectors

•
Capture mesh properties better

•
Accelerate system solving

•
Better feature space for many examples

•
Simulate physical effects like inertia

Computer Graphics Group


Tobias Weyand

34

Conclusion

MeshIK

•
provides IK without skeleton

•
more direct and dynamic

•
runs at interactive speeds

•
compares well to other approaches

•
eg As
-
Rigid
-
As
-
Possible shape interpolation

Computer Graphics Group


Tobias Weyand

35




Thank you for your attention!