using Hidden Markov Model

duewestseaurchinΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

52 εμφανίσεις

Skill Learning in Telerobotics
using Hidden Markov Model

by

Gary Holness

Skill Learning


Human performance
stochastic


Repeated trials of
same task

different


Something about the

task


Uncovering
“nature” of data


Most likely performance
rejecting noise


Stochastic

methods perfect


Why HMM?


Double stochastic process


-

observable

process (motion data)


-

hidden

process (mental state/intent)


Parametric

model with
incremental update


Observations as
“symbols”


Unifying framework for
perception

and
action


Likely human performance from
measured
activity


Software architecture


Pre
-
processing to extract observation “symbols”


Algorithm on real
-
data or simulation

SM
2

configuration


7 DOF


6 DOF free flying hand controller provide control
input

HMM experiment


Orbit replacement unit (ORU)


Nut driver in gripper


Model action
under teleoperator control

as
HMM


Position/trajectory in
Cartesian space


Position/trajectory in
Joint space


Velocity/trajectory in
Cartesian space

HMM Experiment cont’d


Observable symbols
: trajectory


States:

subtasks


Special case on states
: time index
increases left
-
to
-
right (Bakis model)


Fewer parameters

than ergodic HMM


100 trajectories recorded and scored

Position trajectory in Cartesian
space


Forward algorithm used for scoring: P(O|

)


Trajectory 60,90 better than average


Score increase w.r.t iteration


model improvement

Position trajectory in joint space


Iteration 77 best score


Velocity in Cartesian, iteration 49 is best

What was good?



Use mathematical framework for which
many


statistical tools already exist


Integrating
framework


Software engineering


Rigorous:

make
sound statements

about


experiment (not just “it works therefore its proven”)


Clearly
laid out design

What was bad?



Poor initialization
: Baum
-
Welch can converge to


local maxima



(not problem of experiment)



Simplification in left
-
right HMM



(understandable why they did it)



Independence assumption

among r.v. in


R
-
dimensional observation vector



(joints non
-
independent)

Why do I care?


Learning in HMM for
ergodic case


Choose
right
“features”

as observation
symbols


Make use of 80
-
years of
statistical

tools


Beautifully engineered research artifacts


Framework for
skills transfer and re
-
use


Transition among HMMs

still an HMM:


lends itself to
hierarchical descriptions

Conclusion

Representation, representation, representation