Brigham Young University

J. Alan Atherton

Michael Goodrich

Brigham Young University

Department of Computer Science

April 9, 2009


Funded in part by Idaho National Laboratory

And Army Research Laboratory

1


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


2


What is a remote manipulator?


Applications

•
USAR

•
EOD

•
Planetary Exploration


3


Remotely operating a robot is difficult

•
“Soda straw”
—

Maintaining situation awareness

•
Time delay

•
Mental workload


Why is this a

problem?

•
Collisions

•
Slow

•
Stressful

Foster
-
Miller Talon

4

All images adopted from

Yanco
, H. A.; Drury, J. L. &
Scholtz
, J.

“Beyond usability evaluation: analysis of human
-
robot interaction at a major robotics competition”

Hum.
-
Comput
. Interact., L. Erlbaum Associates Inc.,
2004
, 19
, 117
-
149

5


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


6

Idaho National Laboratory

UMass Lowell

Bruemmer
, D. J. et al.

“Shared understanding for collaborative
control.”

IEEE Transactions on Systems, Man and
Cybernetics, Part A,
2005
, 35
, 494
-
504

Yanco
, H. A. et al.

“Analysis of Human
-
Robot Interaction for
Urban Search and Rescue.”

Proceedings of the IEEE International Workshop
on Safety, Security and Rescue Robotics,
2006


7

INL / BYU AV Interface

Ferland

et al.
-

Sherbrooke

C. W. Nielsen, M. A. Goodrich, and B. Ricks.


“Ecological
Interfaces for Improving Mobile Robot
Teleoperation
.”


IEEE Transactions on Robotics and
Automation
.


Vol

23, No 5, pp. 927
-
941, October 2007.



Ferland
, F.;
Pomerleau
, F.;
Dinh
, C. T. L. & Michaud, F.

“Egocentric and exocentric
teleoperation

interface
using real
-
time, 3D video projection.”

Proceedings of the 4th ACM/IEEE international
conference on Human robot interaction, ACM,
2009
,
37
-
44

8

NASA
Viz

Nguyen, L. A.;
Bualat
, M.; Edwards, L. J.;
Flueckiger
, L.;
Neveu
, C.;
Schwehr
, K. ..; Wagner, M. D. &
Zbinden
, E.

“Virtual Reality Interfaces for visualization and control of
remote vehicles”

Autonomous Robots,
2001
, 11
, 59
-
68

9

Kelly, A.; Anderson, D.;
Capstick
, E.; Herman, H. &
Rander
, P.

“
Photogeometric

Sensing for Mobile Robot
Control and
Visualisation

Tasks”

Proceedings of the AISB Symposium on New
Frontiers in Human
-
Robot Interaction,
2009


CMU Robotics Institute


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


10


Requirements

•
Ecological

•
Increase situation awareness

•
Manage workload


Existing Interfaces

•
Lack depth information

•
No manipulation support

•
Not designed for real
-
time operation


11



12


Real
-
time

remote
manipulation

13


Robot

•
Build from kit, modify

•
Player driver

•
Motion planning for arm

•
Swiss Ranger driver


Communication

•
Integrate with INL’s system

•
Network data transfer


User Interface

•
OpenGL display

•
Experiment automation



14

Robot
Controller

User
Interface


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


15

Variant

1

Variant

2

Variant 3

Variant 4

Variant 5

Variant 6

3D +
Video

End
Effector

Video

3D

Joint

Robot
Control

Visualization


Task: collect yellow blocks


30 participants


Between
-
subject
comparison

16


Reduce memorization
effects


Minimize damage to arm


Quick change

17

18


19


Joint control


View
-
dependent end
effector

control

20

21


Robot reaches
for point


User moves
point with
joystick


Point
movement
depends on
view
orientation

3D +
Vid
.

End eff.

3D +
Vid
.

Joint

3D

End eff.

3D

Joint

Video

End eff.

Video

Joint

22

3D +
Vid
.

End eff.

3D +
Vid
.

Joint

3D

End eff.

3D

Joint

Video

End eff.

Video

Joint

3D +
Vid
.

End eff.

3D +
Vid
.

Joint

3D

End eff.

3D

Joint

Video

End eff.

Video

Joint

23

Collisions with
posts, box, table

Collisions with
block in final
adjustments



Interface

Measure

3D

+
Vid
.
End eff.

3D +
Vid
.

Joint

3D

End

eff.

3D

Joint

Video

End eff.

Video

Joint

Time to completion

6th

1st

World collisions

3rd

6th

1st

3rd

6th

Block collisions

1st

6th

6th

Preference

1st

1st

6th

1st

Subjective

mental workload

1st

6th

2nd

3rd

24


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


25


Problems

•
Alignment

•
Time lag

•
Cluttered 3D scan model


Changes

•
Stereo camera exterior orientation

•
Interactive robot arm calibration

•
Simple Quickening

•
Scan Pruning



26


Interactive stereo
camera
calibration




Live robot arm
calibration



27

28


29


30


Background


Related Work


Ecological Interface


User Study


Interface Changes from Study


Conclusions and Future Work


31


3D visualization supports SA


Video is faster


3D + video is a good tradeoff


3D + video might reduce workload

32

33


Head tracking


Ecological
camera video


Haptics