Applying a Predictive Display in a Mixed Reality Interface for a Tele-Manipulator

duewestseaurchinΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 4 χρόνια και 7 μήνες)

112 εμφανίσεις

What is the problem?

In a traditional tele-robotic system, extended and variable time delay decreases the
system performance
What is Our Research About?
In our research, we try to minimise the effect of the time delay for tele-robot system
by utilising a mixed reality interface with predictive display (Figure 1).
We apply predictive display in a mixed reality environment for tele-robotics to
improve the manoeuvrability and situational awareness. A virtual 3D arm robot
model position is synchronised with video overlay to show a predictive display from
both a video and virtual scene. We believe, this approach can help the operator to
improve their performance and minimise the effect of the time delay.
What is Mixed Reality?
Mixed Reality Environment
is the combination of visual information between the
video views and 3D computer generated virtual objects and text.
Video view information

Shows the dynamic information of the tele-robot worksite environment.

Increases the operator’s confidence in performing the tasks.
Virtual information

Provides extra information to the operator.

Shows the condition of tele-robot, including its position and orientation.

Can be used as a predictive display.

Produces viewpoint freedom.
What is predictive display?
A predictive display shows the modelled response of the system to the operator
commands prior to their application rather than waiting on feedback from the remote
Applying a Predictive Display in a Mixed
Reality Interface for a Tele-Manipulator

How is the predictive display different from the
traditional system?
How do we approach the predictive display system
in our research?
Predictive Display with Inverse Kinematics


are applied to generate position and rotation of each
3D link in the model
It works by calculating the required motions (position
and rotation of joint angles) to achieve a desire position.
Example: In figure 4, Inverse Kinematics are used to calculate the required
joint angles (
) o f e a c h l i n k wh e n a t a r g e t p o s i t i o n i s

Predictive Display with Coordinate Transform Approach

Coordinate Transformation
is applied to transform the three dimensional
coordinate system (World coordinates – 3D) to a two dimensional
coordinate system (Camera coordinates – 2D).
Figure 1: A Mixed Reality Interface of Tele-robotics system with predictive display as a
visual feedback concept
By Ida Bagus Kerthyayana Manuaba
, ANU CECS iHcc Group,
Supervisor Panel: Dr. Ken Taylor **, Prof. Tom Gedeon *, Assoc. Prof. Henry Gardner *.

Figure 5. Coordinate transformation

Figure 4. Example of different angels and length used for the inverse kinematical problem

Figure 3. The predictive display is used as visual feedback of tele-manipulator
actions within a Mixed Reality framework.
Figure 2. Traditional system
* Research
Computer Science, College of Engineering & Computer Science, Australian National University.
** Commonwealth Science and Industrial Research Organisation (CSIRO) ICT Center, Canberra, Australia.
Predictive Display

Virtual Information

Real Scene Tele-Manipulator

Visual Feedback

Defined target position
Camera Coordinate
2D Position in Image Plane
Position in 3D world
World Coordinate