What is the problem?
In a traditional tele-robotic system, extended and variable time delay decreases the
What is Our Research About?
In our research, we try to minimise the effect of the time delay for tele-robot system
by utilising a mixed reality interface with predictive display (Figure 1).
We apply predictive display in a mixed reality environment for tele-robotics to
improve the manoeuvrability and situational awareness. A virtual 3D arm robot
model position is synchronised with video overlay to show a predictive display from
both a video and virtual scene. We believe, this approach can help the operator to
improve their performance and minimise the effect of the time delay.
What is Mixed Reality?
Mixed Reality Environment
is the combination of visual information between the
video views and 3D computer generated virtual objects and text.
Video view information
Shows the dynamic information of the tele-robot worksite environment.
Increases the operator’s confidence in performing the tasks.
Provides extra information to the operator.
Shows the condition of tele-robot, including its position and orientation.
Can be used as a predictive display.
Produces viewpoint freedom.
What is predictive display?
A predictive display shows the modelled response of the system to the operator
commands prior to their application rather than waiting on feedback from the remote
Applying a Predictive Display in a Mixed
Reality Interface for a Tele-Manipulator
How is the predictive display different from the
How do we approach the predictive display system
in our research?
Predictive Display with Inverse Kinematics
are applied to generate position and rotation of each
3D link in the model
It works by calculating the required motions (position
and rotation of joint angles) to achieve a desire position.
Example: In figure 4, Inverse Kinematics are used to calculate the required
joint angles (
) o f e a c h l i n k wh e n a t a r g e t p o s i t i o n i s
Predictive Display with Coordinate Transform Approach
is applied to transform the three dimensional
coordinate system (World coordinates – 3D) to a two dimensional
coordinate system (Camera coordinates – 2D).
Figure 1: A Mixed Reality Interface of Tele-robotics system with predictive display as a
visual feedback concept
By Ida Bagus Kerthyayana Manuaba
, ANU CECS iHcc Group, email@example.com.
Supervisor Panel: Dr. Ken Taylor **, Prof. Tom Gedeon *, Assoc. Prof. Henry Gardner *.
Figure 5. Coordinate transformation
Figure 4. Example of different angels and length used for the inverse kinematical problem
Figure 3. The predictive display is used as visual feedback of tele-manipulator
actions within a Mixed Reality framework.
Figure 2. Traditional system
Computer Science, College of Engineering & Computer Science, Australian National University.
** Commonwealth Science and Industrial Research Organisation (CSIRO) ICT Center, Canberra, Australia.
Real Scene Tele-Manipulator
Defined target position
2D Position in Image Plane
Position in 3D world