Ground-based Telerobotic Interfaces for Space Telescience

duewestseaurchinΤεχνίτη Νοημοσύνη και Ρομποτική

14 Νοε 2013 (πριν από 3 χρόνια και 4 μήνες)

133 εμφανίσεις

Ground-based Telerobotic Interfaces for Space Telescience
Paolo Fiorini Jorge Manzano Alfredo Castro
School of Science Jet Propulsion Laboratory Divisione Robotica Instituto de Automática
University of Verona Caltech ENEA Universidad Nacional de San Juan
Italy USA Italy Argentina
So far, Earth-based space science has been carried out from a few dedicated facilities, with
scientists working away from their home institution and with limited participation of the
science community to the experiments. A better approach to ground control of space
experiments would consist of deploying a network of operation stations located at various
institutions and connected via Internet to the relay station communicating with the space
experiment. In order to achieve this objective, several basic technologies need development,
in particular Internet-based teleoperation, which would support science experiments requiring
manipulation of delicate samples. This paper summarises two of our current efforts in the
areas of compensation of Internet time-delay in teleoperation, and of integration of a PC-
based virtual environment with a force reflecting device. These results, combined with
advances in visual-only operator interfaces using web browsing programs, should eventually
lead to the development of distributed, force-reflecting control stations for ground-based
space telescience.
1. Introduction
It is well known that Space Station astronauts will have very limited time to attend science
experiments. Similarly, experiments located in geostationary orbits, or requiring a quieter
environment than the Station, will be unmanned. It is therefore very important to develop
advanced space telerobotic capabilities to carry out these experiments from a remote location,
on Earth or on Station, using telemetry data and teleoperated devices.
Telerobotics capabilities for interactive space telescience are of particular importance for
microgravity experiments requiring the direct manipulation of small, fragile samples. This is
the case, for example, of Protein Crystal Growth (PCG) directed at supporting
crystallographic structure determination. Protein crystallography is currently the principal
method for determining the structure of complex biological molecules, and it requires
relatively large, well-ordered single crystals of useful morphology. Crystals with these
qualities are difficult to produce on Earth for a variety of reasons, some of which are
influenced by gravity. Experiments conducted aboard the Space Shuttle have provided
persuasive evidence that PCG in microgravity yields improvements for a variety of protein
samples. Furthermore, recent studies indicate PCG as one of the microgravity experiments
most likely to receive financial support from industry. These two reasons provide a good
motivation to study space telerobotics techniques. So far, there have been only two
experiments of space teleoperation, the first flown by the German and the second by the
Japanese Space Agency, and they have been both carried out using dedicated facilities in the
two Countries. This type of operation is acceptable for proof of concept demonstrations, but it
is not conceivable for the day-to-day operation of scientific space facilities carrying tens,
possibly hundreds, of long term experiments. Dedicated facilities allow only a very limited
number of scientists to participate to the experiment, are very expensive, and force the
scientist to work from these facilities away from their home institution. These problems were
evident during the extended Mars Pathfinder mission, originally designed for a two-week
duration. A more sensible approach to mission operation would be to leave each scientist at
his/her home institution and access the space experiments using a public computer network,
such as Internet. This approach allows a significant reduction of mission operation cost, an
increase in productivity of the science teams and, even more important, exposes many more
researchers to space experiments. An example of this approach is the Web Interface for
Telescience (WITS), an interactive operator interface currently under development at NASA
Jet Propulsion Laboratory for Mars exploration missions [3]. However, in spite of its
simplicity and elegance, WITS does not support real-time functions and has no provision for
telemanipulation. Precise telemanipulation requires to faithfully feed the experiment forces
back to the ground operator. However, force-reflecting telemanipulation is made very difficult
by the presence of communication time-delays in the communication from space to the Earth
relay station, and from there to the various mission control stations. Due to the unpredictable
load of these communication channels, it is also likely that the delay is variable. To simulate
this situation we use Internet to connect the master and the slave of a telerobot, and we call
Internet-based telerobot this type of system. These telerobotic devices are also potentially
very important on their own merit, since they would stimulate the design of new interactive
telepresence applications. Unfortunately, the control of time-delay systems achieves stability
at the expense of altering the users perception of the remote environment, thus making it
almost impossible to manipulate remote samples accurately.
The control of time-delay robotic devices has mostly addressed the case of constant delay.
The main approaches include passivity theory [1], remote compliance control [18], and wave
variable decomposition [20]. Some of these methods achieve IOD stability, i.e. independent
of delay, as shown in [10]. However, they are not directly applicable to variable time-delay
systems, since a fixed-delay controller may become unstable when the delay is variable [16].
Variable time-delay is compensated by estimating the original data using an n-step predictor
[19], or by including the delay variations directly into the design procedure [7]. One key issue
of variable time-delay compensation is the availability of a model of the delay source. In the
case of Internet-based telerobotics, the model represents a network of computers,
communicating using packet-switched protocols [8]. The delay affecting the data packets
depends on the packet's routes, on the different handling policies at each node traversed, and
on the network congestion [4]. An approximate model consists of queues at each node in the
network, introducing the variable delay and of network congestion causing packets losses
[14]. This model is characterized by the statistics of the packet delay and of the packet losses.
Values of these parameters have been known for some time [5], but, because of the rapid
Internet growth and variable loading conditions, they need frequent updates and on-line
identification [11]. Other issues related to distributed telerobotic systems regard modeling
Figure 1: The experimental set-up.
[9,15], real-time architectures [12,23], and control issues [2]. Force-reflecting interfaces are
also receiving significant attention by several research communities, for example, a few
interesting commercial systems are available for games and virtual-reality applications, but
have limited capabilities to connect to a remote system [13,17,22]. This paper presents results
in two areas relevant to distributed telerobotics. Next Section briefly summarizes experiments
of Internet time-delay compensation carried out using a virtual environment. Section 3
presents the integration of a commercial force-reflecting joystick within a standard desk-top
personal computer. Finally, Section 4 summarizes the paper and presents our plans to
integrate these two technologies into and end-to-end demonstration of Internet teleoperation.
2. Experiments of Internet-Based Force-Reflection
As mentioned earlier, the main factor influencing the control of Internet-based robots is the
transmission delay between the master and the slave. This problem is addressed in [21] by
developing a simple system to design and test Internet-based force reflecting devices. The set-
up consists of a laptop personal computer (PC) using the DOS operating system, and a planar
two degree-of-freedom (dof) direct drive device used as the teleoperation master, as shown in
Figure 1. The computer controls the position and torque of the master, while executing the
simulation of a virtual slave manipulator and several objects. The virtual slave robot follows
the position of the master and the forces generated by the virtual impacts between the robot
and the objects are reflected to the master. To simulate an Internet-based system, we connect
the master to the slave using a remote reflector, i.e. a computer returning the data to the
sender. With this configuration, data are sent from the master to the virtual slave through the
Internet and the reflector, and data from the slave to the master are exchanged internally in the
PC. This approach models the delays of the forward and the feedback data paths on the
Internet,  (t) and  (t), as a single round-trip delay T(t)= (t)+ (t), arbitrarily located in the
forward path, as shown schematically in Figure 2. We represent the Internet segment using
two parameters representing the average packet delay and loss over a given time interval, by
measuring the round trip time T of the data sent to the remote reflector. The controller is a
standard position-based force feedback scheme in which the forces acting on the master are
proportional to the difference between the position of the master and that of the slave.
Host Distance Average Delay Std. Deviation Loss Rate
Local 00.05 Km 1.026 ms 0.189 0.00 %
Remote 10,000 Km 319.0 ms 16.70 51.3 %
Table 1 : Internet caracteristics of remote reflector.
The experiments carried out consist of using a single link of the master as input, commanding
the virtual manipulator to move against a hard surface. In the first test, the reflector is a local
computer, whereas in the second test the reflector is a remote computer. The characteristics of
the Internet connections are summarized in Table 1.
Delay T +T
x(t-T (t)-T (t))
Figure 2: A simplified model of the experimental set-up.
Figure 3-a shows the response of the experimental device when using the local reflector. The
slave is commanded to push harder against the surface, by increasing the position of the
master. The top part of Figure 3-a shows the positions of both master and slave. The slave is
blocked by the surface, and its position shows a small limit cycle due to the slow sampling
rate. The bottom part of Figure 3-a shows the force reflected at the master. The spikes are due
to packet losses. A different type of response is shown in the plots of Figure 3-b, representing
an experiment with the remote reflector. In this example, a slightly larger position error
generates a much lower force feedback. The longer time-delay results also in the slower slope
of the force profile, and in an apparent increase of the system compliance. The longer distance
affects also the packet losses which are more frequent in this case.
These experiments show that variable time delay in Internet-based force reflecting systems
can be compensated, at least in the simple system presented here, although with an alteration
of the perception of the remote environment. To experiment with more realistic systems, we
need to use a more complex master, as described next.
3. Integration of a Force-Reflecting Joystick and a PC
A series of tests were carried out to verify the performance of the standard PC environment
provided by Windows NT when used as a real-time
controller for a force-reflecting joystick [25]. The
commercial force-reflecting joystick Impulse
Engine 2000
, shown in Figure 4, is used in these
tests because of its good price performance ratio
[17]. This device can accurately track motions in
two degrees of freedom and exchange data with a
PC at rates up to 10 KHz. The control of the device
is made simple by the fact that each axis can be
accurately represented by the following dynamical
where m is the mass of the joystick, I the inertia, l
the distance of the center of mass from the
articulated base, and  is the joystick angle
measured from the vertical. The asymmetric design
of the handle adds a systematic error to the torque
in the y direction, which is compensated in
software. Three standard schemes are used to
control the joystick: a PID controller, a state space controller, and a PD controller with gravity

sin  mglbI
time [s]
force [N]
Master force
position [mm]
Master and Slave positions
position [mm]
Master and Slave positions
time [s]
force [N]
Master force
Figure 3: Experiments with a-Local reflector, b- Remote reflector.
Figure 4: The Impulse 2000.
compensation on the y-axis. The last controller shows the best performance and it is used in
the force-reflecting application.. The joystick comes with a software development kit that
allows reading positions and output forces, under DOS and Windows 95/NT. The toolkit was
used to develop the controller software as a Windows NT application organized in a client-
server scheme. JoysSrvr is the server implementing the actual controller of the force feedback
joystick, and is a multi-thread program, with the highest priority given to the controller. The
sampling frequency of the controller is 1 KHz, corresponding to the maximum frequency of
the internal timer used. The client is a portable virtual environment with different
configurations to test the performance of the server. The server communicates with the client
via shared memory or sockets. The virtual environment client adds severe timing constraints
to the controller, since it is computationally very intensive and has a lower update rate than
the controller, 40 Hz versus 500 HZ, respectively. The client implements the graphical
simulation of a four degree-of-freedom manipulator, as shown in Figure 5, is written using
WorldToolKit (WTK) to achieve a portable, high performance, real time, 3D simulation [24],
and is commanded by the two dof and the two buttons of the joystick.
This architecture supports tests with three different client types, as shown schematically in
Figure 6. A local client is represented by block 1. This client is on the same computer than the
server, implements the graphical environment, and exchanges data with the server using a
shared memory. To reduce the effects of the graphical simulation on the performance of the
joystick controller, we give the controller the highest priority in the Windows NT system and
we put the graphics at the idle priority. Block 2 represents a remote client connected to the
server through a segment of a computer network. It may implement the same functions than
the local client, but its communication with the server occurs through a socket connection.
Finally, blocks 3 and 4 implement a compensation of the network data rate fluctuation. This
compensation consists of a simplified model of the graphical environment in block 3, which
communicates at a fast rate with the joystick server, and ensure a steadier data flow to the
server by decoupling the controller from the network connection.
Several tests were carried out using the server implementing the PD with compensation
controller, and the virtual environment located on a remote computer on the same subnet. The
first tests aimed at measuring the maximum bandwidth of the communication channel. Data
were exchanged without visualization in the virtual environment at a maximum rate of 500 Hz
during low traffic periods. The addition of the virtual environment dropped the data exchange
Figure 5. Virtual manipulator
rate to 40 Hz when using a graphical computer. Finally the addition of collision computation
further reduces the transfer rate to the range 8--20 Hz, depending of the complexity of the
graphic model. During these experiments, the execution cycle of the controller on the force
server PC remained constant at 500 Hz, thus providing an acceptable force rendering.
5. Conclusions
The concept of distributed telerobotic workstations for the operation of space missions is
introduced in this paper. Although promising, this concept needs significant development to
overcome the current technological limitations, and extensive evaluation to verify its
applicability to real orbital experiments. Two experimental systems demonstrating specific
features of distributed telerobotic system are described in this paper. The first system consists
of a set-up to experiment with the control of variable time-delay systems, such as
teleoperators connected via Internet. The second system consists of a client-server application
for the control of a force-reflecting joystick in a Windows NT PC, such as those foreseen for
the distributed telerobotic stations. The combined results of the two experiments show the
feasibility of achieving high performance force-reflection using commercial hardware and
software, and of compensating the effects of the Internet variable time-delay with an
appropriate control system design. Naturally, more quantitative experiments need to be
performed to define the applicability of this approach. These experiments will consist of
telemanipulation tasks carried out by manipulators at the University of Verona and at the
ENEA laboratories in Rome, connected via Internet to force-reflecting joysticks located in the
two laboratories. Other experiments will increase our understanding of the effects of time-
delay compensation on the operator perception of a remote environment. We hope that this
area of research will also promote new ground-based applications such as much needed
support for elderly and disabled, remote training, tele and collaborative work on shared
environments, and new forms of interactive entertainment.
1. Bilateral control of teleoperator with time delay, Anderson, R.J. and Spong, M.W., IEEE
Transaction on Automatic Control, pp. 494-501, Vol. 34, N. 5, May 1989.
2. A Modular Control Architecture for Telerobotics, Anderson, R.J, IEEE Robotics and
Automation Magazine, pp. 10-18, Vol. 2, N. 3, September 1995.
Figure 6. Client - Server Architecture
3. WITS a Web-based Interface for Telescience, P. Backes, G. Tharp and K. Tso, IEEE
Conference of Robotics and Automation, Albuquerque, NM, April 1997.
4. Performance Analysis of Transport Protocols over Congestive Networks, Bolot, J.C. and
Shankar, A.U. and Plateau, B.D, Journal on Performance Evaluation, pp. 45-65, N. 11,
5. End-to-end Packet Delay and Loss Behavior in the Internet , Bolot, J.C., SIGCOMM '93,
pp. 289-298, September 1993.
6. Decentralized Robust Control Design for Uncertain Delay Systems, Chen, Y.H., Journal
of Optimization Theory and Applications, pp. 311-323, Vol. 89, N. 2, May 1996.
7. Memoryless Stabilization of Uncertain Dynamic Systems with Time-Varying Delayed
States and Controls, Choi, H.H. and Chung, M.J., Automatica, pp. 1349-1351, Vol. 31,
N. 9, 1995.
8. Interconnecting with TCP/IP , Comer, D.E, Prentice Hall, 1991
9. A Robotics Toolbox for MATLAB , Corke, P.I., IEEE Robotics and Automation
Magazine pp. 24-32, March 1996.
10. Stability Analysis of Bilateral Teleoperation Robotic Systems, Eusebi, A. and
Melchiorri, C., 3rd European Control Conference (ECC`95), Rome Italy 1995.
11. Internet-based Telerobotics: Problems and Approaches , Fiorini, P. and Oboe, R.,
International Conference of Advanced Robotics (Icar'97), July 7-9, 1997, Monterey, CA .
12. A PC-Based Configuration Controller for Dexterous Arms, Fiorini, P. and Seraji, H. and
Long, M., IEEE Robotics and Automation Magazine, September 1997.
13., URL of Haptic Technologies Inc. in Quebec (Canada).
14. Performance Analysis of Local Computer Networks, Hammond, J.L. and {O'Reilly},
P.J.P., Addison-Wesley, March 1988.
15. A Detailed Model of Bilateral Teleoperation, Hannaford, B. and Fiorini, P., IEEE
International Conference on Systems, Man and Cybernetics, Beijin, China, August,.
16. Stability of Systems with variable time delay, Hirai, K. and Satoh, Y., IEEE Transaction
on Automatic Control, pp. 552-554, Vol. Ac-25, N.3 1980.
17., URL of Immersion Corp. of San Jose (CA).
18. Force Reflection and Shared Compliant Control in Operating Telemanipulators with Time
Delay , Kim, W.S. and Hannaford, B. and Bejczy, A.K., IEEE Transaction on Robotics
and Automation, pp. 176-185, Vol. 8, N. 2, April 1992.
19. Experimental Verification of a Delay Compensation Algorithm for Integrated
Communication and Control Systems, Luck, R. and Ray, A, International Journal of
Control, pp. 1357-1372, Vol. 59, N. 6, 1994.
20. Stable Adaptive Teleoperation, Niemeyer, G. and Slotine, J.E, IEEE Journal of Oceanic
Engineering, pp. 152-162, Vol. 16, N. 1, January 1991.
21. A Design and Control Environment for Internet-Based Telerobotics. R. Oboe and P.
Fiorini, The International Journal of Robotics Research, May 1998.
22., URL of Sensable Technologies Inc. of Cambridge (MA).
23. Implementing real-time robotic systems using CHIMERA II. Steward, D. and Schmitz, D.
and Khosla, P., IEEE International Conference on Robotics and Automation, pp. 598-603,
March 1990, Sacramento, CA
24. Sense8 Corporation. WorldToolKit Reference Manual. Sense8 Corporation, July
25. Integration of a Force Feedback Joystick with a VR System. Alfredo Castro. Internal
report Enea, Divisione Robotica e Informatica Avanzata, June 1998.