The Development of a Prédictive Display for Space Manipulator
Positioning Tasks
Paul Breedveld
Delft Univeraity of Technology, Faculty of Mechanical Engineering & Marine Technology, Laboratory for Measurement & Control, Man-Machine System) Group, Mekelweg 2, 2628 CD Delft, the Netherlands.
A b s t r a c t . Currently, the space manipulator E R A (European Robot Arm) is being devel-oped in Europe. It will be used on the International Space Station Alpha. When the E R A is manually controlled, its movements are monitored with camera's, like the end-eftector camera, which is used for positioning the 'hand' of the robot. When the operator controls the ERA, he/she has to cope with slow and flexible dynamics. At the Delft University of Technology, research is carried out on the development of a man-machine interface for the ERA. The movements of the E R A are animated on a graphical Workstation, and the in-put device is a Spaceball controller. This paper briefly discusses a novel graphical display that uses intersections between 3-dimensional objects to improve the spatial observability of the end-effector camera picture, and a real-time simulation model, that accurately simu-lâtes ERA's movements. Next, the paper describes the extension of the display to a novel
prédictive display, that compensâtes for the effects of the slow dynamics and the flexibility.
Finally, the paper discusses the suitability of the prédictive display to compensate the time delays when a space manipulator like the E R A is controlled from Earth. Some try-outs with the prédictive display have shown, that it is very suitable for manually positioning the end-efFector.
Keywords. Man/machine Systems, Man/machine interfaces, Manual control, Telemanip-ulation, Telerobotics, Graphic displays, Prédictive displays
1. I N T R O D U C T I O N
Around the end of this Century, the first segments of the new International Space Station Alpha will be put into orbit around Earth. On the Russian part MIR-2 of the Alpha, a European space manipulator will be
used. The manipulator is called ERA, which is short for 'European Robot Arm', and shown in Fig. 1. It is a 10 m long anthropomorphic manipulator with 6 de-grees of freedom (DOF's), that is being developed and assembled at Fokker Space and Systems B.V. in Leiden, the Netherlands. The joints of the robot are driven by electric brushless D C motors, and its links are made of lightweight carbon fibre. Even though this is a very stiff material, the long and slender upper- and forearm of the E R A are flexible. Since the power of the elec-tric motors is limited, however, the arm can only move slowly, and the vibrations will be small. The 'hand' of the E R A is called end-effector. On the end-effector
front, a gripper is placed, which is used for grasping an object.
On the MIR-2, the E R A will be used for servicing and inspection tasks and for the displacement of Orbital Re-placeable Units or ORU's. These are containers with objects that must be moved from one part of the space station to another. During these activities, the robot will be controlled from inside the MIR-2 by an astro-naut. Due to a lack of direct vision on the MIR-2,
camera pictures will be used to monitor the movements
of the ERA. The camera's are located on the manipu-lator itself and on the space station. A n example of a camera, located on the manipulator itself, is the end-effector camera, that will be used for fine positioning
I the end-effector, e.g. for grasping an ORU. Whenever possible, the E R A will be controlled in supervisory con-trol. In that case, the operator gives commands like
'put that there' to the manipulator, and the robot per-forais these tasks automatically. The operator uses the camera pictures to monitor the movements of the
ma-nipulator. If anything goes wrong, the operator inter-venes, and switches to manual control. In that case,
he/she uses a control device to move the manipulator
into the desired direction.
Pig. 2. The Spaceball controller
faced with three problems, that make controlling the E R A difîicult:
1. The E R A is a multivariable System with a large freedom. o£ movement. It is difîicult for a human to control such a System manually;
2. It is difficult for a human to estimate the 3-dimensional movements of the manipulator from flat, 2-dimensional caméra pictures;
3. The slow dynamics and the flexibility make con-trolling even more difficult.
At the Delft University of Technology, research is car-ried out on the development of a man-machine inter-face for manual control of a space manipulator like the ERA, and to find solutions for thèse problems. Besides space manipulators, the results of the project are also suitable for other applications, like endoscopie surgery, undersea robotics, and robotics in the nuclear industry [Sheridan, 1992].
The hardware facility consists of a fast Silicon Graphics Indigo II Extrême graphical workstation and a Spaceball controller. The graphical workstation is used to
simu-late the movements of the manipulator, and to animate the caméra pictures; both in real-time. The Spaceball controller, Pig 2, is a 6 D O F force operated control de-vice that consists of a sphère on a base. The sphère, or Spaceball, can be slightly translated and rotated in 3 perpendicular directions. The forces and torques the operator imposes on the Spaceball are transformed into translational and rotational velocities of the end-effector. The choice for end-effector velocity control and the sélection of the Spaceball controller as control device for this project is motivâted in [Breedveld, 1994]. Instead of velocity control, also end-effector position control could have been used. In [Kim, 1987], however,
it is shown that position control is more suitable for
fast manipulators with a small workspace, while
veloc-ity control is more suitable for slow space
manipula-tors with a large workspace like the ERA. Joint control
didn't seem to be interesting to investigate, since, for a human operator, this way of controlling is much more difficult to do.
The research of the project is phased as follows: A. Activities in an operating point:
1. Manual control of an idéal manipulator; 2. Manual control of a flexible manipulator. B. Activities along a track
In stage A, which is focused on positioning the end-effector, e.g. for grasping an ORU, the demanded ac-curacy is high, but the risk of an unexpected collision between the manipulator and the environment is small, since the movements of the manipulator are small. In stage B, which is focused on moving the end-effector along a track, e.g. for displacing an ORU, the de-manded accuracy is small, but the risk of a collision
Fig. 3. The end-effector c a m é r a picture. (a) End-effector
(h) baclcpJane (c) sfdeplanes (d) target (e) vision target.
is large. In phase A l , the first and second problem mentioned above are investigated, and the manipula-tor dynamics are not implemented yet. In phase A2, the third problem is investigated, and the dynamics of the manipulator are modelled in an accurate real-time simulation model.
The developments in phase A l of the research are de-scribed in [Breedveld, 1995 & Buiël, 1995a,b]. The so-lutions for the second problem mentioned above in this phase of the research are briefly discussed in section 2. The solutions for the third problem in phase A2 of the research, that have resulted in a prédictive display, are
described in sections 3 and 4. In section 5, the suit-ability of the prédictive display to compensate the time delays when a space manipulator like the E R A is con-trolled from Earth is discussed. The paper ends with some concluding remarks in section 6.
2. DISPLAYS F O R POSITIONING T H E E N D - E F F E C T O R
The end-effector caméra picture
When an object must be grasped, the end-effector cam-éra will be used for fine positioning the end-effector. Fig. 3 shows a stylized impression of the end-effector caméra picture, when the distance between the end-effector front and the object to be grasped is decreased to 1 m. The end-effector caméra is placed on top of
the end-effector and shows a view of the environment. The end-effector front is visible at the bottom of the picture. The environment is simplified to five planes. The oacfcptaraerepresents the object to be grasped. The four sideplanes represent the environment around the
object. O n the backplane, a target is présent. This is a
connection point on which the end-effector front must be accurately positioned with a desired accuracy of 3 mm and 1.5 °, to grasp the object. Above each con-nection point on the MIR-2, a vision target is présent.
This object consists of a black base plate with three white dises, from which the middle one is placed on an élévation. It is used by a proximity sensor. a
com-puter progiam that uses image processing techniques to calculate the distance and orientation of the
end-Fig. 4. Pyramid Display, at a distance of 50 cm from the target.
effector with respect to the target from the camera pic-ture. This distance and orientation is described with six numbers: 3 translations and 3 rotations along the principal axes. These numbers can be used to posi-tion the end-efTector on the target automatically. The slow dynamics and the flexibility of the manipulator, however, make automatic positioning difficuit. In some cases, e.g. if an object is present between the end-effector and the target, or if the automatic controller is damaged, automatic positioning might even be im-possible. Therefore, it is also possible to position the end-effector by hand.
Graphical overlays on the end-efTector camera picture In the last stage of the positioning task, the distance between the end-effector and the backplane becomes so small, that the four sideplanes get out of sight. Then, only the target and the vision target remain to give spatial information to the operator. Unfortunately, for a human, it is very difficuit to estimate the position and orientation of the end-effector from the size and the lo-cations of the three white dises. To assist the operator, the proximity information can be used to superimpose a graphical overlay on the picture that makes it more
easy to position the end-effector.
In [Bos, 1995], a graphical overlay on the end-effector camera picture is described, in which the proximity in-formation is used to animate six analogue dial indica-tors, that display the six translational and rotational misfits individually. In this project, the misfits are not
displayed individually, but integratedin a novel overlay
that consists of 3-dimensional graphical objects. Due to the intégration, the operator can estimate the trans-lational and rotational misfits at a single glance. The
overlay is constructed in such a way, that the animated objects seem to be part of the environment. Due to
this, it increases the operators involvement with the
real situation. The working principle of the overlay, called the Pyramid Display, is briefly described below.
In [Breedveld, 1995], the development of this display is discussed in detail, and some further improvements of the display are described.
The Pyiamid Display
Figs. 4, 5 and 7 show the Pyramid Display, at a distance
F i g . 5. Pyramid Display, at a distance of 5 cm from the target.
Fig. 6. Side view of the insert-box, the-frosted-glass and the pyramid. (a) Backplane, (b) insert-box, partially
eut away, (c) frosted-glass, and (d) pyramid.
of 50 cm, 5 cm and 0.5 cm from the target respectively. To improve the sensation of depth, the proximity sen-sor information is used to project rectangular grids on large unicoloured planes in the environment. Besides the grids, the proximity sensor information is also used to animate three 3-dimensional graphical objects: an
box, a frosted-glass and a pyramid. The
insert-box and the pyramid are fixed to the backplane, at the location of the vision target, which is therefore not vis-ible anymore. The frosted-glass is a transparent square plane, fixed to the end-effector front, like the sight of a gun. In the first phase of the task, Fig. 4, the opera-tor uses the frosted-glass as a viewfinder and moves it towards the pyramid. The rotational misfit of the end-effector can then easily be detected from the orientation of the box. In the second phase of the task, Fig. 5 and Fig. 6 from aside, the frosted-glass intersects the
insert-box. Then, the rectangular grid in the box is used as a distance indicator, perpendicular to the backplane, and the remaining distance can easily be estimated. In the third phase of the task, Fig. 7, when the distance left to the backplane is decreased to only a few millimeters, the frosted-glass intersects the pyramid. The size of
this intersection then visualizes the distance left to the backplane. The larger the intersection, the smaller this distance. The shape of the intersection visualizes the
Fig. 7. Pyramid Display, at a distance of 0.5 cm from the target.
rotational misfit is present, the intersection is a trapez-ium or a kite, l u e in Fig. 7. If all rotational misfits are zero, the intersection is a square. When the intersection finally coincides with the square on the frosted-glass, the front of the end-effector is positioned against the backplane and the positioning task is completed. The intersection with the pyramid is also used to avoid col-lisions against the backplane: a collision occurs, when one of the four vertices of the intersection touches the base of the pyramid, like in Fig. 7.
The graphical objects of the Pyramid Display have con-trasting colours that make them clearly distinguishable. The grids are coloured black, the insert-box blue, the frosted-glass transparent black, and the square on the frosted-glass bright yellow. The colour of the pyramid is given the fonction of accuracy indicator. This was done
like a traffic light: In général, the pyramid is drawn brignt red; an alarming colour, that enforces the op-erator to decrease speed. When all translational and rotational misfits, except for the distance left to the backplane, have reached their desired accuracies of 3 mm and 1.5 °, the colour of the pyramid changes to bright yellow. When the misfits are made even smaller, its colour changes to bright green, and the end-effector can be positioned safely. The yellow colour functions as a kind of security margin.
In phase A l of the project, a number of man-machine experiments have been performed with the Pyramid Display. The experiments proved the display to be very handy for positioning the end-effector. After a relatively short learning phase of about 3.5 hours, al-most all subjects turn out to be able to position the end-effector with surprisingly fast completion times be-tween 17 and 34 s [Buiël, 1995b].
3. T H E E F F E C T S O F T H E D Y N A M I C S The real-time simulation model of the E R A
If the E R A is manually controlled, the operator uses the Spaceball controller to generate the desired veloc-ity of the end-effector. ERA's automatic velocity troller transforms his/her control command into
con-trol signais for the six electric brushless D C motors,
Fig. 8. The principle of a p r é d i c t i v e display.
that make the end-effector move into the desired di-rection. In phase A l of the research, the manipulator dynamics were not implemented yet. It was assumed that the E R A behaves perfectly, and that it responds
immediately and exactly to a given control command. In reality, however, even with an optimized automatic controller, the behaviour of the manipulator is far from perfect.
In order to investigate the effects of the slow dynamics, the flexibility and the time delays in phase A2 of the research, the dynamics of the E R A were modelled in a real-time simulation model. The model is described in detail in [Diepenbroek, 1994]. The flexible dynamics of the E R A were modelled by using software of Fokker Space and Systems B.V. [Woerkom, 1994]. The elec-tric motors and the automatic velocity controller were modelled in this project. The simulation model also includes a novel friction model, that simulâtes the ef-fects of the friction in ERA's joints very accurately, but takes less Computing time.
Some try-outs with the simulation model showed, that ERA's slow, flexible dynamics make manually Control-ling the manipulator difficult. This is mainly caused by the following two effects:
• Since it takes some time for the robot to accel-erate, it is not immediately clear in which direc-tion the end-effector will move when the operator
moves the Spaceball;
• Since it takes some time for the robot to slow down, it is not immediately clear at what position
the end-effector will corne to a standstill when the
Spaceball is released.
In order to reduce the risk of instability, the operator is forced to use a move and wait strategy; a cautious
way of Controlling, in which he/she générâtes a small incrémental control command, then stops and waits for a few seconds to see what happens, then gives another small incrémental control command, etc.
The principle of a prédictive display
The two annoying effects above can be reduced by using a prédictive display. The principle of such a display is shown in Fig. 8. The control commands are sent to the manipulator, and the movements of the manipulator are recorded with a camera and shown on a télévision screen in front of the operator. In order to compensate the two effects mentioned above, the control commands
tuning
self-tuning predictor
Fig. 9. A predictor with tuning. Fig. 10. A predictor that generates a position setpoint
are also used in a computer program, or predictor, that
serves two purposes:
(a) it calculates in which direction the end-effector will move when the operator moves the Spaceball, and
(b) it calculates at what position the end-effector will come to a standstill when the Spaceball is released (this position will further be referred to as the
stopping position).
The output of the predictor is used to overlay an image of a ghost manipulator on the camera picture, that
ini-tially coincides with the real one. When the operator moves the Spaceball, the ghost manipulator will detach from the real manipulator, which, in turn, will accel-erate and follow the prediction. When the Spaceball is released again, the ghost manipulator stops moving, and the real manipulator will slow down, till finaUy, both pictures coincide again. If the prediction is ac-curate, and if the image is spatially easy to interpret, a predictive display can be very helpful for controlling the manipulator.
In phase A2 of the research, the principle above has been used to extend the Pyramid Display to a Pyra-mid Predictive Display. The development of this dis-play will be described in the next section.
4. D E V E L O P M E N T O F T H E P Y R A M I D P R E D I C T I V E D I S P L A Y
The structure of the predictor
If the operator controls the manipulator with a predic-tive display, he uses the stopping position to avoid col-lisions between the manipulator and the environment. In the literature, different ways have been used to cal-culate the stopping position in an accurate way.
In the predictor described in [Bos, 1991, pp. 88-121], the stopping position is calculated by using a simula-tion model with the dynamics of the manipulator. A simulation model, however, is always a simplification of reality. If such a predictor is used in practice, the pre-dicted stopping position will always differ from the real one, and a statical deviation will be present between
the final position of the real- and the ghost manipula-tor on the screen. This deviation can be reduced if the simulation model is made more accurately. However, a complex model takes much computing time. This is
not desired, since the prediction must be calculated in real-time.
In the predictor described in [Breedveld, 1992], mea-sured information from the camera picture is used to
tune the predictor, to eliminate the statical deviation
automatically (Fig. 9). If such a self-tuning predictor is
used, however, the ghost manipulator will not immedi-ately come to a standstill, when the operator releases the Spaceball, but it will slowly drift towards the real stopping position. In order to compensate this drift, the operator will have to generate more control com-mands to make the manipulator do what he/she wants. This is quite irritating, and a drawback of a self-tuning predictor.
Due to the problems with the two predictors above, in this project, it was not decided to use a predictor that describes the behaviour of the robot as accurately as possible, but to use the predictor as setpoint generator
for the robot (Fig. 10). For the predictor, a simple inte-grator is taken, that integrates the velocity commands
of the operator, and calculates a position setpoint. This setpoint is used to animate the ghost manipulator on the screen, and used in the automatic controller of the manipulator, that has been extended with an automatic position controller. This position controller eliminates
the deviation between the setpoint and the real posi-tion of the end-effector automatically and moves the real manipulator towards the ghost.
Since the ghost manipulator now responds immediately and exactly to a given control command, the predic-tor serves both purposes (a) and (b) mentioned above. Since the predictor is a very simple integrator, that doesn't use a simulation model of the robot, no prob-lems with the computing time are present anymore. Furthermore, a statical deviation in the final position of the real manipulator and the ghost is compensated by drifting the real manipulator towards the ghost, instead of the opposite, and the drawback of the self-tuning pre-dictor is not present anymore.
A disadvantage of the strategy is, that the automatic controller of the manipulator must be extended with a position controller, which makes the system more com-plicated. However, the requirements on the position controller are not very large: if it eliminates the stat-ical deviation within a reasonably short time, and if the overshoot is not too large, the operator can rely
Fig. 11. The pyramid (left) and the star (right).
almost completely on the position setpoint, and a sim-ple automatic controller will meet the demands. Only when the manipulator moves very close to an object, the overshoot may cause a collision. In such a case, the operator cannot rely completely on the position set-point, and he/she must also observe the movements of the real manipulator, and smoothen his/her control commands if it vibrâtes too much.
So, instead of using a very complicated simulation model, or a clumsy self-tuning predictor, a simple ex-tension was made to the automatic controller of the robot, and the predictor is used as setpoint generator. With this strategy, the total System, which consists of
both the manipulator and the predictor, has a simple structure and is easy to control. In spite of the extra automation, the movements of the manipulator are still completely manually controlled by the operator.
The visualization of the prédiction
The prédictive display in Fig. 10 shows a 2-dimensional side view of a manipulator. The end-effector camera
picture, however, shows a 3-dimensional front view of
the end-effector and the environment. In this case, the position setpoint must be visualized spatially, which is
much more difficult.
In the prédictive display of Fig. 10, the predictor in-tégrâtes the velocity commands of the operator to a position setpoint for the end-effector relative to the en-vironment. Since the camera picture in Fig. 10 only
shows a manipulator, an image of a ghost manipula-tor was used to visualize the position setpoint. The end-effector camera picture, however, both shows the end-effector front ánd the environment. In this case, either an image of a ghost end-effector front, or an
im-age of a ghost environment, can be superimposed on the
picture to visualize the position setpoint. When the op-erator controls the end-effector camera picture, he/she focuses on the environment, and not on the end-effector front. Therefore, the ghost end-effector front seems to be less suitable to visualize the position setpoint, and the ghost environment is selected.
Note, that, instead of an image of a really existing ob-ject in the end-effector camera picture, also another, more abstract présentation could have been used to vi-sualize the position setpoint. In the end-effector camera overlay described in [Ferro, 1994], for example, three analogue distance scales and a figure that looks like the artificial horizon of a plane, are used to visualize the real and the predicted misfits. In this project, how-ever, it was decided to visualize the position setpoint in such a way, that the operator stays involved with the real situation as much as possible. Therefore, in the end-effector camera picture, a ghost environment is used to visualize the position setpoint, instead of a more abstract présentation, that would divert the at-tention of the operator from the real situation.
Fig. 12. Pyramid P r é d i c t i v e Display, at a distance of 50 ' cm from the target.
So, the prédictive display shows both a ghost- and a real environment. In the prédictive display, initially, the ghost environment coincides with the real one. When the operator moves the Spaceball, the ghost environ-ment will detach from the real environenviron-ment, which, in turn, will accelerate and follow the position set-point. When the Spaceball is released again, the ghost environment stops moving and the real environment will slow down, tili, finally, both environments coincide again. From the display, it must be easy to estimate the translational and rotational misfits of both envi-ronments. Due to the good results with the Pyramid Display in phase A l of the research, the principle of this display is also used in the prédictive display. In the Pyramid Display, an insert-box and a pyramid are projected in the environment, to visualize its position, and orientation. If these two graphical objects are pro-jected on both the ghost- and the real environment, however, the prédictive display would contain 2 insert-boxes and 2 pyramids. This mixture of objects would look very crowded and confusing.
With the predictor of the preceding paragraph, the op-erator can rely almost completely on the position set-point. The movements of the real environment are only important at the end of the positioning task, when the
end-effector is very close to the backplane and the over-shoot may cause a collision. In the Pyramid Display, the operator uses the intersection of the frosted-glass with the pyramid to avoid collisions against the back-plane. The insert-box is not used for this purpose. This means, that, in the real environment, the insert-box can be omitted. This simplifies the prédictive display to a ghost environment with an insert-box and a pyramid, and a real environment with only a pyramid. At the end of the positioning task, when both pyramids in-tersect the frosted-glass, the ghost pyramid is used to fine position the end-effector, and the real pyramid is used to avoid collisions. For the operator, however, the mixture of both intersections would still be difficult to interpret. Therefore, the display is simplified further. In the Pyramid Display, a collision occurs, when one of the four vertices of the intersection on the frosted-glass touches the base of the pyramid (Fig. 7). So, the
Fig. 13. Pyramid P r é d i c t i v e Display, at a distance of 0.5 cm from the target.
operator uses the vértices of the intersection to avoid
collisions. This means, that, in the real environment, it is sufficiënt to draw only the edges of the pyramid.
The object then simplifies to a star with four tips, as
shown in Fig. 11.
The Pyramid Prédictive Display
Figs. 12 and 13 show the finally resulting Pyramid Pré-dictive Display, at a distance of 50 cm and 0.5 cm from
the target respectively. The ghost environment is equal to the environment in the Pyramid Display and consists of the grids, a ghost target, the insert-box and the pyra-mid. The real environment consists of the back- and sideplanes, the real target and the star. The ghost tar-get is drawn transparent, and the star is drawn with a
préférence: it is always visible, whether or ivot it lies
be-hind the pyramid. The colours of the graphical objects are equal to the colours of the Pyramid Display. The accuracy indicator has been extended: the colour of the pyramid now changes from red via yellow to bright green, if the position setpoint reaches its desired
accu-racy, and the colour of the star changes from red via yellow to bright green, if the real position reaches its
desired accuracy.
In the begiiuiing of the positioning task, Fig 12, the op-erator moves the frosted-glass towards the insert-box of the ghost environment and uses the orientation of the box and its intersection with the frosted-glass to po-sition the end-effector in front of the target. At the end of the task, Fig 13, both the pyramid and the star intersect the frosted-glass. Then, the operator uses the pyramid for fine positioning the end-effector on the ghost environment, and the star, to get an im-pression of the overshoot. If the intersection with the star vibrâtes a lot, the operator must be careful and slow down, tül the vibrations of the manipulator are damped sufficiently. When both the pyramid and the star are coloured green, the end-effector can be posi-tioned on the target safely.
Note, that in the last stage of the positioning task, the movements of the real environment, the predicted envi-ronment, and the accuracy indicator, are all visualized
in the middle of the display, which is the location where
the operator focuses on. Due to this, in the last stage of
Fig. 14. The stretched pyramid (left) and the stretched star (right).
the task, when the end-effector must be fine positioned, he/she doesn't have to spread the attention to different parts of the display, which would be quite irritating.
Improved shape of the pyramid and the star
In the display, the pyramid and the star have two pur-poses: they are used to visualize the resulting trans-lational and rotational misfits, and they are used to avoid colüsions. The first purpose requires, that the ob-jects are low: the lower they are, the stronger they
am-plify the misfits when they intersect the frosted-glass. The second purpose, however, requires that the objects are high: the higher they are, the earlier they intersect
the frosted-glass, and the earlier the operator can slow down the end-effector to avoid a collision.
In order to meet both conflicting requirements above better, in the Pyramid Predictive Display, the pyra-mid and the cross are stretched a little bit, as shown
in Fig. 14. The low slope at the base of the stretched objects makes them very suitable to visualize the mis-fits, but their height still makes them very suitable to avoid colüsions. However, the objects should not be stretched too much, since then, the size of the intersec-tion changes strongly non-linear with the distance to the backplane, which is very confusing.
5. DISCUSSION Prediction in the case of time delavs
In future applications, space manipulators like the E R A will also be used on unmanned space vehicles, and man-ually controlled from Earth. Then, the control com-mands of the operator are transmitted into space and arrivé at the manipulator with a time delay A í up to about 3 s [Sheridan, 1992, p. 212]. The delay is
variable, due to changes in the distance and in the
numbei of satellites (1 or 2) between the manipulator and Earth, and due to the processing time within the satellites. The automatic controller of the manipula-tor tiansforms the delayed control signáis into move-ments of the arm that are monitored with a camera. The video signal is transmitted back to Earth, and the camera picture appears on the televisión screen, again with a time delay A i . In the control loop, the overall time delay of 2At up to about 6 s, makes controlling
the robot very difficult.
In the case of the positioning task, the Pyramid Predic-tive Display can also be used to compénsate the time delays. Since, in that case, the computer with the pre-dictor is located on Earth, no delays will be present when the operator controls the predictor. The control commands are transmitted into space, and the real ma-nipulator on the screen wül follow the position setpoint after 2A¿ s.
Compensation of the variations in the time delays If a manipulator like the E R A is controlled from Earth, the control- and video signais will be transmitted dig-itally, as a discrète séries of samples with a constant time interval, for example of 0.05 s. Due to the vari-able time delay A i , the time interval at the arrivai will not be constant anymore, and the signais will be de-formed. In [Breedveld, 1992], a method is described by which the déformation in. the signais can be compen-sated. This method uses two buffera; the first one near
the manipulator and the second one near the télévision screen. At the arrivai, the samples are put into the buffer, which is emptied on-line, again with the con-stant time interval of 0.05 s. Since the time interval between the mimbers is constant again, the signais are not deformed anymore. The time interval at the input of the buffer varies, while the interval at the output is constant. Therefore, the size of the buffet is vari-able. Since the buffer should never become completely empty, the constant time delay between the moment of transmitting and the moment of emptyirtg the buffer, must be somewhat larger than the maximum value of A t . Due to this, the time delay increases a little bit. If a well working prédictive display is used, however, it is not expected that a small increase in the time delay will make Controlling the manipulator more difficult.
6. C O N C L U D I N G R E M A R K S
At the moment of writing this paper, no man-machine experiments have been carried out with the Pyramid Prédictive Display yet. Some try-outs, however, have shown that the display simplifies the positioning task strongly. In the near future of phase A2 of the research, a number of man-machine experiments will be carried out with the display, to investigate the effects of a pré-dictive display on fine positioning the end-effector in the présence of slow, flexible dynamics and time delays. The Pyramid Prédictive Display will also be compared with other kinds of prédictive displays for end-effector positioning tasks, like the display described i n [Ferro,
1992].
The •Pyramid Display and the Pyramid Prédictive Dis-play, both developed in phase A of the research, are developed for fine positioning tasks, and help the Op-erator in positioning the end-effector with a high accu-racy. In phase B of the research, other kinds of displays are being developed for track following tasks, that help the Operator in avoiding collisions between the manip-ulator and parts of space station. In the future of this phase of the research, a number of man-machine exper-iments will be carried out with différent types of such displays, to compare them with each other, and to in-vestigate their suitability for track following tasks.
7. A C K N O W L E D G E M E N T S
This research is supported by the Dutch Technology Foundation S T W and takes place in coopération with Fokker Space fe Systems B.V., the European Space Agency ESA, and the National Aerospace Laboratory NLR.
ment and Control, Delft, the Netherlands, ISBN 90-370-0056-8, 184 p.
Bos J F T (1995), Aiding the Operator in the Manual Control of a Space Manipulator. Contiol Eng. Prac-tica Vol 3, No 2, pp. 223-230; and also in Proc. 5th IFAC Syxnp. on Analysis, Design and Evaluation of Man-Machine Systems, June 9-11, 1992, the Hague,
the Netherlands, paper 1.3.3.
Breedveld P (1992), Controlling a Space Manipula-tor from Earth with Predictive Display Techniques. Proc. llth European Annual Conf. on Human De-cision Making and Manual Control, Nov. 17-19,
Va-Jenciennes, France, paper 27.
B r e e d v e l d P (1994), The Design of an Optimal Man-Machine Interface for a Manually Controlled Space Manipulator. Report N-469, Delft Unirersity of Technology, Fac. of Mechanical Engineering & Marine Technology, Lab. for Measurement & Con-trol, 75 p.
Breedveld P (1995), The Development of a Man-Machine Interface fol Telemanipulator Positioning Tasks. To be published in Prepr. 6th IFAC Symp. on Analysis, Design and Evaluation of Man-Machine Systems, June 27-29, MIT, Cambridge, USA.
Buiel E F T , Breedveld P (1995a), A Labora-tory Evaluation of four Control Methods for Space Manipulator Positioning Tasks. To be published in Prepr. 6th IFAC Symp. on Analysis, Design and Evaluation of Man-Machine Systems, June 27-29, MIT, Cambridge, USA.
Buiel E F T , Breedveld P (1995b), A Labora-tory Evaluation of two Graphical Displays for Space Manipulator Positioning Tasks. To be published in
Proc. 14th European Annual Conf. on Human Deci-sion Making and Manual Control, June 14-16, Delft University of Technology, Delft, the Netherlands.
Diepenbroek A Y (1994) (in Dutch), Ontwikkel-ing van een real-time simulatiemodel van de E R A ruimtemanipulator met daarin wrijving. Report
A-704, Delft University of Technology, Fac. of Mechan-ical Engineering and Marine Technology, Lab. for Measurement and Control, Delft, the Netherlands, 98 p.
Ferro D V (1992), Process and System for Re-motely Controlling an Assembly of a First and a Second Object, US Patent no. 5,119,305, 1992, Assignee: Aerospatiale Societe Nationale Indust., Paris, France.
K i m W S, Tendick F , Ellis S R, Stark L W (1987), A Comparison of Position and Rate Control for Telemanipulations with Consideration of Manip-ulator System Dynamics, IEEE Journal of Robotics
and Automation, vol. RA-3, no. 5, pp. 426-436.
Sheridan T B (1992), Telerobotics, Automation and Human Supervisory Control, MIT Press, USA, ISBN 0-262-19316-7, 413 p.
Woerkom P T L M van, et al. (1994), Develop-ing Algorithms for Efficient Simulation of Flexible Space Manipulator Operations, Proc. 45th Congress of the International Astronautical Federation, Oct. 9-14, Jerusalem, Israel, pp. 1-13.
8. R E F E R E N C E S
Bos J F T (1991), Man-Machine Aspects of Remotely Controlled Space Manipulators, P h D Thesis, Delft University of Technology, Fac. of Mechanical Engi-neering and Marine Technology, Lab. for
