Delayed Trilateral Teleoperation of a Mobile Robot

This paper analyzes the stability of a trilateral teleoperation system of amobile robot.This type of system is nonlinear, time-varying, and delayed and includes a master-slave kinematic dissimilarity. To close the control loop, three P + d controllers are used under a positionmaster/slave velocity strategy.The stability analysis is based on Lyapunov-Krasovskii theory where a functional is proposed and analyzed to get conditions for the control parameters that assure a stable behavior, keeping the synchronism errors bounded. Finally, the theoretical result is verified in practice by means of a simple test, where two human operators both collaboratively and simultaneously drive a 3D simulator of a mobile robot to achieve an established task on a remote shared environment.


Introduction
Robot teleoperation allows the execution of a task in a remote environment; generally dangerous and harmful jobs for the human operator are addressed necessarily by these systems to improve the safety level.They are formed by a human operator, a device called master, a robot called slave acting on a remote environment, and a communication channel that connect user and environment.Besides, the term bilateral indicates that there is a physical coupled via force feedback additional to the common multimedia information such as video and audio.That is, the slave follows the motions carried out by the master while simultaneously the human operator receives multimodal information [1].This feature about tact at distance increases the application field to varied areas such as telemedicine, health assistance, exploration, entertainment, teleservices, and telemanufacturing, among others.Besides, the use of the Internet as a low-cost communication channel allows easily connecting the master and slave devices.However, all Internet-like communication adds nonsymmetric time-varying delays, which if become larger could cause oscillations, low performance [2,3], and poor transparency [4,5].In general, the teleoperation systems are represented by delayed time-varying nonlinear models.So, one of the research topics in delayed teleoperation systems is related to the analysis of different mathematical tools to get delaydependent conditions for the controller parameters that allow assuring the stability of the delayed closed-loop system.
In the field about robot teleoperation, most control schemes are designed and analyzed considering two manipulators called master and slave [3][4][5][6] (and references therein).In general, energy is removed to assure stability or passivity [7].A classic strategy is the use of scattering signals or wave transformations [8,9], which adds an apparent damping to get passivity analyzing the system as a two-or fourchannel port.In the last years, many researches were focused on the use of  +  schemes to get a stable operation in the sense of Lyapunov by using a sufficiently large damping injected into master and slave [10][11][12].Besides, many strategies designed for teleoperation of manipulators were tested also into bilateral teleoperation of WMR (wheeled mobile robots), for example, control based on impedance [13], control based on -passivity [14,15], schemes including prediction and augmented reality [16], haptic teledriving of a WMR coupled with slippage, where an acceleration-level control law is proposed to assure the passivity of the bilateral system [17,18], and also  +  control schemes [19,20].It is important to remark that the stability analysis of a master position/slave velocity coupling is different mathematically to the common position/position used in a teleoperation

Preliminary
This paper will analyze teleoperation systems in which two human operators generally called trainer and trainee, use each one a master device (called master 1 and master 2, resp.) to drive a mobile robot slave in a trilateral way, as it is illustrated in Figure 1.
Master Model.Each human operator drives a manipulator robot to generate velocity commands and also receive force feedback.The standard nonlinear dynamic model of a manipulator robot to represent each master is used [10,11], that is, where index  indicates master 1 or master 2, q m () ∈  ×1 is the joint position of the master; q m () is the joint velocity; M m (q m ) ∈  × is the inertia matrix; C m (q m , q m ) is the matrix representing centripetal and Coriolis torques; g m (q m ) is the gravitational torque; f h is the torque caused by the human operator force; and  m is the torque applied to the master by the controller.Commonly, master 1 is associated with the trainer and master 2 with the trainee.
Mobile Robot Model.For the case of teleoperation of a mobile robot, the dynamic model of a mobile robot with differential traction is considered [14].It has two independently actuated rear wheels and is represented by where  = [V ]  is the robot velocity vector with V and  representing the linear and angular velocity of the mobile robot, f e is the force caused by the elements of the environment on the robot as well as other nonmodeled external forces such as static and dynamic frictions, D = [  0 0  ] is the inertia matrix and Q = [ 0 −  0 ] is the Coriolis matrix where  is the mass of the robot,  is the rotational inertia, and  is the distance between the mass center and the geometric center.In addition, the control action involves a force  1 and torque  2 is applied to the robot.
The devices are numbered as follows: index 1 represents master 1; index 2 indicates master 2; and index 3 represents the mobile robot.Besides, to represent the communication channel of a trilateral system, a forward time delay ℎ  (from device  to device ) and a backward time delay ℎ  (from device  to device ) for each pair master 1-master 2, master 1-slave, and master 2-slave are taken as time-varying and nonpredictable.

Trilateral Teleoperation of a Mobile Robot
The teleoperation trilateral system is used to control the velocity of a mobile robot, where the trainer and trainee (or in general two human operators) permanently send commands and perceive a force feedback.Based on the  +  control schemes applied to bilateral teleoperation of manipulator robots [10][11][12] and mobile robots [19,20], and its recent use in delayed trilateral teleoperation of manipulator robots [25], the stability of a dual-master single mobile robot system under a master position/slave velocity strategy will be analyzed in this work.The set of control actions is established as follows: where the parameters   3 and   3 are positive constant and they represent the proportional gain and accelerationdependent damping added by the velocity controller placed on the remote site,   1 and   2 are the damping injected in the masters, and   1 and   2 represent a relative spring depending on the mismatch between the master position and a reference feedback.Besides, the parameter   linearly maps the master position to a velocity reference, and z represents the available information about the mobile robot acceleration η but an infinitesimal before the current time instant.It is assumed that this signal can be represented by the real mobile robot acceleration η plus an infinitesimal error () ∈  ∞ [20,29]: Furthermore, two dominance factors are used.The  factor determines the authority of trainer over trainee and 1− implies the supremacy of trainee over trainer.Moreover, the dominance factors  and 1 −  indicate the supremacy of trainer and trainee over the mobile robot, respectively.The relation between them is defined in [25] as follows: with 0 <  < 1.As  is closer to 1, the following occurs.
(i) On master device 1, the force feedback from the mobile robot is bigger than the feedback of master device 2; then the trainer has greater perception about the state of the mobile robot than about the command provided by the trainee.
(ii) On master device 2, the force feedback from master device 1 is bigger than the force feedback from the mobile robot; thus the trainee has greater perception about the command generated by the trainer than about the mobile robot state.
(iii) On mobile robot,  increases with  (Figure 2); thus the command of the trainer has higher priority than the command of the trainee.
Relation between  and  can be seen on Figure 2. To the best of the authors' knowledge, there is not a conceptual justification about the above definition, but from it the stability analysis can be completed.The study about different definitions of  that allow keeping the system stability is an open subject.

Stability of the Delayed Closed-Loop System
The stability analysis of the control scheme is based on the Lyapunov-Krasovskii theory applied to trilateral teleoperation of a mobile robot in presence of time delays.Theorem 9. Consider a teleoperation system wherein two human operators (3) use two master devices (1) to drive a slave mobile robot (2) which interacts with an environment described by (4).If the control actions (7) are applied to such system, and if the controller gains hold with then the vector x fl [ q m 1 q m 2  η z ( −   q m 1 ) ( −   q m 2 ) (  q m1 −   q m 2 )]  is bounded, and the variables vectors q m 1 , q m 2 , η ∈  2 .
Proof.A Lyapunov-Krasovskii functional (x) =  1 +  2 +  3 +  4 +  5 +  6 +  7 +  8 > 0 is proposed, in order to analyze its evolution along the system trajectories and then infer the behavior of x fl [ q m 1 q m 2  η z ( −   q m 1 ) ( −   q m 2 ) (  q m1 −  q m 2 )]  which includes the kinetic energy of the two masters and mobile robot as well as the potential energy of the synchronism errors of the trilateral system.Next each   is represented mathematically as follows: where    are arbitrary positive constants which will be determined later.
To get V, the time derivative of ( 11)-( 18) will be computed along the system trajectories.First, V1 is obtained from (11) as follows: Next, the master dynamics (1) are inserted into (19) considering also Properties 1 and 2 of the master model: Now, the control action  m 1 from ( 7) is included in (20) and Relation 8 ( 6) is applied over the delayed terms: Following a similar procedure, the time derivative of ( 12) is given as follows: Next, the dynamics (1) of master 2 are introduced into (22) while Properties 1 and 2 of the master model are used, too: If the control action  m 2 from ( 7) is replaced into (23) and Relation 8 is applied again over delayed terms, the following expression is achieved: The functions V3 and V4 are easily reached from ( 13) and ( 14), respectively: Including the dynamics of the mobile robot ( 2) into ( 26) and then adding the control action  s from ( 7) and the environment force f e of (4) to the resulting equation, we get If relations ( 6) and ( 8) and  from (9) are considered, the terms of ( 27) can be reorganized as follows: Next, the derivative of the synchronism errors of the trilateral system are obtained from ( 15), (16), and ( 17): Finally, V8 is deduced from (18) (see [11,20] for further details) as follows: q m 2  () q m 2 () . (30) Now, observing the expressions obtained for V , there are terms with integrals that difficult the analysis.To solve this, the terms with integrals can be linked by means of Lemma 7 (5) to get quadratic expressions which are convenient to compare the contribution of each factor on the stability.
If each integral term of ( 21), (24), and ( 28) is associated with one integral term of (30) through the structure of ( 5), the following relations can be stated: Joining the functions   and applying relations (31)-( 37), the terms including integrals are replaced with expressions more adequate to comparison: where the constants   are given by As  2 + (1 − ) 2 ≤ 1, (32) can be rewritten as Since f a e () ∈  2 ,  ∞ (Assumption 6) and () ∈  ∞ , if the damping coefficients   1 ,   2 ,   3 are sufficiently high such that  1 ,  2 ,  3 > 0 in (34) then it is possible to state that Then, if (34) is integrated from 0 to , we have Furthermore, as () is considered infinitesimal, the terms containing () in ( 34 As (x) is radially unbounded and from (37) also it is bounded for all , then x ∈  ∞ .Thus, the synchronism errors are bounded.Even more, considering this result in (35), we get In this way, relation (38) allows asserting that the variables q m 1 , q m 2 , η ∈  2 .The proof is complete.
Remark 10.Taking the result achieved in (34), as the parameters   1 ,   2 ,   3 are larger, then q m 1 , q m 2 , η (both master velocities and slave acceleration) will tend to a smaller convergence ball.
Remark 11.From the x vector boundness and employing Properties 2 and 3, Proving that the control actions (7) are bounded is direct.
Remark 12.The following procedure to calibrate the control parameters of a delayed trilateral teleoperation system formed by a single mobile robot and dual-master is proposed: (1) Calibrate the gain parameters   1 ,   2 ,   3 and map   for the nondelayed trilateral teleoperation system considering  = 0 and  = 1.
(2) Set  depending on some training process.

Experimental Results
In this section, the proposed control scheme is tested.Two human operators teleoperate a mobile robot, receiving visual and force feedback.Both master devices employed are a lowcost manipulator, Novint Falcon model http://www.novint.com/which has 3DOF and force feedback.Two DOFs of each master device are used to generate velocity commands for the mobile robot: one for the linear velocity and the other to set the angular velocity reference.To implement the teleoperation system, the following tools are used: MATLAB/Simulink from https://www.mathworks.com/running with the real-time module and the SAS library from https://drive.google.com/drive/folders/0B2jklwyyOJqPNVA2SWFSaGFSNnc? usp=sharing compatible with V-REP environment (http:// www.coppeliarobotics.com/).The methodology used allows easily doing tests of a delayed trilateral teleoperation system using a simulated robot but considering the two human operators inside the system, which commonly is called humanin-the-loop simulations, as it is shown in Figure 3.The communication between the master devices, Simulink, and V-REP is made through shared memory.
For the test, all time delays ℎ 12 , ℎ 13 , ℎ 21 , ℎ 23 , ℎ 31 , ℎ 32 are configured randomly with values from 0.35 seconds to 0.65 seconds while the control parameters are calibrated following the procedure described in Remark 12, obtaining the following setting:   2 = 100; The objective of the experiment is to navigate avoiding the walls for reaching the yellow cube.Then the yellow cube must be pushed from its initial position to the green painted floor, as it is shown in Figure 4.Both operators feel not only the motion exerted by the other operator but also the dynamics of the mobile robot and the box weight.
Figure 5 shows the evolution on the remote site of each component (linear and angular velocity references) of the references   q m 1 ( − ℎ 13 ) and   q m 2 ( − ℎ 23 ) as well as the mobile robot velocity  (where V is the linear velocity and  is the angular velocity).It can be seen that the mobile robot velocity is closer to master 1 reference, due to  = 0.6 and therefore from (8) we get  ≈ 0.7.Last value establishes the priority level over the mobile robot.Figure 6 shows an image sequence captured from the test, in which it is possible to observe that the task is done adequately in spite of the delays added by the trilateral communication channel.
Finally, in Figures 7 and 8, the main signals of the system about its stability are shown, such that it was proven in the theoretical analysis (Theorem 9 and Remarks 10 and 11), the synchronism errors between: master 1 and master 2  e 3 =   q m 1 () −   q m 2 (), master 1 and mobile robot e 1 =   q m 1 ()−(), and master 2 and mobile robot e 2 =   q m 2 ()− (), as well as the mobile robot acceleration z, master 1 velocity q m 1 , and master 2 velocity q m 2 , remain bounded throughout the test.It is important to point out that about 53 seconds (subplot named T5 on Figure 5) the mobile robot, driven by the two human operators, starts to push the box.In the error signals e 1 and e 2 (Figure 7) as well as the mobile robot acceleration z (Figure 6), it is possible to appreciate such action.That is, two human operators both collaboratively and simultaneously drive a mobile robot, fulfilling the required task on the remote environment.
Discussion.The scheme control developed is addressed to keep the motion synchronism errors bounded, that is, e 1 , e 2 , and e 3 ∈  ∞ .On the other, the controller is not designed to achieve force tracking but the control actions are bounded k g q m 1 t − ℎ 13 [1] k g q m 2 t − ℎ 23 [1] k g q m 1 t − ℎ 13 [2] k g q m 2 t − ℎ 23 [2]  human operator performance generally is quantified with indirect measures such as time to complete the task and position errors of the mobile robot with respect to a goal path (navigation task).These indexes could be used to allocate a training process where the expert (trainer) gradually will have less control ( and  are decreased) as the performance of the trainee will be better.

Conclusions
In this paper, the stability of a delayed trilateral teleoperation system of a single mobile robot-dual manipulator-like master is carried out.The kinematics and dynamics mismatch between the mobile robot and the type-manipulator masters makes the analysis presented different with respect to the one applied for the case where all robots are manipulator robots.
To get a result about stability, a new Lyapunov-Krasovskii functional has been proposed, which is analyzed along the system trajectories.The result of the analysis exposed is formalized through Theorem 9, which establishes sufficient conditions to hold bounded the position and velocity of each master, velocity and acceleration of the mobile robot, motion synchronism errors, and all the control actions of the delayed trilateral system.Besides, a procedure suitable in practice for the parameters calibration is proposed considering the six values of time delays present on the trilateral system and some training process.A common training methodology requires that the trainer gradually sets the convenient level of  depending on the learning stage of the trainee, which could be evaluated from indexes such as time to complete the task.Furthermore, a simple test about trilateral teleoperation using a 3D simulator is shown, whose result agrees with the result achieved from the theoretical analysis proposed.Finally, it is important to remark that the main current application field is the remote training of operators to drive different types of mobile machines or robotic systems such as collaborative mobile manipulators.k g q m 1 (t) k g q m 2 (t) k g q m 2 (t)

Figure 1 :
Figure 1: Delayed trilateral teleoperation system of a mobile robot.

Figure 4 :
Figure 4: Setting of the test.

Figure 5 :Figure 6 :
Figure 5: Linear and angular velocities of the mobile robot and the dual-references from masters 1 and 2.

Figure 7 :
Figure 7: Velocities of both masters and acceleration of the mobile robot.
) and (35) can be neglected.Besides, the integral term ∫