Delayed Bilateral Teleoperation of Wheeled Robots including a Command Metric

This paper proposes a control scheme applied to the delayed bilateral teleoperation of wheeled robots with force feedback, considering the performance of the operator’s command execution. In addition, the stability of the system is analyzed taking into account the dynamic model of the master as well as the remote mobile robot under asymmetric and time-varying delays of the communication channel. Besides, the performance of the teleoperation system, where a human operator drives a 3D simulator of a wheeled dynamic robot, is evaluated. In addition, we present an experiment where a robot Pioneer is teleoperated, based on the system architecture proposed.


Introduction
Robot teleoperation allows the execution of different tasks in remote environments including possibly dangerous and harmful jobs for the human operator [1].In the teleoperation systems of robots with force feedback, a user completes some task and physically interacts with the environment through a master-slave system.Thus, several control schemes and strategies for the teleoperation of mobile robots have been developed in order to solve tasks such as land surveying in inaccessible or remote sites, transportation and storage of hazardous material, inspection of high-voltage power lines, deactivation of explosive devices, high-risk fire control, pesticide and fertilizer crop spraying and dusting, mining exploration, and various other tasks [2][3][4][5][6][7][8].Additionally, the use of Internet as a communication channel increases the application of the teleoperation systems.However, it is known that the presence of time delay(s) can induce instability or poor performance in a delayed teleoperation system [9][10][11] as well as a poor transparency [12].Some strategies involve compensation based on a human operator model [13] in which only visual feedback is considered, ordinary structures such as control based on impedance [14], event-based control [15], signals fusion [16], and others based on kinematic models like [17][18][19][20].Finally, [21,22] consider a dynamic model and analyze the r-passivity of the system, while [23] presents a stability analysis of teleoperation systems using Linear Matrix Inequalities (LMIs) approach based on Lyapunov-Krasovskii functionals.
In addition, researchers have considered human factors in the design or evaluation of the teleoperated systems as it is shown in [24], where about 150 papers are described, covering human performance issues and mitigation solutions for teleoperated systems.However [25] emphasizes that human factor (HF) potential is underexploited and [26] expresses that, despite advances in autonomy, there will always be a need for human involvement in vehicle teleoperation.In addition, [27] states that many serious accidents, in which human error has been involved, can be attributed to faulty operator decision making.In this context, an operator can correctly select the actions but executes them poorly by overcorrecting, losing control, or flipping the commands.
This work presents a novel method to assist operators of a remote mobile robot in driving task.The method consists of weighting the operator commands by the risk associated to the current command activating also some safety action in the remote robot but only if the operator command increases the risk of collision.A control scheme based on impedance [28] is proposed to avoid obstacles, and the gain is variable according to the risk-based metric of the user commands  (which is also presented in this work).The benefits of this scheme were analyzed in a previous work presented by the authors [29], where the channel delay was not considered.
Besides, the stability of the system is analyzed considering the dynamics of master and slave robots as well as time-varying delays.Furthermore, the controller is tested from experiences over a virtual environment (Human in the Loop tests), in which a user drives a mobile robot teleoperated in presence of time delay, avoiding obstacles and feeling the force executed by the robot.In addition, an experiment of driving a real robot through a local network is presented.This work is organized as follows.First, in Section 2, the statement of the problem is presented, while in Section 3 the control scheme and a method for a dynamic risk-based command metric are exposed.Then, in Section 4, the stability of the system is analyzed.Then, an explanation of the implementation of the method developed is given and experiments are shown in Section 5. Finally, conclusions are exposed (Section 7).

Preliminary
This paper proposes and analyses teleoperation systems in which a human operator drives a wheeled robot while he perceives the environment near the robot through visual feedback and a force feedback related to the force executed by the mobile robot.In addition, the scheme computes the command performance in order to help the controller, as it is shown in Figure 1.
The master or local device is represented by the typical model where () ∈ R ×1 is the joint position of the master (in this work ) ∈ R × is the matrix representing centripetal and Coriolis torques; () ∈ R ×1 is the gravitational torque;  ℎ ∈ R ×1 is the torque caused by the human operator force; and   ∈ R ×1 is the control torque applied to the master.
For the case of teleoperation of wheeled robots, the dynamic model of a unicycle-type mobile robot is considered [21].It has two independently actuated rear wheels and is represented by where  = [ 1  2 ]  is the robot velocity vector with  1 and  2 representing the linear and angular velocity of the mobile robot,   is the force caused by the elements of the environment on the robot, D = [  0 0  ] is the inertia matrix, and Q = [ 0 −  0 ] is the Coriolis matrix, where  is the mass of the robot,  is the rotational inertia, and  is the distance between the mass center and the geometric center.In addition,  = [ 1  2 ] involves a control force  1 and a control torque  2 , with where   > 0 is the radius of the wheels,  > 0 is the half-width of the cart, and  left and  right are the torques of the left and right rear wheels, respectively.Furthermore, the communication channel adds a forward time delay ℎ 1 and a backward time delay ℎ 2 .Generally, these delays are time-varying and different between them (asymmetric delays).
On the other hand, the following ordinary properties, assumptions, and lemmas will be used in this paper.) is skew-symmetric.

Control Scheme Based on Human's Commands
The scheme proposed includes a PD-like velocity control, feedback of the force executed by the mobile robot, and an impedance loop to avoid obstacles depending on the performance of the delayed user's actions.The control structure is well known in teleoperation systems of manipulator robots [30,31].
Considering the control actions, we have for the local site and for the remote site where the controller is formed by   ∈ R ×1 (torque applied to the master),   ∈ R ×1 (torque applied to the robot), and   ∈ R ×1 (velocity impedance).The parameters   and   are positive constant (diagonal matrices) and they represent the proportional gain and damping added by a remote velocity controller;   is the damping injected in the master,   represents the scaling of force applied to the master, relative to the real force exerted by the mobile robot,   converts position to a velocity command compatible with the restrictions about maximum speed of the mobile robot (all of them are also diagonal matrices),  = [    ] is a fictitious force, and z is defined by with  → 0 + .Signal  represents the mobile robot acceleration η at an infinitesimal time instant before  so it is assumed η ≈  considering ż without discontinuities.In addition,  ℎ ∈ R ×1 is a parameter based on the riskbased command metric and   ∈ R 1×1 is a user impedance tuning parameter.3.1.Impedance Subsystem.In the control loop of the teleoperated robot, an impedance control law is applied based on a fictitious force, which depends on the nearest of the obstacles around the robot, in order to prevent collisions [32].Despite operator commands, the robot automatically reduces its mobility when some object is near it.Therefore, the reference commands sent by the human could differ from the real actions executed by the controller when the impedance is active.
Figure 2 shows a graphics representation of the fictitious force vector, generated from the robot-obstacle interaction.() is the distance between the robot and the obstacle, and a minimum  min and a maximum  max distance are set, in order to restrict the values of the fictitious force vector.It is decomposed in two components that affect the linear and angular speed impedance signal.Equation for computing the vector mentioned is described in [33], where () = (  (),   ()), being () =  −  (), with  and  as positive constant such that  −  max = 0 and  −  min = 1.Then,   = () cos Ω() and   = () sin Ω(), where Ω() is the angle between () and   ().
The impedance signal (  ) is calculated like (5), where speed references will keep the sense and direction of the user commands.
Variation of  ℎ is proportional to the risk-based command metric proposed (  ).That is,  the Operator command execution in front of risk.In order to define   , the following terms are established.

Proposed Definition of the
(i) Real Risk (  (); [ 0, . . ., 1 ℎℎ]).It is the level of risk produced by each obstacle  near to the robot, in the current time.
(ii) Command Risk (  (); [ 0, . . ., 1 ℎℎ]).It is the level of risk that would appear if the robot controller directly applies the user commands (considering   = 0), in the current time.
(iii) Graduation of Command's Performance (  (); [ − 1, . . ., 1 ℎℎ]).It is the level of improvement or worsening of Real risk as an effect of the Command risk, taking into account the ideal commands that are possible to be applied.
We differentiate when the effect of the operator's command (  , command risk) potentially improves, preservers, or worsens the level of risk   .For example, if the current   is valuated in 0.9 and   = 0.5, it means that the command applied by the user would decrease the real risk.
Then, in order to valuate the user skill required for the current situation, the more the high-risk level is, the more strongly should the metric weigh the improvement or worsening.For example, for the same numeric difference between   = 0.9 and   = 0.5 or   = 0.6 and   = 0.2 at a time instant   ,   should throw a higher value for the first case due to the improvement demanding wiser and quicker commands.
Therefore, in order to quantify   () as a function of   and   , a surface is proposed and showed in Figure 3.   () is bounded between −1 and 1 and includes 3 ranges: preservation, improvements, and worsening.Figure 3(b) shows the nonlinear quantification of the   () signal.Figure 3(a) pictures the contour of the surface.Green line corresponds to   = 0. Now, we define   as "The degree of goodness of the command that the user executes before risk situations." is computed as the Distance between the Ideal Action and the Real Action, where the Ideal Action (  id ) is the best possible user's command.That is, where with   id calculated like   but considering   = 0 and with   re calculated as   but taking into account   .Besides,  corresponds to the identification subindex for each obstacle and the variable  is the amount of obstacles in the current time   .
Figure 4 shows the vectorial model proposed in order to compute the metric definition (  ), considering only two obstacles for simplicity.A value of   = 0 implies a perfect operator decision making in command execution, and   = √(1 +   ) limits the worst decision for the  present obstacles.  is the worst value of the range Preservation.The upper limit of   is nonlinear and it is altered by the increment of obstacles in the environment.

Implementation of the Metric in This Work.
A function calculates the collision probability   based on the robot and obstacle position information, while   signal is calculated from the robot's kinematic model and the direct operator's commands.Once the calculation of () and () is made, () = (() ()) is computed through a neural network.
In order to create many points  to train the net, a fourplane surface is defined as it is shown in Figure 5.The net consists in one hidden layer with 10 neurons, and a linear function output layer.Figure 6 shows the approximation reached with the neural network used.After computing  for each obstacle,   is calculated as in (7).

System Stability Analysis
It is important to highlight the common states variables that should be analyzed in a teleoperation system.They are (1) the master-slave tracking error, (2) the master velocity ( q should be bounded or tend to 0), and (3) the acceleration of the master ( q should tend to 0).The stability analysis of the scheme proposed is based on the theory of Lyapunov-Krasovskii.Therefore, a positive definite functional  =  1 +  2 +  3 +  4 > 0 is proposed.It is important to remark that there is not an equilibrium point but a Krasovskii-like equilibrium solution that depends on the state  := [ q ( −   ) ]  in the time interval [−ℎ 1 −ℎ 2 , ] [34,35].The functional is formed by four parts.The first, second, and third subfunctional are proposed in the following manner: The time derivative of  1 , along master dynamics (1) and taking into account Properties 1 and 2, is the following: Now, if the control action   (5) is included in (13), it yields Next, V 2 along the dynamics of wheeled robot (2) is obtained as Later, from including (6) into the derivative of ( 12), and considering (2) and ( 5), V 3 can be written in the following way: It can be appreciated from ( 14) to ( 16) that there are delayed variables that make the stability analysis difficult.For solving the inconvenience of integral terms in the time derivative form of the Lyapunov's candidate,  4 is proposed (17) as a mathematic resource: where , ,  > 0 are positive definite matrices.
Considering the assumption of Preliminary section, V 4 can be formed as ()  () q  ()  q () ()  () q  ()  q () q  () q () . ( On the other hand, each term of V 1 + V 2 + V 3 that includes a delayed variable can be conveniently joined using Lemma 2 (3) with one term of V 4 .Consider ()  () In (22), the integral function is applied in a closed interval to ż () (assumed without discontinuities); therefore the function ∫ Next, joining terms from ( 15) and ( 16), the following inequality can be deduced: with  being a positive definite matrix.Finally, V can be built linking from ( 14) to (23) in the following way: where  is the identity matrix and  =   ℎ 2 .The controller can be set to guarantee that the quadratic terms of ( 24) are negative definite and therefore the signals q , ( −   ), and  will tend to a ball centered in the origin and whose size depends on the maximum magnitude of ,   ,  ℎ , and   .

Experimental Test and Results
In our test, a human drives a mobile robot as quick as possible (Figure 7), from an initial point to a final one, in an environment with multiple fixed and mobile obstacles.Operator drives the robot by using a Novint joystick while he receives the image captured by the camera onboard the robot, in presence of time delays ℎ 1 and ℎ 2 .The accomplishment of the task was evaluated based on some performance metrics proposed in Table 1.
The experimental test consisted in two parts (1 and 2).In the first one, many participant were asked to drive over a simulated robot (running under Matlab-Simulink), where also a simulated channel time delay was considered.Simulated environment and channel allows performing the same task under similar conditions, which makes possible a comparison between differences in human aspects.The second test was made using a real robot Pioneer through a local network.The parameters used for the mobile robot are  = 30 kg,  = 2.5 kg, and  = 0.05.In the stability analysis of the proposed scheme, parameters   ,   ,   ,   , and   were considered as scalar due to simplicity, but the analysis and results are also valid for positive definite diagonal matrices.Therefore, for the PD-like controller in the local site, the parameters set are and, for the controller of the master (local side), Finally, in order to transform the position of the master to a speed reference,   is established as follows: The range of the master positions in both axes is from −0.05 [m] to 0.05 [m]; therefore, the maximum speed linear and angular references are 1 [m/s] and 1 [rad/s].The parameter   is set to 0.8.
In order to analyze the control scheme proposed against classical methods based on velocity impedance or other methods that do not incorporate an impedance law, different  ℎ = [ 1  2 ]  are used: a constant high value (maximum), null, and  ℎ variable value (from 0 to a maximum, depending on the metric proposed (  ()).In Table 1, cases  are for null velocity impedance  ℎ = [0 0]  , cases  for constant velocity impedance  ℎ = [1 1]  (classical method based on impedance), and cases  for impedance varying according to the metric  ℎ = [5  5  ]  .About the fictitious force for the impedance subsystem, the parameters taken are  min = 0.5 [m] and  max = 3.5 [m]; therefore  = 1.16 and  = 0.33 (see Section 3.1).
Different drivers were taken to perform the required task (age range from 20 to 60, students and workers from the Universidad Nacional de San Juan).
Table 1 resumes the average values of evaluation index for the system (computed for experiment 1, 10 trials for each case row) in order to test the control system proposed and compare different cases listed above. represents the percentage of experiments in which the robot crashes against any obstacle.In this case the experiment is stopped.Next,  task measures the time to complete the task.
Then, HCP is an index about high collision probability.On one hand, the main task is required to be done as quickly as possible (evaluated by  task ), while, on the other hand, obstacles over the path represent restrictions to the movement of the robot.Hence, this index is about the risk the robot faces.We describe this evaluation metric as where  ob is the quantity of obstacles,   is the real risk robot− obstacle  ,  exp is the time of the experiment, and   is the collisions probability of the robot against obstacle .Designers can use any method for computing the collision probability.We use a method based on a linear integration of the velocities (velocity in  and  coordinates of the plane) of the robot and obstacle.The cubic function of the collision probability is chosen, due to the fact that it enhances high values of risk (high collision probabilities attempt mission's safety) and reduces the low values (not important for evaluating risk).Standard deviation is also computed for each case (  task and  hcp ).
In order to enhance the understanding of the readers, Figure 8 introduces a graphic of the index of the table.
A short part of one trial of experiment 1 was extracted and the main signals were plotted.Figure 9 shows the trajectories followed by the robot and the obstacles, where the darkness of each point represents the experiment's time for each sample (more dark is addressed to a time increasing).The separation of the points gives an idea of the speed reached by the robot and obstacles.One obstacle moves around a point and the human must take the short path through the other two fixed obstacles, trying to keep a low risk.The situation is present seven times along the path between the initial and final point, in order to stimulate the performance of the user before risk.The torque applied to the master for both speed references (linear and angular) is shown in Figure 10.
The impedance reduces the intensity of the commands applied in some moments (like at time  = 9 s), depending on the value of the metric proposed (Figure 13).As a result of this, the robot dynamics and the channel delays, velocities (linear and angular) reached by the robot, are different with respect to the human's commands references (Figure 11).
Figure 12 shows that q is bound to all time.Figure 13 exposes the real and command collision probabilities of the robot against each obstacle ([    ]), as well as the computation of the command metric proposed.It can be observed for  ≈ 10 that the metric is high because the command tends to worsen the collision probability situation against obstacle 1.Additionally, a cost function to minimize was introduced in Table 1 in order to summarize the information of the index introduced.For each row  of the table,   was computed as where variables in bold represent the normalized value of the index (e.g., N  = [((  −  min )/( max −  min ))100]), and the Figure 12: q of the master.
weight gains were chosen according to the relevance of each index.
Next, in order to achieve a direct performance comparison between schemes, cost functions of each one ( 1 is the first row of , corresponding to the cost functional of the scheme  under low delays) can be averaged as in It can be noted that scheme C (which represents the main proposal of this work) is the one that achieves the minimum cost function.

Experiment 𝐸2.
In order to test the system in a real robot, a Pioneer was teleoperated through a Local Network, based on the video feedback.The video of the experiment is available in Supplementary Material available online at http://dx.doi.org/10.1155/2015/460476.
The channel delay was approximately about 300 ms and despite being lower than the one used in the experiments over a simulated robot, the visual feedback is affected by video limitations like field of view and scope and quality and size of the image.The video was transmitted using the free software YawCam (http://www.yawcam.com).In Figure 14, a resume of the local and remote sides is shown.The robot must go from the starting point to the final one, going between the two obstacles present in the environment.
The results obtained are in agreement with the results achieved with the 3D simulator.

Discussions
6.1.Fix Velocity Impedance (Cases   ,  ℎ =[1 1]  , Classical Method).When the remote control applies a fixed gainimpedance, despite good or wrong operator commands, the robot moves very slowly when he navigates in the vicinity of the obstacle.The navigation of the robot is always safe for any kind of commands, but it offers bad performance in path and time to complete task for all occasions (about 35% more than the cases with the same time delay and using the metric; see  task of Table 1).In this case, the Transparency is unsatisfactory because the user's command is highly modified by the controller, and the operator perceives that the robot does not respond to his orders.This impedance law does not consider human driver performance.The increment in the time delay does not affect significantly the  task (less than 15%).The indexes of collisions probabilities measurements are nice, concluding that a safe robot navigation can be assured by using a fix impedance but with a bad  task (time to complete the task), regardless of operator performance.Therefore, good commands are wasted by the local controller.
6.2.Null Velocity Impedance (Cases   ,  ℎ =[0 0]  ).Here, the safety is low and only depends on the delayed operator's command performance.When bad commands occur, a high risk situation appears, and collisions could happen.This was demonstrated by the high percentage of collisions as a result of the bad commands.All participants express that they felt a good sensation when commanding the robot which exactly follows the joystick (nice Transparency) for low delays but, when time delay rises, the driving becomes quite difficult.Some good drivers achieve good executions, but this condition is difficult to maintain during a long task, or when secondary demanding tasks appear, which overload the attention or cognition of the driver, and therefore, poor decision and execution could be taken.

Variable Velocity
Impedance (Cases   ,  ℎ =[ 1  2 ]  ).In this case, conditions of safety driving do not depend strongly on commands performance.If bad commands occur,   takes high values and the risk situation is bounded.Thus, the collisions are mitigated at a cost of losing transparency.
The impedance reduces the received commands only if the potential risk worsens due to the user executions. is lower with this scheme than in the cases of scheme , showing a safer navigation.Furthermore, standard deviation of  is approximately the same as the deviation for cases , indicating a more homogeneous driving despite the performance of the commands executed.In addition, despite the time delay increment, standard deviation of each index of cases  remains similar, which proves that the homogeneous performance in front of delays increases.Time to complete the task depends somewhat on the commands performance, but it is smaller than case  and closer to the best one reached.

Conclusions
The work has presented a bilateral teleoperation controller for wheeled robots under time delays, in which the master controller is a PD-like scheme with gravity compensation and the slave controller is a scheme that incorporates an impedance compensation based on fictitious force and a variable gain, employed for obstacle avoidance.The variable gain is a function of the human operator performance when facing risk situations.The metric gives an online value that

Figure 1 :
Figure 1: Scheme of a human-robot bilateral teleoperation system.
Command Metric.In this section, we propose a performance index   to quantify Mathematical Problems in Engineering Surface of the , , and

Figure 4 :Figure 5 :
Figure 4: Quantitative components of the metric proposed.

Figure 7 :
Figure 7: Photograph of the experiment 1.Simulated environment and communication channel.

3 X
-axis (m) Y-axis (m)Trajectories of the robot and obstacles

Figure 10 :
Figure 10: Applied torque to the master.

Table 1 :
Table of evaluation metrics computed for experiment 1.
Graphic from Table 1: percentage of collisions, time to complete the task, and index of high collision probability.
Figure 13: Real and command collision probabilities of the robot against each obstacle. 1