A novel control algorithm based on the modified wave-variable controllers is proposed to achieve accurate position synchronization and reasonable force tracking of the nonlinear single-master-multiple-slave teleoperation system and simultaneously guarantee overall system’s stability in the presence of large time-varying delays. The system stability in different scenarios of human and environment situations has been analyzed. The proposed method is validated through experimental work based on the 3-DOF trilateral teleoperation system consisting of three different manipulators. The experimental results clearly demonstrate the feasibility of the proposed algorithm to achieve high transparency and robust stability in nonlinear single-master-multiple-slave teleoperation system in the presence of time-varying delays.
1. Introduction
Teleoperation through which a human operator can manipulate a remote environment expands human’s sensing and decision making with potential applications in various fields such as space exploration, undersea discoveries, and minimally invasive surgery [1–3]. From the teleoperation’s point of view, a teleoperation system can be of two categories, bilateral or multilateral.
A conventional bilateral teleoperation system which consists of a pair of robots allows sensed and command signals flow in two directions between the operator and the environment: the command signals are transmitted from the master to control the slave and the contact force information is simultaneously fed back in the opposite direction in order to provide human operator the realistic experience. System stability is quite sensitive to time delays and even a small time delay may destabilize the overall system. Many researchers have been focusing on guaranteeing robust stability of a teleoperation system in the presence of time delays. Based on the passivity theory and the scattering approach, the stability analysis and controller design for the bilateral teleoperation system have been widely studied [4, 5]. The most remarkable passivity-based approach is the wave-variable method introduced by Niemeyer and Slotine [6]. Numerous studies have explored the application of wave-variable theory to enhance the task performance of the wave-variable-based system as reported in [7]. Yokokohji et al. design a compensator to minimize the performance degradation of the wave-based system [8, 9]. Munir and Book apply the wave prediction method which employs the Smith predictor and Kalman filter to deal with the Internet-based time-varying delay problem [10]. Hu et al. compensate for the bias term to improve the trajectory tracking of the wave-variable-based system [11]. Through adding correction term, Ye and Liu enhance the accuracy of the system’s force tracking [12]. Aziminejad et al. further extend the wave-based system to the four-channel system by introducing measured force reflection [13]. Alise et al. analyze the application of the wave variables in multi-DOF teleoperation [14].
A conventional bilateral teleoperation system usually involves a single slave robot which is controlled by a single operator. However, it is more effective in many applications to have multiple manipulators in a teleoperation system. Therefore, the multilateral teleoperation has been gradually becoming a popular topic and many approaches have been proposed such as H∞ control [15, 16], disturbance-observer-based control [17], and adaptive control [18]. Although the wave-variable transformation can guarantee the communication channels’ passivity, most of the wave-based systems are not suitable to be extended to the multilateral teleoperation since they cannot guarantee the system stability under time-varying delays. Moreover, the wave-based systems also suffer transparency degradation and signals variation and distortion due to the existence of wave reflections. Without reducing the wave reflections, one robot with large variations can seriously influence other robots’ task performance and the users’ perception of the remote environment in the presence of large time-varying delays. Therefore, guaranteeing system stability under time-varying delays and enhancing the system transparency via wave reflections reduction are the two key criteria for the successful application of the wave-variable approach in the multilateral teleoperation.
As a part of multilateral teleoperation control, multiple-masters-single-slave (MMSS) system includes more than one single operator to collaboratively carry out the task [15, 20–23]. Unlike the MMSS system, the single-master-multi-slave (SMMS) system allows one operator to simultaneously control multiple slave robots. The SMMS teleoperation is firstly introduced in [24]. Later, the single-master-dual-slave scenario is investigated under constant time delays for a linear one-DOF teleoperation system in [17, 25–28]. In a SMMS system, the multiple slave robots should not only coordinate their motions (e.g., robotic network as a surveillance sensor network) but also perform cooperative manipulation and grasping of a common object [19], as shown in Figure 1. A SMMS system is suitable for many applications where (1) a single slave robot cannot perform the required level of manipulation dexterity, mechanical strength, robustness to single point failure, and safety (e.g., distributed kinetic energy) and (2) the remote task necessarily requires the human operator’s experience, intelligence, and sensory input, but it is not desired or even impossible to send humans on site. One example of such applications is the cooperative construction/maintenance of space structures (e.g., international space station, Hubble telescope) [29]. It requires high demand for these slave robots to have precise actions following the human operator to perform different remote environmental tasks in the presence of time-varying delays.
Single-master-multiple-slave (SMMS) system [19].
In this paper, a novel modified wave-variable-based control algorithm is designed to guarantee accurate position synchronization and force reflection of all the robots in the nonlinear SMMS teleoperation system in the presence of large time-varying delays. The stability of the multirobots system in different environmental scenarios is also analyzed. The theoretical work presented here is supported by experimental results based on a 3-DOF trilateral teleoperation system consisting of three different haptic devices.
2. Modeling the n-DOF Multilateral Teleoperation System
In this paper, the master robot and the n-slave robots are modeled as a pair of multi-DOF serial links with revolute joints. The nonlinear dynamics of such a system can be modeled as(1)Mmqmq¨m+Cmqm,q˙mq˙m+gmqm=τm+τh,Ms1qs1q¨s1+Cs1qs1,q˙s1q˙s1+gs1qs1=τs1-τe1,Ms2qs2q¨s2+Cs2qs2,q˙s2q˙s2+gs2qs2=τs2-τe2,⋮Msnqsnq¨sn+Csnqsn,q˙snq˙sn+gsnqsn=τsn-τen,where i=m,s, m is master, and s is slave. q¨ij,q˙ij,qij∈Rn are the joint acceleration, velocity, and position, respectively, m denotes master, and sj denotes the jth slave. j∈1,2,…,n denotes the number of the slave robots. Mij(qij)∈Rn×n are the inertia matrices; Ci(qi,q˙i)∈Rn×n are Coriolis/centrifugal effects. gij(qij)∈Rn are the vectors of gravitational forces and τij are the control signals. The forces applied on the end-effector of the master and slave robots are related to equivalent torques in their joints by(2)Fh=JmTτh,Fen=JsnTτen,where Jm, Jsn are the Jacobean of the master robot and the nth slave robot, respectively. Fh and Fen stand for the human and environment forces, respectively.
Important properties of the above nonlinear dynamic model, which will be used in this paper, are as follows [25, 30].
The inertia matrix Mij(qij) for a manipulator is symmetric positive-definite which verifies 0<σmin(Mij(qij(t)))I≤Mij(qij(t))≤σmax(Mij(qij(t)))I≤∞, where I∈Rn×n is the identity matrix. σmin and σmax denote the strictly positive minimum (maximum) eigenvalue of Mij for all configurations qij.
Under an appropriate definition of the Coriolis/centrifugal matrix, the matrix M˙ij-2Cij is skew symmetric, which can also be expressed as(3)M˙ijqijt=Cijqijt,q˙ijt+CijTqijt,q˙ijt.
The Lagrangian dynamics are linearly parameterizable:(4)Mijqijq¨ij+Cijqij,q˙ijq˙ij+gijqij=Yqij,q˙ij,q¨ijθ,
where θ is a constant p-dimensional vector of inertia parameters and Yqm,s,q˙m,s,q¨m,s∈Rn×p is the matrix of known functions of the generalized coordinates and their higher derivatives.
For a manipulator with revolute joints, there exists a positive Z bounding the Coriolis/centrifugal matrix as(5)Cijqijt,xtyt2≤Zxt2yt2.
The time derivative of Cij(qij(t),x(t)) is bounded if qij(t) and q˙ij(t) are bounded.
3. Wave Variable and the Proposed Method
Figure 2 shows the standard wave-variable transformation where the wave variables (um and vs) are defined as(6)um=bq˙m+τm2b,vs=bq˙s-τs2b,where b denotes the wave characteristic impedance and ui and vi are the wave variables being transmitted in the communication channels. The power flow P can be expressed as(7)P=τmtq˙mt-τstq˙st.A system is passive if the output energy is no more than the sum of the initial stored energy and the energy injected into the system [14]. The wave-based teleoperation system is passive when it satisfies (8), where Estore0 is the initial energy stored in the system. Consider(8)∫0t12vsTtvst-vmTtvmt≤∫0t12umTtumt-usTtust+Estore0,kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkki∀t≥0.When applied to the multilateral teleoperation, the wave-variable transformation must meet two requirements, maintaining channels passivity in the presence of random time delays and transmitting signals without large variation and distortion. Considering the time delays, the power flow can be further written as(9)P=12umTtumt-vmTtvmt+vsTtvstkkkkk-usTtust=12umTtumt-umTt-Tftkkkkk·umt-Tft+vsTtvst-vsTt-Tbtvst-Tbt=ddt∫t-TbttvsTηvsη2dη-12T˙2tvsTt-Tbtvs·t-Tbt+ddt∫t-TfttumTηumη2dη-12T˙1tumTt-Tftumt-Tft=Pdiss+dEstoredt,Estore0t=∫t-TfttumTηumη2dη+∫t-TbttvsTηvsη2dη,Pdisst=-12T˙btvsTt-Tbtvst-Tbt-12T˙ftumTt-Tftumt-Tft,where Pdiss(t) is the power dissipation of the communication channels. Pdiss(t)≥0 indicates passiveness of the channels. In this paper, the time-varying delays are assumed not to increase or decrease faster than time itself; that is, T˙f,bt<1 [31]. T˙f,bt is the differential of the time delays. In the presence of constant time delays (T˙f,bt=0), the power dissipation Pdiss(t) is equal to zero based on (10). It means the wave-based controller assures passivity regardless of the value of constant time delay. However, when the time delay is varying, the positive T˙f,bt results in Pdiss(t) to be negative and the system passivity will be degraded. Therefore, the conventional wave-variable transformation cannot guarantee system passivity under time-varying delays.
Standard wave-based teleoperation architecture.
Wave reflection is another main drawback of the standard wave transformation, which is caused by the imperfectly matched junction impedance in the wave-based system as shown in Figure 3. There are three independent channels in the wave-variable transformation in Figure 3, the master’s direct feedback (dotted line 1), the wave reflection (dotted line 2), and the force feedback from the slave (dotted line 3). In channel 1, the master velocity signals directly return in the form of the damping bq˙m. Channel 1 generates a certain amount of damping and this enhances the system stability by sacrificing transparency. Channel 3 feeds signals back from the remote slave side in order to provide useful information to the operator. Wave reflections occur in channel 2.
Wave reflections.
The phenomenon of wave reflection occurs in channel 2. The relationship between the outgoing wave variables um and vs and the incoming wave variables vm and us can be expressed as(10)umt=-vmt+2bq˙mt,(11)vst=-ust+2bτst.Each of the incoming wave variables vm and us is reflected and returned as the outgoing wave variables um and vs. Wave reflections can last several cycles in the communication channels and then gradually vanish. This phenomenon can easily generate unpredictable interference and disturbances that significantly influence transparency [15]. Large signals variation and distortion can be caused by the wave reflections in the presence of large time delays. Therefore, the standard wave-variable transformation is not suitable for multilateral teleoperation when large time-varying delays exist.
In order to guarantee the passivity of the time delayed communication channels between the master robot and each slave robot, the modified wave-variable controllers proposed in [32] are applied in this paper as shown in Figure 4. The main advantage of the modified wave controllers is the efficient reduction in the wave-based reflections while simultaneously guaranteeing channels’ passivity as analyzed in [32].
Modified wave-variable controllers.
The two wave-variable controllers are applied to encode the feed-forward signals VA1 and VB1 with the feedback signals IA1 and IB1. The wave variables in the two controllers are defined as follows:(12)um1t=bVA1t+1/λIA2t-Tft2b,us1t=bVA2t+1/λIA2t2b,(13)vm1t=IA2t-Tbt2b,vs1t=IA2t2b,(14)um2t=bVB1t2b,us2t=bVB1t-Tft2b,(15)vm2t=b/λVB1t-IB1t2b,vs2t=b/λVB1t-Tft-IB2t2b,where b and λ are the characteristic impedances. vs1 and um2 do not contain any unnecessary information from the incoming wave variables us1 and vm2 as shown in (13) and (14). Therefore, wave reflections can be efficiently eliminated.
In the proposed SMMS teleoperation system (Figure 5) in which one master robot is used to control multiple slave robots, the main objective is to have the positions of all the slave robots accurately synchronized to the position of the master robot. A secondary objective is that all the robots should have accurate force tracking with each other, which means when one slave robot comes in contact with the remote environmental object during free motion, it will immediately feed back the force information to all of the other robots to signal them to stop. Via reaching the two targets, all the slave robots will precisely follow the human operator in different environmental scenarios. By applying the two wave controllers, the energy information such as torque, position, and velocity signals can be transmitted through the communication channels without influencing the system passivity. By setting VA1t=C1τht, IB1t=βq˙mt+δqmt, IA2t=-β(q˙st+δqs(t)), and VB2t=C2τet, a new state variable Em for the master robot is introduced as follows:(16)Em=∑j=1nC3j-bjλjC1jτht-C2jτejt-Tbjtbjλjkkkikk+βjq˙sjt-Tbjt+δqsjt-Tbjtkkikkk-βjq˙mt+δqmtkkikkk-bjλjβjq˙mt+δqmtkkkkikkkkk-bjλjβjq˙mt-Tfjt-Tbjtkkkkkkikkkkkkkikkkbjλj+δqmt-Tfjt-Tbjt,where C1–4, β, and δ are diagonal positive-definite matrices. In the slave sides, each slave robot receives control signals from the master robot and the other slave robots. The new master-control state variable Esn∗ for the nth slave robot is written as follows:(17)Esn∗=C1nτht-Tfnt-λnC2nbn-C4nτent+βnq˙mt-Tfnt+δqmt-Tfnt-βnq˙snt+δqsnt-βnbnλnq˙snt+δqsntkkkkk-βnbnλnq˙snt-Tfnt-Tbntkkkkkkkkkkkkβnbnλn+δqsnt-Tfnt-Tbnt.In order to prevent the position drift between the slave robots, each slave robot should also transmit its position information to the other slave robots. Furthermore, In order to achieve the secondary objective which is the accurate force tracking, each slave robot’s environmental force information is also transmitted via slave-slave communication channels to the other slave robots. The channels’ passivity is guaranteed when the wave-variable controller proposed in [33] is applied to encode the yth slave robot’s position signals with the transmitted zth slave robot’s control environmental force (y and z denote the arbitrary two slave robots in the n slave robots). Therefore, the final control variable Esn of the nth slave robot is expressed as(18)Esn=C1nτht-Tfnt-λnC2nbn-C4nτent+βnq˙mt-Tfnt+δqmt-Tfnt-βnq˙snt+δqsnt-βnbnλnq˙snt+δqsntkkkki-βnbnλnq˙snt-Tfnt-Tbntkkkkkkkkkkkβnbnλn+δqsnt-Tfnt-Tbnt+∑j=1n-11-T˙sjtkkkkkkk·βsjq˙sjt-Tsjt+δqsjt-Tsjtkkkkkkkkkk1-T˙sjt-βsjq˙snt+δqsnt-∑j=1n-11-T˙sjtkcjτejt-Tsjt,where Tsj(j∈(1,2,…,n)) denote the time-varying delays in the forward slave-slave communication channels and kcj are diagonal positive-definite matrices. The second last term provides the position control between every two slave robots and the last terms provide force control between every two slave robots. By defining new variables,(19)rijt=q˙ijt+δqijt(16) and (18) can be simplified as follows:(20)Em=∑j=1nC3j-bjλjC1jτht-C2jτejt-Tbjtbjλjkkkkkk+βjrsjt-Tbjt-rmtkkkkkk-bjλjβjrmt-rmt-Tfjt-Tbjt,(21)Esn=C1nτht-Tfnt-λnC2nbn-C4nτent+βnrmt-Tfnt-rsnt-βnbnλnrsnt-rsnt-Tfnt-Tbnt+∑j=1n-1βsj1-T˙sjtrsjt-Tsjt-rsnt-∑j=1n-11-T˙sjtkcjτejt-Tsjt.The main aim of the controller design is to provide a stable multilateral system with accurate position tracking and to enhance the force tracking during manipulations. The position synchronization is derived if(22)limt→∞∑j=1nqmt-Tfjt-qsjt=limt→∞∑j=1nq˙mt-Tfnt-q˙sjt=0,(23)limt→∞∑j=1nqsjt-Tbjt-qmt=limt→∞∑j=1nq˙sjt-Tbjt-q˙mt=0,(24)limt→∞∑j=1nqsjt-Tsjt-qsnt=limt→∞∑j=1nq˙sjt-Tsjt-q˙snt=0,where · is the Euclidean norm of the enclosed signal. We define the position errors epmn, epsn and velocity errors evmn, evsn between the master and the nth slave manipulators as follows:(25)epmnt=qmt-Tfnt-qsnt,(26)evmnt=q˙mt-Tfnt-q˙snt,(27)epsnt=qsnt-Tbnt-qmt,(28)evsnt=q˙snt-Tbnt-q˙mt,(29)epssnt=qsjt-Tsjt-qsnt,(30)evssnt=q˙sjt-Tsjt-q˙snt.The new control laws for the single master robot and the nth slave robot are designed as follows:(31)τm=Em-M^mqmδq˙m-C^mqm,q˙mδqm+g^mqm,τsn=Esn-M^snqsnδq˙sn-C^snqns,q˙snδqsn+g^snqsn,where M^iqi, C^iqi,q˙i, and g^iqi are the estimates of Miqi, Ciqi,q˙i, and giqi(i∈(m,s1,s2,…,sn)). Substituting (24) and (25) into (1) and considering Property 3 which states that the dynamics are linearly parameterizable, the new system dynamics can be expressed as(32)Miqir˙i+Ciqi,q˙iri=Ei-Yiθ~i,where(33)θ~it=θit-θ^it;θ^i are the time-varying estimates of the master’s and the nth slave’s actual constant p-dimensional inertial parameters given by θi. θ~i are the estimation errors. The time-varying estimates of the uncertain parameters satisfy the following conditions [33]:(34)θ^˙m=ψYmTqm,rmrm,θ^˙sn=ΛnYsnTqsn,rsnrsn.
Consider the proposed nonlinear multilateral teleoperation system described by (16)–(34) in free motion where the human-operator force τh and the environmental force τe can be assumed to be zero (τh≡τe≡0). For all initial conditions, all signals in this system are bounded and the master and all of the slave manipulators state are synchronized in the sense of (22) and (24).
Proof.
Based on (13) and (14), Em and Esn have the terms ∑j=1n-(bj/λj)βj(rmt-rm(t-Tfj(t)-Tbj(t))) and -(βn/bnλn)[rsn(t)-rsn(t-Tfnt-Tbnt)], respectively. These two terms can be expressed as ∑j=1n-(bj/λj)βjrm(s)(1-e-s(Tfj(s)+Tbj(s))) and -(βn/bnλn)rsn(s)(1-e-s(Tfn(s)+Tbn(s))) in frequency domain. According to the well-known characteristic of the time delay element [34],(35)e-sTf,b=1,it is true that (1-e-s(Tfj(s)+Tbj(s)))∈[0,2] in the presence of large time-varying delays. It means rmt-rm(t-Tfjt-Tbjt)∈[0,2rmt] and rsn(t)-rsn(t-Tfnt-Tbnt)∈[0,2rsnt] which are varying according to the time delays. Therefore, (rmt-rm(t-Tfjt-Tbjt)) and (rsnt-rsn(t-Tfnt-Tbnt)) can be expressed as the varying dampings ζrmt and ζrsnt where ζ varies between 0 and 2. The values of ζrmt and ζrsnt are scaled by the characteristic impedances b and λ of the applied modified wave controllers. Therefore, (20) and (21) can be expressed as(36)Em=∑j=1nC3j-bjλjC1jτht-C2jτejt-Tbjtbjλjkkik+βjrsjt-Tbjt-rmt-bjλjβjζrmt,Esn=C1nτht-Tfnt-λnC2nbn-C4nτent+βnrmt-Tfnt-rsnt-βnbnλnζrsnt+∑j=1n-1βsj1-T˙sjtrsjt-Tsjt-rsnt-∑j=1n-11-T˙sjtkcjτejt-Tsjt.Define a storage functional V, where(37)V=12rmTtMmqmrmt+∑j=1nrsjTtMsjqsjrsjtkkkkk+θ~mTψ-1θ~m+∑j=1nθ~sjTΛj-1θ~sj+∑j=1nβj211-T˙fj∫t-TfjtrmTηrmηdη+∑j=1nβj211-T˙bj∫t-TbjtrsjTηrsjηdη+n∑j=1n-1βsj2∫t-TfjtrsjTηrsjηdη+∑j=1nqmTtbjζβjλj-T˙fjβj2-2T˙fjδqmt+∑j=1nqsjTtβjζbjλj-T˙bjβj2-2T˙bjδqsjt.In order to make V positive semidefinite, bjζβj/λj-T˙fjβj/(2-2T˙fj)≥0 and βjζ/bjλj-T˙bjβj/(2-2T˙bj)≥0(j∈1,2,…,n) should be satisfied, which can be simplified as(38)T˙fj≤2ζλj/bj+2ζ,T˙bj≤2ζbjλj+2ζ.Due to the assumption that T˙f,b<1, by setting a small value of λj, (38) can be easily satisfied. By using the dynamic equations and Property 3, the derivative of V can be written as(39)V˙=rmTtEmt+∑j=1nrsjTtEsjt+∑j=1nβj2rmTtrmt-βj2rmTt-Tfjtrmkkkkkkkki·t-Tfjt+βjT˙fj2-2T˙fjrmTtrmt+∑j=1nβj2rsjTtrsjt-βj2rsjTt-Tbjtrsjkkkkkkkki·t-Tbjt+βjT˙bj2-2T˙bjrsjTtrsjt+n∑j=1n-1βsj2rsjTtrsjt-1-T˙sjtβsj2rsjTkkkkkkkkiβsj2·t-Tsjtrsjt-Tsjt+∑j=1nq˙mTt2bjζβjλj-T˙fjβj2-2T˙fjδqmt+∑j=1nq˙sjTt2βjζbjλj-T˙bjβj2-2T˙bjδqsjt=-∑j=1nβj2evmjt+δepmjtTevmjt+δepmjt-∑j=1nβj2evsjt+δepsjtTevsjt+δepsjt-n∑j=1n-1βj2evssjt+δepssjtTevssjt+δepssjt-∑j=1nq˙mTtbjζβjλj-T˙fjβj2-2T˙fjq˙mtkkkkkkkk+qmTtbjζβjλj-T˙fjβj2-2T˙fjδ2qmtkkkkkkkk+q˙sjTtβjζbjλj-T˙bjβj2-2T˙bjq˙sjtkkkkkkkk+qsjTtβjζbjλj-T˙bjβj2-2T˙bjδ2qsjt≤0.Based on (39), the differential of the functional V is negative semidefinite. Integrating both sides of (39), we get(40)+∞>V0≥V0-Vt≥∫0t∑j=1nβj2evmjt+δepmjtTkkkkkkkkkkβj2·evmjt+δepmjtkkkkkk+∑j=1nβj2evsjt+δepsjtTkkkkkkkkkkkkβj2·evsjt+δepsjtkkkkkk+n∑j=1n-1βj2evssjt+δepssjtTkkkkkkkkkkkkβj2·evssjt+δepssjtkkkkkk+∑j=1nq˙mTtbjζβjλj-T˙fjβj2-2T˙fjq˙mtkkkkkkkkkkkk+qmTtbjζβjλj-T˙fjβj2-2T˙fjδ2qmtkkkkkkkkkkkk+q˙sjTtβjζbjλj-T˙bjβj2-2T˙bjq˙sjtkkkkkkkkkkkk+qsjTtβjζbjλj-T˙bjβj2-2T˙bjkkkkkkkkkkkk∑j=1nbjζβjλj-T˙fjβj2-2T˙fj·δ2qsjtdt.Since V is positive semidefinite and V˙ is negative semidefinite, limt→∞V exists and is finite. Also, based on (37)–(40), rmt,rsjt,θ~mt,θ~sjt∈L∞, evmjt, epmjt, evsjt, epsjt, qmt, qsjt, evssjt, epssjt, q˙mt, q˙sjt∈L∞∩L2. Since a square integrable signal with a bounded derivative converges to the origin [31, 33, 35], limt→∞epmjt=limt→∞evmjt=limt→∞epsjt=limt→∞evsjt=limt→∞epssjt=limt→∞evssjt=0. Therefore, the master and slave manipulators state synchronize in the sense of (22)–(24).
In free motion, the system’s dynamic model (26) can also be written as (41)q¨it=Mi-1Eit-Yiθ~i-Cirmt-δq˙it.Differentiating both sides of (41),(42)ddtq¨it=ddtMi-1Eit-Yiθ~i-Cirit+Mi-1ddtEit-Yiθ~i-Cirit-δq¨it.For the first terms of the right sides of (42), we have [36](43)ddtMi-1=-Mi-1M˙iMi-1=-Mi-1Ci+CiTMi-1.According to Properties 1 and 4, d/dt(Mi-1) are bounded. Based on Property 5, the terms in bracket of (29) are also bounded. Therefore, d/dtq¨it∈L∞ and q¨it are uniformly continuous (∫0tq¨iηdη=q˙it-q˙i0). Since q˙it→0, it can be concluded that q¨it→0 based on Barbǎlat’s Lemma.
4.2. Environmental Contact with Passive Human Force
Assume the human and environmental forces are passive and can be modeled as(44)τht=-αmrmt,τejt=αsjrsjt,where αm and αsj are positive constant matrices and are the properties of the human and the environment, respectively.
Theorem 2.
The multilateral nonlinear teleoperation system described by (16)–(34) is stable and all signals in this system are ultimately bounded, when the human and environmental forces satisfy (44).
Proof.
Consider a positive semidefinite function V′ for the system as(45)V′=V+∑j=1nC3j-bjλjC1jαm2∫t-TfjtrmTηrmηdη+∑j=1nλjC2j/bj-C4jαsj2∫t-TbjtrsjTηrsjηdη+n∑j=1n-1αsj2n∫t-TsjtrsjTηrsjηdη.The derivative of V′ can be written as(46)V˙′=∑j=1n-C3j-bjλjC1jαm2rmTtrmtkkkkkkkkk+C2jαsjrmTtrst-Tbjkkkkkkkkk+λjC2j/bj-C4jαsj2kkkkkkkkkC3j-bjλjC1jαm2·1-T˙bjrsjTt-Tbjrsjt-Tbj+∑j=1n-λjC2j/bj-C4jαsj2rsTtrstkkkkkkkkkkk+C1jαmrsTtrmt-Tfjkkkkkkkkkkk+C3j-bjλjC1jαm21-T˙fjkkkkkkkkkkkλjC2j/bj-C4jαsj2·rmTt-Tfjrmt-Tfj+n∑j=1n-1-αsn2nrsnTtrsnt+1-T˙sjtkcjαsjkkkkkkkkkkk·rsnTtrsjt-Tfj+αsj2n1-T˙sjrsjTkkkkkkikkkkαsn2n·t-Tsjrsjt-Tsj-αmrmTtrmt+V˙.The Lyapunov approach requires V˙′ to be negative semidefinite. Based on the first three terms of the right side of (46), the sufficient conditions to satisfy this requirement are that(47)11-T˙bjC2j2λjC2j/bj-C4jC3j-bjλjC1jI≤αmαsj-1T,11-T˙fjC1j2λjC2j/bj-C4jC3j-bjλjC1jI≤αsjαm-1T,kcjTkcj≤1n2αsnαsj-1T.By enlarging the values of C3j and decreasing the values of kcj, (47) can be satisfied. Hence, V˙′ will be negative semidefinite and limt→∞V′ exists and is finite.
4.3. Environmental Contact with Nonpassive Human Force
The human operator can not only dampen energy but also generate energy in order to manipulate the robots to move through the desired path. Therefore, in the common case, the human forces are not passive. In this situation, the human and environment can be modeled as(48)τh=α0-αmrm,τej=αsjrsj,where α0 is a bounded positive constant vector, which generates energy as an active term. We define x-j=[qm,qsj,q˙m,q˙sj]T and xj=[qm,qsj,rm,rsj]T. There is a linear map between x-j and xj [33]:(49)x-jt=Γjxjt,where Γj are nonsingular constant matrices.
Theorem 3.
The proposed system is stable and all signals in this system are ultimately bounded, when the human and environmental forces satisfy (48).
Proof.
By choosing the previous Lyapunov function V′, the new derivative V˙∗ can be written as(50)V˙∗=V˙′+∑j=1nrmTC3j-bjλjC1jα0+α0+∑j=1nrsjTλjC2jbj-C4jα0.Note that (51)∑j=1nrmTC3j-bjλjC1jα0+α0≤∑j=1nhTxjC3j-bjλjC1jα0+α0,∑j=1nrsjT∑j=1nrsjTλjC2jbj-C4jα0≤∑j=1nhTxj∑j=1nrsjTλjC2jbj-C4jα0,where vector hT=[1,1,…,1] has the same ranks as rm, rsj. Therefore, it is true that(52)V˙∗≤V˙′+∑j=1n2xjαj,where αj=(C3j-bjλjC1j)α0+α0+(λjC2j/bj-C4j)α0>0. When the system satisfies (47),(53)V˙∗≤-∑j=1nq˙mTtbjζβjλj-T˙fjβj2-2T˙fjq˙mtkkkkkkki+qmTtbjζβjλj-T˙fjβj2-2T˙fjδ2qmtkkkkkkki+q˙sjTtβjζbjλj-T˙bjβj2-2T˙bjq˙sjtkkkkkkki+qsjTtβjζbjλj-T˙bjβj2-2T˙bjδ2qsjt≤∑j=1n-Υjx-j2,where Υj is the smallest eigenvalue of (βjζ/bjλj-T˙bjβj/(2-2T˙bj)), (βjζ/bjλj-T˙bjβj/(2-2T˙bj))δ2, (bjζβj/λj-T˙fjβj/(2-2T˙fj)), and (bjζβj/λj-T˙fjβj/(2-2T˙fj))δ2. Substituting (53) into (52) and setting 0<μ<1,(54)V˙∗≤∑j=1n-Υjx-j2+2xjαj=∑j=1n-Υj1-μΓj2xj2-ΥjμΓj2xj2kkkkkkxj2+2xjαj,(54) can be simplified as(55)V˙∗≤∑j=1n-Υj1-μΓj2xj2,∀xj≥2αjΥjμΓj2.Based on (55), for large values of xj, the Lyapunov function is decreasing. Therefore, xj and x-j are bounded, which means rm, rsj, qm, qsj, q˙m, and q˙sj are also bounded.
5. Experimental Validation
In this section, the performance of the proposed nonlinear multilateral teleoperation system is validated by a series of experiments. The algorithm is applied to three Phantom manipulators. The 6-DOF Phantom (TM)* model 1.5 manipulator (Sensable Technologies, Inc., Wilmington, MA) is chosen to be the master robot which remotely controls a 3-DOF Phantom Omni (Slave 1) and a 3-DOF Phantom Desktop (Slave 2) via the Internet as shown in Figure 3. The three haptic devices have different dynamics and initial parameters. PhanTorque toolkit [36] is applied by two computers to control the two robots. PhanTorque toolkit enables the users to work with the Sensable Phantom haptic devices in the Matlab/Simulink environment in a fast and easy way. Figure 4 shows the trilateral experiment platform.
The control loop is configured as a 1 kHZ sampling rate. Based on the controllers analysis in Section 4, the controller parameters are given as b1=b2=2.5, λ1=λ2=0.5, C1=C2=1, C3=2, C4=1.2, δ=1.2, β1=5, β2=3, βs=2, kc=1.
5.1. Bilateral Teleoperation (1-DOF)
In this subsection, the proposed wave-based architecture is compared with the standard wave-based system in bilateral teleoperation using 1-DOF. The time delay (one way) is 400 ms constant delay.
Figures 7 and 8 show the velocity and position tracking of the two systems in free motion. Based on (10)-(11), due to the wave reflections, the useless signals remain in the communication channels for several circles to the extent that the normal signals transmissions are influenced and the transmitted velocity control signals contain large signals variations. Moreover, considering the conventional wave variables in (6), the signal transmission in the standard wave-based system can be expressed as(56)q˙st=q˙mt-Tf-1bτst-τmt-Tf,(57)τmt=τst-Tb+bq˙mt-q˙st-Tb.The biased terms -(1/b)[τs(t)-τm(t-Tf)] and b[q˙m(t)-q˙s(t-Tb)] also seriously affect the accuracy of the position tracking. Since the standard wave-based system is an overdamped system, by applying the same operation force, the velocity and position of the standard wave-based system are lower than those of the proposed system and the operator feels damped when operating the system. Unlike the standard system, the proposed wave-based system has little signals variations since the wave reflections are almost eliminated. According to (20) and (21), the biased terms affecting position tracking are b/λβ[rm(t)-rm(t-Tf(t)-Tb(t))] and -(β/bλ)[rs(t)-rs(t-Tf(t)-Tb(t))]. Under small time delays, the biased terms are about zero. When the time delays are nonignorable, setting large value of λ can also effectively reduce the biased terms. Therefore, both of the velocity and the position have accurate tracking performances.
Figures 9 and 10 show the torque tracking and position tracking of the two systems in hard contact. As shown in Figure 9, the standard wave-based system can only achieve accurate force tracking in steady state. In the transient state, when the environment undergoes unpredictable changes, wave reflections occur so that the force reflection has large perturbations and the operator can hardly feel the accurate environmental force. Moreover, according to (56), since the standard wave-based system has no direct position transmission, position drift occurs during hard contract. It means that when directly applying the conventional wave-variable transformation in the SMMS system, when one slave robot contacts with the remote environment and is forced to stop, the master robot still keeps moving which can drive other slave robots to move. Therefore, the robots’ motion synchronization will be jeopardized. As shown in Figure 10, the environmental torque quickly tracks the operator’s torque without variation and no position drift occurs during hard contact, which means when applying to the SMMS system, the proposed architecture can not only provide accurate force tracking, but also achieve motion synchronization.
5.2. Multilateral Teleoperation (3-DOF)
In this subsection, the proposed SMMS system is validated. The communication channel of the experimental platform is the Internet. In order to test the performance of the proposed system in the presence of large time-varying delays, the time delay blocks in the Simulink library are applied to introduce the overall system time delays (Figure 6). The one-way delay between the master and the slave sides is from 650 ms to 750 ms. Theoretically, in the real applications, the slave robots are close to each other, so the time delays between two slave robots are not large and not significantly different. The one-way delay between the two slave robots is set as around 100 ms in this experiment. In the first experiment, the system performance in free motion is demonstrated. During free motion, the master manipulator is guided by the human operator in the task space and the two slave robots are coupled to the master robot using the proposed system. Figure 11 demonstrates the position synchronization performances of the proposed teleoperation system. Since the wave reflections are eliminated, the slave robots can closely track the master robot without large vibration and signals distortion. The remaining slight signal perturbations in Figure 7 are caused by the time-varying delays. The two slave robots can perform exactly the same actions during free motion. In the presence of large time-varying delays, although the dynamic models of the master and slaves are quite different and affected by uncertain parameters, both of the slave robots can reasonably track the master robot’s trajectory with little errors. The root mean square errors (RMSEs) for position tracking between every two robots in Figure 7 are shown in Table 1. Therefore, it can be concluded that the main objective is that accurate position tracking of the proposed teleoperation system is achieved.
RMSE (free motion).
Free motion
Master and Slave 1
Master and Slave 2
Slave 1 and Slave 2
Position joint 1
0.0353
0.0429
0.0465
Position joint 2
0.0434
0.0444
0.035
Position joint 3
0.0453
0.038
0.0431
Experimental setup.
Standard wave-based system in free motion.
Proposed wave-based system in free motion.
Standard wave-based system in hard contact.
Proposed wave-based system in hard contact.
Free motion.
In the next experiment, the two slave robots are driven by the master robot to draw a letter “O” and a triangle “Δ” on a table as shown in Figure 8. Friction exists between the manipulators and the table. The RMSEs for position tracking between every two robots in Figure 12 are shown in Table 2. Due to the effect of the friction, the RMSEs are larger than that of free motion. The proposed algorithm still makes all of the robots have reasonable trajectory tracking without large signals distortion.
RMSE (drawing).
Drawing a letter “O”
Master and Slave 1
Master and Slave 2
Slave 1 and Slave 2
x-axis
0.1351
0.1587
0.1265
y-axis
0.1739
0.1704
0.2302
Drawing a triangle “△”
Master and Slave 1
Master and Slave 2
Slave 1 and Slave 2
x-axis
0.1043
0.0996
0.112
y-axis
0.1539
0.1425
0.1053
Drawing a letter “O” and a triangle “Δ.”
In the next experiment, slave manipulators 1 and 2 are guided by the master manipulator to come in contact with different remote environment as shown in Figure 13. The master robot firstly drives the two slave robots to perform the free motion in the first 2 seconds. Then, from the 2nd to the 5th second, Slave 1 starts to contact with a solid wall while Slave 2 is still in free motion. Slave 1 immediately feeds the contact force back to the master robots and Slave 2. The master robot keeps applying force to the two slave robots, but Slave 2 also stops moving to make the motion synchronization with Slave 1 even when no environmental force is applied to its manipulator. In the 5th second, the solid wall is suddenly removed. It can be observed that both of the two slave robots quickly track the master robot’s position with little variation, which proves that the proposed algorithm can deal with the sudden changing environment and the wave reflections will not reinstate. The RMSEs for position tracking between every two robots and the RMSEs for force tracking between the master robot and Slave 1 in Figure 13 are shown in Tables 3 and 4.
RMSE, position (Slave 1 contacting with a reverse wall).
Contacting with a reverse wall
Master and Slave 1
Master and Slave 2
Slave 1 and Slave 2
Position joint 1
0.308
0.2709
0.0856
Position joint 2
0.2507
0.2444
0.0379
Position joint 3
0.2442
0.2378
0.0801
RMSE, force (Slave 1 contacting with a reverse wall).
Contacting with a reverse wall
Master and Slave 1
Force joint 1
0.0639
Force joint 2
0.0962
Force joint 3
0.0852
Slave 1 contacting to a reverse wall.
In the final experiment, the two slave robots are driven by the master robot to simultaneously contact with a solid wall. The position and force tracking are shown in Figure 14. Under the condition of hard contact, both of the two slave robots feed the environmental forces back to the master robots and the human operator can feel the mixed forces from the two slave robots. Figure 14 demonstrates that accurate force tracking between all of the three robots is achieved. The RMSEs of position and force tracking between every two robots are shown in Table 5.
RMSE (hard contact of the two slave robots).
Hard contact
Master and Slave 1
Master and Slave 2
Slave 1 and Slave 2
Position joint 1
0.2501
0.2510
0.0229
Position joint 2
0.2545
0.2587
0.0342
Position joint 3
0.2533
0.2549
0.0247
Force joint 1
0.0678
0.0706
0.025
Force joint 2
0.0712
0.0698
0.0496
Force joint 3
0.0831
0.0845
0.0737
Both of the two slave robots contacting to a solid wall.
6. Conclusion
In this paper, a novel wave-based control approach has been proposed for hybrid motion and force control of a multilateral teleoperation system with one-master-multiple-slave configuration in the presence of large time-varying delays in communication channels. The stability of the proposed multilateral teleoperation system in different environment scenarios is also analyzed in this paper. The feasibility of the proposed algorithm in the presence of large time-varying delays is validated using a 3-DOF nonlinear trilateral teleoperation system.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
PeninL. F.MatsumotoK.WakabayashiS.Force reflection for time-delayed teleoperation of space robots4Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '00)April 2000San Francisco, Calif, USAIEEE3120312510.1109/ROBOT.2000.8451432-s2.0-0033692444ManochaK. A.PernaleteN.DubeyR. V.Variable position mapping based assistance in teleoperation for nuclear cleanupProceedings of the IEEE International Conference on Robotics and Automation (ICRA '01)May 2001Seoul, Korea3743792-s2.0-0034865487MitsuishiM.SugitaN.PitakwatcharaP.Force-feedback augmentation modes in the laparoscopic minimally invasive telesurgical system200712444745410.1109/tmech.2007.9019682-s2.0-34548349335AndersonR. J.SpongM. W.Bilateral control of teleoperators with time delay.198934549450110.1109/9.242012-s2.0-0024663381AndersonR. J.SpongM. W.Asymptotic stability for force reflecting teleoperators with time delay199211213514910.1177/0278364992011002042-s2.0-0026842501NiemeyerG.SlotineJ.-J. E.Stable adaptive teleoperation199116115216210.1109/48.648952-s2.0-0025792942SunD.NaghdyF.DuH.Application of wave-variable control to bilateral teleoperation systems: a survey2014381123110.1016/j.arcontrol.2014.03.0022-s2.0-84900511442YokokohjiY.ImaidaT.YoshikawaT.Bilateral teleoperation under time-varying communication delay3Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '99)October 1999Kyongju, Republic of Korea185418592-s2.0-003331322510.1109/IROS.1999.811748YokokohjiY.ImaidaT.YoshikawaT.Bilateral control with energy balance monitoring under time-varying communication delayProceedings of the IEEE International Conference on Robotics and Automation (ICRA '00)April 2000268426892-s2.0-0033707419MunirS.BookW. J.Internet-based teleoperation using wave variables with prediction20027212413310.1109/TMECH.2002.10112492-s2.0-0036613169HuL.-Y.LiuX. P.LiuG.-P.The wave-variable teleoperator with improved trajectory trackingProceedings of the 8th IEEE International Conference on Control and Automation (ICCA '10)June 201032232710.1109/icca.2010.55240682-s2.0-77957884268YeY.LiuP. X.Improving haptic feedback fidelity in wave-variable-based teleoperation orientated to telemedical applications20095882847285510.1109/TIM.2009.20163682-s2.0-67949115613AziminejadA.TavakoliM.PatelR. V.MoallemM.Transparent time-delayed bilateral teleoperation using wave variables200816354855510.1109/tcst.2007.9082222-s2.0-43449085251AliseM.RobertsR. G.ReppergerD. W.MooreC. A.Jr.TosunogluS.On extending the wave variable method to multiple-DOF teleoperation systems2009141556310.1109/TMECH.2008.20061812-s2.0-61549138660NudehiS. S.MukherjeeR.GhodoussiM.A shared-control approach to haptic interface design for minimally invasive telesurgical training200513458859210.1109/tcst.2004.8431312-s2.0-22444436217SirouspourS.A control architecture for multi-master/multi-slave teleoperationProceedings of the Tenth IASTED International Conference on Robotics and ApplicationsAugust 20042212262-s2.0-10444228899DoN. D.NamerikawaT.Cooperative control based on Force-Reflection with four-channel teleoperation systemProceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC '11)December 201148794884SirouspourS.Modeling and control of cooperative teleoperation systems20052161220122510.1109/TRO.2005.8522542-s2.0-29844436472LeeD.Martinez-PalafoxO.SpongM. W.Bilateral teleoperation of multiple cooperative robots over delayed communication networks: applicationProceedings of the IEEE International Conference on Robotics and Automation (ICRA '05)April 2005IEEE36637110.1109/robot.2005.15701462-s2.0-33846166786MalyszP.SirouspourS.Cooperative teleoperation control with projective force mappingsProceedings of the IEEE Haptics SymposiumMarch 201030130810.1109/haptic.2010.54446412-s2.0-77952702695KhademianB.Hashtrudi-ZaadK.Dual-user teleoperation systems: new multilateral shared control architecture and kinesthetic performance measures201217589590610.1109/tmech.2011.21416732-s2.0-84865386744BacoccoR.MelchiorriC.A performance and stability analysis for cooperative teleoperation systemsProceedings of the 18th IFAC World CongressSeptember 2011109611012-s2.0-8486675357010.3182/20110828-6-it-1002.01348LiZ.DingL.GaoH.DuanG.SuC.-Y.Trilateral teleoperation of adaptive fuzzy force/motion control for nonlinear teleoperators with communication random delays201321461062410.1109/TFUZZ.2012.22241162-s2.0-84881348650KosugeK.IshikawaJ.FurutaK.SakaiM.Control of single-master multi-slave manipulator system using VIMProceedings of the IEEE International Conference on Robotics and AutomationMay 1990117211772-s2.0-0025658573WangY.SunF.LiuH.LiZ.Passive four-channel multilateral shared control architecture in teleoperationProceedings of the 9th IEEE International Conference on Cognitive Informatics (ICCI '10)July 201085185810.1109/coginf.2010.55997942-s2.0-78649880813CheungY.ChungJ. H.ColemanN. P.Semi-autonomous formation control of a single-master multi-slave teleoperation systemProceedings of the IEEE Symposium on Computational Intelligence in Control and Automation (CICA '09)April 200911712410.1109/cica.2009.49827922-s2.0-69549106399TumerdemU.OhnishiK.Multi-robot teleoperation under dynamically changing network topologyProceedings of the IEEE International Conference on Industrial Technology (ICIT '09)February 2009Gippsland, AustraliaIEEE1610.1109/icit.2009.49396212-s2.0-67650323832KatsuraS.OhnishiK.A realization of haptic training system by multilateral control20065361935194210.1109/tie.2006.8854652-s2.0-33947205967LeeD.SpongM. W.Bilateral teleoperation of multiple cooperative robots over delayed communication networks: theoryProceedings of the IEEE International Conference on Robotics and Automation (ICRA '05)April 200536036510.1109/robot.2005.15701452-s2.0-33846158856SpongM. W.HutchinsonS.VidyasagarM.2005New York, NY, USAJohn Wiley & SonsNuñoE.BasañezL.OrtegaR.Passivity-based control for bilateral teleoperation: a tutorial201147348549510.1016/j.automatica.2011.01.0042-s2.0-79952487829SunD.NaghdyF.DuH.Transparent four-channel bilateral control architecture using modified wave variable controllers under time delays201411710.1017/s026357471400191xChopraN.SpongM. W.LozanoR.Synchronization of bilateral teleoperators with time delay20084482142214810.1016/j.automatica.2007.12.0022-s2.0-47249115360SuzukiA.OhnishiK.Novel four-channel bilateral control design for haptic communication under time delay based on modal space analysis201321388289010.1109/TCST.2012.21893982-s2.0-84876815258KhalilH. K.2002Upper Saddle River, NJ, USAPrentice HallHashemzadehF.TavakoliM.Position and force tracking in nonlinear teleoperation systems under varying delays201411410.1017/S026357471400068X