The Lyapunov Optimization for Two-Tier Hierarchical-Based MAC in Cloud Robotics

,


Introduction
Robotics is evolving from a single platform to cluster collaboration for the applications with high complexity, uncertainty, and real time. Although robots share information and cooperate with each other by a decentralized wireless network, the intelligence is still limited by their own hardware configuration. Cloud robotics [1] provides a novel concept to break the individual limitation in low intelligence and processing capacity. By means of the offloading task to cloud dynamically, robots get rid of the huge storage and intensive computing bottleneck.
As shown in Figure 1, the offloading task is divided into a series of flows and is processed on cloud hosts in parallel by virtualization technology. Task flows are then repacked by upper layers with header information and finally encapsulated into MAC frames in the data link. For enhancing the survival ability in complex environment, robots adopt a decentralized ad hoc network and contend for communication channel in the carrier sense multiple access/collision avoidance (CSMA/CA) mode.
As the competition is inevitable, how to allocate the limited resource to guarantee the QoS performance and maximize the total system utility is a big challenge for the offloading scenario. Frame differentiation is easily thought of. For example, simultaneous localization and mapping (SLAM) should enjoy a better real-time than environment monitoring application. However, robots themselves also differ in processing capacity, hardware configuration, task emergency, and so on which is often overlooked. Meanwhile, the network condition is time varying which is hard to predict. When the network is overloading, the system cannot support the QoS for all robots. The data arrival rate should be adjusted to avoid blocking.
For the problems above, the paper proposes a two-tier hierarchical-based MAC (Two-Tier MAC) to realize the classification on both frames and robots. The Lyapunov optimization technique as well as isolated time slots and admission control is adopted to maximize the system utility even in poor network condition. The main contributions are as follows: (i) The classification on both frames and robots is considered. All frames are divided into different priorities waiting in corresponding queues with isolated time slots to reduce collision consumption. Frames in the same queue but from different robots are allocated diverse accessing probability to ensure the energy balance (ii) The system optimization can be decoupled into two independent issues by the Lyapunov method. One is admission control (AC) in which the frames will be rejected when the network is overloading. The other is resource allocation (RA) to adjust the accessing probability to maximize the system utility function The rest of the paper is organized as follows. Chapter II reviews related works. In Chapter III, the scheme of the Two-Tier MAC is detailed. The Lyapunov optimization method is described in Chapter IV. In Chapter V, a series of experiments are operated to test the effectiveness of the Two-Tier MAC. The conclusion and future work are illustrated in Chapter VI.

Related Works
Since the widespread of heterogeneous, crosslayered "thin robots," it is the best way to control the network transmission in the MAC layer or transceivers in the physics layer.
Some researches based on multiqueuing architecture have realized QoS support by assigning different queues with different resource or CSMA/CA parameters, such as contention window size, interframe space, active time, transmission power, TCP congestion window, and data rate [2,3]. In specific, QoS-based MAC [2] adjusts the contention window for diverse QoS requirements and duty cycle to preserve sensory energy. PRIN-MAC [4] can minimize energy consumption and increase system throughput by varying the interarrival time according to the packets priorities. ASMAC [5] avoids overhearing and reduces contention and delay by asynchronously scheduling the wake-up time of neighboring nodes. RF-MAC [6] focuses on how the placement, chosen frequency, and number of RF energy transmitters impact the sensor charging time. CACC-MAC [7] is proposed for crosslayer congestion control by the dynamic TCP congestion window and MAC contention window scheme. The mechanism proposed in [3] operates QoS provisioning at the MAC layer for urgent traffic and provides data rate adjustment at the application layer. Each algorithm can improve QoS performance in certain scenarios. However, they are verified from the view of system realization or protocol design without theory analysis such as system stability and dynamic performance.
Based on this problem, we have tried to apply control theory in this area in recent years. FD-MAC [8] designs a less-step controller to provide proportional delay differentiated (PDD) QoS support. It adopts a linear model for system identification which is not precisely enough and without taking energy consumption into consideration. On this basis, CSFD-MAC [9] proposes a cascade selftuning architecture as well as active power management, and MQEB-MAC [10] adopts a back propagating (BP) neural network to develop an all-in-one mixed QoS  2 Wireless Communications and Mobile Computing insurance and energy-balanced scheme. Although the algorithms above are verified to be valid by experiments, they still have the following limitations: Firstly, they only focus on the difference in frames but neglect the heterogeneity of the robots themselves. Secondly, channel condition is time varying, but few papers notice the system performance under poor channel conditions which is actually a common scenario in the wireless network. Thirdly, the mathematical model above is either too simple such as the linear system or low in real time such as the BP neural network.

System Model
3.1. Two-Tier Hierarchical Architecture. Figure 1 shows the Two-Tier MAC architecture for cloud robotics, and Table 1 exhibits all the variables present in the paper. There are M robots, and all frames are divided into N types waiting in the isolated queues without interference. Note that blocks with the same color come from the same robot, while blocks fulfilled with the same shape have the same frame priority.
To avoid blocking, the arrival rate should be adjusted according to the channel condition. For example, if one queue is fulfilled, the robot should reject new corresponding arrived frames. Queues will be allocated different time slots according to the frame priority, and frames in the same queue are allocated diverse accessing probability according to robot classification.

Time Slot Isolation.
Actually, the robots are power driven related to system lifetime. Isolated time slots are adopted to reduce the collision energy consumption. Frames contend for communication channel by the CSMA/CA mode, with the key algorithm that once the collision is detected, the node will wait for a random back off in the range of ½0, CWðC W min ≤ CW ≤ CW max Þ where CW is called the contention window. The more intensive a robot attempts to transmit frames, the higher the probability collision and retransmission occur, which will not only cause the unnecessary energy wasting but also induce a large end-to-end delay. Define ρ to be the accessing probability, then ρ = 2 CW + 1 : ð1Þ Figure 1 also shows a dynamic time slot isolation framework. There are S time slots in total, and s n means the number of time slots allocated to the n th queue (∑s n ≤ S, n = 1, 2, ⋯N). Frames with the same priority content for channel medium in specified time slots are actually a kind of resource reservation mechanism to avoid the across-class contention. Data arrival rate of robots is independently and identically distributed (i.i.d). Without loss of generality, assume the data priority is prefixed or dynamically negotiated by high level protocol, and robots are classified by battery capacity.

Utility Function.
In the n th queue, suppose that the arrival rate and the admission rate are λ n ðtÞ and R n ðtÞ, respectively. Although each frame expects a higher admission rate, it is still limited by data arrival rate, i.e., It is known that user experience and satisfaction follow logarithmic laws [11]. Since the normalized positive weight vector ½w 1 , w 2 ,⋯w n T ð0 ≤ w n ≤ 1,∑w n = 1Þ is prefixed by a high level protocol, the utility function for the n th queue can be written as 3.4. Queuing Model. Define Q n ðtÞ to be the data backlog (number of frames waiting in the queue) for the n th queue at time t. The queue grows with the admission rate R n ðtÞ and gets short with the service rate μ n ðtÞ. According to the queue theory [12], the recursive formula of queue length is As mentioned before, frames are with the same priority content for the channel medium in specified time slots in the manner of CSMA/CA. So the service rate mainly depends on both the time slots and accessing probability. For every robot, supposing PHY transmission ratio is v m ðn = 1, 2,⋯MÞ, the class factor is τ m , then the service rate is where ρ mn is the accessing probability of the n th frame type in robot m. So there is According to paper [13], an individual queue Q n ðtÞ is the mean-rate-stable if and a network is stable if all individual queues in the network are stable. In a real system, the mean rate stability implies that the long-term average departure rate from the queue is greater than or equal to the long-term average input rate injected into the queue, i.e., lim T→∞ ð1/TÞ Thus, all frames placed in the queue will be finally transmitted when the data queue is mean-rate-stable.
3.5. Problem Formulation. In summary, the utility function can be optimized by adjusting the time slots s n and accessing the probability ρ mn dynamically. The problem can be described as lim where RðtÞ = fR n ðtÞg, sðtÞ = fs n ðtÞg, and ρðtÞ = fρ mn ðtÞg are the optimized parameter vector/matrix to be solved. C1 constrains the admission rate which cannot exceed the arrival rate. C2 constrains the accessing probability which is in the range of [0,1], and the total value of different frame priorities in one robot at a time should equal to 1. C3 constrains the time slots which are limited by an upper bound for energy consumption, and C4 constrains the queue which is mean-rate-stable.

Lyapunov Optimization
4.1. Problem Transformation. The queue system should be stable, so the Lyapunov method is used to ensure the stability in the long term. Define the Lyapunov function as Without loss of generality, assume all queues are empty when t = 0, i.e., L½Gð0Þ = 0. Lyapunov function drift is The drift-minus-reward term has an upper bound according to Lemma 1.
Lemma 1 (see [14]). For any nonnegative real number Q, b, and a, there holds that where B = Ef∑ N n=1 ½R 2 n ðtÞ + μ 2 n ðtÞ/2g is defined as a finite constant to simplify the inequation above, and V is a nonnegative constant parameter that controls the trade-off between drift ΔLðtÞ and satisfaction function. By the Lyapunov optimization technique, problem P1 can be transformed into minimizing the right-hand side (RHS) of (12) subjected to constraints C1-C4.
Furthermore, according to Equation (12), P1 can be decoupled into two subproblems: the first one is the admission control (AC) related to RðtÞ, which means the new frames will be rejected if the queue is full. The second one is the resource allocation (RA) related to sðtÞ and ρðtÞ to guarantee the higher priority could enjoy a better QoS performance.

Admission Control.
Due to the resource limitation, admission control is adopted to ensure that the QoS performance will not degrade below an acceptable level even in 4 Wireless Communications and Mobile Computing the overloading case. Based on Equation (12), the AC problem is formulated as P2 : min Easily verify that P2 is a convex optimization problem. Set the derivative of Q n ðtÞR n ðtÞ − Vw n log 2 ½R n ðtÞ with respect to R n ðtÞ to be 0 to solve the service rate:

Wireless Communications and Mobile Computing
Since the admission rate is also limited by arrival rate, the optimal AC decision is 4.3. Resource Allocation. For the RA problem, both the time slots and accessing probability should be optimized to maximize P3: P3 : max P3 is nonconvex and may have several local minima, which make it different to find the closed-form solution. In recent years, some heuristic algorithms have been applied and proven to be valid. In this paper, the genetic algorithm (GA) as shown in Algorithm 1 is used for it can jump out of local minimal points and obtain the global optimization by finding a better artificial population iteratively.
For the unified value range of variables, take s n ′ = s n /S to replace original s n : The fitness function is denoted as Γ = Ef∑ N n=1 Q n ðtÞv m ρ mn s n ′ g. Since ρ ∈ R M×N , s ′ ∈ R N are interpreted into a binary chromosome by a 16-bit Gray code, each individual can be represented by a ð16MN + 16NÞ-bit-long binary vector, i.e., 16×N . The coding and decoding functions used in Algorithm 1 are defined as I binary = F code ðρ, s ′ Þ and ðρ, s ′ Þ = F decode ðI binary Þ, respectively. The superscript g denotes the number of generations. The output of GA is ½ρ, s ′ , and ρ can be adjusted by CW according to Equation (1).
Since P3 is nonconvex, it is hard to prove the convergence in a mathematical way. However, as shown in Figure 2, the GA algorithm can converge with a finite number of generations by simulation with different crossover probability p c and mutation probability p m .

Experiments and Results
The experiments are operated by ZigBit TM 900 hardware module with Atmel R AVR2025 software package. Specifically, ZigBit TM 900 is a 784/868/915 MHz IEEE 802.15.4 OEM module, while AVR2025 is a configurable MAC stack for ZigBit 900, which provides the fundamental abstract methods for hardware operation and a secondary development supported the MAC stack. 30 robots classified into 3 groups according to their battery configuration are deployed in the radius of 100 m. Every node randomly sends data packets to the other. Packets are encapsulated into MAC frames with 3 different priorities which are preset by upper layers as ω 1 : w 2 : w 3 = 3 : 2 : 1. A larger value means a better service.
The interval of frames obeys the normal distribution with the average of − T t /log ð1 − KÞ, where Kð0 < K < 1Þ is the offered traffic normalized by service data rate. The frame length follows the Pareto distribution with the shape parameter of 1.1 and average of 105 × 8 bits. The transmitted power is 1 mW, CW min = 23, CW max = 28. The average frame length is 105 bytes and the symbol rate is 256 kbps.

Static Performance.
Static performance concerns the throughput, power consumption, and energy efficiency changing with time, which are related to system utility, lifetime, and data transmission efficiency, respectively. Energy efficiency is defines as (2) During 200-400 s, FD-MAC operates for frame differentiation. However, the throughput remains nearly the same, and the presented energy efficiency drops slightly. That is because in channel contention, energy is consumed once a node transmits whether or not the data is correctly received. FDMAC which makes a small CW size with high priority increases    Notice that for a specific MAC protocol, frames with high priority enjoy a large throughput but at the same time will lead to more power consumption. So no matter which frame priority, there is no big difference in energy efficiency of the same protocol (Figure 3(c)). The phenomenon will also be present in Section 5.2. Figure 4 shows the statistic performance of the Two-Tier MAC on each robot. 30 robots are classified into 3 groups according to their battery configuration as shown in Table 2, and the factor is τ 1 : τ 2 : τ 3 = 4 : 3 : 2.

Statistic Performance.
For one robot itself, the frames with different priorities are distinguished. Frames with a higher priority enjoy a better throughput which is in accordance with the results shown in Figure 3.
Among different robots, the ones in the higher class still have a better throughput statistically (Figure 3(a)) but lead to more power consumption (Figure 3(b)). As a result, no matter which frame priority or robot class, the energy efficiency is always approximately equal (Figure 3(c)). This conclusion is greatly meaningful to the system lifetime discussed below.
Define the system lifetime as the moment when the first robot is out of battery. The lifetime in different protocols is shown in Table 3. Obviously, the lifetime in the Two-Tier MAC will be extended to take energy balance into consideration.

Dynamic Performance.
The experiments above only consider the normal load scene. However, traffic burst is more common in cloud robotics. Here, we focus on the effect of varying arrival rates to the system performance, i.e., dynamic performance, especially in the overloading situation. The parameter setting is the same as the former except that the arrival rate changes in the range of 50-350 frame/s. As shown in Figure 5, when the system capacity is unsaturated (50-250 frame/s), the throughput is proportional to the growth of the arrival rate for all protocols. The features of power consumption and energy efficiency present similar characteristic with Section 5.1. After that, robots suffer from traffic bursting which leads to distinct performance in the four protocols.
For 802.15.4, FD-MAC, and MQEB-MAC, network congestion occurs, and the system performance is reduced because of the noncongesting protection mechanism. Data loss will consume extra energy and lead to the degradation of throughput and energy efficiency. On the contrary, thanks to the admission control, the Two-Tier MAC could reject new frames when the network is overloading to avoid network congestion. So the energy efficiency can be maintained even in this severe network environment.

Conclusion
Considering the heterogeneity on both robots and frames, a two-tier hierarchical-based MAC is proposed to guarantee the differentiated QoS performance and maximize the system throughput. Experiments show that compared with IEEE 802.15.4, FD-MAC, and MQEB-MAC, the Two-Tier MAC can improve the overall throughput by the Lyapunov theory, reduce power consumption by the time slot isolation scheme, and expend system lifetime by the energy balance. Even in the overloading condition, the QoS performance can be guaranteed thanks to the admission control.
In the future, we would like to do more work to find the close-formed solution for the optimized problem instead of the heuristic algorithm. In addition, in this paper, all robots operate task offloading to the central cloud which suffers from large end-to-end delay. Along the development of 5G, offloading in the distributed mobile edge computing (MEC) scenario should be studied in the future.

Data Availability
The data used to support the findings of this study are included within the article.