This paper addresses the optimal bandwidth scheduling problem for a doublelayer networked learning control system (NLCS). To deal with this issue, auction mechanism is employed, and a dynamic bandwidth scheduling methodology is proposed to allocate the bandwidth for each subsystem. A noncooperative game fairness model is formulated, and the utility function of subsystems is designed. Under this framework, estimation of distribution algorithm (EDA) is used to obtain Nash equilibrium for NLCS. Finally, simulation and experimental results are given to demonstrate the effectiveness of the proposed approach.
Networked control systems (NCSs) are the multiple feedback control loops closed via a serial communication channel. Compared with the traditional pointtopoint control system, the advantages of NCSs are sharing of information resources, powerful system diagnosis, distributed remote control, modular design, configuration flexibility, and low cost [
It is notable that most of the aforementioned researches are focused on singlelayer network structure; few results have been reported on NCSs with doublelayer structure. As pointed out by [
We know that if each node is allowed to occupy network resource as much as possible according to its own requirements, the overall system performance will be very poor [
The remainder of this paper is organized as follows: Section
A typical doublelayer NLCS structure is shown in Figure
The structure of doublelayer NLCS.
The timing diagram of bandwidth allocation is illustrated in Figure
Timing diagram of bandwidth allocation.
The dynamics of the remote controlled plant is given by the following linear model:
The electromechanical dynamics of the networked dcmotor subsystem used in this paper can be described as
The parameters of the dcmotor subsystem.

Motor winding resistance 


Motor winding inductance 


BackEMF constant 


Electric torque constant 


Motor moment of inertia 


Damping coefficient 

By letting
The performance of the NLCS can be demonstrated jointly by the subsystems, and the performance of the subsystem can be described by a function, such as the integral absolute error (IAE) [
Sampling periods versus IAE.
For each control loop, the relation between sampling period
On one hand, the QoC of each subsystem should be improved, on the other hand, the demands of network bandwidth of each subsystem should be reduced to provide more resources for extra subsystems. Due to the bandwidth limitations, a multiobjective optimization problem can be expressed as follows
The game model based on auction mechanism is discussed in this section. Auction theory is first proposed by Vickrey in 1961, which mainly includes four basic types: English auction (ascendingbid auction), Dutch Auction (descendingbid auction), and first price auction and second price auction (Vickrey auction). The highest offer will win the bid no matter what type of auction is used, and the optimum outcome is that the price exactly equals what the second highest bidder can afford.
It is worth noting that the first two kinds of auction mechanism are bidding open, and the others are sealed. In sealed auction, each bidder submits his price without any information of others, then the auctioneer announces the winner who offers the highest bid. Both of them are suitable for bandwidth allocation in NLCS. The only difference between first and second price auction is the actual price that the winner paid. For second price auction, the winner just needs to pay the second highest bid of others instead of his own. For ease of operation, first price auction is applied in this paper.
The closedloop control subsystem is modeled as a player. Every player is not explicitly aware of the existence of other players and their status. Each player puts forward its own bandwidth strategy denoted as
Every player has the same amount of money
Every player submits the price
ULN will run the bandwidth allocation programs based on the bidding prices and allocate the bandwidth resources to each player.
Every player will pay for the bandwidth they have got.
The price of bandwidth
Further, the revenue function based on auction theory and IAE evaluation method is shown as (
Obviously, the revenue of the player is determined by the received QoC and the payment under certain QoS. Every player will make the bidding strategy based on their own revenue and will not increase the price aimlessly. If a player overly spends its money for a much larger bandwidth than its actual need, it will only get a low payoff due to the reduced network utility. The money each player has to pay is based on the bandwidth they have got, which may not match the initial offer.
Every player in LLN wants to maximize their revenue under the framework of NLCS. ULN will make a network resources allocation strategy which makes it impossible to get more bandwidth resources through changing the offer. Generally speaking, the network utility of a single node depends on the scheduling strategy of others. If there is no node chooses other scheduling strategies when the scheduling strategies of other nodes that are decided, this equilibrium will not be broken in the network. This equilibrium is called Nash equilibrium (NE) [
To address the modelling problem, we introduce the following definitions to prove the existence and uniqueness of Nash equilibrium in NLCS.
A noncooperative networked learning control game (NNLCG) with
The function
A Nash equilibrium exists in game
Theorem
A Nash equilibrium exists in the NNLCG,
By Theorem
The firstorder differential of the revenue function
The secondorder differential of the revenue function
Known from the above equations, the revenue function is continuous and differentiable in
The NNLCG has a unique equilibrium.
The proof of the unique equilibrium can be carried out by following similar lines as in the proof of Theorem 2 in [
Up to now, the problem of network resources allocation under a general framework of doublelayer NLCS has be changed into the problem of Nashequilibriumpoint solving in the noncooperation game model. It is hard to solve the Nash equilibrium point by the traditional numerical method. So, estimation of distribution algorithm (EDA) is introduced in this paper to find the Nash equilibrium point. The EDA algorithm used in this paper is described as follows.
Initialization: generate gain candidates meeting the ratio of bandwidth (RoB) randomly to form an initial population.
Repeat the following steps until the termination criterion is met.
Selection: select the best gain candidates from the parent generation.
Updating: update using the selected promising gain candidates.
Sampling: generate gain candidates meeting RoB based on the updated; copy the best gain candidate in the current population to the next population.
For more details about EDA, please refer to [
In this section, an illustrative example is presented to show the effectiveness of the proposed method. To this end, let us consider a doublelayer NLCS as shown in Figure
Parameters of the rest two subsystems.
Parameter A  Parameter B  

Loop 2 


Loop 3 


The auctionbased bandwidth allocation (ABA) has an initial population of 100 solutions and ten generations. When the overall fitness value is stabilized, the Nash equilibrium point is reached. The bandwidth vector allocated by the ULN in this case is
Simulation results of comparison test.
EBA  ABA  

IAE  RoB  IAE  RoB  
Loop 1 




Loop 2 




Loop 3 




 
Total 




Different step responses of three networked dcmotor subsystems.
Loop 1
Loop 2
Loop 3
As shown in Table
Figure
Statistics of the 30 simulation results.
EBA  ABA  

Mean value 


Standard deviation 


The boxandwhisker diagram.
Motivated by the results, we found that auctionbased bandwidth allocation that optimizes resource scheduling strategy can effectively meet the desired objectives in the resourceconstrained NLCS.
This paper presents a noncooperation game model based on Nash theory and auction mechanism for bandwidth allocation in NLCS with limited resources. And the estimation of distribution algorithm is introduced to solve the problem effectively. The proposed method forces all players sharing the same network to have allocated bandwidths at Nash equilibrium point. Network resources are allocated in the optimum way to reduce delays and packet losses, and the overall performance of systems with communication constraints is significantly improved. The simulation and experiment results indicate the effectiveness and availability of the proposed approach.
The authors would like to thank C. Hao, D. W. Yang, and T. Tony for their valuable comments and suggestions to improve the quality of the paper. The work of X. Yan is supported by the Fundamental Research Funds for the Central Universities under Grant E11JB00310. The work of L. D. Wang and P. Shen is supported by the Fundamental Research Funds for the Central Universities under Grant E12JB00140. The work of H. Li is supported by National Basic Research Program of China (973 Program) under Grant 2012CB821206, the National Natural Science Foundation of China under Grant 61004021, and Beijing Natural Science Foundation (4122037).