Infinite Queue Management via Cascade Control for Industrial Routers in Smart Grid IP Networks

Smart grid applications experience an extremely wide range of communication delay. Data flows of those applications are normally aggregated at industrial network routers in substations, form infinite (long) queues termed bufferbloat issue, andmight damage the operation of transmission control protocol. The default queue management scheme, DropTail, in such routers just drops packets if queue is full while the others in literature are mostly based on one-loop feedback control where an optimal point of performance between queue length and drop rate is limited. In this paper, we study the problem of managing a long queue of industrial router at substation under heterogeneous smart grid networks. Specifically, we propose an enqueue-dequeue dropping cascade control using a two-loop design method to control both window size and queue length. Moreover, our proposal can be easily implemented into router firmware with provided discrete expressions. Finally, our simulation results are presented to validate the possible benefits that can be gained from cascade control and compare the existing queue management methods as well.


Introduction
1.1.Overview.Smart grid is going to be a geographically widespread system soon, which integrated many diverse real-time applications (e.g., sensor) together with energy management applications.Smart grid, which is a digitallyenhanced version of the traditional electric grid, provides infrastructure to support diverse services such as finance, information, and electrical delivery among consumers, assets, and those users who have authorized access.A conceptual model for smart grid communication is presented in [1].At the customer house, energy consumption information is monitored and then sent to service provider for further analysis and better support.At the operator side, data is collected to make decision according to demand response or energy saving.
With an emerging smart grid, various applications can be enabled to improve quality of service and consumer satisfaction.Some popular examples are building automation, automated meter reading, outage and restoration management, and electric vehicles.Most of the smart grid applications have a strict latency requirement in the range of 100 milliseconds to 5 seconds [2].In [3], the delay ranges of different smart grid applications were approximately reported.For example, teleprotection applications allow traffic delay from 8 to 10 ms while they allow traffic delay for more than one second for interval measurement from smart meters.To respect it, we also need a fast communication infrastructure that can handle a huge amount of exchanging data and is able to provide a near real-time response.Latency is defined as the time interval between when the state occurred and when it was acted upon by a smart grid application [2].Various applications own different latency requirements that depend on the kind of system response it is dealing with.In such scenario, network equipment including switches, Internet wired/wireless routers, and operating systems (O/S) are the fundamental components to provide a fast and reliable smart grid communication.Table 1 lists latency and bandwidth requirements for some specific applications in smart grid networks [4].
In such a heterogeneous environment with multiple types of applications, special equipment like industrial routers  at substations are necessary to handle.Commercially, we have seen an example product from the company, Virtual Access, that is, GW2027, which is an industrial router with applications like machine-to-machine (M2M), telemetry, supervisory control and data acquisition (SCADA), roadside, and wireless.Figure 1 illustrates such smart grid network situation in detail, where industrial router has to process data from both wired and wireless links.Among several great software features like remote configuration and fault reporting and so forth we, however, realize that a scheme to manage the input queue of the industrial router is not specified in the data sheet [5] and is usually Drop Tail, a simple scheme which drops packets in case of full buffer and allows packets otherwise.

Motivation.
In general, each part of every network equipment or substation industrial router is usually preinstalled or configured with some amount of buffering, whether it is enough or not, to handle bursts of arriving packets and departing packets to the next link.It is important to ensure good utilization of the network link, especially in cases where arriving rate is greater than departing rate causing bottleneck point to be built up.The buffer then absorbs high-rate traffic packets which wait and are later served on the slower outgoing links [7].Not enough buffering results in high dropping rate of most of the packets and low egress link utilization.For example, if a user transfers files using transmission control protocol (TCP), the user satisfaction is measured by how quickly the file transfer can complete, which is directly related to how effectively the protocol can utilize network links, that is, more buffering is better.Moreover, as the cost of buffers/memory keeps reducing, it is foreseen that large buffering would be put into every piece of network equipment and even preconfigured in operating systems.However, large buffering results in long latency experience for users as well.Besides, most of TCP schemes (Reno, New Reno, CUBIC, etc.) use lossbased congestion control.It means that a loss-based TCP sender continues increasing its congestion window size (i.e., sending rate) until a loss signal is detected by time-out or incorrect acknowledgment packets.In [8], it is suggested that "the widely used rule of thumb leads to buffers which are much larger than they need to be" and could be reduced by scarifying a small amount of bandwidth utilization.Using too much buffering, it obviously takes longer time for loss detection and therefore TCP may function incorrectly.The excessive long latency and the damage of TCP due to too much buffering are two major consequences of a problem recently termed as bufferbloat.
The up-to-date solutions to the bufferbloat issue, or debloat, consist of traffic shaping, TCP window size modification, and active queue management (AQM).In the scope of this paper, we concentrate on AQM algorithms because they can manage queuing delay efficiently, by either a direct or an indirect way.Most of Linux kernels and some open source routers (CeroWRT) have been integrated with some modern AQM algorithms [9,10].For example, depending on where the bottlenecks are, a Linux user can change the computer to use a different AQM qdisc (queuing discipline) via a sysctl or the tc (traffic control) command.Literature has witnessed several coexisting AQM candidate algorithms including Controlled Delay (CoDel) [11], Stochastic Flow Queuing CoDel (sfqCoDel) [12], and Proportional Integral Enhanced (PIE) [13] that have been proposed using varied theory tools (e.g., optimization, queuing, and control theory).Among them, the control theory approach owns some advantages to be implemented in smart grid network equipments, for example, parameterization controllers and well reference tracking.Theoretically, control-based AQM algorithms exploit the additive increase multiplicative decrease (AIMD) model of TCP and the continuous fluid-flow queue approximation [14,15].Then the specific controller's parameters are designed according to a closed-loop transfer function of the whole system.We, however, realize that these models almost design one-loop control for queue length only which has some problems of large overshoots and unacceptable lags (delay) in mixed-traffic scenarios.More loops operating in different network layers should improve the whole system performance, especially the input delay transient (overshoot) behaviour of the aforementioned debloat schemes.

Our Contributions.
In this paper, we develop a novel active queue management using cascade control framework in control theory that is able to reduce large overshoot and therefore improve delay transient behaviour.To the best of our knowledge, our work is the first attempt to adapt cascade control method to bufferbloat research field.Using the wellknown AIMD and fluid queue model [14], we decompose transfer functions into an inner and an outer loop.The inner one adjusts window size based on changing of traffic and feedback window size and the outer one mission adjusts queue length based on feedback queue length value at time instant.Each loop's parameters are tuned using optimal gain and phase margin method [16].This scheme operates at both the network transport layer (adjusting TCP congestion window size) and the network link layer (adjusting queue length).A difficulty when considering this cascade design is the interaction in time scale between two loops.The other side effect is complexity due to two additional controllers.Nevertheless, we show that better delay performance results can be achieved with EDDC via both numerical and simulation results (Section 5.2).Our primary contributions are summarized as follows: (i) We develop a cascade control active queue management (EDDC) scheme by decomposing the TCP/ AQM fluid-flow model into two cascaded loops for two independent controllers.
(ii) By applying the reliable optimal gain and phase margin tuning method, we derive a set of parameters for these controllers based on a given example from single-loop AQM design.
(iii) Our simulations are conducted in network simulator ns-2 using different smart grid application models, including file data transmission (FTP), voice over IP (VoIP), and video conference, which are close to the realistic smart grid applications.In most of cases, the related-delay results using EDDC are well controlled around the target delay (10 ms) while the large overshoot phenomenon disappears.

Related Works
In this section, we discuss the features of the famous existing AQM schemes: one mature (RED) and two modern AQM algorithms (CoDel, sfqCoDel) which are designed specifically for bufferbloat mitigation.Random Early Detection (RED) is one of the first viable AQM algorithms [17].However, it is proved to be so difficult to configure properly that hardly anybody uses it, even though many carrier grade routers implement it.With an appropriate set of parameters, RED is an effective algorithm.However, dynamically predicting this set of parameters was found to be difficult.As a result, RED has not been enabled by default, and its present use in the Internet is limited [18].Other AQM algorithms have been developed since RFC2309 was published, some of which are self-tuning within a range of applicability.Hence, while this memo continues to recommend to deploy AQM, it no longer recommends that RED or any other specific algorithm is used as a default; instead it provides recommendation on how to select appropriate algorithms and that a recommended algorithm is able to automate any required tuning for common deployment scenarios.
"Controlled Delay" (CoDel) AQM algorithm [11] tries to address the problems that RED could not.First, the input signal into the algorithm (sojourn time versus average queue length) is of a different quality; second, CoDel (in its plain form) does drop/mark on dequeue (departure queue), while RED drops/marks on enqueue (arrival queue).Therefore TCP congestion control loop using CoDel responds much quicker than using RED; thus the reaction by the sender will probably be timely and relevant for that congestion epoch.With RED, the congestion signal (lost packet) has to traverse the filled-up buffer first; thus, the control loop time is much larger (includes the instantaneous queue length of the buffer) and is further delayed by the averaging going on.By design, CoDel does not need to be tuned specifically for one particular drain rate (bandwidth) of the queue unlike RED; so it adjusts much better to variable bandwidth MACs (e.g., Wi-Fi and DOCSIS link).
CoDel itself, however, drops packet of a group during each interval without considering the packet's priority level or user side.Recently, Stochastic Flow Queue CoDel (sfqCoDel) [12] is a promising AQM algorithm which demonstrates a satisfied performance to mitigate bufferbloat for users, for example, increase 10x speed of a network under load [9], and it has been implemented in open source routers (i.e., OpenWRT [10]).The sfqCoDel is a hybrid of deficit roundrobin scheduling (DRR) and CoDel.It is renamed from "Fair Queuing" to "Flow Queuing" because flows that build a queue are treated differently than flows that do not.It stochastically classifies incoming packets into different queues; each queue is managed by the CoDel AQM algorithm.Packet ordering within a queue is preserved since queues have FIFO ordering [19].

TCP State-Space Model.
We exploit the TCP fluid-flow model described by nonlinear differential equations that has been extensively studied in network routers interacting with TCP sources (e.g., [14,20,21]).This model captures the additive increase multiplicative decrease (AIMD) feature from TCP [22], without slow start and time-out mechanisms.However, this lack only affects initial start-up of the TCP system.Once the system reaches the stable point, the differential equations solver is able to track changes in the network well [23].
where  is average TCP window size (packets);  is queue length at cable modems (packets); (⋅) is packet dropping probability function (0 ≤  ≤ 1);   is transmission capacity of link  (packets/sec);  is round-trip time (sec); () =   + ()/  with   being propagation delay; and  is number of TCP sessions.The first equation ( 1) describes the TCP-Reno-sender behaviour based on AIMD while the second one (2) models a fluid queue, which allows traffic arrivals to be continuous rather than discrete, as in a classic queuing model like M/M/1 Scientific Programming queue.The operating point ( 0 ,  0 ,  0 ) of model ( 1) can be derived at with ẇ = 0 and q = 0 as follows By doing linearization around operating points ( 0 ,  0 ,  0 ) [14], with  ≐  −  0 ;  ≐  −  0 ; and  ≐  −  0 , a linearized small signal model is obtained as: By defining system state vectors as () = [() ()]  ; control input vector as () = (); external disturbance as (); and system output vector as () = (), we then rewrite the TCP system as a continuous time-invariant type using state-space modeling approach.
where matrices ,   , , , and  represent state (system), input, output, and disturbance matrix, respectively, as follows: The final system model equations which map the input (traffic flow) to the output (queue length) are Using the above state-space TCP/IP system model, there are several approaches to design an efficient controller with the following requirements: (i) Queue length is as short as possible.
(ii) Sensitivity to network parameters is low.
(iii) Link utilization is high, which means we do not waste the cost of link.
(iv) Packet drop rate is low.
We, however, admit a trade-off to satisfy all of those aforementioned requirements.For example, a shorter queue length comes along with a higher packet drop rate.Our algorithm in the next section seeks for an optimal point.

Optimization Formulation of Infinite Buffer (Bufferbloat).
The classic M/M/1 queuing model well represents the infinite buffer or bufferbloat phenomenon of smart grid substation routers.Optimizing the performance of even simple queue like M/M/1 is a difficult problem because of nonlinearity of objective functions and constraints and running time scales exponentially with the problem size.In [24], a convex optimization problem was proposed for M/M/1 queues and with purpose of minimizing the state probability   only.However, minimizing queue length is the main purpose when dealing with the bufferbloat issue.Therefore, we formulate our optimization problem to directly optimizing queue length of infinite buffer, with constraints following the aforementioned TCP state-space model.( The optimization variables are  and , and the constant parameters are ,   , and .

Proposed Algorithm
In this section, we propose an algorithm, named an Enqueue-Dequeue Dropping Cascade control scheme (EDDC).We present two main design steps of our proposed algorithm from the above TCP state-space system model.The first step is to design in the continuous time domain.The second step is transform dropping probability formula into discrete domain so that we are able to implement it into the smart grid industrial router firmware (Figure 2).

EDDC in Continuous Time Domain.
We design two controllers for each loop.The inner controls drop probability  2 based on traffic information and the outer controls drop probability  1 based on difference between measured average queue length and reference queue length.
(i) Inner Loop.An important design requirement is that the inner loop controller should behave quickly.From ( 7), the inner control objective is a linear first-order type: The close-loop transfer function of the inner loop is The () transfer function is converted into frequency domain, with  as frequency: One of quality requirements of closed-loop control system, which is represented by (), is that the output is same to the input signal or the controller  win should bring |()| = 1, ∀, which can be called optimal gain and phase margin tuning method [16].However, due to several reasons of real system, that requirement is rarely satisfied for all frequencies .An acceptable design is that |()| ≈ 1, in a wide band of low frequencies .Hence, we propose to choose   win such that  2  win − 2  ⋅   win  = 0 or   win = 2  .This close-form expression of   win is used to make decision for controller  win .
(ii) Outer Loop.The close-loop transfer function of the outer loop is The outer control objective into zero-pole form is where For the outer loop, the objective function is linear thirdorder type, due to inclusion of ().Hence, we choose proportional-integral-derivative (PID) controller by using the same method at the inner loop design, or with

EDDC in Discrete Time Domain (i) Discrete Outer Loop Controller
where  1 () is the output of the outer loop controller and  1 () is the difference between measured queue length and desired queue length at the outer loop.We apply backward difference method to get the z-transform of the outer loop controller  queue (): Transforming again to time domain, we obtain or a compact form of the outer loop controller  queue (ii) Discrete Inner Loop Controller where  2 () is the output of the inner loop controller and  2 () is the difference between  1 and congestion window size ().Doing similarly as discrete outer loop steps above, we obtain a compact form of the inner loop controller  win in discrete domain as follows, with   =   /  win :

Performance Evaluation
5.1.Simulation Setup.NS2 is a discrete event network simulator, which simulates packet-level events and facilitates the development of communication network scenarios considering the new protocols involved, either wired or wireless technologies.We use and modify the NS2 TCP evaluation tool originally developed by Wang et al. [25] for our AQM algorithm evaluation purpose.We also develop our own EDDC source code files based on the basic proportional integral (PI) algorithm code from NS2 source code tree.The simulation network topology as multiple cross bottlenecks is our choice because of its realistic smart grid IP networks (Figure 3).The simulation configuration is presented in Table 2.

Simulation Results
. We evaluate our proposed algorithm performance according to three main characteristics of the   varied bandwidth because of wireless signal attenuation and fading.
Firstly, Figure 4 presents our comparison results when we change the value of link bandwidth from 1 Mbps to 1000 Mbps.Our algorithm achieves very low and stable mean queue length (≈10%); even bandwidth is varied in a wide range.Meanwhile, the link utilization of EDDC is about 95%, which is higher than the default DropTail or RED and nearly approaches the modern sfqCoDel algorithm (96%).The packet drop rate performance of EDDC, however, is almost the same as the others.It makes sense that when we use a low-bandwidth link, the packet drop rate is high due to full buffer phenomenon.We conclude that our EDDC algorithm promisingly maintain the queue length of substation router, while link utilization or the busy level of link is acceptable under different kinds of links.

Experiment 2:
Varied Link Round-Trip Time. Figure 5 shows our comparison of AQM algorithms performance in case of varied link propagation round-trip time (RTT).This criterion represents the latency of packets when using a lossbased TCP congestion control scheme, for example, TCP Reno.As mentioned before, one consequence of a large buffer (bufferbloat) is that TCP operation might be damaged because of untimely congestion information feedback of acknowledgment (ACK) packets.The higher the propagation RTT is, the higher chance of incorrect ACK feedback to the sources is.We observe the same results of EDDC performance for the mean queue length and link utilization.In Figure 5(c), the packet drop rate results of EDDC, however, are higher than the other AQM algorithms in the RTT range from 1 ms to 100 ms.

Figure 1 :
Figure 1: Long input queue at industrial router at substation.

Figure 2 :
Figure 2: Proposed control framework for queue management in industrial router.

Figure 3 :
Figure 3: Cross-traffic simulation topology for smart grid applications.

Table 1 :
A wide range of delays for some smart grid applications.

Table 2 :
Simulation configuration.  ,   ,   , and 5.2.1.Experiment 1:Varied Link Bandwidth.In the first experiment, we vary the bottleneck link bandwidth from 1 Mbps to 1000 Mbps.This variation represents different types of links in heterogeneous smart grid networks.For example, a wired link such as Ethernet has a bandwidth of 100 Mbps while a wireless link from wireless adapter has