SUBOPTIMAL FEEDBACK CONTROL OF TCP FLOWS IN COMPUTER NETWORK USING RANDOM EARLY DISCARD (RED) MECHANISM

We consider a dynamic model that simulates the interaction of TCP sources with active queue management system (AQM). We propose a modified version of an earlier dynamic model called RED. This is governed by a system of stochastic differential equations driven by a doubly stochastic point process with intensity as the control. The feedback control law proposed observes the router (queue) status and controls the intensity by sending congestion signals (warnings) to the sources for adjustment of their transmission rates. The (feedback) control laws used are of polynomial type (including linear) with adjustable coefficients. They are optimized by use of genetic algorithm (GA) and random recursive search (RRS) technique. The numerical results demonstrate that the proposed model and the method can improve the system performance significantly.


Introduction
Congestion is one of the most significant problems in Internet traffic engineering. To avoid congestion collapse, the Internet Engineering Task Force (IETF) recommended deployment of active queue management (AQM) in routers [2] in 1998. Random early detection (RED), a widely used congestion avoidance mechanism originally proposed in 1993 [7], has proved to be capable of detecting the inception of congestion and avoiding global synchronization. Well-tuned routers can improve queuing delay and provide high throughput.
As illustrated in left side of Figure 1.1, the original RED algorithm has four parameters: min th , max th , max p , and α [7]. min th and max th are defined as two-packet dropping thresholds. The average queue size q a is calculated by applying a lowpass filter (exponentially weighted moving average). No packets are dropped if q a is less than min th . For q a between min th and max th , the packet dropping probability is increased linearly with q a up to max p . When q a exceeds max th , all arriving packets are dropped. The first three parameters determine the dropping decision: when and how aggressively to drop. The last parameter α is the weight given to the current value for average queue size estimation. It controls the cutoff frequency of the lowpass filter, which provides a tradeoff for detecting manageable traffic variations and filtering out transient traffic oscillations [11]. By invoking the TCP congestion avoidance mechanisms, TCP traffic sources respond to packet drops by reducing window sizes as dictated by the router congestion as shown in right side of Figure 1.1. However, RED's parameters are sensitive to traffic load; network operators are required to constantly tune those parameters based on current network conditions. Multiplicity of parameters (here 4) makes adjustment of configuration complicated.
Recently, several algorithms have been proposed to reduce the number of RED's parameters, and adjust the parameters in varying conditions [4,5,6]. Considering largedelay variation, and throughput sensitivity to traffic load, and parameter setting, Floyd et al. [6] suggested automatic adjustment of max p value in order to keep the average queue size q a around (min th +max th )/2. The parameter min th was set as a function of the link bandwidth and max th was chosen as 3 times of min th . Determination of the values of min th and max th is an open question. Cnodder et al. [4] proposed a rate-based RED algorithm which reduced the number of parameters to one. It considers the packet dropping rate as a function of the long-term average arrival rate and the average queue size. However, it has two weaknesses. (1) It may drop packets when the queue is almost empty, which is not necessary. (2) The estimated traffic which is used for calculation of drop probability is based on an exponential function. Recent studies show that network traffic has long-range dependence property [3,8,10], and therefore the estimates based on short-range assumptions are not accurate. To analyze an active queue management system, Poisson-process-driven stochastic differential equations (SDEs) are used to model the behaviors of TCP flows and queues of routers [9]. The methodology and model are helpful for analyzing the dynamics of AQM. However, the approximations, based on assumptions such as E{1/R(q)} = E{(1/R(E(q)))}, (where R is the round trip time, q is current queue size) are seriously questionable. In this paper, we propose a revised model based on the earlier model. We replace the fixed propagation time a i by a nonnegative random variable uniformly distributed over the range of values it may possibly take. For the flow dynamics (window size), we use doubly stochastic Poisson-driven stochastic differential equation where the rate (intensity) λ is an appropriate function of the multiplexor (router load) q size. We introduce an objective functional which includes throughput and congestion and then minimize this functional by an appropriate choice of the rate functional as mentioned above. By numerical simulation, we demonstrate how effective our proposed technique is. Numerical computations are based on genetic algorithm (GA) and recursive random search (RRS) algorithms [11].
The rest of the paper is organized as follows. In Section 2, system model is presented. Objective function and state-space formulation are given in Section 3. We introduce system model with delays in Section 4. Numerical results are presented in Section 5. The paper ends with conclusion in Section 6.

System model
To present the system model, we frequently make use of indicator functions. Let S denote any logical or mathematical statement and define the indicator function of S as follows: The basic philosophy behind the TCP congestion control algorithm constitutes additive increase and multiplicative decrease (AIMD) of window size. For each round-trip time, the system increases the window size by one packet while, if a packet is lost, it halves the window size. For simplicity, here we ignore the slow start and retransmit timer mechanisms. Let n − 1 denote the number of TCP flows connected to the router. The dynamics of the TCP congestion control system can then be described in terms of window size of the sources (users) and q size of the router. Note that the larger the window size is the larger TCP flow rate is. The window size is governed by the following equation: where R i (q(t)) is the round-trip time dependent on the router q size, w i (t) is the window size, and the process N λ i (t) represents the number of packets dropped from the stream i over the time interval [0, t]. The round-trip time is generally given by the following expression: where a i denotes the round-trip propagation time between the source i and the router in case of no congestion. Since the round-trip routes may vary, this is considered to be a random variable with certain distribution discussed later. The second term in this expression represents additional time required for a complete round trip if the router is not free. In summary, the first term on the right-hand side of (2.2) gives the window's opening rate and the second the closing rate.
The router queue dynamics can be described as follows: where q(t) is the current queue size, C is the channel (or link) capacity. The first term on the right-hand side of (2.4) represents the service rate and the second term measures the traffic arrival rate at the router (from all the sources). The model presented above is a modified version of a similar model proposed in [9]. The process N λ i for each i = 1,2,...,n − 1 is a (nondecreasing) counting process with intensity process λ which is a nonnegative function of the current q size at the router. Since q is a random process, N λ i is a doubly stochastic counting process. The variable λ can be chosen by the system designer as follows: where f is a suitable nonnegative, nondecreasing function with the domain, range, and the boundary conditions as follow: with γ being equal to the reciprocal of the smallest propagation time. One possible choice of the feedback policy f is given by is a polynomial of degree n with a i , i = 1,...,n, being real-valued coefficients. Physical interpretation of λ is simple. It denotes the mean rate of congestion warnings sent out to the users (sources). The parameter Q is the size of the (router) multiplexor (defining it's storage limit). The function f defines the packet dropping scheme (programme). This is the function that can be chosen by the network designer. Once this is chosen, (2.5) determines the intensity, and hence the dropping rate. When the current queue size is less than q p , λ=0, no packet is dropped. Our feedback model, once optimized, will automatically determine q p from the first (positive) root of g on the interval [0, Q]. For numerical simulation using our model, we assume that N λ i (t), t ≥ 0, is a conditional Poisson process, conditioned on λ (or doubly stochastic Poisson process). In general, λ should be a function of the current queue size q(t) as indicated above. But since the frequency of variation of q can be very large forcing frequent changes of the window size leading to oscillation (and instability), it is desirable to use a filtered version of this. This can be partially achieved by using the moving average of q denoted by q a which is determined by the following equation: for some κ ∈ [0,1]. The parameter κ is chosen sufficiently small to avoid unnecessary frequent changes in congestion warnings. This frequency cannot exceed, in fact should be sufficiently smaller than, the frequency of acknowledgement (ACK) flows.

Objective function and state-space formulation
For evaluation of network performance over the running period I ≡ [0,T], the objective functional can be chosen as where the set Q α ≡ [αQ,Q] and E{z} denotes the expected value of the random variable z. The first term within the expectation gives the throughput of the router with a negative sign. The second term is a measure of loss due to congestion at the router. The set Q α with α ∈ (0,1] defines the congestion zone. The functions λ i (t), i = 1,2, are nonnegative measurable functions used as relative weights (or importance) given to each of the costs. These can be chosen by network designers to reflect different concerns and scenarios as necessary. Once these are chosen and fixed, one can then attempt to choose the dropping strategy f to minimize the cost functional J. The optimal strategy guarantees maximum expected (average) throughput and minimum expected losses due to jitter and congestion. For further analysis, it is convenient to write the state-space model for the system. Denoting the state of the system by ξ ≡ (w 1 ,w 2 ,...,w n−1 , q) , one can write the system in the state-space form as follows: where F : R n → R n is given by 482 Suboptimal feedback control of TCP flows The function G(ξ) is an n × (n − 1) matrix with entries given by Since λ is determined by the choice of f , we may rewrite the system (3.2) as where f is the packet dropping scheme to be determined for optimum performance. Clearly, the cost functional (3.2) can be compactly written as where denotes the integrand of the expression (3.1). One of the objectives of the network provider is to improve the system performance by using strategies that minimize this cost functional. Our principal objective is to determine the best feedback policy f (·), from the class of polynomial functions modulated by indicator functions as shown in the expression (2.7), that minimizes the cost functional (3.1) or equivalently (3.6). Our approach does not need a priori specification of the threshold q p at which packet drops are initiated. We let the optimization process determine this through the choice of the coefficients of the polynomial g which determine the roots. Due to the presence of indicator functions, the cost function versus the coefficients of the polynomial g is not smooth. Most traditional optimization methods require computation of gradients. In a nonsmooth situation like we have, those methods are difficult to implement and tend to get trapped in local minima. Random recursive search (RRS) algorithm appears to be a good choice to solve this kind of problems. In order to verify the simulation results, we use two different optimization techniques: GA and RRS. The numerical results based on these two techniques are very close. However, the computation time with RRS is approximately 50 percent of that of GA.

System model with delays
The models presented above do not take into account the time delay due to information exchange between the router and the users. Assuming an average delay of δ units of time, the models can be rewritten as follows: In fact, the communication delay is a function of several factors such as time, the physical distance between sources and router, and traffic condition (router queue). If one wants to be exact, the problem becomes mathematically intractable. Therefore, for simplicity one may view δ as the mean of a random variable having uniform distribution (considering the worst case/maximum entropy) with a compact support D 0.

Numerical results
For numerical simulation, we consider a system comprised of three TCP traffic sources connected to one router. The observation (measurement) time interval at router side is taken as 10 milliseconds, the (moving average) factor κ is set as 0.02, the maximum queue size Q = 200Kb, and the set D is taken as D ≡ [10ms,200ms]. This range is based on recent measurement studies [1]. We use RRS to optimize the cost functional (3.1) with feedback policy given by (2.7). For illustration, we choose only polynomials of degree 3, 2, and 1 (linear) in that order. Figures 5.1, 5.2, 5.3 show the values of λ as functions of average queue size based on the optimized feedback law f associated with the polynomials as mentioned above. More precisely, we choose any given polynomial and start with an arbitrary set of coefficients and substitute the corresponding f , and hence λ ≡ f (·) given by (2.5), in the system dynamics (2.2) and (2.4). Then we minimize the cost functional (3.1) using RRS algorithm by iteratively choosing the coefficients. In all of the three cases, the system does not start dropping packets until the queue size has increased to about 125Kb, approximately 62 percent of the router size. Dropping packets too early can reduce throughput even though there is no noticeable trend towards congestion. Comparing Figures 5.1, 5.2, 5.3 controls based on polynomials of degree 2 and 3 lead to global traffic synchronization which is not desirable. The reason is that, for degrees 2 and 3, with the increase of the queue size, the λ values increase much faster than the linear case. This generates (warning) signals too frequently, forcing the sources to cut their window sizes leading to reduced throughput and global synchronization. In fact, when the queue size starts to build up, reducing the traffic rates of one or two users will relieve the congestion while maintaining high level of throughput.
To compare the results corresponding to the linear case with those based on the polynomials of degree 2 and 3, we normalize the maximum values f (Q) (associated with quadratic and cubic polynomials) to the same level γ as the linear case. Table 5.1 shows the cost and losses at the router (queue) for different cases.
The graphs of Figures 5.4,5.5,5.6 show the variation of queue for the three cases. Linear case has the smallest variation. It drops packets more aggressively in the early stages of discard, and thereby it prevents congestion more efficiently. In contrast, the polynomial controls (degree 2 and higher) drop less aggressively at the early stages, which leads to queue build up, and hence more chances for congestion.

Conclusion
In this paper, we have developed a dynamic model that simulates the interaction of TCP sources with active queue management system (AQM). We have proposed a modified version of an earlier model called RED. This is governed by a system of stochastic differential equations driven by doubly stochastic point process with intensity as the control. This proposed model contains a feedback control law that observes the router (queue) status and controls the intensity by sending congestion signals (warnings) to the sources for adjustment of their transmission rates according to the router load. The control laws are optimized by use of genetic algorithm (GA) and random recursive search (RRS) technique. It is observed from our simulation results that the linear control law is better than the polynomial control laws. More precisely, the higher the degree of the polynomial control law is the lower is the performance. The numerical results demonstrate that the proposed model and the method can improve the system performance significantly. As mentioned in the introduction, in previous studies, the choice of the critical parameters such as {min th ,max th ,P max } describing important thresholds for making discarding decision was considered to be an unsolved problem. According to our approach, these parameters or their equivalents are automatically determined by the optimization process itself. Thus this paper provides a technique for determination of critical parameters.