REAL-TIME FEEDBACK CONTROL OF COMPUTER NETWORKS BASED ON PREDICTED STATE ESTIMATION

We present a real-time feedback control strategy to optimize the dynamic performance of a computer communication network. In previous studies closely related to this topic, feedback delay, arising from communication delay, was shown to degrade system performance. Considering this negative impact of delay, we propose a new control law which predicts the traffic in advance and exercises control based on the predicted traffic. We demonstrate through simulation experiments that the predictive feedback control law substantially improves the system performance.


Introduction
With the explosive growth of the Internet over the past twenty years, excessive network congestion occurs frequently.This kind of congestion results in packet losses, transmission delay, and delay variation.Moreover, the increasing integrated applications (e.g., web surfing, teleconferencing, and e-commerce) are sensitive to delay and packet losses.Therefore, it becomes important to efficiently manage and optimally control the traffic, and to ultimately offer a guarantee of quality of service (QoS) for multimedia applications.
A number of different control mechanisms have been proposed to solve these problems.The token bucket (TB) algorithm is one of the methods widely used in the network access control field (see [14,16,17]; see also http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/12cgcr/qos c/qcpart4/qcpolts.htm, and it can dynamically allocate bandwidth and efficiently minimize packet losses.Previous studies [1,2] used TB to construct a dynamic system model.Additionally, different control strategies were proposed to manage traffic flow into the backbone network.The results showed that the feedback control laws can improve network performance by improving throughput, reducing packet losses, and relaxing congestion.On the other hand, in [2], it was observed that the system performance was highly degraded in the presence of feedback delay (arising from communications).Due to the time delay, what we capture in real time is the lagged or delayed traffic information.Control based on delayed information leads to excessive degradation of network performance.Thus, in practice, its impact cannot be ignored and must be taken into consideration and compensated for.However, in [1,2] no attempt was made to deal with this problem and solve the impact of the communication delay.
Traffic prediction methods have been widely used in network management [4,6,7,8,9,12,13,15].By use of prediction techniques, that is, forecasting the future behavior of the traffic, one can effectively prevent traffic jams, traffic congestion, and network crashes.In [4], a method combining the single-step-ahead (SSP) and multi-stepahead (MSP) predictors was developed to forecast MPEG-4 video traffic trace, which employed a nonlinear estimation tool and offline training.To solve the admission control problem in wireless mobile networks, the authors of [7] predicted cell bandwidth requirements subject to the constant bit-rate traffic.In [15], the evolutionary genetic approach (EGA) was used to predict the short-term bandwidth demand in virtual paths (VPs), which enabled intelligent controllers to improve transmission efficiency.Inspired by these ideas, we have applied prediction techniques to solve the problem encountered in [1,2].
For this purpose, we propose a real-time feedback control mechanism based on the predicted state and traffic.The traffic and state information are predicted for different values of prediction times based on their past history (the traffic history measured online).An accurate prediction for the future traffic and state (short-term prediction) is able to provide better control compensating for time delay.Thus the impact of time delay can be minimized and the system performance improved.
In this paper, we developed an online predictor based on the principle of the least mean square error (LMSE), which is one of the simplest methods.It was noted in [6] that LMSE can achieve better accuracy compared to those complex long-memory predictors [9,13] for online measurements.Without the requirement of complex computation, it can be implemented at a high speed.As a result of traffic prediction, the system performance degradation due to delay is reduced by use of proper control actions.According to our results, it is possible to optimize the system performance and minimize the cost function by implementing the new method.
The paper is organized as follows.Section 2 briefly describes TB-based system model in current practice.In Section 3, a cost functional is defined for network performance evaluation.Section 4 presents different control strategies (feedback with time delay and without delay).In Section 5, we propose a real-time feedback control based on the predicted state and traffic.An LMSE predictor is introduced here.Traffic used for simulation is illustrated in Section 6.After choosing different predictor parameters, we seek to optimize the network performance and reduce the system losses.Simulation results will be discussed in Section 7, where we present results showing improvements on system performance using predicted state feedback control.In Section 8, conclusion and suggestions for future work are discussed.

System model with control
In order to understand and solve the performance-related problems in computer communication network, it is critical to build a dynamic model of the information flow through the system.Further, the basic statistical properties of measured trace data must be known.N. U. Ahmed and H. Song 9 Over the last two decades, a number of traffic models have been proposed and studied in the traffic management area.Traditional characterization of the Internet traffic is based on the Poisson process (which exhibits short-range dependence), Bernoulli process, or more generally doubly stochastic Poisson process (DSPP).
A recent study [11] shows that network traffic has self-similarity characteristics and long-range dependence.Self-similarity means that a certain property of traffic behavior is preserved over space and/or time scales, and long-range dependence is said to exhibit long-term correlations which decay at rates slower than exponential ones.On the other hand, the correlation functions of traditional traffic models decay exponentially or faster.
In this paper, a general model is constructed to simulate the incoming traffic illustrated in Figure 2.1, which is similar to those in [1,2].A discrete time series represents a traffic {V (t k ), k = 0,1,2,...,K} generated by the users' applications, where V (t k ) denotes the packet size (measured in bytes) arriving during the non-overlapping time intervals [t k ,t k+1 ).This can be modeled as a marked point process where the event times are the points and the sizes of the packets are the marks.For simplicity we assume that the time intervals are of equal length.
The following notations are used throughout the paper: (1

Basic model for a token bucket.
A TB is described by a pair of parameters (token generation rate u and bucket capacity T).TBs are usually implemented to shape or police traffic between the host and the network or between the routers (see [14,16,17]; see also http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/12cgcr/qosc/ qcpart4/qcpolts.htm).Here TB is used as a traffic policer.Figure 2 If the packet size of the arriving traffic V (t k ) is less than the number of tokens available, the traffic is marked as conforming (traffic) and is immediately passed on to the network for queuing up in the multiplexor.At the same time, a number of tokens equal to the size of the packet is taken out of the token pool.Thus the state of TB at any time is determined by the algebraic sum of three terms: tokens left over from the previous time interval, new tokens added, and tokens consumed during the current time interval.As a result, the dynamics governing the status of the TB is given by the following expression: Note that the last term in the above equation gives the conforming traffic denoted by and hence the nonconforming traffic is given by (2.4)

Complete system model.
To simulate a network, we construct a mathematical model comprised of N individual users (traffic streams), served by N corresponding TBs, all of which are coupled to a multiplexor connected to an outgoing link having (bandwidth) capacity C.This is illustrated in Figure 2.3 [1,2].Each TB implements its algorithm to police the arriving packet.The nonconforming traffic streams are dropped while all the conforming traffic are multiplexed and queued up for entering the multiplexor.As a matter of fact, not all conforming traffic from TBs will be accepted because of the size limitation of the multiplexor (buffer size Q) and the link capacity (speed) of the accessing node.If the sum of these traffics exceeds the multiplexor size, some part of the conforming traffic may be dropped.The discarded traffic is defined as the traffic loss at the multiplexor L(t k ).The dynamics for a single TB is given by (2.2).For a system consisting of N TBs, served by a single multiplexor, the associated variables are given by multidimensional vectors from R N which are listed as follows: (2.5) Since the sources are independent, the dynamics of the system of TBs is now given by a system of identical equations where i = 1,2,...,N.The conforming traffic is accumulated in the multiplexor.The state of the multiplexor, denoted by q(t k ), is given by the volume of traffic waiting in the buffer for service during the kth time interval.Based on the sum of all the conforming traffic and the available buffer space, the multiplexor can only accept the traffic given by Thus the state dynamics of the multiplexor can be written as follows: The first term on the right-hand side represents packets leftover from the previous (k − 1)th time interval, the second term represents packets accepted by the multiplexor during the kth time interval.This is a scalar equation governing the dynamics of the queue in the multiplexor.Thus the dynamics of the whole access control protocol is governed by the system of (N + 1) difference equations (2.6) and (2.8).This leads to the following nonlinear state-space model: where X ≡ (ρ, q) denotes the state vector, u is the control vector, V represents the input traffic vector, and F is the state transition operator determined by the expressions on the right-hand sides of equations (2.6) and (2.8).

Objective function.
In general, the traffic loss at the TBs during the kth time interval is given by while the multiplexor loss during the same time interval is given by In addition to these losses, it is also important to include a penalty for the waiting time or time spent on the queue before being served.For simplicity we assume that it is linearly proportional to queue length.
Adding all these, we obtain the cost functional.Since the incoming source (or user demand) is a random process, we must compute the average cost as being the expected value of the sum of all the costs described above.This is given by where u is the control law which determines the state of the system and hence the individual losses and finally the total cost.The functions α, β, γ represent the relative weights or relative importance given to each of the three distinct losses.
Since the exact stochastic characterization of our traffic is not available or is unknown, the Monte Carlo method is employed to compute the expected values of the performance measures.For applying the Monte Carlo [5,10] technique, we let N s denote the number of samples used and let Ω ≡ {w j , j = 1,2,3,...,N s } denote the elementary events or sample paths with finite cardinality N s .The objective functional (3.3) is then given by The first term of the expression gives the average weighted loss at the multiplexor, the second gives that for TBs, and the last one is the penalty assigned to the average waiting time in the multiplexor.

Performance measure.
In addition to the objective functional, we also use the network utilization to evaluate the overall system performance.The total amount of data successfully transferred to the network is marked as the actual transferred traffic, while the maximum transfer capacity is given by the link capacity.Utilization is then defined as the ratio of the two, measured in percentage by

General feedback control law.
Unlike open-loop controls which do not consider the status of the network resources shared by competing users, the feedback control mechanism (closed-loop control) exercises controls based on available information on the current traffic and system state [8,12,18].By analyzing the current status of the system, the feedback controller can make proper decisions on resource allocation to achieve superior performance.In general, a feedback control law without communication (plus service) delay is of the form where G is a suitable function (to be determined) that maps the available data into a control action.The data may be given by the collection {V ∈ R N , ρ ∈ R N , q ∈ R 1 } which denotes the input traffic, state of TBs, and the multiplexor, respectively.
In the presence of communication and service delay, if the same control law G is used, the actual control action would be different.This is given by the following expression, which is nothing but the delayed version u d of the control u without delay: where m i , i = 1,2,3, denotes the number of time slots by which information reaching the controller is delayed.It is clear from this expression that the current control is decided on the basis of past status information and therefore cannot be expected to be as effective

A simple control law in the absence of time delay.
To avoid cell losses at the multiplexor and monopoly by any users, we include a permission parameter and compute the maximum permissible allocation as follows.The maximum possible allocation for the ith user is given by  where each user's share is determined by the smaller of the fraction of his demand and the fraction of the resource allocation permitted times the available multiplexor space.
Comparing with the real traffic V i (t k ), the true allocation for the ith user during the kth time interval is given by If 1 ≤ e i < N (for one special i), the network provider withdraws the ith user's full permission and allows other users to utilize the supplementary bandwidth.Thus by an appropriate choice of this parameter, the network provider can control monopoly and even assign priorities.
A simple feedback control law that was suggested in our previous papers [1] was found to be very effective.This is given by the following expression: where A i is given by the expressions in (4.4).

Feedback control in the presence of time delay.
Considering all the delays (labeled as communication delay) in real time, the actual control action is given by where delay less than one time slot, 0 ≤ m < 1, is simply considered as no delay.

Prediction scheme and predictive feedback control
In the presence of feedback delay, it is difficult to make a proper decision to effectively allocate network resources or prevent traffic congestion on the basis of delayed information, which results in degradation of system performance [2].To reduce the impact of delay, traffic prediction, that is, accurately providing the statistical characteristics of traffic, becomes one of the key issues in network control engineering.

Prediction scheme.
Recently a number of traffic predictors have been proposed [4,6,7,8,9,12,13,15].However, those predictors either realize the offline control or focus only on constant bit-rate traffic.Here we develop a real-time predictor which is model-free and independent of the statistical properties of the traffic.The algorithm proposed to improve the system performance is called predictive feedback control (PFC).
Monitoring the past history of the system (the traffic and the system state), we are able to make short time prediction of the future traffic and system state.By applying the predicted information to the feedback control law in place of delayed information, excessive packet losses and waste of network resources can be prevented.
Our aim is to find a model-free predictor which does not require excessive computation and can be implemented online.The experiments carried out by Ghaderi, Capka and Boutaba [6,7] have demonstrated that the LMSE predictor can achieve satisfactory performance.The LMSE predictor does not require much computation and is simple to implement.This adaptive predictor is independent of the traffic model (statistics) and is useful for both the short-range-and long-range-dependent processes.In this study, we apply this technique for traffic prediction.
Let V (t k ), k = 0,1,2,...,K, denote any history of the traffic process measured in terms of packets.Our objective is to predict the process by some specified units of time ahead, denoted by T d , based on a segment (window) of the past history of the process denoted by W s .This is mathematically described by the following expression: where T d denotes the number of time slots (ahead of the current time), and W s denotes the length of observation, which is also called observation window width.This is the number of past samples used to predict the future traffic.Here the vector α denotes the weight or importance given to past samples observed.The major objective of traffic prediction is to minimize the mean square difference (error) between the predicted traffic and the actual traffic measured.The choice of the weight vector α determines the level of prediction error.In order to determine the best weight vector, we define the estimation error as follows: (5.2) N. U. Ahmed and H. Song 17 Then where (x, y) = x i y i denotes the standard inner product in R N .If X, Y are two Ndimensional random vectors having finite second moments, then E(X,Y ) denotes the expected value of their inner product.Since there are no constraints on α ∈ R d , (d = W s ), differentiating J with respect to α and setting it equal to zero, we obtain Let A and b denote the matrix and vector as defined below: where (5.6) Then (5.4) can be written compactly as Aα = b, and, if A is nonsingular, the solution is given by α = A −1 b.Hence the estimated traffic can be computed by the following expression: where (A −1 b) r denotes the rth component of the vector A −1 b.

Predictive feedback control.
After we have the best mean square estimate of the future traffic T d steps ahead of current time, based on observation of past history of length W s , the control law can be adapted in advance corresponding to the predicted states of the future traffic.Assuming that the state (ρ, q) is monitored in real time, to minimize the impact of delay we choose the new control law as (5.8)

Basic data used for numerical simulations
Implementation of our system approach to computer communication network is described in this section.Fractional Brownian motion (FBM) is realized by use of Matlab.Using the basic idea of the algorithm introduced in [3], we have developed the traffic models in C++ programming language.Finally, the entire system including feedback control is implemented using Matlab.

System parameters and configurations.
In order to compare the results with previous work [2], we use an identical system to the one in [2].The system consists of three users (sources) regulated by three TBs, which are served by a multiplexor connected to the outgoing link.Three individual traffic traces, each of 4-second duration, are fed into the three TBs directing conformed traffic to the multiplexor.The experiments also follow the previous assumptions of [2] as follows: (a) the three independent traffic traces share the same statistical characteristics, (b) packet size is measured in terms of bytes and one token is consumed for each byte, (c) the traffic traces last 4 seconds.While calculating the objective function, we have to consider the weights assigned to the costs associated with the losses at TB, the multiplexor, and waiting time.They are chosen as follows: α(t k ) = 10, β(t k ) = 5, and γ(t k ) = 0.3 for all t k .
The system parameters are listed in Table 6.1.

Specification of traffic traces.
Two major types of traffic are used in our simulation: (1) Bellcore trace and (2) DSPP generated by FBM as described below.
The traffic traces used here are partitioned into small time slots, which individually contain the information for 4 seconds and are described in bytes.These 4-second data are used as the traffic from the network users.

DSPP traffic traces.
We also construct a self-similar traffic model using a nonnegative function of FBM as the input rate of a Poisson process.This yields a DSPP which exhibits self-similarity and long-range dependence properties.FBM denoted by B H (t) is a self-similar process itself.It depends on the parameter H called the Hurst parameter.In general, its values lie in the interval (0,1), and it represents a measure of self-similarity in the traffic, and "burstiness."We construct the FBM through the following integral transformation of the standard Brownian motion {B(t), t ≥ 0}: where K H (t) is given by and C H is any constant.Then we generate a Poisson process with the intensity function given by the absolute value of the FBM:

Simulation results and numerical analysis
In this section, we show the prediction performance of the LMSE predictor and the improvement of the overall performance of the system by use of the predictive feedback control.

Performance of LMSE predictor.
To illustrate the dependence of estimation error on the observation window size W s and the prediction time T d , we use the Monte Carlo technique to compute the expected value of the (estimation) error given by where w j denotes the jth sample path and N s denotes the number of sample paths used.The inverse of the signal-to-noise ratio (E NSR ) is used as another measure to evaluate the quality of prediction results: Numerical results of prediction based on the LMSE method are presented in the following section for both the Bellcore traffic and DSPP traffic.

Bellcore traffic trace.
Here the performance of the LMSE predictor is evaluated for different observation window sizes and prediction times.Similar studies were reported in [6,7] for a fixed window size and a fixed prediction time.In contrast, we evaluate the prediction performance for different window sizes and prediction times.Since prediction times required vary with the communication delay causing delayed control actions, our results corresponding to required prediction times can be used to evaluate the performance of the system described by (2.6), (2.8), and (5.8).
Dependence of E NSR on observation window size.Figure 7.1 shows the plots of E NSR as a function of observation window size for fixed prediction times (as parameters).It is clear that for a fixed prediction time, the error decreases with the increase of (observation) window size.This is expected.On the other hand, it is also clear from this figure that, for a fixed window size, prediction error increases with the increase of prediction time.Furthermore, for a fixed prediction time, as the window size increases, the prediction error tends to reach a lower limit possibly greater than zero.This means that by simply increasing the window size we cannot expect to improve the performance beyond a limit.
Dependence of E NSR on prediction time.In Figure 7.2, we plot E NSR as a function of prediction time for fixed values of window size.From the plot, it is clearly seen that for any fixed window size, E NSR increases with the increase of prediction time.This is also expected.Again for a fixed prediction time, as the window size increases, the prediction error decreases.Dependence of E NSR on observation window size.Again, it is clear from Figure 7.3, plotted for two different Hurst parameters, that the prediction error decreases when the observation window size increases for a fixed prediction time.This is similar to what was observed in the case of Bellcore traffic.We also find that with the increase of the Hurst parameter, E NSR decreases.Figure 7.4 demonstrates the relationship between E NSR and the Hurst parameter for fixed prediction times and three different window sizes.The result illustrates that the long-range dependence property exists in the traces.The larger the Hurst parameter, the stronger the correlation with the past information resulting in reduced prediction error with increasing observation window size.
Dependence of E NSR on prediction time.Figure 7.5 offers another insight.For any fixed window size, E NSR increases with the increase of prediction time and appears to reach a plateau.As expected, E NSR is smaller for larger Hurst parameters.This is further illustrated in Figure 7.6.

Simulation results
and numerical analysis of system performance.As described in the introduction and Section 4, communication delay in the network, resulting in delay in control actions, adversely affects the performance of the system [2].By use of predictor controls, we demonstrate improved performance.Here simulation results are presented in Figures 4.1,4.2,4.3,7.7,7.8,7.9, and 7.10 to illustrate the improvement of the overall performance of the system.7.2.1.Dependence of system performance on control policies.In this subsection, we present performance results corresponding to predictive feedback control and compare them with those corresponding to feedback controls with or without delay.The results clearly 24 Real-time feedback control of computer networks  demonstrate that by use of the predictive feedback control law, one can achieve substantial improvement of system performance.We consider the following five cases to compare with the results reported in [2].
( In all these cases we use predictive feedback control law with increasing window size.It is clear (Case 3) that use of this control law substantially improves the performance despite communication delay.This is further improved by use of larger window sizes as seen in Cases 4-5.System utilization given by the thin curve shows that utilization is highest in the absence of communication delay and lowest in its presence (Case 2) and then increases if predictive feedback control is used with increasing window size (Cases 3-5).This is explained further on.
26 Real-time feedback control of computer networks  System total cost System total cost For C H = 15,T = 4000 (milliseconds) and H = 0.6, m H = 2.39 × 10 5 and for H = 0.8, m H = 4.8 × 10 6 .Note that the waiting cost, losses at TBs, and the total cost shown in the first row of Figure 4.2 are lower than those in Figure 4.3.Also Figure 4.3, corresponding to H = 0.8, shows higher utilization.These differences are clearly due to difference in the volume of traffic in the two cases.

Dependence of cost on observation window size.
The cost function is a measure of overall system performance.This is plotted as a function of window size for 3 different values of communication delay.
Bellcore traffic trace..8b,we observe that the cost reduction, with the increase of observation window size, is again more pronounced for larger Hurst parameters.This is due to the fact that the process with larger Hurst parameter has stronger correlation with the past and hence the larger window size contains more useful information for more accurate prediction of the future traffic.Bellcore traffic trace.Figure 7.9 gives another aspect of system performance.It is clear from these curves that utilization increases with increasing window size.Note that as the window size increases to a certain level, the growth of utilization slows down.It illustrates that by simply increasing the window size the performance cannot be improved beyond a limit.Again this is due to the same reason as mentioned in Section 7.2.2.
DSPP (H = 0.6,0.8)traffic traces.Again from Figure 7.10, it is clear that as the window size increases, utilization increases, which has been observed also in Figure 7.9 for Bellcore traffic.

Conclusion
In this paper, we have demonstrated that by use of predictive feedback control, it is possible to compensate the impact of communication delay causing performance degradation as reported in [2].The method presented in this paper improves the overall system performance and prevents network instability.The numerical simulation results presented in Section 7 have shown the effectiveness of the proposed predictive feedback control law.
We also explore the relationship between the Hurst parameter of the traffic and prediction performance.It was found that processes with larger Hurst parameter have better prediction performance.In addition, dependence of system cost and utilization on observation window size, corresponding to different values of Hurst parameters, has also been discussed.The results of this paper also lead to a better understanding of the impact of Hurst parameters on network performance.In summary, this work provides a useful tool for design and optimization of future networks using predictive feedback control law thereby avoiding instability.

. 3 )Figure 6 . 2 Figure 6 . 4 .Figure 6 . 5 .
Figure 6.2 shows λ(t) which is generated by the absolute value of the FMB with the Hurst parameter H = 0.6 and C H = 15.Another set of λ(t) with H = 0.8 is plotted in Figure 6.3.They are chosen as the intensity (rate) of a Poisson process.The DSPP traces with the intensities as defined above are generated, and the samples of the traces are plotted in Figures 6.4 and 6.5, respectively.

) Case 1 :
feedback control (without delay, T d = 0).(2) Case 2: feedback control (with delay, T d = 1).(3) Case 3: feedback control (predictive feedback control, T d = 1, W s = 1).(4) Case 4: feedback control (predictive feedback control, T d = 1, W s = 6).(5) Case 5: feedback control (predictive feedback control, T d = 1, W s = 11).Bellcore traffic trace.Numerical results for the system performance are shown in Figure4.1.Here we have listed the weighted TB losses, waiting-time losses, total cost, and utilization.It is clear from Figure4.1 that feedback control without (communication) delay (Case 1) achieves the minimum cost and the highest utilization.The worst situation occurs in Case 2 with one unit of communication delay.Provided with the delayed (traffic) information, the controller cannot supply the required number of tokens to match the incoming traffic.This leads to significant packet losses at the TB resulting in degradation of performance and utilization.On the other hand, by use of predictive feedback control law (Cases 3-5), it is possible to reduce the performance degradation significantly.

7. 2 . 3 .
Dependence of utilization on observation window size.The system utilization depends on the volume of traffic successfully transferred to the network.Here we present the utilization as a function of observation window size for three different values of communication delay (Figures 7.9 and 7.10).
6.2.1.Bellcore traffic trace.Bellcore traffic trace (see http://ita.ee.lbl.gov/html/contrib/BC.html) was captured on an Ethernet at the Bellcore Morristown Research and Engineering facility, which lasts 3142.82seconds and contains one million packets.Figure6.1 shows part of the trace which is divided into 4-second segments.
Utilization versus W s -Bellcore traffic.Among the five cases, Case 1 maintains the lowest system cost and the highest utilization, and with predictive feedback control law, system performance gets better with increasing window sizes.Similar results are plotted in Figures 4.2 and 4.3 corresponding to Hurst parameters H = 0.6 and H = 0.8, respectively.The time average of the expected traffic intensity λ H (t) ≡ |B H (t)| is given by