Continuous and Discrete-Time Optimal Controls for an Isolated Signalized Intersection

1Beijing Key Lab of Urban Intelligent Traffic Control Technology, North China University of Technology, Beijing 100144, China 2Graduate School at Shenzhen, Tsinghua University, Shenzhen 518055, China 3The Institute for Transport Planning and Systems (IVT), ETH Zurich, Zurich, Switzerland 4School of Management Science and Engineering, Central University of Finance and Economics, Beijing 100081, China 5Beijing Key Laboratory for Cooperative Vehicle Infrastructure Systems and Safety Control, School of Transportation Science and Engineering, Beihang University, Beijing 100191, China


Introduction
Along with the fast increase of auto population, urban streets are becoming more crowded nowadays.To relieve congestions and reduce accidents, various traffic control methods have been proposed since the late 1950s [1].
As a typical traffic scenario, oversaturated intersections attracted consistent interest during the last six decades [2,3].The term "oversaturated" means the following: the vehicles that remained since the last cycle plus the vehicles that newly arrived exceed the capacity of the intersection.This leads to the carryover of vehicle queues (at least in one leg of the intersection) to the next cycle.
Discussions on the road networks that consist of many oversaturated intersections can be found in researches done by Chang and Sun [4], Di Febbraro and Sacco [5], Dotoli and Fanti [6], Ma [7,8], and Sun et al. [9,10].In the research done by Varaiya [11] and Le et al. [12], the study of pressure-based signal control developed stability properties of a decentralized signal timing policy for networks with stochastic arrivals.But for a real-time signal timing optimization problem, the data that could be used is the arriving information of vehicles in the recent several signal cycles (data could be gotten by connecting vehicle technology, etc.).The optimization objects and scenarios are different between the model in this paper and pressure-based policies.This paper will focus on the isolated oversaturated intersection.
Usually, researchers aim to find an optimal signal timing plan that minimizes the total delay of vehicles passing this intersection.The total delay is often defined as the time integral of the sum of all queue lengths for all legs of the intersection over a given time horizon.However, the total delay is a nonlinear and nonconvex function of control variables (e.g., green phases), which makes it difficult to optimize.One promising approach is to apply heuristic algorithms to solve the formulated optimization problem.For example, the genetic algorithm was applied by Park et al. [13] to optimize the total delay.An alternative approach is to first approximate the nonconvex total delay with some convex functions and then solve the newly formulated optimization problem.The rest of this paper will focus on the second approach within a typical traffic scenario: an isolated intersection with only two movements.
There are mainly two kinds of convexified models for this scenario.The first kind of models originated from Gazis [3] who used continuous-time differential equations to describe the traffic dynamics.The cycle length, departing flow rates, and arriving flow rates in Gazis [3] were all assumed to be constant.Michalopoulos and Stephanopoulos [14,15] extended the continuous-time model by including the maximum queue lengths constraints and time-variant arrival flow rates.Such formulations led to a classical control problem that can be solved via the Pontryagin Maximum Principle (PMP) [16].However, the obtained continuous-time signal timing plan should be discretized into the corresponding discrete-time signal timing plan that can be executed in practice.
The second kind of models uses discrete-time difference equations to describe the traffic dynamics [7,8,[17][18][19].The corresponding design problem can then be formulated as a linear programming (LP) problem [20][21][22][23][24][25].One interesting question that naturally arises is how to depict the difference and connection between the discrete-time model and the continuous-time model.
Recently, Ioslovich et al. [26] studied the formulated LP problem by considering the corresponding continuous-time approximation model and gave an elegant approximate solution in continuous-time forms.However, it was not verified how this approximate solution differs from the accurate solution (the solution obtained by the continuous-time model).Whether a discretized version of this continuous-time approximate solution is still optimal to the LP problem also needs further discussions.
Zou et al. [27] have given a preliminary result on the relationship between the continuous-time model and the discrete-time model.Zou et al. applied a graphical method to adjust the continuous-time approximate solution to a discrete-time accurate solution.They assumed that both streams are dispatched simultaneously and the adjustment method does not influence the clearance cycle.It is shown that the approximate solution can be adjusted to the optimal solution by changing the green ratio in the switching cycle.However, the discrete-time LP problem in Ioslovich et al. [26] is more complex than the one considered in [27], since the two streams are allowed to be cleared at different times.Merely changing the green ratios at the switching cycle may not obtain the accurate solution.
In this paper, the LP problems proposed by Ioslovich et al. [26] are directly attacked using the strong duality theorem [28].It is first shown that the approximate solution and the accurate solution do not coincide in many situations.Then, the relationship between the approximate solution and the accurate solution will be discussed.The errors introduced by discretization are carefully studied.It is shown that, in many cases, the discretized approximate solution can be converted to the accurate solution within a few gradient descent adjustments.Finally, an algorithm is proposed to implement this conversion.These findings shed light on the connection between the continuous-time and discrete-time signal timing models.To give a detailed analysis, the rest of this paper is arranged as follows.Section 2 introduces the LP problem proposed by Ioslovich et al. [26] and lists the nomenclature used in this paper.Section 3 proposes two counterexamples to show that the optimality of the discretized approximate solution may not hold.Section 4 discusses the difference and connection between the discrete-time model and the continuoustime model.Finally, Section 5 concludes the paper.

Problem Presentation
2.1.Nomenclature and Assumptions.For presentation simplicity, the nomenclature in this paper follows Ioslovich et al. [26] as shown in Nomenclature List.Similar to Ioslovich et al. [26], consider an isolated intersection with two one-way streams  1 and  2 governed by signal lights, as shown in Figure 1.Note that the two-stream isolated intersection is chosen as an initial building block to understand the connection between the continuous-time model and the discretetime model.This is a widely applied treatment even in recent works (e.g., [23]).Nevertheless, this assumption can be relaxed to consider general intersections.Both the continuous model and the discrete model are able to handle cases with more generalized intersections.Although it is hard to directly compare the two types of models in generalized intersections, we expect that the findings in this paper can shed light on more generalized intersections.
Likewise, the following assumptions are imposed as in Ioslovich et al. [26].
Assumption 1.(a) Each signal cycle starts with a green light for  1 .
(b) There is no lost time for each cycle.Suppose that the green ratio for stream  1 in cycle  is denoted as (); the green ratio for stream  2 is then equal to 1 − ().
(c) The arrival flow rates and the saturation departure flow rates of both streams are constant over time.Denote   as the arrival flow rates for   and   as the departure flow rates.
(e) For some reasons (e.g., physical constraints), both streams are constrained by minimum green ratios.Define  min and 1 −  max as the minimum green ratios for  1 and  2 , respectively; then, the constraint on () is represented as  min ≤ () ≤  max .
(f) There exists a signal timing strategy such that queues in both streams can be cleared finally but may not be in the same cycle.Define   =  1 / 1 and   = 1 −  2 / 2 .This assumption is written as   <   ,   <  max , and  min <   .
(g) Cycle lengths are known to be constant .
In Assumption 1, (a)-(f) are inherited from Ioslovich et al. [26] directly and (g) is added to simplify the discussion.In fact, if cycle lengths are not fixed, a similar conclusion can still be drawn but the detailed timing plan is much more complicated.

The Discrete-Time LP Model and Its PMP Solution.
Similar to Ioslovich et al. [26], the discrete-time model is formulated as the following problem to minimize the total delay of vehicles: for  = 0, 1, . . .,  − 1.
In (1)-( 5), the objective function is a linear approximation for total delay.The decision variables are (),  = 0, 1, . . ., .For presentation convenience, it is assumed that the number of total cycles  is large enough such that both streams are cleared before cycle .Equations ( 2) and (3) represent the evolution of both queues in time.Equation (4) gives the upper and lower bound of the green ratio in each cycle.Equation ( 5) is the initial queue lengths.It is easy to verify that this LP problem is equivalent to the Relaxed Discrete-Event Max-Plus Problem proposed by Haddad et al. [22].
Particularly, the following four cases were discussed with respect to the order of   ,   ,  min , and  max : (I) Table 1 lists the solution.The optimal continuous-time solution is denoted asV * ().  is the switching time where the optimal control switches. and  are represented as For example, Figure 2 illustrates the evolution of queue lengths under the optimal solution for Case I(a).It is observed that the optimal solution is a two-stage strategy, namely, bang-bang control.The green ratio remains  max at first, causing the length of stream  1 to decrease sharply and stream  2 to increase.After the switching time   , the green ratio changes into  min and both streams start to decrease.The optimal switch-over point where the solution switches from maximum to minimum green split is given in Table 1.
The discretized solution of V * () is obtained as follows.Suppose the cycle length is ; the discretization is done as For presentation simplicity, the solution   (),  = 0, 1, . . .,  − 1, is called the discretized approximate solution in the rest of this paper.

The Steady State and the Steady-State Solution.
According to Haddad et al. [22], the steady state is defined as the state where the queue lengths are the same in the beginning and end of each cycle.This means, given the green ratio for a certain stream, the queue lengths of this stream keep the same in a certain period.Denote the queue lengths in the steady state as  ,ss ,  = 1, 2, and the green ratio in the steady state as  ss .So, considering a single-cycle version of discrete-time model (see (1)-( 5)), the steady state can be formulated into the following model: min  1,ss +   The objective equation ( 9) is a linear approximation of delay in a single cycle.Equations ( 10) and ( 11) depict the queue evolution.Equation (12) gives the bound of the green ratio.
The optimal green ratio  ss is called the steady-state solution.Haddad et al. [22] gave a neat analytical form of  ss as follows: According to Haddad et al. [22], the two streams that follow the discrete-time model (see ( 1)-( 5)) will eventually enter the steady state under Assumptions (a)-(g).Therefore, in the accurate solution, the green ratios in the steady state are the steady-state solution  ss .

The Nonoptimality of the Discretized Approximate Solution
In this section, it is shown that the discretized approximate solution does not always coincide with the accurate solution.
The term "coincide" here means that the discretized approximate solution is not always optimal for the discrete-time model.In particular, two counterexamples are constructed.
The first counterexample is used to deal with Case I and Case II where the minimum green ratio  min plays an important role in the approximate solution.In contrast, the second counterexample is used to handle Case III and Case IV where the minimum green ratio  min does not influence the approximate solution.The two counterexamples are as follows.
Counter As is shown in Figure 3, the accurate solution deviates from the discretized approximate solution in two ways.In Counterexample 1, the accurate solution roughly remains as a bang-bang form before the system enters steady state.The difference concentrates around the switching cycle and steady state.However, in Counterexample 2, the accurate solution exhibits oscillation and differs from the discretized approximation solution in almost every cycle.
Such deviations are caused by discretization and can be classified into two categories.
The first category of deviations is caused by the discretization of the model constraints.For the continuous-time model, the queue length for stream   is lower-bounded by the constraint  1 () ≥ 0, while for the discrete-time model, the corresponding constraint is  1 () ≥  1 (1 − ()).When cycle length  is large, the discretization of model constraints may introduce huge errors.In Counterexample 1, the switching time locates in the middle of a cycle.As shown in Figure 4, the discretization leads to waste of green ratio around the switching cycle due to the restriction of the minimum green ratio.When  is small enough, the solution of the continuous-time model and the solution of the discrete-time model are coherent.A proper  should be selected to ensure consistency.
The second category deviation is caused by the discretization of the approximate solution.In Counterexample 2, the clearance time of stream  1 locates in the middle of a cycle.So, the discretization causes oscillation in the accurate solution.

The Relationship between the Discretized Approximate Solution and the Accurate Solution
In this section, the connection between the discrete approximate solution and the accurate solution is further discussed.For Case I and Case II, the errors are relatively concentrated.The discretized approximate solution does not deviate largely from the accurate solution in many cases.In fact, the discretized approximate solution can be modified to the accurate solution by some minor adjustments; see Algorithm 1.The details of adjustments will be discussed in Sections 4.1-4.3.
For Case III and Case IV, however, the dispersed errors make it hard to adjust the solution in a few steps.However, for some carefully chosen cycle length, the accurate solution coincides with the discretized approximate solution.
For example, in Case III, if the cycle length is chosen as where  is any positive integer, the accurate solution can be written as It is easy to show that this accurate solution coincides with the corresponding discretized approximate solution.
In the rest of this section, the discussion will focus on the adjustments for Case I, and the adjustments for Case II are similar.
For presentation simplicity, define several denotations before starting the discussion.Define k1 and k2 , respectively, as the clearance cycle indices in which streams  1 and  2 are cleared exactly: k2 fl max { :  2 () > 0} .
Apparently, in Case I(a), it holds that k1 ≤ k2 .So, both streams enter steady state after cycle k2 .Note that there is also a switch in the discretized approximate solution.Define k as the switching cycle index where switch occurs; that is, k : min { :  () <  max } . ( The adjustments contain two parts.Section 4.1 deals with the adjustment in the steady state.Section 4.2 includes However, in the clearance cycle k2 , the optimal green ratio is not  ss due to discretization.So, ( k2 ) can be set as follows: 4.2.The Adjustment around the Switch.The adjustment around the switching cycle is shown in lines ( 2)-(33) in Algorithm 1.There are two cases of adjustment: increasing the green ratios after the switch and decreasing the green ratios before the switch.For presentation simplicity, this subsection only handles the case of increasing the green ratios after the switch.The adjustment to the other case is similar except for some details; see Algorithm 1, lines ( 9)- (20).This adjustment can be viewed as a gradient descent approach to gradually reduce the total delay.It operates in three steps: (1) determine the direction of adjustment; (2) update the green ratio; (3) check feasibility of the new green ratio.It will be proven in the Appendix that such adjustment operations can be finished in a few steps.
Due to constraint equations ( 3) and (4), the discrete-time model is a nonsmooth model.Thus, a subdifferential will be used instead of a gradient.Without loss of generality, assume that both streams are cleared after the switching cycle (i.e., k1 ≥ k ).The case of k1 < k can be transformed easily to the case of k1 > k (Algorithm 1, lines (2)-( 8)).
In the first step, the direction of adjustment is determined by calculating the subdifferential.Denote the subdifferential of the total delay   with respect to ( k ) as   .Define the function ( k1 , k2 , k ) as so as to represent the subdifferential in a more convenient way.
Clearly, the case of increasing the green ratios after the switch corresponds to the condition of ( k1 , k2 , k ) < 0. Step 1 Step 2 Step 3

Discretized solution
Step 4 Step 5 Optimal solution render a better total delay and thus the optimal ( k ) is achieved.
In the second step, the green ratio ( k ) is updated by reducing k1 and increasing k2 .The green ratio ( k ) is calculated such that either stream  1 or stream  2 is cleared exactly in its clearance cycle.The detailed calculation is shown in Algorithm 1 (lines ( 24)-( 29)).
The third step is the feasibility check of whether the new green ratio ( k ) satisfies the constraint equation ( 4).In the case of increasing ( k ), the only possible violation is ( k ) >  max .Therefore, if the feasibility check fails, ( k ) is set as  max and a new bang-bang solution is obtained.Then, the algorithm goes back to the first step and determines the direction of adjustment.
The adjustment of the discretized approximate solution is also shown in Figure 5.

Conclusion
This paper studies the continuous-time and discrete-time signal timing models for the classical isolated signalized intersection with only two one-way vehicle flows.Against intuitions, the nonequivalence between the solutions of the two models is first shown by two counterexamples.Then, the differences between the solutions of the two models are explained.Finally, an algorithm is proposed to transform the discretized continuous-time solution to the discrete-time solution for many cases.Via this gradient descent based algorithm, a more concrete link is set up between the continuoustime and discrete-time signal timing models.

Figure 1 :
Figure 1: An isolated intersection with two one-way movements  1 and  2 .

Figure 2 :
Figure 2: An illustration of the queue lengths and optimal timing plan in Case I(a).

Figure 3 :Figure 4 :
Figure 3: The two counterexamples.The dotted line represents the discretized approximate solution.The solid line represents the accurate solution.

Figure 5 :
Figure 5: The adjustment of the discretized approximate solution.

Table 1 :
The optimal solution to the continuous model.