Optimality Condition-Based Sensitivity Analysis of Optimal Control for Hybrid Systems and Its Application

Gradient-based algorithms are efficient to compute numerical solutions of optimal control problems for hybrid systems OCPHS , and the key point is how to get the sensitivity analysis of the optimal control problems. In this paper, optimality condition-based sensitivity analysis of optimal control for hybrid systemswithmode invariants and control constraints is addressed under a priori fixed mode transition order. The decision variables are the mode transition instant sequence and admissible continuous control functions. After equivalent transformation of the original problem, the derivatives of the objective functional with respect to control variables are established based on optimal necessary conditions. By using the obtained derivatives, a control vector parametrization method is implemented to obtain the numerical solution to the OCPHS. Examples are given to illustrate the results.


Introduction
In many fields of applications, such as powertrain systems of automobiles and multistage chemical processes, dynamics of the systems involve a sequence of distinct modes with fixed mode transition order, forming a hybrid system characterized by the coexistence and interaction of discrete and continuous dynamics the mode is commonly denoted by a discrete state of the systems in hybrid systems literature .To achieve some overall optimal performance for the systems, the duration and the admissible continuous control function of each mode must be determined as a whole 1-3 ; thus, it necessitates the use of theories and techniques for the analysis and synthesis of hybrid dynamical systems.With the growing importance of hybrid models, various classes of hybrid systems for analysis, design, and variables, that is, the mode transition instant sequence and admissible continuous control functions, are derived analytically.As a result, a control vector parametrization method is implemented to obtain the numerical solution to optimal control of the hybrid systems with the obtained derivatives.The sensitivity analysis in Vassiliadis et al. 1, 2 is similar to the work, in which the sensitivity of states w.r.t control parameters is directly obtained from the state equations and the sensitivity of objective functional with respect to control parameters is not involved.In contrast, this paper derives the derivatives of cost functional w.r.t control variables based on the optimality conditions and gives the explicitly expression of the derivatives.Therefore, the main contributions of this paper are listed as follows.a Optimality conditions-based sensitivity analysis of optimal control for hybrid systems with mode invariants are given explicitly, and b following the given derivatives, a control vector parameterization method is designed to obtain the numerical solution.Compared with the existing results on the OCPHS with fixed mode transition order, the settings in this paper cover not only the control constraints, but also the continuous states constraints, which makes the results here more general.
The paper is organized as follows.In the next section, the hybrid system with mode invariants and its optimal control problem are formulated.In Section 3, the equivalent problem and associated optimal conditions are analyzed.The derivatives of the objective functional w.r.t control variables are established in Section 4, and a control vector parametrization approach is also proposed in this section.Some numerical examples are presented in Section 5, and Section 6 contains conclusions.

Terminology and Notation
N denotes the set of positive integers.R and R denote the set of real numbers and nonnegative real numbers, respectively.A T denotes the transpose of a vector or a matrix A. C l a, b , R n denotes the family of continuous functions f from a, b to R n with up to l order derivatives.• denotes the Euclidean norm.mode transition instants and mode transition order, respectively.A pair of t j−1 , i j indicates that at instant t j−1 , the hybrid system transits from mode i j−1 to mode i j .During the time interval t j−1 , t j , mode i j is active and unchanged.The mode transition order π of the considered hybrid dynamical systems is known a priori.Without loss of generality, it is supposed that the mode transition order is {i 1 , i 2 , . . ., i K } over the finite horizon t 0 , t f , i j ∈ I, j 1, 2, . . ., K.Moreover, according to each distinct mode, the continuous states are restricted in a specified range which is referred to as mode invariants.Here, the mode invariants are formulated by a set of inequalities.Thus, for each mode i j ∈ I and its active horizon t j−1 , t j , the dynamics of the considered systems can be formulated by ẋ f i j x, u , p i j x < 0,

Hybrid Systems
is the mode transition instant when a particular mode transition occurs, p i j , ψ i j , and g i j are h i j < n, n and r i j ≤ n dimensional vectors for i j ∈ I, respectively.n, m, h i j , r i j ∈ N. To make the hybrid systems formulated by 2.1 well defined, the following assumption is needed.
and such that a uniform Lipschitz condition holds, that is, there exists K f < ∞ such that Remark 2.2.p i j x < 0 indicates mode invariant for mode i j ∈ I, which describes the conditions that the continuous states have to satisfy at this mode and can be referred to as the path constraints of the continuous states in Vassiliadis et al. 1, 2 .
Remark 2.3.g i j x t − j 0 can be referred to as mode transition conditions which describe the conditions on the continuous states under which a particular mode transition takes place.When mode i j is active over t j−1 , t j , then, at t − j , x meets an n − r i j -dimensional smooth manifold S i j {x | g i j x 0} and mode transition from i j to i j 1 occurs.The mode transition conditions implicitly define the mode i j 's active horizon t j−1 , t j .To prevent Zeno behavior from occurrence, t j−1 < t j is assumed.Physically, the mode transition conditions are always the boundary of closure of the mode invariant p i j < 0.
Remark 2.4.x t j−1 ψ i j x t − j−1 is the outcome of the mode transition and describes the effect that the transition will have on the continuous states.It can be viewed as junction conditions in Vassiliadis et al. 1, 2 .It is assumed that Remark 2.5.Basically, for general hybrid systems, the evaluation of i should be formulated by a function of impulsive control or a graph, which generates mode transition sequence, as formulated in Song and Li 24 and Cassandras and Lygeros 8 .However, the order of the mode transition π is known a prior here thus, the evaluation of i is determined only by the transition instants t j , and the evaluation function of i is omitted here.
Besides Assumption 2.1, to make the considered systems to be well defined, there are some additional assumptions on mode invariants and mode transition conditions should be imposed.Here, it is supposed that the mode invariants and mode transition conditions meet the requirements as in Taringoo and Caines 20 .

Optimal Control Problem for Hybrid Systems
Let L i ∈ C l R n ×U i ; R be a running cost function, ϕ ij ∈ C l R n ; R be a discrete state transition cost function, and φ ∈ C l R n ; R be a terminal cost function, i, j ∈ I, l ≥ 1, l ∈ N, respectively.The optimal control problem for the hybrid systems 2.1 is stated as follows.
Optimal Problem A Consider a hybrid system formulated by 2.1 , given a fixed time interval t 0 , t f and a prespecified mode transition order π {i 1 , i 2 , . . ., i K }, find a continuous control u ∈ U i j in each mode i j ∈ I and mode transition instants θ {t 1 , . . ., t K−1 }, such that the corresponding continuous state trajectory x departs from a given initial state x t 0 x 0 and meets an Remark 2.6.As it is well known, when t 0 and t f are unknown points in some fixed interval T ⊂ R , this problem can be transformed to one with fixed time essentially by introducing an additional state variable.
There are fruitful strategies about how to compute OCPHS see 15 and the references therein , and the basic idea is briefly reviewed as follows for completeness.
Obtaining the optimal control for hybrid systems is very difficult due to the interactions between the continuous states and discrete states which produce a mode transition sequence that increases the feasibility range of the decision variables.One algorithm framework for dealing with this complexity is the decomposition method as follows: where According to this framework, the master problem is how to get the optimum of the inner functional, that is, minimize J u, θ given π.The key point of finding the optimal solution of J u, θ is how to get the sensitivity of the objective with respect to control variables, which provides a better direction for searching and hence reduces computational burden and help associated algorithms converge quickly and accelerate the primary problem convergence eventually.
In next section, the derivatives of cost functional with respect to control variables are established analytically based on optimality condition, which can facilitate the design of associated gradient-based algorithms.

Equivalent Problem and Its Optimal Conditions
When control vector parametrization methods are implemented to obtain numerical solution to the OCPHS, updating the parameters of control profiles should be at the same time point when iterative procedure is running.However, the fact is that the mode active horizon t j−1 , t j for mode i j ∈ I is varying during the procedure running, so a fixed horizon should be introduced, which will guarantee the updating of parameters of control profiles is at the same time point.For this purpose, let τ ∈ 0, K be a time independent variable, and t ∈ t j−1 , t j can be formulated by In addition, to deal with mode invariants constraints p i j x < 0, slack algebraic variable s i j s i j 1 , . . ., s i j h i j T ∈ R h i j is introduced for each mode i j ∈ I, such that p i j x and θ, such that the corresponding continuous state trajectory x 1 departs from a given initial state x 1 0 x 0 and x K meets an n − l f -dimensional smooth manifold S f {x K | ϑ x K 0, ϑ : R n → R l f } at K, and the cost functional where and M is a large positive constant.

Mathematical Problems in Engineering 7
According to Theorems 2 and 3 in Dmitruk and Kaganovich 12 , when M is big enough Optimal Problem B is equivalent to Optimal Problem A.
Remark 3.1.The penalty function term, say, M h i j l 1 p i j l x j s 2 i j l 2 , cannot always guarantee the state satisfies the mode invariant conditions.However, the method works well in practice; moreover, the mode transition order is fixed in this paper which reduces the negative effect of the penalty function method for OCPHS.
For τ ∈ j − 1, j , j 1, . . ., K, let λ j ∈ R n , and define Hamiltonian function H j by H j λ j , x j , u j , s j L i j x j , u j , s j λ T j f i j x j , u j , 3.5 and according to Sussmann 10 , Shaikh and Caines 11 , and Dmitruk and Kaganovich 12 , the following Theorem 3.2 holds.
Theorem 3.2.In order that u and s are optimal for Optimal Problem B, it is necessary that there exist vector functions λ j , j 1, . . ., K, such that the following conditions hold: a for almost any τ ∈ j − 1, j , the following state equations hold: b for almost any τ ∈ j − 1, j , the following costate equations hold: Mathematical Problems in Engineering e transversality conditions for λ j , λ j 1 j β j , j 1, . . ., K − 1,

3.10
where α j ∈ R h i , β j ∈ R n are Lagrangian multipliers.Based on Theorem 3.2, the sensitivity analysis is established in the next section for Optimal Problem B.

Sensitivity Analysis and Parametrization Method
For finding numerical solution to the OCPHS effectively, based on Theorem 3.2, the derivatives of the objective functional J • with respect to the control u, s, and the mode transition instant t j , j 1, . . ., K−1 are established in this section, and by using the obtained derivatives associated parametrization method is proposed.

4.3
Note that x t j is a functional vector of u k , and the expression δx j /δu k is used, where the notation δx j /δu k is the functional derivatives which describe the response of the functional x j to an infinitesimal change in the function u k at each point.
Proof.The proof of 4.1 is only going to be shown for easily reading.The proof for 4.2 can be found in Appendix.
When j 1, . . ., k − 1, x j j − and x j 1 j are independent of t k , and obviously dx j j − / dt k 0 holds.In the case of j k, x k k − is a function of t k which gives rise to Case i. j k 1 .In this case, x k 1 is a function of t k and x k 1 k , and we have Note that in 4.4 , ∂x k 1 /∂t k is produced by the perturbation of t k , and ∂x k 1 / ∂x k 1 k ∂x k 1 k /∂t k is produced by the perturbation of x k 1 k with respect to t k .Obviously, for τ ∈ k, k 1 , The solution to ∂x k 1 τ /∂x k 1 k is given by Equation 4.6 is a linear system about ∂x k 1 /∂x k 1 k .Define the state transition matrix Φ l τ, v by according to 4.6 , and we have Thus, At transition instants t j , since x j 1 j ψ i j 1 x j j − , so According to 4.9 , and we have

4.15
Theorem 4.2.The derivatives of the objective functional J • w.r.t t k , u k and s k are given, respectively, as follows:

4.16
Before proving Theorem 4.2, Lemma 4.3 is firstly given as follows.L i j x j , u j , s j dτ λ j j − 1 T dx j j − 1 dt k − λ j j − T dx j j − dt k .4.17 Proof.For any j k 2, . . ., K, we have

4.18
Since the following holds by Theorem 3.2,

4.20
Obviously, when j k, k 1, we have Now we prove Theorem 4.2.We are only going to show d J/dt k for easily reading.The proofs for δ J/δu k and δ J/δs k can be found in Appendix.
Proof.J θ, u, s can be formulated as

4.23
Since L i j • is independent of t k for j 1, . . ., k − 1, then d J/dt k can be obtained by

4.25
Due to Theorem 3.2 and 4.10 , d J/dt k can be formulated by

4.26
Note that when second-order derivatives are needed, there is no difficulty to obtain the second-order derivatives following the above procedure.

Parametrization Method
To obtain the numerical solution to optimal control for hybrid systems, continuous control profiles are parameterized on each mode active horizon in this section.Then the numerical solution to optimal controls can be computed based on the obtained sensitivity analysis results.The basic idea behind the proposed method using finite parameterizations of the controls is to transcribe the original infinite dimensional problem, that is, C-problem, into a finite dimensional nonlinear programming problem, that is, P -problem 25 .Here, the parametrization method that the control profiles are approximated by a family of Lagrange form polynomials is implemented.
Partition each horizon j − 1, j into N j elements as j − 1 τ j0 < τ j1 < • • • < τ jN j j where τ jl are referred to as collocation points, l 0, . . ., N j .Let u jl denote the value of u j at τ jl , l 0, . . ., N j .Thus, the control variable u j is represented approximately by a Lagrange interpolation profile for j 1, . . ., K, where l l τ N j m 0,m / l τ − τ jm / τ jl − τ jm .s j is also parameterized by where s jl is the value of s j at the collocation points τ jl , l 0, . . ., N j .As a result, based on the obtained derivatives, the numerical solution of u and θ to optimal control for the hybrid systems can be solved simultaneously and efficiently by adopting gradient-based algorithms as described in Xu and Antsaklis 3 and Egerstedt et al. 6 .Note that the derivatives are functions of costate λ j as formulated in Theorem 4.2.When control polynomial profiles are implemented, a multipoint boundary value problem about state and costate expressed by 3.6 , 3.7 , and 3.10 will be solved, which produces the derivatives.
Although the Lagrange interpolation profiles may cause the state or/and control trajectories violate their constraints, this parameterizations method has been proved useful in practice.Moreover, there are some techniques to decrease the defect 1, 2 .1.The first reactor denoted by mode 1 is fitted with a heating coil which can be used to manipulate the reactor temperature u over time and is initially loaded with 0.1 m 3 of an aqueous solution of component x 1 of concentration 2000 mol/m 3 .This reacts to form components x 2 according to the consecutive reaction scheme 2x 1 → x 2 .After completion of the first reaction, an amount of dilute aqueous solution of component x 2 of concentration 600 mol/m 3 is added instantaneously to the products of with x 0 2000 0 0 T .The system transits once at t t 1 t 0 < t 1 < t f from mode 1 to 2 with

Some Examples
The OCPHS is to find an optimal mode transition instant t 1 and an optimal input 298 ≤ u t ≤ 398, t ∈ t 0 , t 1 , to maximize the cost functional max t 1 ,u x 3 t f , 5.3 with x 3 t f ≥ 150 must be satisfied.
By using the proposed method, the optimal mode transition instant is t 1 105 and the corresponding optimal cost is J * 150.0285.The corresponding continuous control and state trajectories are shown in Figure 2. In Vassiliadis et al. 1 , the transition instants and the optimal cost are t 1 106, J * 150.294, respectively, which are solved by software package DAEOPT.
Example 5.2.Example 5.2 comes from Xu and Antsaklis 3 and is also reconsidered by Hwang et al. 9 .Different from the example in the two references, the control constraint is imposed.The example can be referred to as autonomous switching hybrid systems with mode invariants.Consider the hybrid system consisting of Mode 1: Mode 2: ẋ 0.5 0.866 0.866 −0.5 x 1 1 u, 5.5 with x 0 1 1 T .Assume that t 0 0, t f 2 and the system transits once at t t 1 t 0 < t 1 < t f from Mode 1 to 2 when the state trajectories intersect the linear manifold defined by m x x 1 x 2 − 7 0. Mode 1 is active with its mode invariant x 1 x 2 − 7 < 0 and Mode 2 is active with its mode invariant x 1 x 2 − 7 > 0. The OCPHS is to find an optimal mode transition instant t 1 and an optimal input u t ∈ −1, 1 such that the cost functional is minimized.
By using the method developed here, the optimal mode transition instant is t 1 1.1857 and the corresponding optimal cost is J * 0.1246.The corresponding continuous control and state trajectories are shown in Figure 3.In Xu and Antsaklis 3 , the transition instants and the optimal cost are t 1 1.1624, J * 0.1130, respectively.The bad performance results from that the optimal control is approximated by polynomial.

Conclusions
The optimal control problem for hybrid systems OCPHS with mode invariants and control constraints is addressed under a priori fixed mode transition order.By introducing new independent variables and auxiliary algebraic variables, the original OCPHS is transformed into an equivalent optimal control problem, and the optimality conditions for the OCPHS is stated.Based on the optimality conditions, the derivatives of the objective functional w.r.t control variables, that is, mode transition instant sequence and admissible continuous control functions, are established analytically.As a result, a control vector parametrization method is implemented to obtain the numerical solution by using gradient-based algorithms with the obtained derivatives.Compared with the existing results on the OCPHS with fixed mode

A.15
Obviously, when j k, we have

A.16
Proof of δ J/δu k in Theorem 4.2.J θ, u ε , s can be rewritten by ϕ i j i j 1 x j j − . A.17 Applying a δ-operation to A.17 leads to δ J d J ρ, u ε , s dε ε 0 λ j j − T δx j j − − λ j 1 j T δx j 1 j − ∂ϕ i j i j 1 ∂x j j − δx j j − − λ K K T δx K K .

A.18
Due to Theorem 3.2 and A.9 , δ J can be reformulated by

A.19
Then according to the definition of functional derivative, we have Obviously, the functional derivative of J with respect to s k can be directly given by This completes the proof.

Remark 4 . 4 .
Control variable u j can be approximated by several piecewise Lagrange interpolation profiles by further partitioning the element j − 1, j .More detail of the parameterizations methods can be found in Vassiliadis et al. 1, 2 , Kameswaran and Biegler 26 , and the references therein.Only one Lagrange interpolation profile is used here to show the process of the proposed method.

Example 5 . 1 .
To illustrate the effectiveness of the developed method, two examples with different situations are presented in the following.Numerical examples are conducted on an ThinkPad X61 2.10-GHz PC with 2G of RAM.The program is implemented using MatLab 7. The order of Lagrange polynomials in the examples is 3.The prototype of this example comes from Vassiliadis et al. 1 .The hybrid system consists of two batch reactors as shown in Figure

Figure 2 :
Figure 2: State trajectories and control input of Example 5.1.

Since the following holds by Theorem 3
dτ λ j j − 1 T δx j j − 1 − λ j j − T δx j j − .

1 L i j x j , u j , s j dτ k k− 1 L i k x k ε , u k ε , s k dτ K j k 1 j j− 1 L
i j x j ε , u j , s j dτ K−1 j 1 Engineered systems, such as chemical engineering systems and powertrain systems of automobiles, always undergo multiple modes which are represented by a discrete state i taking values from set I .
{1, 2, . . ., M} and pose hybrid characters.The evolution of discrete state i is determined by mode transition sequence.A mode transition sequence schedules the sequence of active modes i j , i j ∈ I and is a sequence of pairs of t j−1 , i j , which can be defined by { t 0 , i 1 , t 1 , i 2 , . ..} .θ, π where θ .{t 0 , t 1 , . ..} and π .{i 1 , i 2 , . ..} are referred to as