Calculus of Variations and Nonlinear Optimization Based Algorithm for Optimal Control of Hybrid Systems with Controlled Switching

This paper investigates the optimal control problem of a particular class of hybrid dynamical systems with controlled switching. Given a prespecified sequence of active subsystems, the objective is to seek both the continuous control input and the discrete switching instants that minimize a performance index over a finite time horizon. Based on the use of the calculus of variations, necessary conditions for optimality are derived. An efficient algorithm, based on nonlinear optimization techniques and numerical methods, is proposed to solve the boundary-value ordinary differential equations. In the case of linear quadratic problems, the two-point boundary-value problems can be avoided which reduces the computational effort. Illustrative examples are provided and stress the relevance of the proposed nonlinear optimization algorithm.


Introduction
The optimization of hybrid dynamical systems has been widely investigated in the last years [1][2][3][4][5][6][7] because such systems can be used to model a wide range of real-world processes in many application fields, such as automotive systems, communication networks, chemical processes, robotics, airtraffic management systems, automated highway systems, embedded systems, and electrical circuit systems, etc [8][9][10][11].The focus of this paper is on the optimal control of a particular class of hybrid dynamical systems called switched systems.The behaviour of interest of such systems is described by a set of time-driven continuous-state subsystems and a switching law specifying the active subsystem at each time instant.A switching happens when an event signal is received.This signal may be an external generated signal or an internal signal generated if a condition on the time evolution, the states, and/or the inputs is satisfied.Consequently, we call a switching triggered by an external event an externally forced switching.So, according to the nature of the switching signal, switched systems may be classified into switched systems with externally forced switching or switched systems with internally forced switching.In the last decade the switched systems have been extensively studied [12][13][14][15][16] and the corresponding optimal control has never been as relevant as it is nowadays.Due to its significance in theory and applications many theoretical results and numerical algorithms have appeared in the literature [16][17][18][19][20].Most of the available theoretical results are concerned with the study of necessary and/or sufficient conditions for a trajectory to be optimal by means of the Pontryagin maximum principle [21,22], the dynamic programming approach [23], or the calculus of variations [24].See [25] for a brief survey on recent progress in computational methods of the optimal control of switched systems.Modeling switched system depends on the different dynamics of its subsystems described by indexed differential or difference equations.Otherwise, if there is no external control influence on the system, we call it an autonomous switched system.To address the optimal control problem of autonomous switched systems, we have to focus on deriving the optimal sequence of switching times, and even if this sequence is the sole control influence, its determination remains a challenging 2 Complexity task.In previous work [10], we have investigated the optimal control problem for autonomous switched systems with autonomous and/or controlled switches.The obtained results were considered to solve a time-optimal control problem for a nonlinear chemical process subject to state constraints.For nonautonomous switched systems, it is necessary to consider the continuous input together with switching times and sequences.Similar to the optimization for autonomous switched systems, optimization techniques are needed to find the optimal solutions for nonautonomous switched systems.But, despite the relevant contributions to find numerical solutions to such problems by employing the established theoretical conditions, effective algorithms still remain to be developed.Inspired by what is cited above, the main contribution of this paper is to solve the optimal control problem for nonautonomous switched systems with controlled switching.So, given a prespecified sequence of active subsystems, the objective is to seek both the continuous control input and the discrete switching instants that minimize a performance index over a finite time horizon.By using the calculus of variations, we derive necessary conditions for optimality.A computational algorithm, based on nonlinear optimization techniques and numerical methods for solving boundaryvalue ordinary differential equations, is then proposed.We also show that, in the case of linear quadratic problems, we can avoid dealing with two-point boundary-value problems and therefore reduce the computational effort.This paper is organized as follows.In Section 2, the description of the studied switched systems is introduced and the corresponding optimal control problem is formulated.The main results of nonlinear optimization problem are presented in Section 3.An algorithm for computing the optimal continuous control input as well as the full information of the switching sequence and time instants is also proposed.In Section 4, we focus on quadratic optimal control problems for linear switched systems.Numerical simulations are performed in Section 5 to demonstrate the efficiency of the derived results and algorithm.Finally, some concluding remarks and suggested future work are given in Section 6.

Problem Statements and Preliminaries
In this paper, we focus on continuous-time switched systems whose dynamics are described for  ∈ [ 0 ,   ] by where  ∈ R  is the continuous-state vector,  ∈ R  is the continuous control input vector, {  : with where  1 , . . .,   are the switching instants and   is the number of the active subsystems during the time interval [  ,  +1 ).
The optimal control problem considered in the present paper can be formulated as follows.

Problem 1.
Given a continuous-time switched system whose dynamics are governed by ( 1) and ( 2) for a fixed time interval [ 0 ,   ], the objective is to find the continuous control  * and the switching instants  *  that minimize the quadratic performance index with ,   ,   , and  are symmetric matrices with   ≥ 0,   ≥ 0,  ≥ 0, and   > 0.   is a desired trajectory over [ 0 ,   ] and   are costs associated with the switches.In order to solve this problem, we will resort to the calculus of variations.

Basic Concepts.
Based on the optimal control problem of continuous systems and the calculus of variations, the variational problem can be stated as follows.Let  be a family of trajectories () defined on some interval [ 0 ,   ] by The problem is to find  * () ∈  that minimize a given cost functional defined as where  and  are real-valued continuously differentiable functions with respect to their arguments.In order to solve the above problem, we need to express the variation of the cost functional , denoted by , in terms of independent increments in all of its arguments.The optimal trajectory is then characterized by imposing the stationary condition  = 0. Let q() = ()+() be a neighboring perturbed trajectory of (), evolving in the time interval as illustrated in Figure 1. 0 ,  0 ,   , and   are small changes in the trajectory at the initial and final instants.The variation of  is given by From the calculus of variations, we can obtain the expression of  as If we introduce the Hamiltonian function () and the conjugate moment () as defined below the variation of  may be rewritten in the following form: Setting to zero the coefficients of the independent increments ,  0 ,  0 ,   , and   yields necessary conditions for a trajectory to be optimal.The obtained results will be used in the following section to solve the hybrid control problem.

Main Results
In this section, we will consider the case of a single switching, but the proposed approach and used methods can be straightforwardly applied to the case of several subsystems and more than one switching.The first problem is then reduced to the following problem.

Problem 2.
A continuous-time switched system is given whose dynamics are governed by where  0 ,  0 , and   are fixed.Find the continuous control  * and the switching instant  * 1 that minimize the quadratic performance index with Complexity   , ,   , and  are symmetric matrices with   ≥ 0,  ≥ 0,  ≥ 0, and   > 0. To solve this problem, we introduce a costate variable () ∈ R  also called Lagrange multiplier to adjoin the system subject to constraints (15) to  (16).The augmented performance index is thus which can be written as with Note that the performance index J (19) is a sum of two cost functionals having the same form as (10) with () = [() ()]  .The variation  J can therefore be obtained by using the results developed in Section 2.2.According to (13), the Hamiltonian function () and the conjugate moment () are expressed by Using ( 14), we can write Since it follows that According to the Lagrange theory, a necessary condition for a solution to be optimal is  J = 0. Setting to zero the coefficients of the independent increments (), (),  1 , , and   yields the costate equation defined as the gradient of the cost functional with respect to u and the gradient of the cost functional with respect to the switching instant  1 Taking into account ( 17) and ( 21), the costate equation ( 26) is rewritten The gradient of the cost functional with respect to u (27) will be described by and the gradient of the cost functional with respect to the switching instant  1 (28) will be expressed as Considering linear controlled systems, we get from (30) Substituting (32) into the costate equation ( 29) and the state equation ( 15) yields the Hamiltonian system expressed by To determine the hybrid optimal control ( * ,  * 1 ), we have to solve (33) and (31).Analytical resolution of the above equations is a difficult task, so we need to resort to the following: (i) Numerical methods for solving boundary-value ordinary differential equations: the Hamiltonian system consists of two boundary-value ordinary differential equations whose solutions must satisfy conditions specified at the boundaries of the time intervals [ 0 ,  1 ) and ( 1 ,   ].To find these solutions, we can use, for example, the shooting method [26] consisting in replacing the boundary-value problem by an initialvalue problem.More details about this method will be further provided. (ii) Nonlinear optimization algorithms: to locate the optimal switching instant  * 1 , we shall use nonlinear optimization techniques which are abundant in the literature [20,27,28].These methods allow finding the instant  1 that satisfies the stationary condition (31).
Therefore, the hybrid optimal control ( * ,  * 1 ) can be found by the implementation of the algorithm detailed in the following subsection.

Algorithm. See Algorithm 1.
Remark 1.Note that, at each iteration k, we have to solve a boundary-value problem to find the continuous control for a fixed switching instant.Numerical methods used for solving such problem are generally iterative, which may lead to heavy computational time.
(54) Substituting ( 42) and ( 51) into (35), we get Note that  * is an affine state feedback.The closed-loop system is therefore governed by In order to compute the optimal continuous control for a fixed switching instant, we need to solve the matrix Riccati equations ( 46)-( 52) and the auxiliary equations (47)-(53).The latter are integrated backward in time to get the matrices () and ().The hybrid optimal control is then determined by the implementation of Algorithm 1 with Step 2 modified in Algorithm 2.

Simulation Results
To illustrate the validity of the proposed result and the efficiency of the algorithms, two examples are considered in this section.The former concerns the optimization of a nonlinear switched system.The latter deals with quadratic optimal control problem for linear switched system.The computation was performed using MATLAB 6.5 on a Celeron 2 GHz PC with 256 Mo of RAM.

First Illustrative Numerical
Example.Consider a nonlinear switched system described by Subsystem 1

Conclusion
Based on nonlinear optimization techniques and numerical methods for solving boundary-value ordinary differential equations, we proposed an algorithm for solving optimal control problems for switched systems with externally forced switching.We assumed that the switching sequence is fixed, and therefore the control variables are only the continuous control input and the discrete switching instants.The effectiveness of the presented algorithm was demonstrated through simulation results.The obtained results will be extended to the optimal control of interconnected switched systems.Otherwise parametric uncertainties and input disturbances are often present in real-life applications.So, analysis procedures and control synthesis algorithm for hybrid systems if additive disturbances and/or parametric uncertainties are present are topics that are starting to deserve the attention of researchers [5].Indeed, uncertainty in hybrid system can be present in the vector fields describing the flow of the system and/or in the switching transition law.It can be of parametric nature or caused by time-varying perturbations of the vector field, switching delays.Thus, the robustness analysis will be investigated and can be handled in our future works.