An Approximations Based Approach to Optimal Control of Switched Dynamic Systems

Our paper deals with a new computational approach to optimal control design for a class of switched systems.The control strategy we propose is based on the conventional proximal point method applied to a specific optimal control problem (OCP) with switched dynamics. The class of OCPs under consideration is widely applicable in optimization of real-world electronic systems. We create constructive approximations for the initially given sophisticated OCP, establish numerical stability (consistency) of the resulting algorithm, and develop an optimal control strategy. We finally discuss some implementability issues, study an illustrative example, and also point to possible generalizations of the elaborated control design in the context of nonlinear hybrid systems.

In the recent years there has been a revival of efficient optimization techniques developed for various types of nonlinear OCPs (see, e.g., [4,10,14,21]).This fact is due to the valuable progress in the area of computational engineering and applied computer sciences.Nowadays the most powerful numerical approaches to switched and hybrid OCPs are based on the so-called "optimality zone" algorithms developed in [9,17] and on the first-order techniques (see [6,12,18] and the references therein).In this paper we study a specific class of switched systems and apply the classic proximal point method from [26] in the corresponding optimal control design procedure.Our contribution proposes a computationally consistent control technique that can be considered as a significant simplification of the celebrated "gradient descent" method (see [6,12,15,16,18,24]) in the particular case of a linear switched-mode system.Recall that the conventional gradient descent approach from [12,24] generates an approximating sequence that possesses the property of a minimizing sequence.In the specific case of a linear switched-mode system we advance the existing convergence result related to the gradient descent method and establish a strong convergence of the minimizing sequence generated by the proposed optimization algorithm.The computational approach we develop involves an auxiliary relaxation procedure associated with the initially given dynamic model.After the basic convexity property of the OCP under consideration is established, we apply a combination of the proximal point regularization algorithm and the usual convex programming techniques [27,28].
The remainder of our contribution is organized as follows: Section 2 contains some necessary theoretical facts.In Section 3 we formulate the initial OCP and introduce the associated relaxation scheme.Section 4 proposes a conceptual numerical approach to the OCP governed by the specific linear switched systems.We establish convexity of the resulting OCP and apply the proximal based algorithm in combination with the convex optimization techniques to the controller design for the given system.Section 5 establishes the algorithmic consistency of the control algorithm we developed and discusses shortly some possible generalizations of the proposed approach.Section 6 contains an illustrative example.Section 7 summarizes the paper.

Mathematical Preliminaries
Let us start by introducing the auxiliary convex minimization problem minimize  () , where  is a real Hilbert space,  ⊂  is a bounded, convex, and closed set.By  :  → R := R ⋃{∞} we denote here an objective functional that is assumed to be proper convex.We also assume that (⋅) is bounded on the set  + , where  is an open unit ball in  and  > 0. Clearly, Let us make the following simple observation: the boundedness of (⋅) implies continuity of this functional on the given set  + .A proper convex, lower semicontinuous functional (⋅) attains its minimum on the bounded, convex, closed set  [28].Thus the existence of an optimal solution  opt ∈  to problem (1) is guaranteed.Note that in general a convex function on a convex subset of an infinite dimensional topological vector space does not need to be continuous on the interior of its domain.The nonempty set of all optimal solutions to (1) is denoted by .
We next introduce the classical proximal mapping where  > 0,  ∈ , and consider the proximal point iterations (see [23]) where {  }, 0 <   ≤  < ∞ is a given sequence.The original problem ( 1) is now replaced by the sequence of the following auxiliary minimization problems: with the augmented strong convex objective functionals.We now suppose that elements of {   },  = 0, 1, . . .satisfy the additional condition for all  = 0, 1, . ... In that case {   } constitutes a numerically stable minimizing sequence, namely, a sequence that converges in the weak topology to an optimal point  opt ∈  (see [26]).Note that the strong convergence property of some modifications of the classical proximal point sequence generated by (5) can also be proved.We refer to [21] for the corresponding analytic results.
The proof of Proposition 1 can be found in [28].
We also assume that all functions   ,  ∈ Q, are continuously differentiable and the corresponding derivatives are bounded.Elements of F do not contain any conventional control parameter.An input of a SSC under consideration is given as a sequence  of switchings such that the length of this sequence is equal to  ∈ N. In fact an admissible sequence of switchings  determines a partitioning of the given time interval [0,   ] by the adjoint subintervals [ −1 ,   ) associated with every locations   ∈ Q,  = 1, . . ., .A switched control system from Definition 2 remains in a location   ∈ Q for all  ∈ [ −1 ,   ).The dynamic behaviour of the SCC in every location is given by the following differential equation: for almost all times  ∈ [ where The specific input variable (control input) in ( 8) can now be written as follows: A set B of all admissible vectors (⋅) is characterized by the natural conditions Note that (⋅) from B does not depend on the trajectory (⋅) of the given SSC.Evidently, B is in one-to-one correspondence to the set of all admissible sequences  from Definition 2. Since trajectory (⋅) of a SSC is a continuous function, our concept describes a class of dynamical systems without any impulse component.Therefore, we have (  ) = ( +1 ) for all  = 1, . . .,  − 1.Let us additionally refer to [1,10,12,17] for some alternative classes of switched and hybrid systems.
It is necessary to stress that dynamic models of the type (8) are widely used in analysis and prototyping of many realworld electronic systems.Let us refer to [3,8,9,12] for some concrete examples.Moreover, the switched model ( 8) provides an adequate theoretic framework for control design of various electric power converters (see, e.g., [19,29] and the references therein).
Given a SSC, we are now in position to introduce the main OCP as minimize  ( (⋅)) := ∫ Here  0 : R  → R is a continuously differentiable function.Evidently, (11) can be interpreted as a problem of a dynamical switchings' optimization.Note that [14] contains a similar optimal control problem formulation.Throughout this paper we assume that problem (11) has an optimal solution  opt (⋅) ∈ B. The corresponding optimal trajectory is next determined as  opt (⋅).
The initial OCP (11) represents an optimization problem with a specific form of the admissible control set B. Evidently, B ⊂ L 2  (0,   ), where L 2  (0,   ) is the standard Lebesgue space of -dimensional square integrable functions.In that situation the celebrated Pontryagin maximum principle (see [30]) cannot be applied directly to problem (11).Recall that the necessary optimality condition in the form of the Pontryagin maximum principle provides an adequate (also in a numerical sense) solution procedure in the full space of measurable control functions (the space L 2  (0,   )).A possible direct application of the conventional maximum principle to (11) does not guarantee admissibility of the obtained optimal solution, namely, the admissibility condition  opt (⋅) ∈ B. This fact is an immediate consequence of the following simple observation: the set of admissible needle variations of a characteristic function  [ −1 ,  ) (⋅) is very "poor." Consequently, in that case it is impossible to derive the adjoint equations associated with the conventional Pontryagin maximum principle (see also [14,22] for theoretical details).The same theoretic observation is also true with respect to a hybrid version of the maximum principle (see [9][10][11]22]).
One of the possible computational approaches to ( 11) is based on a suitable relaxation procedure.Let us reformulate the initial OCP (11) in the full space, namely, as a restricted dynamic optimization problem in L 2  (0,   ) as minimize  (V (⋅)) := ∫ where V(⋅) := (V 1 (⋅), . . ., V  (⋅))  is a new (auxiliary) square integrable control variable, Ψ(⋅) := (  1 (⋅), . . .,    (⋅))  , and  is a small positive number.Note that the system of control restrictions selects the characteristic functions (by V  = 0 or V  = 1).The system of inequalities in (12) corresponds to an approximating (also relaxed) solution procedure associated with the original OCP (11).In that case  > 0 can be interpreted as a usual numerical accuracy of the selected computational algorithm.The OCP ( 12) is in fact a relaxation of the initial problem (11).This relaxed problem provides a main computational basis for the optimal control design of the original switched system SSC in the sense of OCP (11).
It can be constructively solved by using some conventional optimization techniques.

Optimal Linear Switched Mode Dynamics
The theoretical and numerical investigations of the relaxed OCPs of the type (12) involve a question of general interest.Let V opt (⋅) ∈ L 2  (0,   ) be an optimal solution of (12).Is it possible to use V opt (⋅) constructively with the aim to construct an adequate approximation of the optimal vector  opt (⋅) ∈ B from the initial problem (11)?In this and in the next sections we give a positive answer to the above question and also present a concrete approximation procedure in the special case of linear switched-mode system (8).
Let us make some additional technical assumptions.We now consider the linear variant of F with   () =   , where   ∈ R × for all  ∈ Q. Assume that every component of all matrices   is positive.Recall that such matrices are called Metzler matrices (see e.g., [21]).The ensuing analysis is also restricted to a monotonically nondecreasing, convex function  0 (⋅).In that specific case the control system in (12) has the following linear form: where Ψ() := (  1 , . . .,    )  is an ordered set of given matrices.Consider now the initial and relaxed OCPs under the technical assumptions given above and introduce the following convexity concept from [13].
Definition 3. If the relaxed OCP ( 12) is equivalent to a convex optimization problem (1), then we call (12) a convex OCP.
Definition 4. Let  V (⋅) be an absolutely continuous solution to the relaxed control system ( 14) generated by an admissible control input V(⋅).Then ( 14) is called a convex control system if every functional is convex.
The proof of this result can be found in [13, Theorem 1, Page 994].We also refer to [21] for some related facts.We now show that under the above additional assumptions the corresponding variant of the relaxed OCP ( 12) is equivalent to a specific convex OCP (see Definition 3).Theorem 6. Assume that the above additional conditions are satisfied.Then the relaxed OCP ( 12) associated with the corresponding linear switched system is equivalent to the convex OCP of the following type: where ,  = 1, . . .,  and  1 <  2 are solutions of the simple quadratic equation V 2 − V +  = 0 for a small enough  > 0.
Proof.The right-hand side of the linear switched system ( 14) is convex (linear) with respect to  := (V, ).From Proposition 5 it follows that ( 14) is a convex control system in the sense of Definition 4. It means that every   (V(⋅)) is a convex functional for every  = 1, . . ., .From Proposition 1 we next deduce the convexity of the function  0 ( V ()) for all  ∈ [0,   ] and also the convexity property of the objective functional (V(⋅)).Let V 1 (⋅), V 2 (⋅) be admissible controls in the sense of the OCP (18).Then for a convex combination where  ∈ (0, 1), we obtain Thus the equality constrain in (18) involves a convex subset of the basic Hilbert space L 2  (0,   ).The linear inequalities constraints from (18) also determine a convex subset of L 2  (0,   ).Since an intersection of a finite number of convex sets constitutes a convex set, problem (18) for  = 1, . . .,  is a convex OCP.Observe that the system of the original inequality/equality constraints in ( 12) is equivalent to the system of inequalities from (18) for all  = 1, . . ., .We can conclude that the relaxed OCP ( 12) is equivalent to the convex OCPs of the type (18).The proof is completed.
Problem (18) from Theorem 6 is equivalent to the abstract convex program (1) discussed in Section 2. Therefore, we are now in position to apply the proximal point algorithm and write the regularized variant of the OCPs (18) minimize   (V (⋅)) :=  (V (⋅)) where ,  = 1, . . ., .Here V  (⋅) is an admissible -iteration of the proximal point method and V  (⋅) is an admissible initial control.As mentioned in Section 2, a sequence {V  (⋅)} of optimal solutions to ( 22) is a numerically stable (weakly convergent) minimizing sequence in the sense of problem (12).This sequence converges in the weak L 2  (0,   )-topology to an optimal solution V opt (⋅) of (18).
We now have derived the final representation of the approximating OCP, namely, problem ( 22) that provides a necessary basis for the computational control design associated with a SSC from Section 3.This regularized OCP (22) can be solved by a standard numerical convex optimization technique.It is necessary to stress that ( 22) is a convex optimization problem with a strictly convex functional   (⋅).Therefore, {V  (⋅)} is uniquely defined.In this case (22) can be solved by a first-order numerical procedure, for example, by a suitable gradient-type algorithm (see [6,18,27]).The gradient ∇  (V(⋅)) of   (V(⋅)) in that case can be computed as follows: where is the Hamiltonian associated with (18) and (⋅) is a solution of the corresponding system of adjoint equation (see [30] for details) Evidently, in the case of the above SSC of linear type we have Finally let us note that the proximal point method for solving the regularized OCP ( 22) can also be combined with an effective second-order numerical optimization procedure for convex programming.This possible combination involves an alternative numerical procedure for the optimal control design of switched systems under consideration.

Strong Convergence Property of the Proximal Based Algorithm
In this section we consider the proposed numerical approach to the initial OCP (11) in the specific case that was indicated by additional assumptions given in Section 4. Our aim is to study the convergence property of the sequence {V  (⋅)} generated by the proximal regularized problems (22) and the relationship to the original optimal solution  opt (⋅) of ( 11).From the qualitative point of view, the numerically stable (proximal based) optimal solutions to ( 18) with a "small enough" parameter  > 0 represent a well-determined approximation of the optimal vector of characteristic functions in (11).We now will express this qualitative statement more precisely.First, let us note that the measurable function (⋅) (see Section 3) takes its values in the set ver(  ) of vertices of the following -dimensional simplex: We now give the constructive geometric characterization of the weak convergence of a sequence {  (⋅)},  ∈ N from   .
We refer to [21] for the complete proof of this auxiliary geometrical result.Theorem 7 can be interpreted as follows: the L 2  (0,   )-weak convergence to a characteristic function coincides with the strong convergence.Note that this fact is true due to specific type of functions under consideration, namely, for the concrete characteristic functions we study.Using this abstract result, we are now able to formulate our (improved) convergence result for the proposed proximal point based algorithm.Theorem 8. Let all the additional conditions from Section 4 be satisfied.Consider a sequence {V  (⋅)} of solutions to (22).Then this sequence converges strongly (in the norm topology of L 2  (0,   )) to a solution   (⋅) of the initial OCP (18).
Proof.From the standard properties of the classical proximal point method in a real Hilbert spaces, we deduce the weak convergence of {V  (⋅)} to an optimal solution of (18).Using the equivalency of problems ( 18) and ( 12) established in Theorem 6, we conclude that {V  (⋅)} converges weakly to an optimal solution of ( 12), namely, to an admissible function Recall that  opt (⋅) is an optimal solution to the original OCP (11).The above fact also follows from Theorem 6.It is easy to see that for the decreasing  the solution  1 from this theorem also decreases.The solution  2 increases for the decreasing .This means that the diameters of the convex sets that are determined by the corresponding inequalities constraints in Theorem 6 decrease.This observation and the equivalency relation between problems ( 18) and ( 12) imply relation (29).Evidenlty, a combination of the weak and strong (given by ( 29)) convergent sequences generates a weakly convergent sequence.Therefore, the control sequence {V  (⋅)} converges weakly to  opt (⋅).Finally, from the abstract Theorem 7 we deduce the strong convergence of the proximal point sequence {V  (⋅)} to an optimal solution  opt (⋅) of the initial OCP (11).The proof is finished.
Theorem 8 gives a conceptual response to the question formulated at the beginning of Section 4. The sequence {V  (⋅)} generated by the proximal point algorithm for (18) and ( 12) converges in the strong sense (in the sence of the usual L 2  (0,   )-norm) to an optimal solution of the main OCP (11).This theoretical fact is of fundamental importance for the corresponding computational control design procedure.The strongly convergent algorithm from Theorem 8 makes it possible to use the proximal point techniques in several computational approaches to the switched OCPs of the type (11).As mentioned in Section 1 the numerical method we propose can also be considered as a specific case of the conventional gradient descent method (see [6,12,15,16,18,24]) applied to a linear switched-mode system.In general, the gradient descent method generates only a minimizing sequence for the associated switched-type OCP.We refer to [12,24] for the corresponding theoretical results.In that context Theorem 8 establishes strong convergence properties of the optimization method we developed in comparison to the "weak" convergence of the approximating sequence obtained by a gradient descent algorithm application [12,24].
Note that the additional linearity assumptions from Section 4 are too restrictive.Otherwise, it is also possible to use the proximal point techniques and the associated convex programming methodology in combination with some classical linearization procedures associated with an initially given nonlinear SSC.Recall that several linearization schemes have been long time recognized as a powerful tool for solving sophisticated control design problems (see, e.g., [21,27]).Note that a linearization based approach to dynamic optimization can usually be considered only as a first-step auxiliary solution procedure.In the case of a general nonlinear OCP (11), a possible linearized optimization problem related to the corresponding relaxed problem (12) can be written as follows: minimize Ĵ ( (⋅)) , subject to LSSC, where V(⋅) ∈ D is a fixed control function and the set D of admissible control inputs V(⋅) is given by the usual conditions By LSSC in (30) we denote an adequate linearized dynamics of the original control system from (12).For instance, a suitable LSSC can be given by the following linear dynamics (see [21]): where (⋅) is a state of the linearized system and x(⋅) is a "reference" trajectory (for the linearization procedure) of the original SSC (8).The objective functional Ĵ(⋅) in ( 30) can be assumed to be linear where  V+ (⋅) denotes a solution of the above-mentioned linearized dynamical system generated by the control function V(⋅) + (⋅).The linear nature of the obtained approximating problem (30) implies the convexity property in the sense of Definition 4 such that the dynamic optimization problem (30) constitutes a convex OCP.As mentioned above one can apply the proximal based approach in combination with appropriate techniques from the convex programming to the corresponding sequence of the linearized OCPs of type (30).

A Numerical Example
We now apply the OCP-based control algorithm developed in previous sections to optimization of a practically motivated engineering system, namely, to optimal control of an electric AC/DC converter.
Example 1.Consider the simplified electric scheme of a three-phase circuit associated with an AC/DC converter (Figure 1).The control system is characterized by the switched dynamics of the admissible control inputs   () ∈ {−1, 0, 1}, where  = , , .The corresponding state variables represent the initially given AC currents   (),  = , ,  where  = , , .By   () we denote here the voltage of the th source.The resulting DC voltage of the converter is given by the expected quasiconstant voltage   () > 0. We next put  = 10 −3  and  = 10 Ω for the given inductance and resistance, respectively.Moreover, we put Note that we have assumed that the frequency of the electrical network under consideration is equal to 60 Hz.Moreover, the minimal length (switching period) of the time interval Evidently, (36) can be easily interpreted as a best converting performance requirement under all possible switching strategies.Applying the proximal regularization and the gradient based algorithm from Section 4, we now compute the optimal switching control inputs  opt  (⋅) for  = , ,  as solutions of (36).The computed optimal switching strategy is presented in Figure 2.
The corresponding optimal trajectory (the resulting output DC currents) is shown in Figure 3.
Note that the "output" currents (after converting) presented in Figure 3 are, as expected, the DC-type currents.The obtained optimal switching strategy  opt  () constitutes a high frequency function (see Figure 2).Moreover, the optimal currents satisfy the natural algebraic condition, namely, the fundamental junction rule for the given AC/DC circuit   () +   () +   () = 0. (37) Using (37) and (38) in combination with (34), we next calculate the associated optimal switchings  opt  (),  opt  (), and the resulting optimal states (see Figure 2).Finally note that the numerical results presented above are obtained using the authors program and the standard MATLAB packages.

Concluding Remarks
In this paper, we have shown that the proximal point approach can be used for numerically tractable approximations of some OCPs associated with switched-mode dynamics.An abstract proximal regularization scheme in combination with a specific relaxation approach and extended by an effective convex programming algorithms makes it possible to establish stable convergence properties of the resulting algorithm.The iterative updates of the approximation level we proposed are based on an increasing sequence of "embedded" consistent approximations of the relaxed problem.The stable regularization properties of the proximal point method and a specific analytical formalization of the switching structure guarantee the strong convergence of the obtained numerical scheme that contains the changeover from one specific infinite-dimensional control space to another.
The above-mentioned numerically stable specification of the proposed approximations constitutes a new analytical approach to practical optimization of switched-mode dynamical systems with controllable location transitions.Note that the proximal point regularization can be combined with diverse numerical procedures for a concrete computational treatment of the auxiliary relaxed problem, for example, with an adequate modification of the conventional gradient based method and also with some second-order optimization procedures.
Finally note that the proposed numerical approach and the corresponding theoretical results can be applied to some alternative classes of optimal control processes governed by switched or by some specific hybrid dynamic systems.