Optimal Control of Switching Topology Networks

We present an extension of a previously proposed approach based on the method of moments for solving the optimal control problem for a switching system considering now a continuous external input. This method is based on the transformation of a nonlinear, nonconvex optimal control problem, into an equivalent optimal control problemwith linear and convex structure, which allows us to obtain an equivalent convex formulation more appropriate to be solved by high-performance numerical computing. Finally, the design of optimal logic-based controllers for networked systems with a dynamic topology is presented as an application of this work.


Introduction
The design of optimal logic-based controllers, while satisfying physical and operational constraints, plays a fundamental role in modern technological systems such as networked dynamic systems, where the information exchange is modeled via a communication graph.In this work, the communication graph has a dynamic topology which can be used to model asynchronous consensus problems [1,2].Networked systems with a dynamic topology are known as switching networks [1,3].The switching network is modeled as a switching control system.In general, switched control systems are characterized by a set of several continuous state dynamics with a logic-based controller which determines simultaneously a sequence of switching times, a sequence of modes, and a continuous external input.In the last years, several researchers have considered the optimal control of switched systems (see [4] and references therein).However, it is widely perceived that the best numerical methods available for switched optimal control problems involve mixed integer programming (MIP) [5,6].Even though great progress has been made in recent years in improving these methods, the MIP is an NP-hard problem, so scalability is problematic.
One solution for this problem is to use the traditional nonlinear programming techniques such as sequential quadratic programming (SQP) which reduces dramatically the computational complexity over existing approaches [7].However, the development of computational efficient tools to these problems is still an important research area.
Recently, an alternative approach to solve effectively the optimal control problem for an autonomous (without an external input) nonlinear switched system based on probability measures has been presented in [4].Following this previous work, we present in this paper an extension to include an external continuous input which presents new theoretical results.This extension is done by considering the set of probability measures associated with the set of both control variables, that is, switching signal and continuous input.The main contribution of this paper is twofold.First, the natural extension including a continuous external input into the control problem opens several possibilities for more real and more complex applications.Secondly, the dynamic network consensus problem is treated by the proposed approach.The proposed approach is based on the fundamental concepts of the theory of moments introduced for global optimization 2 Mathematical Problems in Engineering with polynomials in [8,9] and later extended to nonlinear 0-1 programs using an explicit equivalent positive semidefinite program in [10].We also use some results recently introduced for optimal control problems with the control variable expressed as polynomials [11][12][13].The moment approach for global polynomial optimization based on semidefinite programming (SDP) is consistent as it simplifies and/or has better convergence properties when solving convex problems.This approach works properly when the control variables (i.e., the switching signal and the continuous external input) can be expressed as polynomials.Essentially, this method transforms a nonlinear, nonconvex optimal control problem (i.e., the switched system) into an equivalent optimal control problem with linear and convex structure which allows us to obtain an equivalent convex formulation more appropriate to be solved by high-performance numerical computing.In other words, we transform a given controllable switched system into a controllable continuous system with a linear and convex structure in the control variables.
This paper is organized as follows.In Section 2 we present the problem statement and the basic concepts.The main contribution based on the method of moments is developed in Section 3. Some applications of the proposed method for networks with switching topology are presented in Section 4, and finally in Section 5 some conclusions are drawn.

Switching Optimal Control Problem.
A switched optimal control problem can be stated in a general form as follows.Given the switched system in (2) and a Bolza cost functional , the switched optimal control problem is given by min (),()  ( 0 ,   ,  () ,  () ,  ()) = ∫ subject to the state equation where () is the state,   : R  × R   → R  is the th vector field, ( 0 ) =  0 are fixed initial values, () ∈ R  is the continuous control function, and  : [ 0 ,   ]  → Q ∈ {0, 1, 2, . . ., } is a piecewise constant function of time, with  0 and   as the initial and final times, respectively.Every mode of operation corresponds to a specific subsystem ẋ () =   ((), ()), for some  ∈ Q, and the switching signal  determines which subsystem is followed at each point of time into the interval [ 0 ,   ].The control input  is a measurable function.In addition, we consider a non-Zeno behavior; that is, we exclude an infinite switching accumulation point in time.Finally, we assume that the state does not have jump discontinuities.Moreover, for the interval [ 0 ,   ], the control functions must be chosen so that the initial and final conditions are satisfied, and the running switched costs  () : R + × R   → R are continuously differentiable for each  ∈ Q.
Additionally, we assume that each function   and   can be expressed as polynomials in the control variable .In general we have the functionals as and the state equations as where  is the maximum degree of the polynomials in terms of , setting in zero the coefficients of monomials of degree less than , and   () and   () are the coefficients in  related to the respective polynomial term in   .General conditions for the subsystems functions should be satisfied.
Assumption 1.The nonlinear switched system satisfies growth, Lipschitz continuity, and coercivity qualifications concerning the mappings   : R  × R   → R  and   : R  R  × R   → R to ensure existence of solutions of (2).
The switching optimal control problem can have the usual variations of fixed or free initial or terminal state, free terminal time, and so forth.Definition 2. A control for the switched system in (2) is a triplet consisting of (a) a finite sequence of modes, , (b) the optimal value for the continuous external control, , (c) a finite sequence of switching times such that  0 <  1 < ⋅ ⋅ ⋅ <   =   .

A Continuous Polynomial Representation.
A continuous nonswitched control system can be obtained from (2) as it has been shown in [14].The polynomial expression in the control variable able to mimic the behavior of the switched system is developed using a variable V which works as a control variable.
A polynomial expression in the new control variable V() can be obtained through Lagrange polynomial interpolation quotients defined as in [15] and a constraint polynomial set defined by . Consider a switched system of the form given in (2).There exists a unique continuous state system with polynomial dependence in the control variable V, F(, V, ) of degree  in V, with V ∈ Ω as follows (see [14] for details): where   are the coefficient that results from the factorization of Lagrange polynomial interpolation quotients.Similarly, we define a polynomial equivalent representation for the running cost  () .Consider a switched running cost of the form given in (1).There exists a unique polynomial running cost equation L(, V, ) of degree  in V, with V ∈ Ω, which is an extension of the previous work [4,14], as follows: We now can state an equivalent optimal control problem based on the equivalent polynomial representation presented before as min L (, V, ) s.t.Equation (6) .
Traditional optimization solvers perform poorly on polynomial constraints that are nonconvex with a disjoint feasible set as the necessary constraint qualification is violated.This makes this problem intractable directly by traditional nonlinear optimization solvers.We propose then a convexification of the equivalent polynomial representation using the special structure of the control variables V and  which improves the optimization process.

The Method of Moments and SDP-Relaxations
In this section, we present the first main result of this work.The inclusion of a external input in the optimal switched system using the method of moments.Before the method is introduced, some basic definitions of probability measures are necessary.Let Δ = Ω × R  be the set of admissible controls V() and ().The set of probability measures associated with the set Δ is where  is a probability measure supported (supp(⋅)) in Δ.We obtain a problem reformulation defined on the set Λ of probability measures as follows: subject to This reformulation is an infinite dimensional linear program which is not tractable as it stands.Considering the special polynomial structure of the problem in terms of the control variables, the theory of moments can be used to obtain a semidefinite program or linear matrix inequality relaxation, with finitely many constraints and variables, called the method of moments.

The Moments Approach.
When an optimal control problem can be stated in terms of polynomial expressions in the control variables, we can use the method of moments.By means of moments variables, an equivalent convex formulation can be obtained which is more appropriate to be solved by numerical computing.The method of moments takes a proper formulation in probability measures of a nonconvex optimization problem ( [9,16] and references therein) and, thus, when the problem can be stated in terms of polynomial expressions in the control variable, we can transform the measures into algebraic moments to obtain a new convex problem defined in a new set of variables that represent the moments of every measure [8,9,13].
We define the space of moments as where Λ(Δ) is the convex set of all probability measures supported in Δ.In addition, a sequence  = {  } has a representing measure  supported in Ω only if these moments are restricted to be entries on positive semidefinite moments and localizing matrices [8,10].
In this work, we are dealing with polynomials in two variables,  and V. Let a basis be defined for the vector space of real-valued polynomials of degree at most  as 1, V, , V 2 , V,  2 , . . ., V 2 , . . .,  2 .Given a moment vector  = [1,  10 ,  01 ,  20 ,  11 ,  20 , . . .,  20 , . . .,  02 ] ⊤ associated with the polynomial basis.The moment matrix   () is the block matrix with rows and columns labeled lexicographically with the polynomial basis; it follows that the row   V  with column   V  entry of   () is  +,+ .For example, we consider the case with  = 1 which corresponds to the application presented in Section 4. The polynomial basis is 1, V, , V 2 , V,  2 ; the corresponding vector of moments is  = [1,  10 ,  01 ,  20 ,  11 ,  02 ] ⊤ .The respective moment matrix is The localizing matrix is defined based on the corresponding moment matrix whose positivity is directly related to the existence of a representing measure with support in Ω as follows.Consider the set Ω defined by the polynomial (V) =  0 +  1 V + ⋅ ⋅ ⋅ +   V  .It can be represented in moment variables as () =  0 +  1  10 + ⋅ ⋅ ⋅ +    0 , or in compact form as () = ∑  =0    0 .Suppose that the entries of the corresponding moment matrix are  0 , with  ∈ [0, 1, . . ., 2].Thus, every entry of the localizing matrix is defined as  0 = ∑  =0  0  +,0 .Note that the localizing matrix has the same dimension of the moment matrix; that is, if  = 1 and the polynomial  = V + 2V 2 , then we have the moment matrix as above and the localizing matrix is More details on the method of moments can be found in [10,17].
Since  is a polynomial in V of degree  and in  of degree , the criterion ∫ L  is linear in the moment variables.Hence, we replace  with the finite sequence  = {  } of all its moments.We can then express the linear combination of the functional  and the space of moments Γ as the following problem in moments variables: min subject to where   are the coefficients resulting from the factorization of (15) as above.We now have a problem in moment variables which can be solved by efficient computational tools as shown below.

Semidefinite Programs.
We can use the functional and the state equation with moment structure to rewrite the relaxed formulation as a semidefinite program (SDP).First, we need to redefine the control set Δ to be coherent with the definitions of localizing matrix and representation results.We treat the polynomial (V) as two opposite inequalities, that is,  1 (V) = (V) ≥ 0 and  2 (V) = −(V) ≥ 0, and we redefine the compact set to be Δ = {  (V) ≥ 0,  = 1, 2}.Also, we define a prefixed order of relaxation which is directly related to the number of subsystems.
Let  be the degree of the polynomial (V), which is equivalent to the degree of the polynomials  1 and  2 .Considering its parity, we have that if  is even (odd) then  = /2 ( = (+1)/2).In this case,  corresponds to the prefixed order of relaxation.We use a direct transcription method to obtain an SDP to be solved through a nonlinear programming (NLP) algorithm [18].Using a discretization method, the first step is to split the time interval [ 0 ,   ] into N subintervals as  0 <  1 <  2 < ⋅ ⋅ ⋅ <  N =   , with a time step ℎ predefined by the user.The integral term in the functional is implicitly represented as an additional state variable, transforming the original problem in Bolza form into a problem in Mayer form, which is a standard transformation [18].Therefore, we obtain a set of discrete equations in moment variables.Thus, the optimal control problem can be formulated as an SDP.
Consider a fixed  in the time interval [ 0 ,   ] and let Assumption 1 hold.We can state the following SDP of relaxation order  (SDP  ).
SDP  : for every  = {1, 2, . . ., N}, a semidefinite program SDP  can be described by In order to solve a traditional NLP, we use the particular characteristics of the moment and localizing matrices form.We know that the moment and localizing matrices are symmetric positive definite, which implies that every principal subdeterminant is positive [12].So we can use the set of subdeterminants of each matrix as algebraic constraints.

Problem Analysis and Optimal Solutions.
Once a solution has been obtained in a subinterval [ −1 ,   ], we obtain a vector of moments  * (  ) = [ * 1 (  ),  * 2 (  ), . . .,  *  (  )].We need to verify if we have attained an optimal solution.Based on a rank condition of the moment matrix [17], we can test if we have obtained a global optimum at a relaxation order .Also, based on the same rank condition, we can check whether the optimal solution is unique or if it is a convex combination of several minimizers.The next result is based on an important result presented in [17] and used in [10] for optimization of 0-1 problems.Lemma 3. Suppose that the SDP  is solved with a moment vector solution  * (  ) for the th relaxation.If the flat extension condition holds, that is, then the global optimum has been reached and the problem has ]  global minimizers.
Using the previous result we can state some relations between solutions that can be used to obtain the switching signal and the continuous control input for every   .First, we state the following result valid for the unique solution case.(17) is solved and the rank condition (18) is verified with ]  = rank   ( * (  )) = 1, then the vector of moments  * (  ) has attained a unique optimal global solution and therefore the optimal switching signal of problem (1) is obtained as

Theorem 4. If for a fixed 𝑡
and the optimal continuous control input is obtained as where  * 10 (  ) and  * 01 (  ) are the first and second component of the vector of moments  * (  ), respectively.
Proof.Assume that the SDP r -program has been solved for a fixed   .Assume also that  * (  ) is the obtained solution and the rank condition ( 18) has been verified.It is known (see [10]) the equivalence between the optimization problem in a probability measure space and in the moment space; that is, min where  * (  ) is the vector of moments.From this equivalence, it follows that the optimal solution in the probability measure space corresponds with the optimal solution in the moment space, and, due to the rank condition (18), this solution is unique.Therefore, this solution is the solution of the equivalence polynomial problem (8); that is, which implies that  * 10 = V * (  ) and  * 01 =  * (  ).On the other hand, based on the polynomial equivalence, we know that the solutions of the polynomial problem (7) are also solutions of the switching system.Hence, we can state that  * (  ) = V * (  ), which in turn implies that  * (  ) = V * (  ) =  * 10 (  ).It also follows that  * (  ) =  * 01 (  ), which is the external input for the switching system.This result states a correspondence between the minimizers of the SDP  and the solutions of the original switched problem, and it can be used to obtain a switching signal and a continuous control input directly from the solution of the SDP  .However, it is not always the case.Sometimes we obtain a nonoptimal solution that arises when the rank condition is not satisfied; that is, ]  > 1.But we still can use information from the solution to obtain a switching suboptimal solution and a suboptimal continuous control input.In [19] a sum-up rounding strategy is presented to obtain a suboptimal switched solution from a relaxed solution in the case of mixed-integer optimal control.We use a similar idea but extended to the case when the relaxed solution is any integer instead of the binary case.
Consider the first moment  10 (⋅) : [ 0 ,   ]  → [0, ], which is a relaxed solution of the NLP problem for   when the rank condition is not satisfied.We can state a correspondence between the relaxed solution and a suboptimal switching solution, which is close to the relaxed solution in average and is given by where ⌈⋅⌉ and ⌊⋅⌋ are the ceiling and floor functions, respectively.For the continuous control input we use the extraction algorithm presented in [20].

Networks with Switching Topology
This section provides an application of the theoretical results presented in this work for optimal control of switching systems.As it has been shown in [2], the possibility of deploying a network of small, simple, and cheap units or agents to execute tasks cooperatively leads to consider the communication network between agents as a fundamental part of the complete system design and control.In general, a given set of agents can communicate with each other.The information exchange is modeled via a communication graph.Each node of the graph represents an agent; an edge represents the possibility for an agent to receive information from another one.For the sake of clarity, a dynamic network is shown in Figure 1, where a communication graph of five nodes is only strongly connected through the switching signal (), which operates as the switching control variable.In the case considered in this work, the communication graph has a dynamic topology; this could be used to model asynchronous consensus or the fact that two agents are not always able to communicate because of possible energy limitations.Networked systems with a dynamic topology are known as switching networks [1].The switching network is modeled using a dynamic graph G () parameterized by a switching signal () : R  → Q = {0, 1, . . ., }, the set of vertices V, and the set of edges E. We deal with an algorithm that is based on the average consensus concept.An average consensus algorithm is a distributed strategy to compute the average of the number of each agent.The average consensus protocol is mathematically expressed as follows.Assume that each node  has a state   () which is initialized to a number  0 ; that is,   (0) =  0 .Then, each node updates   () according to the iteration where N  = { ∈ V \ {} : (, ) ∈ E} is the set of neighbors of the agent .It is also assumed that   ( + 1) is a convex combination of all the states available to the agent  (i.e.,   ≥ 0 and   + ∑ ∈N    = 1).The set of topologies of the network is G  = {G 1 , G 2 , . . ., G  }.
We can rewrite the previous iteration in a matrix form as where the entries of the matrix  ∈ R × are   in position ,  consistent with the graph G. Result for stability analysis of networks with switching topology is mainly based on assumptions of necessity for strong connectivity of all graphs in all time instances [1].However, weaker form of network connectivity is crucial in analysis of asynchronous consensus, which is the case treated in this work.An interesting result for periodically connected topologies can be used to guarantee that the algorithm proposed can reach a consensus with a switching sequence.Consider the discrete-time consensus algorithm in (25); a switching network with the set of topologies G  is periodically connected with a period  G > 1 if the unions of all graphs over a sequence of intervals are connected graphs.
Theorem 5 (see [1]).Consider the system in (25) with G  ∈ G  for  ∈ Q. Assume the switching network is periodically connected.Then, an alignment is asymptotically reached.
Considering that we are dealing with a discrete-time linear switched system, we can use a similar result for discrete-time linear system.It is well known that the system is stable when its poles are located in the open unit ball of the complex plane.For stabilizability of switched linear systems, we have algebraic criteria as follows [21].Theorem 6. Suppose that the switched linear system (25) is stabilizable.Then, there is a  ∈ Q such that |∏  eig =1   (  )| ≤ 1, where   (), 1 ≤  ≤  eig , are the eigenvalues of matrix .
Using Theorem 6, we can state that the switched system (25) is stabilizable, which implies that a switching signal exists that leads the states of the switched system to a stable point.
On the other hand, the solution of (25) can be expressed as with The convergence of the algorithm depends on whether the infinite product of nonnegative matrices   has a limit.The consensus value is a quantity in the convex hull of all initial values.

Example 1.
In this work, we consider a switching topology with an external input  that can be used to change the value of convergence of the average consensus.In order to illustrate the optimal problem, we consider two communications graphs which are shown in Figure 2. Figure 2(a) represents a network with an unconnected node (node 3), which implies that this system cannot reach consensus.The system in Figure 2(b) is the same network with node 3 connected so that the system can reach consensus.The control objectives are to lead the dynamics of node 3 to a particular reference through the external input  spending minimum energy consumption, which is applied to node 1, as it is shown in Figure 2.
The optimal control problem is described as follows.A discrete linear switched system consisting of two subsystems associated with the graphs in Figure 2 with matrices  0 and  1 .An external input  connected directly to node 1.Consider The functional is the same for the two subsystems and it considers both optimization objectives, that is, a reference for  1 and  3 , and minimum energy consumption.Notice the polynomial form of  in the functional: In Figure 3, the trajectories, the switching signal, and the external control input of the switching system obtained for an order of relaxation  = 1 are shown.The simulations show that due to the unconnected node, the system has to switch between the two network topologies to accomplish the control objective.It is also shown that the external input is necessary to change the average consensus value to a value of 10, which is part of the control objective.Once the system has reached the average value, the external control input is zero.It is noted that the system response reaches a stable value and meets the control objectives.The computational efficiency of the proposed algorithm is based on the semidefinite structure of the relaxed problem obtained.

Example 2: Consensus under Communication Limitations.
In this example, we consider a leader-following system consisting of a leader and two agents.The communication network is shown in Figure 4, where three switching topologies G 1 , G 2 , and G 3 can be observed.None of the graphs has a spanning tree, which implies that the system has to switch in order to converge to a consensus (see Theorem 5).The optimal control objectives are to minimize the disagreement as a quadratic function while the energy consumption is minimized as described in Example 1 but with matrices It is observed in the simulation that the controlled system switches between the three subsystems (see switching signal in Figure 5) to spread the information through all nodes and then to reach consensus (see Figure 5).

Conclusions
In this paper, we have extended our previous work on the new method for solving the optimal control problem of switched systems based on a polynomial approach including the external input.We follow the same line of thought, transforming the original problem into a polynomial system, which is able to mimic the switching behavior with a continuous polynomial representation.Then, we transform the polynomial problem into a relaxed convex problem using the method of moments.From a theoretical point of view, we have provided sufficient conditions for the existence of the minimizer by using particular features of the relaxed convex formulation.We have introduced the moment approach as a computational useful tool to solve this kind of problems, which has been illustrated by means of interesting networked control systems, that is, a network with switching topology modeled as switching systems.