The Optimization of Solutions of the Dynamic Systems with Random Structure

and Applied Analysis 3 then the system 1.1 can be rewritten to the form dX t dt G t, ξ t X t , 1.7 and the functional 1.4 to the form J ∫∞ 0 〈X∗ t H t, ξ t X t 〉dt. 1.8 We suppose also that, together with every jump of random process ξ t in time tj , the solutions of the system 1.7 submit to the random transformation X ( tj 0 ) CskX ( tj − 0 ) , s, k 1, 2, . . . , n, 1.9 if the conditions ξ tj 0 θs, ξ tj − 0 θk hold. Definition 1.3. Let ak t , k 1, . . . , n, t ≥ 0 be a selection of n different positive functions. If ξ tj 0 θs, ξ tj − 0 θk, s, k 1, . . . , n, and for tj ≤ t ≤ tj 1 the equality a t, ξ t θs as t − tj holds, then the function a t, ξ t is called semi-Markov function. The application of semi-Markov functions makes it possible to use the concept of stochastic operator. In fact, the semi-Markov function a t, ξ t is an operator of the semiMarkov process ξ t , because the value of the semi-Markov function a t, ξ t is defined not only by the values t and ξ t , but it is also necessary to specify the function as t , t ≥ 0 and the value of the jump of the process ξ t in time tj which precedes the moment of time t. Our task is the construction of Lyapunov function for the new class of systems of linear differential equations with semi-Markov coefficients and then applying the function to solve the optimization problem which minimizes the quality criterion. 2. Auxiliary Results In the proof of Theorem 3.1 in Section 3, we will employ two results concerning the construction of the Lyapunov function and the construction of the optimal control for the system of linear differential equations in a deterministic case. We will derive these auxiliary results in this part. 2.1. The Construction of the Lyapunov Function Let us consider the system of linear differential equations dX t dt A t, ξ t X t 2.1 associated to the system 1.1 . 4 Abstract and Applied Analysis Let us define a quadratic form w t, x, ξ t x∗B t, ξ t x, B t, ξ t > 0, 2.2 where elements of the matrix B t, ξ t are the semi-Markov processes. The matrix B t, ξ t is defined by such a set of n different symmetric and positive definite matrices Bk t , t ≥ 0, k 1, . . . , n, that the equality ξ t θs for tj ≤ t ≤ tj 1 implies B t, ξ t Bs ( t − tj ) , s 1, 2, . . . , n. 2.3 Our purpose in this section is to express the value of the functional ν ∫∞ 0 〈w t, X t , ξ t 〉dt 2.4 in a convenient form, which can help us to prove the L2-stability of the trivial solution of the system 2.1 . At first, we introduce the particular Lyapunov functions νk x ∫∞ 0 〈w t, X t , ξ t | X t x, ξ 0 θk〉dt, k 1, 2, . . . , n. 2.5 If we can find the values of the particular Lyapunov functions in the form νk x x∗Ckx, k 1, 2, . . . , n, then value of the functional ν can be expressed by the formula


The Statement of the Problem
The optimal control theory as mathematical optimization method for deriving control policies plays an important role in the development of the modern mathematical control theory.The optimal control deals with the problem of finding such a control law for a given system that a certain optimality criterion is achieved.The background for the optimization method can be found in the work of Lev Pontryagin with his well-known Pontryagin's maximum principle.The optimal control has been applied in diverse fields, such as economics, bioengineering, process control, and many others.Some real-life problems are described by a continuoustime or discrete-time linear system of differential equations, but a lot of them are described by dynamic systems with random jumping changes, for example economics systems.The general theory of random structure systems can be found in the work of Artemiev and Kazakov 1 .The optimization of linear systems with random parameters are considered in many works, for example in 2-12 .Particularly, the original results concerning the stabilization of the systems with random coefficients and a random process are derived using moment equations and Lyapunov functions in 4 .These results create a more convenient technique for applying the method in practice using suitable software for engineering or economics investigation.Our aim is the expansion of the achieved results to a new class of systems of linear differential equations with semi-Markov coefficients and random transformation of solutions performed simultaneously with jumps of semi-Markov process.We will focus on using the particular values of Lyapunov functions for the calculation of coefficients of the control vector which minimize the quality criterion.We will also establish the necessary conditions of the optimal solution which enables the synthesis of the optimal control for the considered class of systems.
Let us consider the linear control system dX t dt A t, ξ t X t B t, ξ t U t 1.1 on the probability basis Ω, Ì, P, F ≡ {F t : t ≥ 0} and together with 1.1 we consider the initial conditions

1.2
The coefficients of the system are semi-Markov coefficients defined by the transition intensities q αk t , α, k 1, 2, . . ., n, from state θ k to state θ α .We suppose that the vectors U t belong to the set of control U and the functions q αk t , α, k 1, 2, . . ., n, satisfy the conditions 13 : We suppose also that, together with every jump of random process ξ t in time t j , the solutions of the system 1.7 submit to the random transformation . ., n, and for t j ≤ t ≤ t j 1 the equality a t, ξ t θ s a s t − t j holds, then the function a t, ξ t is called semi-Markov function.
The application of semi-Markov functions makes it possible to use the concept of stochastic operator.In fact, the semi-Markov function a t, ξ t is an operator of the semi-Markov process ξ t , because the value of the semi-Markov function a t, ξ t is defined not only by the values t and ξ t , but it is also necessary to specify the function a s t , t ≥ 0 and the value of the jump of the process ξ t in time t j which precedes the moment of time t.
Our task is the construction of Lyapunov function for the new class of systems of linear differential equations with semi-Markov coefficients and then applying the function to solve the optimization problem which minimizes the quality criterion.

Auxiliary Results
In the proof of Theorem 3.1 in Section 3, we will employ two results concerning the construction of the Lyapunov function and the construction of the optimal control for the system of linear differential equations in a deterministic case.We will derive these auxiliary results in this part.

The Construction of the Lyapunov Function
If we can find the values of the particular Lyapunov functions in the form ν k x x * C k x, k 1, 2, . . ., n, then value of the functional ν can be expressed by the formula where the scalar value is called the scalar product of the two matrices N ν kj , S s kj and has the property 14 The first auxiliary result contains two equivalent, necessary, and sufficient conditions for the L 2 -stability see in 4 of the trivial solution of the system 2.1 and one sufficient condition for the stability of the solutions.
Theorem 2.1.The trivial solution of the system 2.1 is L 2 -stable if and only if any of the next two equivalent conditions hold: 2 the sequence of the approximations Moreover, the solutions of the system 2.1 are L 2 -stabile, if there exist symmetric and positive definite matrices Proof.We will construct a system of equations, which will define the particular Lyapunov functions ν k x , k 1, 2, . . ., n.Let us introduce the auxiliary semi-Markov functions

2.12
For the state ξ t θ k , t ≥ 0 of the random process ξ t , the equalities are true.Simultaneously, with the jumps of the random process ξ t , the jumps of solutions of 2.1 occurred, so in view of 2.12 , we derive the equations Further, if we introduce denoting then 2.14 can be rewritten as the system of integral equations for the matrix u k t in the form

2.18
Similarly, integrating the system of 2.14 , we get the system of equations determining the particular Lyapunov functions

2.19
Let us denote

2.20
If there exist such positive constants λ 1 , λ 2 that hold, then the matrices H k , k 1, 2, . . ., n are symmetric and positive definite.Using 2.17 , the system 2.18 can be rewritten to the form

2.23
It is easy to see that the system 2.23 is conjugated to the system 2.9 .Therefore, the existence of a positive definite solution C k > 0, k 1, 2, . . ., n of the system 2.23 is equivalent to the existence of a positive definite solution B k > 0, k 1, 2, . . ., n and it is equivalent to L 2stability of the solution of the system 2.1 .On the other hand, if the existence of the particular Lyapunov functions ν k x , k 1, 2, . . ., n in 2.5 implies L 2 -stability of the solutions of the system 2.1 , then, in view of conditions 2.22 and the convergence of the integral 2.17 , we get the inequality The theorem is proved.
Remark 2.2.If the system of linear differential equations 2.1 is a system with piecewise constant coefficients and the function w t, X t , ξ t has the form then the system 2.18 can be written in the form

2.26
Particularly, if the semi-Markov process ξ t is identical with a Markov process, then the system 2.26 has the form

2.28
Moreover, under the assumption that the integral in 2.28 converges, the system 2.28 is equivalent to the system of matrices equations which can be written as the system The trivial solution of 2.32 is L 2 -stable, if c 1 > 0 and c 2 > 0. Let the intensities of semi-Markov process ξ t satisfy the conditions Then, using the Theorem 2.1, the conditions are sufficient conditions for the L 2 -stability of solutions of 2.32 .

The Construction of an Optimal Control for the System of Linear Differential Equations in the Deterministic Case
Let us consider the deterministic system of the linear equations in the boundary field G, where X ∈ Ê m , U ∈ Ê l , and together with 2.37 we consider the initial conditions X t x 0 .

2.38
We assume that the vector U t belongs to the control set U. The quality criterion has the form of the quadratic functional which minimizes the quality criterion 2.39 is called the optimal control.The optimization problem is the problem of finding the optimal control 2.40 from all feasible control U, or, in fact, it is the problem of finding the equation to determine S t , dim S t l × m.
Theorem 2.4.Let there exist the optimal control 2.40 for the system of 2.37 .Then the control equations where the matrix K t satisfies the Riccati equation

2.55
In the end, we define the matrix K t as the integral manifolds of solutions of the system equations

2.56
If we derive the system 2.56 with respect to t regarding the system 2.55 and extract the vector Ψ * , then we obtain the matrix differential equation

The Main Result
Theorem 3.1.Let the coefficients of the control system 1.1 be the semi-Markov functions and let them be defined by the equations

3.1
Then the set of the optimal control is a nonempty subset of the control U, which is identical with the family of the solutions of the system where the matrix R s t is defined by the system of Riccati type of differential equations 3.3

The Proof of Main Result Using Lyapunov Functions
It should be recalled that the coefficients of the systems 1.1 , 1.7 and of the functionals 1.4 , 1.8 have the form In addition to this, we have

3.5
The formula is useful for the calculation of the particular Lyapunov functions ν k x ≡ x * C k x, k 1, . . ., n of the functional 1.8 .We get or, the more convenient form 3.8 Then the system 3.1 has the form Let us assume that for the control system 1.1 the optimal control exists in the form 1.5 independent of the initial value X 0 .Regarding the formula 3.6 , there exist minimal values of the particular Lyapunov functions ν k x , k 1, . . ., n, which are associated with the optimal control.It also follows from the fact that the functions ν k x , k 1, . . ., n are particular values of the functional 3.6 .Finding the minimal values ν k x , k 1, . . ., n by choosing the optimal control U k x is a well-studied problem, for the main results see 16 .It is significant that all matrices C s , s 1, . . ., n of the integrand in the formula 3.8 are constant matrices, hence, solving the optimization problem they can be considered as matrices of parameters.
Therefore, the problem to find the optimal control 1.5 for the system 1.1 can be transformed to n problems to find the optimal control for the deterministic system 3.9 , which is equivalent to the system of linear differential equations of type 2.37 .

The Proof of the Main Result Using Lagrange Functions
In this part, we get one more proof of the Theorem 3.1 using the Lagrange function.
We are looking for the optimal control which reaches the minimum of quality criterion

3.10
Let us introduce the Lagrange function

3.11
where Y t is the column-vector of Lagrange multipliers.In accordance with Pontryagin's maximum principle, we put the first variations of the functionals ∂I x , ∂I y equal to zero and we obtain the system of linear differential equations

3.12
Then the optimal control U t can be expressed by The synthesis of the optimal control needs to find the integral manifolds of the solutions of the system 3.12 in the form Y t K t X t , K T 0.

3.14
According to the theory of integral manifolds 19 we construct the differential matrix equations of the Riccati type for the matrix K t .Integrating them from time t T to time t 0 and using the initial condition K T 0 we obtain Lagrange functions for the optimal control U t −L −1 t B * t K t X t .

3.16
We will prove that 3.17 Differentiating the equality 3.17 with respect to t we obtain the matrix equation and extracting the optimal control U t we obtain differential equation for K t identical with 3.15 .The equality K t K * t follows from the positive definite matrices Q t , L t for t < T. Therefore, from 3.17 we get K t 0; moreover, from 3.10 it follows that C K 0 .
Applying the formulas 3.15 , 3.16 to the system 3.8 with minimal functionals 3.9 , the expression for the optimal control can be found in the form where symmetric matrices K s t satisfy the matrix system of differential equations

3.20
The systems 3.9 , 3.20 define the necessary condition such that the solutions of the systems 1.4 will be optimal.In addition to this, the system 3.8 defines the matrices S k t , k 1, 2, . . ., n, of the optimal control in the form

3.21
We define matrices C s from the system equations 3.20 in the view of C s K s 0 , s 1, 2, . . ., n.

3.22
In regards to it can makes the system 3.20 simpler.Then the system 3.20 takes the form 3.3 , and formula 3.2 defines the optimal control.Remark 3.2.If the control system 1.1 is deterministic, then q ks t ≡ 0, Ψ s t ≡ 0, k, s 1, 2, . . ., n and the system 3.3 is identical to the system of the Riccati type equations 3.15 .

Particular Cases
The optimal control U t for the system 1.1 has some special properties, and the equations determining it are different from those given in the previous section in case the coefficients of the control system 1.1 have special properties or intensities q sk t satisfy some relations or some other special conditions are satisfied.Some of these cases will be formulated as corollaries. Corollary

4.6
Remark 4.2.In the corollary we mention piecewise constant coefficients of the control system 4.1 .The coefficients of the functional 4.2 will be piecewise as well, but the optimal control is nonstationary.

4.7
Then the optimal control U t will be piecewise constant.
Taking into consideration that the optimal control is piecewise constant, we find out that the matrices R k t , k 1, 2, . . ., n in 4.5 are constant, which implies the form of the system 4.6 is changed to the form q sk t Ψ k t C * sk R k C sk 0, k 1, . . ., n.

4.8
The system 4.8 has constant solutions R k , k 1, 2, . . ., n, if conditions 4.7 hold.Moreover, if the random process ξ t is a Markov process then the conditions 4.7 have the form Ψ k t a kk const, q sk t Ψ k t a sk const, k,s 1, 2, . . ., n, k / s, 4.9 and the system 4.8 transforms to the form

4.14
In this case, the search for the matrix K s t , s 1, 2, . . ., n in concrete tasks is reduced to integration of the matrix system of differential equations 3.15 on the interval 0, T s with initial conditions 4.14 .In view of Ψ s T s 0, s 1, 2, . . ., n, we can expect that every equation 3.15 has a singular point t T s .If Ψ s t has simple zero at the point t T s , then the system 4.6 meets the necessary condition Ψ s T s R s T s n k 1 q sk T s C * ks R s 0 C ks 0, s 1, . . ., n 4.15 for boundary of matrix R s t in the singular points.
Example 2.3.Let the semi-Markov process ξ t take two states θ 1 , θ 2 and let it be identical with the Markov process described by the system of differential equations Let the state θ s of the semi-Markov process ξ t be no longer than T s > 0. Then the system 3.8 has the formν k x ≡ x * C k x s 0, s 1, 2, . . ., n.