AN APPROXIMATION TO DISCRETE OPTIMAL FEEDBACK CONTROLS

We study discrete solutions of nonlinear optimal control problems. By value functions, we construct diﬀerence equations to approximate the optimal control on each interval of “small” time. We aim to ﬁnd a discrete optimal feedback control. An algorithm is proposed for computing the solution of the optimal control problem.


Introduction and statement of the problem.
For nonlinear analytic systems, there have been many works on control problems since the eighties of the last century.In [3,4,5,6], Lie series was used in studying the controllability of nonlinear analytic systems.In [1], the discrete method in solving Hamilton-Jacobi-Bellman equations for value functions of nonlinear problems was discussed.In this paper, we use Lie series to construct difference equations by value functions in obtaining the discrete solutions of nonlinear optimal control problems.Taking advantage of the uniform convergence of Lie series on an interval of "small" time, we focus on the integral of the optimal control function.We aim to find a discrete optimal feedback control.We see that the optimal controls of a given problem can be constructed by these integral dates.We propose an algorithm which includes the process of pre-estimation and correction of an approximation to the solution of the optimal control problem.
We begin by considering the following nonlinear control system: where f : R n → R n and G : R n → R n×m , G = g 1 ,...,g m , (1.2) are real analytic mappings.We consider the admissible controls u(t), which take values in some compact set U ⊂ R m , to be integrable.Throughout this paper, it is assumed that the state space X is bounded.Let Q(x) be a Lipschitz function.Denote by x u (t) the solution of system (1.1) relative to the control u.We pose the following optimal control problem: find an admissible control û(•) such that where x(•) is the solution of (1.1) relative to the optimal control û(•).

2.
A lemma on Lie series.We introduce Lie series for system (1.1) and real analytic function P (x) (see [4, pages 698-699]) in the following lemma.
Lemma 2.1.Let P (x) be a real analytic function on R n and let be compact in R n .Suppose that x 0 ∈ and consider the admissible controls u(•) satisfying u(•) ≤ M, a.e. on [0,T ].The solution of (1.1) corresponding to an admissible control u(•) is denoted by x u (•).For a given positive integer l, denote N = ml.Further, define f 0 = f and f i = g i , i = 1, 2,...,m.Meanwhile, for each positive integer k, denote where u 0 (t) ≡ 1 and (f i 1 P )(x) = L f i 1 P , and in turn, for each positive integer k, ( Then there exists a positive number δ < 1 which depends only on P , f , G, M, and such that if 0 < t < δ, P x u (t) − P x 0 = Ser N (u)(P ) t, x 0 + R N (u)(P ) t, x 0 , ( where R N (u)(P )(t, x 0 ) is uniformly convergent to zero, when N → ∞, as long as x 0 ∈ and u(s) ≤ M, a.e. on [0,t].Moreover, for the sufficiently large N, where C only depends on N, , M, m, f i (i = 0, 1, 2,...,m), and P .

The formulation of discrete solutions to the optimal control problem.
We define the value function, for where x u (s, x) is the solution of the following equation: It is well known that the value function is granted the following boundary condition: We introduce the discrete scheme as follows.For a given positive integer L, divide [0,T ] into L parts: [(i−1)T /L, iT /L], i = 1, 2,...,L.Denote t i = iT /L.Let x i , i = 1, 2,...,L, be the state point associated with t i .For every j, j = 1, 2,...,n, denote e T j = (0,...,0, 1, 0,...,0) which has a 1 on the jth place and zeros on other places.For every vector x ∈ R n , x (j) = e T j x, which is denoted by P j (x).We need the following elementary lemma.Lemma 3.1.Let {x i , i = 1,...,L} be a set of state points.Suppose that, for each i ∈ {1, 2,...,L}, (3.4) is an optimal control for the nonlinear optimal control problem (1.3).
Remark 3.2.By Lemma 2.1, we see that when τ (= T /L) is sufficiently small, û(•), given by Lemma 3.1, steers x i−1 to x i , which amounts to, for j = 1, 2,...,n, where the term R N ( û)(P j )(τ, x i−1 ) converges uniformly to zero (N → ∞).On the other hand, condition (3.4) ensures the optimality of state points.Now we derive the following difference equations.For each i = 0, 1, 2,...,L−1, when x i is obtained, where for i = 0, 1,...,L − 1, S k (i) stands for the following integral form for an admissible control u(•): We establish We see that (3.9) can be rewritten in the vector form as follows: (3.12) Substituting expression (3.12) for x i+1 in (3.11), we have (3.13) The discrete solution for each i will be constructed by solving (3.13) for Next suppose we have got x i+1 such that and ..,m), such that We take a control ũi (•) on each [t i ,t i+1 ], i = 0, 1,...,L− 1, such that Suppose that ũi (•) steers system (1.1) from x i to xi+1 .We have, by Lemma 2.1, Further, we have noting by hypothesis that Q(x) is a Lipschitz function and so is V (t,x) by Lemma 3.3.Therefore, we have Sk Proof.Given x , x ∈ X, we show, for arbitrarily given > 0, that By definition (3.1), we are able to get admissible controls u ,u such that Then we see that Noting that Q(x) is a Lipschitz function, f (x) and G(x) are real analytic, and the state space is bounded, we deduce that by means of Gronwall inequality [2, page 829].Since > 0 is arbitrary, we see that V (t,x) is a Lipschitz function with respect to x.
In the following, we indicate that carrying out Algorithm 4.1, we can compute to get approximation to an optimal endpoint and an optimal control in general.
Theorem 4.3.Suppose that there is an optimal control û(•) for problem (1.1), (1.2), and (1.3).Denote by x(T ) the endpoint of (1.1) with respect to û(•), that is, Meanwhile, for each positive integer L, with τ = T /L, let ũ(•) and xL be obtained by Algorithm 4.1.Then there is a positive real C which only depends on the state space X (which is assumed to be bounded) and f (x), G(x) such that, for every large positive real Proof.By (3.20) and (3.21), we have we have, by (4.7) and (4.8), By noting that, for each x ∈ X, V (T ,x) = 0, we deduce, from (4.10), that Since x is the optimal endpoint, by the definition of V (0,x 0 ) (see (3.1)), we have Combining (4.11) and (4.12), noting that the state space is assumed to be bounded, we conclude that there is a positive constant C: At the end of this section, we would state that in some cases by Algorithm 4.1 one can compute an exact optimal control.For example, if we consider the simple linear system ẋ = u in R 1 , the algorithm will be as follows.
5. An example demonstrating the results of Section 4. We would demonstrate the above process by the following simple example.
Example 5.1.Consider the following system in R 1 : Let Q(x) = x.We pose the optimal control problem min Since u(•) is integrable and |u| ≤ 1, we have x e 1−t − 1 , for x < 0.
6.An application in singular linear quadratic optimal control problems.We would like to present an application of the approximation approach in Sections 2 and 3 in singular linear quadratic optimal control problems by the following example.We aim to find a discrete optimal feedback control.