Abstract and Applied Analysis

Volume 2014 (2014), Article ID 436362, 8 pages

http://dx.doi.org/10.1155/2014/436362

Research Article

On the Hamilton-Jacobi-Bellman Equation by the Homotopy Perturbation Method

Abdon Atangana,¹ Aden Ahmed,² and Suares Clovis Oukouomi Noutchie³

¹Institute for Groundwater Studies, Faculty of Natural and Agricultural Sciences, University of the Free State, Bloemfontein 9300, South Africa

²Department of Mathematics, Texas A&M University, MSC 172, 700 University Boulevard, Kingsville, TX, USA

³Department of Mathematical Sciences, North-West University, Mafikeng Campus, Mmabatho 2735, South Africa

Received 16 April 2014; Revised 24 May 2014; Accepted 1 June 2014; Published 22 June 2014

Academic Editor: Guo-Cheng Wu

Copyright © 2014 Abdon Atangana et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Our concern in this paper is to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation (HJB). The approach is obviously extremely well organized and is an influential procedure in obtaining the solutions of the equations. We portrayed particular compensations that this technique has over the prevailing approaches. We presented that the complexity of the homotopy decomposition method is of order . Furthermore, three explanatory examples established good outcomes and comparisons with exact solution.

1. Introduction

Theory and application of optimal control have been widely used in different fields such as biomedicine, aircraft systems, and robotics. However, optimal control of nonlinear systems is a challenging task which has been studied extensively for decades. In the past two decades, the indirect methods have been extensively developed. It is well known that the nonlinear optimal control leads to a nonlinear two-point boundary value problem or a Hamilton-Jacobi-Bellman (HJB) partial differential equation. Many recent researches have been devoted to solving these two problems. In general, the HJB equation is a nonlinear partial differential equation that is hard to solve in most cases. An excellent literature review on the methods for solving the HJB equation is provided in [1], where a successive Galerkin approximation method is also considered. In the Galerkin approximation method a sequence of generalized Hamilton-Jacobi-Bellman equations is solved iteratively to obtain a sequence of approximations approaching the solution of Hamilton-Jacobi-Bellman equation. However, the proposed sequence may converge very slowly or even diverge. There are various efficient methods such as those reported in [2, 3] for the computation of open-loop optimal controls. However, feedback controls are much preferred in many engineering applications. Many other numerical methods [4–6]. In those, a sequence of nonhomogeneous linear time-varying TPBVPs is solved instead of directly solving the nonlinear TPBVP derived from the maximum principle. However, solving time-varying equations is much more difficult than solving time-invariant ones. Thus, it is required to solve HJB equation by an approximate-analytic method. Some of these equations were also evaluated in [7]; they used the so-called piecewise homotopy perturbation method to derive approximate solution for (HJB). Their method was a simple modification of the well-known homotopy perturbation method; the modification was based upon an algorithm in a sequence of small interval meaning time step for finding more accurate approximate solutions to the corresponding (HJB) equation.

In this paper we use homotopy decomposition method that was recently proposed in [8] to solve the Hamilton-Jacobi-Bellman equation. The method was first used to solve the groundwater flow equation [9]. To show the efficiency of the method, the following three problems are solved.

Problem 1. Consider a single-input scalar system as follows [10]:

The corresponding Hamiltonian function will be

Problem 2. Consider the following purely mathematical optimal control problem:

The corresponding Hamiltonian function will be

Problem 3. Consider the following nonlinear optimal control problem [11]:

The corresponding Hamiltonian function will be

The paper is organized as follows. In Section 2, the basics idea of homotopy decomposition method is presented. In Section 3, the advantages of the chosen method are presented. In Section 4, nonlinear time-variant HJB equation is presented. Conclusion is made in Section 5.

2. Homotopy Decomposition Method [12, 13]

To exemplify the primitive thought of this approach we study an overall nonlinear nonhomogeneous partial differential equation with initial conditions of the following form:

subject to the initial condition

Somewhere, is a known function, is the general nonlinear differential operator, and represents a linear differential operator. The first step of the method here is to apply the inverse operator of both sides of (7) to obtain

The multi-integral in (9) can be distorted to

so that (9) can be reformulated as

Employing the homotopy arrangement the resolution of the overhead integral equation is assumed in series form as

and the nonlinear term can be decomposed as

where is an embedding parameter. is He’s polynomials [14] that can be generated by

The homotopy decomposition method is obtained by the beautiful coupling of decomposition method with He’s polynomials and is given by

with

Comparing the terms of same powers of , give solutions of various orders. The initial guess of the approximation is [13]. It is important to point out that the initial guess is Taylor series of order of the exact solution of the main problem [13].

2.1. Advantage of the Method

The homotopy decomposition method is chosen to solve this nonlinear problem because of the following advantages the method has over the existing methods.

(1)The method does not require the linearization or assumptions of weak nonlinearity.(2)The solutions are not generated in the form of general solution as in Adomian decomposition method. With ADM the recursive formula allows repetition of terms in the case of nonhomogeneous partial differential equation and this leads to the noisy solution [15].(3)The solution obtained is noise-free compared to the variational iteration method [16].(4)No correctional function is required as in the case of the variational homotopy decomposition method.(5)No Lagrange multiplier is required in the case of the variational iteration method [16].(6)It is more realistic compared to the method of simplifying the physical problems.(7)If the exact solution of the partial differential equation exists, the approximated solution via the method converges to the exact solution [8].(8)A construction of a homotopy is not needed as in the case of the homotopy perturbation method [14].

3. Complexity of the Homotopy Decomposition Method

It is very important to test the computational complexity of the method or algorithm. Complexity of an algorithm is the study of how long a program will take to run, depending on the size of its input and length of loops made inside the code [13]. We compute a numerical example which is solved by the homotopy decomposition method. The code has been presented with Mathematica 8 according to the following code [13].

Step 1. Set .

Step 2. Calculating the recursive relation after the comparison of the terms of the same power is done.

Step 3. If with being the ratio of the neighbourhood of the exact solution [8] then go to Step 4; else and go to Step 2.

Step 4. Print out

as the approximation of the exact solution.

Lemma 1. If the exact solution of the partial differential equation (7) exists, then

Proof. Let ; since the exact solution exists, then we have the following:

The last inequality follows from [13].

Lemma 2. The complexity of the homotopy decomposition method is of order .

Proof. The number of computations including product, addition, subtraction, and division is in Step 2:

 : 0 because, obtains directly from the initial guess [13], : 3,  : 3.

Now in Step 4 the total number of computations is equal to .

4. Nonlinear Time-Variant Hamilton-Jacobi-Bellman Equation

In this paper, we consider a general nonlinear control system described by

where a state is and vector   is a control signal. The objective is to find the optimal control law , which minimizes the following cost function:

In this cost function, and are arbitrary convex functions and is final time of system operation. It is supposed [4] that

Problem 4. Consider a single-input scalar system as follows [9]:

The corresponding Hamiltonian function will be

Our concern here is to find , that is, stationary point for the Hamiltonian function. Therefore, differentiating (24) with respect to , we obtain

Applying the second derivative test, we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our concern is to find the minimum value. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain

To solve the above system of equation via the homotopy decomposition method, we transform it to the integral equation as follows:

Using the homotopy scheme the solution of the above integral equation is given in series form as

verifying

Comparing the terms of same powers of , we obtain the following system of integral equations:

Lemma 3. If over its domain, then .

Proof. Notice that

Replacing (32) in (26) we obtain

Arranging this we obtain

Solving the above, we obtain

Solving the integral equations we obtain the following terms:

Using the package Mathematica, in the same manner, one can obtain the rest of the components. But, here, 16 terms were computed and the asymptotic solution is given by

This verifies Lemma 3.

The analytical solution for this problem of state and the control is given as [4]

with

where, for simplicity,

To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors for as shown in Figure 1 that is showing the comparison between the two.

Figure 1: Approximated solution.

Problem 5. Consider the following purely mathematical optimal control problem:

The corresponding Hamiltonian function will be

Our concern here is to find , that is, stationary point for the Hamiltonian function. Therefore, differentiating (42) with respect to we obtain

Applying the second derivative test we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain

subjected to the following condition:

The exact solution of this form was proposed in [4] as

Shadowing the homotopy decomposition structure and equaling the relations of the same powers of , we obtain the following system of integral equations:

The following solutions are obtained. Here we chose the first term to be so that

With the similar routine one can acquire the remaining terms of the components. But 8 terms were computed and the asymptotic solution is given by

To access the accuracy of the homotopy decomposition method, we determine numerically the absolute errors for as shown in Figure 4. Figures 2 and 3 show approximate solution of equation and numerical error respectively.

Figure 2: Exact solution.

Figure 3: Numerical error.

Figure 4: Numerical error.

Problem 6. Consider the following nonlinear optimal control problem [14]:

The corresponding Hamiltonian function will be

Following the discussion presented earlier, we obtain

Applying the second derivative test we obtain , since the second derivative is positive for all ; it follows that our turning point is a minimum, which is acceptable because our aim is to minimise. Thus, by substituting in Hamilton-Jacobi-Bellman equation we obtain

subjected to the following condition:

Following the homotopy decomposition scheme and comparing the terms of the same powers of , we obtain the following system of integral equations:

The following solutions are obtained. Here we chose the first term to be so that

The asymptotic solution is given by

Figure 5 shows the graphical representation of .

Figure 5: Numerical approximation of .

5. Conclusion

The purpose of this paper was to use the homotopy decomposition method to solve the Hamilton-Jacobi-Bellman equation. The method is clearly a very efficient and powerful technique in finding the solutions of the equations. We presented some advantages the method has over the existing methods. We showed that the complexity of the homotopy decomposition method is of order . In addition, three illustrative examples demonstrated good results and comparisons with exact solution. Although the technique used in this paper is very accurate, we have to mention that there exist more appropriate techniques that can be used to handle this type of nonlinear equation.

Conflict of Interests

All authors confirmed that there is no conflict of interests in this paper.

Authors’ Contribution

Abdon Atangana wrote the first draft; Aden Ahmed and Suares Clovis Oukouomi Noutchie revised and corrected all mathematical and language mistakes; all three authors read and agreed to submit the final version of the paper.

References

R. W. Beard, G. N. Saridis, and J. T. Wen, “Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation,” Journal of Optimization Theory and Applications, vol. 96, no. 3, pp. 589–626, 1998.

B. D. Craven, Control and Optimization, Chapman & Hall, 1995.

K. L. Teo, C. J. Goh, and K. H. Wong, A Unified Computational Approach to Optimal Control Problems, Longman Scientific & Technical, Essex, UK, 1991.

G. C. Wu and D. Baleanu, “Chaos synchronization of the discrete fractional logistic map,” Signal Processing, vol. 102, pp. 96–99, 2014.

A. H. Bhrawy and M. A. Alghamdi, “A shifted Jacobi-Gauss-Lobatto collocation method for solving nonlinear fractional Langevin equation involving two fractional orders in different intervals,” Boundary Value Problems, vol. 2012, article 62, 2012.

G. C. Wu and D. Baleanu, “Discrete chaos in fractional delayed logistic maps,” Nonlinear Dynamics, 2014.

S. Effati, H. S. Nik, and M. Shirazian, “An improvement to the homotopy perturbation method for solvingthe Hamilton-Jacobi-Bellman equation,” IMA Journal of Mathematical Control and Information, 2013.

A. Atangana and A. Secer, “The time-fractional coupled-Korteweg-de-Vries equations,” Abstract and Applied Analysis, vol. 2013, Article ID 947986, 8 pages, 2013.

S. A. Yousefi, M. Dehghan, and A. Lotfi, “Finding the optimal control of linear systems via He's variational iteration method,” International Journal of Computer Mathematics, vol. 87, no. 5, pp. 1042–1050, 2010.

A. Jajarmi, N. Pariz, A. V. Kamyad, and S. Effati, “A novel modal series representation approach to solve a class of nonlinear optimal control problems,” International Journal of Innovative Computing, Information and Control, vol. 7, no. 3, pp. 1413–1425, 2011.

J.-H. He, “Homotopy perturbation technique,” Computer Methods in Applied Mechanics and Engineering, vol. 178, no. 3-4, pp. 257–262, 1999.

A. Atangana and J. F. Botha, “Analytical solution of the groundwater flow equation obtained via homotopy decomposition method,” Journal of Earth Science & Climatic Change, vol. 3, article 115.

A. Atangana and D. Baleanu, “Nonlinear fractional Jaulent-Miodek and Whitham-Broer-Kaup equations within Sumudu transform,” Abstract and Applied Analysis, vol. 2013, Article ID 160681, 8 pages, 2013.

A. Atangana and A. Kılıçman, “Analytical solutions of boundary values problem of 2D and 3D poisson and biharmonic equations by homotopy decomposition method,” Abstract and Applied Analysis, vol. 2013, Article ID 380484, 9 pages, 2013.

A. Atangana, “New class of boundary value problems,” Information Sciences Letters, no. 12, pp. 67–76, 2012.

G. C. Wu and D. Baleanu, “Discrete fractional logistic map and its chaos,” Nonlinear Dynamics, vol. 75, pp. 283–287, 2014.