Optimal Homotopy Asymptotic Method-Least Square for Solving Nonlinear Fractional-Order Gradient-Based Dynamic System from an Optimization Problem

In this paper, we consider an approximate analytical method of optimal homotopy asymptotic method-least square (OHAM-LS) to obtain a solution of nonlinear fractional-order gradient-based dynamic system (FOGBDS) generated from nonlinear programming (NLP) optimization problems. The problem is formulated in a class of nonlinear fractional di ﬀ erential equations, (FDEs) and the solutions of the equations, modelled with a conformable fractional derivative (CFD) of the steepest descent approach, are considered to ﬁ nd the minimizing point of the problem. The formulation extends the integer solution of optimization problems to an arbitrary-order solution. We exhibit that OHAM-LS enables us to determine the convergence domain of the series solution obtained by initiating convergence-control parameter C j ′ s . Three illustrative examples were included to show the e ﬀ ectiveness and importance of the proposed techniques. ,


Introduction
Consider a nonlinear programming-constrained optimization problems (NLPCOPs) of the form min x∈R n f x ð Þ subject to g k x ð Þ ≤ 0 and h k x ð Þ = 0∀k ∈ I = 1, 2:: where f : R n ⟶ R, h k : R n ⟶ R, and g k : R n ⟶ R, k, are C 2 functions. Let X 0 = fx ∈ R n | h k = 0, g k ≤ 0, i ∈ Ig be the feasible set of Equation (1), and we assume that X 0 is not empty. The general idea of obtaining an approximate analytical solution to Equation (1) is to transform to an unconstrained nonlinear programming problem by any suitable technique such as augmented Lagrange method, barrier method, and penalty method [1,2]; it can then be solved by any unconstrained optimization numerical method like the steepest descent method, conjugate gradient method, and Newton method. In optimization, the penalty method is the most efficient method to transform a constrained optimization problem into an unconstrained optimization problem [3][4][5]. An efficient penalty function for equality and inequality problem Equation (1) is given below where σ = 2. It can be seen that under some conditions, the solutions to Equation (1) are solutions of the unconstrained below [6], where μ > 0 is an auxiliary penalty variable. The corollary connecting the minimizer of the constraint problem in Equation (1) and unconstrained problem in Equation (4) is seen in [7]. The gradient descent method as a standard optimization algorithm has been widely applied in many engineering applications, such as optimization machine learning and image [8][9][10]. Through diverse research and studies, it is established that the gradient method is one of the most reliable and efficient ways to find the optimal solution of optimization problems [11]. Nowadays, one of the critical points of the gradient method is how to improve the performance further. As an important area of mathematics, fractional calculus is believed to be an excellent tool to enhance the old gradient descent method, mainly because of its special long memory characteristics and nonlocality [12][13][14]. In the past decade, several methods have been considered to solve unconstrained nonlinear optimization in the form of ordinary differential equation (ODE) dynamic system of which the gradient-based method is one of the approaches. The technique transforms the nonlinear optimization problem to an ODE dynamic system with some optimality conditions, to obtain optimal solutions to the optimization problem. The gradient-based method was first proposed by [15], was developed by [16,17], and was later extended to solve differential nonlinear programming problems [18]. However, the studies of nonlinear fractional-order gradient-based dynamic systems are still in the infant stage and are considered further in this paper. Arbitrary-order ODEs, which are the generalizations of integer-order ODEs, are mostly used to model problems in applied sciences. Several numerical methods had been used to solve linear and nonlinear problems of FDEs, such as the Adomian decomposition method (ADM) [19], variational iteration method (VIM) [20], homotopy perturbation method for solving fractional Zakharov-Kuznetsov equation [21], a numerical method for FDEs [22], and multivariate padé approximation (MPA) [23]. The usefulness of an arbitrary-order started receiving tremendous attention of researchers in the field of applied science and engineering in the last two decades where some authors in the area of optimization focused on developing approximate analytical methods for different types of nonlinear constrained optimization problems in the form of IVPs of nonlinear FDE systems including multistage ADM for NLP [24], a fractional dynamics trajectory approach [25], the convergence of HAM and application [26], fractional steepest descent approach [27], studied optimal solution of fractional gradient [28], gradient descent direction with Caputo derivative sense for BP neural networks [29], fractional-order gradient methods [30], and conformable fractional gradient-based system [31]. In 2008, Marinca and Herisanu [32] introduced a numerical method called OHAM to solve a nonlinear problem, later extended by Azimi et al. [33] for strong nonlinear differential equations (NLDEs). This powerful tool called OHAM has not been applied in the area of FOGBDS, which motivates this work.
So, in this paper, we showed that the steady-state solutions xðtÞ of the proposed system can be approximated analytically to the expected exact optimal solution x * of the nonlinear programming constrained optimization problem by OHAM-LS as t ⟶ ∞. The significant contribution is summarized as follows: (1) The reason why OHAM-LS is preferable to be the method used [25,31] to solve FOGBDS (2) The reason why some existing approximate analytical method cannot guarantee the convergence of the series solution is discussed (3) From the previous approximation analytical method of solving FOGBDS, accurate optimal values control-convergence parameter had been a little bit difficult to achieve which is easily address with least square optimization techniques (4) OHAM-LS with guaranteed convergence ability is proposed with conformable fractional derivative sense to solve FOGBDS. The fastest convergence ability of the proposed compared with fourth-order Runge-Kutta is also shown We arrange the paper as follows: a brief introduction to the fractional calculus and OHAM-LS derivation is given in Section 2. Section 3 is devoted to problem formulation of OHAM-LS with FOGBDS and the key contributions. In Section 4, we solved some NLP constrained optimization problems to show the effectiveness of the proposed method. The results obtained from OHAM-LS are plotted in several figures with numerical method comparisons to confirm the validity and ability of the method to solve the problem. In the last section are the conclusions.

Preliminaries
2.1. Fractional Calculus. The most common arbitrary-order in literature is the Riemann-Liouville's and the Caputo fractional derivative. The arbitrary-order definitions are generally used for mathematical modelling within many areas, especially when the classical-order derivative operator fails or additional memory effect is required. However, the limitation of these two definitions is that they do not provide some of the features that the classical derivative provides, such as chain rule, quotient rule, product rule, and derivative of Advances in Mathematical Physics constant. Recently, Khalil et al. [34] have characterized a new fractional derivative operator, which is an extension of the usual conformable fractional derivative, to overcome these deficiencies. Besides these advantages, the conformable fractional derivative does not show the memory effect, which is inherent for the other classical fractional derivatives.
Definition 1. Let f : ½0, ∞Þ ⟶ R be a given function. The α th order CFD of f given by ∀x > 0 and α ∈ ð0, 1 This new definition preserves many properties of the classical derivatives refer to [34,35]. Some features that we will adopt are as follows: where the integral is the regular Riemann improper integral, and α ∈ ð0, 1. We start from the fundamental principle of OHAM as described in [36][37][38]. Consider the IVPs with initial conditions where L i is a linear operator, N i is a nonlinear operator, t is an independent variable, z i ðtÞ is an unknown function, φ is the problem domain, and g i ðtÞ is a known function. According to OHAM, one can construct an homotopy map H i ðϕ i ðt, pÞ: φ × ½0, 1 ⟶ φ which satisfies where p ∈ ½0, 1 is an embedding parameter, H i ðpÞ is a nonzero auxiliary function for p ≠ 0, Hð0Þ = 0, and ϕ i ðt, pÞ is an unknown function. Obviously, when p = 0 and p = 1, it holds that ϕ i ðt, 0Þ = z i,0 ðtÞ and ϕ i ðt, 1Þ = z i ðtÞ, respectively. Thus, as p varies from 0 to 1, the solution ϕ i ðt, pÞ approaches from z i,0 ðtÞ to z i ðtÞ where z i,0 ðtÞ is the initial guess that satisfies the linear operator which is obtained from Equation (8) for p = 0 as H i ðpÞ is chosen in the form where C j would be determined in the last part of this work. We consider Equation (8) in the form Now substituting Equation (11) in Equation (8) and equating the coefficient of like power of p, we obtain the governing equation of z i,0 ðtÞ in a linear form, given in Equation (9). The firstand second-order problems are given by and the general governing equations for z i,k ðtÞ are given by where N i,m ðz 0 ðtÞ, z i,1 ðtÞ ⋯ , z i,m ðtÞÞ is the coefficient of p m , obtained by expanding N i ðϕ i ðt, p, C j ÞÞ in series with respect to the embedding parameter p where ϕ i ðt, p, C j Þ is obtained from Equation (11). It should noted that z i,k for k ≥ 0 is governed by the linear Equations (9), (12), and (14) with linear initial conditions that come from the original problem, which can be easily solved. It has been shown that the convergence of the series Equation (16) depends upon the C j . If it is convergent at p = 1, we have The result of the mth-order approximation is given as 3

Advances in Mathematical Physics
Substituting Equation (18) in Equation (6), we get the following expression for the residual If R i ðt, C j Þ = 0, thenz i ðt, C j Þ is the exact solution. Usually, such a case does not arise for nonlinear problems. Several methods [39,40] can be used to find the optimal values of convergence-control parameters C j ′s like the method of the least square method, collocation method, Ritz method, and Galerkin's method. By applying the least square method, we have minimized the functional where the value a and b depends on the given problem.
With these known C j ′ s, the approximate solution (of mth-order) is well determined. The correctness of the method by (1) Error Norm L 2 .
The OHAM-LS is based on hybridization of OHAM with the least square method of optimization technique. The OHAM enable us to determine the convergence domain of the series solution, and the least square method allows us to obtain the optimal values of the C s k .
Remark 7. The existing approximate analytical for FOGBDS cannot guarantee convergence mainly because they possess no criteria for the establishment for convergence of the series solution Equations (20) and (21).

Construction of OHAM-LS with FOGBDS Generated by NLPCOPs
We begin by considering a NLP constrained in the form where f : R n ⟶ R is the objective function, h k ðxÞ: R n ⟶ R are equality constraint functions, g k ðxÞ: R n ⟶ R are inequality constraint functions, and C 2 are continuous differentiable functions. One of the main ideas of solving unconstrained NLP is by searching for the next point by choosing proper search direction d k and the stepsize α k as in the Newton direction [45], trust-region algorithm for unconstrained optimization [46]; the descent method [47], conjugate gradient method [48], three-term conjugate gradient method [49], and subspace method for nonlinear optimization [50]; the hybrid method for convex NLP [51]; CCM for optimization problem and application [52]; and descent direction stochastic approximation for optimization problem [53]. But there are studies for other approaches. In this paper, we obtain the minimizing point of the problem by solving a certain initial-value system of FDEs. This kind of FOGBDS was first proposed by Evirgen and Özdemir [24]. Using the penalty function Equation (2) and (3) for Equation (24) with ρ = 2, the conformable FOGBDS model can be constructed as subject to the initial conditions where ∇ x Fðx, μÞ is the gradient vector of Equation (25) with respect to x k ∈ R n and T α is the CFD of 0 < α ≤ 1.
Note that a point x e is called an equilibrium point of Equation (25) if it satisfies the RHS of Equation (25). We reformulate fractional dynamic system Equation (25) as We used OHAM-LS to obtain the solution of system Equation (27) by constructing the following homotopy where k = 1, 2 ⋯ , n and p ∈ ½0, 1. If p = 0, Equation (28) becomes and when p = 1, the homotopy Equation (28) becomes Advances in Mathematical Physics subject to the initial conditions, The correction functional for the system of conformable fractional nonlinear differential equation Equation (30), according to OHAM-LS, can be constructed as Thus as p varies from 0 to 1, the solution φ k ðt, pÞ approaches from x k,0 ðtÞ to x k ðtÞ where x k,0 ðtÞ is the initial guess that satisfies the linear operator which is obtained from Equation (32) for p = 0 as where C j can be determined later. We get an approximate solution by expanding φ k ðt, p, C j Þ in Taylor's series with respect to p; we have Now using Equation (35) in Equation (32) and equating the coefficient of like power of p, we obtain the governing equation of x i,0 ðtÞ in a linear form, given in Equation (33). The 1st-and 2nd-order problems are given by and the general governing equations for x k,i ðtÞ are given by where N k,m ðx 0 ðtÞ, x k,1 ðtÞ ⋯ , x k,m ðtÞÞ is the coefficient of p m , obtained by expanding N k ðφ k ðt, p, C j ÞÞ in series with respect to p.
It has been shown that the convergence of the series Equation (38) depends upon the C j . If it is convergent at p = 1, one has The solution of Equation (30) is determined approximately in the form, Substituting Equation (40) in Equation (30), we get the following expression for the residual error If R k ðt, C j Þ = 0, thenx k ðt, C j Þ is the exact solution. Usually, such a case does not arise for nonlinear problems. Using the least square method as below minimizes the functional where the value of a and b depends on the given problem.
With these known C k , the analytical approximate solution (of mth-order) is well determined.
The steps for optimal homotopy asymptotic methodleast square (OHAM-LS) are as follows: Step 1. We transform the nonlinear constrained optimization problem to the unconstrained optimization problem by a penalty method.
Step 2. We find the gradient of the unconstrained optimization problem, with given initial conditions.
Step 3. We choose the linear and nonlinear operators for OHAM-LS.
Step 4. We construct homotopy for the conformable fractional nonlinear differential equation which includes embedding parameter, auxiliary function, and the unknown function.
Step 5. We substitute the series solution results into the governing equation and equate to zero for an exact solution. Usually, such case a does not arise in nonlinear problems.

Advances in Mathematical Physics
Step 6. We find the optimal values for C j ′ s by using the optimization method called least square method, for good analytical approximate solution.

Numerical Examples and Results
In Minimize f x ð Þ = 100 whose exact solution is not known, but expected optimal solution is x * 1 = 1:9993, x * 2 = 3:9998. First, we transform the constraint problem to an unconstrained problem by quadratic penalty function for σ = 2; then, we have where μ ∈ R + , and so that the nonlinear FOGBDS can be given as where 0 < α ≤ 1. By using OHAM-LS with auxiliary penalty variable μ = 200, the terms of the OHAM-LS solutions for fractional order are acquired by using the concept of homotopy. According to Equation (6)), we choose the linear and nonlinear operators in the following forms:

Advances in Mathematical Physics
We can construct the following homotopy where Substituting Equations (56)-(58) into Equations (54) and (55) and equating the coefficient of the same powers of p result to the following set of linear FDEs.

Example 2. Consider the NLPCOPs test problem from Schittkowski [54] [No 320].
Minimize This is a practical problem, and the exact solution is not known, but the expected optimal solution is x * 1 = 9:395, x * 2 = −0:6846. First, the quadratic penalty function is used to get the unconstrained optimization problem as follows: where μ ∈ R + and so that the nonlinear FOGBDS be given as By using OHAM-LS with μ = 10 6 , the terms of the OHAM-LS solutions for fractional order are acquired by using the concept of homotopy. According to Equation (6), we choose the linear and nonlinear operators in the following forms:   Table 2: Comparisons and absolute error between OHAM-LS and RK4, α = 1.
Minimize f x ð Þ = x 2 1 + x 2 2 + 2x 2 This is a practical problem, and the exact solution is not known, but the expected optimal solution is x * 1 = 0, x * 2 = 1, x * 3 = 2, and x * 4 = −1. From the above procedure, the secondorder approximate solution obtained by OHAM-LS at α = 0:9, for p = 1, is and for x 2 2 ðtÞ, Substituting these optimal values into Equations (106)-(109), we havẽ  Tables 5 and 6 show the C k at different values of α for Example 3. Tables 7 and 8 show the comparisons and the absolute error between OHAM-LS and RK4 at α = 1. Also, Figure 3 shows the comparisons of OHAM-LS at α = 1, 0:9, 0:8, and 0:7 with RK4 at α = 1, which verifies the performance of the present method as an excellent tool for NLPCOPs. For α = 1, it can be seen that the approximate analytical solution agrees with the ideal solution. Thus, as α approaches 1, the classical solution for the system is recovered.  Variable

Conclusions
In this paper, we implemented OHAM-LS for solving nonlinear FOGBDS from the optimization problem. The fractional derivative is considered in a new conformable fractional derivative sense. The optimization minimization approach of the least square method helps to obtain optimal values of the C s j for accurate approximate analytical solutions. The comparisons between the fourth-order Runge-Kutta (α = 1) and OHAM-LS show that our present method performs rapid convergence to the expected optimal solutions of the optimization problem. The results obtained are in close agreement with the exact solution, and those from the RK4 and OHAM-LS are reliable, dependable, and efficient for finding an approximate analytical solution for nonlinear FOGBDS optimization problem.