A New Stochastic Technique for Painlevé Equation-I Using Neural Network Optimized with Swarm Intelligence

A methodology for solution of Painlevé equation-I is presented using computational intelligence technique based on neural networks and particle swarm optimization hybridized with active set algorithm. The mathematical model of the equation is developed with the help of linear combination of feed-forward artificial neural networks that define the unsupervised error of the model. This error is minimized subject to the availability of appropriate weights of the networks. The learning of the weights is carried out using particle swarm optimization algorithm used as a tool for viable global search method, hybridized with active set algorithm for rapid local convergence. The accuracy, convergence rate, and computational complexity of the scheme are analyzed based on large number of independents runs and their comprehensive statistical analysis. The comparative studies of the results obtained are made with MATHEMATICA solutions, as well as, with variational iteration method and homotopy perturbation method.


Introduction
The history of Painlevé equations is more than one century old. These are six second-order nonlinear irreducible equations that define new transcendental functions known as Painlevé Transcendents. These functions describe different physical processes and have been extensively used in both pure [1] and applied mathematics [2], along with theoretical physics [3]. For instance, Painlevé Transcendents are used in solutions to Korteweg-de Vries (KdV), cylindrical KdV and Bussinesq equations [4,5], bifurcations in nonlinear nonintegral models [6], matrix models of quantum gravity with continuous limits [7,8], among others. The history, importance, and applications of Painlevé transcendents can be seen elsewhere [9,10]. In recent publications, Painlevé equation-I (PE-I) is used in Tronquée [11] and hyperasymptotic [12] solutions, as well as in modeling the viscous shocks in Heleshaw flow and Stokes phenomena [13]. Beside this, many researchers have employed PE-I in diverse fields of applied science and engineering [14][15][16][17][18][19][20][21][22].
In this article, our investigation is confined to find out solution of initial value problem (IVP) of nonlinear secondorder PE-I, written in the following form: y (t) = 6y(t) 2 + t, t ∈ (0, 1), In the present study, the strength of feed forward artificial neural networks (ANNs) is exploited for approximate mathematical model of PE-I. The real strength of such model to solve the differential equations can be achieved by using modern stochastic solvers for optimization of weights based on particle swarm optimization (PSO) technique hybrid with local search methods. For example, the solution of linear and nonlinear differential equations and their systems is reported 2 Computational Intelligence and Neuroscience extensively by many researchers [23][24][25][26]. Recently, solution of well-known fractional-order systems in engineering based on Bagley-Torvik and Riccati equations are other illustrative examples of such approaches [27,28].
In this paper, ANNs supported with PSO and active set algorithm (ASA) are used to find the solution of IVP of PE-I. Monte-Carlo simulations of proposed scheme are performed and analyzed statistically to verify and validate its effectiveness. Comparison with standard MATHEMATICA, as well as homotopy perturbation method (HPM) [29,30] and variational iteration method (VIM) [29,30] results is also carried out for the given scheme.
The organization of the paper is as follows: the objective function creation with neural network mathematical modeling and its training methodology are introduced in Section 2. A brief introduction to particle swarm optimization algorithm is revealed in Section 3. A detailed application of the designed method along with some discussion on the results is presented in Section 4. Comparative analysis of the results is presented in Section 5. Last section concludes the findings along with some directions for future research.

Mathematical Modeling
The Feed-forward ANNs is well known to be used as a universal function approximator for any continuous function along with its derivatives on a compact set. In the modeling of differential equation through ANN methodology, following continuous mapping is used for the solution y(t), firstorder derivative y (t) and second-order derivative y (t), respectively, given as [31][32][33][34] where α i , w i , and β i are real-valued bounded adaptive parameters, m is the number of neurons, and f being the activation function taken as log sigmoid for hidden layers: The linear combination of the networks represented by (2) can arbitrarily model the nonlinear PE-I. It is called differential equation neural network (DE-NN) of the equation and its architecture is shown in Figure 1.
The objective or fitness function e for solving PE-I has been formulated by defining an unsupervised error, given as the sum of mean square errors: Hidden layer Output where j is the iteration number, M represents the total number of iterations, and the error term e 1 is related with the differential equation and is written as Here, the interval t ∈ (0, 1) is divided into N number of steps t ∈ (t 0 = 0, t 1 , t 2 , . . . , t N = 1) with step size h and y(t) and y (t) are DE-NN given in set of (2).
Similarly, the error term e 2 is due to initial conditions and it is given as It is quite clear that, with the availability of weights α i , w i , and β i in DE-NN model for which the values of functions e 1 and e 2 approach zero, the value of unsupervised error e also approaches zero. Therefore, the solution y(t) of the PE-I is approximated by y(t) as given in (2).

Learning Methodology
The learning procedures are introduced here for finding the unknown weights of DE-NN networks representing PE-I using PSO and ASA algorithms. The swarm intelligence techniques often referred to as PSO algorithm was first introduced by Kennedy and Eberhart [35]. It is a kind of global optimization technique which was based on the model of social behavior of bird flocking and fish schooling. Its discrete and continuous versions have been developed and used in different optimization problems of applied science and engineering. Few examples include the application to mobile communications, sensor networks, inventory control, multiprocessor scheduling, controls, stock market prediction, and steganographic methods [36][37][38][39] and so forth.
In PSO algorithm, each single candidate solution to an optimization problem is taken as a particle in the search Computational Intelligence and Neuroscience  space. The exploration of a problem space in PSO algorithm is made with a population of particles called a swarm. All particles in the swarm have own fitness values associated with problem specific objective function. The PSO algorithm is initialized with a swarm of particles randomly and is used to search for optimal solution iteratively. The broader spread of initial swarm results in optimal performance of the algorithm. The position and the velocity of each particle are updated according to its known previous local best position P n−1 lb and the global best position of all particles P n−1 gb in the swarm so far. The updating formula for each particle velocity and position in continuous standard PSO is written as where x i is vector to represent ith particle of the swarm, i = 1, 2, . . . , p, p is an integer giving the total number of particles in a swarm. The v i is the velocity vector associated with ith particle, c 1 and c 2 are the local and global social acceleration constants, respectively, ω is the inertia weight linearly decreasing over the course of search between 0 and 1, and r 1 and r 2 are random vectors with elements distributed between 0 and 1.

The elements of velocity are bounded as
where v max is maximum velocity. If the velocity goes beyond its maximum value, it will be set to v max . This parameter has its own importance and controls the convergence rate of the algorithm. The execution of the algorithm is terminated on the criterion to maximum flights/cycles is completed or a specific value of the fitness is achieved. The generic flowchart showing the process of proposed algorithm is presented in Figure 2.
The algorithm is given in the following steps.
Step 1. Initialization. The initial swarm of particle is generated randomly using bounded real values. Each particle represents the candidate solution with as many members as number of unknown parameters in DE-NN networks.
The scattering in values of initial swarm is maintained for better search space for the algorithm. Initialize the values of parameters as given in Table 1 before execution of algorithm.
Step 3. Termination Criteria. Terminate the algorithm if any of following criteria satisfies the following: (i) predefined value of fitness achieved that is, e ≤ 10 −10 , (ii) number of maximum flights/cycles is executed.
If yes, then go to Step 6.
Step 4. Ranking. Rank each particle on the basis of minimum of the fitness function values.
Step 5. Renewal. Update the velocity and position are updated using the equations given in (7). Repeat the algorithm from Step 2 to Step 5 until total number of flights is reached.
Step 6. Refinement. MATLAB built-in formulation is used for running active set algorithm (ASA) for fine tuning of the results. The global best particle of PSO is used as a start point of the algorithm and other parameter settings for ASA is also provided in Table 1.
Step 7. Statistical Analysis. Store the value of global best particle along with its fitness value and time of executions for algorithm. Repeat Step 1 to 6 for sufficient large number of runs for reliable statistical analysis.

Simulation and Results
The well-known analytic solvers like adomian decomposition method (ADM), variational iteration method (VIM), homotopy perturbation method (HPM), homotopy analysis method (HAM), and their modified versions are used extensively to provide the solutions of strong nonlinear Painlevé equations [29,30,[40][41][42]. The generic solution is provided in all these mentioned methods in the form of infinite series, but the results are determined using finite number of terms, whose accuracy can be enhanced with the increase in number of terms. Before describing the designed approach, a brief description of VIM and HPM has been provided for PE-I. The approximate solution of y(t) for PE-I (1) in case of both 4 Computational Intelligence and Neuroscience VIM and HPM is represented in the form of a function u(t). To describe the VIM, consider the following nonlinear differential equation: where L and N are linear and nonlinear operators, respectively, and g(t) is an inhomogeneous term. In standard VIM, the correct functional can be constructed as here, λ(τ) represents a general Lagrangian multiplier, n and n + 1 are the present and next approximation, u represents restricted variation, that is, δ u = 0. With suitable choice of u 0 , the fixed point of the correction functional (9) can be considered as an approximate solution of the equation and successive approximation u n+1 of the solution u will be readily obtained upon by determining the Lagrange multiplier. Consequently, the solution is given by u = lim n → ∞ u n . For interesting readers, the detail description of VIM along with its applicability to various kinds of differential equations can be seen in [43][44][45][46]. The governing iterative formulation for solving the PE-I using VIM with the optimal value for λ(τ) = τ − t can be written as The approximate solution u(t) by VIM for first three iterations using u 0 (t) = t + (1/6)t 3 can be written as On the other hand to explain the HPM, let us consider the following general equation of type: where A represents the differential operator, Ω is the domain and g(t) is a known analytical function. The operator A can consist of linear part L and a nonlinear part N. In standard HPM, the Hopotopy v(r, p) : Ω × [0, 1] → R can be constructed which satisfies here, p ∈ [0, 1] is an embedding parameter and u 0 is an initial approximation of (12). By using small embedding parameter p, the vth solution of (12) can be given as power series in p, that is, for p = 1, the approximate solution for (12) is given as The method provides enough convergence in most of the cases and the rate of convergence depends on the nonlinear operator A(v). The detail description of the HPM and its applications can be seen in [47][48][49][50].
Computational Intelligence and Neuroscience 5 Using HPM, the solution of PE-I (1) starting from u 0 (t) = t + (1/6)t 3 can be given as using the initial conditions (1), the homotopy constructed by the method can be given as Inserting ∞ i=0 p i u i instead of u and comparing the coefficients of 1, 2, 3, 4, and 5 powers of p, the approximate solutions by HPM for PE-I are, respectively, given as Along with the analytical solutions given by VIM and HPM, the results are also calculated with MATHEMATICA y M (t). These results are used for comparison with the solution provided by the proposed design scheme.
Mathematical model of the equation is made with DE-NN networks given by (2) The PSO algorithm runs iteratively in order to compute the minimum of fitness function (19) and the best particle of PSO technique is passed to ASA algorithm as a start point for rapid local search. The parameter settings for the execution of algorithms are provided in Table 1.
Similarly, hundred independent runs of PSO and PSO hybrid with ASA (PSO-ASA) algorithms have been performed for finding the optimal weights. The weights obtained by the algorithms are used in (2) to obtained the proposed solutions y(x) of PE-I. One set of weights for which the value of fitness function 1.3699×10 −04 and 9.1210×10 −08 for PSO and PSO-ASA, respectively, are provided in Table 2. Results calculated by these weights are provided in Tables 3  and 4 for inputs between 0 and 1 with a step of 0.1. The solutions of 3rd order VIM, 5th order HPM for the same inputs are also given in Table 3 while detailed information has been provided in Table 4 Figure 3(b). On the basis of acceptability criteria of MAE ≤ 10 −03 , the convergence for PSO and PSO-ASA is 1% and 100%, respectively.

Comparative Analysis of Results
In this section, the comparative study for results is presented based on values of MAE, execution time (ET), mean fitness achieved (MFA), global mean absolute error (GMAE), 6 Computational Intelligence and Neuroscience       (19) are analyzed. By taking the number of grid points 21 instead of 11, the new fitness function is formulated using the input of the training set t ∈ (0, 1) with a step of h = 0.05 and given as  Table 6. It is found that the values of MFA are of the order 10 −02 and 10 −06 for PSO and PSO-ASA, respectively. The values of MAE are matched with MATHEMATICA solution y M (t) upto 2 to 3 and 5 to 8 decimal places for PSO and PSO-ASA techniques. The value of GMAE for 11 and 21 grids points are determined and given in Table 6. It is found that values of GMAE are of the order 10 −04 for both the cases for inputs between 0 and 1 with a step of h = 0.1. So, by increasing number of grid points the same level of the accuracy in results is maintained for both PSO and PSO-ASA techniques. The increase in number of grids points, that is, decrease in the mesh size h, does not guaranty improvement of results for stochastic solvers and same is observed in our simulations by changing the value of step size h = 0.1 to h = 0.05.
The convergence rates of the proposed stochastic solvers are determined based on the value of MAE ≤ 10 −02 , 10 −03 , and 10 −04 , as well as the value of MFA e ≤ 10 −03 , 10 −04 , and 10 −05 . Results are tabulated in Table 6 for both PSO and PSO-ASA techniques. It can be seen that, with the acceptability 8 Computational Intelligence and Neuroscience of the results based on value of MFA e ≤ 10 −05 , the hybrid approach PSO-ASA achieved this criteria for 99% and 98% of the independents runs for 11 and 21 numbers of grid points, respectively. In addition to that, results are consistently convergent for larger number of independents runs for PSO-ASA method than that of PSO algorithms. Moreover, the computational complexity is analyzed by  Table 6. It can be seen that values of M-ET for PSO and PSO-ASA techniques are 76.93 seconds and 92.17 seconds, respectively, for h = 0.1, and almost double values of M-ET in case for h = 0.05. The time taken for the execution for hybrid approach PSO-ASA is relatively greater than that of PSO algorithm, but this can be overshadow by its invariable dominance in accuracy and convergence. The time analysis provided in this article is carried out using Dell Latitude D630 laptop with Intel(R) Core(TM) 2 Duo CPU T9300, 2.50 GHz, and running with MATLAB version R2009b.

Conclusion
On the basis of the simulations and their comparative studies provided in the last sections, it can be concluded that the IVP of PE-I can be solved effectively by the designed computational intelligence algorithms using neural networks supported with PSO and ASA techniques. Results obtained using the weights for DE-NN networks of PE-I trained by PSO-ASA algorithm are found to be better than from PSO techniques. The values of absolute error from MATHEMATICA solution y M (t) of PSO-ASA lies in the range 10 −04 to 10 −07 and match upto 5 to 7 decimal places with state of art analytical techniques of VIM and HPM.
On the basis of 100 independent runs for each algorithm to solve PE-I using 11 and 21 grid points, the PSO-ASA technique is found to be invariable superior than PSO algorithm using the criteria based on the values of mean absolute error, global mean absolute error, mean fitness achieved, and convergence rate. However, as far as computational complexity is concerned that PSO-ASA technique is slightly more computational exhaustive, but it can be overlook due to its overall dominance in the performance. This study can also be extended to use other biological inspired computational algorithms for solving the PE-I, as well as other Painlevé transcendents.