Solving Partial Differential Equations Using a New Differential Evolution Algorithm

This paper proposes an alternative meshless approach to solve partial differential equations (PDEs). With a global approximate function being defined, a partial differential equation problem is converted into an optimisation problem with equality constraints fromPDEboundary conditions. An evolutionary algorithm (EA) is employed to search for the optimumsolution. For this approach, the most difficult task is the low convergence rate of EA which consequently results in poor PDE solution approximation. However, its attractiveness remains due to the nature of a soft computing technique in EA. The algorithm can be used to tackle almost any kind of optimisation problem with simple evolutionary operation, which means it is mathematically simpler to use. A new efficient differential evolution (DE) is presented and used to solve a number of the partial differential equations. The results obtained are illustrated and compared with exact solutions. It is shown that the proposed method has a potential to be a future meshless tool provided that the search performance of EA is greatly enhanced.


Introduction
In the field of computational mechanics, many engineering systems can be viewed as continuum domains and their physics are represented by using differential equations, for example, heat transfer, structures, and fluid mechanics.Some of such differential equations can be solved analytically; however, in most practical cases, the domains are too complicated for an analytical approach.This leads to the creation of numerical approximation of the differential equations.Some of the established approaches include finite element methods [1][2][3][4], finite different methods [5,6], and finite volume methods [7][8][9].Those previously mentioned methods are classified as the mesh-based or grid-based methods.The concept is to approximate a complicated domain of differential equations with a finite number of simple grids enabling one to find the approximate solutions of the differential equations.
Nevertheless, although the mesh-based methods like finite element analysis have become a traditional engineering tool with numerous commercial software packages being developed around the world, they still have some drawbacks, for example, difficulties in mesh generation and mesh distortion in large deformation analysis of structures.This causes researchers and engineers to search for alternative numerical approaches and the so-called meshless methods were originated.The method unlike the traditional finite element or finite volume methods uses only distribution of nodes on a differential equation domain to achieve its approximate solution.Over the last few decades, there have been numerous meshless methods proposed to solve a number of engineering applications.Starting with one of the earliest methods, the smooth particle hydrodynamics [10,11] approach; then, there have been other established methods such as the element-free Galerkin method [12], the reproducing kernel particle method [13], and the local Petrov-Galerkin method [14][15][16].A good review on meshless methods can be found in [17,18].For some up-to-date development, see [19][20][21][22].The meshless methods, despite having some advantages, have some weak points such as difficulties in dealing with boundary conditions (BCs).Therefore, more research and ideas are still needed in this field.
This paper proposes an alternative way to achieve solutions of partial differential equations (PDEs).The approach is classified as a meshless method since only nodes are generated on a differential equation domain.A partial differential equation problem is converted into an optimisation problem while a metaheuristic (MH) or an evolutionary algorithm (EA) is employed to search for the optimum solution.Such an idea has been presented in [23,24] but there are many obstacles to overcome if one wants to use it in practice.The most important issue is the low convergence rate of EAs which consequently results in a not-so-good approximate solution.However, the attractiveness of this approach remains due to the nature of a soft computing technique in EA.The algorithm can be used to tackle almost any kind of optimisation problem with simple random-directed evolutionary operation, which means it is mathematically simpler to use.In this paper, a new efficient differential evolution (DE) is presented and used to solve a number of the partial differential equations.The results obtained are illustrated and compared with exact solutions.The rest of this paper is organised as follows.
The second section briefly reviews the differential evolution method where the new DE is proposed.Section 3 details how to solve the partial differential equations by means of evolutionary optimisation.The results are shown and discussed in Section 4 while conclusions are drawn in Section 5.

Differential Evolution Algorithm
Some of the earliest evolutionary algorithms were proposed a few decades ago where genetic algorithms are arguably the best known method.Since then, this kind of soft computing has gained increasing popularity year after year because of their simplicity to use, derivative-free feature, and robustness.So far, there have been a thousand of EAs and their variants developed and presented in research literature.Among such an incredibly large number of algorithms, differential evolution is one of the top performers for general optimisation purposes.The method is a parallel random-directed search method that was first proposed by Storn and Price [25].DE is powerful yet very simple to understand, code, and implement which resulted in its great popularity nowadays.A large number of DE variants have been proposed, for example, in [26,27].Similar to most EAs, a DE computational procedure starts with a randomly generated initial real-code population.Then the DE operators (mutation, crossover, and selection) are iteratively activated to improve the population approaching to an optimum.DE since it was first introduced has various schemes.In this paper, we use 4 versions of the original differential evolution algorithm: DE/rand/1/bin, DE/rand/1/exp, DE/current-to-best/2/bin, and DE/current-tobest/2/exp.Note that the notation of DE schemes is described as where  is the vector to be mutated ("rand" for a randomly chosen vector and "current-to-best" for a current population vector that is mutated by the best so far vector),  is a number of pair of different vectors used in the mutation operation, and  is a crossover type ("bin" for binomial crossover and "exp" for exponential crossover).

Differential Evolution Operators.
There are three main operators for DE which can be briefly detailed as follows.
2.1.1.Mutation.For a population vector x , ,  = 1, 2, 3, . . ., where  is a generation number and  is a population size, a mutant vector for the rand scheme is produced using (2) and for the current-to-best scheme is produced using (3).Consider where  1 ,  2 , and  3 are random integer indices of population vectors ( 1 ,  2 ,  3 ∈ {1, 2, 3, . . .}), x , is current population vector of last generation, x best is a population vector that has the best (minimum) fitness in the previous generation,  is a scaling factor ( ∈ [0, 2]) that controls amplification of difference vectors, and  is an additional control variable ( ∈ [0, 1]) that controls influence of the best population vector.

Crossover.
In the crossover operation, a target vector (x ,+1 ) will be bred with a mutant vector (k ,+1 ) leading to a trial vector (u ,+1 ).The probability of crossover ( ∈ [0, 1]) will be predefined.Once the operator is revoked, there can be two types of crossover as binomial and exponential crossovers which are given in Algorithms 1 and 2, respectively, where CR is a prespecified constant and  is a dimension of a design vector.

Selection.
Before adding trial vectors into a new generation population, the selection operator will compare the fitness function value of a trial vector to that of its target vector.Then the vector that has a lower fitness function value is selected into the new generation population.

Termination Criteria.
In this paper, we specify two stopping criteria.The calculation will be terminated when maximum function evaluation is reached or allowable minimum fitness function and constraint values are reached.

New Differential Evolution Algorithm.
In this work, we present new DE with variation of population size during an optimisation run.This DE scheme will be termed DE-New.The differential operators (Mutation, crossover, and selection) and stopping criteria of DE-New are the same as those used in the original DE.The new DE starts searching with an initial population size.The modification is based on the premise that a larger population size tends to produce some better design solutions.Nevertheless, using a large population size throughout the optimisation run can be time consuming.Thus, the concept of population size variation is employed.As the algorithm continues, if a convergence speed of the algorithm at the current iteration is not satisfied, then additional members will be generated and added to the population.The flowchart of DE-New is shown in Figure 1.
During the search process in which the best fitness for all iterations are obtained, if the standard deviation of the best fitness values of the previous npv iterations is less than a predefined value pvc, then the number of solution members in a population is increased but it must be limited in such a way that the total number of population members must not exceed a predefined maximum population size.In cases that the current population size is less than maximum population size and the aforementioned criterion is met, additional solution vectors will be generated and added to the population.Given that NP is the size of the current population, other NP solutions in which half of them are randomly generated (exploration) and the rest are created by means of DE operation (exploitation) will be added to the population.Note that, in this paper, DE/rand/1/exp and DE/current-to-best/2/exp are used with this population variation concept.The DE/rand/1/exp scheme is used to update a regular population for each iteration while the DE/current-to-best/2/exp scheme is used in the additional population generation process.

Numerical Examples
The proposed approach is somewhat similar to the literature [24] in the sense that a partial differential equation problem is altered to be a minimisation problem with equality constraints.Initially, nodes are generated inside and on the boundary of a PDE domain.A function for approximating a PDE solution is selected where boundary conditions are treated as equality constraints.The optimisation problem is posed to find function coefficients in such a way to minimise residuals or square errors.In the previous work, evolution strategies (ES) were used as an optimiser while an approximate function is a partial sum of Fourier cosine series.In this work, DE will be used instead of ES as it is found to be more powerful whereas a polynomial function is used to estimate the PDE solution.In order to demonstrate the performance of a new approach, 6 PDE problems are used for numerical test which can be detailed as follows: while BC means boundary condition.PDE solutions are approximated by a polynomial expansion as in (4).Consider where   are polynomial coefficients to be determined and  is a maximum of the polynomial order for  and .
The number of approximation terms is ( + 1) 2 , which means that there are  = ( + 1) 2 design variables for an optimisation problem.Since ( 4) is in a polynomial form, we can calculate any orders of derivatives, which is simple to be used with a wide range of differential equations.A typical partial differential equation can be written in a general form as with BCs ℎ  (, ) = ℎ 0, on Γ  for  = 1, . . ., .
Let  be the number of nodes being assigned on a PDE domain Ω and boundary where   is the number of nodes on the  boundaries Γ  .If we substitute (4) into a PDE for all points on Ω, we can have  equations with  unknowns.Similarly, by substituting (4) on the BCs for all points on Γ  , we have   equality constraints.This implies that we convert the PDE (5) to be a constrained optimisation problem where the  equations is replaced by an objective function to be minimised [23] which is calculated by means of a weighted which is subject to ℎ  (a) = 0 for  = 1, . . .,   , where  = { 1 , . . .,  (+1)×(+1) }  is a vector of design variables or polynomial coefficients, (, ) is a residual function, (, ) is the weight function, in this case, which is set as unity as used in [23],  is an area of the domain, and ℎ  are obtained from substituting (4) into the boundary conditions for all nodes on the boundaries.Figure 2 shows how to compute the integration (6) on a particular subarea which is discretised by the nodes on the domain.The value of the residual function || on the subarea  is the average value of || from those 4 points.The ideal solution is to have the objective function WRF becoming zero.
In order to handle the equality constraints ℎ 0 , a penalty function is used which can be expressed as where   are penalty parameters (set to be 50 in this study).Thus, the fitness function (FFV) for DE can be calculated as The accuracy of the final solutions is measured using a root mean squared error (RMSE) which can be computed as The optimisation parameter settings are given as  = 0.7,  = 0.5,  = 0.8, and  = 0.9.An optimiser is stopped when either the maximum number of function evaluations (500,000) is reached or the objective and penalty function values are less than 0.001.The best solution of the last generation of DE is classified as an optimum.Population variation criteria are npv = 10 and pvc = 0.001.The population size for the original DE is set to be 400 while the initial population size for DE-New is set as 100.The maximum population size for DE-New is 400.For all PDE problems, the number of nodes is 100.Nodes are equispaced throughout the square domains for PDE1-4.The nodal distributions for PDE5-6 are illustrated in Figures 3 and 4, respectively.

Results and Discussion
The numerical experiment was carried out on a personal computer where the program is written using MATLAB computing language.Having employed five DE strategies for tackling six optimisation problems of PDE1-6, the results obtained are compared in Table 1.In the table, there are   3 and 4.
For approximation accuracy based on RMSE, the results from the literature [24] are compared.It should be noted that, in [24], different fitness function is used, thus, optimisation convergence from [24] and this work cannot be compared.Also, the nodal distributions of PDE5-6 from [24] are also different from those used in this work.From the table, DE-New gives the best results for PDE1-3 while the approach in [24] gives the best results in cases of PDE4-6.Nevertheless, it should be noted that the previous work used ES in combination with a partial sum Fourier cosine series with approximately 1,000,000 function evaluations which is twice the total number of function evaluations used in this paper.DE-New significantly outperforms all other DE versions for PDE1-3.It is said to be as good as DE/ran/1/exp in cases of PDE4 while DE/ran/1/exp along with DE/ran/1/bin is slightly better for the last two test cases.
Graphical comparisons of the approximate solutions from using DE-New and the exact solutions of PDE1-6 are displayed in Figures 11, 12, 13, 14, 15, and 16, respectively.The  left-hand side is an approximate solution while the righthand side is the exact solution.It is shown that DE-New gives good approximation for PDE1-4 while the approximation of PDE5-6 is not so good.One of the notable reasons is that PDE5-6 have nodal distribution with poor quality.Figure 15 (PDE5) shows that the number of nodes on the boundary is insufficient.A similar conclusion can be said for PDE6 (Figure 16).

Conclusions and Future Work
An alternative numerical meshless approach for solving PDEs based on evolutionary computation is proposed.A PDE problem is converted into an optimisation problem while, in this paper, DE is used as an optimiser.A new DE with a population variation concept is proposed.The results show that the proposed method in this paper is comparable to the previous work while using approximately twice less the number of function evaluations in the optimisation process.Most of the approximate solutions of PDE are accurate except for some cases which use poorly distributed nodes.The performance of an optimiser is probably the most important factor for this type of meshless analysis.The choice of approximate function also affects the accuracy of estimating the PDE solutions.Apart from that, nodal distribution also plays a vital role.The weak points can be listed as difficulty in dealing with equality constraints of a metaheuristic, and its slow convergence rate.Moreover, the calculation of WRF is, to some extent, difficult in the sense that adjacent nodes for each subarea have to be identified initially.For our future work, the performance enhancement of EAs or MHs is the first issue to be dealt with.EAs for optimisation with equality constraints should be rigorously investigated.The hybrid between EAs and some efficient gradient-based optimisers will be first focused.The choice of

Figure 1 :Figure 2 :Figure 3 :
Figure 1: Flowchart of new DE with population size variation.
ya x i s x -a x is (b)

Table 1 :
Fitness and RMSE.where (  ,   ) is an approximate function value at node ,  exact (  ,   ) is the exact function value at node , and  is the number of nodes.