ANAAdvances in Numerical Analysis1687-95701687-9562Hindawi Publishing Corporation91281010.1155/2012/912810912810Research ArticleApproximation Solution of Fractional Partial Differential Equations by Neural NetworksAlmarashiAdel A. S.SyamMuhammed I.Department of MathematicsFaculty of EducationThamar University, ThamarYementhuniv.edu.ye201227122011201201102011211120112012Copyright © 2012 Adel A. S. Almarashi.This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Neural networks with radial basis functions method are used to solve a class of initial boundary value of fractional partial differential equations with variable coefficients on a finite domain. It takes the case where a left-handed or right-handed fractional spatial derivative may be present in the partial differential equations. Convergence of this method will be discussed in the paper. A numerical example using neural networks RBF method for a two-sided fractional PDE also will be presented and compared with other methods.

1. Introduction

In this paper, I will use neural network method to solve the fractional partial differential equation (FPDE) of the form:u(x,t)t=  c+(x,t)αu(x,t)+xα+c-(x,t)αu(x,t)-xα+s(x,t).

On a finite domain L<x<R, 0tT. Here, I consider the case 1α2, where the parameter α is the fractional order (fractor) of the spatial derivative. The function s(x,t) is source/sink term . The functions c+(x,t)0 and c-(x,t)0 may be interpreted as transport-related coefficients. We also assume an initial condition u(x,t=0)=F(x) for L<x<R and zero Dirichlet boundary conditions. For the case 1<α2, the addition of classical advective term -v(x,t)u(x,t)/x on the right-hand side of (1.1) does not impact the analysis performed in this paper and has been omitted to simplify the notation.

The left-hand (+) and right-hand (-) fractional derivatives in (1.1) are the Riemann-Liouville fractional derivatives of order α  defined by(DLα+f)(x)=dαf(x)d+xα=1Γ(n-α)dndxnLxf(ξ)(x-ξ)α+1-ndξ,  (DRα-f)(x)=dαf(x)d-xα=(-1)nΓ(n-α)dndxnxRf(ξ)(x-ξ)α+1-ndξ, where n is an integer such that n-1αn. If α=m is an integer, then the above definitions give the standard integer derivatives, that is(DLm+f)(x)=dmf(x)dxm,(DRm-f)(x)=(-1)mdmf(x)dxm=dmf(x)d(-x)m.

When α=2, and setting c(x,t)=  c+(x,t)+c-(x,t), (1.1) becomes the following classical parabolic PDE:u(x,t)t=c(x,t)2u(x,t)x2+s(x,t).

Similarly, when α=1 and setting c(x,t)=  c+(x,t)-c-(x,t), (1.1) reduces to the following classical hyperbolic PDE:u(x,t)t=c(x,t)u(x,t)x+s(x,t).

The case 1<α<2 represents a superdiffusive process, where particles diffuse faster than the classical model (1.4) predicts. For some applications to physics and hydrology, see .

I also note that the left-handed fractional derivative of f(x) at a point x depends on all function values to the left of the point x, that is, this derivative is a weighted average of such function values. Similarly, the right-handed fractional derivative of f(x) at a point x depends on all function values to the right of this point. In general, left-handed and right-handed derivatives are not equal unless α is an even integer, in which case these derivatives become localized and equal. When α is an odd integer, these derivatives become localized and opposite in sign. For more details on fractional derivative concepts and definitions, see [1, 3, 5, 6]. Reference  provides a more detailed treatment of the right-handed fractional derivatives as well as a substantial treatment of the left-handed fractional derivatives.

Published papers on the numerical solution of fractional partial differential equation are scarce. A different method for solving the fractional partial differential equation (1.1) is pursued in the recent paper . They transform this partial differential equation into a system of ordinary differential equations (method of lines), which is then solved using backward differentiation formulas. Another very recent paper,  develops a finite element method for two-point boundary value problem, and , finds the numerical solution of (1.1) by finite differences method.

2. Multilayer Neural Networks

The Rumelhart-Hinton-William’s multilayer network , that we consider here, is a feed-forward type network with connections between adjoining layers only. Networks generally have hidden layers between the input and output layers. Each layer consists of computational units. The input-output relationship of each unit is represented by inputs xi output y, connection weights wi, threshold θ, and differential function ϕ as follows:y(x1,,xn)=j=1mvjϕ(Wj-X+θj),where  xiR,  WjRn,  vj,θjR the learning rule of this network is known as the backpropagation algorithm , which is an algorithm that uses a gradient descent method to modify weight and thresholds as the error between the desired output and the output signal of the network is minimized. I generally use a bounded and monotonic increasing differentiable function which is called a radial basis function for each unit output function.

If a multilayer network has n input units and m output units, then the input-output relationship defines a continuous mapping from n-dimensional Euclidean space to m-dimensional Euclidean space. I call this mapping the input-output mapping of the network. I study the problem of network capability from the point of view of input-output mapping. It observed that for the study of mappings defined by multilayer networks, it is sufficient to consider networks whose ϕ(x) and whose output functions for input and output layers are linear.

3. Approximate Realization of Continuous Mappings by Neural Networks

Reference  considers the possibility of representing continuous mappings by neural networks whose output functions in hidden layers are sigmoid function, for example, ϕ(x)=1/(1+e-x). It is simply noted here that general continuous mappings cannot be exactly represented by Rumelhart-Hinton-William’s networks. For example, if a real analytic output function such as the sigmoid function ϕ(x)=1/(1+e-x) is used, then an input-output mapping of this network is analytic and generally cannot represent all continuous mappings.

Let points of n-dimensional Euclidean space Rn be denoted by x=(x1,,xn) and the norm of x defined by x=(i=0nxi2).

Definition 3.1 (see [<xref ref-type="bibr" rid="B1">11</xref>]).

Let H=Rn is a linear space. A function f:HR is called a radial basis function that can be represented in the form f=goϕ, where ϕ:RR, g:RnR, and g(x)=x-wi2, x,wiRn.

In this paper, I study approximate U by using y which is a representation model of neural networks, through analyzing previous theoretical studies. Also we present a study of convergent of solution U by approximate solution y, where ϕ(wj,x,θ) is radial basis function.

We also give some theorems for soliciting the conditions to converge the approximation solution for (1.1) by neural networks method.

Theorem 3.2 (Irie-Miyake, [<xref ref-type="bibr" rid="B9">6</xref>]).

Let ψ(x)L1(R), that is, let ψ(x) be absolutely integrable and u(x1,,xn)L2(Rn). Let Ψ(ξ) be integrable and U(x1,,xn) be Fourier transforms of ψ(x) and u(x1,,xn), respectively. If  Ψ(1)0, then u(x1,,xn)=--ψ(i=1nxiwi-w0)1(2π)nΨ(1)U(w1,,wn)×exp(iw0)dw0dw1dwn.

Theorem 3.3.

Let ϕ(x) be a radial basis function, nonconstant, bounded, and monotone increasing continuous function. Let K be a compact subset of Rn and u(x1,,xn) a real value continuous function on K. Then for an arbitrary ε>0, there exists an integral N and real constants vj,θj,wij  (i=1,2,,n,  j=1,2,,m) such that y(x1,,xn)=j=1mvjϕ[(i=1n(wij-xi)2]1/2+  θj) satisfies maxxK|y(x1,,xn)-  u(x1,,xn)|  <  ε. In other words, for an arbitrary ε>0, there exists a three-layer network whose output function for the hidden layer is ϕ(x), whose output functions for input and output layers are linear, and which has an input-output function y(x1,,xn) such that maxxK|y(x1,,xn)-u(x1,,xn)|<ε.

Proof.

First

Since u(x)(x=(x1,,xn)) is a continuous function on a compact subset K of Rn, u(x) can be extended to be a continuous function on Rn with compact support.

If I operate the mollifier ρα* on u(x), ρα*u(x)  is  C-function with compact support. Furthermore, ρα*u(x)u(x)(α+0) uniformly on Rn. Therefore, we may suppose u(x) is a C-function with compact support for proving Theorem 3.3. By the Paley-Wiener theorem , the Fourier transform U(w)(w=w1,,wn) of u(x) is real analytic and, for any integer N, there exists a constant CN such that |U(w)|CN(1+|w|)-N. In particular U(w)L1L2(Rn).

I define IA(x1,,xn), I,A(x1,,xn), and JA(x1,,xn) as follows: IA(x1,,xn)=-AA-AAψ([i=1n(xi-wi)2]1/2-w0)1(2π)nΨ(1)U(w1,,wn)×exp(iw0)dw0dw1dwn,I,A(x1,,xn)=-AA-AA[-ψ([i=1n(xi-wi)2]1/2-w0)1(2π)nΨ(1)U(w1,,wn)×exp(iw0)dw0([i=1n(xi-wi)2]1/2-w0)]dw1dwn,JA(x1,,xn)=1(2π)n-AA-AAU(w1,,wn)exp[ii=1n(xi-wi)2]1/2dw1dwn, where ψ(x)L1 is defined by ψ(x)=ϕ(x/δ+α)-ϕ(x/δ-α), δ,α>0.

The essential part of the proof of Irie-Miyake’s integral formula  is the equality I,A(x1,,xn)=JA(x1,,xn), and this is derived from -ααψ([i=1n(xi-wi)2]1/2-w0)exp(iw0)dw0=exp([ii=1n(xi-wi)2]1/2)Ψ(1), in our discussion, using the estimate of U(w), it is easy to prove that limAJA(x1,,xn)=u(x1,,xn) uniformly on Rn. Therefore, limAI,A(x1,,xn)=u(x1,,xn) uniformly on Rn. I can state that for any ε>0, there exists A>0 such that maxxRn|I,A(x1,,xn)-u(x1,,xn)|<ε2.

Second

I will approximate I,A by finite integrals on K. For ε>0, fix A which satisfies (3.5). For A>0, set IA,A(x1,,xn)=-AA-AA[-AAψ([i=1n(xi-wi)2]1/2-w0)×1(2π)nΨ(1)U(w1,,wn)×exp(iw0)dw0([i=1n(xi-wi)2]1/2-w0)]dw1dwn. I will show that, for ε>0, I can take A>0 so that maxxK|IA,A(x1,,xn)-I,A(x1,,xn)|<ε2. Using the following equation: -AAψ([i=1n(xi-wi)2]1/2-w0)exp(iw0)dw0=i=1nxiwi-Ai=1nxiwi+Aψ(t)exp(-it)dtexp[ii=1n(xi-wi)2]1/2, the fact U(x)L1 and compactness of [-A,A]n×K, we can take A so that |-AAψ([i=1n(xi-wi)2]1/2-w0)exp(iw0)dw0--ψ([i=1n(xi-wi)2]1/2-w0)exp(iw0)dw0|ε(2π)2|Ψ(1)|(2--|U(x)|dx+1)×-AA-AA|U(x)|dxon  K. Therefore, maxxK|IA,A(x1,,xn)-I,A(x1,,xn)|ε(2--|U(x)|dx+1)×-AA-AA|U(x)|dx<ε2.

Third

From (3.5) and (3.7), I can say that for any ε>0, there exists A,A>0 such that maxxK|u(x1,,xn)-IA,A(x1,,xn)|<ε,u(x) can be approximated by the finite integral IA,A(X) uniformly on K. The integral of IA,A(X) can be replaced by the real part and is continuous on [-A,A]××[-A,A]×K, hence IA,A(X) can be approximated by the Riemann sum uniformly on K. Since ψ([i=1n(xi-wi)2]1/2-w0)=ϕ([i=1n(xi-wi)2]1/2δ-w0+α)-ϕ([i=1n(xi-wi)2]1/2δ-w0-α), the Riemann sum can be represented by a three-layer network. Therefore u(x) can be represented approximately by three-layer network.

Theorem 3.4.

Let H be a linear space. The set of radial basis functions denoted by Φ={expoϕ:ϕH*} is fundamental in C(H).

Proof.

Let T denotes the linear span of the set, Φ as I will prove T an algebra in C(H), for xH and ϕ,ψH* (where H* is the dual space of H)  [(expoϕ)(expoψ)](x)=(expoϕ)(x)(expoψ)(x)=exp(ϕ(x))exp(ψ(x))=exp(ϕ(x)+ψ(x))=exp(ϕ+ψ)(x). Let ϕ0 be the zero functional in H*, then (expoϕ0)=1 for all xH. Hence, T contains constants.

Now if x,yH and xy, then there exists ϕH* such that ϕ(x-y)0. exp[ϕ(x-y)]exp(0)=1,exp[ϕ(x)-ϕ(y)]1,exp[ϕ(x)]exp(-ϕ(y))1,exp[ϕ(x)]exp(ϕ(y))1,expϕ(x)exp(ϕ(y)), hence T separates point of H.

Now, I will prove each basic neighborhood of a fixed point u in C(H) intersects T; K is compact set such that KH, a basic neighborhood of u corresponding to K, and has the form BK,ε={gC(H):u-gK<ε, now restrict u and all members of T to K, then {h/K:hT} is still on algebra-containing constants separating of K, hence {h/K:hT} is dense in C(K), and consequently, T intersects BK,ε, as required. Therefore, T is fundamental in C(H) and required.

4. Numerical Methods

Conceding a feed forward network with input layer of a single hidden layer, and an output layer consisting of a single unit, I have purposely chosen a single output unit to simplify the exposition without loss of generality.

The network is designed to perform a nonlinear mapping form the input space to the hidden space, followed by a linear mapping from the hidden space to the output space.

Given a set of N different points {xiRn,  i=1,2,,n} and a corresponding set N real numbers {uiRn,  i=1,2,,n}, we find a function f: RnR that satisfied the interpolation conditionsf(xi)=yi,i=1,2,,n.

For strict interpolation as specified here, the interpolating surface (i.e.; function f(x)) is constraining to pass through all the training data points, initial w(0)Rn+m, θ(0),d(0)R, v(0)Rn, we will use algorithm of backpropagation neural network  to find the best weights W,V,d, and θ, inserting the interpolation conditions of (1.1) in (2.1), we obtain the following set of simultaneous linear equations for the unknown coefficients of the expansion[ϕ11ϕ12ϕ1nϕ21ϕ22ϕ2nϕn1ϕn2ϕnn][ν1ν2νn]=[y1y2yn], where ϕij=ϕ(Wij-Xi), (i,j)=1,2,,n.

Compute the error E=(1/2)i=1N(yi-ui)2, and δi=(yi-ui)*ϕ(yi).

Calculate Δνi=αδi*Zi  =αδi*Xi-Wij  (α  is training rat   0  <  α  <  1),Δdj=α*δj,ρi=j=1NδjVj,βi=ρiα(Xi-Wij),ΔWij=αβixi,Δθi=αβi,Vinew=Vjold+ΔVj,Wijnew=Wijold+ΔWij,θinew=θjold+Δθi,dinew=djold+Δdi.

After that the best weights W, V, θ, d are fined. Given the substitute in (2.1), the approximate solution is found.

The above method is supposed to work in this situation. In such case, the values of absence of the exact solution of independent variables will be substituted in the boundary conditions in order to get the exact values. Those values can be used in the phase of training of the considered backpropagation neural network. The approach is proved to get good weights of (2.1). The radial basis function in Theorems 3.3 and 3.4 was used in order to obtain converge training of the backpropagation neural network.

5. Numerical Example

The following two-sided fractional partial differential equationu(x,t)u=c+(x,t)1.8u(x,t)+x1.8+c-(x,t)1.8u(x,t)-x1.8+s(x,t) were considered on a finite domain 0<x<2 and t>0 with the coefficient functions c+(x,t)=Γ(1.2)x1.8 and c-(x,t)=Γ(2-x)x1.8, and the forcing function s(x,t)=-32e-t[x2+(2-x)2-2.5(x3+(2-x)3)+2522(x4+(2-x)4)], initial condition u(x,0)=4x2(2-x)2, and boundary conditions u(0,t)=u(2,t)=0. This fractional PDE has the exact solution u(x,t)  =4e-tx2(2-x)2, which can be verified applying the fractional formulas DLα+(x-L)p=Γ(p+1)Γ(p+1-α)(x-L)p-α,DRα-(R-x)p=Γ(p+1)Γ(p+1-α)(R-x)p-α (see ).

Table 1 shows the training data by values of the boundary conditions. Table 2 shows the comparisons between exact solution u(xi,ti) and the approximated solution test points y(xi,yi) at Δx=0.2, Δt=0.1. Table 3 compares maximum error between approximate solution by artificial neural networks method and finite difference numerical method .

The training data by values of the boundary conditions.

x  t  U(x,t)Y(x,t)×e-5
0.100.14440.1678
0.201.04041.5493
0.500.25000.9654
1.303.31246.7412
1.502.254.7759
1.701.04045.7418
20.100
20.400
20.600

The comparisons between exact solution u(xi,ti) and the approximated solution test points y(xi,yi) at Δxi=0.2, Δti=0.1.

 x t U(x,t) Y(x,t) 0.2 0.1 0.4691 0.4246 0.4 0.2 1.3414 1.3752 0.6 0.3 0.2490 0.2831 0.8 0.4 2.0906 2.0597 1.0 0.5 2.4711 2.4244 1.2 0.6 2.4261 2.4997 1.4 0.7 2.0231 2.0488 1.6 0.8 0.7362 0.7651 1.8 0.9 0.2107 0.2239 Maximum error 0.0736

Compares maximum error between approximate solutions.

 Δxi Δti FDM NNS Maximum error Maximum error 0.2 0.1 0.1417 0.0736 0.1 0.05 0.0571 0.0541 0.05 0.025 0.0249 0.0217 0.025 0.0125 0.0113 0.0082
6. Conclusion

Referring back to Tables 13, it is clear that the approximation of fractional partial FPDE using neural network with RBF obtains a good approximated solution. The converge of this method is given as well. In addition, the discussed method is able to solve fractional partial differential equations with more than two variables where many other methods fail, clearly. The suggested method provides a general approximated solution to the interval domain and depends on boundary conditions.

MeerschaertM. M.TadjeranC.Finite difference approximations for two-sided space-fractional partial differential equationsNSFgrant DMC2004563573ChavesA. S.A fractional diffusion equation to describe Lévy flightsPhysics Letters A19982391-21316161610310.1016/S0375-9601(97)00947-XZBL1026.82524BensonD. A.debenson@dri.eduWheatcraftS. W.MeerschaertM. M.The fractional-order governing equation of Levy motionWater Resources Research20003661413142310.1029/2000WR900032LiuF.AonV.TurnerI.Numerical solution of the fractional advection-dispersion equation2002, http://academic.research.microsoft.com/Publication/3471879MillerK. S.RossB.An introduction to the fractional calculus and fractional differential equations1993New YorkJohn Wiley & Sonsxvi+366A Wiley-Interscience Publication1219954GorenfloR.MainardiF.ScalasE.RabertoM.Fractional calculus and continuous-time finance. III. The diffusion limitMathematical Finance2001Basel, SwitzerlandBirkhäuser171180Trends Math.1882830SamkoS. G.KilbasA. A.MarichevO. I.Fractional Integrals and Derivatives: Theory and Applications1993Yverdon, SwizerlandGordon and Breach Science Publishersxxxvi+9761347689FixG. J.RoopJ. P.Least squares finite-element solution of a fractional order two-point boundary value problemComputers & Mathematics with Applications2004487-810171033210738010.1016/j.camwa.2004.10.003ZBL1069.65094HaykinS.Neural Networks2006Prntice-HallFunahashiK.-I.On the approximate realization of continuous mappings by neural networksNeural Networks198923183192Al-MarashiA.Al-WagihK.Approximation solution of boundary values of partial differential equations by using neural networksThamar University Journal20076121136