Solving Fractional Differential Equations by Using Triangle Neural Network

,


Introduction
Fractional differential equations (FDEs) have been a hot topic in many scientific fields, such as dynamical system control theory, fluid flow, modelling in rheology, dynamic process of self-similar porous structure, diffusion transport similar to diffusion, electric network, and probability statistics [1][2][3][4][5][6][7][8][9]. These problems in science and engineering sometimes require us to get the solutions of various fractional differential equations. But as we know, it is difficult to find the exact solutions in most cases. So, we have to use numerical methods to solve fractional differential equations.
In recent years, some scholars try to use the neural network to solve differential equations [18][19][20]. Lagaris et al. [21] proposed an artificial neural network method for solving initial and boundary value problems. In their work, a trial solution is adopted and written as the sum of two parts. The first part satisfies the initial or boundary conditions and does not contain adjustable parameters while the construction of the second part does not affect the initial and boundary conditions. Then, the neural network is trained to satisfy the differential equation at many selected points. The question for this method is that it is difficult to construct the first part of the trial solution and this method cannot be applied to fractional partial differential equations.
Piscopo et al. [22] also introduced a method to find the numerical solutions of many types of differential equations. The proposed method does not depend on the trial solution and therefore has more flexibility in many cases. It can be used for solving many types of ODE and PDE. The two mentioned neural network techniques motivate us to develop more neural network methods to solve FDEs, but how to get the fractional order of the neural network is a difficult problem.
To overcome this difficulty, in this work, we use a triangle base neural network as basis function to propose an alternative method called triangle neural network methods. This paper is organized as follows. In Section 2, we study the fractional derivative of the triangle base neural network and present the numerical method for solving many types of FDEs. In Section 3, we show the efficiency of the proposed method by some numerical examples. Section 4 is the conclusion.

Fractional Derivative of Triangle Neural
Network and Numerical Algorithm

Ordinary Fractional Differential Equation.
To solve the following fractional initial value problem (1) and boundary value problem (2), where aðxÞ and bðxÞ are real functions, 1 < α ≤ 2, 0 < β ≤ 1. D ðαÞ and D ðβÞ are Caputo fractional derivative operators. We consider the following triangle base neural network 1 (see Figure 1) to approximate the solution of problems (1) and (2), where w j are weights for the neural networks and C j ðxÞ are triangle base functions as the following: where C j ðxÞ are activation function of neurons in the hidden layer of the above neural network and N is an integer and x ∈ ½0, π. Let the weight matrix be w = ½w 0 , w 1 , ⋯, w 2N T and the activation matrix be CðxÞ = ½c 0 ðxÞ, c 1 ðxÞ, ⋯, c 2N ðxÞ T . The triangle base neural network can be written as When this neural network is used to be the numerical solution of problem (1), the loss function is For problem (2), The loss function is where x t , t = 1, 2, ⋯, m are training points. We have two methods to minimize the loss function to get the corresponding numerical solution. One is the gradient descent algorithm, and another one is the optimization process. For both methods, we need to compute the α derivative of the triangle neural network. For this purpose, we have the following theorems.
Theorem 1. For given α ∈ R + , f ðxÞ ∈ C 1 ðRÞ, then Proof. Since We have Let λτ = u, we have We also have the following. 2 Journal of Function Spaces Proof.
From Theorem 1, we have The second part of this theorem can be verified in the same way. Based on Theorems 1 and 2, we can get the α derivative of the triangle base neural network.
In fact, let the solution to problems (1) and (2) be We can get Thus, we get the loss function for problem (1): To carry out the gradient descent process, we have for j = 0, 1, 2, ⋯, N, and we also have for j = N + 1, N + 2, ⋯, 2N. So, we can see that getting the numerical solution of (1) is equivalent to finding w j s by minimizing the loss function J. Usually, we have two methods to do this work. One is the gradient descent method, and another one is adopting the optimization process.
The gradient descent method is as below: where η is the step size for the gradient descent. If the function f ðx, yÞ is a linear function of y, the initial value problem can also be reduced to That is, which is a quadratic optimization problem. For fractional 3 Journal of Function Spaces boundary value problem (2), the numerical solution can be reduced to the following optimization process: So, there are two methods to solve this problem. One method is to get the solution through the gradient descent method. Another method is using the optimization technique.

Fractional Partial Differential Equation.
For fractional partial differential equation problem where 0 < α ≤ 1, and problem where 1 < β ≤ 2. We use the triangle base neural network (see Figure 2) to approximate the solution of problems (22) and (23). The triangle base neural network can be written as where a i1 , a i2 are weights for the import layer in the neural network, b i are bias parameters for the hidden layer in the neural network, w i are weights for the export layer in the neural network, and sin ðxÞ is the activation function of neurons in the hidden layer. Based on Theorems 1 and 2, we can get the fractional derivative of the neural network as below: The loss function for problem (22) is where ðt k , x k Þ are training points and B is the boundary of problem (22) and The loss function for problem (23) can be given in the same way.
We use the gradient descent algorithm to train the neural network. In fact, we can train the neural network by layers. First, we train the export layer to get w i s, then the bias parameters to get b i s in the hidden layer, and finally, we train the import layer to get a i1 s and a i2 s.

Numerical Experiment
3.1. Numerical Test 1. Consider the following example 1: where 0 < α ≤ 1. The exact solution to this problem is y = x 2 + 1. We let α be 0:5, 0:9 and use the optimization method when N = 5, N = 15 and gradient descent method when N = 5, N = 15, respectively. The computational results are listed in Tables 1 and 2. 3.2. Numerical Test 2. Consider the following example 2 for boundary value problem: y 0 ð Þ = 0, y π 2 = 1,  Journal of Function Spaces where 0 < α ≤ 1. The exact solution to this problem is y = sin x. We let α be 0:5 and N = 30, N = 45, respectively; the computational error is listed in Figure 3.
We let α be 0:8 and N = 20, N = 30. The computational error is listed in Figure 4.
As we see in example 1, the solution becomes more accurate when N is increased. And for the boundary value problem, we use two constraints when we use the optimization process.

Numerical Test 3. Consider the following example 3:
where 0 < α ≤ 1. The exact solution to this problem is u = t 3 sin x.
We use the gradient descent method to solve this problem, and the computational error is listed in Figure 5. In the training process of the neural network, we set a stopping criteria J < 10 −5 for the computing process to stop. If this stopping criteria cannot be achieved, the computing will be stopped when 10 5 times of training is completed.

Numerical Test 4. Consider the following example 4:
where 1 < β ≤ 2. The exact solution to this problem is u = t 3 sin x.
We use the gradient descent method to solve this problem, and the computational error is listed in Figure 6.

Conclusion
The neural network method is a promising approach for solving fractional differential equations. The difficulty for this method is how to calculate the fractional derivatives of the involved neural network. In this paper, we propose numerical methods for solving fractional differential equations including the initial problem, boundary value problem, and partial FDEs by using the triangle base neural network and gradient descending method. All the involved fractional derivatives in this work are considered as Caputo type. We first analyze the fractional derivative of the triangle base neural network. Then, based on the loss function, the proposed numerical methods reduce the fractional differential equation into the gradient descent process or the quadratic optimization problem. By carrying out the gradient descent process or the quadratic optimization process, we can get the numerical solutions. Numerical tests show that this approach is easy to implement and the solution is accurate when applied to many types of FDEs.

Data Availability
All the data supporting this study are available within the article.

Conflicts of Interest
The authors declare that they have no conflicts of interest.

Authors' Contributions
All authors contributed equally and significantly to this paper. All authors read and approved the final manuscript.   Journal of Function Spaces