This paper investigates the solution of Ordinary Differential Equations (ODEs) with initial conditions using Regression Based Algorithm (RBA) and compares the results with arbitrary and regressionbased initial weights for different numbers of nodes in hidden layer. Here, we have used feed forward neural network and error back propagation method for minimizing the error function and for the modification of the parameters (weights and biases). Initial weights are taken as combination of random as well as by the proposed regression based model. We present the method for solving a variety of problems and the results are compared. Here, the number of nodes in hidden layer has been fixed according to the degree of polynomial in the regression fitting. For this, the input and output data are fitted first with various degree polynomials using regression analysis and the coefficients involved are taken as initial weights to start with the neural training. Fixing of the hidden nodes depends upon the degree of the polynomial. For the example problems, the analytical results have been compared with neural results with arbitrary and regression based weights with four, five, and six nodes in hidden layer and are found to be in good agreement.
Differential equations play vital role in various fields of engineering and science. The exact solution of differential equations may not be always possible [
Lee and Kang [
As per the review of the literatures, it reveals that authors have taken the parameters (weights/biases) as arbitrary (random) and the numbers of nodes in hidden layer are considered by trial and error method. In this paper, we propose a method for solving ordinary differential equations using feed forward neural network as a basic approximation element and error back propagation algorithm [
Rest of the paper is organized as follows. In Section
Let us consider the following general differential equations which represent both ordinary and partial differential equations [
Now,
The error computation not only involves the outputs but also the derivatives of the network output with respect to its inputs. So, it requires finding out the gradient of the network derivatives with respect to its inputs. Let us now consider a multilayered perceptron with one input node, a hidden layer with
The derivatives of
Let
Let us consider first order ordinary differential equation as below
In this case, the ANN trail solution may be written as
The error function for this case may be formulated as
In this case, the second order ordinary differential equation may be written in general as
The ANN trail solution may be discussed as
The error function to be minimized for second order ordinary differential equation will be
Three layer architecture of ANN for the present problem is considered. Usually numbers of nodes in the hidden layer are taken by trial and error method. Here, we fix the number of nodes in hidden layer by using regressionbased weight generation [
Threelayered neural network architecture with single input and single output node.
In this section, we present solution of two example problems as mentioned earlier. In all cases, we have used error back propagation algorithm and one hidden layer. The weights are taken as arbitrary and regression based for comparison of the training method. Sigmoid function
Let us consider the first order ordinary differential equation as follows:
The trial solution is written as
Analytical and neural solutions with arbitrary and regression based weights (Example
Input data  Analytical  Neural results  






Deviation% 

Deviation%  
0  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  0.00  1.0000  0.00 
0.05  0.9536  1.0015  0.9998  1.0002  0.9768  0.9886  3.67  0.9677  1.47 
0.10  0.9137  0.9867  0.9593  0.9498  0.9203  0.9084  0.58  0.9159  0.24 
0.15  0.8798  0.9248  0.8986  0.8906  0.8802  0.8906  1.22  0.8815  0.19 
0.20  0.8514  0.9088  0.8869  0.8564  0.8666  0.8587  0.85  0.8531  0.19 
0.25  0.8283  0.8749  0.8630  0.8509  0.8494  0.8309  0.31  0.8264  0.22 
0.30  0.8104  0.8516  0.8481  0.8213  0.9289  0.8013  1.12  0.8114  0.12 
0.35  0.7978  0.8264  0.8030  0.8186  0.8051  0.7999  0.26  0.7953  0.31 
0.40  0.7905  0.8137  0.7910  0.8108  0.8083  0.7918  0.16  0.7894  0.13 
0.45  0.7889  0.7951  0.7908  0.8028  0.7948  0.7828  0.77  0.7845  0.55 
0.50  0.7931  0.8074  0.8063  0.8007  0.7960  0.8047  1.46  0.7957  0.32 
0.55  0.8033  0.8177  0.8137  0.8276  0.8102  0.8076  0.53  0.8041  0.09 
0.60  0.8200  0. 8211  0.8190  0.8362  0.8246  0.8152  0.58  0.8204  0.04 
0.65  0.8431  0.8617  0.8578  0.8519  0.8501  0.8319  1.32  0.8399  0.37 
0.70  0.8731  0.8896  0.8755  0.8685  0.8794  0.8592  1.59  0.8711  0.22 
0.75  0.9101  0.9281  0.9231  0.9229  0.9139  0.9129  0.31  0.9151  0.54 
0.80  0.9541  0.9777  0.9613  0.9897  0.9603  0.9755  2.24  0.9555  0.14 
0.85  1.0053  1.0819  0.9930  0.9956  1. 0058  1.0056  0.03  0.9948  1.04 
0.90  1.0637  1.0849  1.1020  1.0714  1.0663  1.0714  0.72  1.0662  0.23 
0.95  1.1293  1.2011  1.1300  1.1588  1.1307  1.1281  0.11  1.1306  0.11 
1.00  1.2022  1.2690  1.2195  1.2806  1.2139  1.2108  0.71  1.2058  0.29 
Analytical and neural results with arbitrary and regression based weights for six nodes in hidden layer are compared in Figures
Plot of comparison between (analytical results) and (neural results) with arbitrary weights (Example
Plot of comparison between (analytical results) and (neural results) with regressionbased weights for six nodes (Example
Error plot between analytical and regressionbased weights approximation solution (Example
It may be seen that by increasing the number of nodes in hidden layer from four to six, the results are found to be better. Although the authors increased the number of nodes in hidden layer beyond six, but the results were not improving.
The first problem has also been solved by a wellknown numerical method, namely, using Euler and Rungekutta method. Table
Comparison of the results (Example
Input data  Analytical  Euler  RungeKutta 


0  1.0000  1.0000  1.0000  1.0000 
0.0500  0.9536  0.9500  0.9536  0.9677 
0.1000  0.9137  0.9072  0.9138  0.9159 
0.1500  0.8798  0.8707  0.8799  0.8815 
0.2000  0.8514  0.8401  0.8515  0.8531 
0.2500  0.8283  0.8150  0.8283  0.8264 
0.3000  0.8104  0.7953  0.8105  0.8114 
0.3500  0.7978  0.7810  0.7979  0.7953 
0.4000  0.7905  0.7721  0.7907  0.7894 
0.4500  0.7889  0.7689  0.7890  0.7845 
0.5000  0.7931  0.7717  0.7932  0.7957 
0.5500  0.8033  0.7805  0.8035  0.8041 
0.6000  0.8200  0.7958  0.8201  0.8204 
0.6500  0.8431  0.8178  0.8433  0.8399 
0.7000  0.8731  0.8467  0.8733  0.8711 
0.7500  0.9101  0.8826  0.9102  0.9151 
0.8000  0.9541  0.9258  0.9542  0.9555 
0.8500  1.0053  0.9763  1.0054  0.9948 
0.9000  1.0637  1.0342  1.0638  1.0662 
0.9500  1.1293  1.0995  1.1294  1.1306 
1.000  1.2022  1.1721  1.2022  1.2058 
Let us consider the following second order damped free vibration equation:
As discussed above, we can write the trail solution as
Analytical and neural results which are obtained for random initial weights are depicted in Figure
Analytical and neural solutions with arbitrary and regressionbased weights (Example
Neural results  

Input data  Analytical 






(four nodes)  (four nodes)  (five nodes)  (five nodes)  (six nodes)  (six nodes)  
0  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 
0.1  1.0643  1.0900  1.0802  1.0910  1.0878  1.0923  1.0687 
0.2  1.0725  1.1000  1.0918  1.0858  1.0715  1.0922  1.0812 
0.3  1.0427  1.0993  1.0691  1.0997  1.0518  1.0542  1.0420 
0.4  0.9885  0.9953  0.9732  0.9780  0.9741  0.8879  0.9851 
0.5  0.9197  0.9208  0.9072  0.9650  0.9114  0.9790  0.9122 
0.6  0.8433  0.8506  0.8207  0.8591  0.8497  0.8340  0.8082 
0.7  0.7645  0.7840  0.7790  0.7819  0.7782  0.7723  0.7626 
0.8  0.6864  0.7286  0.6991  0.7262  0.6545  0.6940  0.6844 
0.9  0.6116  0.6552  0.5987  0.6412  0.6215  0.6527  0.6119 
1.0  0.5413  0.5599  0.5467  0.5604  0.5341  0.5547  0.5445 
1.1  0.4765  0.4724  0.4847  0.4900  0.4755  0.4555  0.4634 
1.2  0.4173  0.4081  0.4035  0.4298  0.4202  0.4282  0.4172 
1.3  0.3639  0.3849  0.3467  0.3907  0.3761  0.3619  0.3622 
1.4  0.3162  0.3501  0.3315  0.3318  0.3274  0.3252  0.3100 
1.5  0.2738  0.2980  0.2413  0.2942  0.2663  0.2773  0.2759 
1.6  0.2364  0.2636  0.2507  0.2620  0.2439  0.2375  0.2320 
1.7  0.2036  0.2183  0.2140  0.2161  0.2107  0.2177  0.1921 
1.8  0.1749  0.2018  0.2007  0.1993  0.1916  0.1622  0.1705 
1.9  0.1499  0.1740  0.1695  0.1665  0.1625  0.1512  0.1501 
2.0  0.1282  0.1209  0.1204  0.1371  0.1299  0.1368  0.1245 
2.1  0.1095  0.1236  0.1203  0.1368  0.1162  0.1029  0.1094 
2.2  0.0933  0.0961  0.0942  0.0972  0.0949  0.0855  0.09207 
2.3  0.0794  0.0818  0.0696  0.0860  0.0763  0.0721  0.0761 
2.4  0.0675  0.0742  0.0715  0.0849  0.0706  0.0526  0.0640 
2.5  0.0573  0.0584  0.0419  0.0609  0.0543  0.0582  0.0492 
2.6  0.0485  0.0702  0.0335  0.0533  0.0458  0.0569  0.0477 
2.7  0.0411  0.0674  0.0602  0.0581  0.0468  0.0462  0.0409 
2.8  0.0348  0.0367  0.0337  0.0387  0.0328  0.0357  0.03460 
2.9  0.0294  0.0380  0.0360  0.0346  0.0318  0.0316  0.0270 
3.0  0.0248  0.0261  0.0207  0.0252  0.0250  0.0302  0.0247 
3.1  0.0209  0.0429  0.0333  0.0324  0.0249  0.0241  0.0214 
3.2  0.0176  0.0162  0.0179  0.0154  0.0169  0.0166  0.0174 
3.3  0.0148  0.0159  0.0137  0.0158  0.0140  0.0153  0.0148 
3.4  0.0125  0.0138  0.0135  0.0133  0.0130  0.0133  0.0129 
3.5  0.0105  0.0179  0.0167  0.0121  0.0132  0.0100  0.0101 
3.6  0.0088  0.0097  0.0096  0.0085  0.0923  0.0095  0.0090 
3.7  0.0074  0.0094  0.0092  0.0091  0.0093  0.0064  0.0071 
3.8  0.0062  0.0081  0.0078  0.0083  0.0070  0.0061  0.0060 
3.9  0.0052  0.0063  0.0060  0.0068  0.0058  0.0058  0.0055 
4.0  0.0044  0.0054  0.0052  0.0049  0.0049  0.0075  0.0046 
Plot of comparison between (analytical results) and (neural results) with arbitrary weights (for six nodes) (Example
Plot of comparison between (analytical solutions) and (neural solutions) with regression based weights (for six nodes) (Example
Error plot between analytical, and regressionbased weights solution (Example
Now we consider an initial value problem as follows:
The ANN trial solution is written as
Analytical and traditional neural results obtained using random initial weights with six nodes are depicted in Figure
Analytical and neural solutions with arbitrary and regressionbased weights (Example
Input data  Analytical  Euler  RungeKutta  Neural results  








0  0  0  0  0  0  0  0  0  0 
0.1  0.0671  0.1000  0.0671  0.0440  0.0539  0.0701  0.0602  0.0565  0.0670 
0.2  0.0905  0.1241  0.0904  0.0867  0.0938  0.0877  0.0927  0.0921  0.0907 
0.3  0.0917  0.1169  0.0917  0.0849  0.0926  0.0889  0.0932  0.0931  0.0918 
0.4  0.0829  0.0991  0.0829  0.0830  0.0876  0.0806  0.0811  0.0846  0.0824 
0.5  0.0705  0.0797  0.0705  0.0760  0.0748  0.0728  0.0714  0.0717  0.0706 
0.6  0.0578  0.0622  0.0577  0.0492  0.0599  0.0529  0.0593  0.0536  0.0597 
0.7  0.0461  0.0476  0.0461  0.0433  0.0479  0.0410  0.0453  0.0450  0.0468 
0.8  0.0362  0.0360  0.0362  0.0337  0.0319  0.0372  0.0370  0.0343  0.0355 
0.9  0.0280  0.0271  0.0280  0.0324  0.0308  0.0309  0.0264  0.0249  0.0284 
1.0  0.0215  0.0203  0.0215  0.0304  0.0282  0.0255  0.0247  0.0232  0.0217 
Plot of comparison between (analytical results) and (neural results) with arbitrary weights (for six nodes) (Example
Plot of comparison between (analytical solutions) and (neural solutions) with regression based weights (for six nodes) (Example
Error plot between analytical, and regressionbased weights solution (Example
Here, we consider a standard differential equation which represents exponential growth as follows:
Here
Analytic result may be found as
The ANN trial solution in this case is
Analytical and neural solutions with arbitrary and regressionbased weights (Example
Neural results  

Input data  Analytical 






(four nodes)  (four nodes)  (five nodes)  (five nodes)  (six nodes)  (six nodes)  
0  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 
0.1000  1.1052  1.1069  1.1061  1.1093  1.1060  1.1075  1.1051 
0.2000  1.2214  1.2337  1.2300  1.2250  1.2235  1.2219  1.2217 
0.3000  1.3499  1.3543  1.3512  1.3600  1.3502  1.3527  1.3498 
0.4000  1.4918  1.4866  1.4921  1.4930  1.4928  1.4906  1.4915 
0.5000  1.6487  1.6227  1.6310  1.6412  1.6456  1.6438  1.6493 
0.6000  1.8221  1.8303  1.8257  1.8205  1.8245  1.8234  1.8220 
0.7000  2.0138  2.0183  2.0155  2.0171  2.0153  2.0154  2.0140 
0.8000  2.2255  2.2320  2.2302  2.2218  2.2288  2.2240  2.2266 
0.9000  2.4596  2.4641  2.4625  2.4664  2.4621  2.4568  2.4597 
1.0000  2.7183  2.7373  2.7293  2.7232  2.7177  2.7111  2.7186 
Plot of comparison between (analytical results) and (neural results) with arbitrary weights (for six nodes) (Example
Plot of comparison between (analytical solutions) and (neural solutions) with regression based weights (for six nodes) (Example
Error plot between analytical and regression based weights solution (Example
In traditional artificial neural network, the parameters (weights/biases) are usually taken as arbitrary (random) and the number of nodes in hidden layer is considered by trial and error method. Also, few authors have used optimization technique to minimize the error. In this investigation, a regressionbased artificial neural network with combinations of initial weights (arbitrary and regression based) in the connections is considered. We have fixed the number of nodes in hidden layer according to the degree of polynomial of regression fitting. The initial weights from input to hidden and hidden to output layer are taken by using regressionbased weight generation. Back propagation algorithm has been employed for modification of the parameters without use of any optimization technique. Also, time of computation is less than traditional artificial neural architecture. Table
Time of computation.
Problems  Time of computation in hours  

Traditional ANN  Proposed ANN 
Proposed ANN 
Proposed ANN  
Four nodes  Five nodes  Six nodes  
Example 
1.57 hrs  1.51  1.49  1.31  1.27  1.09 
Example 
3.06  3.00  2.44  2.23  1.55  1.38 
It is well known that the other numerical methods are usually iterative in nature, where we fix the step size before the start of the computation. After the solution is obtained, if we want to know the solution in between steps, then again the procedure is to be repeated from initial stage. ANN may be one of the reliefs where we may overcome this repetition of iterations. The authors are not claiming that the method presented is most accurate. As it may be seen by the comparison in Tables
Here, we have considered three, four, and five degree polynomial for regression fitting. One may consider higher degree polynomial in the simulation but it has been seen that by increasing the degree of the polynomials, the accuracy does not usually increase. In the future, it needs to develop a methodology about what degree polynomial one should use to get a result with acceptable accuracy. This is however not of the scope of this paper and the authors are working in this direction and hope to communicate the findings in the future.
This paper presents a new approach to solve ordinary differential equations by using regression based artificial neural network model. Accuracy of the proposed method has been examined by solving a first order and a second order damped free vibration problem. The main value of the paper is that the numbers of nodes in hidden layer are fixed according to the degree of polynomial in the regression. Accordingly, here, comparisons of different neural architectures corresponding to different regression models are investigated. Moreover, the algorithm is unsupervised and error back propagation algorithm is used to minimize the error function. Corresponding initial weights from input to hidden and hidden to output are all obtained by the proposed procedure. The trail solution is closed and differentiable. One may see from the tables and graphs that the initial weights generated by regression model make the results more accurate. Lastly, it may be mentioned that the implemented Regression Based Neural Network (RBNN) algorithm is simple, computationally efficient, and straight forward.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The first author is thankful to the Department of Science and Technology (DST), Government of India for the financial support under Women Scientist SchemeA.