A Comparative Study on Qualification Criteria of Nonlinear Solvers with Introducing Some New Ones

. In order to compare di ﬀ erent solvers for systems of nonlinear equations, some novel goodness and quali ﬁ cation criteria are de ﬁ ned in this paper. These use all parameters of a nonlinear solver such as convergence order, number of function evaluations, number of iterations, CPU time, etc. To achieve the criteria, di ﬀ erent algorithms to solve nonlinear systems are categorised to three kinds. For any category, two criteria are de ﬁ ned to compare di ﬀ erent algorithms in that category. As numerical results show, these new criteria can use to compare di ﬀ erent algorithms which solve systems of nonlinear equations. Further, we present some corrected formulas for some classical e ﬃ ciency indices and change them to be more applicable. Also, some suggestions are presented about the future works.


Introduction
One of the attractive problems for mathematicians, engineers, and physicians is finding a (the) root of systems of nonlinear equations. A system of nonlinear equations can be written as the following generic form: where F : ℝ n ⟶ ℝ n is a vector valued function. It is known that in general, one cannot solve (1) analytically. Hence we have to solve it numerically or more precise by an iterative method. Even though the classical Newton's method to solve (1) is a basic iterative method and widely used, but many new iterative methods are also introduced every year. It is well-known that the Newton's method is a second-order method if Jacobian of F in (1) is nonsingular in a neighborhood of the solution. To show superiority of new methods versus the Newton's method, different criteria have been used. Some authors use order of convergence, some use number of iterations to achieve convergence, some use CPU time, or a combination of some of these mentioned cri-teria. To remove the weakness of these criteria, for algorithms which solve nonlinear systems like (1), it is important to take into account the number of required operations, because in any iteration one must solve some linear systems. For these cases the classical efficiency index (EI) is defined by Ostrowski [1] as where p is the order of convergence and d = mn 2 + rn wherein n is the size of the system, r is the number of functional evaluations per step, and m shows the number of Jacobian evaluations per step. Further, Traub [2] introduced the operational index as where op shows the number of required products or quotients per iteration. Except these criteria, another ones were defined by some researchers for example Grau A and Grau N [3] defined the computational cost per iteration as Cðα, β, nÞ = αkn + βrn 2 + PðnÞ, where k and r represent the number of evaluations of FðxÞ and its Jacobian matrix, respectively. Besides, α and β are the ratios between multiplications and evaluations required to express the value of Cðα, β, nÞ in terms of multiplications. Also, PðnÞ is the number of products in any iteration. By these notations, they introduced the computational efficiency index (CEI) as where p is the order of convergence. It must be noted that, CEI is an extension of EI, to see this extension, it is enough that one set α = β = 1, and PðnÞ = 0. Also, the flops-like efficiency index (FLEI) is defined by Montazeri et al. [4] as where p is the order of convergence of the method, c = A + Bn 2 + Cn which A shows the cost of LU factorization (based on the flops). Also, n and C, respectively, are the size of the system and the number of functional evaluations per step. Finally, B is the number of Jacobian evaluations in any step. The above mentioned criteria are useful but for two different reasons, they cannot express superiority of an algorithm solely. The different categories of nonlinear solvers are the first reason. Existing different parameters in any algorithm is the second one. Further, computing the values of parameters are not easy and straight forward. Therefore, there are some important gaps for these existing known criteria in the literature, I op , CEI, and FLEI. In this paper, we introduce some new qualification criteria to remove these lacks. For example, for algorithms which their order of convergence is known, we present a criterion which contains order of convergence, CPU time, number of iterations, number of function evaluations, etc. These parameters have two distinguished roles; the positive and the negative ones. For example, order of convergence shows a positive index for any algorithm while CPU time presents a negative one. Hence our criterion can have a fractional form as

Order of Convergence A Combination of Negative Parameters · ð6Þ
Further, we categorise different algorithms and for any kind we introduce two relevant criteria to compare them altogether. Finally, we present some corrected formulas for the classical efficiency indices, I op , CEI, and FLEI and change them to be more applicable.

Test Problems.
To investigate qualification of nonlinear solvers in any category, we applied them to solve four systems of nonlinear equations. These are selected form different areas which need solving of a nonlinear system. These are moderately simple nonlinear systems such that any algorithms can solve them. As a matter of fact, we only want to compare the considered algorithms; therefore these test problems were selected. Besides, all of them at least have one known exact solution.

First Category
The first category of the nonlinear solvers contains algorithms which use Jacobian evaluation, and their order of convergence is known. The parameters of these solvers are order of convergence (OC) which is an important index for the method and in the investigation of quality of the method has a positive role, number of iterations (IT), number of total function evaluations (FE), total number of Jacobian evaluations (JE), and CPU time (CPU) that have negative roles. The most common criteria for these algorithms can be written as follows: Journal of Mathematics (ii) Criterion 2: 2.1. Some Selected Iterative Methods. To show goodness of our criteria, we have chosen some different iterative methods which are presented in the last two decades. We have computed values of both criteria to compare qualification of these iterative methods. Bigger qualification value of any criterion shows the better qualification of the solver. These methods are selected from different categories which use Jacobian of function F. We use them to solve Test Problems 1-4. Method 1: The classical Newton's method (NM) is a quadratic order convergent one which has the following formula: where J F ð·Þ shows the Jacobian matrix of function F. Method 2 (M2): Darvishi and Barati [5] introduced the following third order convergent method in (2007): Method 3 (M3): The following super cubic convergent method is introduced by Darvishi and Barati [6] in (2007). Later, Babajee et al. [7] proved that the method is in fact a fourth order convergent method.

Journal of Mathematics
Method 10 (M10): As the last method, we investigate qualification criteria for the following sixth order convergent iterative method which is introduced by Lotfi et al. [13] in (2015): 2.2. Numerical Results. As a matter of fact, our aim is showing superiority of different methods on the classical Newton's one. Hence any method which has better results for the criteria will be better than the Newton's method. In fact, we would like to test the mentioned methods M1-M10 to pass or fail our examinations from the criteria. Any method which has small values for a criterion in respect with the Newton's method is failed from this evaluation. Also, we can compare the methods altogether. All test problems are solved for two different values of n, namely, n = 100,500. The numerical results are reported for these cases as follows. Case n = 100. Tables 1 and 2, respectively, show the values of CR1 and CR2 to solve all test problems by iterative Methods 1-10. From Tables 1 and 2, M9 has bigger values in comparing with the other methods. Hence for case n = 100, M9 acts better than the other ones.
Case n = 500. Values of our criteria, for n = 500 are reported in Tables 3 and 4 for test problems 1-4, respectively. We can see from Tables 3, and 4 that M3 is better than the other methods for solving all test problems in both criteria CR1 and CR2, clearly. From these tables, we conclude that by increasing size of the system, M3 acts better than the other methods.
2.3. Discussion on Numerical Results for the First Category. As we can see from Tables 1-4 four methods, namely, M3, M7, M9, and M10 have better results among all methods. Hence, we compared those methods altogether for different sizes of our problems. As a matter of fact, we investigated the qualification of these methods by increasing n. The results are reported only for Test Problem 1. Plots of Figure 1 shows the results graphically. As one can see from these plots, even though methods M3, M7, and M9 have better results for small n but by increasing the size of the problem, the third order method M3 has better results. It must be    (i) Criterion 3: The third criterion is defined as (ii) Criterion 4: To give a big weight to the order of convergence, the next criterion receives the following formula: There are similar interpretations for the other criteria.
(iii) Criterion 5: (iv) Criterion 6: (v) Criterion 7: (vi) Criterion 8: To decrease influence of the order of convergence, we use the following:  (vii) Criterion 9: 3. Second Category: Jacobian Free Methods In this category, we consider Jacobian free methods which apply to solve systems of nonlinear equations and compare them by introducing two qualification criteria. In this category, our parameters are: order of convergence (OC) or computational order of convergence (COC), number of iteration(IT), total number of function evaluations(FE), total number of linear operations evaluations (LOE), number of LU-decomposition (LU), and CPU time(CPU). It must be noted that for some iterative methods, we cannot compute their order of convergence, instead we use the computational order of convergence [14] which is computed as follows: Hence for the second category, we define our criteria as follows: (i) Jacobian Free Criterion 1 (JFC1): The first criterion is defined as (ii) Jacobian Free Criterion 2 (JFC2): The second criterion is defined as 3.1. Some Selected Iterative Methods. To show usefulness of our criteria, we have chosen some different Jacobian free methods which are presented in different years. We have applied both criteria (31) and (32) to compare qualification of the following iterative methods. Jacobian Free Newton Method (JFNM): The Jacobian free version of the classical Newton's method (with OC = 2) has the following form: where the Jacobian matrix J F ðx ðkÞ Þ has replaced by the linear operator ½w, x ; F where x = x ðkÞ and w = x ðkÞ + Fðx ðkÞ Þ satisfy in To compute the elements of (34), we can use the classical first-order divided difference operator (cf. [15]) where Jacobian Free Method 2 (JFM2): This method is presented by Amiri et al. [16] in 2018, with OC/COC = 3 and has the following form: Jacobian Free Method 3 (JFM3): Authors: Chicharro et al. [17]; Published year: 2020; OC/COC = 3: Jacobian Free Method 4(JFM4): Authors: Sharma and Arora [18]; Published year: 2013; OC/COC = 4: Jacobian Free Method 5 (JFM5): Authors: Cordero et al. [19]; Published year: 2019; OC/COC = 5: where a = x ðkÞ + Fðx ðkÞ Þ, b = x ðkÞ − Fðx ðkÞ Þ and α = ð2 − γÞ, β = ðγ − 1Þ 2 /γ, γ = 1/5.

Numerical
Results. Case n = 25. Tables 5 and 6 show the results of criteria JFC1 and JFC2 for Test Problems 1-4. As we can see from these tables, JFM6 is better among all the Jacobian free methods which are mentioned in this paper only for Test Problems 1-3; this method fails in solving the fourth test problem. Even though just for Test Problems 2 and 3, JFM5 is better, but this method fails to solve Test Problem 4 and diverges. Thus, we can claim that for case n = 25, JFM8 is a suitable method.
Case n = 80. Tables 7 and 8, respectively, show the results of criteria JFC1 and JFC2 for Test Problems 1-4. From the numerical results, JFM8 is the best method among the other Jacobian free methods for case n = 80.
3.3. Discussion on Numerical Results for Jacobian Free Methods. As computation of Jacobian matrix in solving nonlinear systems is the most consuming part of CPU time, hence applying Jacobian free solvers is very attractive. In this section, we selected just ten Jacobian free methods to compare them altogether. As our numerical results show, we can claim that high order convergence for a method can show qualification of a Jacobian free method. Therefore, our criteria can be useful to show qualification of new Jacobian free methods in the future. Meanwhile, JFM8 can be a good method to compare any new method with it.

The Third Category: Frozen Jacobian Methods
One of the attractive schemes in multistep iterative methods for solving nonlinear systems is the frozen Jacobian method.
In any iteration of multistep frozen Jacobian method, the Jacobian matrix is same in all steps. Hence, one computes the inverse of the Jacobian matrix only once. For example, this inversion is performed in LU decomposition only once in any iteration. In this paper, we compare the frozen Jacobian iterative methods which are used to solve nonlinear systems by introducing two qualification criteria. The (ii) Frozen Jacobian Criterion 2(FJC2): The second criterion is defined as 4.1. Some Selected Frozen Jacobian Methods. To show goodness of our criteria, we have chosen some different frozen Jacobian methods which are presented in the last two decades. We have applied our proposed criteria (44) and (45) to compare qualification of these iterative methods. Newton Method (NM): As our goal is the comparing iterative methods with the classical Newton's algorithm, for this category we consider the Newton's method as a one-step fro-zen Jacobian method. Therefore, the Newton's method is the first frozen Jacobian method with the following information: where J F ð·Þ −1 is the inverse of Jacobian matrix. Frozen Jacobian Method 2 (FJM2): A two-step, second order frozen Jacobian method is presented by Darvishi and Barati [6] in 2007 with the following information: FJM2 : Frozen Jacobian Method 3 (FJM3): The third order frozen Jacobian method is presented by Qasim et al. [23] in 2016 with the following information: FJM3 : Frozen Jacobian Method 4 (FJM4): The fourth order frozen Jacobian method is presented by Ahmad et al. [24] in 2016 which has the following information: FJM4 : Frozen Jacobian Method 5 (FJM5): The fifth order frozen Jacobian method is presented by Qasim et al. [23] in 2016 as follows: Frozen Jacobian Method 6 (FJM6): The sixth order frozen Jacobian method is presented by Cordero et al. [25] in 2016 as: FJM6 : Frozen Jacobian Method 7 (FJM7): The following seventh order frozen Jacobian method is presented by Montazeri et al. [4] in 2012: FJM7 : Frozen Jacobian Method 8 (FJM8): The following eighth order frozen Jacobian method is presented by Kouser et al. [26] in 2018: FJM8 : Frozen Jacobian Method 9 (FJM9): The following ninth order frozen Jacobian method is presented by Alzahrani et al. [27] in 2016: Frozen Jacobian Method 10 (FJM10): The following tenth order frozen Jacobian method is presented by Ahmad et al. [28] in 2016 with the following information: FJM10 : Frozen Jacobian Method 11 (FJM11): The following eleventh order frozen Jacobian method is presented by Qasim et al. [29] in 2016:  Case n = 500. Tables 11 and 12 show the results of criteria FJC1 and FJC2 for Test Problems 1-4. As Table 11 for FJC1 shows, FJM8 acts better than the other methods for Test Problems 1, 3, and 4 while for the other problem, FJM10 acts better. For FJC2 as Table 12 shows, six methods FJM6-FJM11 are better than the others.    Figure 2 shows that, FJM8 demonstrates brilliant results by increasing n. It must be noted that to obtain these results, the CPU time is computed in minutes. From these figures, we can conclude that order of convergence, solely, cannot show an advantage for an algorithm in this category. It must be noted that, order of convergence for FJM8 is 7 while for FJM9, FJM10, and FJM11 are 9, 9, and 10, respectively.

A Real Test Problem
In this part, we consider a real problem as our last test problem. This is the well-known Van der Pol nonlinear differential equation. Test Problem 5. The Van der Pol equation governs the flow of current in a vacuum tube, and it has the following form [30].
with boundary conditions To solve the boundary value problem (57), numerically, first we use the following second-order finite difference approximations: This leads to the following system of nonlinear equations: In system (60), h = 2/n is the step size and u 1 , u 2 , u 3 , ⋯ are the approximations of the unknown uðx i Þ, i = 1, 2, 3, ⋯.
Results for the first category. Tables 13 and 14 show the values of parameters of the algorithms in the first category for cases n = 100 and n = 250, respectively. Table 15 shows the values of CR1 and CR2 for the algorithms of this category for n = 100. As this table shows, algorithm M9 is the best algorithm among the others. By increasing the number of equations in (60) to n = 250, M3 has the best values for criteria CR1 and CR2 as Table 16 shows. These confirm our previous discussion on Test Problems 1-4.
Results for the second category. As one may see from Table 17, in this category and for n = 10, JFM5 works better than the other algorithms. Because our criteria JFC1 and JFC2 have better values for this method. By increasing n to n = 25, again JFM5 is the best algorithm among the others. We can see this superiority from the reported results in Table 18.
Results for the third category. For the algorithms of the third category and for n = 100, FJM11 is the best algorithm for this case, see Table 19, while for n = 250 as Table 20 shows, FJM6 is the best algorithm among the others.

The Classical Efficiency Indices and their Corrections
As we mentioned in the first section, there are some qualification criteria for nonlinear solvers which we call them the classical efficiency indices. To have a comparison between our qualification criteria and the classical ones, in this section we present numerical results for the classical efficiency indices for all mentioned methods in each category to solve Test Problem 1. Table 21 shows values of the classical efficiency index for the methods of the first category. As this table shows, methods M9 and M10 work better than the other methods.

The First Category. The classical efficiency index (EI ).
The operational index (I op ). Table 22 shows the values of I op for the methods in the first category. Therefore, by this criterion, M2 is the best method among the others which conflicts with EI.
The computational efficiency index (CEI). Table 23 shows the values of CEI for the methods in the first category. From the results of this table, we conclude that using this criterion, M2 is the best method among the others for n ≥ 10.
The flops-like efficiency index (FLEI ). Table 24 shows the values of FLEI for the methods in the first category. Here also, M2 is the distinguished method.
where p is the order of convergence, d is the number of functional evaluations per step, and op shows the number of products/quotients per step. Table 25 shows the values of CI for the methods in the first category which shows M2 is the best method among the other methods. Therefore, by EI, CEI, FLEI, and CI, M2 is the best method among all methods of the first category.
6.2. The Second Category. The classical efficiency index. Table 26 shows the values of EI for the methods in the second category. This criterion selects JFM5 as the best method.
The operational index. Table 27 shows values of I op for the methods in the second category. This criterion fails to introduce the best method. For small values of n, namely n = 3, JFNM shows a better result, for 5 ≤ n ≤ 80, JFM5 is the better method while for n = 80, JFM6 is the best method.    The computational efficiency index. Table 28 shows the results of CEI for the methods in the second category. By this index, JFM6 is the best method among all methods.
The flops-like efficiency index. The values of FLEI are presented in Table 29 for the methods in the second category. By this criterion, JFM5 is the best method.
The computational index. Table 30 shows results of CI for the methods in the second category. From the reported results in this table, we conclude that by this criterion JFM5 is the best method. Therefore for the solvers in the second category, three criteria EI, FLEI, and CI introduce JFM5 as the best method while JFM6 is selected as the best method by I op , and CEI.
6.3. The Third Category. The classical efficiency index. Table 31 shows the values of EI for the methods in the third category. For large values of n, FJM8 is better than the other methods by criterion EI.
The operational index. Table 32 shows the values of I op for the methods in the third category. By this index, FJM11 works better than the other methods for large values of n while for small values of the problem size, FJM6 is the best method.
The computational efficiency index. The values of CEI for the methods in the third category are presented in Table 33. By CEI, there is a similar situation to I op .
The flops-like efficiency index. Table 34 shows the values of FLEI for the methods in the third category. By this criterion, one cannot select a distinguished method which works better than the other methods for all values of n.
The computational index. Table 35 shows the values of CI for the methods in the third category. As this table shows, for n ≤ 10, the value of CI for FJM6 is better with respect to     Hence for this category, one cannot select the best method by all indices.
6.4. Corrections on the Classical Efficiency Indices. As we know, CPU time is a very important parameter to show quality of an algorithm, but this has not considered in the classical efficiency indices. In this part, we enter this parameter to the classical efficiency indices and compute the values of corrected indices to solve Test Problem 1. Since for all nonlinear solvers CPU time has a negative role, hence we introduce the corrected efficiency indices as follows:    The results of these corrected indices for solving Test Problem 1 are reported in Tables 36-38 for our three categories and some selected values of n. As we can see from Tables 26-28 and 31-35, by using the classical efficiency indices, we cannot crucially talk about the best method in some categories. But as Tables 36-38 show, for the corrected efficiency indices and for any case, there is a unique method which acts better than the other methods. For the first category, M2 is the best method (Cf . Table 36). For the second category, JFM5 is the best method (the results may be found in Table 37). Finally, for the third category, FJM2 is the best method (Cf.

Concluding Remarks
Solving systems of nonlinear equations is a very applicable field in different areas of mathematics, physics, etc. Each year many different algorithms are presented to solve nonlinear systems. As there are different types for these solvers, hence comparing them must be done in a right category. In this paper, some new qualification criteria for nonlinear solvers were introduced. They use all parameters in each relevant algorithm, such as order of convergence, CPU time, number of function, and Jacobian evaluations. Hence, they are suitable to compare different algorithms which solve       Journal of Mathematics systems of nonlinear equations. In this paper, just to show usefulness of these criteria, we only compared some nonlinear solvers from different categories. As a matter of fact, we must categorise different algorithms, e.g., for their order of convergence, after this categorization, we can compare any algorithm with same convergence order. In fact, we cannot compare a fourth order method with a second or a third order one. Therefore, we can use some of the presented cri-teria in this paper to nonlinear solvers from a similar category. Thus, to compare different algorithms, first we categorised nearly all solvers to three different categories, namely, solvers which use Jacobian matrix, the Jacobian free methods, and the frozen Jacobian ones. After that, for any category two qualification criteria were presented which use all parameters of the algorithm. Next, in each category we presented numerical results of our criteria and between   As a suggestion, we can compute the CPU time in seconds or minutes for small sizes of the problems. But for large values of n, it is better that computes in hours.
The presented criteria will be useful for researchers which work on nonlinear systems. They can use a relevant criterion from the category of their method and present its value to show qualification of their algorithm. A new solver must be able to solve any problem with any size as well. Finally from our presented results, we suggest the followings to solve nonlinear systems and/or presenting any qualification criterion: (i) To investigate validity of an algorithm, using one criterion is not enough. Hence, applying two or more criteria can show a better validity   (ii) A qualification criterion depends on the algorithm and type of the nonlinear system. Hence, a suitable criterion must be selected for the problem. This will need more works in the future (iii) The suitable criterion directly depends on the category of the used algorithm (iv) The numerical results of this paper have obtained by four, ten, or eleven digits depending on how the results are near together. But it must be noted that, the choice of data type influences the performance of the solvers. For example, using single, double, and quad precision floating-point data. In fact, the performance usually suggests precision evaluation (v) The practical implementations of methods heavily depend on computer type. For example, specialized computational platforms such as FPGA (Field Programmable Gate Array) or vector computers can speed up the calculations for specific algorithms (vi) Here, we only considered explicit algorithms. Introducing qualification criteria for using implicitexplicit solvers will be a new research and an attractive work for the future (vii) The CPU time is a very important index to be in any criterion. Therefore, the corrected classical efficiency indices will be very useful to show the quality of any new nonlinear solver In summary, for the first time we have presented some new robust qualification criteria which remove the gaps of the previous ones. These criteria consider all parameters of the nonlinear solvers. Further, we have proposed some suggestions for each old criterion by entering the CPU time in its formula. Also, some outlines for the continuation of this work are presented for the future researches.

Data Availability
The data that support the findings of the study are available upon reasonable request from the authors.