Journal of Applied Mathematics

Volume 2013 (2013), Article ID 875935, 6 pages

http://dx.doi.org/10.1155/2013/875935

Research Article

An Improved Diagonal Jacobian Approximation via a New Quasi-Cauchy Condition for Solving Large-Scale Systems of Nonlinear Equations

Mohammed Yusuf Waziri1,2 and Zanariah Abdul Majid1,3

¹Department of Mathematics, Faculty of Science, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia

²Department of Mathematical Sciences, Faculty of Science, Bayero University, Kano PMB 3011, Nigeria

³Institute for Mathematical Research, Universiti Putra Malaysia, 43400 Serdang, Selangor, Malaysia

Received 8 August 2012; Revised 14 December 2012; Accepted 15 December 2012

Academic Editor: Turgut Öziş

Copyright © 2013 Mohammed Yusuf Waziri and Zanariah Abdul Majid. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

We present a new diagonal quasi-Newton update with an improved diagonal Jacobian approximation for solving large-scale systems of nonlinear equations. In this approach, the Jacobian approximation is derived based on the quasi-Cauchy condition. The anticipation has been to further improve the performance of diagonal updating, by modifying the quasi-Cauchy relation so as to carry some additional information from the functions. The effectiveness of our proposed scheme is appraised through numerical comparison with some well-known Newton-like methods.

1. Introduction

Let us consider the systems of nonlinear equations

where is a nonlinear mapping. Often, the mapping is assumed to be satisfying the following assumptions:

(A1) there exists an s.t ;(A2) is a continuously differentiable mapping in a neighborhood of ;(A3) is invertible.

The well-known method for finding the solution to (1) is the classical Newton’s method which generates a sequence of iterates from a given initial point via

where . The attractive features of this method are rapid convergence and being easy to implement. Nevertheless, Newton’s method requires the computation of the matrix entails the first-order derivatives of the systems. In practice, computations of some functions derivatives are quite costly, and sometimes they are not available or could not be done precisely. In this case, Newton’s method cannot be applied directly.

Moreover, some substantial efforts have been made by numerous researchers in order to eliminate the well-known shortcomings of Newton’s method for solving systems of nonlinear equations, particularly large-scale systems (see, e.g., [1, 2]). Notwithstanding, most of these modifications of Newton’s method still have some shortfalls as Newton’s counterpart. For example, Broyden’s method and Chord Newton’s method need to store an matrix, and their floating points operations, are , respectively.

To tackle these disadvantages, a diagonally Newton’s method has been suggested by Leong et al. [3] and showed that their updating formula is significantly cheaper than Newton’s method and some of its variants. Based on this fact, it is pleasing to present an approach which will improve further the diagonal Jacobian approximation, as well as reducing the computational cost, floating points operations and number of iterations. This is what leads to the idea of this paper. The anticipation has been to further improve the performance of diagonal updating, by modifying the quasi-Cauchy relation so as to carry some additional information from the functions. We organized the paper as follows. In the next section, we present the details of the proposed method. Convergence results are present in Section 3. Some numerical results are reported in Section 4. Finally, conclusions are made in Section 5.

2. Derivation Process

This section presents a new diagonal quasi-Newton-like method for solving large-scale systems of nonlinear equations. The quasi-Newton method is an iterative method that generates a sequence of points from a given initial guess via the following form:

where is a step length and is an approximation to the Jacobian inverse which can be updated at each iteration for ; the updated matrix is chosen in such a way that it satisfies the secant equation, that is,

It is clear that the only Jacobian information we have is , and this is only approximation information. To this end, we incorporate more information from and to in order to present a better approximation to the Jacobian matrix. We consider the modification on presented by Li and Fukushima [4]:

where .

Our aim here is to build a square matrix, say , using diagonal updating scheme which is an approximation to the Jacobian inverse, and we let satisfy the quasi-Cauchy equation, that is,

In addition, the deviation between and is minimized under some norms; hence, in the following theorem, we state the resulting update formula for .

Theorem 1. Assume that be the diagonal update of a diagonal matrix . Let us denote the deviation between and as . Suppose that which is defined by (5). Consider the following problem:

where denotes the Frobenius norm. Hence, the optimal solution of (7) is given by

where , , and is the trace operation.

Proof. Consider the Lagrangian function of (7):

where is the corresponding Lagrangian multiplier. By differentiating with respect to each , and setting them all equal to zero, we obtain

Multiplying both sides of (10) by and summing them all give

Differentiating with respect to , and since , then we have

Equating (11) and (12) and substituting the relation into (10), finally we have

Since is a diagonal component of , is the component of vector , then and . We further rewrite (13) as

which completes the proof.

Hence, the best possible updating formula for diagonal matrix is given by

Now, we can describe the algorithm for our proposed method as follows.

Algorithm IDJA

Step 1. Choose an initial guess , , , , , and let .

Step 2. Compute , and If stop.

Step 3. Compute .

Step 4. If , retain and go to Step 5. Otherwise set and repeat Step 4.

Step 5. If , retain and go to Step 6. Otherwise set and repeat Step 5.

Step 6. Let .

Step 7. If stop. Also go to Step 8.

Step 8. If where , compute as defined by (15); if not, .

Step 9. Set and go to Step 2.

3. Convergence Result

This section presents local convergence results of the IDJA methods. To analyze the convergence of these methods, we will make the following assumptions on nonlinear systems .

Assumption 2. is differentiable in an open convex set in .

There exists such that ; is continuous for all .

satisfies the Lipschitz condition of order one that is there exists a positive constant such that

for all .

There exist constants such that for all and .

We can state the following result on the boundedness of by assuming that, without loss of generality, the updating matrix (15) is always used, then we have the following.

Theorem 3. Suppose that is generated by Algorithm IDJA where is defined by (15). Assume that Assumption 2 holds. There exists , , and , such that if and satisfies for all then

for some constant , .

Proof. Since , it follows that

For and assuming , we have

where is the largest element among .

After multiplying (19) by and substituting , we have

Since , then (20) turns into

From Assumption 2 and , (21) becomes

where .

Since for , it follows that

Hence, we obtain

Suppose , then

From the fact that , it follows that

where .

Therefore, if we assume that , then

therefore, .

Hence, by induction, for all .

4. Numerical Results

In this section, the performance of IDJA method has been presented, when compared with Broyden’s method (BM), Chord Newton’s method (CN), Newton’s method (NM), and (DQNM) method proposed by [3], respectively. The codes are written in MATLAB 7.4 with a double precision computer; the stopping condition used is

The identity matrix has been chosen as an initial approximate Jacobian inverse.

We further design the codes to terminates whenever one of the following happens:

(i)the number of iteration is at least 200 but no point of that satisfies (28) is obtained; (ii)CPU time in seconds reaches 200; (iii)Insufficient memory to initial the run.

The performance of these methods are compared in terms of number of iterations and CPU time in seconds. In the following, some details on the benchmarks test problems are presented.

Problem 1. Spares 1 function of Shin et al. [5]:

Problem 2. Trigonometric function of Spedicato [6]

Problem 3. System of nonlinear equations

Problem 4. System of nonlinear equations

Problem 5. System of nonlinear equations

Problem 6. System of nonlinear equations

Problem 7. System of nonlinear equations

Problem 8. System of nonlinear equations

The numerical results presented in Tables 1, 2, 3, 4, and 5 demonstrate clearly the proposed method (IDJA) shows good improvements, when compared with NM, CN, BM, and DQNM, respectively. In addition, it is worth mentioning, the IDJA method does not require more storage locations than classic diagonal quasi-Newton’s methods. One can observe from the tables that the proposed method (IDJA) is faster than DQNM methods and required little time to solve the problems when compared to the other Newton-like methods and still keeping memory requirement and CPU time in seconds to only .

Table 1: Numerical results of NM, CN, BM, DQNM, and IDJA methods.

prob
Dim
NM
CN
BM
DQNM
IDJA

NI
CPU
NI
CPU
NI
CPU
NI
CPU
NI
CPU

1
50
7
0.046
55
0.031
15
0.031
14
0.016
2
0.011

2
50
9
0.078
344
0.062
15
0.031
15
0.031
13
0.031

3
50
10
0.062
—
—
—
—
20
0.016
10
0.016

4
50
—
—
—
—
—
—
19
0.031
9
0.031

5
50
12
0.078
—
—
42
0.031
16
0.016
8
0.015

6
50
8
0.064
—
—
16
0.032
14
0.031
7
0.014

7
50
8
0.094
—
—
—
—
25
0.031
14
0.010

8
50
11
0.064
—
—
11
0.0312
11
0.016
9
0.016

Table 2: Numerical Results of NM, CN, BM, DQNM, and IDJA methods.

prob
Dim
NM
CN
BM
DQNM
IDJA

NI
CPU
NI
CPU
NI
CPU
NI
CPU
NI
CPU

1
100
7
0.156
98
0.094
15
0.043
14
0.016
2
0.011

2
100
10
0.187
—
—
18
0.062
16
0.032
13
0.032

3
100
7
0.203
—
—
24
0.140
15
0.031
7
0.015

4
100
—
—
—
—
—
—
13
0.031
10
0.030

5
100
13
0.265
—
—
53
0.109
17
0.031
12
0.031

6
100
8
0.203
—
—
16
0.047
14
0.031
7
0.017

7
100
8
0.185
—
—
—
—
26
0.031
16
0.030

8
100
11
0.234
—
—
11
0.094
11
0.032
10
0.016

Table 3: Numerical Results of NM, CN, BM, DQNM, and IDJA methods.

prob
Dim
NM
CN
BM
DQNM
IDJA

NI
CPU
NI
CPU
NI
CPU
NI
CPU
NI
CPU

1
250
7
0.359
100
0.109
15
0.101
14
0.034
2
0.032

2
250
11
0.640
—
—
21
0.218
18
0.032
8
0.031

3
250
8
0.499
—
—
29
0.250
16
0.016
9
0.016

4
250
—
—
—
—
—
—
15
0.031
10
0.032

5
250
14
0.827
—
—
—
—
19
0.031
8
0.016

6
250
8
0.686
—
—
24
0.250
14
0.031
10
0.031

7
250
8
0.499
—
—
—
—
27
0.031
14
0.031

8
250
11
0.484
—
—
11
0.125
11
0.031
10
0.016

Table 4: Numerical results of NM, CN, BM, DQNM, and IDJA methods.

prob
Dim
NM
CN
BM
DQNM
IDJA

NI
CPU
NI
CPU
NI
CPU
NI
CPU
NI
CPU

1
500
7
0.796
101
0.702
15
0.671
14
0.016
2
0.011

2
500
13
1.997
—
—
23
0.972
19
0.031
9
0.032

3
500
7
1.4352
—
—
—
—
17
0.031
9
0.031

4
500
—
—
—
—
—
—
12
0.030
10
0.031

5
500
15
2.449
—
—
—
—
21
0.031
9
0.031

6
500
8
2.184
—
—
23
0.998
14
0.032
10
0.045

7
500
8
1.498
—
—
—
—
32
0.047
15
0.047

8
500
11
1.451
—
—
11
0.515
11
0.031
9
0.031

Table 5: Numerical results of NM, CN, BM, DQNM, and IDJA methods.

prob
Dim
NM
CN
BM
DQNM
IDJA

NI
CPU
NI
CPU
NI
CPU
NI
CPU
NI
CPU

1
1000
7
2.730
103
3.167
38
9.438
14
0.016
2
0.011

2
1000
—
—
—
—
31
7.722
20
0.032
8
0.043

3
1000
9
5.819
—
—
—
—
17
0.031
9
0.031

4
1000
—
—
—
—
—
—
11
0.064
10
0.064

5
1000
16
8.705
—
—
—
—
22
0.031
10
0.031

6
1000
8
6.474
—
—
—
—
14
0.062
11
0.061

7
1000
8
4.321
—
—
—
—
38
0.062
31
0.047

8
1000
11
4.882
—
—
11
2.418
11
0.032
10
0.031

5. Conclusions

In this paper, we present an improved diagonal quasi-Newton update via new quasi-Cauchy condition for solving large-scale Systems of nonlinear equations (IDJA). The Jacobian inverse approximation is derived based on the quasi-Cauchy condition. The anticipation has been to further improve the diagonal Jacobian, by modifying the quasi-Cauchy relation so as to carry some additional information from the functions. It is also worth mentioning that the method is capable of significantly reducing the execution time (CPU time), as compared to NM, CN, BM, and DQNM methods while maintaining good accuracy of the numerical solution to some extent. Another fact that makes the IDJA method appealing is that throughout the numerical experiments it never fails to converge. Hence, we can claim that our method (IDJA) is a good alternative to Newton-type methods for solving large-scale systems of nonlinear equations.

References

J. E. Dennis, Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall, Englewood Cliffs, NJ, USA, 1983.

C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, vol. 16, SIAM, Philadelphia, Pa, USA, 1995.

W. J. Leong, M. A. Hassan, and M. Waziri Yusuf, “A matrix-free quasi-Newton method for solving large-scale nonlinear systems,” Computers & Mathematics with Applications, vol. 62, no. 5, pp. 2354–2363, 2011.

D.-H. Li and M. Fukushima, “A modified BFGS method and its global convergence in nonconvex minimization,” Journal of Computational and Applied Mathematics, vol. 129, no. 1-2, pp. 15–35, 2001.

B.-C. Shin, M. T. Darvishi, and C.-H. Kim, “A comparison of the Newton-Krylov method with high order Newton-like methods to solve nonlinear systems,” Applied Mathematics and Computation, vol. 217, no. 7, pp. 3190–3198, 2010.

E. Spedicato, “Cumputational experience with quasi-Newton algorithms for minimization problems of moderatetly large size,” Tech. Rep. CISE-N-175 3, pp. 10–41, 1975.