ANA Advances in Numerical Analysis 1687-9570 1687-9562 Hindawi Publishing Corporation 563872 10.1155/2013/563872 563872 Research Article Modular Analysis of Sequential Solution Methods for Almost Block Diagonal Systems of Equations El-Mistikawy Tarek M. A. Benzi Michele Department of Engineering Mathematics and Physics Cairo University Giza 12211 cu.edu.eg Egypt 2013 24 2 2013 2013 18 05 2012 21 08 2012 2013 Copyright © 2013 Tarek M. A. El-Mistikawy. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Almost block diagonal linear systems of equations can be exemplified by two modules. This makes it possible to construct all sequential forms of band and/or block elimination methods. It also allows easy assessment of the methods on the basis of their operation counts, storage needs, and admissibility of partial pivoting. The outcome of the analysis and implementation is to discover new methods that outperform a well-known method, a modification of which is, therefore, advocated.

1. Introduction

Systems of equations with almost block diagonal (ABD) matrix of coefficients are frequently encountered in numerical solutions of sets of ordinary or partial differential equations. Several such situations were described by Amodio et al. , who also reviewed sequential and parallel solvers to ABD systems and came to the conclusion that sequential solution methods needed little further study.

Traditionally, sequential solution methods of ABD systems performed 𝕃𝕌 decompositions of the matrix of coefficients 𝔾 through either band (scalar) elimination or block tridiagonal elimination. The famous COLROW algorithm , which is highly regarded for its performance, was incorporated in several applications . It utilizes Lam’s alternating column/row pivoting  and Varah’s correspondingly alternating column/row scalar elimination . The efficient block tridiagonal methods included Keller’s Block Tridiagonal Row (BTDR) elimination method [10, Section 5, case i] and El-Mistikawy’s Block Tridiagonal Column (BTDC) elimination method . Both methods could apply a suitable form of Keller’s mixed pivoting strategy , which is more expensive than Lam’s.

The present paper is intended to explore other variants of the 𝕃𝕌 decomposition of 𝔾. It does not follow the traditional approaches of treating the matrix of coefficients as a banded matrix or casting it into a block tridiagonal form. It, rather, adopts a new approach, modular analysis, which offers a simple and unified way of expressing and assessing solution methods for ABD systems.

The matrix of coefficients 𝔾 (or, more specifically, its significant part containing the nonzero blocks) is disassembled into an ordered set of modules. (In fact, two different sets of modules are identified.) Each module Γ is an entity that has a head and a tail. By arranging the modules in such a way that the head of a module is added to the tail of the next, the significant part of 𝔾 can be reassembled. The module exemplifies the matrix, but is much easier to analyze.

All possible methods of 𝕃𝕌 decomposition of 𝔾 could be formulated as decompositions of Γ. This led to the discovery of two new promising methods: Block Column/Block Row (BCBR) Elimination and Block Column/Scalar Row (BCSR) Elimination.

The validity and stability of the elimination methods are of primary concern to both numerical analysts and algorithm users. Validity means that division by a zero is never encountered, whereas stability guards against round-off-error growth. To insure validity and achieve stability, pivoting is called for . Full pivoting is computationally expensive requiring full two-dimensional search for the pivots. Moreover, it destroys the banded form of the matrix of coefficients. Partial pivoting strategies, though potentially less stable, are considerably less expensive. Unidirectional (row or column) pivoting makes a minor change to the form of 𝔾 by introducing few extraneous elements. Lam’s alternating pivoting , which involves alternating sequences of row pivoting and column pivoting, maintains the form of 𝔾. When 𝔾 is nonsingular, Lam’s pivoting guarantees validity, and if followed by correspondingly alternating elimination it produces multipliers that are bounded by unity, thus enhancing stability. This approach was proposed by Varah  in his 𝕃𝔾~𝕌 decomposition method. It was developed afterwards into a more efficient 𝕃𝕌 version—termed here Scalar Column/Scalar Row (SCSR) Elimination—that was adopted by the COLROW solver .

The present approach of modular analysis shows that Lam’s pivoting (with Varah’s arrangement) applies to the BCBR and BCSR elimination methods, as well. It even applies to the two block tridiagonal elimination methods BTDR and BTDC, contrary to the common belief. A more robust, though more expensive strategy, Local Pivoting, is also identified. It performs full pivoting over the same segments of 𝔾 (or Γ) to which Lam’s pivoting is applied. Keller’s mixed pivoting  is midway between Lam’s and local pivoting.

Modular analysis also allows easy estimation of the operation counts and storage needs, revealing the method with the best performance on each account. The method having the least operation count is BCBR elimination, whereas the method requiring the least storage is BTDC elimination . Both methods achieve savings of leading order importance, for large block sizes, in comparison with other methods.

Based on the previous assessment, and realizing that programming factors might affect the performance, four competing elimination methods were implemented. The COLROW algorithm, which was designed to give SCSR elimination its best performance, was modified to perform BTDC, BCBR, or BCSR elimination, instead. The four methods were applied to the same problems, and the execution times were recorded. BCSR elimination proved to be an effective modification to the COLROW algorithm.

2. Problem Description

Consider the almost block diagonal system of equations 𝔾z=g whose augmented matrix of coefficients 𝔾+=[𝔾g] has the form (1) The blocks with leading characters A, B, C, and D have m, n, m, and n columns, respectively. The blocks with leading character g have r columns indicating as many right-hand sides. The trailing character m or n (and subsequently p=m+n) indicates the number of rows of a block or the order of a square matrix such as an identity matrix I and a lower L or an upper U triangular matrix. Blanks indicate zero blocks.

The matrix of unknowns z is written, similarly, as (2)z=[zm1tzn1tzm2tzn2tzmjtznjtzmJtznJt]t, where the superscript t denotes the transpose.

Such a system of equations results, for example, in the finite difference solution of p first order ODEs on a grid of J points with m conditions at one side to be marked with j=1 and n conditions at the other side that is to be marked with j=J. Then, each column of the submatrix [zmjtznjt]t contains the p unknowns of the jth grid point, corresponding to a right-hand side.

3. Modular Analysis

The description of the decomposition methods for the augmented matrix of coefficients 𝔾+ can be made easy and concise through the introduction of modules of 𝔾+. Two different modules are identified:

The Aligned Module (A-Module) (3) The Displaced Module (D-Module) (4) (For convenience, we will occasionally drop the subscript and/or superscript identifying a module and its components (given later), as well as the subscript identifying its blocks.)

As a rule, the dotted line defines the partitioning to left and right entities. The dashed lines define the partitioning (5) to the following components: the stem σj, the head ηj+[ηjθj], and the fins ψj and ϕj+[ϕjρj].

Each module has a tail τj+[τjλj]. For Γj+A, τj+A=[Cmj#Dmj#gmj#] which is defined through the head-tail relation (6)

For which is, likewise, defined through the head-tail relation (7)

This makes it possible to construct the significant part of 𝔾+ by arranging each set of modules in such a way that the tail of Γj+ adds to the head of Γj-1+, for j=0J. Minor adjustments need only to be invoked at both ends of 𝔾+. Specifically, we define the truncated modules (8)

The head of the module Γ+ is yet to be defined. It is taken to be related to the other components of Γ+ by (9)η+=ψσ-1ϕ+, in order to allow for decompositions of Γ+ having the form (10)

The generic relations σ=MN, ψ=ΨN,  ϕ+=MΦ+, and η+=ΨΦ+ then hold, leading to η+=ΨΦ+=(ψN-1)(M-1ϕ+)=ψ(N-1M-1)ϕ+=ψσ-1ϕ+1 as defined in (9).

3.1. Elimination Methods

All elimination methods can be expressed in terms of decompositions of the stem σj. Only those worthy methods that allow alternating column/row pivoting and elimination are presented here. Several inflections of the blocks of 𝔾 are involved and are defined in the appendix section. The sequence in which the blocks are manipulated for: decomposing the stem, processing the fins, and handling the head (evaluating the head and applying the head-tail relation to determine the tail of the succeeding module Γj+1), is mentioned along with the equations (from the appendix section) involved. The correctness of the decompositions may be checked by carrying out the matrix multiplications, using the equalities of the appendix section, and comparing with the un-decomposed form of the module.

The following three methods can be generated from either module. They will be given in terms of the aligned module.

3.1.1. Scalar Column/Scalar Row (SCSR) Elimination

This is the method implemented by the COLROW algorithm. It performs scalar decomposition of the stem σ. The triangular matrices L and U appear explicitly. If unit diagonal, they are marked with a circumflex ^: (11)

The following sequence of manipulations applies:

: LmU^m(A7), Dm(A16), An(A9), Bn#(A3b), L^nUn(A6).

: Am(A8), Bm#(A2b), Bm(A14), Cn(A10), Dn(A11).

: Cm#(A4b), Dm#(A5b).

3.1.2. Block Column/Block Row (BCBR) Elimination

The method performs block decomposition of the stem σ, in which the decomposed pivotal blocks C˘m#(LmU^m) and B˘n#(L^nUn) appear: (12)

The following sequence of manipulations applies:

C˘m#(A7), Dm(A16), Dm*(A17), Bn#(A3a), B˘n#(A6).

Bm#(A2a), Bm(A14), Bm*(A15).

Cm#(A4a), Dm#(A5a).

3.1.3. Block Column/Scalar Row (BCSR) Elimination

The method has the decomposition (13)

The following sequence of manipulations applies:

C˘m#(A7), Dm(A16), Dm*(A17), Bn#(A3a), L^nUn(A6).

Bm#(A2a), Bm(A14), Cn(A10), Dn(A11).

Cm#(A4b), Dm#(A5b).

3.1.4. Block-Tridiagonal Row (BTDR) Elimination

This method can be generated from the aligned module only. It performs the identity decomposition σ=Ipσ˘, leading to the decomposition (14)

In [10, Section 5, case (i)], scalar row elimination was used to obtain the decomposed stem σ˘. However, σ˘ can, by now, be obtained by any of the nonidentity (scalar and/or block) decomposition methods given in Sections 3.1.13.1.3.

Using SCSR elimination, the following sequence of manipulations applies:

LmU^m(A7), Dm(A16), An(A9), Bn#(A3b), L^nUn(A6).

Am(A8), Bm#(A2b), Bm(A14), Bm*(A15), Am(A12), Am*(A13).

Cm#(A4a), Dm#(A5a).

3.1.5. Block-Tridiagonal Column (BTDC) Elimination

This method can be generated from the displaced module only. It performs the identity decomposition σ=σ˘Ip, leading to the decomposition (15)

In , σ˘ was obtained by scalar column elimination. As with BTDR elimination, σ˘ can, by now, be obtained from any of the nonidentity decompositions given in Sections 3.1.13.1.3.

Using SCSR elimination, the following sequence of manipulations applies:

L^nUn(A6), Cn(A10), Bm(A14), Cm#(A4b), LmU^m(A7).

Dn(A11), Dm#(A5b), Dm(A16), Dm*(A17), Dn(A18), Dn*(A19).

Bn#(A3a), Bm#(A2a).

3.2. Solution Procedure

The procedure for solving the matrix equation 𝔾+z=g, which can be described in terms of manipulation of the augmented matrix 𝔾+[𝔾g], can, similarly, be described in terms of manipulation of the augmented module Γj+[Γjγj]. The manipulation of 𝔾+ applies a forward sweep which corresponds to the decomposition 𝔾+=𝕃𝕌+𝕃[𝕌Z] that is followed by a backward sweep which corresponds to the decomposition 𝕌+=𝕌[Iz]. Similarly, the manipulation of Γj+ applies a forward sweep involving two steps. The first step performs the decomposition Γj+=μjνj+. The second step evaluates the head ηj+=ΨjΦj+ then applies the head-tail relation to determine the tail of Γj+1+. In a backward sweep, two steps are applied to νj+[NjΦjΖj] leading to the solution module zj(zjA=[zmjtznjt]t,zjD=[znjtzmj+1t]t). With zj+1 known, the first step uses in the back substitution relation to contract νj+ to Nj+[Njζj]. The second step solves Njzj=ζj for zj which is equivalent to the decomposition Nj+=Nj[Ipzj].

3.3. Operation Counts and Storage Needs

The modules introduced previously allow easy evaluation of the elimination methods. The operation counts are measured by the number of multiplications (mul) with the understanding that the number of additions is comparable. The storage needs are measured by the number of locations (loc) required to store arrays calculated in the forward sweep for use in the backward sweep, provided that the elements of 𝔾+ are not stored but are generated when needed, as is usually done in the numerical solution of a set of ODEs, for example.

Per module (i.e., per grid point), each method requires, for the manipulation of 𝔾+, as many operations as it requires to manipulate Γ+. All methods require (p3-p)/3 (mul) for decomposing the stem, pmn (mul) for evaluating the head, and 2p2r (mul) to handle the right module γ. The methods differ only in the operation counts for processing the fins ψ and ϕ, with BCBR elimination requiring the least count pmn (mul).

Per module, each method requires as many storage locations as it requires to store ν+[νZ]. All methods require pr (loc) to store Z. They differ in the number of locations needed for storing the ν’s, with BTDC elimination requiring the least number pn (loc). Note that, in SCSR and BCSR eliminations, square blocks need to be reserved for storing the triangular blocks U^m and/or Un.

Table 1 contains these information, allowing for clear comparison among the methods. For example, when p1, BCBR elimination achieves savings in operations that are of leading order significance ~p3/8 and ~p3/2, respectively, in the two distinguished limits m~n~p/2 and (m,n)~(p,1), as compared to SCSR elimination.

Operation counts and storage needs.

 Method Operation counts (mul) Storage needs (loc) 2 p 2 r + ( p 3 - p ) / 3 + 2 p m n + p r + p n + BCBR 0 p n SCSR ( m 3 + n 3 - m 2 - n 2 ) / 2 p n + m 2 BCSR ( n 3 - n 2 ) / 2 p n BTDC p n 2 0
3.4. Pivoting Strategies

Lam’s alternating pivoting  applies column pivoting to τA=[Cm#Dm#] in order to form and decompose a nonsingular pivotal block Cm# and applies row pivoting to τD=[Bn#tBm#t]t in order to form and decompose a nonsingular pivotal block Bn#. These are valid processes since τA is of rank m and τD is of rank n, as can be shown following the reasoning of Keller [10, Section 5, Theorem].

To enhance stability further we introduce the Local pivoting strategy which applies full pivoting (maximum pivot strategy) to the segments τA and τD. Note that Keller’s mixed pivoting , if interpreted as applying full pivoting to τA and row pivoting to τD, is midway between Lam’s and local pivoting.

Lam’s, Keller’s, and local pivoting apply to all elimination methods of Section 3.1. Moreover, in any given problem, each pivoting strategy would produce the same sequence of pivots, regardless of the elimination method in use.

4. Implementation

The COLROW algorithm, which is based on SCSR elimination, is modified to perform BTDC, BCBR, or BCSR elimination, instead. As an illustration, the four methods are applied to three systems of equations having the augmented matrix given in (1), with J=11 and r=1. The solution procedures are repeated k times, so that reasonable execution times can be recorded and compared. All calculations are carried out in double precision via Compaq Visual Fortran (version 6.6) that is optimized for speed, on a Pentium 4 CPU rated at 2 GHz with 750 MB of RAM.

The first system has p=11 and k=106. Different combinations of m/n=10/1, 9/2, 8/3, 7/4, and 6/5 are considered. The execution times, without pivoting, are given in Table 2. All entries include 22 seconds that are required to read, write, and run empty Fortran-Do-Loops. Pivoting requires additional 20 seconds in all methods.

Execution times in seconds for the first system.

m / n a SCSR BTDC BCBR BCSR
10/1 109 96 93 91
9/2 113 104 106 99
8/3 118 116 115 110
7/4 120 126 122 117
6/5 123 133 133 125

a m + n = 11 .

The second (third) system has p=21 (51) and k=105 (104). The execution times, with Lam’s pivoting, are given in Table 3 (4), for different combinations of m/n.

Execution times in seconds for the second system.

 m / n a SCSR BTDC BCBR BCSR 20/1 67.9 52.0 51.8 51.4 18/3 70.6 59.9 60.0 55.0 16/5 72.0 68.3 73.8 66.3 14/7 73.0 74.2 80.8 73.8 12/9 73.1 78.3 81.4 74.8 11/10 73.6 81.7 82.3 75.9

a m + n = 21 .

Although the modified COLROW algorithms may not produce the best performances of BTDC, BCBR, and BCSR eliminations, Tables 2, 3, and 4 clearly indicate that they, in some cases (when mn), outperform the COLROW algorithm that is designed to give SCSR elimination its best performance.

Execution times in seconds for the third system.

m / n a SCSR BTDC BCBR BCSR
50/1 60.51 38.05 39.19 38.04
46/5 62.71 48.30 57.55 49.15
41/10 65.56 58.97 76.25 61.57
36/15 68.12 65.86 87.53 68.35
31/20 69.09 76.38 110.61 85.24
26/25 69.86 83.95 115.95 94.00

a m + n = 51 .

5. Conclusion

Using the novel approach of modular analysis, we have analyzed the sequential solution methods for almost block diagonal systems of equations. Two modules have been identified and have made it possible to express and assess all possible band and block elimination methods. On the basis of the operation counts, storage needs, and admissibility of partial pivoting, we have determined four distinguished methods: Block Column/Block Row (BCBR) Elimination (having the least operation count), Block Tridiagonal Column (BTDC) Elimination (having the least storage need), Block Column/Scalar Row (BCSR) elimination, and Scalar Column/Scalar Row (SCSR) elimination (implemented in the well-known COLROW algorithm). Application of these methods within the COLROW algorithm shows that they outperform SCSR elimination, in cases of large top-block/bottom-block row ratio. In such cases, BCSR elimination is advocated as an effective modification to the COLROW algorithm.

Appendix

In Section 3, two modules, ΓA and ΓD, of the matrix of coefficients 𝔾 are introduced and decomposed to generate the elimination methods. The process involves inflections of the blocks of 𝔾, which proceed for a block E, say, according to the following scheme: (A.1)EE#ifpivotalE˘#EE′′ifpivotalE*.

The following equalities are to be used to determine the blocks with underscored leading character. The number of multiplications involved is given between braces:

E # -Blocks (A.2)Bm-B_m#=aAmDm*=bAmDm′′{m2n},(A.3)Bn-B_n#=aAnDm*=bAnDm′′{mn2},(A.4)Cm-C_m#=aBm*Cn=bBm′′Cn{m2n},(A.5)Dm-D_m#=aBm*Dn=bBm′′Dn{mn2}E˘#-Blocks (decomposed pivotal blocks) (A.6)Bn#=B˘_n#(L_nU_n){n3-n3}(A.7)Cm#=C˘_m#(L_mU_m){m3-m3},

E -Blocks (A.8)A_mU^m=Am{m2-m2m},(A.9)A_nU^m=An{m2-m2n},(A.10)L^nC_n=Cn{n2-n2m}(A.11)L^nD_n=Dn{n2-n2n},E′′ and E*-Blocks (A.12)A_m′′=Am-Bm*An{m2n},(A.13)A_m*Lm=Am′′{m2+m2m}(A.14)B_m′′Un=Bm#{n2+n2m},(A.15)B_m*L^n=Bm′′{n2-n2m},(A.16)LmD_m′′=Dm#{m2+m2n},(A.17)U^mD_m*=Dm′′{m2-m2n}(A.18)D_n′′=Dn-CnDm*{mn2},(A.19)UnD_n*=Dn′′{n2-n2n}.

Amodio P. Cash J. R. Roussos G. Wright R. W. Fairweather G. Gladwell I. Kraut G. L. Paprzycki M. Almost block diagonal linear systems: sequential and parallel solution techniques, and applications Numerical Linear Algebra with Applications 2000 7 5 275 317 1766916 Díaz J. C. Fairweather G. Keast P. FORTRAN packages for solving certain almost block diagonal linear systems by modified alternate row and column elimination ACM Transactions on Mathematical Software 1983 9 3 358 375 10.1145/356044.356053 791972 Cash J. R. Wright M. H. A deferred correction method for nonlinear two-point boundary value problems: implementation and numerical evaluation SIAM Journal on Scientific and Statistical Computing 1991 12 4 971 989 10.1137/0912052 1102418 Cash J. R. Moore G. Wright R. W. An automatic continuation strategy for the solution of singularly perturbed linear two-point boundary value problems Journal of Computational Physics 1995 122 2 266 279 10.1006/jcph.1995.1212 1365435 Enright W. H. Muir P. H. Runge-Kutta software with defect control for boundary value ODEs SIAM Journal on Scientific Computing 1996 17 2 479 497 10.1137/S1064827593251496 1374292 Keast P. Muir P. H. Algorithm 688: EPDCOL: a more efficient PDECOL code ACM Transactions on Mathematical Software 1991 17 2 153 166 2-s2.0-0026168065 10.1145/108556.108558 Wright R. W. Cash J. R. Moore G. Mesh selection for stiff two-point boundary value problems Numerical Algorithms 1994 7 2–4 205 224 10.1007/BF02140684 1283099 Lam D. C. Implementation of the box scheme and model analysis of diffusion-convection equations. [Ph.D. thesis] 1974 Ontario, Canada University of Waterloo: Waterloo Varah J. M. Alternate row and column elimination for solving certain linear systems SIAM Journal on Numerical Analysis 1976 13 1 71 75 0411199 Keller H. B. Accurate difference methods for nonlinear two-point boundary value problems SIAM Journal on Numerical Analysis 1974 11 305 320 0351098 El-Mistikawy T. M. A. Solution of Keller's box equations for direct and inverse boundary-layer problems AIAA journal 1994 32 7 1538 1541 2-s2.0-0028468771 Duff I. S. Erisman A. M. Reid J. K. Direct Methods for Sparse Matrices 1986 Oxford, UK Clarendon Press 892734