Modular Analysis of Sequential Solution Methods for Almost Block Diagonal Systems of Equations

Almost block diagonal linear systems of equations can be exemplified by two modules. This makes it possible to construct all sequential forms of band and/or block elimination methods. It also allows easy assessment of the methods on the basis of their operation counts, storage needs, and admissibility of partial pivoting.The outcome of the analysis and implementation is to discover new methods that outperform a well-known method, a modification of which is, therefore, advocated.


Introduction
Systems of equations with almost block diagonal (ABD) matrix of coefficients are frequently encountered in numerical solutions of sets of ordinary or partial differential equations. Several such situations were described by Amodio et al. [1], who also reviewed sequential and parallel solvers to ABD systems and came to the conclusion that sequential solution methods needed little further study.
The present paper is intended to explore other variants of the LU decomposition of G. It does not follow the traditional approaches of treating the matrix of coefficients as a banded matrix or casting it into a block tridiagonal form. It, rather, adopts a new approach, modular analysis, which offers a simple and unified way of expressing and assessing solution methods for ABD systems.
The matrix of coefficients G (or, more specifically, its significant part containing the nonzero blocks) is disassembled into an ordered set of modules. (In fact, two different sets of modules are identified.) Each module Γ is an entity that has a head and a tail. By arranging the modules in such a way that the head of a module is added to the tail of the next, the significant part of G can be reassembled. The module exemplifies the matrix, but is much easier to analyze.
All possible methods of LU decomposition of G could be formulated as decompositions of Γ. This led to the discovery of two new promising methods: Block Column/Block Row (BCBR) Elimination and Block Column/Scalar Row (BCSR) Elimination.
The validity and stability of the elimination methods are of primary concern to both numerical analysts and algorithm users. Validity means that division by a zero is never encountered, whereas stability guards against roundoff-error growth. To insure validity and achieve stability, pivoting is called for [12]. Full pivoting is computationally expensive requiring full two-dimensional search for the pivots. Moreover, it destroys the banded form of the matrix of coefficients. Partial pivoting strategies, though potentially less stable, are considerably less expensive. Unidirectional (row 2 Advances in Numerical Analysis or column) pivoting makes a minor change to the form of G by introducing few extraneous elements. Lam's alternating pivoting [8], which involves alternating sequences of row pivoting and column pivoting, maintains the form of G. When G is nonsingular, Lam's pivoting guarantees validity, and if followed by correspondingly alternating elimination it produces multipliers that are bounded by unity, thus enhancing stability. This approach was proposed by Varah [9] in his LGU decomposition method. It was developed afterwards into a more efficient LU version-termed here Scalar Column/Scalar Row (SCSR) Elimination-that was adopted by the COLROW solver [2].
The present approach of modular analysis shows that Lam's pivoting (with Varah's arrangement) applies to the BCBR and BCSR elimination methods, as well. It even applies to the two block tridiagonal elimination methods BTDR and BTDC, contrary to the common belief. A more robust, though more expensive strategy, Local Pivoting, is also identified. It performs full pivoting over the same segments of G (or Γ) to which Lam's pivoting is applied. Keller's mixed pivoting [10] is midway between Lam's and local pivoting.
Modular analysis also allows easy estimation of the operation counts and storage needs, revealing the method with the best performance on each account. The method having the least operation count is BCBR elimination, whereas the method requiring the least storage is BTDC elimination [11]. Both methods achieve savings of leading order importance, for large block sizes, in comparison with other methods.
Based on the previous assessment, and realizing that programming factors might affect the performance, four competing elimination methods were implemented. The COLROW algorithm, which was designed to give SCSR elimination its best performance, was modified to perform BTDC, BCBR, or BCSR elimination, instead. The four methods were applied to the same problems, and the execution times were recorded. BCSR elimination proved to be an effective modification to the COLROW algorithm.

Problem Description
Consider the almost block diagonal system of equations Gz = g whose augmented matrix of coefficients The blocks with leading characters , , , and have , , , and columns, respectively. The blocks with leading character have columns indicating as many right-hand sides. The trailing character or (and subsequently = + ) indicates the number of rows of a block or the order of a square matrix such as an identity matrix and a lower or an upper triangular matrix. Blanks indicate zero blocks.
The matrix of unknowns z is written, similarly, as where the superscript denotes the transpose.
Such a system of equations results, for example, in the finite difference solution of first order ODEs on a grid of points with conditions at one side to be marked with = 1 and conditions at the other side that is to be marked with = . Then, each column of the submatrix [ ] contains the unknowns of the th grid point, corresponding to a right-hand side.

Modular Analysis
The description of the decomposition methods for the augmented matrix of coefficients G + can be made easy and concise through the introduction of modules of G + . Two different modules are identified: The Aligned Module (A-Module) The Displaced Module (D-Module) (For convenience, we will occasionally drop the subscript and/or superscript identifying a module and its components (given later), as well as the subscript identifying its blocks.) As a rule, the dotted line defines the partitioning to left and right entities. The dashed lines define the partitioning For Γ + , which is, likewise, defined through the head-tail relation This makes it possible to construct the significant part of G + by arranging each set of modules in such a way that the tail of Γ + adds to the head of Γ + −1 , for = 0 → . Minor adjustments need only to be invoked at both ends of G + . Specifically, we define the truncated modules The head of the module Γ + is yet to be defined. It is taken to be related to the other components of Γ + by in order to allow for decompositions of Γ + having the form The generic relations = , = Ψ , + = Φ + , and + = ΨΦ + then hold, leading to + = ΨΦ + = ( −1 )( −1 + ) = ( −1 −1 ) + = −1 +1 as defined in (9).

Elimination Methods.
All elimination methods can be expressed in terms of decompositions of the stem . Only those worthy methods that allow alternating column/row pivoting and elimination are presented here. Several inflections of the blocks of G are involved and are defined in the appendix section. The sequence in which the blocks are manipulated for: decomposing the stem, processing the fins, and handling the head (evaluating the head and applying the head-tail relation to determine the tail of the succeeding module Γ +1 ), is mentioned along with the equations (from the appendix section) involved. The correctness of the decompositions may be checked by carrying out the matrix multiplications, using the equalities of the appendix section, and comparing with the un-decomposed form of the module.
The following three methods can be generated from either module. They will be given in terms of the aligned module.

Scalar Column/Scalar Row (SCSR)
Elimination. This is the method implemented by the COLROW algorithm. It performs scalar decomposition of the stem . The triangular matrices and appear explicitly. If unit diagonal, they are marked with a circumflex̂: The following sequence of manipulations applies: Stem:̂( 7), ( 16), The following sequence of manipulations applies: The following sequence of manipulations applies: .

Block-Tridiagonal Row (BTDR) Elimination.
This method can be generated from the aligned module only. It performs the identity decomposition =˘, leading to the decomposition In [10, Section 5, case (i)], scalar row elimination was used to obtain the decomposed stem˘. However,˘can, by now, be obtained by any of the nonidentity (scalar and/or block) decomposition methods given in Sections 3.1.1-3.1.3.

Block-Tridiagonal
In [11],˘was obtained by scalar column elimination. As with BTDR elimination,˘can, by now, be obtained from any of the nonidentity decompositions given in Sections 3.1.1-3.1.3.
Using SCSR elimination, the following sequence of manipulations applies: , .

Operation Counts and Storage
Needs. The modules introduced previously allow easy evaluation of the elimination methods. The operation counts are measured by the number of multiplications (mul) with the understanding that the number of additions is comparable. The storage needs are measured by the number of locations (loc) required to store arrays calculated in the forward sweep for use in the backward sweep, provided that the elements of G + are not stored but are generated when needed, as is usually done in the numerical solution of a set of ODEs, for example. Per module (i.e., per grid point), each method requires, for the manipulation of G + , as many operations as it requires to manipulate Γ + . All methods require ( 3 − )/3 (mul) for decomposing the stem, pmn (mul) for evaluating the head, and 2 2 (mul) to handle the right module . The methods differ only in the operation counts for processing the fins and , with BCBR elimination requiring the least count pmn (mul).
Per module, each method requires as many storage locations as it requires to store + ≡ [ . . . ]. All methods require pr (loc) to store . They differ in the number of locations needed for storing the 's, with BTDC elimination requiring the least number pn (loc). Note that, in SCSR and BCSR eliminations, square blocks need to be reserved for storing the triangular blockŝand/or .    and decompose a nonsingular pivotal block # and applies row pivoting to = [ # # ] in order to form and decompose a nonsingular pivotal block # . These are valid processes since is of rank m and is of rank , as can be shown following the reasoning of Keller [10,Section 5,Theorem].
To enhance stability further we introduce the Local pivoting strategy which applies full pivoting (maximum pivot strategy) to the segments and . Note that Keller's mixed pivoting [10], if interpreted as applying full pivoting to and row pivoting to , is midway between Lam's and local pivoting.
Lam's, Keller's, and local pivoting apply to all elimination methods of Section 3.1. Moreover, in any given problem, each pivoting strategy would produce the same sequence of pivots, regardless of the elimination method in use.

Implementation
The COLROW algorithm, which is based on SCSR elimination, is modified to perform BTDC, BCBR, or BCSR elimination, instead. As an illustration, the four methods are applied to three systems of equations having the augmented matrix given in (1), with = 11 and = 1. The solution procedures are repeated times, so that reasonable execution times can be recorded and compared. All calculations are carried out in double precision via Compaq Visual Fortran (version 6.6) that is optimized for speed, on a Pentium 4 CPU rated at 2 GHz with 750 MB of RAM.
Although the modified COLROW algorithms may not produce the best performances of BTDC, BCBR, and BCSR eliminations, Tables 2, 3, and 4 clearly indicate that they, in some cases (when ≫ ), outperform the COLROW algorithm that is designed to give SCSR elimination its best performance.

Conclusion
Using the novel approach of modular analysis, we have analyzed the sequential solution methods for almost block diagonal systems of equations. Two modules have been identified and have made it possible to express and assess all possible band and block elimination methods. On the 6 Advances in Numerical Analysis basis of the operation counts, storage needs, and admissibility of partial pivoting, we have determined four distinguished methods: Block Column/Block Row (BCBR) Elimination (having the least operation count), Block Tridiagonal Column (BTDC) Elimination (having the least storage need), Block Column/Scalar Row (BCSR) elimination, and Scalar Column/Scalar Row (SCSR) elimination (implemented in the well-known COLROW algorithm). Application of these methods within the COLROW algorithm shows that they outperform SCSR elimination, in cases of large top-block/bottom-block row ratio. In such cases, BCSR elimination is advocated as an effective modification to the COLROW algorithm.