Almost block diagonal linear systems of equations can be exemplified by two modules. This makes it possible to construct all sequential forms of band and/or block elimination methods. It also allows easy assessment of the methods on the basis of their operation counts, storage needs, and admissibility of partial pivoting. The outcome of the analysis and implementation is to discover new methods that outperform a well-known method, a modification of which is, therefore, advocated.
1. Introduction
Systems of equations with almost block diagonal (ABD) matrix of coefficients are frequently encountered in numerical solutions of sets of ordinary or partial differential equations. Several such situations were described by Amodio et al. [1], who also reviewed sequential and parallel solvers to ABD systems and came to the conclusion that sequential solution methods needed little further study.
Traditionally, sequential solution methods of ABD systems performed 𝕃𝕌 decompositions of the matrix of coefficients 𝔾 through either band (scalar) elimination or block tridiagonal elimination. The famous COLROW algorithm [2], which is highly regarded for its performance, was incorporated in several applications [3–7]. It utilizes Lam’s alternating column/row pivoting [8] and Varah’s correspondingly alternating column/row scalar elimination [9]. The efficient block tridiagonal methods included Keller’s Block Tridiagonal Row (BTDR) elimination method [10, Section 5, case i] and El-Mistikawy’s Block Tridiagonal Column (BTDC) elimination method [11]. Both methods could apply a suitable form of Keller’s mixed pivoting strategy [10], which is more expensive than Lam’s.
The present paper is intended to explore other variants of the 𝕃𝕌 decomposition of 𝔾. It does not follow the traditional approaches of treating the matrix of coefficients as a banded matrix or casting it into a block tridiagonal form. It, rather, adopts a new approach, modular analysis, which offers a simple and unified way of expressing and assessing solution methods for ABD systems.
The matrix of coefficients 𝔾 (or, more specifically, its significant part containing the nonzero blocks) is disassembled into an ordered set of modules. (In fact, two different sets of modules are identified.) Each module Γ is an entity that has a head and a tail. By arranging the modules in such a way that the head of a module is added to the tail of the next, the significant part of 𝔾 can be reassembled. The module exemplifies the matrix, but is much easier to analyze.
All possible methods of 𝕃𝕌 decomposition of 𝔾 could be formulated as decompositions of Γ. This led to the discovery of two new promising methods: Block Column/Block Row (BCBR) Elimination and Block Column/Scalar Row (BCSR) Elimination.
The validity and stability of the elimination methods are of primary concern to both numerical analysts and algorithm users. Validity means that division by a zero is never encountered, whereas stability guards against round-off-error growth. To insure validity and achieve stability, pivoting is called for [12]. Full pivoting is computationally expensive requiring full two-dimensional search for the pivots. Moreover, it destroys the banded form of the matrix of coefficients. Partial pivoting strategies, though potentially less stable, are considerably less expensive. Unidirectional (row or column) pivoting makes a minor change to the form of 𝔾 by introducing few extraneous elements. Lam’s alternating pivoting [8], which involves alternating sequences of row pivoting and column pivoting, maintains the form of 𝔾. When 𝔾 is nonsingular, Lam’s pivoting guarantees validity, and if followed by correspondingly alternating elimination it produces multipliers that are bounded by unity, thus enhancing stability. This approach was proposed by Varah [9] in his 𝕃𝔾~𝕌 decomposition method. It was developed afterwards into a more efficient 𝕃𝕌 version—termed here Scalar Column/Scalar Row (SCSR) Elimination—that was adopted by the COLROW solver [2].
The present approach of modular analysis shows that Lam’s pivoting (with Varah’s arrangement) applies to the BCBR and BCSR elimination methods, as well. It even applies to the two block tridiagonal elimination methods BTDR and BTDC, contrary to the common belief. A more robust, though more expensive strategy, Local Pivoting, is also identified. It performs full pivoting over the same segments of 𝔾 (or Γ) to which Lam’s pivoting is applied. Keller’s mixed pivoting [10] is midway between Lam’s and local pivoting.
Modular analysis also allows easy estimation of the operation counts and storage needs, revealing the method with the best performance on each account. The method having the least operation count is BCBR elimination, whereas the method requiring the least storage is BTDC elimination [11]. Both methods achieve savings of leading order importance, for large block sizes, in comparison with other methods.
Based on the previous assessment, and realizing that programming factors might affect the performance, four competing elimination methods were implemented. The COLROW algorithm, which was designed to give SCSR elimination its best performance, was modified to perform BTDC, BCBR, or BCSR elimination, instead. The four methods were applied to the same problems, and the execution times were recorded. BCSR elimination proved to be an effective modification to the COLROW algorithm.
2. Problem Description
Consider the almost block diagonal system of equations 𝔾z=g whose augmented matrix of coefficients 𝔾+=[𝔾⋮g] has the form
(1)
The blocks with leading characters A, B, C, and D have m, n, m, and n columns, respectively. The blocks with leading character g have r columns indicating as many right-hand sides. The trailing character m or n (and subsequently p=m+n) indicates the number of rows of a block or the order of a square matrix such as an identity matrix I and a lower L or an upper U triangular matrix. Blanks indicate zero blocks.
The matrix of unknowns z is written, similarly, as
(2)z=[zm1tzn1tzm2tzn2t⋯zmjtznjt⋯zmJtznJt]t,
where the superscript t denotes the transpose.
Such a system of equations results, for example, in the finite difference solution of p first order ODEs on a grid of J points with m conditions at one side to be marked with j=1 and n conditions at the other side that is to be marked with j=J. Then, each column of the submatrix [zmjtznjt]t contains the p unknowns of the jth grid point, corresponding to a right-hand side.
3. Modular Analysis
The description of the decomposition methods for the augmented matrix of coefficients 𝔾+ can be made easy and concise through the introduction of modules of 𝔾+. Two different modules are identified:
The Aligned Module (A-Module)
(3)
The Displaced Module (D-Module)
(4)
(For convenience, we will occasionally drop the subscript and/or superscript identifying a module and its components (given later), as well as the subscript identifying its blocks.)
As a rule, the dotted line defines the partitioning to left and right entities. The dashed lines define the partitioning
(5)
to the following components: the stem σj, the head ηj+≡[ηj⋮θj], and the fins ψj and ϕj+≡[ϕj⋮ρj].
Each module has a tail τj+≡[τj⋮λj]. For Γj+A, τj+A=[Cmj#Dmj#⋮gmj#] which is defined through the head-tail relation
(6)
For which is, likewise, defined through the head-tail relation
(7)
This makes it possible to construct the significant part of 𝔾+ by arranging each set of modules in such a way that the tail of Γj+ adds to the head of Γj-1+, for j=0→J. Minor adjustments need only to be invoked at both ends of 𝔾+. Specifically, we define the truncated modules
(8)
The head of the module Γ+ is yet to be defined. It is taken to be related to the other components of Γ+ by
(9)η+=ψσ-1ϕ+,
in order to allow for decompositions of Γ+ having the form
(10)
The generic relations σ=MN, ψ=ΨN, ϕ+=MΦ+, and η+=ΨΦ+ then hold, leading to η+=ΨΦ+=(ψN-1)(M-1ϕ+)=ψ(N-1M-1)ϕ+=ψσ-1ϕ+1 as defined in (9).
3.1. Elimination Methods
All elimination methods can be expressed in terms of decompositions of the stem σj. Only those worthy methods that allow alternating column/row pivoting and elimination are presented here. Several inflections of the blocks of 𝔾 are involved and are defined in the appendix section. The sequence in which the blocks are manipulated for: decomposing the stem, processing the fins, and handling the head (evaluating the head and applying the head-tail relation to determine the tail of the succeeding module Γj+1), is mentioned along with the equations (from the appendix section) involved. The correctness of the decompositions may be checked by carrying out the matrix multiplications, using the equalities of the appendix section, and comparing with the un-decomposed form of the module.
The following three methods can be generated from either module. They will be given in terms of the aligned module.
This is the method implemented by the COLROW algorithm. It performs scalar decomposition of the stem σ. The triangular matrices L and U appear explicitly. If unit diagonal, they are marked with a circumflex ^:
(11)
This method can be generated from the aligned module only. It performs the identity decomposition σ=Ipσ˘, leading to the decomposition
(14)
In [10, Section 5, case (i)], scalar row elimination was used to obtain the decomposed stem σ˘. However, σ˘ can, by now, be obtained by any of the nonidentity (scalar and/or block) decomposition methods given in Sections 3.1.1–3.1.3.
Using SCSR elimination, the following sequence of manipulations applies:
This method can be generated from the displaced module only. It performs the identity decomposition σ=σ˘Ip, leading to the decomposition
(15)
In [11], σ˘ was obtained by scalar column elimination. As with BTDR elimination, σ˘ can, by now, be obtained from any of the nonidentity decompositions given in Sections 3.1.1–3.1.3.
Using SCSR elimination, the following sequence of manipulations applies:
The procedure for solving the matrix equation 𝔾+z=g, which can be described in terms of manipulation of the augmented matrix 𝔾+≡[𝔾⋮g], can, similarly, be described in terms of manipulation of the augmented module Γj+≡[Γj⋮γj]. The manipulation of 𝔾+ applies a forward sweep which corresponds to the decomposition 𝔾+=𝕃𝕌+≡𝕃[𝕌⋮Z] that is followed by a backward sweep which corresponds to the decomposition 𝕌+=𝕌[I⋮z]. Similarly, the manipulation of Γj+ applies a forward sweep involving two steps. The first step performs the decomposition Γj+=μjνj+. The second step evaluates the head ηj+=ΨjΦj+ then applies the head-tail relation to determine the tail of Γj+1+. In a backward sweep, two steps are applied to νj+≡[NjΦj⋮Ζj] leading to the solution module zj(zjA=[zmjtznjt]t,zjD=[znjtzmj+1t]t). With zj+1 known, the first step uses in the back substitution relation to contract νj+ to Nj+≡[Nj⋮ζj]. The second step solves Njzj=ζj for zj which is equivalent to the decomposition Nj+=Nj[Ip⋮zj].
3.3. Operation Counts and Storage Needs
The modules introduced previously allow easy evaluation of the elimination methods. The operation counts are measured by the number of multiplications (mul) with the understanding that the number of additions is comparable. The storage needs are measured by the number of locations (loc) required to store arrays calculated in the forward sweep for use in the backward sweep, provided that the elements of 𝔾+ are not stored but are generated when needed, as is usually done in the numerical solution of a set of ODEs, for example.
Per module (i.e., per grid point), each method requires, for the manipulation of 𝔾+, as many operations as it requires to manipulate Γ+. All methods require (p3-p)/3 (mul) for decomposing the stem, pmn (mul) for evaluating the head, and 2p2r (mul) to handle the right module γ. The methods differ only in the operation counts for processing the fins ψ and ϕ, with BCBR elimination requiring the least count pmn (mul).
Per module, each method requires as many storage locations as it requires to store ν+≡[ν⋮Z]. All methods require pr (loc) to store Z. They differ in the number of locations needed for storing the ν’s, with BTDC elimination requiring the least number pn (loc). Note that, in SCSR and BCSR eliminations, square blocks need to be reserved for storing the triangular blocks U^m and/or Un.
Table 1 contains these information, allowing for clear comparison among the methods. For example, when p≫1, BCBR elimination achieves savings in operations that are of leading order significance ~p3/8 and ~p3/2, respectively, in the two distinguished limits m~n~p/2 and (m,n)~(p,1), as compared to SCSR elimination.
Operation counts and storage needs.
Method
Operation counts (mul)
Storage needs (loc)
2p2r+(p3-p)/3+2pmn+
pr+pn+
BCBR
0
pn
SCSR
(m3+n3-m2-n2)/2
pn+m2
BCSR
(n3-n2)/2
pn
BTDC
pn2
0
3.4. Pivoting Strategies
Lam’s alternating pivoting [8] applies column pivoting to τA=[Cm#Dm#] in order to form and decompose a nonsingular pivotal block Cm# and applies row pivoting to τD=[Bn#tBm#t]t in order to form and decompose a nonsingular pivotal block Bn#. These are valid processes since τA is of rank m and τD is of rank n, as can be shown following the reasoning of Keller [10, Section 5, Theorem].
To enhance stability further we introduce the Local pivoting strategy which applies full pivoting (maximum pivot strategy) to the segments τA and τD. Note that Keller’s mixed pivoting [10], if interpreted as applying full pivoting to τA and row pivoting to τD, is midway between Lam’s and local pivoting.
Lam’s, Keller’s, and local pivoting apply to all elimination methods of Section 3.1. Moreover, in any given problem, each pivoting strategy would produce the same sequence of pivots, regardless of the elimination method in use.
4. Implementation
The COLROW algorithm, which is based on SCSR elimination, is modified to perform BTDC, BCBR, or BCSR elimination, instead. As an illustration, the four methods are applied to three systems of equations having the augmented matrix given in (1), with J=11 and r=1. The solution procedures are repeated k times, so that reasonable execution times can be recorded and compared. All calculations are carried out in double precision via Compaq Visual Fortran (version 6.6) that is optimized for speed, on a Pentium 4 CPU rated at 2 GHz with 750 MB of RAM.
The first system has p=11 and k=106. Different combinations of m/n=10/1, 9/2, 8/3, 7/4, and 6/5 are considered. The execution times, without pivoting, are given in Table 2. All entries include ≈22 seconds that are required to read, write, and run empty Fortran-Do-Loops. Pivoting requires additional ≈20 seconds in all methods.
Execution times in seconds for the first system.
m/na
SCSR
BTDC
BCBR
BCSR
10/1
109
96
93
91
9/2
113
104
106
99
8/3
118
116
115
110
7/4
120
126
122
117
6/5
123
133
133
125
^{
a}m+n=11.
The second (third) system has p=21 (51) and k=105 (10^{4}). The execution times, with Lam’s pivoting, are given in Table 3 (4), for different combinations of m/n.
Execution times in seconds for the second system.
m/na
SCSR
BTDC
BCBR
BCSR
20/1
67.9
52.0
51.8
51.4
18/3
70.6
59.9
60.0
55.0
16/5
72.0
68.3
73.8
66.3
14/7
73.0
74.2
80.8
73.8
12/9
73.1
78.3
81.4
74.8
11/10
73.6
81.7
82.3
75.9
^{
a}m+n=21.
Although the modified COLROW algorithms may not produce the best performances of BTDC, BCBR, and BCSR eliminations, Tables 2, 3, and 4 clearly indicate that they, in some cases (when m≫n), outperform the COLROW algorithm that is designed to give SCSR elimination its best performance.
Execution times in seconds for the third system.
m/na
SCSR
BTDC
BCBR
BCSR
50/1
60.51
38.05
39.19
38.04
46/5
62.71
48.30
57.55
49.15
41/10
65.56
58.97
76.25
61.57
36/15
68.12
65.86
87.53
68.35
31/20
69.09
76.38
110.61
85.24
26/25
69.86
83.95
115.95
94.00
^{
a}m+n=51.
5. Conclusion
Using the novel approach of modular analysis, we have analyzed the sequential solution methods for almost block diagonal systems of equations. Two modules have been identified and have made it possible to express and assess all possible band and block elimination methods. On the basis of the operation counts, storage needs, and admissibility of partial pivoting, we have determined four distinguished methods: Block Column/Block Row (BCBR) Elimination (having the least operation count), Block Tridiagonal Column (BTDC) Elimination (having the least storage need), Block Column/Scalar Row (BCSR) elimination, and Scalar Column/Scalar Row (SCSR) elimination (implemented in the well-known COLROW algorithm). Application of these methods within the COLROW algorithm shows that they outperform SCSR elimination, in cases of large top-block/bottom-block row ratio. In such cases, BCSR elimination is advocated as an effective modification to the COLROW algorithm.
Appendix
In Section 3, two modules, ΓA and ΓD, of the matrix of coefficients 𝔾 are introduced and decomposed to generate the elimination methods. The process involves inflections of the blocks of 𝔾, which proceed for a block E, say, according to the following scheme:
(A.1)E→E#→ifpivotalE˘#↓↓E′→E′′→ifpivotalE*.
The following equalities are to be used to determine the blocks with underscored leading character. The number of multiplications involved is given between braces:
E′-Blocks
(A.8)A_m′U^m=Am{m2-m2m},(A.9)A_n′U^m=An{m2-m2n},(A.10)L^nC_n′=Cn{n2-n2m}(A.11)L^nD_n′=Dn{n2-n2n},E′′ and E*-Blocks
(A.12)A_m′′=Am′-Bm*An′{m2n},(A.13)A_m*Lm=Am′′{m2+m2m}(A.14)B_m′′Un=Bm#{n2+n2m},(A.15)B_m*L^n=Bm′′{n2-n2m},(A.16)LmD_m′′=Dm#{m2+m2n},(A.17)U^mD_m*=Dm′′{m2-m2n}(A.18)D_n′′=Dn′-Cn′Dm*{mn2},(A.19)UnD_n*=Dn′′{n2-n2n}.
AmodioP.CashJ. R.RoussosG.WrightR. W.FairweatherG.GladwellI.KrautG. L.PaprzyckiM.Almost block diagonal linear systems: sequential and parallel solution techniques, and applicationsDíazJ. C.FairweatherG.KeastP.FORTRAN packages for solving certain almost block diagonal linear systems by modified alternate row and column eliminationCashJ. R.WrightM. H.A deferred correction method for nonlinear two-point boundary value problems: implementation and numerical evaluationCashJ. R.MooreG.WrightR. W.An automatic continuation strategy for the solution of singularly perturbed linear two-point boundary value problemsEnrightW. H.MuirP. H.Runge-Kutta software with defect control for boundary value ODEsKeastP.MuirP. H.Algorithm 688: EPDCOL: a more efficient PDECOL codeWrightR. W.CashJ. R.MooreG.Mesh selection for stiff two-point boundary value problemsLamD. C.VarahJ. M.Alternate row and column elimination for solving certain linear systemsKellerH. B.Accurate difference methods for nonlinear two-point boundary value problemsEl-MistikawyT. M. A.Solution of Keller's box equations for direct and inverse boundary-layer problemsDuffI. S.ErismanA. M.ReidJ. K.