Mathematical Problems in Engineering
Volume 2016 (2016), Article ID 9065438, 16 pages
http://dx.doi.org/10.1155/2016/9065438

Research Article

Computing the Pseudoinverse of Specific Toeplitz Matrices Using Rank-One Updates

Predrag S. Stanimirović,¹ Vasilios N. Katsikis,² and Igor Stojanović³

¹University of Niš, Faculty of Sciences and Mathematics, Višegradska 33, 18000 Niš, Serbia
²Department of Economics, Division of Mathematics and Informatics, National and Kapodistrian University of Athens, Sofokleous 1 Street, 10559 Athens, Greece
³Faculty of Computer Science, Goce Delčev University, Goce Delčev 89, 2000 Štip, Macedonia

Received 29 February 2016; Accepted 3 July 2016

Academic Editor: Masoud Hajarian

Copyright © 2016 Predrag S. Stanimirović et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Application of the pure rank-one update algorithm as well as a combination of rank-one updates and the Sherman-Morrison formula in computing the Moore-Penrose inverse of the particular Toeplitz matrix is investigated in the present paper. Such Toeplitz matrices appear in the image restoration process and in many scientific areas that use the convolution. Four different approaches are developed, implemented, and tested on a number of numerical experiments.

1. Introduction

Let and denote the set of all complex matrices and the set of all complex matrices of , respectively. The identity matrix of an appropriate order is denoted by . The conjugate transpose, the range, the rank, and the null space of are denoted by , , , and , respectively.

Representation and computation of various generalized inverses are closely related to the following Penrose equations:

The set of all matrices obeying the conditions contained in a subset is denoted by . Any matrix from is called -inverse of and is denoted by . By we denote the set of all -inverses of with . For any matrix there exists a single element in the set , called the Moore-Penrose inverse of and denoted by .

A rank-one modification of a matrix is the matrix , which is created from and two vectors and . The Sherman-Morrison formula (S-M shortly) gives the basic relationship between the inverses and (for more details see, e.g., [1]):

The identity (2) provides a numerically efficient way to compute the inverse of the rank-one update . The S-M formula is important in many different fields of numerical computation; see for example [2–7].

On the other hand, Toeplitz matrices arise in a number of various theoretical investigations and applications. A number of iterative processes for finding generalized inverses of an arbitrary Toeplitz matrix by modifying Newton’s method have been developed so far. The main results were stated in [8–13]. Adaptations of the iterative processes to the Toeplitz structure are based on the usage of the displacement operator as well as the concept of displacement representation and -displacement rank of matrices.

A variety of methods for computing the Moore-Penrose inverse of a rank-one modified matrix have been developed so far. Main results were derived in [14–18]. Relationships between various generalized inverses of an arbitrary matrix and corresponding generalized inverses of its rank-one modifications were investigated in [19]. The leading idea in [15, 16] was successive computation of the symmetric rank-one (SR1) updates of a given matrix , where denotes the th row of and . The authors of the paper [15] introduced a computational procedure for the Moore-Penrose inverse of a symmetric rank-one perturbed matrix. Using this method, the authors of [16] proposed a finite method for computing the minimum-norm least-squares solution of the linear system .

The results derived in [20] reveal that both the SR1 updates techniques and the S-M recursive rule are useful tools in the computation of various matrix products involving the Moore-Penrose inverse of certain symmetric matrices. Particularly, the algorithms introduced in [20] are numerically efficient in computation of and inverses.

In the present paper, we investigate the possibilities to apply the SR1 update procedure and the S-M formula in the computation of the Moore-Penrose inverse of specific Toeplitz matrices that appear in the image restoration process. Our main motivation arises from the convenience to apply the SR1 update and the S-M procedure in removing the blur which is always present in digital images. Firstly, both the SR1 update and the S-M formula are based on the usage of columns (or rows) of the input matrix. On the other hand, the matrices which appear in the mathematical model of blur in computer-generated images possess very specific structure which can be used to accelerate SR1 and S-M procedures. Namely, entries in Toeplitz matrices are constant along main diagonal parallels and, moreover, possess a significant proportion of zero elements.

The paper is organized as follows. Some basic notations and necessary facts are restated in Section 2. Also, some additional motivation is presented in the same section. Usage of the pure SR1 update algorithm, proposed in [15], in the computation of the Moore-Penrose inverse of a kind of Toeplitz matrices is considered in Section 3. A hybrid combination of the SR1 and the S-M recursive rules is defined in Section 4. An improvement of the SR1 procedure, which is derived on the basis of the specific structure of the underlying Toeplitz matrix, is presented in the same section. An application of introduced methods in image restoration is presented in Section 5.

2. Preliminaries and Motivation

Toeplitz matrices or diagonally constant matrices are matrices having constant diagonal entries. Toeplitz matrices which are applicable in the image restoration process contain nonzero main diagonal parallels above the main diagonal, where defines the blurring process. In what follows, let us consider the Toeplitz matrix of such form:

The assumption is active.

To clarify notation, Toeplitz matrices of the general form (3) will be denoted shortly by

We investigate the use of the SR1 update method, as described in [15, Algorithm  2], during the numerical computation of the Moore-Penrose inverse of Toeplitz matrices satisfying the pattern. Also, we examine different improvements of the original method. The improvements are based on appropriate adaptations of the SR1 method and the S-M formula to the characteristic structure of underlying matrices of type . The method of SR1 updates is based on the expression which computes the Moore-Penrose inverse of the first columns of the initial matrix using the Moore-Penrose inverse of its first columns. In detail, the SR1 method from [15] starts from the well-known representation of the Moore-Penrose. If the th row of is denoted by , then

Chen and Ji in [15] defined the matrix sequences and , as

Clearly, is the rank-one modification of and

Recall that the Moore-Penrose inverse of a general rank-one modified matrix , where is an arbitrary matrix and ,   are arbitrary vectors, is obtained in [14, Theorem  3.1.3]. The general theorem from [14] suggests six different cases that one has to follow in order to establish a relation between and . In [15], the authors proved that the real number , corresponding to the term in (2), satisfies . Later using this result in conjunction with the fact that is a positive semidefinite matrix, the six cases of Theorem  3.1.3 from [14] can be reduced to the two-case problem. This reduction simplifies the SR1 updates formulas.

Let us denote the first columns of by

Also, the last columns of are denoted by . Then the matrix is given in the block form

where the square block

is the nonsingular band Toeplitz matrix and

collects the last columns of .

An application of Greville’s partitioning method from [21] and the block partitioning method (BP method, shortly) from [22] in computing the Moore-Penrose inverse of the matrix is presented in [23]. According to Algorithms  1 and 2 and Lemma  3 from [23], it is clear that the specific structure of matrix enables computation simply by computing the inverse of the nonsingular block .

In this paper, we investigate some alternative methods for computing using the SR1 updates and the S-M formula. Our intention is to decrease computational complexity as much as possible using a specific structure of Toeplitz matrices of the general form .

3. Computing the Pseudoinverse of a Toeplitz Matrix by Rank-One Updates

Our first attempt consists in applying the unmodified SR1 method from [15] in order to compute the pseudoinverse of the matrix that belongs to the class . Obtained results are compared with corresponding results derived by applying Algorithm  2 from [23] (the BP method). The results of this comparison are presented in Table 1, where , , and are the parameters which define the Gaussian blur modeled by the Toeplitz matrix . In the rest of the paper, it is assumed that the matrix is of the order , where and represents the width of the blurring function.

Table 1: Comparison of the rank-one updates method and the BP method.

Method
Sizes , ℓ,  
CPU time

BP
50, 15, 10
0.0025
2.0011e − 15
2.0385e − 14
3.773e − 15
1.6463e − 14

SR1
50, 15, 10
0.0490
2.2762e − 14
5.5984e − 13
1.2594e − 14
4.0623e − 13

BP
50, 20, 10
0.0028
6.9626e − 16
2.8774e − 14
3.4226e − 15
2.7706e − 14

SR1
50, 20, 10
0.0364
6.16e − 14
2.0705e − 12
1.3341e − 14
1.3718e − 12

BP
50, 15, 500
0.0029
8.0717e − 16
1.6354e − 14
3.7149e − 15
7.0839e − 15

SR1
50, 15, 500
0.0501
1.9666e − 14
4.8611e − 13
8.979e − 15
3.5898e − 13

BP
50, 20, 500
0.0016
6.8218e − 16
2.1407e − 14
3.2663e − 15
6.0107e − 15

SR1
50, 20, 500
0.0362
2.0844e − 14
5.3265e − 13
7.9594e − 15
4.1980e − 13

BP
250, 15, 500
0.0059
3.4724e − 15
2.3693e − 13
1.5467e − 14
7.4894e − 14

SR1
250, 15, 500
23.9762
7.81e − 12
3.6574e − 11
1.2707e − 13
2.2523e − 11

BP
400, 20, 500
0.0135
4.731e − 15
3.7769e − 13
2.6494e − 14
9.909e − 14

SR1
400, 20, 500
320.0578
3.8409e − 11
1.4246e − 10
2.2311e − 13
8.0923e − 11

If an approximation of is denoted by , then the residual norms are denoted by

For the sake of simplicity, let us denote Algorithm  2 from [15] by SR1 and Algorithm  2 from [23] by BP.

From Table 1, it is observable that the SR1 method is marginally efficient in terms of accuracy, but it is time consuming, especially for larger dimensions. There is a theoretical explanation for these results. The number of multiplications and divisions included in the SR1 algorithm is about in the case and in the case . Therefore, as it is stated in [15], the rank-one updates algorithm works efficiently in cases where it holds or . In our case, , where represents the width of the blurring process. Since, in the general case, is a relatively small integer, the conditions and are satisfied. This fact causes relative inefficiency of the SR1 method.

Our second attempt is to apply Algorithm SR1 to the matrix , from (10), in order to generate its inverse. Then the Moore-Penrose inverse of comes from the block partitioning method (called BP), as it was described in [23]. This approach is shortly denoted by HSR1 + BP.

It is observable from Table 2 that the method HSR1 + BP is marginally efficient in terms of accuracy, but it is time consuming; see, for example, the case .

Table 2: Comparison of the combination of AlgSR1 and AlgBP methods and the block partitioning method.

Method
Sizes , ℓ,  
CPU time

BP
50, 15, 10
0.0037
2.0011e − 15
2.0385e − 14
3.773e − 15
1.6463e − 14

HSR1 + BP
50, 15, 10
0.0592
2.6951e − 14
2.9305e − 13
1.3171e − 13
4.0769e − 14

BP
50, 20, 10
3.4243e − 04
6.9626e − 16
2.8774e − 14
3.4226e − 15
2.7706e − 14

HSR1 + BP
50, 20, 10
0.0305
5.3084e − 14
4.1354e − 13
3.1072e − 13
9.9898e − 14

BP
50, 15, 500
0.0019
8.0717e − 16
1.6354e − 14
3.7149e − 15
7.0839e − 15

HSR1 + BP
50, 15, 500
0.0411
2.1453e − 14
2.2518e − 13
7.0401e − 14
3.2624e − 14

BP
50, 20, 500
3.2256e − 04
6.8218e − 16
2.1407e − 14
3.2663e − 15
6.0107e − 15

HSR1 + BP
50, 20, 500
0.0288
6.029e − 15
1.3385e − 13
2.6565e − 14
1.3927e − 14

BP
250, 15, 500
0.0048
3.4724e − 15
2.3693e − 13
1.5467e − 14
7.4894e − 14

HSR1 + BP
250, 15, 500
21.9978
1.8868e − 12
3.4807e − 10
1.154e − 11
4.0825e − 12

BP
400, 20, 500
0.0132
4.731e − 15
3.7769e − 13
2.6494e − 14
9.909e − 14

HSR1 + BP
400, 20, 500
306.1949
7.3251e − 12
2.7048e − 09
6.4634e − 11
2.0001e − 11

From the previous analysis of data included in Tables 1 and 2, we conclude that both algorithms SR1 and HSR1 + BP are not efficient approaches for computing with respect to the BP method in terms of computational speed, especially for large matrices. But both approaches show marginal efficiency in terms of the accuracy.

4. A Combination of SR1 Updates and S-M Formula

Two additional algorithms are defined in the current section. The first one uses the SR1 updates to compute the matrix in the first step and the S-M formula to compute in the subsequent step. The second algorithm replaces the usage of the SR1 updates from the first step by a direct construction of the matrix . The matrix can be generated efficiently in view of the specific structure of the input matrix .

4.1. SR1 Updates in Conjunction with the S-M Formula

As usual, the th column of is denoted by . Then, since is of full row rank, it is advantageous to define a proper SR1 recursive computational method for generating the Moore-Penrose inverse of on the basis of the relation

For that purpose, it is necessary to define the matrix sequence

Lemma 1 reveals some of basic properties of the sequence . Following the notation from [20], the notation means that a column is linearly dependent on the previous columns and indicates that is linearly independent of .

Lemma 1. The following statements are valid:

(i), for each .(ii), for each .(iii), for each , and , for each .(iv)The following concluding relation is valid:

Proof. (i) This part of the proof is implied by the fact that the columns of are linearly independent.
(ii) According to the part (i), immediately follows , for each . Later, it is easy to verify

which implies

(iii) This part of the proof can be verified by using (i) and (ii), together with the fact that are all matrices, for each .
(iv) According to part (iii), matrices , , are invertible. Since

is invertible, then it holds

which completes the proof.

Remark 2. According to Lemma 1 the matrices are of the order and it holds , for each , while , for each . Since , the invertibility of is guaranteed, so that we are able to apply the S-M formula which relates with , for each .
According to Lemma 1, matrices , , are invertible. Therefore, it is possible to define the following matrix sequence:

Identity (15) immediately implies

Lemma 3. It holds , for each .

Proof. The proof of lemma follows from the fact that is regular, for each .

Now, we turn our attention to (20). Since the matrices , are regular, it is possible to find a relation between and by applying the Sherman-Morrison formula (2). This relation is given in Proposition 4.

Proposition 4. Let the matrix be defined as in (3). Assume that the matrix sequence is defined in (14). The inverse of is equal to

for each .

In order to present the finite recursive algorithm for computing the Moore-Penrose inverse of with the help of the actual SR1 update formulas (20) and (21), we first define . For each , it is necessary to compute . This requires calculated values of the vectors , and the matrix .

Theorem 5. Let the matrix be defined as in (3), the matrix sequence defined in (14), and the matrix sequence defined in (20). Then the following recursive relations are true:

Proof. Note that from it follows that . Based on the use of (21), the recursive sequences are defined for as in the following:

Then the statement (22) can be easily verified using (24).
Also, matrices , are defined as

So, (23) holds.

According to Theorem 5, we present Algorithm 1 for computing the Moore-Penrose inverse of the Toeplitz matrix . The algorithm is based on the SR1 modifications to compute and subsequently the S-M formula to compute . For this purpose, we use the abbreviation HSR1 + S-M to denote Algorithm 1.

Require: Start with and . Let be the th column of , for , .

(1) (SR1 Step) For , compute .

(2) Set for all and .

(3) (S-M Step) Compute and by using (22) and (23), respectively for all .

(4) Output .

Algorithm 1: Computing the Moore-Penrose inverse of the Toeplitz matrix .

A Matlab implementation of Algorithm 1 is placed in Appendix.

4.2. An Improvement of the Hybrid Combination in Algorithm 1

By taking into account the fact that the Toeplitz matrix has a specific structure, the matrix can be generated in an efficient way. In this way, an improvement of step  (1) of Algorithm 1 is introduced. The effectiveness of this method will be justified in the numerical experiments presented in Section 4.3.

According to the proper structure of the matrix and step  (1) of Algorithm 1, the general structure of the matrix is defined as

Let be one-dimensional vector; then the symmetric Toeplitz matrix generated by is denoted by :

Also, for two appropriate vectors , we denote by the (symmetric) Hankel matrix whose first column is and whose last row is .

Theorem 6. The matrix can be expressed as the difference of a specific banded symmetric Toeplitz matrix and a partitioned matrix of the form

where the block is defined by , for an appropriate Hankel matrix .

Proof. Let and be vectors defined by

respectively. Then we define the following matrices:

A simple verification shows that .

Next we present an example, in low dimensions, in order to illustrate Theorem 6.

Example 7. Consider the Toeplitz matrix , defined in (3) for ,  , and :

According to (14),

We exploit the matrix

Let and ; then

Finally, it is necessary to construct the matrix

It is easy to see that

According to Theorem 6 we present the following improved version of Algorithm 1, called Algorithm 2. A Matlab implementation of Algorithm 2 is placed in Appendix. It is appropriate to use the term IHSR1 to denote the improvement of the HSR1 step of Algorithm 1 and IHSR1 + S-M to denote Algorithm 2.

Require: Start with , . Let be the th column of , for .

(1) (Improved HSR1 Step) Define and as in Theorem 6. Then, set .

(2) Set for all and .

(3) (S-M Step) Compute and by using (22) and (23), respectively for all .

(4) Output .

Algorithm 2: Computing the Moore-Penrose inverse of the Toeplitz matrix .

Note that proposed Algorithms 1 and 2 are not using directly, at any step, the Sherman-Morrison formula to compute the inverse of some matrix. In Section 4.4, we provide more information regarding this situation.

4.3. Numerical Experiments

In this subsection we analyze numerical data arising during the computation of the Moore-Penrose inverse of the Toeplitz matrix by applying a Matlab implementation of Algorithms 1 and 2. In order to test the time efficiency as well as the accuracy of considered methods, we enrich our collection of matrices used in Tables 1 and 2 with larger matrices. In order to complete our numerical study, we also compare the results derived by applying Algorithm  2 from [23] (Algorithm BP). A comparison of Algorithms 1 and 2 and BP is presented in Table 3.

Table 3: Comparison of Algorithm 1 (HSR1 + S-M), Algorithm 2 (IHSR1 + S-M), and BP.

Method
Sizes  , ℓ,
CPU time

HSR1 + S-M
50, 15, 10
0.0054
1.6449e − 14
2.5845e − 12
3.5344e − 13
1.9362e − 14

IHSR1 + S-M
50, 15, 10
0.0035
2.0486e − 14
2.7318e − 12
5.4426e − 13
2.3222e − 14

BP
50, 15, 10
0.0018
2.0011e − 15
2.0385e − 14
3.773e − 15
1.6463e − 14

HSR1 + S-M
50, 20, 10
0.0040
1.8748e − 14
8.8674e − 13
2.2185e − 13
1.5379e − 14

IHSR1 + S-M
50, 20, 10
0.0032
9.4733e − 15
8.9539e − 13
1.9076e − 13
1.4775e − 14

BP
50, 20, 10
0.0027
9.2684e − 16
1.6646e − 14
3.2679e − 15
5.8648e − 15

HSR1 + S-M
50, 20, 500
0.0053
1.2862e − 14
9.6327e − 13
1.2288e − 13
1.4447e − 14

IHSR1 + S-M
50, 20, 500
0.0044
8.6971e − 15
1.5068e − 12
2.2331e − 13
1.5216e − 14

BP
50, 20, 500
0.0022
6.8218e − 16
2.1407e − 14
3.2663e − 15
6.0107e − 15

HSR1 + S-M
250, 15, 500
0.0482
2.5535e − 13
3.7054e − 11
4.7471e − 12
2.5608e − 13

IHSR1 + S-M
250, 15, 500
0.0195
2.209e − 13
3.7065e − 11
5.5401e − 12
2.4239e − 13

BP
250, 15, 500
0.0032
3.4724e − 15
2.3693e − 13
1.5467e − 14
7.4894e − 14

HSR1 + S-M
400, 20, 500
0.3234
4.5138e − 13
6.0769e − 10
1.4669e − 11
5.5243e − 13

IHSR1 + S-M
400, 20, 500
0.1466
5.881e − 13
6.0878e − 10
1.5866e − 11
5.3143e − 13

BP
400, 20, 500
0.0116
6.5483e − 15
6.052e − 13
3.0382e − 14
1.3117e − 13

HSR1 + S-M
1200, 20, 500
11.0062
3.0653e − 12
1.7107e − 08
1.4049e − 10
3.2612e − 12

IHSR1 + S-M
1200, 20, 500
2.3018
3.5064e − 12
1.711e − 08
1.3136e − 10
3.1795e − 12

BP
1200, 20, 500
0.1049
1.2805e − 14
3.179e − 12
8.9833e − 14
2.3321e − 13

HSR1 + S-M
1500, 20, 500
21.7539
4.3812e − 12
3.2457e − 08
2.2848e − 10
4.2712e − 12

IHSR1 + S-M
1500, 20, 500
3.7358
4.2345e − 12
3.2456e − 08
2.0156e − 10
4.2919e − 12

BP
1500, 20, 500
0.1886
1.3786e − 14
4.0422e − 12
1.1081e − 13
3.0133e − 13

HSR1 + S-M
2000, 50, 500
74.5230
5.5113e − 12
4.6497e − 08
5.1185e − 10
6.7183e − 12

IHSR1 + S-M
2000, 50, 500
30.8622
5.3682e − 12
4.6504e − 08
5.4329e − 10
6.6102e − 12

BP
2000, 50, 500
0.4038
1.8118e − 14
6.5462e − 12
1.408e − 13
3.6715e − 13

From Table 3, it is observable that the level of improvement in CPU time that can be achieved by using Algorithm 2 instead of Algorithm 1 is significant. Also, it is clear that the accuracy of both algorithms is almost the same. Clearly, the BP method requires minimal CPU time.

4.4. Rounding Error Analysis

The starting motivation of this subsection is the following basic problem: approximation error grows with repeated use of the S-M formula. As a consequence, computations which involve the S-M rule become unstable. For this purpose, it is interesting to investigate the error curves corresponding to the four Penrose equations during the iterations included in step  (3) of Algorithm 2. Nevertheless, it is important to note that the proposed Algorithm 2 does not use in each step the S-M rule. Of course, Theorem 5 is in the heart of Algorithm 2 but also it is clear that the poor stability of the S-M formula has no serious influence on the stability of proposed algorithms.

So, in this subsection, we shall present error estimations regarding Algorithm 2. That is, we record and investigate the residual matrix norms , and , corresponding to the four Penrose equations during the execution of step  (3) of Algorithm 2. The results are presented in Figures 1, 2, and 3 for the cases: (a) , , and , (b) , , and , and (c) , , respectively. The choice of these matrices from Table 3 is random.

Figure 1: Errors recorded for , , and while the loop of step (3) is iterating.

Figure 2: Errors recorded for , , and while the loop of step (3) is iterating.

Figure 3: Errors recorded for , , and while the loop of step (3) is iterating.

The purple line in each of Figures 1–3 corresponds to the values of the matrix norm , . Since and the matrix is symmetric, the matrix is symmetric. For this reason, the graph corresponding to these values is almost a constant line.

The stable convergence is observable from each of these figures. Also, it is possible to notice very fast convergence in the terminal phase of the convergence as well as a relatively slower convergence in the middle of the recurrent process. This fact is understandable if one takes into account that it is necessary to take into consideration all the columns of the input matrix in order to get the complete information on the pseudoinverse.

5. Application in Image Restoration

Several image restoration examples are presented in this subsection. Experiments are done using Matlab programming package on an Intel(R) Core(TM) i5-2540M CPU @ 2.6 GHz 64-bit system with 8 GB RAM memory.

Numerical results corresponding to the Moore-Penrose inverse are derived using Algorithm 2. Results are derived using and and different values of . Suppose that the matrix corresponds to the original image and is the matrix corresponding to the degraded image.

The model of the nonuniform blurring is the same as in [23] and assumes that the blurring of columns in the image is independent with respect to the blurring of its rows. Therefore, the relations between the original and the blurred image are expressed by the matrix equation

where , , is length of the vertical blurring, and is length of the horizontal blurring (in pixels). Following the approach used in [23], the Moore-Penrose inverse approach is used to restore the blurred image , which produces the next approximation of :

The algorithm for computing the Moore-Penrose inverse of the blurring matrix used in [24, 25] is based on the method for computing the Moore-Penrose inverse of a full rank matrix, introduced in [26, 27]. In our case, we use Algorithm 2.

The first approach is based on the usage of the pure SR1 update algorithm, proposed in [15]. The second approach starts with the SR1 update algorithm and, after its completion, continues with the BP method. The third approach is a hybrid combination which comprises the SR1 updates in the first step and the S-M formula in the second phase (Algorithm 1). The fourth approach is based on the improvement of the SR1 update step method of Algorithm 1, which leads to Algorithm 2 with the best performances. The improvement is achieved using the specific structure of the underlying Toeplitz matrix.

The most popular algorithm for image restoration based on the matrix pseudoinverse is developed using the singular value decomposition (SVD) [28]. But, the SVD is sensitive to singular values very close to zero. For this purpose, it is common approach to leave out small singular values, corresponding to high-frequency components. The resulting method is known as the truncated SVD (or TSVD shortly) [28]. On the other hand, the methods proposed in the present paper are based on different approach and do not require any kind of the regularization. In order to verify the applicability of the proposed method in the image restoration, we present a comparison of results generated by the TSVD and the rank-one updates.

Parameters of Barbara image degradation are , , , and and salt pepper noise with the noise density of is assumed. The original and damaged image are presented in Figure 4.

(a)

(b)

Figure 4: (a) Image of Barbara. (b) Blurred noisy image of Barbara.

Comparison with the state-of-the-art methods in image processing is presented. Many of them are constrained least-squares filter (CLS); Wiener filter (WF); Lucy-Richardson algorithm (LR); and truncated singular value decomposition (TSVD) [28]. Improvements in signal-to-noise ratio (ISNR) and Dice Coefficient (DC) are considered.

Results corresponding to ISNR values are presented in Figures 5 and 6.

Figure 5: ISNR versus for the removal of Gaussian white noise with zero mean and variance 0.05 and Gaussian blur (, , and ) for Barbara image.

Figure 6: ISNR versus for the removal of Gaussian white noise with zero mean and variance 0.05 and Gaussian blur (, and ) for Barbara image.

Values corresponding to Dice Coefficient (DC) [29, 30] are presented in Figure 7.

Figure 7: DC versus for the removal of Gaussian white noise with zero mean and variance 0.05 and Gaussian blur (, and ) for Barbara image.

Restored images are presented in Figures 8 and 9.

(a)

(b)

Figure 8: (a) Moore-Penrose restored image. (b) Lucy-Richardson restored image.

(a)

(b)

(c)

Figure 9: (a) Wiener filter restored image, (b) constrained least-squares filter restored image, and (c) truncated singular value decomposition restored image.

6. Conclusion

The present paper investigates the application of the SR1 updates procedure and S-M formula in computation of the Moore-Penrose inverse of specific Toeplitz matrices that appear in the image restoration process.

The first approach is based on the usage of the pure SR1 update algorithm, proposed in [15]. The second approach starts with the SR1 update algorithm and, after its completion, continues with the BP method. The third approach is a hybrid combination which comprises the SR1 updates in the first step and the S-M formula in the second phase (Algorithm 1). The fourth approach is based on the improvement of the SR1 update step method of Algorithm 1, which leads to Algorithm 2 with the best performances. The improvement is achieved using the specific structure of the underlying Toeplitz matrix.

An application of Algorithm 2 in image restoration is considered. In this case, a comparison with the modified BP method from [23] is presented.

Appendix

The Matlab implementation of Algorithm 1 is given in Algorithm 3 under the function . Note that the correct performance of the function requires the presence of yltXl function for computing the sequences and .

function alg41 = alg41(A)

%***************************%

% General Information.             %

%***************************%

% Synopsis:

% alg41 = alg41(A)

% Input:

% A = the initial matrix of interest.

% Output:

% alg41 = mp-inverse by using rank-one updates and the Sherman_Morrison formula.

% This function follows Algorithm 1.

% The correct performance of alg41b requires the presence of yltXl function.

Astar = A’;

m,n = size(A);

G0 = zeros(m,m);

if m > 1

  for i = 1:m-1

     Gkout = G0+A(:,i)*Astar(i,:);

     G0 = Gkout;

  end

  Gm = Gkout+A(:,m)*Astar(m,:);

  else

     Gm = G0;

end

Gkout = Gm;

Y02 = Gkout∖A;

X02 = Y02’;

for l = m+1:n-1

yltout,Xlout = yltXl(Y02,X02,Astar,l,m,n);

Y02 = yltout;

X02 = Xlout;

end

alg41 = X02 - Astar*Y02(:,n)*Y02(:,n)’/(1+Astar(n,:)*Y02(:,n));

Algorithm 3

The Matlab implementation of Algorithm 2 is given in Algorithm 4 under the function . Note that the correct performance of requires the presence of yltXl function for computing the sequences and .

function alg42 = alg42(A,ll)

%***************************%

% General Information.             %

%***************************%

% Synopsis:

% alg42 = alg42(A)

% Input:

% A = the mxn initial matrix of interest.

% ll=n-m+1

% Output:

% alg42 = improvement of Algorithm 1 according to Theorem 6.

% This function follows Algorithm 2.

% The correct performance of alg42 requires the presence of yltXl function.

Astar = A’;

m,n = size(A);

f = A(1,1:ll);

E0 = zeros(1,m);

product = f’*f;

for i = 1:ll

        E0(1,i) = sum(diag(product,i-1));

end

E1=E0;

E = toeplitz(E1);

G0=zeros(ll-1);

Gnew = hankel(zeros(1,ll-2) f(end), flip(f(1,2:end)));

for i = 1:ll-1

        Gkoutnew = G0+Gnew(:,i)*Gnew(i,:);

        G0=Gkoutnew;

end

Gmnew = G0;

fullmatrix = zeros(m-ll+1) zeros(m-ll+1,ll-1);zeros(ll-1,m-ll+1) Gmnew;

Gkout = E-fullmatrix;

Y02 = Gkout∖A;

X02 = Y02’;

for l = m+1:n-1

yltout,Xlout = yltXl(Y02,X02,Astar,l,m,n);

Y02 = yltout;

X02 = Xlout;

end

alg42 = X02 - Astar*Y02(:,n)*Y02(:,n)’/(1+Astar(n,:)*Y02(:,n));

Algorithm 4

The yltXl Function. See Algorithm 5.

function yltout,Xlout = yltXl(Y0,X0,G,l,m,n)

d = n-l;

Y0l = Y0(:,l);

g = 1+G(l,:)*Y0l;

g1 = Y0l*G(l,:);

multiplier = eye(m)-g1/g;

Y0lherm = Y0l*Y0l’;

 Xl = X0 - G*Y0lherm/g;

     Xlout = Xl;

ylt = zeros(m,n);

    for i = 1:d

    t = l+i;

        ylt(:,t) = multiplier*Y0(:,t);

    yltout = ylt;

 end

Algorithm 5

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

Predrag S. Stanimirović gratefully acknowledges support from the Research Project 174013 of the Serbian Ministry of Science. Vasilios N. Katsikis gratefully acknowledges that this work was carried out at the Department of Economics, University of Athens, Greece, and supported by Special Account Research Grant 11698. Predrag S. Stanimirović and Igor Stojanović gratefully acknowledge support from the Project “Applying Direct Methods for Digital Image Restoring” of the Goce Delev University.

References

D. K. Faddeev and V. N. Faddeeva, Computational Methods of Linear Algebra, W.H. Freeman, San Francisco, Calif, USA, 1963.

C. G. Broyden, “A class of methods for solving nonlinear simultaneous equations,” Mathematics of Computation, vol. 19, pp. 577–593, 1965.

R. Bru, J. Cerdán, J. Marín, and J. Mas, “Preconditioning sparse nonsymmetric linear systems with the Sherman-Morrison formula,” SIAM Journal on Scientific Computing, vol. 25, no. 2, pp. 701–715, 2003.

A. S. Householder, The Theory of Matrices in Numerical Analysis, Blaisdell Publishing, New York, NY, USA, 1964.

P. Maponi, “The solution of linear systems by using the Sherman-Morrison formula,” Linear Algebra and its Applications, vol. 420, no. 2-3, pp. 276–294, 2007.

K. Moriya, L. Zhang, and T. Nodera, “An approximate matrix inversion procedure by parallelization of the Sherman-Morrison formula,” The ANZIAM Journal, vol. 51, no. 1, pp. 1–9, 2009.

E. D. Nino-Ruiz, A. Sandu, and J. Anderson, “An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula,” Statistics and Computing, vol. 25, no. 3, pp. 561–577, 2015.

D. Bini and B. Meini, “Approximate displacement rank and applications,” in Structured Matrices in Mathematics, Computer Science and Engineering II, V. Olshevsky, Ed., vol. 281 of Contemporary Mathematics, pp. 215–232, American Mathematical Society, Rhode Island, RI, USA, 2001.

D. A. Bini, G. Codevico, and M. Van Barel, “Solving Toeplitz least squares problems by means of Newton's iteration,” Numerical Algorithms, vol. 33, no. 1–4, pp. 93–103, 2003.

J.-F. Cai, M. K. Ng, and Y.-M. Wei, “Modified Newton's algorithm for computing the group inverses of singular Toeplitz matrices,” Journal of Computational Mathematics, vol. 24, no. 5, pp. 647–656, 2006.

M. Miladinović, S. Miljković, and P. Stanimirović, “Modified SMS method for computing outer inverses of Toeplitz matrices,” Applied Mathematics and Computation, vol. 218, no. 7, pp. 3131–3143, 2011.

W. R. Trench, “An algorithm for the inversion of finite Toeplitz matrices,” SIAM: SIAM Journal on Applied Mathematics, vol. 12, no. 3, pp. 515–522, 1964.

Y. Wei, J. Cai, and M. K. Ng, “Computing Moore-Penrose inverses of Toeplitz matrices by Newton's iteration,” Mathematical and Computer Modelling, vol. 40, no. 1-2, pp. 181–191, 2004.

S. L. Campbell and C. D. Meyer, Generalized Inverses of Linear Transformations, Pitman, London, UK, 1979.

X. Chen and J. Ji, “Computing the Moore-Penrose inverse of a matrix through symmetric rank-one updates,” American Journal of Computational Mathematics, vol. 1, no. 3, pp. 147–151, 2011.

X. Chen and J. Ji, “The minimum-norm least-squares solution of a linear system and symmetric rank-one updates,” Electronic Journal of Linear Algebra, vol. 22, pp. 480–489, 2011.

X. Chen, “The generalized inverses of perturbed matrices,” International Journal of Computer Mathematics, vol. 41, no. 3-4, pp. 223–236, 1992.

C. D. Meyer Jr., “Generalized inversion of modified matrices,” SIAM Journal on Applied Mathematics, vol. 24, pp. 315–323, 1973.

J. K. Baksalary and O. M. Baksalary, “Relationships between generalized inverses of a matrix and generalized inverses of its rank-one-modifications,” Linear Algebra and Its Applications, vol. 388, pp. 31–44, 2004.

P. S. Stanimirović, V. N. Katsikis, and D. Pappas, “Computing {2,4} and {2,3}-inverses by using the Sherman-Morrison formula,” Applied Mathematics and Computation, vol. 273, pp. 584–603, 2016.

T. N. E. Grevile, “Some applications of the pseudo-inverse of matrix,” SIAM Review, vol. 3, pp. 15–22, 1960.

F. E. Udwadia and R. E. Kalaba, “General forms for the recursive determination of generalized inverses: unified approach,” Journal of Optimization Theory and Applications, vol. 101, no. 3, pp. 509–521, 1999.

P. S. Stanimirović, M. Miladinović, I. Stojanović, and S. Miljković, “Application of the partitioning method to specific Toeplitz matrices,” International Journal of Applied Mathematics and Computer Science, vol. 23, no. 4, pp. 809–821, 2013.

S. Chountasis, V. N. Katsikis, and D. Pappas, “Applications of the Moore-Penrose inverse in digital image restoration,” Mathematical Problems in Engineering, vol. 2009, Article ID 170724, 12 pages, 2009.

S. Chountasis, V. N. Katsikis, and D. Pappas, “Digital image reconstruction in the spectral domain utilizing the Moore-Penrose inverse,” Mathematical Problems in Engineering, vol. 2010, Article ID 750352, 14 pages, 2010.

S. Karanasios and D. Pappas, “Generalized inverses and special type operator algebras,” Facta Universitatis, Series: Mathematics and Informatics, vol. 21, pp. 41–48, 2006.

V. N. Katsikis and V. D. Pappas, “Fast computing of the Moore-Penrose inverse matrix,” Electronic Journal of Linear Algebra, vol. 17, pp. 637–650, 2008.

P. C. Hansen, J. G. Nagy, and D. P. O'Leary, Deblurring Images: Matrices, Spectra, and Filtering, SIAM, Philadelphia, Pa, USA, 2006.

R. C. Craddock, G. A. James, P. E. Holtzheimer III, X. P. Hu, and H. S. Mayberg, “A whole brain fMRI atlas generated via spatially constrained spectral clustering,” Human Brain Mapping, vol. 33, no. 8, pp. 1914–1928, 2012.

L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945.