Abstract and Applied Analysis

Volume 2013 (2013), Article ID 781276, 10 pages

http://dx.doi.org/10.1155/2013/781276

Research Article

Norm-Constrained Least-Squares Solutions to the Matrix Equation

An-bao Xu and Zhenyun Peng

Guilin University of Electronic Technology, Guilin 541004, China

Received 16 March 2013; Accepted 9 May 2013

Academic Editor: Masoud Hajarian

Copyright © 2013 An-bao Xu and Zhenyun Peng. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

An iterative method to compute the least-squares solutions of the matrix over the norm inequality constraint is proposed. For this method, without the error of calculation, a desired solution can be obtained with finitely iterative step. Numerical experiments are performed to illustrate the efficiency and real application of the algorithm.

1. Introduction

Throughout this paper, denotes the set of all real matrices. represents the identity matrix of size implied by context. , denote, respectively, the transpose and the Frobenius norm of the matrix . For the matrices ,, represents the Kronecker production of the matrices and defined as . The inner product in the matrix set space is defined as for all the matrices . Obviously, is a Hilbert inner product space and the norm of a matrix generated by this inner product space is the Frobenius norm.

Solutions to the well-known linear matrix equation with special structures have been widely studied. See, for example, [1–5] for symmetric solutions, skew-symmetric solutions, centro-symmetric solutions, symmetric R-symmetric solutions, or (R,S)-symmetric solutions. To the best of our knowledge, the solutions to the matrix equation subject to the norm inequality constraint, however, have not been studied directly in the literature. In this paper we consider the solutions to the following least-squares problem over the norm inequality constraint:

where , , and is a nonnegative real number.

Problem (1) can be regarded as a natural generalization of the unconstrained least-squares problem of the matrix equation . In fact, when we let be big enough, the problem will turn out to be the unconstrained least-squares problem of the matrix equation . According to [6], moreover, the problem (1) is equivalent to the classical Tikhonov regulation approach of the matrix equation

where is the regularization parameter. While Tikhonov regularization involves the computation of a parameter that does not necessarily have a physical meaning in most problems, the problem (1) has the advantage that, in some applications, the physical properties of the problem either determine or make it easy to estimate an optimal value for the norm constraint . This is the case, for example, in image restoration where represents the energy of the target image [7].

In this paper, an iterative method is proposed to compute the solutions of the problem (1). We will use the generalized Lanczos trust region algorithm (GLTR) [8], which is based on Steihaug-Toint algorithm [9, 10], as the frame method for deriving this iterative method. The basic idea is as follows. First, by using the Kronecker production of matrices, we transform the least-squares problem (1) into the trust-region subproblem in vector form which can be solved by the GLTR algorithm. Then, we transform the vector iterative method into matrix form. In the end, numerical experiments are given to illustrate the efficiency and real application of the proposed iteration algorithm.

2. Iteration Methods to Solve Problem (1)

In this section we first give the necessary and sufficient conditions for the problem (1) to have a solution. Then we propose an iteration method to compute the solution to the problem. And some properties of this algorithm are also given.

Obviously, problem (1) is equivalent to the following problem

This equivalent form of the problem (1) makes us more convenient to prove the following theorem.

Theorem 1. Matrix is a solution of the problem (3) if and only if there is a scalar such that the following conditions are satisfied:

Proof. Assume that there is a scalar such that the conditions (4) are satisfied. Let

For any matrix , we have

This implies that is a global minimizer of the function . Since for all , we have

The equality implies that . Consequently, the following inequality always holds:

Hence, from , we have for all with . And so is a global minimizer of (3).

Conversely, assuming that is a global solution of the problem (3), we show that there is a nonnegative such that satisfies conditions (4). For this purpose we consider two cases: and .

In case , is certainly an unconstrained minimizer of . So satisfies the stationary point condition ; that is, . This implies that the properties (4) hold for . In the case , the second equality is immediately satisfied, and also solves the constrained problem

By applying optimality conditions for constrained optimization to this problem, we know that there exists a scalar such that the Lagrangian function defined by

has a stationary point at . By setting to zero, we obtain

Now the proof is concluded by showing that . Since the equality (11) holds, then minimizes , and so we have

for all . Suppose that there are only negative values of that satisfy (11). Then we have from (12) that

Since we already know that minimizes for , it follows that is in fact a global, unconstrained minimize of . Therefore conditions (11) hold when , which contradicts our assumption that only negative values of can satisfy condition (11). The proof is completed.

We give an iteration method to solve problem (1) as in Algorithm 2.

Algorithm 2. (i) Given matrices , and a small tolerance . Compute

 ,,,. Set .

(ii) Computing , , ,

 , where .

(iii) If , compute .

 If , computing , , , else, go to Step 4. If , stop, else set and go to Step 2.

(iv) Find the solution to the following optimization problem:

(v) If (here represents the last column of identity matrix ), set

and then stop, else set and go to Step 2.

The basic iteration route of Algorithm 2 to solve problem (1) includes two cases: First, using CG method (Step 3) to compute the solution of problem (1) in feasible region. When the first case is failure, the solution of problem (1) in feasible region cannot be obtained by using CG method, and then the solution of problem (1) on the boundary can be obtained by solving the optimization problem (14). The properties about Algorithm 2 are given as follows.

Theorem 3. Assume that the sequences , , and are generated by Algorithm 2; then the following equalities hold for all , ,:

Proof. Since holds for all matrices and , we only need to prove that the conclusion holds for all . Using induction and two steps are required.

Step 1. Show that , and hold for all . We also use the principle of mathematical induction to prove these conclusions. When , we have

Assume that conclusion holds for all ; then

By the principle of induction, , , and hold for all .

Step 2. Assume that , , and for all and , show that , , and . The proof is as follows:

From Steps 1 and 2, we have by principle induction that , , hold for .

Theorem 4. Assume that the sequence is generated by Algorithm 2; then the following equalities hold:

Proof. By the definition of , we immediately know that . Similar to the proof of Theorem 3, we also use the principle of mathematical induction to prove this conclusion with the two following cases.

Step 1. Show that for all .

When , we have

Assume that conclusion holds for all ; then

By the principle of induction, holds for all .

Step 2. Assume that for all and show that . The proof is as follows:

From steps 1 and 2, we have by the principle of mathematical induction that hold for all .

Theorem 5. Assume that the sequences , , and are generated by Algorithm 2. Let

Then the following equality holds:

where represents the first column of identity matrix .

Proof. By the definition of and , we have

Hence, we have

So the equality (24) holds. In addition, from above equality, we have for all. Sois positive semi-definite. The proof is completed.

Theorem 6. Assume that the sequences , , , , , and are generated by Algorithm 2, then the following equalities hold for all :

Proof. (the proof of the first equality in (27)). By the definition of and , we have

where , are positive numbers. These equalities imply that and belong to the same space

And furthermore we can have

By Theorems 3 and 4, we have and . Hence and must be linear correlation, so there exists a real number such that . Noting that , we have by (28) that .

(The proof of the second equality in (27)). Noting that the first equality in (27) holds, then, when , we have

When , we have

(The proof of the third equality in (27)). By the definition of , we have

Hence the third equality in (27) holds. The proof is completed.

Remark 7. This theorem shows the relationship between the sequences  , , , , , and to lower down the cost of calculation.

3. The Main Results and Improvement of the Iteration Method

We will show that the solution of the problem (1) can be obtained within finite iteration steps in the absence of round-off errors. And we give the detail to solve the problem (14) in order to complete Algorithm 2. By discussing the characterization of the proposed iteration method, the further optimization method for the proposed iteration method is given at the end of this section.

Theorem 8. Assume that the sequences , are generated by Algorithm 2. Then the following equalities hold for all :

Proof. We use the principle of mathematical induction to prove this conclusion. When , obviously, the conclusion holds. Assume that the conclusion holds for ; then

This implies that the conclusion holds for . By the principle of mathematical induction, we know that the conclusion holds for all .

Remark 9. For Theorem 3, the sequences are orthogonal to each other in the finite dimension matrix space ; it is certain that there exists a positive number such that . So without the error of calculation, the first stopping criterion in the algorithm will perform with finite steps. From Theorem 8, we get . According to Theorem 1, when we set , is a solution of the problem (3).

Theorem 10. Assume that the sequences , , and are generated by Algorithm 2. Let

Then, for all , there exists a nonnegative number such that

Proof. Assume that is the solution of optimization problem (14); then there exists a nonnegative number such that the following optimality Karush-Kuhn-Tucker (KKT) conditions are satisfied:

Noting that and the second equality in (38) hold, we know that the second equality in (37) holds.

Since the first equality in (38) can be rewritten as

so we have

Hence, we have

The proof is completed.

Theorem 11. Assume that , and . Then is the solution of the problem (1).

Proof. Since and , we have by Theorem 10 that

which implies that is the solution of the problem (1).

Remark 12. According to Theorem 4, the sequences are orthogonal each other in the finite dimension matrix space ; it is certain that there exists a positive number such that . Since , then . So without the error of calculation, the second stopping criterion in the algorithm also performs with finite steps.

Remark 13. According to Remarks 9 and 12, we have that, without the error of calculation, a desired solution can be obtained with finitely iterative step by Algorithm 2.

Theorem 14. The solution of the problem (14) obtained by Algorithm 2 is on the boundary. In other words, is the solution of the following optimization problem:

Proof. Assuming that the solution of the problem (14) obtained by Algorithm 2 is inside the boundary, we have by (38) that . By Theorem 5, we know is a positive semidefinite matrix. If is positive definite, then with is a unique solution of the problem (14). Hence, we have by Theorem 5 that with is a unique solution of the problem (1). In this case, the step of solving the problem (14) in Algorithm 2 cannot be implemented. If is positive semidefinite and not positive definite, then there exists a matrix such that and which implies that is a solution to the problem (1) on the boundary. This contradicts our assumption.

Now we use the following Algorithm 15, which was proposed by More and Sorensen in paper [11], to solve the problem (43).

Algorithm 15. (I) Let a suitable starting value and be given.

(II) For until convergence.

(a)Factorize , where and are unit bidiagonal and diagonal matrix, respectively.(b)Solve .(c)Solve .(d)Set .

In the implementation of Algorithm 15, the initial secular can be chosen by the following principles: If , let ; else let , where is obtained by the ()th iterative steps of Algorithm 2. The stopping criteria can be used as , where is a small tolerance.

By fully using the result of Theorem 6, Algorithm 2 can be optimized as in Algorithm 16.

Algorithm 16. (i) Given matrices , and a small tolerance .

Computing ,.

Set ,,, and .

(ii) If , compute

where .

Else, computing (the first one ,

(iii) If , computing , .

If , stop. Else, setting and go to Step 2.

Else, go to Step 4.

(iv) Using Algorithm 15 to compute the solution of the problem (43).

(v) If , setting , then stop.

Else, setting and go to step 2.

4. Numerical Experiments

In this section, we present numerical examples to illustrate the availability and the real application of the proposed iteration method. All tests are performed using MATLAB 7.1 with a 32-bit Windows XP operating system. Our experiments are performed on an FOUNFER computer of mode E520 with 2.8 GHz CPU and 3.25 G RAM. Because of the error of calculation, the iteration will not stop with finite steps. Hence, we regard the approximation solution as the solution of problem (1) if the , where

Example 17. Given the matrices ,, as follows:

When , using Algorithm 16 and iterate 43 steps, we obtain the approximation solution

When , using the Algorithm 16 and iterative 23 steps, we obtain the approximation solution

Given a nonnegative real number , with iterate 45 steps, we obtain the approximation solution

Example 18. We work with a 2D first-kind Fredholm integral equation of the generic form

where and are function. Based on [12], we have that the discretization of the problem (51) leads to the linear relation between the discrete solution and the discrete data , where

An example of such problem is image denoising with a Gaussian point spread function:

which is used as a model for out-of-focus as well as atmospheric turbulence blur [13]. In Figure 1(a), the original image is the standard test image of Lena with size , which is also a matrix . After the image was blurred by Gaussian kernel (53) with , we get Figure 1(b); that is the matrix . In image denoising, our target is to get the solution of . Tikhonov regularization is needed to treat this problem in order to control the effect of the noise on the solution. As we have said in Section 1, Tikhonov regularization is equivalent to over the norm inequality constraint matrix equation

(a)

(b)

(c)

Figure 1: (a) The original image. (b) The noisy image. (c) The recovered image.

Based on [7], represents the energy of the target image, so we get . Solving the above problem by Algorithm 16, we get the recovered image in Figure 1(c). It means our algorithm is suitable for image denoising.

Acknowledgments

The research was supported by National Natural Science Foundation of China (11261014) and Innovation Project of GUET Graduate Education (XJYC2012023).

References

Z.-Y. Peng, “A matrix LSQR iterative method to solve matrix equation ,” International Journal of Computer Mathematics, vol. 87, no. 8, pp. 1820–1830, 2010.

Y.-X. Peng, X.-Y. Hu, and L. Zhang, “An iteration method for the symmetric solutions and the optimal approximation solution of the matrix equation ,” Applied Mathematics and Computation, vol. 160, no. 3, pp. 763–777, 2005.

Z.-Y. Peng, “New matrix iterative methods for constraint solutions of the matrix equation ,” Journal of Computational and Applied Mathematics, vol. 235, no. 3, pp. 726–735, 2010.

M.-L. Liang, C.-H. You, and L.-F. Dai, “An efficient algorithm for the generalized centro-symmetric solution of matrix equation ,” Numerical Algorithms, vol. 44, no. 2, pp. 173–184, 2007.

G.-X. Huang, F. Yin, and K. Guo, “An iterative method for the skew-symmetric solution and the optimal approximate solution of the matrix equation ,” Journal of Computational and Applied Mathematics, vol. 212, no. 2, pp. 231–244, 2008.

M. Rojas and D. C. Sorensen, “A trust-region approach to the regularization of large-scale discrete forms of ill-posed problems,” SIAM Journal on Scientific Computing, vol. 23, no. 6, pp. 1842–1860, 2002.

M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging, Institute of Physics Publishing, Bristol, UK, 1998.

N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint, “Solving the trust-region subproblem using the Lanczos method,” SIAM Journal on Optimization, vol. 9, no. 2, pp. 504–525, 1999.

T. Steihaug, “The conjugate gradient method and trust regions in large scale optimization,” SIAM Journal on Numerical Analysis, vol. 20, no. 3, pp. 626–637, 1983.

L. Toint, “Towards an efficient sparsity exploiting Newton method for minimization,” in Proceedings of the Conference on Sparse Matrices and Their Uses, Academic Press, London, UK, 1981.

J. J. Moré and D. C. Sorensen, “Computing a trust region step,” SIAM Journal on Scientific and Statistical Computing, vol. 4, no. 3, pp. 553–572, 1983.

P. C. Hansen, M. Jacobsen, J. M. Rasmussen, and H. Sorensen, “The PP-TSVD algorithm for image restoration problems,” in Methods and Applications of Inversion, P. C. Hansen, B. H. Jacobsen, and K. Mosegaard, Eds., vol. 92 of Lecture Notes in Earth Sciences, pp. 171–186, Spring, Berlin, Germany, 2000.

M. R. Banham and A. K. Katsaggelos, “Digital image restoration,” IEEE Signal Processing Magazine, vol. 14, no. 2, pp. 24–41, 1997.