𝑠 -Goodness for Low-Rank Matrix Recovery

. Low-rank matrix recovery (LMR) is a rank minimization problem subject to linear equality constraints, and it arises in many fields such as signal and image processing, statistics, computer vision, and system identification and control. This class of optimization problems is generally NP hard. A popular approach replaces the rank function with the nuclear norm of the matrix variable. In this paper, we extend and characterize the concept of 𝑠 -goodness for a sensing matrix in sparse signal recovery (proposed by Juditsky and Nemirovski (Math Program, 2011)) to linear transformations in LMR. Using the two characteristic 𝑠 -goodness constants, 𝛾 𝑠 and ̂𝛾 𝑠 , of a linear transformation, we derive necessary and sufficient conditions for a linear transformation to be 𝑠 -good. Moreover, we establish the equivalence of 𝑠 -goodness and the null space properties. Therefore, 𝑠 -goodness is a necessary and sufficient condition for exact 𝑠 -rank matrix recovery via the nuclear norm minimization.


Introduction
Low-rank matrix recovery (LMR for short) is a rank minimization problem (RMP) with linear constraints or the affine matrix rank minimization problem which is defined as follows: minimize rank () , where  ∈ R × is the matrix variable, A : R × → R  is a linear transformation, and  ∈ R  .Although specific instances can often be solved by specialized algorithms, the LMR is NP hard.A popular approach for solving LMR in the systems and control community is to minimize the trace of a positive semidefinite matrix variable instead of its rank (see, e.g., [1,2]).A generalization of this approach to nonsymmetric matrices introduced by Fazel et al. [3] is the famous convex relaxation of LMR (1), which is called nuclear norm minimization (NNM): where ‖‖ * is the nuclear norm of , that is, the sum of its singular values.When  =  and the matrix  := Diag(),  ∈ R  , is diagonal, the LMR (1) reduces to sparse signal recovery (SSR), which is the so-called cardinality minimization problem (CMP): where ‖‖ 0 denotes the number of nonzero entries in the vector  and Φ ∈ R × is a given sensing matrix.A well-known heuristic for SSR is the ℓ 1 -norm minimization relaxation (basis pursuit problem): where ‖‖ 1 is the ℓ 1 -norm of , that is, the sum of absolute values of its entries.LMR problems have many applications and they appeared in the literature of a diverse set of fields including signal and image processing, statistics, computer vision, and system identification and control.For more details, see the recent paper [4].LMR and NNM have been the focus of some recent research in the optimization community, see; for example, [4][5][6][7][8][9][10][11][12][13][14][15].Although there are many papers dealing with algorithms for NNM such as interior-point methods, fixed point and Bregman iterative methods, and proximal point methods, there are fewer papers dealing with the conditions that guarantee the success of the low-rank matrix recovery via NNM.For instance, following the program laid out in the work of Candès and Tao in compressed sensing (CS, see, e.g., [16][17][18]), Recht et al. [4] provided a certain restricted isometry property (RIP) condition on the linear transformation which guarantees that the minimum nuclear norm solution is the minimum rank solution.Recht et al. [14,19] gave the null space property (NSP) which characterizes a particular property of the null space of the linear transformation, which is also discussed by Oymak et al. [20,21].Note that NSP states a necessary and sufficient condition for exactly recovering the low-rank matrix via nuclear norm minimization.Recently, Chandrasekaran et al. [22] proposed that a fixed -rank matrix  0 can be recovered if and only if the null space of A does not intersect the tangent cone of the nuclear norm ball at  0 .
In the setting of CS, there are other characterizations of the sensing matrix, under which ℓ 1 -norm minimization can be guaranteed to yield an optimal solution to SSR, in addition to RIP and null-space properties, see; for example, [23][24][25][26].In particular, Juditsky and Nemirovski [24] established necessary and sufficient conditions for a Sensing matrix to be "-good" to allow for exact ℓ 1 -recovery of sparse signals with  nonzero entries when no measurement noise is present.They also demonstrated that these characteristics, although difficult to evaluate, lead to verifiable sufficient conditions for exact SSR and to efficiently computable upper bounds on those  for which a given sensing matrix is -good.Furthermore, they established instructive links between -goodness and RIP in the CS context.One may wonder whether we can generalize the -goodness concept to LMR and still maintain many of the nice properties as done in [24].Here, we deal with this issue.Our approach is based on the singular value decomposition (SVD) of a matrix and the partition technique generalized from CS.In the next section, following Juditsky and Nemirovski's terminology, we propose definitions of goodness and -numbers,   and γ , of a linear transformation in LMR and then we provide some basic properties of -numbers.In Section 3, we characterize -goodness of a linear transformation in LMR via -numbers.We consider the connections between the -goodness, NSP, and RIP in Section 4. We eventually obtain that Let  ∈ R × ,  := min{, }, and let  =  Diag(())  be an SVD of , where  ∈ R × ,  ∈ R × , and Diag(()) is the diagonal matrix of () = ( 1 (), . . .,   ())  which is the vector of the singular values of .Also let Ξ() denote the set of pairs of matrices (, ) in the SVD of ; that is, For  ∈ {0, 1, 2, . . ., }, we say  ∈ R × is an -rank matrix to mean that the rank of  is no more than .For an rank matrix , it is convenient to take  =  ×     × as its SVD where  × ∈ R × ,  × ∈ R × are orthogonal matrices and   = Diag(( 1 (), . . .,   ())  ).For a vector  ∈ R  , let ‖ ⋅ ‖  be the dual norm of ‖ ⋅ ‖ specified by ‖‖  := max V {⟨V, ⟩ : ‖V‖ ≤ 1}.In particular, ‖ ⋅ ‖ ∞ is the dual norm of ‖ ⋅ ‖ 1 for a vector.Let ‖‖ denote the spectral or the operator norm of a matrix  ∈ R × , that is, the largest singular value of .In fact, ‖‖ is the dual norm of ‖‖ * .Let ‖‖  := √⟨, ⟩ = √ Tr(  ) be the Frobenius norm of , which is equal to the ℓ 2 -norm of the vector of its singular values.We denote by   the transpose of .For a linear transformation A : R × → R  , we denote by A * : R  → R × the adjoint of A.

Definitions and Basic Properties
2.1.Definitions.We first go over some concepts related to goodness of the linear transformation in LMR (RMP).These are extensions of those given for SSR (CMP) in [24].
(ii) -number γ (A, ) is the infimum of  ≥ 0 such that for every matrix  ∈ R × with  nonzero singular values, all equal to 1, there exists a vector  ∈ R  such that A *  and  share the same orthogonal row and column spaces: If there does not exist such  for some  as above, we set   (A, ) = +∞ and to be compatible with the special case given by [24] we write   (A), γ (A) instead of   (A, +∞), γ (A, +∞), respectively.
From the above definition, we easily see that the set of values that  takes is closed.Thus, when   (A, ) < +∞, for every matrix  ∈ R × with  nonzero singular values, all equal to 1, there exists a vector  ∈ R  such that Similarly, for every matrix  ∈ R × with  nonzero singular values, all equal to 1, there exists a vector ŷ ∈ R  such that A * ŷ and  share the same orthogonal row and column spaces: Observing that the set {A *  : ‖‖  ≤ } is convex, we obtain that if   (A, ) < +∞ then for every matrix  with at most  nonzero singular values and ‖‖ ≤ 1 there exist vectors  satisfying (10) and there exist vectors ŷ satisfying (11).

Basic Properties of 𝐺-Numbers.
In order to characterize the -goodness of a linear transformation A, we study the basic properties of -numbers.We begin with the result that -numbers   (A, ) and γ (A, ) are convex nonincreasing functions of .
The above inequality follows immediately if one of  1 ,  2 is +∞.Thus, we may assume  1 ,  2 ∈ [0, +∞).In fact, from the argument around (10) and the definition of   (A, ⋅), we know that for every matrix  =  Diag(())  with  nonzero singular values, all equal to 1, there exist vectors It is immediate from (13 Moreover, from the above information on the singular values of A *  1 , A *  2 , we may set A *   =  +   ,  ∈ {1, 2} such that This implies that for every  ∈ [0, 1] and hence rank[ 1 +(1−) 2 ] ≤ −, , and have orthogonal row and column spaces.Thus, noting that for every  ∈ [0, 1].Combining this with the fact we obtain the desired conclusion.
For a given pair ,  as above, take ỹ := (1/(1 + )).Then we have ‖ ỹ‖  ≤ (1/(1 + )) and where the first term under the maximum comes from the fact that A *  and  agree on the subspace corresponding to the nonzero singular values of .Therefore, we obtain Now, we assume that γ := γ (A, ) < 1/2.Fix orthogonal matrices  ∈ R × ,  ∈ R × .For an -element subset  of the index set {1, 2, . . ., }, we define a set   with respect to orthogonal matrices ,  as In the above,  denotes the complement of .It is immediately seen that   is a closed convex set in R  .Moreover, we have the following Proof.Note that   is closed and convex.Moreover,   is the direct sum of its projections onto the pair of subspaces and its orthogonal complement Let  denote the projection of   onto   .Then,  is closed and convex (because of the direct sum property above and the fact that   is closed and convex).Note that   can be naturally identified with R  , and our claim is the image  ⊂ R  of  under this identification that contains the ‖ ⋅ ‖ ∞ball   of radius (1 − γ) centered at the origin in R  .For a contradiction, suppose   is not contained in .Then there exists V ∈   \.Since  is closed and convex, by a separating hyperplane theorem, there exists a vector  ∈ R  , ‖‖ 1 = 1 such that Let  ∈ R  be defined by By definition of γ = γ (A, ), for -rank matrix  Diag()  , there exists  ∈ R  such that ‖‖  ≤  and where  and  Diag()  have the same orthogonal row and column spaces, ‖A *  −  Diag()  ‖ ≤ γ and ‖(A * ) − ‖ ∞ ≤ γ.Together with the definitions of   and , this means that  contains a vector V with |V  − sign(  )| ≤ γ, ∀ ∈ {1, 2, . . ., }.Therefore, By V ∈   and the definition of , we obtain where the strict inequality follows from the facts that V ∈  and  separates V from .The above string of inequalities is a contradiction, and hence the desired claim holds.
Using the above claim, we conclude that for every  ⊆ {1, 2, . . ., } with cardinality , there exists an  ∈   such that   = (1 − γ), for all  ∈ .From the definition of   , we obtain that there exists  ∈ R  with ‖‖  ≤ (1 − γ) −1  such that where Thus, we obtain that To conclude the proof, we need to prove that the inequalities we established are both equations.This is straightforward by an argument similar to the one in the proof of [24,Theorem 1].We omit it for the sake of brevity.

𝑠-Goodness and 𝐺-Numbers
We first give the following characterization result of -goodness of a linear transformation A via the -number   (A), which explains the importance of   (A) in LMR.

𝑠-Goodness, NSP, and RIP
This section deals with the connections between -goodness, the null space property (NSP), and the restricted isometry property (RIP).We start with establishing the equivalence of NSP and -number γ (A) < 1/2.Here, we say A satisfies NSP if for every nonzero matrix  ∈ Null(A) with the SVD  =  Diag(())  , then we have For further details, see, for example, [14,[19][20][21] and references therein.
Proposition 9.For the linear transformation A, γ (A) < 1/2 if and only if A satisfies NSP.
Proof.We first give an equivalent representation of the number γ (A, ).We define a compact convex set first: For the above, we adopt the convention that whenever  = +∞, ‖A‖ is defined to be +∞ or 0 depending on whether ‖A‖ > 0 or ‖A‖ = 0. Next, we consider the connection between restricted isometry constants and -number of the linear transformation in LMR.It is well known that, for a nonsingular matrix (transformation)  ∈ R × , the RIP constants of A and A can be very different, as shown by Zhang [30] for the vector case.However, the -goodness properties of A and A are always the same for a nonsingular transformation  ∈ R × (i.e., -goodness properties enjoy scale invariance in this sense).Recall that the -restricted isometry constant   of a linear transformation A is defined as the smallest constant such that the following holds for all -rank matrices  ∈ R × : In this case, we say A possesses the RI (  )-property (RIP) as in the CS context.For details, see [4,[31][32][33][34] and the references therein.
For the RIP constant  2 , Oymak et al. [21] gave the current best bound on the restricted isometry constant  2 < 0.472, where they proposed a general technique for translating results from SSR to LMR.Together with the above arguments, we immediately obtain the following theorem.The above theorem says that -goodness is a necessary and sufficient condition for recovering the low-rank solution exactly via nuclear norm minimization.

Conclusion
In this paper, we have shown that -goodness of the linear transformation in LMR is a necessary and sufficient conditions for exact -rank matrix recovery via the nuclear norm minimization, which is equivalent to the null space property.Our analysis is based on the two characteristic -goodness constants,   and γ , and the variational property of matrix norm in convex optimization.This shows that -goodness is an elegant concept for low-rank matrix recovery, although   and γ may not be easy to compute.Development of efficiently computable bounds on these quantities is left to future work.Even though we develop and use techniques based on optimization, convex analysis, and geometry, we do not provide explicit analogues to the results of Donoho [35] where necessary and sufficient conditions for vector recovery special case were derived based on the geometric notions of face preservation and neighborliness.The corresponding generalization to low-rank recovery is not known, currently the closest one being [22].Moreover, it is also important to consider the semidefinite relaxation (SDR) for the rank minimization with the positive semidefinite constraint since the SDR convexifies nonconvex or discrete optimization problems by removing the rank-one constraint.Another future research topic is to extend the main results and the techniques in this paper to the SDR.