We introduce the notion of Khatri-Rao product for operator matrices acting on the direct sum of Hilbert spaces. This notion generalizes the tensor product and Hadamard product of operators and the Khatri-Rao product of matrices. We investigate algebraic properties, positivity, and monotonicity of the Khatri-Rao product. Moreover, there is a unital positive linear map taking Tracy-Singh products to Khatri-Rao products via an isometry.
Thailand Research Fund1. Introduction
In matrix theory, there are various matrix products which are of interest in both theory and applications, such as the Kronecker product, Hadamard product, and Khatri-Rao product; see, for example, [1–3]. Denote by Mm,n(C) the set of m-by-n complex matrices and abbreviate Mn,n(C) to Mn(C). Recall that the Kronecker product of A=[aij]∈Mm,n(C) and B∈Mp,q(C) is given by (1)A⊗^B=aijBij∈Mmp,nqC.The Hadamard product of A,B∈Mm,n(C) is defined by the entrywise product (2)A⊙^B=aijbij∈Mm,nC.Now, let A and B be complex matrices partitioned into blocks Aij and Bij for each i, j (the sizes of Aij and Bij may be different). Then, the Khatri-Rao product [4] of A and B is defined by (3)A⊡^B=Aij⊗^Bijij.When A and B are nonpartitioned (i.e., each has only one block), their Khatri-Rao product is just their Kronecker product. If A and B are entrywise partitioned (i.e., each block is a 1×1 matrix), then their Khatri-Rao product is their Hadamard product. Interesting algebraic, order, and analytic properties of this product were studied in the literature; see, for example, [5–12]. Their applications in statistics, computer science, and related fields can be seen, for example, in [13, 14].
The tensor product of Hilbert space operators is a natural extension of the Kronecker product to the infinite-dimensional setting. Let H, H′, K, and K′ be Hilbert spaces. Recall that the tensor product of two operators A:H→H′ and B:K→K′ is the unique bounded linear operator from H⊗K into H′⊗K′ such that, for all x∈H and y∈K, (4)A⊗Bx⊗y=Ax⊗By.
In this paper, we generalize the tensor product of operators to the Khatri-Rao product of operator matrices acting on a direct sum of Hilbert spaces. We investigate fundamental properties of this operator product. Algebraically, this product is compatible with the addition, the scalar multiplication, the adjoint operation, and the direct sum of operators. By introducing suitable operator matrices, we can prove that there is a unital positive linear map taking the Tracy-Singh product A⊠B to the Khatri-Rao product A⊡B. Hence, the Khatri-Rao product can be viewed as a generalization of the Hadamard product of operators. Moreover, positivity, strict positivity, and operator orderings are preserved under the Khatri-Rao product. Our result extends well-known results for Khatri-Rao products of complex matrices (see [4, 9, 15, 16]).
This paper is organized as follows. In Section 2, we provide some preliminaries about Tracy-Singh products for operators. These facts will be used in Sections 4 and 5. In Section 3, we introduce the Khatri-Rao product for operator matrices and deduce its algebraic properties. Section 4 explains how the Khatri-Rao product can be regarded as a generalization of the Hadamard product. Section 5 discusses positivity and monotonicity of Khatri-Rao products.
2. Preliminaries on Tracy-Singh Products for Operators
Throughout, let H, H′, K, and K′ be complex separable Hilbert spaces. When X and Y are Hilbert spaces, denote by B(X,Y) the Banach space of bounded linear operators from X into Y, and abbreviate B(X,X) to B(X). If an operator A∈B(H) satisfies 〈Ax,x〉>0, we write A>0. For self-adjoint operators A,B∈B(H), we write A⩾B to mean that A-B is a positive operator, while A>B means that A-B>0.
Decompose (5)H=⨁j=1nHj,H′=⨁i=1mHi′,K=⨁l=1qKl,K′=⨁k=1pKk′,where all Hj, Hi′, Kl, and Kk′ are Hilbert spaces. For each j, l, let Mj:Hj→H and Nl:Kl→K be the canonical embeddings. For each i and k, let Pi:H′→Hi′ and Qk:K′→Kk′ be the orthogonal projections. Two operators A∈B(H,H′) and B∈B(K,K′) can thus be represented uniquely as operator matrices (6)A=Aiji,j=1m,n,B=Bklk,l=1p,q,where Aij=PiAMj∈B(Hj,Hi′) and Bkl=QkBNl∈B(Kl,Kk′) for each i, j, k, and l. We define the Tracy-Singh product of A and B to be the bounded linear operator from ⨁j,l=1n,qHj⊗Kl to ⨁i,k=1m,pHi′⊗Kk′ expressed in a block-matrix form (7)A⊠B=Aij⊗Bklklij.Basic properties of the Tracy-Singh product are listed below.
Lemma 1.
The Tracy-Singh product (A,B)↦A⊠B is a bilinear map for operators. It is positive in the sense that if A⩾0 and B⩾0, then A⊠B⩾0.
3. Compatibility of Khatri-Rao Products with Algebraic Operations
In this section, we define the Khatri-Rao product for operator matrices and show that this product is compatible with certain algebraic operations of operators.
From now on, fix the following orthogonal decompositions of Hilbert spaces: (8)H=⨁j=1nHj,H′=⨁i=1mHi′,K=⨁j=1nKj,K′=⨁i=1mKi′.That is, we fix how to partition any operator matrix in B(H,H′) and B(K,K′). We now extend the Khatri-Rao product of matrices [4] to that of operators on a Hilbert space.
Definition 2.
Let A=Aiji,j=1m,n∈B(H,H′) and B=Biji,j=1m,n∈B(K,K′) be operators partitioned into matrices according to decomposition (8). We define the Khatri-Rao product of A and B to be the bounded linear operator from ⨁j=1nHj⊗Kj to ⨁i=1mHi′⊗Ki′ represented by the block-matrix (9)A⊡B=Aij⊗Biji,j=1m,n.
If both A and B are 1×1 block operator matrices, then A⊡B is A⊗B. When Hi=Ki=C and Hj′=Kj′=C for all i, j, the Khatri-Rao product is the Hadamard product of complex matrices.
Next, we shall show that the Khatri-Rao product of two linear maps induced by matrices is just the linear map induced by the Khatri-Rao product of these matrices. Recall that, for each A∈Mm,n(C) and B∈Mp,q(C), the induced maps, (10)LA:Cn⟶Cm,x⟼Ax,LB:Cq⟶Cp,y⟼By,are bounded linear operators. We identify Cn⊗Cq with Cnq together with the canonical bilinear map (x,y)↦x⊗^y for each (x,y)∈Cn×Cq.
Lemma 3.
For any A∈Mm,n(C) and B∈Mp,q(C), one has (11)LA⊗LB=LA⊗^B.
Proof.
Recall that the Kronecker product of matrices has the following property (see, e.g., [3]): (12)A⊗^BC⊗^D=AC⊗^BDprovided that all matrix products are well defined. It follows that, for any x∈Cn and y∈Cq, (13)LA⊗LBx⊗y=LAx⊗LBy=LAx⊗^LBy=Ax⊗^By=A⊗^Bx⊗^y=A⊗^Bx⊗y=LA⊗^Bx⊗y.The uniqueness of tensor products implies that LA⊗LB=LA⊗^B.
Proposition 4.
For any complex matrices A=[Aij] and B=[Bij] partitioned in block-matrix form, one has (14)LA⊡LB=LA⊡^B.
Proof.
Recall that the (i,j)th block of the matrix representation of LA is LAij. By Lemma 3, we obtain LA⊡LB=LAij⊗LBijij=LAij⊗^Bijij=LA⊡^B.
The next result states that the Khatri-Rao product is bilinear and compatible with the adjoint operation.
Proposition 5.
Let A∈B(H,H′) and B,C∈B(K,K′) be operator matrices, and let α∈C. Then, (15)A⊡B∗=A∗⊡B∗,(16)A⊡B+C=A⊡B+A⊡C,(17)B+C⊡A=B⊡A+C⊡A,(18)αA⊡B=αA⊡B=A⊡αB.
Proof.
Since A∗=Aji∗ij and B∗=Bji∗ij, we obtain (19)A⊡B∗=Aij⊗Bij∗ij=Aji∗⊗Bji∗ij=A∗⊡B∗.The fact that (B+C)ij=Bij+Cij for all i, j together with the left distributivity of the tensor product over the addition implies (20)A⊡B+C=Aij⊗Bij+Cijij=Aij⊗Bij+Aij⊗Cijij=A⊡B+A⊡C.Similarly, we obtain property (17). Since (αA)ij=αAij for all i, j, we get (21)αA⊡B=αAij⊗Bijij=αAij⊡Bijij=αA⊡B.Similarly, A⊡(αB)=α(A⊡B).
By property (15), the self-adjointness of operators is closed under taking Khatri-Rao products; that is, if A and B are self-adjoint, then so is A⊡B. The next proposition shows that, in order to compute the Khatri-Rao product of operator matrices, we can freely merge the partition of each operator.
Proposition 6.
Let A=Aiji,j=1m,n∈B(H,H′) and B=Biji,j=1m,n∈B(K,K′) be operator matrices represented according to decomposition (8). We merge the partition of A to be A=Aklk,l=1r,s, where r, s are given natural numbers such that r⩽m and s⩽n. Here, each operator Akl is of mk×nl block in which the (g,h)th block of Akl is the (u,v)th block of A, where (22)u=g,k=1∑i=1k-1mi+g,k>1,∑k=1rmk=m,v=h,l=1∑j=1l-1nj+h,l>1,∑l=1snl=n.Similarly, we repartition B=Bklk,l=1r,s, where each operator Bkl is of mk×nl block in which the (g,h)th block of Bkl is the (u,v)th block of B. Then, (23)A⊡B=Akl⊡Bklkl=A11⊡B11⋯A1s⊡B1s⋮⋱⋮Ar1⊡Br1⋯Ars⊡Brs.That is, each (k,l)th block of A⊡B is just Akl⊡Bkl.
Proof.
Write A⊡B=Cklk,l=1r,s, where Ckl is mk×nl block operator matrix such that the (g,h)th block of Ckl is the (u,v)th block of A⊡B. We know that the (u,v)th block of A⊡B is Auv⊗Buv. Then, (24)C11=A11⊗B11⋯A1n1⊗B1n1⋮⋱⋮Am11⊗Bm11⋯Am1n1⊗Bm1n1=A11⋯A1n1⋮⋱⋮Am11⋯Am1n1⊡B11⋯B1n1⋮⋱⋮Bm11⋯Bm1n1=A11⊡B11.Similarly, we have Ckl=Akl⊡Bkl for all k=1,…,r and l=1,…,s.
Recall that the direct sum of Ai∈B(Hi,Hi′),i=1,…,n, is defined to be the operator matrix (25)A1⊕⋯⊕An=A10⋯00A2⋯0⋮⋮⋱⋮00⋯An.The next result shows that the Khatri-Rao product is compatible with the direct sum of operators.
Proposition 7.
For each i=1,…,n, let Ai∈B(Hi,Hi′) and Bi∈B(Ki,Ki′) be compatible operator matrices. Then, (26)⨁i=1nAi⊡⨁i=1nBi=⨁i=1nAi⊡Bi.
Proof.
It follows directly from Proposition 6.
In summary, the Khatri-Rao product is compatible with fundamental algebraic operations for operators.
4. The Khatri-Rao Product as a Generalization of the Hadamard Product
In this section, we explain how the Khatri-Rao product can be viewed as a generalization of the Hadamard product. To do this, we construct two isometries which identify which blocks of the Tracy-Singh product we need to get the Khatri-Rao product.
Fix a countable orthonormal basis E for H. Recall that the Hadamard product of A and B in B(H) is defined to be the operator A⊙B in B(H) such that (27)A⊙Be,e=Ae,eBe,efor all e∈E. More explicitly, it was shown in [15] that (28)A⊙B=U∗A⊗BU,where U:H→H⊗H is the isometry defined by Ue=e⊗e for all e∈E. When H=Cn and E is the standard ordered basis of Cn, the Hadamard product of two matrices reduces to the entrywise product (2).
We now extend selection matrices in [9] to selection operators. Fix an ordered 4-tuple (H,H′,K,K′) of Hilbert spaces endowed with decomposition (8). For each r=1,…,m, consider the operator matrix (29)Er=Eghrg,h=1m,m:⨁i=1mHi′⊗Ki′⟶⨁i=1mHr′⊗Ki′,where Egh(r) is the identity operator if g=h=r and the others are zero operators. Similarly, for s=1,…,n, we define the operator matrix (30)Fs=Fghsg,h=1n,n:⨁j=1nHj⊗Kj⟶⨁j=1nHs⊗Kj,where Fgh(s) is the identity operator if g=h=s and the others are zero operators. Now, construct (31)Z1=E1⋮Em,Z2=F1⋮Fn.We call Z1 and Z2 selection operators associated with the ordered tuple (H,H′,K,K′). Notice that Z1 depends only on the ordered tuple (H′,K′) and how we decomposed H′ and K′. The operator Z2 depends on (H,K) and how we decomposed H and K. For instance, an ordered tuple (H,H′,K,K′) with decompositions (32)H=H1⊕H2⊕H3,H′=H1′⊕H2′,K=K1⊕K2⊕K3,K′=K1′⊕K2′has the following selection operators: (33)Z1=IH1′⊗K1′⊕00⊕IH2′⊗K2′,Z2=IH1⊗K1⊕0⊕00⊕IH2⊗K2⊕00⊕0⊕IH3⊗K3.In the case of H=H′ and K=K′, construction (31) gives (34)Z1=Z2=Z.If (Z1,Z2) is the ordered pair of selection operators associated with the ordered tuple (H,H′,K,K′) with decompositions given by (8), then (Z2,Z1) is the ordered pair of selection operators associated with the ordered collection (H′,H,K′,K) with the same decompositions.
Lemma 8.
Let Z1 and Z2 be selection operators defined by (31). Then, for i=1,2,
Zi∗Zi=I; that is, Zi is an isometry;
0⩽ZiZi∗⩽I.
Proof.
A direct computation shows that Z1∗Z1=I and Z2∗Z2=I. We know that EiEi∗ is an m×m block operator matrix which consists only of zero and identity operators. More precisely, the (i,i)th block of EiEi∗ is the identity operator and EiEj∗=0 for i≠j. Then, (35)Z1Z1∗=E1E1∗E1E2∗⋯E1Em∗E2E1∗E2E2∗⋯E2Em∗⋮⋮⋱⋮EmE1∗EmE2∗⋯EmEm∗=E1E1∗0⋯00E2E2∗⋯0⋮⋮⋱⋮00⋯EmEm∗.Since EiEi∗⩽I for all i=1,…,m, we have Z1Z1∗⩽I. Similarly, Z2Z2∗⩽I.
Next, we relate the Khatri-Rao and the Tracy-Singh product of operators.
Theorem 9.
For any operator matrices A∈B(H,H′) and B∈B(K,K′), one has (36)A⊡B=Z1∗A⊠BZ2,where Z1 and Z2 are the selection operators defined by (31). If H=H′ and K=K′, A∈B(H) and B∈B(K), then (37)A⊡B=Z∗A⊠BZ,where Z is the selection operator defined by (34).
Proof.
Let B(i) denote the ith column of B for i=1,…,n. Then, we have (38)Z1∗A⊠BZ2=E1∗⋯Em∗A11⊠B⋯A1n⊠B⋮⋱⋮Am1⊠B⋯Amn⊠BF1⋮Fn=E1∗⋯Em∗A11⊠BF1+⋯+A1n⊠BFn⋮Am1⊠BF1+⋯+Amn⊠BFn=E1∗⋯Em∗A11⊠B1⋯A1n⊠Bn⋮Am1⊠B1⋯Amn⊠Bn=A11⊗B11⋯A1n⊗B1n⋮⋱⋮Am1⊗Bm1⋯Amn⊗Bmn=A⊡B.If H=H′ and K=K′, then Z1=Z2 and (36) becomes (37).
We mention that Theorem 9 is an extension of both [9, Theorem 3] and result (28) in [15].
Remark 10.
If we partition A and B into row operator matrices, we have (39)A⊡B=A⊠BZ2.If both A and B are column operator matrices, then (40)A⊡B=Z1∗A⊠B.
Comparing (28) and (37), Theorem 9 shows that the Khatri-Rao product can be regarded as a generalization of the Hadamard product.
Recall that a map Φ between two C∗-algebras is said to be positive if Φ preserves positive elements. The map Φ is unital if Φ preserves the multiplicative identity.
Corollary 11.
There is a unital positive linear map (41)Φ:B⨁i,j=1n,nHi⊗Kj⟶B⨁i=1nHi⊗Kisuch that Φ(A⊠B)=A⊡B for any A∈B(H) and B∈B(K).
Proof.
Define Φ(X)=Z∗XZ, where Z is the selection operator defined by (37) in Theorem 9. The map Φ is clearly linear and positive. The map Φ is unital since Z is an isometry (Lemma 8).
Corollary 11 provides a natural way to derive operator inequalities concerning Khatri-Rao products from existing inequalities for Tracy-Singh products.
The next result extends [16, Corollary 3] to the case of Khatri-Rao and Tracy-Singh products of operators.
Corollary 12.
Let A=A1⊕⋯⊕An and B=B1⊕⋯⊕Bn be operators in B(H) and B(K), respectively. Then, (42)Z∗A⊠B=A⊡BZ∗,A⊠BZ=ZA⊡B.
Proof.
Using the fact that EiEi∗X=Xi=XEiEi∗ and EiEj∗X=0=XEiEj∗ if i≠j, where X=X1⊕⋯⊕Xn, we compute (43)ZZ∗A⊠B=E1E1∗⋯0⋮⋱⋮0⋯EnEn∗A1⊠B⋯0⋮⋱⋮0⋯An⊠B=E1E1∗A1⊠B⋯0⋮⋱⋮0⋯EnEn∗An⊠B=A1⊠BE1E1∗⋯0⋮⋱⋮0⋯An⊠BEnEn∗=A1⊠B⋯0⋮⋱⋮0⋯An⊠BE1E1∗⋯0⋮⋱⋮0⋯EnEn∗=A⊠BZZ∗.By applying Theorem 9, we get (44)Z∗A⊠B=Z∗ZZ∗A⊠B=Z∗A⊠BZZ∗=A⊡BZ∗.Similarly, (A⊠B)Z=Z(A⊡B).
5. Positivity and Monotonicity of Khatri-Rao Products
In this section, we show that the Khatri-Rao product preserves positivity and strict positivity. It follows that operator orderings are preserved under Khatri-Rao products.
Theorem 13.
Let A∈B(H) and B∈B(K) be operator matrices. If A⩾0 and B⩾0, then A⊡B⩾0.
Proof.
It follows from the positivity of the Tracy-Singh product (Lemma 1) and Theorem 9.
The next result provides the monotonicity of Khatri-Rao product which is an extension of [9, Theorem 5] to the case of operators.
Corollary 14.
Let A1,A2∈B(H) and B1,B2∈B(K). If A1⩾A2⩾0 and B1⩾B2⩾0, then A1⊡B1⩾A2⊡B2.
Proof.
Applying Proposition 5 and Theorem 13 yields (45)A1⊡B1-A2⊡B2=A1⊡B1-A2⊡B1+A2⊡B1-A2⊡B2=A1-A2⊡B1+A2⊡B1-B2⩾0.Thus, A1⊡B1⩾A2⊡B2.
Now, we will develop the result of [9, Theorem 6] to the case of Khatri-Rao product of operators.
Theorem 15.
Let A∈B(H) and B∈B(K) be operator matrices. If A>0 and B>0, then A⊡B>0.
Proof.
The strict positivity of A and the spectral theorem imply the existence of an increasing sequence (Hn)n=1∞ of closed subspaces of H such that, for each n∈N, (46)Ax,x⩾1nx2for each x∈Hn. Let Pn be the orthogonal projection onto Hn for each n∈N. There are similar subspaces Kn and orthogonal projections Qn for the operator B. Then, for each n∈N, we have A⩾(1/n)Pn and B⩾(1/n)Qn and hence (47)A⊡B⩾1n2Pn⊡Qnby Corollary 14. Since the union of the subspaces Hn in H and of the subspaces Kn in K is dense, it follows that, for any z∈H⊗K, there is m∈N for which 〈(Pm⊡Qm)z,z〉>0. Hence, (48)A⊡Bz,z⩾1m2Pm⊡Qmz,z>0.This shows that A⊡B>0.
Corollary 16.
Let A1,A2∈B(H) and B1,B2∈B(K). If A1>A2>0 and B1>B2>0, then A1⊡B1>A2⊡B2.
Proof.
The proof is similar to that of Corollary 14. Instead of Theorem 13, we apply Theorem 15.
Finally, we mention that, by using the results in this paper, we can develop further operator identities/inequalities parallel to matrix results for Khatri-Rao products.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This work was supported by the Thailand Research Fund. The second author would like to thank the Thailand Research Fund for the financial support.
LiuS.TrenklerG.Hadamard, Khatri-Rao and other matrix products20084160177Zbl1159.15008Van LoanC. F.The ubiquitous Kronecker product20001231-28510010.1016/s0377-0427(00)00393-9MR17985202-s2.0-0034320029ZhangH.DingF.On the Kronecker products and their applications20132013829618510.1155/2013/296185MR3070498KhatriC. G.RaoC. R.Solutions to some functional equations and their applications to characterization of probability distributions196830167180MR0238416Al ZhourZ. A.KilicmanA.Extension and generalization inequalities involving the Khatri-Rao product of several positive matrices20062006218087810.1155/jia/2006/80878MR22534102-s2.0-33746878528CaoC.-G.ZhangX.YangZ.-P.Some inequalities for the Khatri-Rao product of matrices2002927628110.13001/1081-3810.1090MR19381332-s2.0-3042572723CivcivH.TaurkmenR.On the bounds for lp norms of Khatri-Rao and Tracy-Singh products of Cauchy-Toeplitz matrices2005624352FengX. X.YangZ. P.Lowner partial ordering inequalities on the Khatri-Rao product of matrices200219106110Zbl1025.15032LiuS.Matrix results on the Khatri-Rao and Tracy-Singh products19992891–326727710.1016/s0024-3795(98)10209-4MR1670989LiuS.Several inequalities involving Khatri-Rao products of positive semidefinite matrices200235417518610.1016/s0024-3795(02)00338-5MR1927654YangZ.-P.ZhangX.CaoC.-G.Inequalities involving Khatri-Rao products of Hermitian matrices200291125133MR1876518ZhangX.YangZ.-P.CaoC.-G.Matrix inequalities involving the Khatri-Rao product2002384265272MR1942656LiuS.1995Amsterdam, The NetherlandsThesis PublishersTinbergen Institute Research Series no. 106RaoC. R.RaoM. B.1998SingaporeWorld Scientific10.1142/9789812779281MR1660868FujiiJ. I.The Marcus-Khan theorem for Hilbert space operators1995413531535MR1339012VisickG.A quantitative version of the observation that the Hadamard product is a principal submatrix of the Kronecker product20003041–3456810.1016/s0024-3795(99)00187-1MR1734209