JMATH Journal of Mathematics 2314-4785 2314-4629 Hindawi Publishing Corporation 10.1155/2016/8301709 8301709 Research Article Khatri-Rao Products for Operator Matrices Acting on the Direct Sum of Hilbert Spaces http://orcid.org/0000-0001-6299-917X Ploymukda Arnon 1 http://orcid.org/0000-0002-9885-5685 Chansangiam Pattrawut 1 Meyer Ralf Department of Mathematics Faculty of Science King Mongkut’s Institute of Technology Ladkrabang Chalongkrung Rd. Bangkok 10520 Thailand kmitl.ac.th 2016 28112016 2016 02 07 2016 20 09 2016 18 10 2016 28112016 2016 Copyright © 2016 Arnon Ploymukda and Pattrawut Chansangiam. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We introduce the notion of Khatri-Rao product for operator matrices acting on the direct sum of Hilbert spaces. This notion generalizes the tensor product and Hadamard product of operators and the Khatri-Rao product of matrices. We investigate algebraic properties, positivity, and monotonicity of the Khatri-Rao product. Moreover, there is a unital positive linear map taking Tracy-Singh products to Khatri-Rao products via an isometry.

Thailand Research Fund
1. Introduction

In matrix theory, there are various matrix products which are of interest in both theory and applications, such as the Kronecker product, Hadamard product, and Khatri-Rao product; see, for example, . Denote by Mm,n(C) the set of m-by-n complex matrices and abbreviate Mn,n(C) to Mn(C). Recall that the Kronecker product of A=[aij]Mm,n(C) and BMp,q(C) is given by (1)A^B=aijBijMmp,nqC.The Hadamard product of A,BMm,n(C) is defined by the entrywise product (2)A^B=aijbijMm,nC.Now, let A and B be complex matrices partitioned into blocks Aij and Bij for each i, j (the sizes of Aij and Bij may be different). Then, the Khatri-Rao product  of A and B is defined by (3)A^B=Aij^Bijij.When A and B are nonpartitioned (i.e., each has only one block), their Khatri-Rao product is just their Kronecker product. If A and B are entrywise partitioned (i.e., each block is a 1×1 matrix), then their Khatri-Rao product is their Hadamard product. Interesting algebraic, order, and analytic properties of this product were studied in the literature; see, for example, . Their applications in statistics, computer science, and related fields can be seen, for example, in [13, 14].

The tensor product of Hilbert space operators is a natural extension of the Kronecker product to the infinite-dimensional setting. Let H, H, K, and K be Hilbert spaces. Recall that the tensor product of two operators A:HH and B:KK is the unique bounded linear operator from HK into HK such that, for all xH and yK, (4)ABxy=AxBy.

In this paper, we generalize the tensor product of operators to the Khatri-Rao product of operator matrices acting on a direct sum of Hilbert spaces. We investigate fundamental properties of this operator product. Algebraically, this product is compatible with the addition, the scalar multiplication, the adjoint operation, and the direct sum of operators. By introducing suitable operator matrices, we can prove that there is a unital positive linear map taking the Tracy-Singh product AB to the Khatri-Rao product AB. Hence, the Khatri-Rao product can be viewed as a generalization of the Hadamard product of operators. Moreover, positivity, strict positivity, and operator orderings are preserved under the Khatri-Rao product. Our result extends well-known results for Khatri-Rao products of complex matrices (see [4, 9, 15, 16]).

This paper is organized as follows. In Section 2, we provide some preliminaries about Tracy-Singh products for operators. These facts will be used in Sections 4 and 5. In Section 3, we introduce the Khatri-Rao product for operator matrices and deduce its algebraic properties. Section 4 explains how the Khatri-Rao product can be regarded as a generalization of the Hadamard product. Section 5 discusses positivity and monotonicity of Khatri-Rao products.

2. Preliminaries on Tracy-Singh Products for Operators

Throughout, let H, H, K, and K be complex separable Hilbert spaces. When X and Y are Hilbert spaces, denote by B(X,Y) the Banach space of bounded linear operators from X into Y, and abbreviate B(X,X) to B(X). If an operator AB(H) satisfies Ax,x>0, we write A>0. For self-adjoint operators A,BB(H), we write AB to mean that A-B is a positive operator, while A>B means that A-B>0.

Decompose (5)H=j=1nHj,H=i=1mHi,K=l=1qKl,K=k=1pKk,where all Hj, Hi, Kl, and Kk are Hilbert spaces. For each j, l, let Mj:HjH and Nl:KlK be the canonical embeddings. For each i and k, let Pi:HHi and Qk:KKk be the orthogonal projections. Two operators AB(H,H) and BB(K,K) can thus be represented uniquely as operator matrices (6)A=Aiji,j=1m,n,B=Bklk,l=1p,q,where Aij=PiAMjB(Hj,Hi) and Bkl=QkBNlB(Kl,Kk) for each i, j, k, and l. We define the Tracy-Singh product of A and B to be the bounded linear operator from j,l=1n,qHjKl to i,k=1m,pHiKk expressed in a block-matrix form (7)AB=AijBklklij.Basic properties of the Tracy-Singh product are listed below.

Lemma 1.

The Tracy-Singh product (A,B)AB is a bilinear map for operators. It is positive in the sense that if A0 and B0, then AB0.

3. Compatibility of Khatri-Rao Products with Algebraic Operations

In this section, we define the Khatri-Rao product for operator matrices and show that this product is compatible with certain algebraic operations of operators.

From now on, fix the following orthogonal decompositions of Hilbert spaces: (8)H=j=1nHj,H=i=1mHi,K=j=1nKj,K=i=1mKi.That is, we fix how to partition any operator matrix in B(H,H) and B(K,K). We now extend the Khatri-Rao product of matrices  to that of operators on a Hilbert space.

Definition 2.

Let A=Aiji,j=1m,nB(H,H) and B=Biji,j=1m,nB(K,K) be operators partitioned into matrices according to decomposition (8). We define the Khatri-Rao product of A and B to be the bounded linear operator from j=1nHjKj to i=1mHiKi represented by the block-matrix (9)AB=AijBiji,j=1m,n.

If both A and B are 1×1 block operator matrices, then AB is AB. When Hi=Ki=C and Hj=Kj=C for all i, j, the Khatri-Rao product is the Hadamard product of complex matrices.

Next, we shall show that the Khatri-Rao product of two linear maps induced by matrices is just the linear map induced by the Khatri-Rao product of these matrices. Recall that, for each AMm,n(C) and BMp,q(C), the induced maps, (10)LA:CnCm,xAx,LB:CqCp,yBy,are bounded linear operators. We identify CnCq with Cnq together with the canonical bilinear map (x,y)x^y for each (x,y)Cn×Cq.

Lemma 3.

For any AMm,n(C) and BMp,q(C), one has (11)LALB=LA^B.

Proof.

Recall that the Kronecker product of matrices has the following property (see, e.g., ): (12)A^BC^D=AC^BDprovided that all matrix products are well defined. It follows that, for any xCn and yCq, (13)LALBxy=LAxLBy=LAx^LBy=Ax^By=A^Bx^y=A^Bxy=LA^Bxy.The uniqueness of tensor products implies that LALB=LA^B.

Proposition 4.

For any complex matrices A=[Aij] and B=[Bij] partitioned in block-matrix form, one has (14)LALB=LA^B.

Proof.

Recall that the (i,j)th block of the matrix representation of LA is LAij. By Lemma 3, we obtain LALB=LAijLBijij=LAij^Bijij=LA^B.

The next result states that the Khatri-Rao product is bilinear and compatible with the adjoint operation.

Proposition 5.

Let AB(H,H) and B,CB(K,K) be operator matrices, and let αC. Then, (15)AB=AB,(16)AB+C=AB+AC,(17)B+CA=BA+CA,(18)αAB=αAB=AαB.

Proof.

Since A=Ajiij and B=Bjiij, we obtain (19)AB=AijBijij=AjiBjiij=AB.The fact that (B+C)ij=Bij+Cij for all i, j together with the left distributivity of the tensor product over the addition implies (20)AB+C=AijBij+Cijij=AijBij+AijCijij=AB+AC.Similarly, we obtain property (17). Since (αA)ij=αAij for all i, j, we get (21)αAB=αAijBijij=αAijBijij=αAB.Similarly, A(αB)=α(AB).

By property (15), the self-adjointness of operators is closed under taking Khatri-Rao products; that is, if A and B are self-adjoint, then so is AB. The next proposition shows that, in order to compute the Khatri-Rao product of operator matrices, we can freely merge the partition of each operator.

Proposition 6.

Let A=Aiji,j=1m,nB(H,H) and B=Biji,j=1m,nB(K,K) be operator matrices represented according to decomposition (8). We merge the partition of A to be A=Aklk,l=1r,s, where r, s are given natural numbers such that rm and sn. Here, each operator Akl is of mk×nl block in which the (g,h)th block of Akl is the (u,v)th block of A, where (22)u=g,k=1i=1k-1mi+g,k>1,k=1rmk=m,v=h,l=1j=1l-1nj+h,l>1,l=1snl=n.Similarly, we repartition B=Bklk,l=1r,s, where each operator Bkl is of mk×nl block in which the (g,h)th block of Bkl is the (u,v)th block of B. Then, (23)AB=AklBklkl=A11B11A1sB1sAr1Br1ArsBrs.That is, each (k,l)th block of AB is just AklBkl.

Proof.

Write AB=Cklk,l=1r,s, where Ckl is mk×nl block operator matrix such that the (g,h)th block of Ckl is the (u,v)th block of AB. We know that the (u,v)th block of AB is AuvBuv. Then, (24)C11=A11B11A1n1B1n1Am11Bm11Am1n1Bm1n1=A11A1n1Am11Am1n1B11B1n1Bm11Bm1n1=A11B11.Similarly, we have Ckl=AklBkl for all k=1,,r and l=1,,s.

Recall that the direct sum of AiB(Hi,Hi),i=1,,n, is defined to be the operator matrix (25)A1An=A1000A2000An.The next result shows that the Khatri-Rao product is compatible with the direct sum of operators.

Proposition 7.

For each i=1,,n, let AiB(Hi,Hi) and BiB(Ki,Ki) be compatible operator matrices. Then, (26)i=1nAii=1nBi=i=1nAiBi.

Proof.

It follows directly from Proposition 6.

In summary, the Khatri-Rao product is compatible with fundamental algebraic operations for operators.

4. The Khatri-Rao Product as a Generalization of the Hadamard Product

In this section, we explain how the Khatri-Rao product can be viewed as a generalization of the Hadamard product. To do this, we construct two isometries which identify which blocks of the Tracy-Singh product we need to get the Khatri-Rao product.

Fix a countable orthonormal basis E for H. Recall that the Hadamard product of A and B in B(H) is defined to be the operator AB in B(H) such that (27)ABe,e=Ae,eBe,efor all eE. More explicitly, it was shown in  that (28)AB=UABU,where U:HHH is the isometry defined by Ue=ee for all eE. When H=Cn and E is the standard ordered basis of Cn, the Hadamard product of two matrices reduces to the entrywise product (2).

We now extend selection matrices in  to selection operators. Fix an ordered 4-tuple (H,H,K,K) of Hilbert spaces endowed with decomposition (8). For each r=1,,m, consider the operator matrix (29)Er=Eghrg,h=1m,m:i=1mHiKii=1mHrKi,where Egh(r) is the identity operator if g=h=r and the others are zero operators. Similarly, for s=1,,n, we define the operator matrix (30)Fs=Fghsg,h=1n,n:j=1nHjKjj=1nHsKj,where Fgh(s) is the identity operator if g=h=s and the others are zero operators. Now, construct (31)Z1=E1Em,Z2=F1Fn.We call Z1 and Z2 selection operators associated with the ordered tuple (H,H,K,K). Notice that Z1 depends only on the ordered tuple (H,K) and how we decomposed H and K. The operator Z2 depends on (H,K) and how we decomposed H and K. For instance, an ordered tuple (H,H,K,K) with decompositions (32)H=H1H2H3,H=H1H2,K=K1K2K3,K=K1K2has the following selection operators: (33)Z1=IH1K100IH2K2,Z2=IH1K1000IH2K2000IH3K3.In the case of H=H and K=K, construction (31) gives (34)Z1=Z2=Z.If (Z1,Z2) is the ordered pair of selection operators associated with the ordered tuple (H,H,K,K) with decompositions given by (8), then (Z2,Z1) is the ordered pair of selection operators associated with the ordered collection (H,H,K,K) with the same decompositions.

Lemma 8.

Let Z1 and Z2 be selection operators defined by (31). Then, for i=1,2,

ZiZi=I; that is, Zi is an isometry;

0ZiZiI.

Proof.

A direct computation shows that Z1Z1=I and Z2Z2=I. We know that EiEi is an m×m block operator matrix which consists only of zero and identity operators. More precisely, the (i,i)th block of EiEi is the identity operator and EiEj=0 for ij. Then, (35)Z1Z1=E1E1E1E2E1EmE2E1E2E2E2EmEmE1EmE2EmEm=E1E1000E2E2000EmEm.Since EiEiI for all i=1,,m, we have Z1Z1I. Similarly, Z2Z2I.

Next, we relate the Khatri-Rao and the Tracy-Singh product of operators.

Theorem 9.

For any operator matrices AB(H,H) and BB(K,K), one has (36)AB=Z1ABZ2,where Z1 and Z2 are the selection operators defined by (31). If H=H and K=K, AB(H) and BB(K), then (37)AB=ZABZ,where Z is the selection operator defined by (34).

Proof.

Let B(i) denote the ith column of B for i=1,,n. Then, we have (38)Z1ABZ2=E1EmA11BA1nBAm1BAmnBF1Fn=E1EmA11BF1++A1nBFnAm1BF1++AmnBFn=E1EmA11B1A1nBnAm1B1AmnBn=A11B11A1nB1nAm1Bm1AmnBmn=AB.If H=H and K=K, then Z1=Z2 and (36) becomes (37).

We mention that Theorem 9 is an extension of both [9, Theorem 3] and result (28) in .

Remark 10.

If we partition A and B into row operator matrices, we have (39)AB=ABZ2.If both A and B are column operator matrices, then (40)AB=Z1AB.

Comparing (28) and (37), Theorem 9 shows that the Khatri-Rao product can be regarded as a generalization of the Hadamard product.

Recall that a map Φ between two C-algebras is said to be positive if Φ preserves positive elements. The map Φ is unital if Φ preserves the multiplicative identity.

Corollary 11.

There is a unital positive linear map (41)Φ:Bi,j=1n,nHiKjBi=1nHiKisuch that Φ(AB)=AB for any AB(H) and BB(K).

Proof.

Define Φ(X)=ZXZ, where Z is the selection operator defined by (37) in Theorem 9. The map Φ is clearly linear and positive. The map Φ is unital since Z is an isometry (Lemma 8).

Corollary 11 provides a natural way to derive operator inequalities concerning Khatri-Rao products from existing inequalities for Tracy-Singh products.

The next result extends [16, Corollary 3] to the case of Khatri-Rao and Tracy-Singh products of operators.

Corollary 12.

Let A=A1An and B=B1Bn be operators in B(H) and B(K), respectively. Then, (42)ZAB=ABZ,ABZ=ZAB.

Proof.

Using the fact that EiEiX=Xi=XEiEi and EiEjX=0=XEiEj if ij, where X=X1Xn, we compute (43)ZZAB=E1E100EnEnA1B00AnB=E1E1A1B00EnEnAnB=A1BE1E100AnBEnEn=A1B00AnBE1E100EnEn=ABZZ.By applying Theorem 9, we get (44)ZAB=ZZZAB=ZABZZ=ABZ.Similarly, (AB)Z=Z(AB).

5. Positivity and Monotonicity of Khatri-Rao Products

In this section, we show that the Khatri-Rao product preserves positivity and strict positivity. It follows that operator orderings are preserved under Khatri-Rao products.

Theorem 13.

Let AB(H) and BB(K) be operator matrices. If A0 and B0, then AB0.

Proof.

It follows from the positivity of the Tracy-Singh product (Lemma 1) and Theorem 9.

The next result provides the monotonicity of Khatri-Rao product which is an extension of [9, Theorem 5] to the case of operators.

Corollary 14.

Let A1,A2B(H) and B1,B2B(K). If A1A20 and B1B20, then A1B1A2B2.

Proof.

Applying Proposition 5 and Theorem 13 yields (45)A1B1-A2B2=A1B1-A2B1+A2B1-A2B2=A1-A2B1+A2B1-B20.Thus, A1B1A2B2.

Now, we will develop the result of [9, Theorem 6] to the case of Khatri-Rao product of operators.

Theorem 15.

Let AB(H) and BB(K) be operator matrices. If A>0 and B>0, then AB>0.

Proof.

The strict positivity of A and the spectral theorem imply the existence of an increasing sequence (Hn)n=1 of closed subspaces of H such that, for each nN, (46)Ax,x1nx2for each xHn. Let Pn be the orthogonal projection onto Hn for each nN. There are similar subspaces Kn and orthogonal projections Qn for the operator B. Then, for each nN, we have A(1/n)Pn and B(1/n)Qn and hence (47)AB1n2PnQnby Corollary 14. Since the union of the subspaces Hn in H and of the subspaces Kn in K is dense, it follows that, for any zHK, there is mN for which (PmQm)z,z>0. Hence, (48)ABz,z1m2PmQmz,z>0.This shows that AB>0.

Corollary 16.

Let A1,A2B(H) and B1,B2B(K). If A1>A2>0 and B1>B2>0, then A1B1>A2B2.

Proof.

The proof is similar to that of Corollary 14. Instead of Theorem 13, we apply Theorem 15.

Finally, we mention that, by using the results in this paper, we can develop further operator identities/inequalities parallel to matrix results for Khatri-Rao products.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

This work was supported by the Thailand Research Fund. The second author would like to thank the Thailand Research Fund for the financial support.

Liu S. Trenkler G. Hadamard, Khatri-Rao and other matrix products International Journal of Computer and System Sciences 2008 4 160 177 Zbl1159.15008 Van Loan C. F. The ubiquitous Kronecker product Journal of Computational and Applied Mathematics 2000 123 1-2 85 100 10.1016/s0377-0427(00)00393-9 MR1798520 2-s2.0-0034320029 Zhang H. Ding F. On the Kronecker products and their applications Journal of Applied Mathematics 2013 2013 8 296185 10.1155/2013/296185 MR3070498 Khatri C. G. Rao C. R. Solutions to some functional equations and their applications to characterization of probability distributions Sankhya 1968 30 167 180 MR0238416 Al Zhour Z. A. Kilicman A. Extension and generalization inequalities involving the Khatri-Rao product of several positive matrices Journal of Inequalities and Applications 2006 2006 21 80878 10.1155/jia/2006/80878 MR2253410 2-s2.0-33746878528 Cao C.-G. Zhang X. Yang Z.-P. Some inequalities for the Khatri-Rao product of matrices Electronic Journal of Linear Algebra 2002 9 276 281 10.13001/1081-3810.1090 MR1938133 2-s2.0-3042572723 Civciv H. Taurkmen R. On the bounds for lp norms of Khatri-Rao and Tracy-Singh products of Cauchy-Toeplitz matrices Selcuk Journal of Applied Mathematics 2005 6 2 43 52 Feng X. X. Yang Z. P. Lowner partial ordering inequalities on the Khatri-Rao product of matrices Gongcheng Shuxue Xuebao 2002 19 106 110 Zbl1025.15032 Liu S. Matrix results on the Khatri-Rao and Tracy-Singh products Linear Algebra and Its Applications 1999 289 1–3 267 277 10.1016/s0024-3795(98)10209-4 MR1670989 Liu S. Several inequalities involving Khatri-Rao products of positive semidefinite matrices Linear Algebra and Its Applications 2002 354 175 186 10.1016/s0024-3795(02)00338-5 MR1927654 Yang Z.-P. Zhang X. Cao C.-G. Inequalities involving Khatri-Rao products of Hermitian matrices The Korean Journal of Computational & Applied Mathematics 2002 9 1 125 133 MR1876518 Zhang X. Yang Z.-P. Cao C.-G. Matrix inequalities involving the Khatri-Rao product Archivum Mathematicum 2002 38 4 265 272 MR1942656 Liu S. Contributions to Matrix Calculus and Applications in Econometrics 1995 Amsterdam, The Netherlands Thesis Publishers Tinbergen Institute Research Series no. 106 Rao C. R. Rao M. B. Matrix Algebra and Its Applications to Statistics and Econometrics 1998 Singapore World Scientific 10.1142/9789812779281 MR1660868 Fujii J. I. The Marcus-Khan theorem for Hilbert space operators Mathematica Japonica 1995 41 3 531 535 MR1339012 Visick G. A quantitative version of the observation that the Hadamard product is a principal submatrix of the Kronecker product Linear Algebra and Its Applications 2000 304 1–3 45 68 10.1016/s0024-3795(99)00187-1 MR1734209