We establish an asymptotic approach for checking the appropriateness of an assumed multivariate spatial regression model by considering the set-indexed partial sums process of the least squares residuals of the vector of observations. In this work, we assume that the components of the observation, whose mean is generated by a certain basis, are correlated. By this reason we need more effort in deriving the results. To get the limit process we apply the multivariate analog of the well-known Prohorov’s theorem. To test the hypothesis we define tests which are given by Kolmogorov-Smirnov (KS) and Cramér-von Mises (CvM) functionals of the partial sums processes. The calibration of the probability distribution of the tests is conducted by proposing bootstrap resampling technique based on the residuals. We studied the finite sample size performance of the KS and CvM tests by simulation. The application of the proposed test procedure to real data is also discussed.
Ministry of Research, Technology and Higher Education, Indonesia1. Introduction
As mentioned in the literatures of model checks for multivariate regression, the appropriateness of an assumed model is mostly verified by analyzing the least squares residual of the observations; see, for example, Zellner [1], Christensen [2], pp. 1–22, Anderson [3], pp. 187–191, and Johnson and Wichern [4], pp. 395–398. A common feature of these works is the comparison between the length of the matrix of the residuals under the null hypothesis and that of the residuals under a proposed alternative.
Instead of considering the residuals directly MacNeill [5] and MacNeill [6] proposed a method in model check for univariate polynomial regression based on the partial sums process of the residuals. These popular approaches are generalized to the spatial case by MacNeill and Jandhyala [7] for ordinary partial sums and Xie and MacNeill [8] for set-indexed partial sums process of the residuals. Bischoff and Somayasa [9] and Somayasa et al. [10] derived the limit process in the spatial case by a geometric method generalizing a univariate approach due to Bischoff [11] and Bischoff [12]. These results can be used to establish asymptotic test of Cramér-von Mises and Kolmogorov-Smirnov type for model checks and change-point problems. Model checks for univariate regression with random design using the empirical process of the explanatory variable marked by the residuals was established in Stute [13] and Stute et al. [14]. In the papers mentioned above the limit processes were explicitly expressed as complicated functions of the univariate Brownian motion (sheet).
The purpose of the present article is to study the application of set-indexed partial sums technique to simultaneously check the goodness-of-fit of a multivariate spatial linear regression defined on high-dimensional compact rectangle. In contrast to the normal multivariate model studied in the standard literatures such as in Christensen [2], Anderson [3], and Johnson and Wichern [4] or in the references of model selection such as in Bedrick and Tsai [15] and Fujikoshi and Satoh [16], in this paper we will consider a multivariate regression model in which the components of the mean of the response vector are assumed to lie in different spaces and the underlying distribution model of the vector of random errors is unknown.
To see the problem in more detail let n1≥1,…,nd≥1 be fixed. Let a p-dimensional random vector Z≔(Zi)i=1p be observed independently over an experimental design given by a regular lattice: (1)Ξn1⋯nd≔jknkk=1d∈Id:1≤jk≤nk,k=1,…,d,where Id≔∏k=1d0,1 is the d-dimensional unit cube. Let g≔(gi)i=1p be the true but unknown Rp-valued regression function on Id which represents the mean function of the observations. Let Zj1⋯jd≔(Zi,j1⋯jd)i=1p and g≔(gi,j1⋯jd)i=1p be the observation and the corresponding mean in the experimental condition (jk/nk)k=1d. Under the null hypothesis H0 we assume that Zj1⋯jd follows a multivariate linear model. That is, we assume a model (2)H0:Zj1⋯jd=∑w=1diβiwfiwjknkk=1di=1p+Ej1⋯jd,1≤j1≤n1,…,1≤jd≤nd,where, for i=1,…,p, βi≔(βiw)w=1di∈Rdi is a di-dimensional vector of unknown parameters; fi≔(fiw)w=1di is a di-dimensional vector of known regression functions whose components are assumed to be square integrable with respect to the Lebesgue measure λId on Id, that is, fiw∈L2(λId), for all i and w. Ej1⋯jd≔(εi,j1⋯jd)i=1p is the mutually independent p-dimensional vector of random errors defined on a common probability space (Ω,F,P). We assume that, for all 1≤j1≤n1,…,1≤jd≤nd, E(Ej1⋯jd)=0∈Rp, and Cov(Ej1⋯jd)=Σ=(σuv)u,v=1p,p. Let Zn1⋯nd≔(Zj1⋯jd)j1=1,…,jd=1n1,…,nd be the p-dimensional pyramidal array of random observations and let En1⋯nd≔(Ej1⋯jd)j1=1,…,jd=1n1,…,nd be the p-dimensional pyramidal array of random errors taking values in the Euclidean space ∏i=1pRn1×⋯×nd. Then under H0 the observations can be represented by (3)Zn1⋯nd=∑w=1diβiwfiwΞn1⋯ndi=1p+En1⋯nd,where fiw(Ξn1⋯nd)≔fiw(jk/nk)k=1dj1=1,…,jd=1n1,…,nd. Under the alternative H1 a multivariate nonparametric regression model (4)Zn1⋯nd=gΞn1⋯nd+En1⋯ndis assumed, where g(Ξn1⋯nd)≔g(jk/nk)k=1dj1=1,…,jd=1n1,…,nd∈∏i=1pRn1×⋯×nd. By applying the similar argument as in Christensen [2] and Johnson and Wichern [4], the p-dimensional array of the least squares residuals of the observations is given by the following component-wise projection: (5)Rn1⋯nd≔rj1⋯jdj1,…,jd=1n1,…,nd≔Zn1⋯nd-pr∏i=1pWi,n1⋯ndZn1⋯nd,with rj1⋯jd≔(ri,j1⋯jd)i=1p, for 1≤jk≤nk, and k=1,…,d. Thereby, for i=1,…,p, we define Wi,n1⋯nd≔[fi1(Ξn1⋯nd),…,fidi(Ξn1⋯nd)] as the subspace of Rn1×⋯×nd spanned by the arrays {fi1(Ξn1⋯nd),…,fidi(Ξn1⋯nd)}. It is worth mentioning that the Euclidean space Rn1×⋯×nd is furnished with the inner product denoted by 〈·,·〉Rn1×⋯×nd and defined by(6)An1⋯nd,Bn1⋯ndRn1×⋯×nd≔∑j1=1n1⋯∑jd=1ndaj1⋯jdbj1⋯jd,for every An1⋯nd≔(aj1⋯jd)j1=1,…,jd=1n1,…,nd, and Bn1⋯nd≔(bj1⋯jd)j1=1,…,jd=1n1,…,nd∈Rn1×⋯×nd.
Next we define the set-indexed partial sums operator. Let A be the family of convex subset of Id, and let ηλId be the Lebesgue pseudometric on A defined by ηλId(A1,A2)≔λId(A1ΔA2), for A1,A2∈A. Let C(A) be the set of continuous functions on A under ηλId. We embed the array of the residual Rn1⋯nd into a p-dimensional stochastic process indexed by A by using the component-wise set-indexed partial sums operator (7)Vn1⋯nd:∏i=1pRn1×⋯×nd⟼CpA≔∏i=1pCA,such that, for any B∈A,(8)Vn1⋯ndRn1⋯ndB≔∑j1=1n1⋯∑jd=1ndn1⋯ndλIdB∩Cj1⋯jdrj1⋯jd,where, for 1≤j1≤n1,…,1≤jd≤nd, Cj1⋯jd≔∏k=1d((jk-1)/nk,jk/nk]. Let us call this process the p-dimensional set-indexed least squares residual partial sums process. The space Cp(A) is furnished with the uniform topology induced by the metric φ defined by (9)φu,w≔∑i=1pui-wiA=∑i=1psupA∈AuiA-wiA, for u≔(ui)i=1p and w≔(wi)i=1p∈Cp(A).
We notice that, in the works of Bischoff and Somayasa [9], Bischoff and Gegg [17], and Somayasa and Adhi Wibawa [18], the limit process of the partial sums process of the least squares residuals has been investigated by applying the existing geometric method of Bischoff [11, 12]. However, the method becomes not applicable anymore in deriving the limit process of Vn1⋯nd(Rn1⋯nd) as the dimension of Wi,n1⋯nd varies. Therefore, in this work, we attempt to adopt the vectorial analog of Prohorov’s theorem; see, for example, Theorem 6.1 in Billingsley [19] for obtaining the limit process. For our result we need to extend the ordinary partial sums formula to p-dimensional case defined on Id as follows. Let K≔{1,2,…,d} and CkK be the set of all k-combinations of the set K, with k=1,…,d. For a chosen value of k, we denote the ith element of CkK by a k-tuple (i(k1),i(k2),…,i(kk)), for 1≤i≤CkK, where CkK is the number of elements of CkK which is clearly given by (10)CkK=dd-1d-2⋯d-k+1kk-1k-2⋯1.For example, let K={1,2,3}. Then, for k=1, we denote the elements of C1K as 1(k1)≔1, 2(k1)≔2, and 3(k1)≔3. In a similar way, we denote the elements of C2K which consists of {(1,2),(1,3),(2,3)}, respectively, by (1(k1),1(k2))≔(1,2), (2(k1),2(k2))≔(1,3), and (3(k1),3(k2))≔(2,3). Finally the element of C3K={(1,2,3)} is sufficiently written by (1(k1),1(k2),1(k3)). Hence the p-dimensional ordinary partial sums operator transforms any p-dimensional array An1⋯nd=(aj1⋯jd)j1=1,…,jd=1n1,…,nd∈∏i=1pRn1×⋯×nd to a continuous function on Id defined by(11)Tn1⋯ndAn1⋯ndt≔1n1⋯nd∑j1=1⋯jd=1n1t1⋯ndtdaj1⋯jd+∑k=1d∑i=1C2K∏u=1ktiku-nikutikunikun1⋯ndn1⋯nd/nik1⋯nikd∑j1=1,…,jd=1/jik1,…,jikdn1t1,…,ndtdaj1⋯jd∣jik1=nik1tik1+1,…,jikd=nikdtikd+1,for every t≔(t1,…,td)⊤∈Id, where for positive integers bu,bu+1,…,bu+m∈Z+ we define a notation (12)aj1⋯jd∣ju=bu,…,ju+m=bu+m≔aj1,…,ju-1,bu,…,bu+m,ju+m+1,…,jd.It is clear that the partial sums process of the residuals obtained using (11) is a special case of (8) since for every t≔(t1,…,td)⊤∈Id it holds (13)Tn1⋯ndAn1⋯ndt=Vn1⋯ndAn1⋯nd∏k=1d0,tk.
It is worth noting that the extension of the study from univariate to multivariate model and also the expansion of the dimension of the lattice points are strongly motivated by the prediction problem in mining industry and geosciences. As for an example recently Tahir [20] presented data provided by PT Antam Tbk (a mining industry in Southeast Sulawesi). The data consist of a joint measurement of the percentage of several chemical elements and substances such as Ni, Co, Fe, MgO, SiO2, and CaO which are recorded in every point of a three-dimensional lattice defined over the exploration region of the company. Hence, by the inherent existence of the correlation among the variables, the statistical analysis for the involved variables must be conducted simultaneously.
There have been many methods proposed in the literatures for testing H0. Most of them have been derived for the case of W1=⋯=Wp under normally distributed random error. Generalized likelihood ratio test which has been leading to Wilk’s lambda statistic or variant of it can be found in Zellner [1], Christensen [2], pp. 1–22, Anderson [3], pp. 187–191, and Johnson and Wichern [4], pp. 395–398. Mardia and Goodall [21] derived the maximum likelihood estimation procedure for the parameters of the general normal spatial multivariate model with stationary observations. This approach can be straightforwardly extended for obtaining the associated likelihood ratio test in model check for the model. Unfortunately, in the practice especially when dealing with mining data, normal distribution is sometimes found to be not suitable for describing the distribution model of the observations, so that the test procedures mentioned above are consequently no longer applicable. Asymptotic method established in Arnold [22] for multivariate regression with W1=⋯=Wp can be generalized in such a way that it is valid for the general model. As a topic in statistics it must be well known. However, we cannot find literatures where the topic has been studied.
The rest of the paper is organized as follows. In Section 2 we show that when H0 is true Σ-1/2Vn1⋯nd(Rn1⋯nd) converges weakly to a projection of the p-dimensional set-indexed Brownian sheet. The limit process is shown to be useful for testing H0 asymptotically based on the Kolmogorov-Smirnov (KS-test) and Cramér-von Mises (CvM-test) functionals of the set-indexed p-dimensional least squares residual partial sum processes, defined, respectively, by (14)KSn1⋯nd,A≔supA∈AΣ-1/2Vn1⋯ndRn1⋯ndARpCvMn1⋯nd,A≔1n1⋯nd∑A∈AΣ-1/2Vn1⋯ndRn1⋯ndARp2.For both tests the rejection of H0 is for large value of the KS and CvM statistics, respectively. Under localized alternative the above sequence of random processes converges weakly to the above limit process with an additional deterministic trend (see Section 3). In Section 4, we define a consistent estimator for Σ. In Section 5 we investigate a residual based bootstrap method for the calibration of the tests. Monte Carlo simulation for the purpose of studying the finite sample behavior of the KS and CvM tests is reported in Section 6. Application of the test procedure in real data is presented in Section 7. The paper is closed in Section 8 with conclusion and some remarks for future research. Auxiliary results needed for deriving the limit process are presented in the appendix. We note that all convergence results derived throughout this paper which hold for n1,…,nd simultaneously go to infinity, that is, for nk→∞, for all k=1,…,d; otherwise they will be stated in some way. The notion of convergence in distribution and convergence in probability will be conventionally denoted by →D and →P, respectively.
Let B≔{B(A):A∈A} be the one-dimensional set-indexed Brownian sheet having sample path in C(A). We refer the reader to Pyke [23], Bass and Pyke [24], and Alexander and Pyke [25] for the definition and the existence of B. Let us consider a subspace of C(A) which is closely related to B, defined by(15)HB≔hv:A⟶R,∃v∈L2λId, s.t. hvA≔∫Avtλdt.Under the inner product and the norm defined by (16)hv,hwHB≔∫IdvtwtλIdt,hv2≔∫Idv2tλIdt,it is clear that HB and L2(λId) are isometric. For our result we need to define subspaces Wi and WiHB associated with the regression functions fi1,…,fidi, where Wi≔[fi1,…,fidi]⊂L2(λId) and WiHB≔[hfi1,…,hfidi]⊂HB, for i=1,…,p.
Now we are ready to state the limit process of the sequence of p-dimensional set-indexed residual partial sums processes for the model specified under H0.
Theorem 1.
For i=1,…,p, let {fi1,…,fidi} be an orthonormal basis (ONB) of Wi. We assume that Wi⊆Wi+1, for i=1,…,p-1. Let Bp≔{(Bi(A))i=1p:A∈A} be the p-dimensional set-indexed Brownian sheet with the covariance function Cov(Bp(A),Bp(A′))≔λId(A∩A′)Ip, for A,A′∈A, where Ip is the p×p identity matrix. Suppose {fi1,…,fidi} are in C(Id)∩BVH(Id), where C(Id) is the space of continuous functions on Id (see Definition A.4 for the definition of BVH(Id)). Then under H0 it holds that (17)Σ-1/2Vn1⋯ndRn1⋯nd⟶DBp,fH0≔Bp-pr∏i=1pWiHB∗Bp,where (18)pr∏i=1pWiHB∗Bp≔prWiHB∗Bii=1p.Thereby prWiHB∗ is a projector such that, for every u∈C(A) and A∈A, (19)prWiHB∗uA≔∑j=1dihfij,uhfijA,wherehfij,u≔∫IdRfijtdut.For t≔(tk)k=1d∈Id, we set u(t) for u(∏k=1d[0,tk]), and ∫(R) stands for the integral in the sense of Riemann-Stieltjes. Moreover, the limit process Bp,fH0 is a centered Gaussian process with the covariance function given by (20)KA,C≔diagλA∩C-∑j=1d1hf1jAhf1jC,…,λA∩C-∑j=1dphfpjAhfpjC.
Proof.
By applying the linear property of Vn1⋯nd and Lemma C.2, we have under H0,(21)Vn1⋯ndRn1⋯nd=Vn1⋯ndEn1⋯nd-pr∏i=1pWi,n1⋯ndHBVn1⋯ndEn1⋯nd.It can be shown by extending the uniform central limit theorem studied in Pyke [23], Bass and Pyke [24], and Alexander and Pyke [25] to its vectorial analog that the term Σ-1/2Vn1⋯nd(En1⋯nd) on the right-hand side of (21) converges weakly to Bp. Therefore we only need to show that the second term satisfies the weak convergence: (22)Zn1⋯nd≔Σ-1/2pr∏i=1pWi,n1⋯ndHBVn1⋯ndEn1⋯nd⟶DU≔pr∏i=1pWiHB∗Bp,where U is a p-dimensional centered Gaussian process with the covariance matrix given by (23)KUA1,A2=diag∑j=1d1hf1jA1hf1jA2,…,∑j=1dphfpjA1hfpjA2,for A1,A2∈A.By Prohorov’s theorem it is sufficient to show that, for any finite collection of convex sets A1,…,Ar in A and real numbers c1,…,cr, with r≥1, it holds that (24)∑k=1rckZn1⋯ndAk⟶D∑k=1rckUAk, where the left-hand side has the covariance which can be expressed as (25)Cov∑k=1rckZn1⋯ndAk=∑k=1r∑l=1rckclΣ-1/2EBiAkBjAli,j=1pΣ-1/2,where Bi(Ak)≔prWi,n1⋯ndHBVn1⋯nd(εi,n1⋯nd)(Ak), for i=1,…,p and k=1,…,r. Let k and l be fixed. Then by a simultaneous application of the definition of prWi,n1⋯ndHB (see Lemma C.3), (11), the definition of the Riemann-Stieljes integral (cf. Stroock [26], pp. 7–17), and the independence of the array {εi,j1⋯jd:1≤jk≤nk}, we further get (26)EBiAkBjAl=∑w=1di∑w′=1dj∫Aks~iwn1⋯ndtλIddt∫Als~jw′n1⋯ndtλIddtE∫IdRs~iwn1⋯ndtdVn1⋯ndεi,n1⋯ndt∫IdRs~jw′n1⋯ndtdVn1⋯ndεj,n1⋯ndt=∑w=1di∑w′=1dj∫Aks~iwn1⋯ndtλIddt∫Als~jw′n1⋯ndtλIddtσijn1⋯nd∑j1=1,…,jd=1n1,…,nds~iwn1⋯ndj1n1,…,jdnds~jw′n1⋯ndj1n1,…,jdnd.By recalling Lemma C.3 the last expression clearly converges to (27)σij∑w=1di∑w′=1djfiw,fjw′L2λIdhfiwAkhfjw′Al=σij∑w=1dihfiwAkhfjwAl.Hence the matrix E(Bi(Ak)Bj(Al))i,j=1p converges element wise to the symmetric matrix which can be represented as Σ⊙D, for a matrix D defined by (28)D≔κ11Ak,Alκ12Ak,Al⋯κ1pAk,Alκ21Ak,Alκ22Ak,Al⋯κ2pAk,Al⋮⋮⋮⋮κp1Ak,Alκp2Ak,Al⋯κppAk,Alwith κij(Ak,Al)≔∑w=1dmin{i,j}hfiw(Ak)hfjw(Al), for i,j=1,…,p. Thereby ⊙ denotes the Hadarmard product defined, for example, in Magnus and Neudecker [27], pp. 53-54. Since Σ-1/2(Σ⊙D)Σ-1/2=D⊙Ip, we successfully have shown that Cov∑k=1rckZn1⋯ndAk converges to the following general linear combination:(29)∑k=1r∑l=1rckcldiag∑w=1d1hf1wAkhf1wAl,…,∑w=1dphfpwAkhfpwAl,which is actually the covariance of ∑k=1rckU(Ak). Next we observe that, by applying the definition of Vn1⋯nd and the definition of the Riemann-Stieltjes integral, we can also write ∑k=1rckZn1⋯nd(Ak) as follows: (30)∑k=1rckZn1⋯ndAk=∑j1=1n1⋯∑jd=1ndΣ-1/2∑k=1rΛij1n1,…,jdnd;Aki=1p,where (31)Λij1n1,…,jdnd;Ak≔∑w=1dickn1⋯nds~iwn1⋯ndj1n1,…,jdndεi,j1⋯jdhs~iwn1⋯ndAk.Let Γf≔max1≤w≤di;1≤i≤pfiw∞, M≔max1≤k≤rck, and Σ-1/2 be the Euclidean norm of Σ-1/2. Then by considering the stochastic independence of the array of the p-vector of the random errors, it holds that (32)0≤LZn1⋯ndϵ≔∑j1=1n1⋯∑jd=1ndE∑k=1rΛij1n1,…,jdnd;Aki=1pRp21∑k=1rΛij1n1,…,jdnd;Aki=1pRp2≥ϵ≤rM2dpΓf22Σ-122EEj1⋯jdRp21Ej1⋯jdRp≥ϵn1⋯ndrMdpΓf2∀ϵ>0,in which by the well-known bounded convergence theorem (cf. Athreya and Lahiri [28], pp. 57-58) the last term converges to zero. Thus the Lindeberg condition is satisfied. Therefore by the Lindeberg-Levy multivariate central limit theorem studied, for example, in van der Vaart [29], pp. 16, it can be concluded that ∑k=1rckZn1⋯nd(Ak) converges in distribution to ∑k=1rckU(Ak), where ∑k=1rckU(Ak) has the p-variate normal distribution with mean zero and the covariance given by (29).
The tightness of Zn1⋯nd can be shown as follows. By the definition Zn1⋯nd can also be expressed as (33)Zn1⋯nd=Σ-1/2pr∏i=1pWi,n1⋯ndHBΣ1/2Σ-1/2Vn1⋯ndEn1⋯nd.Since Σ-1/2Vn1⋯nd(En1⋯nd) is tight, hence by recalling Lemma 1 in Billingsley [19], pp. 38–40, it is sufficient to show that the mapping Σ-1/2pr∏i=1pWi,n1⋯ndHBΣ1/2 is continuous on Cp(A) for every n1≥1,…,nd≥1. Proposition C.4 finishes the proof.
Corollary 2.
By Theorem 1 and the well-known continuous mapping theorem (cf. Theorem 5.1 in Billingsley [19]) the distribution of the statistics KSn1⋯nd,A and CvMn1⋯nd,A can be approximated, respectively, by those of (34)KS≔supA∈ABp-pr∏i=1pWiHB∗BpARpCvM≔∫IdBp-pr∏i=1pWiHB∗BpARp2dA.Let t~1-α and c~1-α be the (1-α)th quantile of the distributions of KS and CvM, respectively. When KSn1⋯nd,A is used, H0 will be rejected at level α if and only if KSn1⋯nd,A≥t~1-α. Likewise if CvMn1⋯nd,A is used, then H0 will be rejected at level α if and only if CvMn1⋯nd,A≥c~1-α.
The test procedures derived above are consistent in the sense of Definition 11.1.3 in Lehmann and Romano [30]. That is, the probability of rejection of H0 under the competing alternative converges to 1. As an immediate consequence we cannot observe the performance of the tests when the model moves away from H0. Therefore, to be able to investigate the behavior of the tests, we consider the localized model defined as follows:(35)Zn1⋯nd=1n1⋯ndgΞn1⋯nd+En1⋯nd.When H0 is true we get the similar least squares residuals as given in Section 1. Therefore, observing Model (35), the test problem will not be altered.
In the following theorem, we present the limit process of the p-dimensional set-indexed partial sums process of the residuals under H1 associated with Model (35).
Theorem 3.
For i=1,…,p, let {fi1,…,fidi} be an ONB of Wi with Wi⊆Wi+1, for i=1,…,p-1. If g=(gi)i=1p∈∏i=1pBVV(Id) and {fi1,…,fidi}∈C(Id)∩BVH(Id) (see Definition A.3 for the notion of BVV(Id)), then, observing (35), we have under H1 that (36)Σ-1/2Vn1⋯ndRn1⋯nd⟶DΣ-1/2Ip-pr∏i=1pWiHB∗hg+Bp,fH0,where hg≔(hgi)i=1p:A→Cp(A), with hgi(A)≔∫Agi(t)λId(dt).
Proof.
Considering the linearity of Vn1⋯nd and Lemma C.2, when H1 is true we have (37)Σ-1/2Vn1⋯ndRn1⋯nd=Σ-1/21n1⋯ndVn1⋯ndgΞn1⋯nd-Σ-1/2pr∏i=1pWi,n1⋯ndHB1n1⋯ndVn1⋯ndgΞn1⋯nd+Σ-1/2Vn1⋯ndEn1⋯nd-Σ-1/2pr∏i=1pWi,n1⋯ndHBVn1⋯ndEn1⋯nd.Since, for i=1,…,p, gi is in BVV(Id), it can be shown that (1/n1⋯nd)Vn1⋯nd(g(Ξn1⋯nd))(B) converges uniformly to hgB, for every B∈A. Also the last two terms on the right-hand side of the preceding equation converge in distribution to Bp,fH0 by Theorem 1. Thus to the rest we only need to show that pr∏i=1pWi,n1⋯ndHB(1/n1⋯nd)Vn1⋯nd(gn1⋯nd) converges to pr∏i=1pWiHBBV(Id)∗hg. By the definition of component-wise projection we have (38)pr∏i=1pWi,n1⋯ndHB1n1⋯ndVn1⋯ndgΞn1⋯nd=prWi,n1⋯ndHB1n1⋯ndVn1⋯ndigiΞn1⋯ndi=1p=∑j=1di∫IdRs~ijn1⋯ndtd1n1⋯ndVn1⋯ndigiΞn1⋯ndths~ijn1⋯ndi=1p=∑j=1di1n1⋯nd∑j1=1n1⋯∑jd=1nds~ijn1⋯ndj1n1,…,jdndgij1n1,…,jdndhs~ijn1⋯ndi=1p.The right-hand side of the last expression is obtained directly from the definition of the ordinary partial sums (11) and the definition of the Riemann-Stieltjes integral on Id. Since s~ij(n1⋯nd) converges uniformly to fij and gi has bounded variation on Id for all i and j, then the last expression clearly converges component-wise to ∑j=1di∫Id(R)fij(t)dhgi(t)hfiji=1p, which can be written as pr∏i=1pWiHB∗hg. We are done.
Corollary 4.
By Theorem 3 the power function of the KS and CvM tests at a level α can now be approximated by computing the probabilities of the form (39)PsupA∈AΣ-1/2hg-pr∏i=1pWiHBBVId∗hg+Bp,fH0Rp≥t~1-αP∫IΣ-1/2hg-pr∏i=1pWiHBBVId∗hg+Bp,fH0Rp2dA≥c~1-α,for a fixed g∈∏i=1pBVV(Id), respectively. In Section 5 we investigate the empirical power functions of the KS and CvM tests by simulation.
4. Estimating the Population Covariance Matrix
If the covariance matrix Σ is unknown, as it usually is, it is impossible to use KSn1⋯nd;A and CvMn1⋯nd;A in practice. What we propose to do is to employ a consistent estimate of Σ. We need some further notations for expressing the residuals of the model. For i=1,…,p, let Zi(n1⋯nd), gi(n1⋯nd), and εi(n1⋯nd) be (n1⋯nd)-dimensional column vectors defined by (40)Zin1⋯nd≔Zij1n1,…,jdndj1=1,…,jd=1n1,…,nd∈Rn1⋯ndgin1⋯nd≔gij1n1,…,jdndj1=1,…,jd=1n1,…,nd∈Rn1⋯ndεin1⋯nd≔εij1n1,…,jdndj1=1,…,jd=1n1,…,nd∈Rn1⋯nd.Furthermore, let Z(n1⋯nd), g(n1⋯nd), and E(n1⋯nd) be (n1⋯nd)×p-dimensional matrices whose ith column is given, respectively, by the column vectors Zi(n1⋯nd), gi(n1⋯nd), and εi(n1⋯nd), i=1,…,p. Then Model (35) can also be represented as follows:(41)Zn1⋯nd=1n1⋯ndgn1⋯nd+En1⋯nd,where, for u,v=1,…,p, Cov(εu(n1⋯nd),εv(n1⋯nd))=σuvIn1⋯nd, with In1⋯nd being the identity matrix in R(n1⋯nd)×(n1⋯nd).
Associated with the subspace Wi,n1⋯nd we define the design matrix Xi(n1⋯nd) as an element of R(n1⋯nd)×di whose uth column is given by the (n1⋯nd)-dimensional column vector: (42)fiuj1n1,…,jdndj1=1,…,jd=1n1,…,nd∈Rn1⋯nd,u=1,…,di,i=1,…,p.We denote the column space of Xi(n1⋯nd) by C(Xi(n1⋯nd))⊂R(n1⋯nd)×di for the sake of brevity. We also define the column-wise projection of any matrix U(n1⋯nd)≔(U1(n1⋯nd),…,Up(n1⋯nd)) in Rn1⋯nd×p into the product space ∏i=1pC(Xi(n1⋯nd)) by (43)pr∏i=1pCXin1⋯ndUn1⋯nd≔prCX1n1⋯ndU1n1⋯nd,…,prCXpn1⋯ndUpn1⋯nd.A reasonable estimator of the covariance matrix Σ is denoted by Σ^n1⋯nd, defined by (44)Σ^n1⋯nd≔1n1⋯ndpr∏i=1pCXin1⋯nd⊥Zn1⋯nd⊤pr∏i=1pCXin1⋯nd⊥Zn1⋯nd,where (45)pr∏i=1pCXin1⋯nd⊥≔In1⋯nd-pr∏i=1pCXin1⋯ndconstitutes the component-wise orthogonal projector into the orthogonal complement of the product space ∏i=1pC(Xin1⋯nd).
Zellner [1] and Arnold [22] investigated the consistency of Σ^n1⋯nd toward Σ in the case of the multivariate regression model with W1=⋯=Wp. Some difficulties appear when the situation is extended to the case of W1≠⋯≠Wp, since it involves the problem of finding the limit of matrices with the components given by inner products of two vectors.
Theorem 5.
Suppose the localized model (41) is observed. If H0 is true, then, under the conditions of Theorem 1, we have Σ^n1⋯nd→PΣ.
Proof.
If H0 is true, it can be easily shown that (46)n1⋯ndΣ^n1⋯nd=pr∏i=1pCXin1⋯nd⊥En1⋯nd⊤pr∏i=1pCXin1⋯nd⊥En1⋯nd.For technical reason we assume without lost of generality that Xi(n1⋯nd) is an orthogonal matrix, for i=1,…,p. Hence we further get the representation (47)n1⋯ndΣ^n1⋯nd=εun1⋯nd⊤εvn1⋯ndu,v=1p,p-Xvn1⋯nd⊤εun1⋯nd⊤Xvn1⋯nd⊤εvn1⋯ndu,v=1p,p-Xun1⋯nd⊤εun1⋯nd⊤Xun1⋯nd⊤εvn1⋯ndu,v=1p,p+Xun1⋯nd⊤εun1⋯nd⊤Xvn1⋯nd⊤εvn1⋯ndu,v=1p,p.Since εu(j1/n1,…,jd/nd)εv(j1/n1,…,jd/nd)u,v=1p,p are independent and identically distributed random matrices with mean Σ, by the well-known weak law of large numbers, we get (48)1n1⋯nd∑j1=1n1⋯∑jd=1ndεuj1n1,…,jdndεvj1n1,…,jdndu,v=1p,p⟶PΣ.Note that in the practice we consider the polynomial regression model. Hence, for every v=1,…,p, the design matrix Xv(n1⋯nd) satisfies the so-called Huber condition (cf. Pruscha [31], pp. 115–117). By this reason, for the rest of the terms, we can immediately apply the technique proposed in Arnold [22] to show Xv(n1⋯nd)⊤εu(n1⋯nd)→DNdv(0,σuuIdv), for all u,v=1,…,p. Therefore, we finally get the following component-wise convergence: (49)1n1⋯ndXvn1⋯nd⊤εun1⋯nd⊤Xvn1⋯nd⊤εvn1⋯ndu,v=1p,p⟶POp×p1n1⋯ndXun1⋯nd⊤εun1⋯nd⊤Xun1⋯nd⊤εvn1⋯ndu,v=1p,p⟶POp×p1n1⋯ndXun1⋯nd⊤εun1⋯nd⊤Xvn1⋯nd⊤εvn1⋯ndu,v=1p,p⟶POp×p,where Op×p is the p×p-zero matrix.
Remark 6.
Since Σ^n1⋯nd-1/2Vn1⋯nd(Rn1⋯nd)=Σ^n1⋯nd-1/2Σ1/2Σ-1/2Vn1⋯nd(Rn1⋯nd), without altering the convergence result presented in Theorem 1, the population variance-covariance matrix Σ can be directly replaced by the consistence estimator Σ^n1⋯nd.
5. Calibration of the Tests
The limits of the test statistics are not distribution-free and we need therefore calibration for the distribution of the statistical tests. For the calibration we adapted the idea of residual based bootstrap for multivariate regression studied in Shao and Tu [32] for approximating the distributions of KSn1⋯nd,A and CvMn1⋯nd,A.
For fixed n1,…,nd, let Fn1⋯nd be the empirical distribution function of the vectors of least squares residuals {rj1⋯jd-R¯n1⋯nd:1≤jk≤nk,k=1,…,d} centered at zero vector, where R¯n1⋯nd≔(1/(n1⋯nd))∑j1=1n1⋯∑jd=1ndrj1⋯jd. Let En1⋯nd∗≔(Ej1⋯jd∗)j1=1,…,jd=1n1,…,nd be an array of independent and identically distributed random vectors sampled from Fn1⋯nd and let g^(Ξn1⋯nd) be the ordinary LSE of g(Ξn1⋯nd), where g^(Ξn1⋯nd)=pr∏i=1pWiZn1⋯nd. Then we generate the array of p-dimensional bootstrap observations which is denoted in this paper by Zn1⋯nd∗≔(Zj1⋯jd∗)j1=1,…,jd=1n1,…,nd through the model: (50)Zn1⋯nd∗=pr∏i=1pWiZn1⋯nd+En1⋯nd∗.Based on this model we get the array of p-dimensional bootstrap least squares residuals which is given by the component-wise projection of the bootstrap observations: (51)Rn1⋯nd∗≔rj1⋯jd∗j1=1,…,jd=1n1,…,nd=Zn1⋯nd∗-pr∏i=1pWiZn1⋯nd∗. Hence, the bootstrap analog of KSn1⋯nd,A and CvMn1⋯nd,A is (52)KSn1⋯nd,A∗≔supA∈AΣ^n1⋯nd∗-1/2Vn1⋯ndRn1⋯nd∗ARpCvMn1⋯nd,A∗≔1n1⋯nd∑A∈AΣ^n1⋯nd∗-1/2Vn1⋯ndRn1⋯nd∗ARp2, where (53)Σ^n1⋯nd∗≔1n1⋯ndpr∏i=1pCXin1⋯nd⊥Z∗n1⋯nd⊤pr∏i=1pCXin1⋯nd⊥Z∗n1⋯nd.
The question regarding the consistency of the bootstrap approximation of the p-dimensional processes Σ^n1⋯nd∗-1/2Vn1⋯nd(Rn1⋯nd∗)(A) for Σ^n1⋯nd-1/2Vn1⋯nd(Rn1⋯nd)(A) is summarized in the following theorem.
Theorem 7.
Let {fi1,…,fidi} be an ONB of Wi, for i=1,…,p. Suppose the conditions of Theorem 1 are fulfilled. Then under H0 it holds that (54)Σ∗-1/2Vn1⋯ndRn1⋯nd∗⟶DBp-pr∏i=1pWiHB∗Bp,where pr∏i=1pWiHB∗Bp is defined in Theorem 1.
Proof.
We notice that {Ej1⋯jd∗:1≤jk≤nk,k=1,…,d} are independent and identically distributed with E∗(E1⋯1∗)=0 and Cov∗E1⋯1∗=Σ^n1⋯nd-R¯n1⋯ndR¯n1⋯nd⊤. Hence, the invariance principle implies that (Σ^n1⋯nd-R¯n1⋯ndR¯n1⋯nd⊤)-1/2Vn1⋯nd(En1⋯nd∗) converges in distribution to Bp. Hence under H0 we have Rn1⋯nd∗=En1⋯nd∗-pr∏i=1pWiEn1⋯nd∗ and Σn1⋯nd∗-1/2 can be written as (55)Σn1⋯nd∗-1/2=Σn1⋯nd∗-1/2Σ^n1⋯nd-R¯n1⋯ndR¯n1⋯nd⊤1/2Σ^n1⋯nd-R¯n1⋯ndR¯n1⋯nd⊤-1/2,where it can be shown easily that (56)R¯n1⋯ndR¯n1⋯nd⊤=oP1,E∗Σ^n1⋯nd∗=Σ^n1⋯nd+oP1,with oP(1) being the collection of the terms converging in probability to Op×p. Then by recalling Theorem 5 and the linearity of Vn1⋯nd we only need to show that (57)Σ^n1⋯nd-R¯n1⋯ndR¯n1⋯nd⊤-1/2Vn1⋯ndpr∏i=1pWiEn1⋯nd∗⟶Dpr∏i=1pWiHB∗Bp.The proof is established by imitating the steps of proving convergence result of Theorem 1. We are done.
6. Simulation Study
In this section, we report on a simulation study designed to investigate the finite sample size behavior of the KS and CvM tests. We simulate a multivariate model with four components defined on the unit rectangle I2. The hypothesis under study is H0:g∈∏i=14Wi against H1:g∉∏i=14Wi, where W1≔[f1], W2≔[f1,f2,f3], W3≔[f1,f2,f3,f4,f5,f6], and W4≔[f1,f2,f3,f4,f5,f6,f7]. Thereby we define f1(t,s)≔1, f2(t,s)=t, f3(t,s)=s, f4(t,s)=t2, f5(t,s)=s2, f6(t,s)=ts, and f7(t,s)=t3, for (t,s)∈I2. The samples are generated from a localized model Y(l/n1,k/n2)=(1/n1n2)g(l/n1,k/n2)+E(l/n1,k/n2) under the experimental design given by Ξn1×n2={(l/n1,k/n2)∈I2:1≤l≤n1,1≤k≤n2}, where g(l/n1,k/n2) is defined as (58)gln1,kn2≔5+ρexpln1kn210-5ln1+10kn2+γexpln1kn210+20ln1-25kn2+10l2n12-5k2n22+20ln1kn2+δexpln1kn230-30ln1-5kn2+20l2n12-15k2n22+10ln1kn2+10l3n13+κexpln1kn2,for constants ρ, γ, δ, and κ determined prior to the generation of the samples. For fixed n1 and n2 and 1≤l≤n1 and 1≤k≤n2 the vector of random errors E(l/n1,k/n2) is generated independently from the 4-variate normal distribution with mean zero and variance-covariance matrix given by (59)Σ4×4≔93-612326-7-11-6-79712-11765;however, we assume in the computation that Σ4×4 is unknown. It is therefore estimated using Σ^n1n2 defined in Section 4. It is important to note that for computational reason we restricted the index set to the Vapnick Chervonenkis Classes (VCC) of subsets of I2 which is given by the family of closed rectangle with the point (0,0) as the essential point. That is, the family {[0,t]×[0,s]:0≤t,s≤1}.
When ρ, γ, δ, and κ are set simultaneously to zero then we get the samples which coincide to the model specified under H0. Conversely, when at least one of them takes a nonzero value then we obtain the samples which can be regarded as from the alternative whose corresponding samples are generated by assigning nonzero values to either one of the constants, ρ, γ, δ, and κ, or the combinations of them.
Table 1 presents the empirical probabilities of rejection of H0 for α=0.05 and some selected values of ρ, γ, δ, and κ. The empirical powers of the KS and CvM tests are denoted by α^KS and α^CvM, respectively. The notations σ^KS and σ^CvM stand, respectively, for the standard deviation of the samples. The critical values of the statistics KSn1,n2;A and CvMn1,n2;A are 6.0742 and 7.9910 which are approximated by simulation. For ρ=γ=δ=κ=0 the values of α^KS and α^CvM fluctuate around 0.05 as it should be. This means that, independent of the selected number of the lattice points, both tests attain the specified level of significance.
The empirical probabilities of rejection of the KS and CvM tests for several selected values of ρ, γ, δ, and κ. The sample sizes are selected to 35×40 and 50×60. The simulation results are based on 10000 runs.
Sample size
ρ
γ
δ
κ
α^KS
σ^KS
α^CvM
σ^CvM
35×40
0
0
0
0
4.90%
1.31299
4.60%
2.52127
10
0
0
0
10.60%
1.52902
11.28%
4.11538
20
0
0
0
32.80%
1.71455
33.40%
6.26419
40
0
0
0
88.64%
1.86015
87.22%
10.37546
0
50
0
0
5.03%
1.28540
5.33%
2.63926
0
200
0
0
4.43%
1.22630
5.90%
2.57881
0
300
0
0
4.60%
1.18783
7.03%
2.52180
0
500
0
0
4.63%
0.90516
11.00%
2.32439
0
0
50
0
4.83%
1.27888
5.03%
2.59347
0
0
100
0
5.17%
1.29476
5.20%
2.68005
0
0
200
0
6.03%
1.28627
5.23%
2.53931
0
0
500
0
8.23%
1.27694
7.73%
2.71275
0
0
0
50
5.00%
1.27241
5.03%
2.69462
0
0
0
100
4.77%
1.23645
4.63%
2.56912
0
0
0
300
4.77%
1.27318
5.07%
2.53470
0
0
0
500
4.97%
1.26322
4.97%
2.55223
10
50
0
0
11.34%
1.55144
12.22%
3.82300
10
100
0
0
11.12%
1.49017
12.70%
3.81009
10
200
0
0
11.20%
1.52118
12.06%
3.71303
10
500
0
0
12.46%
1.10222
22.44%
2.87591
20
50
50
50
32.78%
1.76263
33.62%
5.75683
20
50
100
50
31.80%
1.71323
32.52%
5.98971
20
50
200
50
28.16%
1.68594
29.16%
5.79669
20
50
500
50
27.08%
1.52512
28.86%
4.92925
50
50
50
20
98.60%
1.86506
97.60%
13.43552
50
50
50
40
98.60%
1.83424
97.95%
13.10968
50
50
50
60
98.55%
1.91241
98.25%
13.49794
50×60
0
0
0
0
4.75%
1.26388
4.55%
2.52770
10
0
0
0
11.35%
1.48925
11.40%
3.70839
20
0
0
0
33.65%
1.75109
33.40%
5.82381
30
0
0
0
65.70%
1.82802
64.90%
8.31756
0
50
0
0
4.70%
1.26907
4.95%
2.60854
0
200
0
0
4.75%
1.23471
5.90%
2.53725
0
300
0
0
5.35%
1.19808
7.35%
2.62291
0
0
50
0
0.0485
1.27867
5.15%
2.57206
0
0
100
0
4.35%
1.23893
4.45%
2.40576
0
0
200
0
5.80%
1.27524
5.50%
2.62313
0
0
500
0
8.45%
1.28708
6.50%
2.64556
0
0
0
50
5.00%
1.28525
4.85%
2.71581
0
0
0
100
4.50%
1.25700
4.70%
2.57227
0
0
0
500
4.70%
1.25102
5.10%
2.51076
20
50
0
0
32.10%
1.70264
32.30%
5.72887
20
100
0
0
33.55%
1.74878
32.50%
5.76105
20
200
0
0
33.30%
1.72065
34.35%
5.71531
20
500
0
0
33.90%
1.61848
46.30%
5.77714
30
50
100
0
62.55%
1.78912
61.80%
8.08037
30
50
200
0
60.50%
1.79929
58.60%
7.76765
30
50
500
0
56.35%
1.78203
54.00%
7.76498
20
20
20
20
32.55%
1.70930
33.05%
5.60510
30
50
50
50
63.10%
1.86549
61.65%
8.63149
50
50
50
50
97.75%
1.88146
97.10%
13.27029
50
100
50
50
97.60%
1.89668
96.90%
13.30456
Furthermore, Figure 1 exhibits the graphs of the empirical power function of the KS and CvM tests for α=0.05 associated with hypothesis H0 specified above against H1:gi∉Wi, for i=1,2,3,4. For the four cases we generate the error vectors independently from the same 4-variate normal distribution mentioned above. In the clockwise direction the left-top panel presents the graphs of the power function for testing H0 against H1:g1∉W1, the right-top panel is for H0 against H1:g2∉W2, the right-bottom is for H0 against H1:g3∉W3, and left-bottom is for H0 against H1:g4∉W4. The common characteristic of the tests is that the power gets larger as the the model moves away from H0. The KS tests represented by smooth line tend to have slightly larger power. However, somewhat unexpectedly, in the second case, the CvM test has much larger power.
The empirical power functions of the KS test (smooth line) and CvM test (dotted line) for the 4-variate model using 50×60-point regular lattice. The simulation is under 10000 runs.
7. Example of Application
In this example, the proposed method is applied to a mining data studied in Tahir [20]. As introduced in Section 1, the data consist of a simultaneous measurement of the percentages of Nickel (Ni), Cobalt (Co), Ferum (Fe), and other substances like Calcium-Monoxide (CaO), Silicon-Dioxide (SiO2), and Magnesium-Monoxide (MgO). The sample was obtained by drilling bores set according to a three-dimensional lattice of size 7×14×10 with 7 equidistance rows running west to east, 14 equidistance columns running south to north, and 10 equidistance depths from the surface of the earth to the bottom. To simplify the computation of the test statistics we consider the experimental design as a two-dimensional lattice of size 7×14 by taking the average value of the samples measured in the same position. We further assume that the exploration region is given by a closed rectangle so that by suitable rescaling it can be transformed into a closed unit rectangle I2. Table 2 exhibits, respectively, the pairs scatter plot and Pearson’s correlation coefficient among the percentages of Ni, CaO, Co, the logarithm of the percentages of SiO2 (logSiO2), MgO (logMgO), and Fe (logFe). By this reason a multivariate analysis must be conducted in the statistical modelling taking into account the unknown covariance matrix of the vector of the variables. Furthermore, based on the individual scatter plot of the samples which are not presented in this work, it can be inferred that polynomials of lower order seem to be adequate to approximate the population model. More precisely, let Y≔(Y1,Y2,Y3,Y4,Y5,Y6)⊤ be the vector of observations representing the observed percentages of CaO, logSiO2, logMgO, Co, Ni, and logFe, respectively. We aim to test the hypothesis (60)H0:EY=β11β21+β22t+β23sβ31+β32t+β33sβ41+β42t+β43sβ51+β52t+β53s+β54t2+β55ts+β56s2β61+β62t+β63s+β64t2+β65ts+β66s2,0≤t,s≤1,for some unknown constants βij, i=1,2,3,4,5,6 and j=1,…,di, with d1=1, d2=d3=d4=3 and d5=d6=6. For this case we have W1≔[f1], W2=W3=W4≔[f1,f2,f3], and W5=W6≔[f1,f2,f3,f4,f5,f6], with f1(t,s)≔1, f2(t,s)=t, f3(t,s)=s, f4(t,s)=t2, f5(t,s)=ts, and f6(t,s)=s2.
Pearson’s correlation matrix of the percentages of CaO, logSiO2, logMgO, Ni, logFe, and Co observed over a regular lattice of size 7×14. Source of data: Tahir [20].
CaO
logSiO2
logMgO
Ni
logFe
Co
CaO
1.0000
0.3949
0.4045
-0.1285
-0.1167
-0.0665
logSiO2
0.3949
1.0000
0.8459
-0.0003
-0.5414
-0.4556
logMgO
0.4045
0.8459
1.0000
-0.1331
-0.4968
-0.3134
Ni
−0.1285
-0.0003
-0.1331
1.0000
0.1652
0.1068
logFe
−0.1166
-0.5414
-0.4968
0.1652
1.0000
0.5937
Co
−0.0665
-0.4556
-0.3134
0.1068
0.5937
1.0000
We obtained the values KS7×14;A=1.44109 and CvM7×14;A=0.936687 with the associated simulated p values of 0.98350 and 0.93670, respectively. We notice that in the computation we consider the VCC {[0,t]×[0,s]:0≤t,s≤1} as the index sets instead of A. Hence when using the KS test as well as CvM test the hypothesis will be not rejected for almost all commonly used values of α. There exists a significant evidence that the assumed model is appropriate for describing the functional relationship between the experimental conditional and the percentages of those elements.
In the practice some computational difficulties appear for testing using our proposed method. First, to the knowledge of the authors, the analytical formula for computing the critical and p values of the tests have been not yet available in the literatures; therefore we need to approximate them by simulation using computer. Second, although the test procedures are established for a much larger family of sets A, in the application the computation is always restricted to the VCC of subsets of Id like that of {∏i=1d[0,ti]⊂Id:0<ti≤1,i=1,…,d} or {∏i=1d[si,ti]⊂Id:0≤si<ti≤1,i=1,…,d}.
8. Concluding Remark
In this article we have developed an asymptotic method for checking the validity of a general multivariate spatial regression model by considering the multidimensional set-indexed partial sums of the residuals. For the calibration of the distribution of the test statistics we propose the residual based bootstrap for multivariate regression. It is shown by applying imitation technique that the residual bootstrap resampling technique is consistent. In a simulation study the finite sample size behavior of the KS and CvM statistics is investigated in greater detail. For the first-order model CvM test has much larger power, whereas for constant, second-order, and third-order models the powers of the two tests are almost the same.
Other possibilities of tests for multidimensional case can be obtained by incorporating a sampling technique according to an arbitrary experimental design. Sometimes because of technical, economic, or ecological reason, practitioners will not or cannot sample the observations equidistantly. One possible approach is to sample according to a continuous probability measure; see, for example, the sampling method proposed in Bischoff [11]. Under this concern we get the so-called weighted KS and CvM tests which can be viewed as generalization of the KS and CvM tests studied in this paper.
Instead of considering the least squares residuals of the observations we can also define a test by directly investigating the partial sums of the observations. The limit process will be given by a type of signal plus a noise which is given by the multidimensional set-indexed Brownian sheet. Observing the limit process we can formulate likelihood ratio test based on the Cameron-Matrin-Girsanov density formula of the limit process. Establishing such type of test will be of our concern in our future research project.
AppendixA. Function of Bounded Variation on I<inline-formula>
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M594">
<mml:mrow>
<mml:mmultiscripts>
<mml:mrow>
<mml:mo> </mml:mo></mml:mrow>
<mml:mprescripts />
<mml:none />
<mml:mrow>
<mml:mi>d</mml:mi></mml:mrow>
</mml:mmultiscripts></mml:mrow>
</mml:math></inline-formula>
Definition A.1.
Let f:Rd→R be a real-valued function with d variables. For αk,βk∈R, let Δαkβkf be a real-value function defined on Rd, given by (A.1)Δαkβkf≔fx1,…,xk-1,βk,xk+1,…,xd-fx1,…,xk-1,αk,xk+1,…,xd,for k=1,…,d. Furthermore, for α≔(αk)k=1d and β≔(βk)k=1d∈Rd, Δαβf is defined on Rd recursively starting from the last components of α and β. More precisely, (A.2)Δαβf≔Δα1β1⋯Δαd-1βd-1Δαdβdf⋯.Let {j1,…,jd} be permutation of {1,2,…,d}; then it holds that(A.3)Δαβf=Δαj1βj1⋯Δαjd-1βjd-1Δαjdβjdf⋯=Δα1β1⋯Δαd-1βd-1Δαdβdf⋯.This means that the operation of Δαβf does not depend on the order. By this reason we write Δαβf by Δα1β1⋯Δαd-1βd-1Δαdβdf ignoring the brackets. The reader is referred to Yeh [33] and Elstrodt [34], pp. 44-45.
Definition A.2 (see Yeh [<xref ref-type="bibr" rid="B33">33</xref>]).
Let Γk≔{[xk0,xk1],[xk1,xk2],…,[xkMk-1,xkMk]} be a collection of Mk rectangles on the unit interval [0,1] with 0=xk0≤xk1≤⋯≤xkMk=1, for k=1,…,d. The Cartesian product K≔∏k=1dΓk which consists of M1×M2×⋯×Md rectangles is called a nonoverlapping finite exact cover of Id. The family of all nonoverlapping finite exact cover of Id is denoted by J(K).
Definition A.3 (see Yeh [<xref ref-type="bibr" rid="B33">33</xref>]).
For 1≤wk≤Mk, with k=1,…,d, let Jw1⋯wd be the element of K defined by Jw1⋯wd≔∏k=1d[xkwk-1,xkwk]. Let ψ:Id→R be a real-valued function on Id. Operator ΔJw1⋯wd acting on a function ψ is defined by (A.4)ΔJw1⋯wdψ≔Δx1w1-1x1w1Δx2w2-1x2w2⋯Δxdwd-1xdwdψ.The variation of ψ over the finite exact cover K is defined by (A.5)vψ;K≔∑w1=1M1⋯∑wd=1MdΔJw1⋯wdψ.Accordingly, the total variation of ψ over Id is defined by (A.6)Vψ;Id≔supK∈JKvψ;K.Furthermore, function ψ is said to have bounded variation in the sense of Vitaly on Id if there exists a real number M>0 such that V(ψ;Id)≤M for some real number M>0. The class of such functions is denoted by BVV(Id).
Definition A.4 (see Yeh [<xref ref-type="bibr" rid="B33">33</xref>]).
Let (xk)k=1d be a variable in Id. For fixed k, let Ik≔[0,1]k be a k-dimensional unit closed rectangle constructed in the following way. We choose d-k components of the variable (xk)k=1d. For each choice from all possible elements of the set Cd-kd, we set each xi with 0 or 1 and let the remaining k variables satisfy 0≤xi≤1. Then for each k we get 2d-kCd-kd unit closed rectangles Ik. For convention we denote the collection of all 2d-kCd-kd of closed rectangles Ik by Bk and the jth element of Bk will be denoted by Ijk. Function ψ is said to have bounded variation in the sense of Hardy on Id, if and only if for each k=1,…,d and j=1,…,2d-kCd-kd there exists a real number Mjk>0 such that V(ψIjk(·);Ijk)≤Mjk, where, for k=1,…,d and j=1,…,2d-kCd-kd, ψIjk(·) is a function with k variables obtained from the function ψ(x1,x2…,xd) by setting the d-k selected variables with 0 or 1, whereas the remaining k variables lie in the interval [0,1]. The class of such functions will be denoted by BVH(Id).
B. Integration by Parts on I<inline-formula>
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M686">
<mml:mrow>
<mml:mmultiscripts>
<mml:mrow>
<mml:mo> </mml:mo></mml:mrow>
<mml:mprescripts />
<mml:none />
<mml:mrow>
<mml:mi>d</mml:mi></mml:mrow>
</mml:mmultiscripts></mml:mrow>
</mml:math></inline-formula>
For family Bk defined in Definition A.4, let I≔⋃k=0dBk, where, for k=0, the family B0 is a collection of 2d different points in Id. As an example, for d=2, we have B0={I10=(0,0),I20=(0,1),I30=(1,0),I40=(1,1)}. For each k=1,…,d and j=1,…,2d-kCk-dd, let ♯(Ijk) be the number of 1’s appearing in Ijk. Next, let φ and ψ be defined on Id. If φ is Riemann-Stieltjes integrable with respect to ψ on Ijk∈Bk, we denote the integral by ∫Ijk(R)φdkψ. For k=0, it is understood that ∫Ij0(R)φd0ψ is defined as the product of φ and ψ at that point of Ij0 (see Yeh [33]).
Theorem B.1 (integration by parts (see Yeh [<xref ref-type="bibr" rid="B33">33</xref>])).
Let φ be Riemann-Stieltjes integrable with respect to ψ on each member of I. Then ψ is Riemann-Stieltjes integrable with respect to φ on Id, and we have the formula(B.1)∫IdRψx1,…,xddφx1,…,xd=∑k=0d∑j=12d-kCd-kd-1d-♯Ijk∫IjkRφdkψ.Moreover, if ψ have bounded variation in the sense of Hardy on Id and φ is continuous on Id, then we have the inequality (B.2)∫IdRψx1,…,xddφx1,…,xd≤φ∞2dψ∞+∑k=1d∑j=12d-kCd-kdVψIjk·;Ijk.
C. Some Property of the Partial Sums OperatorLemma C.1 (see Bischoff and Somayasa [<xref ref-type="bibr" rid="B10">9</xref>]).
For every one-dimensional pyramidal array An1⋯nd≔(aj1⋯jd)j1=1,…,jd=1n1,…,nd, it holds that Vn1⋯nd(i)(An1⋯nd)∈HB, where HB is the subspace defined in (15). Furthermore, for any arrays An1⋯nd≔(aj1⋯jd)j1=1,…,jd=1n1,…,nd and Bn1⋯nd≔(bj1⋯jd)j1=1,…,jd=1n1,…,nd, we have (C.1)Vn1⋯ndiAn1⋯nd,Vn1⋯ndiBn1⋯ndHB=An1⋯nd,Bn1⋯ndRn1×⋯×nd,where Vn1⋯nd(i) is the one-dimensional component of the partial sums operator Vn1⋯nd.
Proof.
Associated with An1⋯nd we can construct a step function sAn1⋯nd:Id→R defined by (C.2)sAn1⋯ndt≔∑j1=1n1⋯∑jd=1ndaj1⋯jd1Cj1⋯jdt,t∈Id,where Cj1⋯jd=∏k=1d(jk-1/nk,jk/nk], for 1≤jk≤nk. For any B∈A, it holds that (C.3)hsAn1⋯ndB≔∫BsAn1⋯ndtλIddt=1n1⋯ndVn1⋯ndiAn1⋯ndB.Hence, Vn1⋯nd(i)(An1⋯nd)∈HB having n1⋯ndsAn1⋯nd as the L2(λId) density. By the definition of the inner product 〈·,·〉HB, we further get (C.4)Vn1⋯ndiAn1⋯nd,V