Two Bootstrap Strategies for a k-Problem up to Location-Scale with Dependent Samples

This paper extends the work of Quessy and Éthier (2012) who considered tests for the k-sample problem with dependent samples. Here, the marginal distributions are allowed, under H 0 , to differ according to their mean and their variance; in other words, one focuses on the shape of the distributions. Although easily stated, this problem nevertheless requires a careful treatment for the computation of valid P values. To this end, two bootstrap strategies based on the multiplier central limit theorem are proposed, both exploiting a representation of the test statistics in terms of a Hadamard differentiable functional. This accounts for the fact that one works with empirically standardized data instead of the original observations. Simulations reported show the nice sample properties of the method based on Cramér-von Mises and characteristic function type statistics. The newly introduced tests are illustrated on the marginal distributions of the eight-dimensional Oil currency data set.


Introduction
Testing for the equality in distribution of two or more realvalued random variables is a classical statistical problem that has been extensively investigated in the literature.Formally, the goal is to test for H 0 :  1 = ⋅ ⋅ ⋅ =   , where   () = (  ≤ ),  ∈ {1, . . ., }, and  1 , . . .,   are usually assumed to be independent random variables.In the bivariate case ( = 2), the most popular procedures for H 0 are those based on the sign, Wilcoxon rank-sum, Kolmogorov-Smirnov, and Cramér-von Mises statistics.The generalization to the -sample situation has been considered by Kiefer [1] and Bickel [2] using Kolmogorov-Smirnov and Cramérvon Mises statistics and by Scholz and Stephens [3] using the Anderson-Darling functional.More recent contributions include Zhang and Wu [4], Martínez-Camblor and de Uña-Álvarez [5], Martínez-Camblor [6], and Wyłupek [7], among others.
When  1 , . . .,   cannot be taken as independent, the testing of H 0 must be handled with care.In that case, one knows from a celebrated theorem of Sklar [8] that there exists a copula  : [0,1]  → [0, 1] such that the joint distribution of X = ( 1 , . . .,   ) can be written as (X ≤ x) = { 1 ( 1 ), . . .,   (  )}, where x = ( 1 , . . .,   ).If the marginal distributions  1 , . . .,   are continuous, then  is unique; see Nelsen [9] for details.As underlined by Quessy and Éthier [10], the possible asymmetry of  invalidates the use of permutation methods.An alternative statistical procedure where  values are estimated from an adapted version of the multiplier bootstrap method was proposed by these authors.
The aim of this paper is to extend the test procedures of Quessy and Éthier [10] to a version of the -sample problem when dependent random variables are rescaled with respect to their mean and variance.More specifically, consider  random variables  1 , . . .,   and let   = (  ) and   = std(  ) be unknown.The null hypothesis of interest in this work is where F () = ( X ≤ ) and X = (  −   )/  .In other words, one wants to test the shape hypothesis that  distributions are equal up to location and scale factors.
The main interest behind this hypothesis is that the actual

Test Statistics
2.1.Empirical Process of the Rescaled Observations.Let X 1 , . . ., X  be independent copies of X = ( 1 , . . .,   ), where X  = ( 1 , . . .,   ).For each  ∈ {1, . . ., }, the random variable   is assumed to have a finite moment of order two.The estimation of the rescaled distribution functions F1 , . . ., F will be based on the empirically standardized observations where, for each  ∈ {1, . . ., }, μ and σ are the usual empirical mean and variance based on  1 , . . .,   .Specifically, the univariate distribution function F will be estimated for  ∈ R by If  1 , . . .,   denote the usual empirical distribution functions, one obtains easily F () =   (σ   + μ ).Letting F = ( F1 , . . ., F ) ⊤ , one then has Remark 1.The estimation of F1 , . . ., F from empirically standardized observations is somewhat similar to the estimation of a quantile function as considered by Parzen [11].Under the null hypothesis that a random sample  1 , . . .,   comes from a distribution  of the form () = F{( − )/} from some fixed distribution function F, empirically standardized observations are defined as Under the null hypothesis, X1 , . . ., X are approximately uniform on [0, 1], which motivated Parzen [11] to compare their empirical quantile function to the uniform quantile function as a goodness-of-fit criteria.
Proposition 3. If  1 , . . .,   have bounded densities  1 , . . .,   and finite moments of order two, then F converges weakly to a centered Gaussian process F of the form where B = (B 1 ∘ F1 , . . ., B  ∘ F ) and B 1 , . . ., B  are Brownian bridges with covariance structure given by Γ   .The covariance structure of F is given by G(, ) = { F() F() ⊤ }, where, for any ,   ∈ {1, . . ., }, Proposition 3 can be specified to the case when the null hypothesis H ⋆ 0 holds true.In that case, there is a distribution function F such that The result is stated as a corollary to Proposition 3.

Corollary 4.
Suppose that F has a bounded density f whose moment of order two exists.Under H ⋆ 0 , the empirical process F converges weakly to the centered Gaussian process whose covariance structure is given by where the entries of Γ are Γ   for ,   ∈ {1, . . ., }.

Cramér-von Mises and Characteristic Function Test Statistics. Consider the Cramér-von Mises and characteristic function type functionals
with Ψ() = ∫ R   F() being the component-wise characteristic function of F = ( 1 , . . .,   ), and Ψ() is its complex conjugate.Here, G() > 0 is a weight function that gives nonnull mass at each  ∈ R. Alternate representations for L G Cf are given in the next lemma; the latter will prove useful in the sequel.
The statistics that will be investigated in this work are Here, O ∈ R × is a combination matrix such that Oz = 0  if and only if z = 1  for some  ∈ R, where 0  = (0, . . ., 0) ⊤ ∈ R  and 1  = (1, . . ., 1) ⊤ ∈ R  .The null hypothesis can then be reformulated as H ⋆ 0 : O F = 0  .The asymptotic behavior of   and  G  under H ⋆ 0 is stated in the next proposition.
Proposition 6. Suppose that F has a bounded density f whose moment of order two exists.Also assume that G is a weight where F is the limit of F identified in Corollary 4.
An interesting feature of the test statistics   and  G  is that simple formulas for their computation are available in terms of product of matrices.Explicit expressions similar to those given by Quessy and Éthier [10] are described in Remarks 7 and 8.
Since by the assumption that where the entries of

Resampling
3.1.Bootstrapping F  .The challenging issue of computing valid  values for test statistics based on   and  G  is addressed in this section, where two bootstrap methods are developed.One version is based directly on the functional Υ, while the other one is built around its Hadamard derivative.
Proposition 9.Under the conditions of Proposition 3, one has in the space ℓ ∞ (R) that where F(1)  , . . ., F() are independent copies of F.

Multiplier Bootstrap
Explicit and easy-to-implement formulas for these multiplier statistics are given in Appendix B. The asymptotic validity of these bootstrapped statistics is a straightforward consequence of Proposition 9 and of the continuous mapping theorem.In other words, one has the weak convergence result (  ,  (1)   , . . .,  ()  )  (,  (1)   , . . .,  ()  ) for  = 1, 2, and similarly for the characteristic function statistics.Hence, asymptotically valid  values for the tests based on   and  G  are given, respectively, for  = 1, 2, by One rejects H ⋆ 0 whenever P, <  or P G , <  for some predetermined significance level .These testing procedures are consistent.Indeed, on one side, if the null hypothesis fails to hold, then O F() ̸ = 0, for each , in a subset of R with nonnull probability; as a consequence, and then   → +∞ and  G  → +∞ in probability.On the other side, the weak convergence result (  ,  (1)   , . . .,  ()  )  (,  (1)   , . . .,  ()  ) (and also for the characteristic function tests) still holds even under a failure of the null hypothesis so that the bootstrap replications are bounded in probability.It shows that the probability of rejection of H ⋆ 0 tends to 1 as  → ∞.

Sample Properties of the Tests
The asymptotic behavior of   and  G  as well as the asymptotic validity of two multiplier bootstrap methods has been established in the preceding sections.These results tell little, however, about the sample properties of the tests in small and moderate sample sizes.It is thus important to investigate their ability to keep their nominal level under H ⋆ 0 as well as their power under various alternatives in situations when only a limited number of observations are available.This will be done here via Monte-Carlo simulations.
The models considered for the marginal distributions are the Normal (N), Student with ] degrees of freedom ( ] ), and double-exponential (DE) distributions.In each of the scenarios under study,  1 is the Normal distribution and  2 = ⋅ ⋅ ⋅ =   are either the N,  9 ,  6 ,  3 , or DE distribution.Because the test statistics are invariant under location/scale transformations, it is enough to consider only the standard version of each of these distributions, that is, when the mean and variance are set to 0 and 1, respectively.
Because one expects that the dependence structure underlying the multivariate distribution has an influence on the results, symmetric and asymmetric dependence structures have been considered.They are all based on the Normal copula that one can extract from the multivariate Normal distribution.If ℎ  is the -variate Normal density with correlation matrix , then it is implicitly given for u = ( 1 , . . .,   ) ∈ [0, 1]  by Note that unlike the resampling methods based on permutation, our procedure is valid under asymmetric joint distributions.Asymmetric versions of    obtained from Khoudraji's device [14] are given by where  = ( 1 , . . .,   ) ∈ [0,1]  .In the simulation study, only the special case when  = (1/2, 0, . . ., 0) was considered, yielding the asymmetric copula model    (u) = √ 1    (√ 1 ,  2 , . . .,   ).For all the simulation results reported, the multiplier random variables are exponential with mean one.For the case  = 2, the number of bootstrap samples has been set to  = 250; this has been reduced to  = 100 when  = 3, 4 in order to speed up the simulations.For the first bootstrap strategy, the grid for the approximation has one hundred points and f is a kernel density estimator based on the whole sample of the  ×  empirically standardized observations X ,  ∈ {1, . . ., },  ∈ {1, . . ., }; that is, where ∫ R () = 1.In the simulations presented,  is the Epanechnikov kernel (see Epanechnikov [15]) and ℎ is the optimal bandwidth for the estimation of Normal densities.
For the characteristic function statistics, two weight functions have been considered.The first is G 1 () =  − 2 , for which

The corresponding test statistics based on the combination matrix
and  G 2  .The results for the case  = 2 are in Table 1 ( = 100) and Table 2 ( = 200).First, note that the ability of the tests to keep their 5% nominal level is quite good, except for   when using the second bootstrap strategy; simulations not reported here indicate that as high as  = 1000 observations are needed in order for   to keep its size, as predicted from the asymptotic theory.Overall, considering the complexity of the bootstrap methods involved in the computation of  values, the results are very satisfying.
The tests are also very good at rejecting the null hypothesis in case of departures from the equality of standardized distributions.Of course, the probability of rejection is higher when  = 200 than when  = 100, as expected from the consistency of the tests.Also, the Normal distribution is well distinguished from the Student distribution  ] when ] is low; it is also the case of the double-exponential distribution.Note that the observed powers are larger for a high level of dependence, that is,  = 1/2, compared to low level of dependence, that is,  = −1/2.However, for a given level of dependence, the asymmetric and symmetric dependence structure yielded similar results.The bootstrap method that is use in conjunction with a test statistic, has a significant influence on the power results.For   , it is when the first bootstrap is used that the best powers are observed; the opposite comment applies to  G 1  .For  G 2  , the results are similar for both resampling methods.Generally speaking, the best test statistic when using the first bootstrap method is  G 2  , which is slightly better than is markedly less powerful.For the second bootstrap, the results vary with respect to the underlying dependence structure.Indeed, under Student alternatives,  G 1  and  G 2

𝑛
have similar power, and they are significantly more powerful than   ; under a double-exponential alternative, it is  G 2

𝑛
which is significantly more powerful than its two competitors.
The results for  = 3 and  = 4 are presented, respectively, in Tables 3 and 4. One can see that the comments for the case  = 2 are still valid here.Hence, globally, one can say that the newly introduced test statistics and the two bootstrap methods yield very reliable statistical procedures: they keep their nominal level well under moderate sample sizes and they are powerful under a large variety of alternatives.These properties are well maintained under various dependence structures, including symmetric and asymmetric copulas, as well as negative and positive dependence.

Illustration on the Oil Currency Data Set
This data set consists of daily log-returns of the oil price ( 1 ), Standard and Poor's 500 ( 2 ), and six currency exchange rates, namely, those of Great Britain pound ( 3 ), United States dollar ( 4 ), Swiss franc ( 5 ), Japanese yen ( 6 ), Danish krone ( 7 ), and Swedish krona ( 8 ) registered from May 1985 to June 2004.It has been analyzed by Kluppelberg and Kuhn [16] to illustrate their newly introduced copula structure analysis.With a goodness-of-fit test specifically designed for the metaelliptical families of distributions, Quessy and Bellerive [17] concluded that a Student copula with an estimated sixteen degrees of freedom was suitable for these data when considering the last  = 904 observations.The tests based on   ,  G 1  , and  G 2

𝑛
for the equality of the standardized distributions will now be illustrated on the twenty-eight possible pairs; the results can be found in Table 5.For each of these three test statistics, their associated  value has been estimated from  = 250 multiplier bootstrap samples from each of the two bootstrap strategies that have been proposed; these resampling methods are referred to as PV I and PV II in Table 5.Overall, all tests are generally in agreement on the acceptance or rejection of Table 3: Percentages of rejection, as estimated from 1000 replicates, of the null hypothesis of the equality of  = 3 distributions up to location/scale under Normal and asymmetric Normal dependence structures when  = 200;  1 is the Normal distribution;  2 =  3 are either the N,  9 ,  6 ,  3 , or DE distribution.
Copula the null hypothesis, whatever the bootstrap method that was employed is.As a specific example, consider the pair (3,4) for which the null hypothesis is accepted for each test; this formal conclusion could also be expected while looking at the standardized densities presented in Figure 1(d).It is clear from the nonstandardized densities (Figure 1(c)) that the conclusion would have been different if the data were not standardized by their respective means and variances.From the scatterplot in Figure 1(a), one can see that there is a significant positive relationship between the two random variables; it can also be seen from the inspection of the scatterplot of the normalized ranks (Figure 1(b)).It just illustrates the importance of having a statistical procedure that takes into account the possible dependence.For the pair (4, 5), all the tests agree on a clear rejection of H 0 ; this conclusion is in accordance with the standardized densities that one can see in Figure 2.

Figure 1 :
Figure 1: Last  = 904 observations of the pair (3, 4) in the Oil currency data set.(a) Scatterplot of the original data; (b) scatterplot of the standardized ranks; (c) estimated densities; (d) estimated standardized densities.

Table 1 :
Percentages of rejection, as estimated from 1000 replicates, of the null hypothesis of the equality of  = 2 distributions up to location/scale under Normal and asymmetric Normal dependence structures when  = 100;  1 is the Normal distribution;  2 is either the N,  9 ,  6 ,  3 , or DE distribution.

Table 2 :
Percentages of rejection, as estimated from 1000 replicates, of the null hypothesis of the equality of  = 2 distributions up to location/scale under Normal and asymmetric Normal dependence structures when  = 200;  1 is the Normal distribution;  2 is either the N,  9 ,  6 ,  3 , or DE distribution.

Table 4 :
Percentages of rejection, as estimated from 1000 replicates, of the null hypothesis of the equality of  = 4 distributions up to location/scale under Normal and asymmetric Normal dependence structures when  = 200;  1 is the Normal distribution;  2 =  3 =  4 are either the N,  9 ,  6 ,  3 , or DE distribution.