T 3-Plot for Testing Spherical Symmetry for High-Dimensional Data with a Small Sample Size

High-dimensional data with a small sample size, such as microarray data and image data, are commonly encountered in some practical problems for which many variables have to be measured but it is too costly or time consuming to repeat the measurements for many times. Analysis of this kind of data poses a great challenge for statisticians. In this paper, we develop a new graphical method for testing spherical symmetry that is especially suitable for high-dimensional data with small sample size. The new graphical method associated with the local acceptance regions can provide a quick visual perception on the assumption of spherical symmetry. The performance of the new graphical method is demonstrated by a Monte Carlo study and illustrated by a real data set.


Introduction
Studies on highly complicated random systems or structures pose the problem of measuring a large number of variables simultaneously.Modern technology makes it possible to collect high-dimensional data.Because of the high cost or the great difficulty in measuring a large number of variables at the same time, it is quite common that high-dimensional data are usually associated with small sample size.For example, microarray data are usually obtained by measuring thousands of variables, but the sample size is possibly less than 100; image data could be obtained by measuring more than 10,000 variables at the same time but possibly with sample size of only several hundreds.Analysis of high-dimensional data with a small sample size has become an important research topic in statistics.Many authors have been making great efforts in developing various dimension-reduction techniques, among which sliced inverse regression called SIR, see, Li 1 and its extension Li 2 ; Cook 3 ; Cook and Li 4 are powerful.The critical assumption of SIR on population is spherical or elliptical symmetry.Thus, test of high-dimensional spherical or elliptical symmetry

Background of the ST 3 -Plot
The ST 3 -plot is focused on looking for evidence of departure from a spherical distribution for an i.i.d.d-dimensional sample {x 1 , . . ., x n }.Spherical distributions possess very similar marginal distributions to those of the multivariate normal distribution.For example, all univariate 1-dimensional marginals of a multivariate normal distribution are still normal.In comparison with this marginal property, all univariate marginals of a spherical distribution are scale-invariant.This scale invariance is an important characteristic of the family of spherical distributions.More discussions and induced results from the scale invariance for a spherical distribution can be found from Fang et al. 12 .Because most existing plotting methods for testing goodness of fit are only applicable for univariate distributions, the key idea of the ST 3 -plot is to extend the existing univariate T 3 -plot 16 to testing multivariate spherical symmetry from some "special principal component" directions by employing the same idea as in Fang et al. 17 .Section 2.2 gives a simple review on Ghosh's 16 T 3 -plot and Section 2.3 summarizes the theoretical details on the extension of Ghosh's 16 T 3 -plot to the ST 3 -plot.

A Review of the T 3 -Plot
Let x 1 , . . ., x n be an i.i.d.univariate sample and x x 1 , . . ., x n .Testing goodness-of-fit for univariate normality is to test whether the underlying distribution of the sample is normal N μ, σ 2 with unknown μ and σ 2 .The EMGF empirical moment generating function for the studentized data is defined as where x and s are the sample mean and the sample standard deviation, respectively, R 1 stands for the set of all real numbers.Denote the ith derivative of M x, t with respect to t by M i x, t i ≥ 0 .Ghosh 16 defined the T n 3 x, t a function of t for fixed x as

2.2
The graphical method for detecting nonnormality of the underlying distribution of the sample {x 1 , . . ., x n } is based on x, t is a stochastic process with an index t ∈ −a, a for some a > 0 .Under the normal assumption, Ghosh 16 obtained the asymptotic distribution of x, t , which is a zero-mean Gaussian process with a covariance function K t, s t, s ∈ −a, a .In particular, K t, t t 6 9t 4 18t 2 6 exp t 2 − 2t 6 .

2.3
The behavior of T n 3 x, t at t 0 reflects evidence of departure from normality for the sample {x 1 , . . ., x n }.For example, T n 3 x, 0 is proportional to the sample skewness and the slope of T n 3 x, t at t 0 is proportional to the sample kurtosis.

Extension of the T 3 -Plot to the ST 3 -Plot
Let x 1 , . . ., x n be an i.i.d.d-dimensional sample from a population characterized by a ddimensional random vector x.We want to test spherical symmetry of the sample.The theoretical principle for extending the T 3 -plot to the ST 3 -plot is based on Lemma 2.
where the sign " d " means that the two sides of the equality have the same distribution.
Lemma 2.1 implies that for any scale-invariant statistic t x associated with a spherically distributed random vector x, its distribution is the same as that from taking the spherical random vector x to be the standard normal N n 0, I n .Theorem 2.2.Let x 1 , . . ., x n be i.i.d. with a d-dimensional spherical distribution and P x 1 0 0, and matrix X is a vector function that is uniquely determined by X X. Define the random vector z X d : d × 1.

2.7
Then z has a d-dimensional spherical distribution.
Proof.First, we point out that the random matrix X x 1 , . . ., x n has a left spherical matrix distribution 18 .That is, it satisfies for any d × d orthogonal matrix Γ that is independent of X.We can write the random vector z in 2.7 as that is, we obtain for any d × d orthogonal matrix Γ that is independent of z.This shows that the random vector z given by 2.7 has a spherical distribution by definition.This completes the proof.
The random vector d in 2.7 acts as a direction for projecting a left spherically distributed random matrix X into a spherically distributed random vector z by 2.7 .This idea is due to Läuter 19 in constructing tests for the multivariate normal mean μ in N d μ, Σ with an unknown Σ.
Based on Lemma 2.1 and Theorem 2.2, we can extend the T 3 -plot to the ST 3 -plot for testing high-dimensional spherical symmetry.At first, we point out that the T 3 function T n 3 x, t given by 2.2 is scale-invariant, that is, for any constant a > 0. Therefore, by Lemma 2.1, if x is spherically distributed, then

2.12
Similarly, for the K-S type statistic given by 2.4 , it is also true that where z, t is given by 2.15 and K t, t by 2.3 .By 2.12 and 2.13 , the principle for using the ST 3 -plot to detect high-dimensional nonspherical symmetry can be summarized as: plot the T 3 -function T d 3 z, t given by 2.15 versus t ∈ −1, 1 .If the plot shows a significant departure from the horizontal axis y 0 in R 2 , hypothesis 2.14 is rejected, and as a result, the i.i.d.sample {x 1 , . . ., x n } can be considered from a population of nonspherical distribution.The K-S type statistic KS d z in 2.17 can be employed to evaluate the significance of departure from spherical symmetry.
It is obvious that there are numerous choices of the function f X X in Theorem 2.2 in constructing the projection direction d.We will study the empirical performance of the choices recommended by Läuter 19 ,and Läuter et al. 21 in next section.

An Overview
Graphical methods can only serve as descriptive statistical inference without associated acceptance regions with the plots.One of the impressive characteristics of Ghosh's 16 T 3 -plot for detecting departure of univariate normality is its associated acceptance regions, which make it possible the plotting method as analytical statistical inference.The purpose of the Monte Carlo study in this section is to provide simulated acceptance regions for the ST 3 -plot developed in Section 2 by using a similar Monte Carlo method to that in Ghosh 16 and in Fang et al. 17 .Based on the acceptance regions, the empirical performance of the ST 3 -plot can be partially evaluated through counting the rejection rate type I error from a selected set of spherical distributions which serve as the null hypothesis and counting the rejection rate empirical power from a selected set of nonspherical alternative distributions which serve as the alternative hypothesis when applying the ST 3 -plot.

The Local Acceptance Regions
A local acceptance region for the plot of T d 3 z, t given by 2.15 is a critical band for the plot on t ∈ −1, 1 .If the plot goes outside the critical band, it is an indication that z is not spherically distributed, and as a result, the hypothesis of spherical symmetry is rejected.A critical band can be constructed by simulating the percentiles of the finite-sample distribution of the K-S type statistic in 2.17 .By Lemma 2.1, we have

3.2
The quadratic curves given by 3.2 are suitable for estimating the percentiles of the K-S type statistic KS d z for d in the range 10 ≤ d ≤ 50. Figure 1 shows the plots of the simulated percentiles of KS d z and the estimated percentiles c d α given by 3.2 by the least squares method.The fit in Figure 1 seems to be acceptable.By using the quadratic curves given by 3.2 for estimating the finite-sample percentiles of the statistic KS d z in 2.17 for dimension 10 ≤ d ≤ 50, the acceptance region for the T 3 -function T d 3 z, t in 2.15 for testing spherical symmetry is given by where c d α is given by 3.2 and K t, t is given by 2.3 .We will call the acceptance region determined by 3.3 the local acceptance region for the ST 3 -plot in testing high-dimensional spherical symmetry.When the plot of T d 3 z, t t ∈ −1, 1 goes outside the acceptance region determined by 3.3 , hypothesis 2.14 is rejected, and as a result, the underlying distribution of the sample {x 1 , . . ., x n } shows evidence of nonspherical symmetry.

Type I Error Rates
We already pointed out that there are numerous choices for the projection direction d in 2.7 to plot the T 3 -function 2.15 .We will perform a Monte Carlo study on the following choices of d that were suggested by Läuter 19 ,and Läuter et al. 21 .
1 Solution to the eigenvalue problem: where To ensure the unique solution, the random matrix D is assumed to have positive diagonal elements.The following directions are chosen: where the sign x stands for the integer part ≤ x of a real number x e.g., 2.9 2 and 3.2 3 .
2 The direction based on the SS-test discussed by Läuter et al. 19 .Choose where Diag X X denotes the diagonal matrix with the same diagonal elements as those of X X, and 1 n is an n × 1 vector of ones.
where k is given by 3.5 .
The following seven directions are chosen for a Monte Carlo study on the type I error rates and power when using the ST 3 -plot for testing spherical symmetry: 3.9 The Monte Carlo study on type I error rates of the local acceptance region 3. 2 and N 20; 5 the Pearson type II distribution PII with m 3/2; 6 the Cauchy distribution.The TFWW algorithm 22 pages 166-170 23 , is employed to generate empirical samples from these spherical distributions except the normal distribution whose samples can be generated from the MATLAB internal function.Table 1 gives the empirical type I error rates α 5% of the ST 3 -plot in testing spherical symmetry for dimensions d 20 and d 30, where the seven directions are given by 3.9 and the type I error rates were calculated by type I error rate number of rejections number of replications .

3.10
The local acceptance region given in 3.3 for α 5% is used to count the number of rejections for the selected null distributions.The simulation was done with 2,000 replications.
Based on Table 1, we can summarize the following empirical conclusions: 1 the ST 3 -plot seems to have better control of the type I error rates by using the five directions d 1 , . . ., d 5 than by using the two directions d 6 and d 7 , which tend to have lower type I error rates than the significance level α 5%.The ST 3 -plot based on these two directions may over-accept the null hypothesis of spherical symmetry; 2 the performance of the ST 3 -plot on controlling the type I error rates tends to be slightly affected by the sample size.This may be due to the arrangement of the observation matrix X in Theorem 2.2.So we can expect the ST 3 -plot to have good control on type I error rates in the case of high dimension with a small sample size.This is a good indication for high-dimensional data analysis.
For α 1% and 10%, we obtained similar results to those in Table 1 on the type I error rates of the ST 3 -plot by using the acceptance region 3.3 and the same directions in 3.9 .These are not presented to save space.

Power Study
The power of the ST 3 -plot in testing spherical symmetry is computed by using the acceptance regions 3.3 and the formula given by 3.10 .The KS-type statistic 2.17 is computed in the same way as in Section 3.2.The following six nonspherical alternative distributions are selected: 1 the multivariate χ 2 -distribution comprises i.i.d.χ 2  By a standard Monte Carlo technique, a d × n sample matrix X 0 x 1 , . . ., x n can be generated from the above six nonspherical alternative distributions according to their marginal distributions.Then X 0 is centerized by where the mean value E X 0 is taken for each element of X 0 .By this way, we obtain nonspherical samples distributed in the space R d like those of spherical samples.Table 2 presents the power α 5% of the T 3 -plot in testing spherical symmetry for the six nonspherical distributions by using the seven directions in 3.9 .
Based on Table 2, we can summarize the following empirical conclusions: 1 the ST 3 -plot based on the direction d 1 remarkably outperforms has much higher power the other six directions under the four choices of the sample sizes and almost all of the selected nonspherical distributions.So d 1 can be considered as a general choice for the T 3 -plot in testing high-dimensional spherical symmetry; 2 the ST 3 -plot is slightly affected by an increase of the sample size.Because of the rearrangement of the observation matrix X x 1 , . . ., x n : d × n, instead of n × d, the sample size n is implicitly taken as the sample dimension d, and the sample dimension d is taken as the sample size n.This is a rotation of the regular observation matrix X x 1 , . . ., x n : n × d.This rotation results in a power decrease of the ST 3 -plot when the sample size n increases, and it results in a power increase of the T 3 -plot when the sample dimension d increases.This can be observed from Table 2. Therefore, the ST 3 -plot is especially suitable for testing very highdimensional spherical symmetry with a relatively small sample size.

Practical Illustration
To illustrate how to apply the ST 3 -plot in practice, we employ a subset of a real data set.The data set was used in Walker and Wright 24 and was called the VDP vertical density profile data set.As described by Walker and Wright 24 , manufacturers of engineered wood boards, which include particle board and medium density fiberboard, are very concerned about the density properties of the board produced.The density is measured using a profilometer which uses a laser device to take a series of measurements across the thickness of the board.A profilometer takes multiple measurements on a sample usually a 2 × 2 inch piece to form the vertical density profile of the board.The VDP data subset that we illustrate here consists of 45 measurements taken 0.014 inch apart, and comprises 2 groups: A group A consists of 9 subjects A1, . . ., A9; B group B consists of 11 subjects B1, . . ., B11.
We can consider each subject as an observation with 45 measurements.Then each observation has a dimension of 45.Based on the structure of the complete VDP data set, groups A and B have a sample size 9 and 11, respectively.Because a spherical distribution always has a zero mean vector, the selected two groups of VDP data should be shifted to have the origin as its central location.This is realized by subtracting the group sample mean from each observation in groups A and B, respectively.That is, CA. centerized subgroup A: the sample mean from the 9 subjects is subtracted from each observation in subgroup A; CB. centerized subgroup B: the sample mean from the 11 subjects is subtracted from each observation in subgroup B.
After the centerization, each of the above subgroups is comparable to a sample from a spherical distribution, which has a zero mean.
For illustration purpose, in choosing the projection directions for the ST 3 -plots for the data of group A and group B as defined above, we consider four directions: On each of these four directions, the T 3 -function 2.15 i.e., the ST 3 -plot and the acceptance region given by 3.3 are plotted in Figure 2 for the centerized subsets CA and CB.
The following facts can be observed: 1 the ST 3 -plot for the centerized data in subgroup A the CA plots at the projection direction d 5 goes beyond the 90%-acceptance region, showing evidence of nonspherical symmetry of the data at the significance level α 10%.As a result, it can be concluded that the null hypothesis of spherical symmetry for the centerized data in subgroup A is rejected at α 10%; 2 the ST 3 -plots for the centerized data in subgroup B the CB plots at the projection directions d 1 , d 2 , and d 6 go beyond the 95%-acceptance regions, showing evidence of nonspherical symmetry of the data at the significance level α 5%.As a result, it can be concluded that the null hypothesis of spherical symmetry for the centerized data in subgroup B is rejected at α 5%.
The evidence of nonspherical symmetry of the data in subgroups A and B above implies that it may be inappropriate to set up a random effects model x i − x σ i with spherically distributed random effects i for the centerized data {x i − x : i 1, . . ., n} n 9 for subgroup A and n 11 for subgroup B , where σ stands for a scale parameter.The illustration of detecting nonspherical symmetry in Figure 2 could provide a way to regression diagnostics with the assumption of spherically distributed error terms.One of the regression diagnostic techniques is to check if the residual random vectors like {y i − y i : i 1, . . ., n} are approximately i.i.d.d-dimensional normal deviates N d 0, σ 2 I d by using the probabilityplot method, where I d stands for the identity matrix and σ denotes an unknown standard deviation.If it shows a lack of fit for the normal assumption, one could consider testing the spherical symmetry for the residual random vectors {y i − y i : i 1, . . ., n} by providing the ST 3 -plots as in Figure 2. If no evidence of nonspherical symmetry can be detected from  1 and 2, respectively.For each plot, the region between the two dashdotted curves stands for the 99% acceptance region, the region between the two real-line curves for the 95% acceptance region, and the region between the two dashed curves for the 90% acceptance region.

Journal of Probability and Statistics
the ST 3 -plots, the regression model for the observed data {y i : i 1, . . ., n} could be extended to have spherically distributed error terms.More discussion on regression models with spherically distributed error terms and statistical inference under elliptical distributions which contains spherical distributions as a special case can be referred to Fraser and Ng 25 and Fang and Zhang 18 .The illustration of detecting nonspherical symmetry for highdimensional data by the ST 3 -plot provides a graphical tool for goodness-of-fit problems in generalized multivariate analysis.

Concluding Remarks
Ghosh's 16 original T 3 -plot is an effective graphical method for detecting nonnormality of univariate data.The T 3 -plot was extended to detecting nonmultinormality of highdimensional data by Fang et al. 17 .In this paper we found another application of the T 3 -plot in testing high-dimensional spherical symmetry by providing approximate local acceptance regions.The simulation results in Section 3 show that the local acceptance regions given by 3.3 have feasible performance.Although we have not been able to find an optimal projection direction in applying the ST 3 -plot to real high-dimensional data analysis, those directions used in the Monte Carlo study in Section 3 can provide potential users with a good reference.Theoretically, any projection direction subject to the condition in Theorem 2.2 can be applied to the ST 3 -plot for testing high-dimensional spherical symmetry.Some directions may perform better than others, as demonstrated in Section 3.For general purpose, the idea in analysis of principal components can provide a guideline for choosing projection directions when applying the ST 3 -plot to real high-dimensional data analysis.This is illustrated by 3.4 , 3.7 and the VDP data set in Section 3.
In this paper we emphasize the ST 3 -plot for testing spherical symmetry for the case of high dimension with a small sample size.For regular cases of testing spherical symmetry, the Q-Q plots proposed by Li et al. 11 , and those analytical methods summarized in Fang and Liang 13 , or the methods mentioned in the relatively new references in Section 1, should be used.There has been a lack of new effective methods for analysis of high-dimensional data with a small sample size since the past few years.So the ST 3 -plot in this paper sheds some additional light on the area of high-dimensional data analysis.

3
The direction based on the PC-test discussed by Läuter et al.21 .Let D be the solution matrix to the eigenvalue problem X X D D Λ, D Diag X X D I n , 3.7 where similar conditions to those on the matrices D and Λ in 3.4 are imposed on the matrices D and Λ in 3.7 to ensure the unique solution.Let D γ 1 , . . ., γ n .The three directions are chosen: 3 is carried out by generating spherical samples from the following six spherical distributions by MATLAB code.These null distributions are discussed in detail in Chapter 3 of Fang et al. 12 .Here we only point out the corresponding parameters for the chosen spherical distributions without explaining their meaning.The six chosen spherical null distributions are: 1 the standard normal distribution N d 0, I d ; 2 the multivariate t-distribution with degrees of freedom m 5; 3 the Kotz type distribution with N 2, s 1, and r 0.5; 4 the Pearson Journal of Probability and Statistics type VII distribution PVII with m

d 6 hFigure 2 :
Figure 2: ST 3 -Plots for the two centerized subsets of data in Groups A and B from the VDP data on four selected projection directions.d 1 , d 2 , d 5 , and d 6 have the same meaning as that for d 1 , d 2 , d 5 , and d 6 as in Tables1 and 2, respectively.For each plot, the region between the two dashdotted curves stands for the 99% acceptance region, the region between the two real-line curves for the 95% acceptance region, and the region between the two dashed curves for the 90% acceptance region.
Fang et al. 17 proposed the following Kolmogorov-Smirnov K-S type statistic KS n : test associated with the T 3 -plot.Large values of KS n x indicate nonnormality of the univariate sample x x 1 , . . ., x n .The exact finite-sample distribution of KS n is not readily obtained under the normal assumption but its percentiles can be well fitted by a quadratic function of 1/ √ n using the least squares method.Fang et al. 17 provided these quadratic functions of 1/ √ n for significance levels α 1%, 5%, and 10% based on a Monte Carlo study.
20w we propose a series of necessary tests for high-dimensional the dimension d of data is very large spherical symmetry by Theorem 2.2.The meaning for necessary test is the same as in Fang et al.20.That is, when the null hypothesis is not rejected, it implies insufficient information to draw a statistical conclusion from the sampled data.Instead of testing the hypothesis of spherical symmetry for an i.i.d.sample {x 1 , . . ., x n } directly, we turn to test a series of hypotheses defined by the least squares method to find the approximate relation of c d α to the sample dimension d.The following quadratic curves based on values d 10 2 50 i.e., d 10, 12, 14, . . ., 50 were obtained from the least squares fitting: 3.1 if the null hypothesis H 0d in 2.14 is true.Therefore, in simulating the percentiles of the finite-sample null distribution of KS d z in 2.17 , we can simply generate the data for z from the standard normal N d 0, I d .By generating the normal N d 0, I d data for z with 2,000 replications, we record the 100 1 − α %-percentiles e.g., α 5% of KS d z , where the KS d z in 3.1 is approximately calculated by taking the supremum on the discrete values of t 0.01 0.01 0.99 i.e., t 0.01, 0.02, . . ., 0.99 .A quadratic curve of 1/ √ d : a α b α / √ d c α /d, is fitted for the 100 1 − α %-percentiles c d α of KS d z by

Table 1 :
Type I error rates of the ST 3 -plot for testing spherical symmetry.
6 the distribution t Kotz has a similar meaning to that for the distribution given by 5 , one marginal has a d/2 -dimensional t-distribution with parameter m 5 and the other has a d 1 -dimensional Kotz type distribution with parameters N 2, s 1, and r 0.5.

Table 2 :
Power of the ST 3 -plot in testing spherical symmetry for six nonspherical distributions.