Comparison of Some Estimators under the Pitman ’ s Closeness Criterion in Linear Regression Model

Batah et al. (2009) combined the unbiased ridge estimator and principal components regression estimator and introduced the modified r-k class estimator. They also showed that the modified r-k class estimator is superior to the ordinary least squares estimator and principal components regression estimator in the mean squared error matrix. In this paper, firstly, we will give a new method to obtain the modified r-k class estimator; secondly, we will discuss its properties in some detail, comparing the modified r-k class estimator to the ordinary least squares estimator and principal components regression estimator under the Pitman closeness criterion. A numerical example and a simulation study are given to illustrate our findings.


Introduction
Consider the following multiple linear regression model: here,  is an  × 1 vector of observation,  is an  ×  known matrix of rank ,  is a  × 1 vector of unknown parameters, and  is an ×1 vector of disturbances with expectation () = 0 and variance-covariance matrix Cov() =  2   .The ordinary least squares estimator (OLSE) of  is given as follows: The OLSE is no longer estimator in the existence of multicollinearity.So in order to reduce the multicollinearity, many remedial actions have been proposed.One popular method is considering the biased estimator.The best known biased estimator is the ridge estimator introduced by Hoerl and Kennard [1]: As we all know  → ∞, β() approaches 0 which is a stable but biased estimator of .
Crouse et al. [2] proposed the unbiased ridge estimator as a convex combination of prior information with the OLSE estimator, which is given as follows: where  is a random vector with  ∼ (,( 2 /)) and  > 0. Özkale and Kac ¸iranlar [3] use two different ways to propose the unbiased ridge estimator and they also compared the unbiased ridge estimator with the OLSE, principal components regression estimator, ridge estimator, and - class estimator under the mean squared error matrix.Another popular way to combat the multicollinearity is the principal components regression (PCR) estimator [4].For this, let us consider the spectral decomposition of the matrix    given as where Λ  and Λ − are diagonal matrices such that that the main diagonal elements of the Baye and Parker [5] introduced the - class estimator which is given as follows: Batah et al. [6] combined the PCR estimator and unbiased ridge estimator and proposed the modified - (-) class estimator: where  is a random vector with  ∼ (, ( 2 /)) and  > 0.
The - class estimator β (, ) has the following properties: Batah et al. [6] also compared the - class estimator to OLSE, PCR, and - class estimator in the sense of mean squared error matrix, and obtained the necessary and sufficient conditions for the - class estimator superior over the OLSE and PCR.
Though mean squared error matrix has been regarded as the primary criterion for comparing different estimators, Pitman [7] closeness (PC) criterion has received a great deal of attention in recent years.Rao [8] has discussed the similarities and differences of mean squared error and PC and has aroused great interest in PC.The monograph by Keating et al. [9] provided an illuminating account of PC and a long list of publications on comparisons of estimators of scalar functions of univariate parameters [10].After that, many authors have used PC to compare estimators, such as, Wencheko [11] who compared some estimators under the PC criterion in linear regression model, Yang et al. [10] compared two linear estimators under the PC criterion, and Ahmadi and Balakrishnan [12,13] compared some order statistic under the PC criterion.Jozani [14] studied the PC using the balanced loss function.
Though, in most cases, the PC criterion is more suitable for comparing estimators, in this paper, firstly, we give a new method to obtain the - class estimator; then we will give the comparison of the - class estimator with the OLSE and PCR; we will obtain under certain conditions that the - class estimator is superior to the OLSE and PCR estimator in the PC criterion.
The rest of the paper is organized as follows.In Section 2, we will give a new method to obtain the - class estimator and the comparison results are given in Section 3. In Sections 4 and 5 we will give a numerical example and a simulation study to illustrate the behaviour of the estimators, respectively.Finally, some concluding remarks are given in Section 5.

The 𝑚𝑟-𝑘 Class Estimator
The handling of multicollinearity by means of PCR corresponds to the transition from the model ( 1) to the reduced model by omitted  − .
We suppose that there are stochastic linear restrictions on the parameter  as where  is an  ×  matrix of rank  ≤ , ℎ is an  × 1 vector, and  is an  × 1 vector of disturbances with mean 0 and variance and covariance  2 . is assumed to be known and positive definite.Furthermore, it is also supposed that the random vector  is stochastically independent of .Now, let us consider that the restriction (11) as ℎ =   +.Under the idea of the PCR, the original restriction (11) becomes where   =   .Then, Wu and Yang [15] introduced the following estimators: Let ℎ be a random vector.The expectation and covariance of β is given as: Now, we let ℎ = ,  = ,  = (1/)  , and  = 0; then (13) equals the - class estimator, that is, In the next section, we will give the comparison of the - class estimator to the OLSE and PCR estimator under the PC criterion.

Superiority of the 𝑚𝑟-𝑘 Class Estimator under the PC Criterion
Firstly, we will give the definition of the PC and PC criterion.

Numerical Example
To illustrate our theoretical results, we now consider in this section the data set on total national research and development expenditure as a percent of gross national product originally due to Gruber [16] and later considered by Akdeniz and Erol [17].In this paper, we use the same data and try to show that the - class estimator is superior to the OLSE and PCR estimator.Firstly, we assemble the data as follows: Then, the values of PC1 and PC2 are computed in Figures 1 and 2, respectively.
From Figure 1, we can see that the values of PC1 are not always bigger than 0.5; that is to say, the - class estimator is not always superior to the OLSE, which is agreeing with our Theorem 3. When we see Figure 2, we may see that the values of PC2 are always bigger than 0.5; that is to say, the - class estimator is always superior to the PCR.

Simulation Results
In order to further illustrate the behaviour of the - class estimator, we are now to consider a Monte Carlo simulation by using different levels of multicollinearity in this section.The explanatory variables are generated by the following equation [18]: where   are independent standard normal pseudorandom numbers and  is specified so that the correlation between any two explanatory variables is given by  2 .Then, the observations on the dependent variable are then generated by where   are independent normal pseudorandom numbers with mean zero and variance  2 .In this simulation study, we choose  = 50,  = 4, and  = (1, 2, 2, 4)  .The simulation results are given in Table 1 From the simulation results in Table 1, we see that, in most cases, the - class estimator gives better performance than the OLSE, which agrees with our theoretical results.And the - class estimator is always better than the PCR estimator.So by the numerical example and simulation study, we can see that the - class estimator is better than the PCR estimator.

Concluding Remarks
In this paper, firstly, we give a new method to propose the - class estimator.Then, we compare the - class estimator to the OLSE and PCR estimators under the PC criterion.The comparison results show that, under certain conditions, the - class estimator is superior to the OLSE.Finally, a numerical example and a simulation study are given to illustrate the theoretical results.

) 3 . 1 .
Comparison of the - Class Estimator and the OLSE under the PC Criterion.Now, we give the comparison of the - class estimator and the OLSE under the PC criterion.

Theorem 4 .
Class Estimator and the PCR under the PC Criterion.Now we give the comparison of the - class estimator and the PCR under the PC criterion For  > 0, the Pitman measure of closeness (PMC) of the - relative to the PCR estimator is given as follows:

Figure 1 :
Figure 1: The PC of - class estimator relative to OLSE.

Figure 2 :
Figure 2: The PC of - class estimator relative to PCR.

Table 1 :
The values of  for different values of  and .