An Unbiased Two-Parameter Estimation with Prior Information in Linear Regression Model

We introduce an unbiased two-parameter estimator based on prior information and two-parameter estimator proposed by Özkale and Kaçıranlar, 2007. Then we discuss its properties and our results show that the new estimator is better than the two-parameter estimator, the ordinary least squares estimator, and explain the almost unbiased two-parameter estimator which is proposed by Wu and Yang, 2013. Finally, we give a simulation study to show the theoretical results.


Introduction
Consider the following linear regression model: where shows an × 1 vector of observations on the dependent variable, shows an × known design matrix of rank , shows a × 1 vector of unknown regression coefficients, and shows an × 1 vector of disturbances with ( ) = 0 and variance-covariance matrix Cov( ) = 2 . As we all know, the ordinary least squares (OLS) estimator OLS = ( ) −1 has been regarded as the best estimator for a long time. However, when the multicollinearity occurs, the OLS estimator is no longer a good estimator. To treat this problem, many approaches have been presented. One method is to consider the biased estimator, such as Hoerl and Kennard [1], Swindel [2], Farebrother [3], Liu [4], Sakallog lu and Akdeniz [5],Özkale and Kaçıranlar [6,7], Yang and Chang [8], and Wu and Yang [9,10]. Although these biased estimators can treat multicollinearity, these estimators have big bias. In order to reduce the bias, Crouse et al. [11] and Sakallog lu and Akdeniz [5] based on ridge estimator and Liu estimator proposed the unbiased ridge estimator and unbiased Liu estimator with prior information, respectively. The unbiased ridge estimator and unbiased Liu estimator not only can deal with multicollinearity, but also have no bias.
In this paper, we will introduce an unbiased twoparameter estimator with prior information and show some properties of the new estimator.
The reminder of this paper is organized as follows. In Section 2, we give the unbiased two-parameter estimator and comparisons with OLS, two-parameter estimator proposed byÖzkale and Kaçıranlar [7], and almost unbiased twoparameter estimator proposed by Wu and Yang [9] in the sense of MMSE criterion. The estimators of the parameters and are proposed in Section 3. A simulation study is given to explain the theoretical results in Section 4 and some conclusion remarks are given in Section 5.

Analysis of Unbiased Two-Parameter Estimator with Prior Information
In this section, we also consider the general linear regression model (1) and thus thêO LS ∼ ( , 2 −1 ) for = . Crouse et al. [11] presented the unbiased ridge estimator based on the ridge estimator and prior information , which is defined as follows: with being uncorrelated witĥO LS and ∼ ( , ). In (2), = ( 2 / ) . And in (2) the prior information is a random vector for specified mean and covariance.
Then in the following section we will give the comparisons of the new estimator with the OLS estimator, the TP estimator, and the AUTP estimator in the matrix mean squared error. Firstly, we give the definition of the matrix mean squared error (MMSE).
The matrix mean squared error (MMSE) is denoted as follows: where shows an estimator of and ( ) and bias( ) present the dispersion matrix and bias vector of , respectively.
Lemma 3 (see [13]). Suppose that is a positive definite matrix and is an nonnegative definite matrix; then Proof. Since so from the definition of MMSE, we have Then from (13) and (14), we obtain that is a nonnegative definite matrix for > 0 and 0 < < 1.
The proof of Theorem 4 is completed.

Comparison of TP Estimator and the Unbiased Two-Parameter (UTP)
Estimator. Now we state the following theorem to compare the unbiased two-parameter estimator (UTP) with the TP estimator in the sense of MMSE.

Theorem 5. The unbiased two-parameter estimator (UTP) is superior to the TP estimator in the sense of MMSE if and only if
The Scientific World Journal Thus, from (14) and (17), we obtain Since > 0, 0 < < 1 and using Lemma 2, we obtain that is nonnegative definite matrix if and only if So we can conclude that the unbiased two-parameter estimator (UTP) is superior to the TP estimator in the sense of MMSE if and only if

Comparison of AUTP Estimator and the Unbiased Two-Parameter (UTP)
Estimator. Now we state the following theorem to compare the unbiased two-parameter estimator (UTP) with the AUTP estimator proposed by Wu and Yang [9] in the sense of MMSE.

the unbiased two-parameter estimator (UTP) is superior to the AUTP estimator in the sense of MMSE if and only if
Proof. By (4), we have Thus, Now we consider the following difference: the UTP is better than the AUTP estimator.

Estimation of the Parameter and Parameter
In this section, we discuss how to estimate the biasing parameters and . 4 The Scientific World Journal From (26), if 2 is known, for a fixed , we can get an unbiased estimator of found as follows: When 2 is unknown, we use the following 2 to estimate 2 : and then an estimate of iŝ where tr( −1 ) = ∑ =1 1/ and is the eigenvalue of . Note that in (27) and (29) the estimator of may be negative. So when being in this situation, one might try to denotê= 1. Summing up these results, thêmay be presented as follows.
Case II. Assuming 2 is unknown, (ii) otherwisê * 3.2. The Estimating of the Biasing Parameter . From (26), if 2 is known, for a fixed , an unbiased estimate of is defined as follows: When 2 is unknown, similarly an estimate of iŝ Note that in (34) and (35) the estimator of may be negative. So when being in this situation, one might try to denotê= 0. However, there always exists a such that the unbiased twoparameter estimator̂( , ) has smaller MSE than̂O LS .
Case I. Assuming 2 is known, Case II. Assuming 2 is unknown, where 2 = ( −̂O LS ) ( −̂O LS )/( − ) is an unbiased estimator of 2 . In applications there may be other estimates of 2 that may also be used. It is worthwhile to point that the proposed and provide an unbiased two-parameter estimator of while the twoparameter estimator is biased.

A Simulation Study
In this section, we will give a simulation study to explain the theoretical results. Following McDonald and Galarneau [14], the explanatory variables are produced using the following device: = (1 − 2 ) + ( +1) , = 1, . . . , , = 1, . . . , , where and ( +1) show independent standard normal pseudorandom numbers and is specified so that the correlation between any two explanatory variables is given by 2 .

Conclusion
In this paper, we introduce an unbiased two-parameter estimator with prior information. We also show the superiority of the new estimator over the OLS estimator, the TP estimator, and the AUTP estimator in the MMSE sense. Furthermore, the estimators of the biasing parameters are also discussed in this paper.