A Variance Shift Model for Detection of Outliers in the Linear Measurement Error Model

and Applied Analysis 3 l ∗ i (β, σ 2 , w i , X, y)


Introduction
Outliers are observations that appear inconsistent with the rest of the data set and can have a profound destructive influence on the statistical analysis.To detect these kinds of observations in the linear models, different approaches have been suggested, among those one can refer to the casedeletion and variance shift models.The first approach is based on the assumption that outliers result from a shift in the mean of contaminated observations (see Barnett and Lewis [1] or Weisberg [2]) and the second procedure takes into account the assumption that an outlier arises from an error term with an increased variance (Cook and Weisberg [3]).Cook et al. [4] indicated that the maximum likelihood estimates for the position of the outlier under two methods could be different, unless the largest absolute studentized residual corresponds to the largest absolute residual.Using the residual maximum likelihood (REML), Thompson [5] showed that the residual variance and outlier position are the same under both methods.
In the linear mixed models, case deletion method, variance shift outlier model, and related diagnostics are studied widely by different authors.Christensen et al. [6] presented case deletion diagnostics for both fixed effects and variance components models.Banerjee and Frees [7] introduced case deletion diagnostics for both fixed effects and random subject effect in linear longitudinal models.Xuping and Bocheng [8] presented a unified diagnostic method for linear mixed models based upon the joint likelihood given by Robinson [9].They showed that the estimates of parameters in case deletion method are equivalent to those in mean shift outlier model.Haslett and Dillane [10] proved a "delete = replace" identity in linear models and applied it to deletion diagnostics for estimators of variance components.Zewotir and Galpin [11] provided routine diagnostic tools for fixed effects, random effects, and variance components, which are computationally inexpensive.Li et al. [12] considered subset deletion diagnostics for fixed effects, random effects and one variance component in varying coefficient mixed models.Gumedze et al. [13] extended the variance shift outlier model (VSOM) to the linear mixed model.
In linear regression models, independent variables are often susceptible to nonnegligible errors, and then it will be more appropriate to consider the measurement error models (see Fuller [14] and Stefanski [15]).However, in measurement error models ordinary maximum likelihood (ML) estimates lose the consistency.In order to correct the bias on parameter estimation, a method in which the score function itself is corrected for measurement errors, is available for the estimation of parameters.This method is based on the corrected log-likelihood of Nakamura [16] (see also Giménez and Bolfarine [17] for more discussion).
On diagnostic methods for measurement error models, some previous works are due to Kelly [18] and Wellman and Gunst [19].Zhong et al. [20] obtained case deletion and mean shift outlier model for linear measurement error models based upon the corrected likelihood of Nakamura [16].Rasekh [21] studied multiple outlier detection in multivariate functional measurement error models based on the suitable definition of standardized residuals.Giménez and Galea [22] studied influence measures on corrected score estimators in functional heteroscedastic measurement error models and a local influence study on functional comparative calibration models with replicated data is developed by Giménez and Patat [23].
In this paper, we concentrate on the variance shift model of Cook et al. [4], for the linear measurement error model, using the corrected likelihood [16].In Section 2, we present the basis of the corrected score method and obtain the estimates of parameters of the model.In Section 3, a variance shift model for the linear measurement error model is derived and the joint corrected maximum likelihood estimates are characterized.In Section 4, we develop the likelihood ratio and the score test statistics.Furthermore, a parametric bootstrap procedure is used to generate the empirical distribution of these statistics.In Section 5, to verify the performance of the proposed test statistics, a simulation study is reported and finally, an illustrative example is given in Section 6.

Corrected Log-Likelihood of Measurement Error Models
Consider the linear measurement error models: where  = ( 1 ,  2 , . . .,   )  ,  is an × matrix of unobservable regressors,  is a  × 1 vector of unobservable parameters, and  2 is the unknown common variance.The matrix  is the observed value of  with the measurement error .Furthermore,  and  are independent, Λ is a  ×  matrix of known values with nonnegative diagonal elements (Fuller [14]) and   is an × identity matrix.The model ( 1) is known as functional linear measurement error model.The log-likelihood of  is given by If we replace  by  without considering the measurement errors, then the ML estimates are not consistent in general.

A Variance Shift Model in the Linear Measurement Error
Suppose the th observation is considered with inflated error variance.A variance shift model for this observation takes the form where   is an  × 1 vector with value 1 in the th element and zero elsewhere and   is an unknown random coefficient of the form (0,   ).Model (6) can be considered as a linear mixed measurement error model in which   is a random effect with the variance   and the covariance matrix of the data are as follows: where   =   / 2 + 1 and Var (  ) =    2 , for   ≥ 1.
The log-likelihood and the corrected log-likelihood for this model are given by respectively, which have the following properties: Now, for   fixed, the corrected score estimates of  and  2 will be obtained with differentiating from the corrected loglikelihood of the variance shift model given in (8) with respect to  and  2 [24].Then we have respectively.In the following theorem we derive the asymptotic expressions of the β (  ) and σ2  (  ) as functions of β and σ2 given no outliers.
Theorem 1.For model (6), we have where Proof.From the estimate of  given in (10), we have On the other hand, we know that, , and [ +   ( −1 )] −1 =  +   ( −1 ) (see the appendix for more details).Therefore, the first term in the right hand side of (13) will be and the second term is Combining both terms of (13), we have Consequently, we have Multiplying the above matrix expression out and simplifying them, β (  ) will be derived.Next, substituting β (  ) in the estimate of  2 given in (11), we can write and hence, σ2  (  ) will be obtained.
In the rest of paper, we define β (  ) = σ2 and for   > 0 we have Remark 2. β and σ2  are the corrected score estimates of  and  2 , respectively, in a mean shift outlier model for th observation, given by [20,25].Now  *  (,  2 ,   , , ) evaluated at (,  2 ) = ( β (  ), σ2 (  )) is, except for an additive constant, proportional to Using Taylor series expansion, an approximate to the ℎ (  ) analogues to one given by Cook et al. [4] will be as The existence of the value of   , say w , over the range of [1, +∞) that maximizes h (  ), was proved by Cook et al. [4].

Analogue of Likelihood Ratio Test and Score Test Statistics
In this section we derive analogues of likelihood ratio test and score test statistics for testing the null hypothesis that the observation is not unusual (  = 1) against the alternative that it has an inflated variance,   > 1.
Because the null hypothesis is on the boundary of the parameter space, the standard asymptotic theory does not apply in this case [27].Therefore, a parametric bootstrap procedure can be used to approximate the distributions of the likelihood ratio and the score test statistic (see Section 4.3 and [13]).

Empirical Distribution.
Based on the Gumedze et al. [13] the following parametric bootstrap procedures, for test statistics CLRT  and SC  , can be used to derive the empirical distributions of these statistics under the hypothesis of no outliers exist in the observations: Step 1. Fit model (1) to the data and calculate estimates Ẑ (see Zare and Rasekh [28]), β and σ2 , where Step 2a.Generate a new data vector from where  * is randomly generated as (0, σ2   ) and  is randomly generated as  (0,   ⊗ Λ).Fit model (1) to  * .
Step 3. Repeat Steps 2a and 2b  times, for  acceptably large, for example,  = 10000.Therefore, an empirical distribution of size  is generated for each order statistic.
Step 4. Calculate the 100(1 − ) percentile for each order statistic for the level of size .
The percentile of the th order statistic can be considered as a threshold for the th largest value of the test statistic from the original data and if the  largest values of the test statistic from the original data all exceed their respective thresholds, then it is concluded that these are all outliers.

Simulation Study
A parametric bootstrap simulation study is carried out to demonstrate the empirical performance of the various proposed test statistics in terms of the probability of a type  error and power on a single unit.
For each simulated data set, the CLRT and the score test statistics were calculated for the first observation.The choice of the first observation was arbitrary.To generate an empirical distribution of the test statistic under the null hypothesis, data sets for  = 1, 2, . . ., 2500 were simulated as  *  = Ẑ β +  *  , where  *  ∼ (0, σ2    ), β , Ẑ , and σ2  are the corrected estimate of ,  and  2 from   .The probability of a type  error estimate for a given test statistic and  = 0.05 was calculated as the number of data sets for which the test statistic exceeded the 95th percentile of the empirical distribution, divided by 1100 [13].
The CLRT and the score test statistics were performed for a variance shift model for the first observation of each simulated data and 95th percentiles from the empirical distribution of each test statistics were used as threshold values for the test statistics observed on the original data set   .The empirical probability of type  errors for thresholds derived from the empirical distribution under the null hypothesis are calculated for the corrected likelihood ratio and score test statistics for  = 0.05 (Tables 1, 2, and 3).A glance at the results of these tables indicate that in general the probability of a type  errors of both CLRT and score test statistics are close to the nominal value of 0.05.
In order to access the relative sensitivity of the CLRT and score test statistics, we introduce the shift values 1, 3, and 5 for the first observation and again for each combination of parameters, 1100 data sets are generated from the following model: for  = 1, 3, or 5, where  1 is an  × 1 vector with value 1 in the first element and zero elsewhere.The CLRT, the score test statistic, and their empirical distribution were calculated as for the probability of a type  error given above, consequently the power of the test statistics are also derived.Results of Tables 1-3 show that with increase of the displacement, , the power of the CLRT and score test statistic, increases in general.Moreover, we can see that power of the test statistics also increase as sample size increases.These tables also show the result of the CLRT and score test statistics are nearly identical in the empirical probability of a type  errors and power.

Example: Concrete Compressive Strength Data
These data were given by Wellman and Gunst [19] and contain comprehensive strength measurements of 41 sample of concrete.It was desired to use a linear regression model to predict comprehensive strength of concrete 28 days after pouring from the strength measurements taken two days after pouring.Zhong et al. [20] analyzed this data set using the linear measurement error model with Λ = diag (0, 1).with relatively large values of  2  and w and a small estimated σ2  .
Next, the corrected likelihood ratio and score test statistics were calculated for each observation under model (6), and then 10000 simulated data sets were generated from the fitted model under the null hypothesis ( 0 :   = 1).In each simulation, a variance shift model was fitted for each observation and the test statistics were sorted and used  to generate the empirical distribution of the order statistics for each test [13].Figures 4 and 5 give plots of the test statistics from the real data and 95th percentile from the empirical distribution of the first, second and third largest values for each test statistic.These figures show that the statistics for observation 21 is larger than the 95th percentile of the distribution of the corresponding order statistics.Finally, the fitted regression line measurement error model and the variance shift measurement error model for case 21 are shown in Figure 6.

Conclusions
We extended the variance shift model to the linear measurement error models based on the corrected likelihood of Nakamura [16].We derived the approximate estimate of parameters of the proposed model under the variance shift and indicated that if the variance shift parameter tends to infinity, these estimates will be the same as those obtained from a mean shift outlier model.Also, we proposed a corrected likelihood ratio test and derived the score test statistic for testing that an observation stands out as possible outlier and it is shown that the score test statistic is function of studentized residuals of model.The performance of both the corrected likelihood ratio and the score test statistics is studied using a parametric bootstrap simulation, and it was found out that with the increase of  or sample size, the power of both test statistics increases.

Figure 5 :Figure 6 :
Figure 5: Score test statistic for each case, with 95th percentile of the empirical distribution under  0 shown for the first  order statistics for score test:  = 1 (solid line),  = 2 (dashed line), and  = 3 (dotted line).
2, , ), which satisfies In order to derive the score test statistic, we only require the estimate of parameters under the null hypothesis.In the following theorem based on the observed information matrix we obtain this test statistic under the hypothesis of   = 1.The score test statistic for the th observation (  ), based on the observed information matrix for testing  0 :   = 1, is given by σ2 , , ) −  *  ( β (w  ) , σ2  (w  ) , w , , )] .(23) 4.2.Score Test Statistic.

Table 1 :
The probability of a type Ι error ( = 0.05) and power of CLRT and score test statistics for a variance shift model in a linear measurement error with combination different of parameters ,  2 , Λ, and  = 1.

Table 2 :
The probability of a type Ι error ( = 0.05) and power of CLRT and score test statistics for a variance shift model in a linear measurement error with combination different of parameters ,  2 , Λ, and  = 3.

Table 3 :
The probability of a type Ι error ( = 0.05) and power of CLRT and score test statistics for a variance shift model in a linear measurement error with combination different of parameters ,  2 , Λ, and  = 5.
(6) zero diagonal element in Λ corresponds to the constant term, a predictor variable measured without error.They indicated that the sample 21 exhibits a strong influence on the fitted model.Here we consider a variance shift measurement error model for this data set.Figures1, 2, and 3 show plots of the square of studentized residual  2  of the data under model (1), estimates of the variance shift parameter w , and estimated variance σ2  under model(6), versus case numbers, respectively.From these figures it is obvious that case 21 stands out as a possible outlier