Detection of Outliers in Panel Data of Intervention Effects Model Based on Variance of Remainder Disturbance

Thepresence of outliers can result in seriously biased parameter estimates. In order to detect outliers in panel datamodels, this paper presents a modeling method to assess the intervention effects based on the variance of remainder disturbance using an arbitrary strictly positive twice continuously differentiable function.This paper also provides a Lagrange Multiplier (LM) approach to detect and identify a general type of outlier. Furthermore, fixed effects models and random effects models are discussed to identify outliers and the corresponding LM test statistics are given.The LM test statistics for an individual-basedmodel to detect outliers are given as a particular case. Finally, this paper performs an application using panel data and explains the advantages of the proposed method.


Introduction
Outliers are observations in the dataset that appear to be unusual and discordant.If the sample contains outliers, the inappropriate estimation from contaminated observations may be strongly distorted and leads to unreliable results.Intervention effects may cause serious bias in estimating parameters as explained in the work of Fox [1], Martin and Yohai [2], Chang et al. [3], and Verardi and Croux [4].Therefore, it is very important to identify these outliers in large datasets for both natural science and social science disciplines such as engineering, biology, education, medicine, economy, and sociology.
Detection of potential outliers plays a very important role in obtaining valuable and accurate information particularly in the field of engineering.An outlier of engineering observations may be due to an error in data transmission or human error in measurement.Some experts in the field of engineering noted the importance of identification of outliers.Febrero et al. [5] highlighted its importance: the analysis of outliers is an important aspect of any statistical analysis of data and especially is important to identify days or periods in which the NO  levels are significantly large.Outlier identification is important to analyze the traffic volume data collected and used for a variety of purposes in intelligent transportation system [6].Outlier detection is also important in the field of ecological engineering, as identification of atypical observations is an important concern in water quality monitoring [7].
Initial researches in outlier detection focused on time series-based outliers.Some influential studies have made contributions to the detection and identification of outliers [1,3,[8][9][10][11][12][13].Fox [1] proposed a fundamental approach for detecting and identifying outliers in a time series model.For linear regression models, it is a general approach to detect outliers to employ the mean-shift outlier model and some test statistics equivalent to the external studentized residuals [8,9].Chang et al. [3] provided an iterative procedure to detect and identify the outliers.Two distinct kinds of outliers are considered, namely, additive outlier (AO) and innovation outlier (IO), and the outliers in time series are regarded as being generated by dynamic intervention models at unknown time points.In recent years, researchers begin to be concerned about the outliers' detection of more complicated models.Shi and Chen [14] developed outlier detection for multilevel models and the proposed test can be used to detect outliers at any level in multilevel models and for any combinations of units.Chen [15] provided an approach to estimate the panel data model with a mixed fractional ARIMA remainder process, in which the data may contain different types of outliers.By the modified inverse Fourier transform, the outliers for the spectral Whittle approach can be quickly detected and identified.Willems et al. [16] diagnosed multivariate outliers.Riani et al. [17] found an unknown number of multivariate outliers.Cerioli [18] developed multivariate outlier tests based on the highbreakdown Minimum Covariance Determinant estimator.Yan [19] proposed a novel method integrating self-organizing map (SOM) with adaptive nonlinear map (ANLM) for multivariate outlier detection.Yuen and Mu [20] proposed a novel probabilistic method for robust parametric identification and outlier detection in linear regression problems.The crux of this method is to calculate the probability of outlier, which quantifies how probable a data point is an outlier.Rapallo [21] made use of log-linear models and exact goodnessof-fit tests to specify the notions of outlier and pattern of outliers.Kuhnt et al. [22] introduced a new technique for the detection of outliers in contingency tables, where outliers are unusual cell counts with respect to classical log-linear Poisson models.
It is very difficult to identify outliers directly by eye, especially when faced by a panel data model with unfamiliar features and uncertain large datasets.As pointed out in Bramati and Croux [23], outliers are not always detectable by looking at residuals from a least squares fit, and diagnostic measures like the Cook distance suffer from the masking effect, as soon as multiple outliers are present.Not much effort has been given to the diagnostics and influence assessment of outliers in panel data models.Although a few researchers tend to be aware of this, there is little literature on the detection of outlier in a panel data model as an important issue.
This paper focuses on the detection and identification of outliers in panel data models with individual effects.Because of the presence of individual effects in panel data models, the traditional mean-shift model cannot differentiate individual effects from the mean disturbance.Thus, the mean-shift model cannot be applied in detecting outliers for panel data models with individual effects.A panel data model with outliers is likely to contaminate the residuals.This means that the variance of error term probably has a large deviation in the outlier model.Therefore, in this paper, a variance intervention effects model is proposed to study the detection of outlier.This paper is concerned with outlier detections through a method of variance intervention effects on the remainder disturbance using arbitrary strictly positive twice continuously differentiable function.Even if the error term  was observable in Baltagi [24], the equations of maximum likelihood (ML) estimator would still be highly nonlinear and difficult to solve explicitly.The test statistics based on Lagrange Multiplier (LM) approach are derived, since this LM test is based on the parameters' estimation under the null hypothesis and its computation is simple only requiring residuals.This paper focuses on a more general type of outlier that has specific impacts on subsequent observations, of which an individual outlier model is a particular case.The test statistics of a general type of outlier and an individual outlier are, respectively, calculated through Lagrange Multiplier (LM) approach.Furthermore, this paper would demonstrate outlier detection of fixed effects models and random effects models by the corresponding LM test statistics.
The rest of the paper is organized as follows.Section 2 briefly presents panel data models with individual effects.Section 3 proposes the variance intervention effects outlier model based on the remainder disturbance.Section 4 describes maximum likelihood estimator.Section 5 provides an LM testing approach for the detection and identification of a general type of outlier.Furthermore, fixed effects and random effects models are discussed with outliers and the corresponding LM test statistics are given.The LM test statistics of an individual outlier model as a particular case are given.Section 6 performs an application of the proposed method using a panel data and explains the advantages of the method.Finally, Section 7 provides the concluding remarks.Proofs of the main results are provided in the appendix.

Panel Data Models with Individual Effects
Firstly, the following panel data model with individual effects is considered: where the subscript  denotes individuals and  denotes time.This means that  represents the cross-section dimension and  denotes the time series dimension.Here   is a dependent variable observed for individual  at time ,   is a  × 1 column vector of observable independent variables (   = ( 1 ,  2 , . . .,   )),  is a  × 1 column vector of regression parameters,   is the unobserved time-invariant individual effect, and   denotes the remainder disturbance term that is uncorrelated over time and across cross-sectional units.
The panel data model can be represented in a matrix form, and then where  = (

An Intervention Effects Outlier Model Based Variance
In a panel data model the standard assumption is that the majority of data follow a certain specified distribution.Unfortunately, a certain small percentage of the panel data take values unlikely to follow this same distribution.The residuals are likely to be contaminated with outliers in a panel data model.This means that the error term has the deviation of variance in the model.Since the main purpose of this paper is to propose a method of outlier detection based on the remainder disturbance, it will be assumed that there is no intervention effects problem based on variance if   is present, and therefore this paper will not deal with the inference with the variance of   .The following is the panel data model for (1): =   +     +   ,  = 1, 2, . . ., ;  = 1, 2, . . ., . (3) Suppose outliers interfere with the remainder disturbance   and there are the following two conditions.One is variation across both individuals and time where var(  ) =  2  ; the other is variation across only individuals where var(  ) =  2   .Here deviations of the variance across only individual can be assumed, so the variance of the remainder disturbance will be var where   (  ) is arbitrary strictly positive twice continuously differentiable function satisfying the conditions   (  ) > 0,   (0) = 1, and  (1)   (0) ̸ = 0.  (1)   () denotes the first derivative of   () with respect to .
The panel data outlier model is based on the variance of remainder disturbance in a matrix form, and then where ),   (   2 ), . . .,   (    ); then ; here  = { 1 ,  2 , . . .,   }.   denotes a  ×  matrix with its th diagonal matrix being   and other matrixes being zeros.
Remark 1.The research scholars of science subjects such as applied mathematics, statistics, and engineering have made active and useful contributions to the detection of outliers and provided some methods to identify outliers, such as Grubbs test, -test, Dixon test, and Nair test.Grubbs test and Dixon test are not generally effective against identification of multiple outliers.-test calculation is more complex and generally used for small samples.Nair test requires variance which is assumed to be known.A number of observations do not contain only one outlier, but there are several outliers; thus, the traditional methods especially for detection of a single outlier have less resistance for the pollution of multiple outliers and are highly likely producing the shielding effect once when there are multiple outliers in the sample data.
Remark 2. The outlier model with intervention effects based variance can be applied in a wide range and considers the case in which there are multiple outliers in the observations.By hypothesis testing to construct a valid test statistic, this model develops a new method for the detection of multivariate outliers.It has a strong ability to resist the pollution from outliers.This method can avoid the shielding effect due to the presence of multivariate outliers and also can be used continuously for the detection of multiple outliers.

Maximum Likelihood (ML) Estimation
Under the assumption of normality, the log-likelihood function for the model ( 2) can be written as where  =  −  and   =   −   .Ω is the variance-covariance matrix of the error term .The variancecovariance matrix can be computed as where and   = (1, 1, . . ., 1)  is the  × 1 column vector.In order to obtain the ML estimator of the regression coefficients, Ω −1 needs to be computed as The maximum likelihood estimators of ,  2  , and  2  are obtained by solving the following normal equations: It can be noted that these equations would be nonlinear and difficult to solve explicitly.Let β, σ2 , and σ2  denote the ML estimates of ,  2  , and  2  .

Outlier Detection and Testing
In the following context, outliers in panel data will be detected and the corresponding LM test for the variance intervention effects model will be derived.

LM Test with a General Type of Outlier for 𝐻
The log-likelihood function for variance intervention effects model can be written as where  = ( 2  ,  2  , )  = {  }.

Mathematical Problems in Engineering
The Hessian matrix of  can be obtained: Following Magnus [25], Lejeune [26], and Baltagi [24], the information matrix is given by  = − 0 [ | ; , ] denoting expectation taken with respect to the true distribution.To calculate the information matrix, it is noted that  2 /  =   Ω −1  and the first derivatives of the likelihood function with respect to , evaluated at the restricted MLE of , are zero.Thus, cov(/, /) = 0.
Thus the information matrix under the null hypothesis is The information matrix is block-diagonal between  and , and the part of the information matrix corresponding to  is ignored in computing the LM statistic since the null hypothesis only involves .Therefore, the LM statistic may be written as where D = /( ρ) is a vector of partial derivatives of the log-likelihood with respect to each element of , evaluated at the restricted MLE ρ. ψ = [− 2 /  ]( ρ) is the information matrix, evaluated at the restricted MLE ρ.Under the null hypothesis, this statistic is asymptotically distributed as a  2 with   degrees of freedom,   being the number of parameters in the vector .
The first derivative D can be obtained as For the information matrix, the formula is given by Baltagi [24]: Based on (10), log-likelihood function may be written as Here Testing for outliers in this model  0 : Ω  =  )  , and under the null hypothesis  0 , the inverse matrix of the covariance matrix is where Under the null hypothesis  0 , this statistic is asymptotically distributed as  2 with  degrees of freedom.

𝐿𝑀 Test of Fixed Effects or Random Effects Model with
Outliers for  0 :   =0.This section would consider fixed effects model of panel data with outliers and random effects model of panel data with the th individual outliers.
Proposition 4. If  > 1 and  2  = 0, so the LM statistic for fixed effects model of panel data with a number of individuals outliers is given by where .Under the null hypothesis  0 , this statistic is asymptotically distributed as  2 with  degrees of freedom.Proposition 5.If  = 1 and  2  > 0, so the LM statistic for random effects model of panel data with the th individual outliers is given by where ũ ).Under the null hypothesis  0 , this statistic is asymptotically distributed as  2 with one degree of freedom. where .Under the null hypothesis  0 , this statistic is asymptotically distributed as  2 with one degree of freedom.

Application Analysis
In this section, an application of the above proposed method using a panel data will be performed and the advantages of the proposed method will be explained.
To evaluate the performance and advantages of the proposed method to detect outliers in panel data, a sample is used as the test dataset to explore main variables' effects on regional environment.The dataset consists of 570 observations for 30 provinces in China from 1992 to 2010.The basic model is ln CO2 , =   +  1 ln EX , +  2 ln IM , +  3 ln GP , +  , (24) with the regional carbon emissions CO 2 as dependent environmental variable, regional export volume EX and import volume IM as main independent trade variables, and regional gross domestic product GP as main regional development level variables.The source data of independent variables are mainly downloaded from National Bureau of Statistics of China and carbon emissions are obtained by following the method of Lin and Sun [27] based on the quantity of fossil fuel consumption data and the CO2 emissions factors of various types of energy from the Intergovernmental Panel on Climate Change (IPCC) reference approach.

Outlier Detection of the Tested Panel Data.
Although the residual plots (Figure 1) of the basic model show that some data points can be initially judged as abnormal, detection of the outliers only from the residual plots is not strict enough because the standard of diagnostic of outliers from residual plots is very vague and even sometimes we will encounter some tougher cases to judge outliers from residual plot.This paper would apply the LM test method to identify outliers accurately.
For those nonidentified outlier tests, which mean that we do not know in advance which data is outlier point, the test statistic LM max = LM  .The Bonferroni inequality will be used to approximate the function, and then the process to identify nonidentified outliers with the confidence level as  will be as follows: if LM max ≥  2 / (1) ( is sample size), the data estimated will be outliers.
Let the confidence level  = 0.05, and then the critical value  2 0.05/30 (1) = 9.88 for detection of nonidentified outliers.Thus, we can find that individuals such as #5, #19, #21, and #29 in Table 1 are detected as outliers as their LM test statistics are more than the critical value.
When data analysis, it can be probably happened that univariate does not meet the characteristics of structure and ) . ( When taking into account the presence of outliers, the regression results of correction models are compared with the basic model and the estimated value is changing. value of the basic model shows that the coefficient of ln IM does not pass the test, while the coefficient of the correction model 1 passes the test at 10% confidence level and the correction model 2 passes the test at 5% confidence level.The coefficients of other two variables (ln EX, ln GP) in two correction models pass the test at 1% confidence level.The results in Table 2 show that the indicators of  value and std.err.are improving in two correction models; thus, it can be explained that the presence of outliers does affect the accuracy of the estimated value and correction based on the suggested method allows the regression results to be more accurate.
To further evaluate the regression results of correction models, the residuals will be tested whether they are stable or not, following the methods such as LLC test [28], HT test [29], IPS test [30], and Fisher-ADF test [31].Table 3 shows the results of the diagnostic test for residuals of panel data.The results show that the correction models have smooth residuals of panel data at the 1% significance level, which indicates that regression results of correction models are better than the basic model with evaluated robustness.
This application case shows the efficiency of LM test to identify outliers and reflects the good effect of detection through avoidance of shielding effect.

Conclusion
The presence of individual outliers' properties in panel data can affect parameter estimates and statistical test in a dramatic way.There are some literatures available on time series model with respect to outlier, whereas there are very few studies on panel data models.In case of the existence of individual effect, the traditional mean-shift model cannot be applied in detecting the outliers for panel data with individual effect.Therefore, this paper developed the variance intervention effects model based on the remainder disturbance to detect outliers.Note that the equations of MLE with outliers' model would be nonlinear and difficult to solve explicitly and directly.These methods of the outlier detection are available through the Lagrange Multiplier (LM) test in panel data.The accurate distribution of the test statistics is not available.Although the test statistics based on LM are asymptotically distributed as  2 , some researches prove that LM test is also very effective even in small samples.This paper presented a more effective LM testing approach for a general type of outliers' detection based on variance intervention effects model.Furthermore, the corresponding LM test statistics of fixed effects model and random effects model with outliers are obtained.The LM test statistics of an individual outlier model as a particular case are given.This maximum likelihood estimation (ML) can be performed in Stata by using the regress command.Through the implemented estimation, the ML estimates of  2  and  2  are obtained under the null hypothesis.The LM test statistics of outlier detection for each individual are calculated as proposed.Following the procedure, this paper recognized and identified individual outliers by using the results of the LM test statistics.Outlier detection and identification can provide some additional and valuable information which improve the robust direction of statistical model.

Appendix
This appendix derives the LM test for the intervention model with the variance of   , and the null hypothesis is given by  0 :    = 0.Under  0 , the following are obtained: (ũ    ũ   ) . (A.6) Using (15), the elements of the information matrix are given by  ) . (A.10) Next, the corresponding Lagrange Multiplier (LM) test for null hypotheses  0 is derived:

Figure 1 :Proposition 6 .
Figure 1: Residual plots of the basic model.
It is the th individual variance interventions' effects model.If the outliers impact not only the th individual observations, but also the subsequent observations, then without loss of generality, it can be assumed that the outliers employ sequence impacts, which are respectively,   (   1 2  [diag(  ,   , . . .,   (   )  ,   , . . .,   )] and    ̸ = 0.   (   )  denotes the th diagonal matrix.

Table 1 :
Outlier detection results of the basic model through LM test.

Table 2 :
Regression results of the basic and correction models.

Table 3 :
Unit root test results of correction models., =   +  1 ln EX , +  2 ln IM , +  3 ln GP , +  , , The null hypothesis of LLC, HT, IPS, and Fisher-ADF test is the data which contain a unit root.correlation among the variables.With the LM test method, significant outliers which could disrupt such relationship can be effectively identified, while the traditional test methods are not in a position to deal with this.Outliers can provide us with valuable information; for example, some significant discoveries often can be obtained on the condition of observations beyond the degree of dispersion of the next random errors.Also, outliers could help us to get an approach of model modification and optimization.