The Effect of a Single Point on Correlation and Slope

By augmenting a bivariate data set with one point, the correlation coefficient and/or the slope of the regression line can be changed to any prescribed values. For the target value of the correlation coefficient or the slope, the coordinates of the new point are found as a function of certain statistics of the original data. The location of this new point with respect to the original data is investigated.


INTRODUCTION.
The correlation coefficient and slope of a least squares regression line are known to be sensitive to a few outliers in the data.For this reason, these estimators are called nonrobust or nonresistant.It is not unexpected that changing one data point or introducing a new data point will greatly perturb the regression line and various statistics associated with it.Actually, we can judiciously introduce a new data point and create a line of our choosing.The effect of adding one new data point is the subject of this paper.
In situations where we have a choice, this addition of a point is a way to lie with statistics.For example, a fit of payroll data, which we fear could reveal our bias, might be changed in this manner by appropriately hiring one more employee.Or, more benignly, in composing an example DAVID L. FARNSWORTH for a classroom exercise or a made-up case study, we may want to produce certain outcomes.These might be obtained with the introduction of just one additional point.
The idea of adding one special point in order to force a regression line through the origin was studied by Casella  [1].The location of the point was shown to be related to various statistics for the line.
The robustness of an estimator is in part gauged by the influence function.The influence curve or function for a given estimator is defined pointwise as a limit as a vanishing weight is placed on the data point.These curves are derivatives and measure infinitesimal asymptotic (large sample) influence on the given estimator.The earliest work on this concept was by Hampel [2] and [3].Also, see Belsley et al. [4], Cook and Weisberg [5], and Huber [6].
Finite sample influence curves are difference quotients, not limits.For sample size n, the numerator consists of the difference between a statistic of interest with a given point included in its formulation and the same statistic without that point.The denominator is 1/n.This sample influence curve is used to measure the influence of a single data point within a sample.The deletion of an influential data point would lead to large values of the influence curve.The diagnostic tests for influence presented in Section 5 are standardized versions of the sample influence curve.
In Sections 2, 3 and 4 for any target value of the slope or the correlation coefficient, the coordinates of one additional point are found as a function of certain statistics of the original data.
The locations of these points are presented both analytically and graphically.They are realizations of curves of constant influence where the original n-1 data are parameters.The trouble is that we may be caught tampering with the data in this way if conventional diagnostics flag the new point.In Section 5, four of the customary measures of leverage and influence are briefly discussed.In Section 6, an illustrative numerical example is given.
We are given the original data {(x{,y)" 1,2,... ,n-1}.For convenience take the center 2 _y/(n-1) 1, that is, x and (5,) (0,0) and variances s x/(n-1) 1 and s 2 and 2 are the full sample size since that choice yi axe in standard units.The denominators of s s leads to considerable simplification of subsequent formulas.Thus, the regression line is y bx, and the correlation coefficient is r b.The additional point (x,, y,,) is expressed in standard deviation units of the original n-1 data as (u, v) (x,/s, y,,/s) (x,, y,).
The condition that the correlation coefficient of the augmented data is the prescribed value p is =. ( /(-+ =)(.+ ,) The dependence of p upon the original data is only through the values of n and r.Equation (2.1) (2.2) For each p, symmetries of the solution curves are about the origin and the lines v u and v -tt since equation (2.1) is invariant under replacement of (u, v) with (-u,-v) or (4-v, +u).
Vertical asymptotes appear at (1 p2)tt2 np 0, that is, tt 4-V/-ffp/x/'l f 4-k.By symmetry, horizontal asymptotes are v 4-k.Note that k is a monotonically increasing function of rt and p2.For p 0, the asymptotes are the axes, and equation (2.1) becomes uv -rtr.
The discriminant of equation (2.2)is 4rip'In + u2][(1-p2)u + n(r -p2)] which is nonnegative for tt > n(p -r2)/(1-p2).Thus, for p2 < r there is no restriction upon it.For p > r the nonnegativity of the discriminant does restrict tt from a strip about the v-axis.If r 0, the p r solution curve becomes the two axes.If r 1, there is just the p < r type of solution curves.
Representative solution curves are shown in Figure 1 for r b 0.5 and n 12.Because of the symmetries, only the right half-plane is displayed.The regression line v 0.5u and the asymptotes u 2 and v 2 for thep r 0.5 curve are shown.Thep -0.6 and the p 4-0.6 curves have asymptotes u 4-2.59 and v 4-2.59.Only the bulge about v u for the p 0.9 curve is visible.Its shape is similar to that of the curve for p 0.6.The p 0.9 curve doubles back, changing its concavity and approaching the vertical asymptote u 7.15 from the left and the horizontal asymptote v 7.15 from below.All of the p 0 curve is outside the square {-2 < u < 2,-2 < v < 2}, indicating how unusual the new point (u, v) would have to be to convert from r 0.5 to p 0. Of particular interest are the two branches of the p r 0.5 curve.The correlation coefficient is unchanged even by points which are very distant along this curve.2 displays selected solution curves for b r 0.5 and n 12.Because of symmetry, just the right half-plane is shown.Of course, the d 0 curve in Figure 2 is the same as the p 0 curve in Figure 1.From Figures 1 and 2 it is apparent that the slope can be drastically chaged while the corre- lation coefficient is numerically the same.Setting p r b, equations ( 2 If n is large, then u or v must be very large for d to be much different from b. Within the constraints p E (--,1,1) and sign p sign d, the two values p and d can be chosen arbitrarily by specifying the two numbers u and v.However, the issue of remoteness of (u, v) from (, ) (0, 0) and from V bx is important and is addressed below.
5. DETECTING THE NEW POINT.Among the available diagnostic tests for gauging whether a data point (xi, W) is an interloper, four re the most common.Each measures a different feature of data.See Neter et al. [7], Atldnson [8], or Cook and Weisberg [5].The notation and critical values are not uniformly agreed upon in the literature.Those suggested in Chapter 11 of Neter et M. [7] are used below.
First, the point (u, v) could be an x-outlier, that is, it could be far in an horizontal direction from the mean of MI n data.Such points influence slope the most for a given chrome in the coordinate and are said to have high leverage.The usual measure of leverage for (xi,w) is the diagonal element hii of the hat matrix.See Hoaglin and Welsch [9].Each hii depends upon all the x-coordinates but does not depend upon the y-coordinates of the data.If 4In < hii <_ 1, then (xi, W) will be said to have high leverage.If 1/n < h < 4/n, then (x,, y) has low leverage.
The next three diagnostic measures arise from the deletion technique.The impact of a data point is often detected by using this technique: The data point is removed; the statistic of interest is computed; and a standardized difference between that statistic and the corresponding statistic utilizing all the data is analysed as the diagnostic measure.The unstandardized differences divided by l/n, are the sample influence curves discussed in the Introduction.
Second, the point (u, v) could be outlying with respect to the !/-value.A diagnostic statistic for measuring whether (xi, yi) is a V-outlying point is the externally studentized residual dT, which is a standardized vertical distance at xi between the point (xi,yi) and the regression line created without (Xi,l/i).Eah d is t-distributed with df n-3.Using the prediction limits for a new point (xi, yi) based on the other n-1 data's regression line as the criteria for designating a point as outlying is equivalent to using d, as shown in Neter et al. [7], page 399.
Third, (u, v) could inordinately change predicted values.The statistic (DFFITS)i is a stan- dardized difference between the predicted values at xi one computed with, and one without, the data point (x, y).The point (x, yi) is said to be influential in this sense if I(DFFITS)I > for large n and if I(DFFITS)I > 1 for small to moderate n.This test is based on the usual confi- dence bands for regression lines.
Fourth, (u, v) could unduly impact the slope.The statistic (DFBETA) is a standardized difference between the slope of the two regression lines --one with (x, y) and one without (z, yi).
If I(DFBETA)il > 1 for small to moderate n and if I(DFBETA)il > 2/V/' for large n, then (xi, yi)   is called influential in this sense.
Point C is the local minimum point of the d 0.7 curve.
Each of the four diagnostic statistics discussed in Section 5 is presented in Table 1 for each of these four points representing (x,, y,).The symbol * next to a numerical value means that the point would be deemed unusual, that is, considered of high leverage or influence, by the criteria given in Section 5.The level of significance is 0.05 for the d, column.Points on the d 0.7 curve with x, u _> 3.59 will be flagged with (DFBETA), > 1.
Table 1 shows that introducing point C will give us the desired change of slope but will not be detected with these four diagnostic statistics.Other points on the d 0.7 curve may or may not be detected with various test statistics depending upon their locations.
The original eleven data, the regression line based on the original data, and a branch of the d 0.7 curve with the four points labeled A, B, C, and D are shown in Figure 3.For purposes of scaling of d, the hyperbolic 95% prediction band is displayed as well.

IIiFigure 1 :
Figure 1: Curves along which (u, v) may be placed to change the value of the correlation coefficient to selected values p for r b 0.5 and n 12.

3 .
PRESCRIBING THESLOPE.The condition that the slope of the least squares regression line based on the n points is the prescribed value d is nb + uv =d (3.1) n+u or .(d-b)/+ d.For each d the solution curve is symmetric with respect to the origin.Asymptotes are u 0 and v du for d b.Maxima and minima occur at (-I-V/n(d b)/d,4-2dV/n(d b)/d) for (d-b)/d > O. Intersections with the u-axis occur at (-l-v/n(bd)/d, 0) for (d-b)/d < O. Placing d b in equation (3.1) implies u 0 and v bu.

Figure
Figure2displays selected solution curves for b r 0.5 and n 12.Because of symmetry, just the right half-plane is shown.Of course, the d 0 curve in Figure2is the same as the p 0

Figure 2 :
Figure 2: Curves along which (u, v) may be placed to change the value of the slope to selected values d for r b 0.5 and n 12.

4 .
PRESCRIBING BOTH THE CORRILATION COEFFICIENT AND THE SLOPE.