International Journal of Quality, Statistics, and ReliabilityVolume 2011 (2011), Article ID 357814, 9 pagesdoi:10.1155/2011/357814

Research Article

Bayes Estimation of Two-Phase Linear Regression Model

Mayuri Pandya,¹ Krishnam Bhatt,¹ and Paresh Andharia²

¹Department of Statistics, Bhavnagar University, University Campus, Near Gymkhana, Bhavnagar 364002, India²Department of Mathematics, Bhavnagar University, University Campus, Near Gymkhana, Bhavnagar 364002, India

Received 28 December 2010; Accepted 9 May 2011

Academic Editor: Kwai Sang Chin

Copyright © 2011 Mayuri Pandya et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Let the regression model be , where are i. i. d. N () random errors with variance but later it was found that there was a change in the system at some point of time and it is reflected in the sequence after by change in slope, regression parameter . The problem of study is when and where this change has started occurring. This is called change point inference problem. The estimators of , are derived under asymmetric loss functions, namely, Linex loss & General Entropy loss functions. The effects of correct and wrong prior information on the Bayes estimates are studied.

1. Introduction

Regression analysis is an important statistical technique to analyze data in social, medical, and engineering sciences. Quite often, in practice, the regression coefficients are assumed constant. In many real-life problems, however, theoretical or empirical deliberations suggest models with occasionally changing one or more of its parameters. The main parameter of interest in such regression analyses is the shift point parameter, which indexes when or where the unknown change occurred.

A variety of problems, such as switching straight lines [1], shifts of level or slope in linear time series models [2], detection of ovulation time in women [3], and many others, have been studied during the last two decades. Holbert [4], while reviewing Bayesian developments in structural change from 1968 onward, gives a variety of interesting examples from economics and biology. The monograph by Broemeling and Tsurumi [5] provides a complete study of structural change in the linear model from the Bayesian viewpoint.

Bayesian inference of the shift point parameter assumes availability of the prior distribution of the changing model parameters. Bansal and Chakravarty [6] had proposed to study the effect of an ESD prior for the changed slope parameter of the two-phase linear regression (TPLR) model on the Bayes estimates of the shift point and also on the posterior odds ratio (POR) to detect a change in the simple regression model.

In this paper, we studied a TPLR model. In Section 2, we have given a change point model TPLR. In Sections 3.1 and 3.2, we obtained posterior densities of considering unknown and of and considering known, respectively. We derive Bayes estimators of , and under symmetric loss functions in Section 4 and asymmetric loss functions in Section 5. We have studied the sensitivity of the Bayes estimators of when prior specifications deviate from the true values in Section 6. In Section 7, we have presented a numerical study to illustrate the above technique on generated observations. In this study, we have generated observations from the proposed model and have computed the Bayes estimates of and of other parameters. Section 8 concludes the research paper.

2. Two-Phase Linear Regression Model

The TPLR model is one of the many models, which exhibits structural change. Holbert [4] used a Bayesian approach, based on TPLR model, to reexamine the McGee and Kotz [7] data for stock market sales volume and reached the same conclusion that the abolition of splitups did hurt the regional exchanges.

The TPLR model is defined as where ’s are i. i. d. N random errors with variance , is a nonstochastic explanatory variable, and the regression parameters (α ₁, β ₁) ≠ (α ₂, β ₂). The shift point is such that if there is no shift, but when exactly one shift has occurred.

3. Bayes Estimation

The ML method, as well as other classical approaches are based only on the empirical information provided by the data. However, when there is some technical knowledge on the parameters of the distribution available, a Bayes procedure seems to be an attractive inferential method. The Bayes procedure is based on a posterior density, say, , which is proportional to the product of the likelihood function , with a prior joint density, say, representing uncertainty on the parameters values where denotes .

The likelihood function of β ₁, β ₂, and , given the sample information , is where

3.1. Using Gamma Prior on and Conditional Informative Priors on ₁, ₂ with Unknown

We consider the TPLR model (1) with unknown . As in Broemeling and Tsurumi [5], we suppose that the shift point is a priori uniformly distributed over the set and is independent of β ₁ and β ₂. We also suppose that some information on β ₁ and β ₂ are available that can be expressed in terms of conditional prior probability densities on β ₁ and β ₂.

We have conditional prior density on β ₁ and β ₂ given , with as We also suppose that some information on is available and that technical knowledge can be given in terms of prior mean and coefficient of variation . We suppose marginal prior distribution of to be gamma distribution with mean where is gamma function same as explained in (8).

The integral representation of is as below , Gradshteyn and Ryzhik [8, page 934]

The gamma function (Euler’s integral of the second kind)

, (Euler) Gradshteyn and Ryzhik [8, page 933], is defined as

If the prior information is given in terms of prior mean and coefficient of variation , then the parameters and can be obtained by solving

Hence, joint prior pdf of and , say is where

Joint posterior density of , and say, (β 1, β 2, , ) is where , and are as given in (4).

is the marginal density of given by where is as given in (13) where denotes ,, , , and are as given in (4).

is the gamma function as explained in (8).

Marginal posterior density of change point , say is is as given in (15).

3.2. Using Conditional Informative Priors on with Known

We consider the TPLR model (1) with known . We also assume same prior consideration of β ₁, β ₂, and as explained in Section 3.1.

Hence, joint prior pdf of β ₁, β ₂, and , say β ₁, β ₂, ) is where Joint posterior density of β ₁, β ₂ and say, is where, , and are as given in (4).

Where is the marginal density of given by

And the integrals,

So using (23) and (24) results in (22), it reduces to where is as given in (21). and are as given in (23) and (24).

, and are as given in (4).

Marginal posterior density of change point , β ₁, and β ₂ is and are as given in (21), (23), (24), and (25), respectively.

, are as given in (4).

4. Bayes Estimates under SymmetricLoss Function

The Bayes estimator of a generic parameter (or function there of) α based on a squared error loss (SEL) function: where is a decision rule to estimate α, is the posterior mean. As a consequence, the SEL function relative to an integer parameter,

Hence, the Bayesian estimate of an integer-valued parameter under the SEL function is no longer the posterior mean and can be obtained by numerically minimizing the corresponding posterior loss. Generally, such a Bayesian estimate is equal to the nearest integer value to the posterior mean. So, we consider the nearest value to the posterior mean as Bayes Estimate.

The Bayes estimator of under SEL is where and are as given in (15) and (25).

Other Bayes estimators of α based on the loss functions is the posterior median and posterior mode, respectively.

5. Asymmetric Loss Function

The Loss function provides a measure of the financial consequences arising from a wrong decision rule d to estimate an unknown quantity (a generic parameter or function thereof) α. The choice of the appropriate loss function depends on financial considerations only and is independent from the estimation procedure used. The use of symmetric loss function was found to be generally inappropriate, since for example, an overestimation of the reliability function is usually much more serious than an underestimation.

A useful asymmetric loss, known as the Linex loss function was introduced by Varian [9]. Under the assumption that the minimal loss occurs at α, the Linex loss function can be expressed as

The sign of the shape parameter reflects the direction of the asymmetry, if overestimation is more serious than underestimation, and vice versa, and the magnitude of reflects the degree of asymmetry.

The posterior expectation of the Linex loss function is where denotes the expectation of with respect to the posterior density . The Bayes estimate is the value of that minimizes provided that exists and is finite.

5.1. Assuming Unknown

Minimizing the posterior expectation of the Linex loss function . Where denotes the expectation of with respect to the posterior density given in (17), we get the Bayes estimate of by means of the nearest integer value to (37), say , as under. We get the Bayes estimators of using Linex loss function, respectively, as

Another loss function, called general entropy (GE) loss function, proposed by Calabria and Pulcini [10], is given by The Bayes estimate is the value of that minimizes : provided that exists and is finite.

Combining the General Entropy Loss with the posterior density (17), we get the estimate by means of the nearest integer value to (40), say , as below. We get the Bayes estimates of using General Entropy loss function as where is as given in (15).

5.2. Assuming Known

Combining the Linex loss with posterior density (26), we get the Bayes estimate of by means of the nearest integer value to (41), say as below.

Combining the Linex loss with the posterior distributions (27) and (28), respectively, we get the Bayes estimators of β ₁ and β ₂ using Linex loss function as where , , and are same as given in (21), (22), (23), and (24), respectively. are as given in (4).

Minimizing expectation and then taking expectation with respect to posterior density , we get the estimate by means of the nearest integer value to (44) say , as below. We get the Bayes estimates of using General Entropy loss function as where is same as given in (25).

Note 1. The confluent hypergeometric function of the first kind [11] is a degenerate form of the hypergeometric function which arises as a solution to the confluent hypergeometric differential equation. It is also known as Kummer's function of the first kind and denoted by , defined as follows: With Pochhammer coefficients for and [12, page 755], also has an integral representation The symbols Γ and denoting the usual functions gamma and beta, respectively.When and are both integer, some special results are obtained. If , and either or , the series yields a polynomial with a finite number of terms. If integer , the function is undefined.

Note 2. ] is called a generalized hypergeometric series and defined as (Gradshteyn and Ryzhik [8, page 1045]).] has series expansion In many special cases hypergeometric is automatically converted to other functions.For , hypergeometric [ list, list, ] has a branch cut discontinuity in the complex plane running from 1 to .Hypergeometric (Regularized) is finite for all finite values of its argument so long as .

Note 3. is the beta function Euler’s integral of the first kind defined as Gradshteyn and Ryzhik [8, pages 948, 950], The gamma function is as explained in (8). Minimizing expected loss function and using posterior distributions (27) and (28), we get the Bayes estimates of using General Entropy loss function, respectively, as where Hypergeometric and hypergeometric are hypergeometric functions same as explained in Notes 1 and 2, respectively. , and are as explained in (23) and (50), respectively. is as given in (21). and are gamma functions same as explained in (8). , and are as given in (4) where Hypergeometric and hypergeometric are hypergeometric functions same as explained in Notes 1 and 2, respectively. , and are as explained in (23) and (50), respectively. is as given in (21). is as given in (21). and are gamma functions same as explained in (8). , and are as given in (4).

Remark 1. Putting in (40) and (44), we get the Bayes estimators of , posterior means under the squared error loss as given in (31) and (32). Note that for , the GE loss function reduces to the squared error loss function.

6. Numerical Study

6.1. Illustration

Let us consider the two-phase regression model where 's are i.i.d. random errors. We take the first 15 values of and from Table 4.1 of Zellner [13] to generate 15 sample values . The generated sample values are given in Table 1. The , and themselves were random observations. and were from standard normal distribution and precision was from gamma distribution with =1 and coefficient of variation , respectively, in .

Table 1: Zellener data for two-phase linear regression model.

T

T

T

1
1.7
4.6
6
4.1
11.27
11
5.1
16.90

2
2.2
6.4
7
4.2
13.90
12
5.2
17.32

3
2.7
6.9
8
4.5
14.18
13
6.0
17.99

4
3.3
8.0
9
4.7
15.45
14
6.2
18.92

5
3.9
13.75
10
4.8
16.40
15
6.5
19.36

We have calculated posterior mean, posterior median and posterior mode of m. The results are shown in Table 2.

Table 2: The values of Bayes estimates of shift point, regression coefficients and precision.

Prior
Bayes estimates of

Posterior Mean
Posterior Median
Posterior Mode

Unknown
4
5
4

Known
4
5
4

We also compute the Bayes estimators mE of using (40) for unknown and (44) for known and using (37) for unknown and (41) for known for data given in Table 1. The results are shown in Table 3.

Table 3: The Bayes estimates of change point using asymmetric loss functions.

Prior
Shape parameter
Bayes estimates of change point

q₁
q₃

Unknown
1.5
1.5
2
2

1.2
1.2
3
3

0.9
0.9
4
4

0.5
0.5
4
4

0.2
0.2
4
4

0.1
0.1
4
4

−1.0
−1.0
6
4

−2.0
−2.0
7
6

q₁
q₃

Known
1.5
1.5
2
2

1.2
1.2
3
3

0.9
0.9
4
4

0.5
0.5
4
4

0.2
0.2
4
4

0.1
0.1
4
4

−1.0
−1.0
5
4

−2.0
−2.0
6
5

Table 3 shows that for small values of , 0.5, 0.2, 0.1 Linex loss function is almost symmetric and nearly quadratic and the values of the Bayes estimate under such a loss is not far from the posterior mean. Table 3 also shows that for , 1.2, Bayes estimates are less than actual value of .

It can be seen from Table 3 that positive sign of shape parameter of loss functions reflects overestimation is more serious than underestimation. Thus, problem of overestimation can be solved by taking the value of shape parameter of Linex and General Entropy loss functions positive and high.

For , −2, Bayes estimates are quite large than actual value . It can be seen from Table 3 that the negative sign of shape parameter of loss functions reflects underestimation is more serious than overestimation. Thus, problem of underestimation can be solved by taking the value of shape parameters of Linex and General Entropy loss functions negative.

We get Bayes estimators , and of and using (42), (43), (49), and (51), respectively, for the data given in Table 1 and for different value of shape parameter and . The results are shown in Table 4.

Table 4: The Bayes estimates of Bayes estimates of regression coefficients using asymmetric loss functions.

Shape parameter
Bayes estimates of regression co-efficients

β ₁
β ₂

q₁

Known
1.5
1.3
2.2

1.2
1.8
2.4

0.9
2.5
3.2

0.5
3.0
3.5

0.2
3.2
3.7

0.1
3.4
3.9

q₃
** 1E
** 2E

Known
1.5
1.0
2.6

1.2
1.5
2.8

0.9
2.9
3.3

0.5
3.1
3.4

0.2
3.3
3.6

0.1
3.5
3.8

Tables 3 and 4 show that the values of the shape parameters of Linex and General Entropy loss functions increase, the values of Bayes estimates decrease.

7. Sensitivity of Bayes Estimates

In this section, we study the sensitivity of the Bayes estimator, obtained in Sections 4 and 5 with respect to change in the prior of parameters. The mean μ of gamma prior on has been used as prior information in computing the parameters of the prior. We have computed posterior mean using (31) and using (32) for the data given in Table 1 considering different sets of values of (μ). Following Calabria and Pulcini [10], we also assume the prior information to be correct if the true value of is closed to prior mean μ and is assumed to be wrong if is far from μ. We observed that the posterior mode appears to be robust with respect to the correct choice of the prior density of and also with a wrong choice of the prior density of . This can be seen from Table 5.

Table 5: The values of Posterior mean of change point for different values of prior means.

Values of prior mean μ
Bayes estimates of change point

Posterior mean
Posterior median
Posterior mode

0.5
5
6
4

1
4
5
4

1.5
6
6
4

Table 5 shows that when prior mean μ = 1 = actual value of , it means correct choice of prior of , The values of Bayes estimator posterior mode is 4. It gives correct estimation of change point. Now, when μ = 0.5 and 1.5 (far from true value of = 1), it means wrong choice of prior of . The value of Bayes estimator of posterior mode remains 4. But, posterior mean and posterior median do not remain same for wrong choice of prior of . Thus, posterior mode is not sensitive with wrong choice of prior density of . While, posterior mean and posterior median are sensitive with wrong choice of prior density of .

8. Simulation Study

In Sections 4 and 5 we have obtained Bayes estimates of on the basis of the generated data given in Table 1 for given values of parameters. To justify the results, we have generated 10,000 different random samples with , 3.3, 3.4, , 3.6, 3.7 and obtained the frequency distributions of posterior mean, median of with the correct prior consideration. The result is shown in Tables 2 and 3. The value of shape parameter of the general entropy loss and Linex loss used in simulation study for shift point is taken as 0.1.

We have also simulated several standard normal samples. For each β ₁, β ₂, and and , 1000 pseudorandom samples from two-phase linear regression model discussed in Section 2 have been simulated and Bayes estimators of change point has been computed using and for different prior mean μ.

Table 6 leads to conclusion that performance of , posterior mode’s and posterior median’s has better performance than that of posterior mean of change point explained in Sections 4 and 5. 46% values of posterior mean are closed to actual value of change point with correct choice of prior. 62% values of posterior median are closed to actual value of change point with correct choice of prior. 70% values of posterior mode are close to correct values of change point with correct prior considerations. 65% values of are closed to actual values of . 66% values of are closed to actual values of .

Table 6: Frequency distributions of the Bayes estimates of the change point.

Bayes estimate
% Frequency for

01-02
03–05
06–15

Posterior mean
35
46
19

Posterior median
23
62
15

Posterior mode
12
70
18

20
65
15

22
66
12

9. Conclusions

In this study, we are discussing the Bayes estimator of shift point, the integer parameter, posterior mean is less appealing. Posterior median and posterior mode appear as better estimators as they would be always integer. Our numerical study showed that the Bayes estimators posterior mode of is robust with respect to the correct choice of the prior specifications on and wrong choice of the prior specifications on , posterior median and posterior mode are sensitive in case prior specifications on deviate simultaneously from the true values. Here, we discussed regression model with one change point, in practice it may have two or more change point models. One can apply these models to econometric data such as poverty and irrigation.

Acknowledgments

The authors would like to thank the editor and the referee for their valuable suggestions which improved the earlier version of the paper.

References

P. E. Ferreira, “A bayesian analysis of switching linear model: a known number of regimes,” Journal of the American Statistical Association, vol. 70, pp. 370–374, 1975.

A. F. M. Smith, “Change point problems: approaches and applications,” in Bayesian Statistics, J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, Eds., pp. 83–98, University Press, Valencia, España, 1980.

R. L. Carter and B. J. N. Blight, “A bayesian change point problem with an application to the prediction and detection of ovulation in women,” in Practical Bayesian Statistics, G. K. Kanji, Ed., pp. 229–230, Longman, London, UK, 1983.

D. Holbert, “A Bayesian analysis of a switching linear model: a known number of regimes,” Journal of the American Statistical Association, vol. 70, pp. 370–374, 1982.

L. D. Broemeling and H. Tsurumi, Econometrics and Structural Change, Marcel Dekker, New York, NY, USA, 1987.

A. K. Bansal and S. Chakravarty, “Bayes estimation and detection of a change in prior distribution of the regression parameter,” in Bayesian Analysis in Statistics and Econometrics, Donald A. Berry and M. Kathryn, Eds., pp. 257–266, Wiley-Interscience, New York, NY, USA, 1996.

V. E. McGee and S. Kotz, “Piecewise regression,” Journal of the American Statistical Association, vol. 65, pp. 1109–1124, 1970.

I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series and Products, Academic Press, New York, NY, USA, 1965.

H. R. Varian, “A Bayesian approach to real estate assessment,” in Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage, S. E. Fienberg and A. Zellner, Eds., pp. 195–208, North-Holland, Amsterdam, The Netherlands, 1975.

R. Calabria and G. Pulcini, “An engineering approach to Bayes estimation for the Wseibull distribution,” Microelectronics Reliability, vol. 34, no. 5, pp. 789–802, 1994.

E. E. Kummer, “Über die hypergeometrische reihe F(a,b;x),” Journal für die reine und angewandte Mathematik, vol. 15, pp. 39–83, 1836.

G. Arfken, “Confluent hypergeometric functions,” in Mathematical Methods for Physicists, chapter 13.6, pp. 753–758, Academic Press, Orlando, Fla, USA, 3rd edition, 1985.

A. Zellner, An Introduction to Bayesian Inference in Econometrics, Wiley, New York, NY, USA, 1971.