Bayes Estimation of Two-Phase Linear Regression Model

Let the regression model be Yi = β1Xi + εi, where εi are i. i. d. N (0, σ2) random errors with variance σ2 > 0 but later it was found that there was a change in the system at some point of time m and it is reflected in the sequence after Xm by change in slope, regression parameter β2. The problem of study is when and where this change has started occurring. This is called change point inference problem. The estimators of m, β1,β2 are derived under asymmetric loss functions, namely, Linex loss & General Entropy loss functions. The effects of correct and wrong prior information on the Bayes estimates are studied.


Introduction
Regression analysis is an important statistical technique to analyze data in social, medical, and engineering sciences. Quite often, in practice, the regression coefficients are assumed constant. In many real-life problems, however, theoretical or empirical deliberations suggest models with occasionally changing one or more of its parameters. The main parameter of interest in such regression analyses is the shift point parameter, which indexes when or where the unknown change occurred.
A variety of problems, such as switching straight lines [1], shifts of level or slope in linear time series models [2], detection of ovulation time in women [3], and many others, have been studied during the last two decades. Holbert [4], while reviewing Bayesian developments in structural change from 1968 onward, gives a variety of interesting examples from economics and biology. The monograph by Broemeling and Tsurumi [5] provides a complete study of structural change in the linear model from the Bayesian viewpoint.
Bayesian inference of the shift point parameter assumes availability of the prior distribution of the changing model parameters. Bansal and Chakravarty [6] had proposed to study the effect of an ESD prior for the changed slope parameter of the two-phase linear regression (TPLR) model on the Bayes estimates of the shift point and also on the posterior odds ratio (POR) to detect a change in the simple regression model.
In this paper, we studied a TPLR model. In Section 2, we have given a change point model TPLR. In Sections 3.1 and 3.2, we obtained posterior densities of m considering σ 2 unknown and of β 1 , β 2 and m considering σ 2 known, respectively. We derive Bayes estimators of β 1 , β 2 , and m under symmetric loss functions in Section 4 and asymmetric loss functions in Section 5. We have studied the sensitivity of the Bayes estimators of m when prior specifications deviate from the true values in Section 6. In Section 7, we have presented a numerical study to illustrate the above technique on generated observations. In this study, we have generated observations from the proposed model and have computed the Bayes estimates of m and of other parameters. Section 8 concludes the research paper.

Two-Phase Linear Regression Model
The TPLR model is one of the many models, which exhibits structural change. Holbert [4] used a Bayesian approach, based on TPLR model, to reexamine the McGee and Kotz [7] data for stock market sales volume and reached the same conclusion that the abolition of splitups did hurt the regional exchanges.
The TPLR model is defined as 2 International Journal of Quality, Statistics, and Reliability where ε t 's are i. i. d. N (0, σ 2 ) random errors with variance σ 2 > 0, x t is a nonstochastic explanatory variable, and the regression parameters (α 1 , β 1 ) / = (α 2 , β 2 ). The shift point m is such that if m = n there is no shift, but when m = 1, 2, . . . , n − 1 exactly one shift has occurred.

Bayes Estimation
The ML method, as well as other classical approaches are based only on the empirical information provided by the data. However, when there is some technical knowledge on the parameters of the distribution available, a Bayes procedure seems to be an attractive inferential method. The Bayes procedure is based on a posterior density, say, g(β 1 , β 2 , σ −2 , m | z), which is proportional to the product of the likelihood function L(β 1 , β 2 , σ −2 , m | z), with a prior joint density, say, g(β 1 , β 2 , σ −2 , m) representing uncertainty on the parameters values where Z denotes g(β 1 , The likelihood function of β 1 , β 2 , σ −2 and m, given the where 3.1. Using Gamma Prior on 1/σ 2 and Conditional Informative Priors on β 1 , β 2 with σ −2 Unknown. We consider the TPLR model (1) with unknown σ −2 . As in Broemeling and Tsurumi [5], we suppose that the shift point m is a priori uniformly distributed over the set {1, 2, . . . , n−1} and is independent of β 1 and β 2 . We also suppose that some information on β 1 and β 2 are available that can be expressed in terms of conditional prior probability densities on β 1 and β 2 .
We have conditional prior density on β 1 and β 2 given σ 2 , with N (0, σ 2 ) as We also suppose that some information on 1/σ 2 is available and that technical knowledge can be given in terms of prior mean μ and coefficient of variation ∅. We suppose marginal prior distribution of 1/σ 2 to be gamma (c, d) distribution with mean μ where Γd is gamma function same as explained in (8 The gamma function (Euler's integral of the second kind) Γ(z)[Re z > 0], (Euler) Gradshteyn and Ryzhik [8, page 933], is defined as If the prior information is given in terms of prior mean μ and coefficient of variation ∅, then the parameters c and d can be obtained by solving Hence, joint prior pdf of β 1 , β 2 , σ −2 and m, say g 1 (β 1 , β 2 , σ −2 , m) is where Joint posterior density of β 1 , β 2 , σ −2 , and m say, International Journal of Quality, Statistics, and Reliability 3 where S m3 , S m4 , S m1 , S m2 and A are as given in (4). h 1 (z) is the marginal density of z given by where k 2 is as given in (13) where x denotes S m1 + 1 · S n1 − S m1 + 1 , S m3 , S m4 , S m1 , S m2 , S n1 , and A are as given in (4).
is the gamma function as explained in (8).
Marginal posterior density of change point m, say g(m | z) is T 1 (m) is as given in (15).
Where h 2 (z) is the marginal density of z given by And the integrals, So using (23) and (24) results in (22), it reduces to where k 4 is as given in (21). G 1m and G 2m are as given in (23) and (24). S m3 , S m4 , S m1 , and S m2 are as given in (4).

Bayes Estimates under Symmetric Loss Function
The Bayes estimator of a generic parameter (or function there of) α based on a squared error loss (SEL) function: where d is a decision rule to estimate α, is the posterior mean. As a consequence, the SEL function relative to an integer parameter, Hence, the Bayesian estimate of an integer-valued parameter under the SEL function L 1 (m, v) is no longer the posterior mean and can be obtained by numerically minimizing the corresponding posterior loss. Generally, such a Bayesian estimate is equal to the nearest integer value to the posterior mean. So, we consider the nearest value to the posterior mean as Bayes Estimate.
The Bayes estimator of m under SEL is where T 1 (m) and T 2 (m) are as given in (15) and (25). Other Bayes estimators of α based on the loss functions is the posterior median and posterior mode, respectively.

Asymmetric Loss Function
The Loss function L(α, d) provides a measure of the financial consequences arising from a wrong decision rule d to estimate an unknown quantity (a generic parameter or function thereof) α. The choice of the appropriate loss function depends on financial considerations only and is independent from the estimation procedure used. The use of symmetric loss function was found to be generally inappropriate, since for example, an overestimation of the reliability function is usually much more serious than an underestimation. A useful asymmetric loss, known as the Linex loss function was introduced by Varian [9]. Under the assumption that the minimal loss occurs at α, the Linex loss function can be expressed as The sign of the shape parameter q 1 reflects the direction of the asymmetry, q 1 > 0 if overestimation is more serious than underestimation, and vice versa, and the magnitude of q 1 reflects the degree of asymmetry.
The posterior expectation of the Linex loss function is where provided that E α {exp (−q 1 α)} exists and is finite.
Another loss function, called general entropy (GE) loss function, proposed by Calabria and Pulcini [10], is given by The Bayes estimate α * E is the value of d that minimizes E α [L 5 (α, d)]: provided that E α (α −q3 ) exists and is finite.
Combining the General Entropy Loss with the posterior density (17), we get the estimate m by means of the nearest integer value to (40), say m * E , as below. We get the Bayes estimates m * E of m using General Entropy loss function as where T 1 (m) is as given in (15).

Minimizing expectation E[L 5 (m, d)]
and then taking expectation with respect to posterior density g 2 (m | z), we get the estimate m by means of the nearest integer value to (44) say m * E , as below. We get the Bayes estimates m * E of m using General Entropy loss function as where T 2 (m) is same as given in (25).

Note 1.
The confluent hypergeometric function of the first kind 1 F 1 (a, b; x) [11] is a degenerate form of the hypergeometric function 2 F 1 (a, b, c; x) which arises as a solution to the confluent hypergeometric differential equation. It is also known as Kummer's function of the first kind and denoted by 1 F 1 , defined as follows: With Pochhammer coefficients (a, m) = Γ(a + m)/Γ(a) for m ≥ 1 and (a, 0) = 1 [12, page 755], also has an integral representation The symbols Γ and B denoting the usual functions gamma and beta, respectively. When a and b are both integer, some special results are obtained. If a < 0, and either b > 0 or b < a, the series yields a polynomial with a finite number of terms. If integer b ≤ 0, the function is undefined.
In many special cases hypergeometric p F q is automatically converted to other functions. For p = q + 1, hypergeometric p F q [a list, b list, z] has a branch cut discontinuity in the complex z plane running from 1 to ∞.
Hypergeometric p F q (Regularized) is finite for all finite values of its argument so long as p ≤ q.

Note 3. β(x, y) is the beta function Euler's integral of the first kind defined as Gradshteyn and Ryzhik [8, pages 948, 950],
The gamma function is as explained in (8).

Illustration. Let us consider the two-phase regression model
where ε t 's are i.i.d. N(0, 1) random errors. We take the first 15 values of x t and ε t from Table 4.1 of Zellner [13] to generate 15 sample values (x t , y t ) t = 1, 2, . . . , 15. The generated sample values are given in Table 1. The β 1 , β 2 , and σ 2 themselves were random observations. β 1 and β 2 were from standard normal distribution and precision 1/σ 2 was from gamma distribution with μ =1 and coefficient of variation ∅ = 1.4, respectively, in c = 0.5, d = 0.5. We have calculated posterior mean, posterior median and posterior mode of m. The results are shown in Table 2.
We also compute the Bayes estimators m E of m using (40) for unknown σ 2 and (44) for known σ 2 and m L using (37) for unknown σ 2 and (41) for known σ 2 for data given in Table 1. The results are shown in Table 3. Table 3 shows that for small values of |q|, q = 0.9, 0.5, 0.2, 0.1 Linex loss function is almost symmetric and nearly quadratic and the values of the Bayes estimate under such a loss is not far from the posterior mean. Table 3 also shows that for q 1 = q 3 = 1.5, 1.2, Bayes estimates are less than actual value of m = 4.
It can be seen from Table 3 that positive sign of shape parameter of loss functions reflects overestimation is more serious than underestimation. Thus, problem of overestimation can be solved by taking the value of shape parameter of Linex and General Entropy loss functions positive and high.
For q 1 = q 3 = −1, −2, Bayes estimates are quite large than actual value m = 4. It can be seen from Table 3 that the negative sign of shape parameter of loss functions reflects underestimation is more serious than overestimation. Thus, problem of underestimation can be solved by taking the value of shape parameters of Linex and General Entropy loss functions negative.
We get Bayes estimators β * * 1L , β * * 2L , β * * 1E , and β * * 2E of β 1 and β 2 using (42), (43), (49), and (51), respectively, for the data given in Table 1 and for different value of shape parameter q 1 and q 3 . The results are shown in Table 4. Tables 3 and 4 show that the values of the shape parameters of Linex and General Entropy loss functions increase, the values of Bayes estimates decrease.

Sensitivity of Bayes Estimates
In this section, we study the sensitivity of the Bayes estimator, obtained in Sections 4 and 5 with respect to change in the prior of parameters. The mean μ of gamma prior on σ −2 has been used as prior information in computing the parameters c, d of the prior. We have computed posterior mean m * using (31) and m * * using (32) for the data given in Table 1 considering different sets of values of (μ). Following Calabria and Pulcini [10], we also assume the prior information to be correct if the true value of σ −2 is closed to prior mean μ and is assumed to be wrong if σ −2 is far from μ. We observed that the posterior mode m * appears to be robust with respect to the correct choice of the prior density of σ −2 and also with a wrong choice of the prior density of σ −2 . This can be seen from Table 5. Table 5

Simulation Study
In Sections 4 and 5, we have obtained Bayes estimates of m on the basis of the generated data given in Table 1 for given values of parameters. To justify the results, we have generated 10,000 different random samples with m = 4, n = 15, β 1 = 3.2, 3.3, 3.4, β 2 = 3.5, 3.6, 3.7 and obtained the frequency distributions of posterior mean, median of m, m * L , m * E with the correct prior consideration. The result is shown in Tables 2 and 3. The value of shape parameter of the general entropy loss and Linex loss used in simulation study for shift point is taken as 0.1.
We have also simulated several standard normal samples. For each β 1 , β 2 , and m and n, 1000 pseudorandom samples from two-phase linear regression model discussed in Section 2 have been simulated and Bayes estimators of change point m has been computed using q 3 = 0.9 and for different prior mean μ. Table 6

Conclusions
In this study, we are discussing the Bayes estimator of shift point, the integer parameter, posterior mean is less appealing. Posterior median and posterior mode appear as better estimators as they would be always integer. Our numerical study showed that the Bayes estimators posterior   mode of m is robust with respect to the correct choice of the prior specifications on σ −2 and wrong choice of the prior specifications on σ −2 , posterior median and posterior mode are sensitive in case prior specifications on 1/σ 2 deviate International Journal of Quality, Statistics, and Reliability 9 simultaneously from the true values. Here, we discussed regression model with one change point, in practice it may have two or more change point models. One can apply these models to econometric data such as poverty and irrigation.