A Measure of Uncertainty regarding the Interval Constraint of Normal Mean Elicited by Two Stages of a Prior Hierarchy

This paper considers a hierarchical screened Gaussian model (HSGM) for Bayesian inference of normal models when an interval constraint in the mean parameter space needs to be incorporated in the modeling but when such a restriction is uncertain. An objective measure of the uncertainty, regarding the interval constraint, accounted for by using the HSGM is proposed for the Bayesian inference. For this purpose, we drive a maximum entropy prior of the normal mean, eliciting the uncertainty regarding the interval constraint, and then obtain the uncertainty measure by considering the relationship between the maximum entropy prior and the marginal prior of the normal mean in HSGM. Bayesian estimation procedure of HSGM is developed and two numerical illustrations pertaining to the properties of the uncertainty measure are provided.

Bayesian analysis of the model (1) begins with the specification of prior distributions for unknown parameters and the noise variance 2 . Specifically, we assign a normal prior distribution for and an inverse gamma (IG) prior for 2 , that is, ∼ ( , 2 ) and 2 ∼ IG( , ), which are commonly used in a normal model as conjugate priors, where all the hyperparameters and 2 , , and are assumed to be known in the first place. On the other hand, when we are completely sure about a functional constraint of a priori; a suitable restriction on the parameter space Θ = R, such as using a truncated normal distribution, is expected. However, it is often the case that the actual observations of (1) may violate the constraint on account of the measurement error or due to some other reasons. Further, the data may provide strong evidence that the constraint is inappropriate and therefore may appear to contradict the theory associated with the constraint. In this respect, it is expected that the uncertainty about the constraint is taken into account in the estimation procedure. O'Hagan and Leonard [1] indeed proposed two stages of a prior hierarchy based on the truncated prior distribution, which reflects the uncertainty about the parameter constraint. Liseo and Loperfido [2], Kim [3], and Kim and Choi [4] among others considered the Bayesian estimation of normal models with uncertain interval constraints using the idea of two stages of a prior hierarchy. In particular, Kim [3] obtained the marginal prior of as the normal selection distribution (e.g., [5]) and thus exploited the class of weighted normal distribution by Kim [6] for reflecting the uncertain prior belief on .
Although the two-stage prior is applied by many investigators, there is no objective method to measure (or control) the degree of uncertainty regarding the interval constraint of accounted for by using the two stages of a prior hierarchy. This is a major hindrance factor in developing the idea of the two stages of a prior hierarchy which is advocated by O'Hagan and Leonard [1]. Thus, such practical problem motivates us to develop a formal measure of uncertainty about the constraint in order to show how the uncertainty of the prior information regarding interval constrained parameter space Θ of can be reflected by utilizing the two stages of a prior hierarchy. This topic is tackled in this paper.

2
The Scientific World Journal To propose the uncertainty measure, we consider the Bayesian inference of the normal mean in (1), but subject to an uncertain interval constraint. Because the maximum entropy prior by Jaynes [7,8] is useful for describing (or measuring) the relative levels of uncertainty about the distribution of the prior parameter, our investigation focuses on the theoretical relationship between the two stages of a prior hierarchy of by O'Hagan and Leonard [1] and the maximum entropy prior subject to an uncertain interval constraint. The remainder of this paper is organized as follows. In Section 2, we briefly discuss the two-stage prior of , which will be used for the Bayesian analysis of subject to uncertainty regarding the interval constraint. Accordingly, influenced by the seminal work of O'Hagan and Leonard [1], we provide a normal model based on the two-stage prior distribution of , referred to as hierarchical screened Gaussian model (HSGM). In Section 3, we explore the theoretical properties of the two-stage prior of by using Boltzmann's maximum entropy theorem [9,10]. Based on the properties, we propose an objective measure of uncertainty regarding the interval constraint of that is accounted for by the two-stage prior. In Section 4, we explore Bayesian estimation procedure by analytically deriving the posterior distribution of the unknown parameters under HSGM, and we discuss the properties of the proposed measure of uncertainty that can be explained in the context of HSGM. Finally, the concluding remarks along with a discussion are made in Section 5.

Hierarchical Screened Gaussian Model
Let us assume the normal model (1) and consider the two stages of a prior hierarchy in the following way: = + , ∼ (0, 2 ) , = 1, . . . , , where ( ≤ ≤ ) ( , 2 1 ) denotes a doubly truncated ( , 2 1 ) distribution with the lower truncation point and upper truncation point . In practice, there are certain cases in which we have a priori information that is highly likely to have an interval constraint, and thus the value of needs to be located with uncertainty in a restricted space Θ, In order to elicit the prior distribution on the uncertain interval constraint, we utilize the two-stage hierarchical model as in (2), which was initially advocated by O'Hagan and Leonard [1] in which values of that do not belong to Θ are penalized to a lesser extent. In this respect, the normal model structure of (2) is referred to as HSGM in the remainder of the paper, because the two stages of a prior hierarchy on are considered and the resulting marginal prior distribution of becomes the weighted normal (or interval screened normal) distribution studied by Kim [6]. This is shown as follows. Since 0 ∼ ( , ) ( , 2 1 ), the marginal prior of under HSGM is where (⋅; , 2 ) and Φ(⋅), respectively, denote the density of ( , 2 ) and the distribution function of (0, 1), We see that (4) is the density of a weighted normal (WN) distribution by Kim [6]. This leads to the following assertion in Lemma 1. (2), eliciting the uncertain interval constraint (3), is marginally distributed as a weighted normal distribution,

Lemma 1. The two-stage prior of in HSGM of
Note that = [ | ≤ ≤ ], where ( , ) ⊤ ∼ 2 ( , Ω), a bivariate normal distribution. See Kim [6] for various properties of the WN ( , ) ( , Ω) distribution. According to O'Hagan and Leonald [1], the first stage variance, 2 0 , of the mean may measure the degree of uncertainty in the constraint. However, there is no systematic method to assign the values of 2 0 (or 2 1 ), according to a priori specified degree (say, (1 − ) × 100%) of uncertainty about the interval constraint (3). This is a major hindrance factor in developing the HSGM.

A Maximum Entropy
Prior. Sometimes we have a situation where partial prior information is available, outside of which it is desired to use a priori that is as noninformative as possible. A useful method of dealing with this problem is through the concept of entropy by Jaynes [7,8]. As discussed by Rosenkrantz [11], entropy has a direct relationship to information theory and in a sense measures the amount of uncertainty inherent in the probability distribution.
Assume now that we can specify the partial information concerning a location parameter (including the normal mean) with continuous space Θ of the form The Scientific World Journal 3 but with nothing else about our prior distribution ( ). Then the maximum entropy prior can be obtained by choosing ( ) that maximizes the entropy in the presence of the partial information in the form of (7). A straightforward application of the calculus of variation leads us to Boltzmann's maximum entropy theorem. This tells us that the density ( ) that maximizes ( ), subject to the constraints [ ( )] = , = 1, . . . , , takes the -parameter exponential family form where 1 , 2 , . . . , can be determined, via the -constraints, in terms of 1 , . . ., . See Leonard and Hsu [12] for the proof.

Maximum Entropy Prior for Constrained Normal Mean.
Let the location parameter of our interest be the normal mean in (1). Then the results of the previous subsection can be applied to the prior for the normal mean . This subsection considers the case where the partial priori information of is in the form of an interval, that is, ∈ Θ, Θ = [ , ] with < , and examines how the maximum entropy prior of , that is, ( ), has different formula according to the degree of uncertainty regarding the interval constraint. Case 1. Θ = R, and ∈ Θ with certainty.
When we have partial priori information about that we can specify values for both mean and variance 2 . Then the ( , 2 ) prior specification is the maximum entropy prior for (e.g., [12]). Thus the prior density and its entropy for the Case 1 are respectively.
On the other hand, when we are completely sure about the priori interval constraint of , a suitable restriction on the parameter space Θ such as using a truncated distribution is expected. This case supposes that we can specify values for both [ ] = and [( − ) 2 ] = 2 on the space ∈ R by a priori information. Further suppose that we have certain prior information that the parameter is concentrated on the region [ , ], that is, but nothing else about our prior distribution ( ). The last condition is equivalent to [ ( ≤ ≤ )] = 1. Therefore, by Bolzmann's maximum entropy theorem, the prior density of for the Case 2 is by (9), because 1 ( ) = , 1 = , 2 ( ) = ( − ) 2 , 2 = 2 , and 3 ( ) = ( ≤ ≤ ). Since 3 = 1, the maximum entropy prior, subject to these three restrictions, is thus provided that we choose 1 = 0 Some algebra using the moments of the ( , ) ( , 2 ) distribution in Johnson et al. [13] yields the entropy of 2 ( ) which is Case 3. Θ = [ , ], and ∈ Θ with (1 − ) × 100% uncertain.
Each graph of Figure 1 depicts the difference between Ent( 3 ( )) and Ent( 2 ( )) as a function of ∈ (0, 1) for three values of 2 , two cases of Θ = [ , ], and = 0. In Figure 1, the difference is denoted by Diff Ent . Since each graph coincides with the results of Corollary 3, we can obtain the following implications from the figure. (i) As expected, we see that Ent( 1 ( )) > Ent( 3 ( )) > Ent( 2 ( )) for ∈ (0, 1). (ii) The entropy of 3 ( ) is a monotone decreasing function of . (iii) Each entropy of the three priors increases as 2 becomes large. (iv) Diff Ent is closely related with degree of uncertainty (i.e,. (1 − ) × 100%) for it is a monotone decreasing function of ∈ (0, 1) and is a function of . (v) Diff Ent is a monotone increasing function of 2 for the case where a value of is given. measure for the degree of uncertainty regarding a prior interval constraint accounted for by HSGM. The following theorem proposes the objective measure using the same notations as used in Lemmas 1 and 2.
Theorem 4 provides an exact measure of the uncertainty about the priori interval constraint on accounted for by HSGM, and it shows that the uncertainty measure is different from that of O'Hagan and Leonald [1] which mainly depends on the first stage variance, 2 0 , of in HSGM. Theorem 4 also indicates that HSGM in (2) can be used to elicit the priori uncertain interval information associated with 3 ( ). Further, the entropy of the two-stage prior defined by the HSGM (i.e., ( ) in (4)) can be calculated by using the formula of Ent( 3 ( )) in (23). We can visualize the degree of uncertainty about the priori interval constraint, ∈ [ , ], by plotting 1 − for different values of ∈ (0, 1) in Figure 2.
From Figure 2, we can see, for fixed value of 2 , that HSGM with small values of tends to increase the uncertainty regarding the priori constraint. This coincides with the result which is asserted by Corollary 3. Further, we see from the figure that, for a fixed value of ∈ (0, 1), we can enlarge (or reduce) the uncertainty about the priori constraint by increasing (or decreasing) the amount of 2 (or equivalently 2 0 and 2 1 ) in the two stages of a prior hierarchy of . When the data information is ∼ ( , 2 ), it is well known that the maximum entropy priors (4) and (9) are conjugate priors for the normal mean when 2 is known. This is obtained from the following posteriors: where * = ( 2 + 2 )/( 2 + 2 ) and * 2 = 2 2 /( 2 + 2 ). Thus, we see that each prior satisfies the conjugate property that ( ) and ( | ) belong to the same family of distributions for = 1, 2. The following corollary provides this conjugate property which also applies to 3 ( ) in (22). (28) * = ( 2 + 2 )/( 2 + 2 ), and * 2 = 2 2 /( 2 + 2 ).

Posterior Estimation
Let us revisit the HSGM with the following two stages of a prior hierarchy of : According to Theorem 4, we see that the two-stage prior of is essentially the same as the maximum entropy prior which properly elicits the partial priori information of an uncertain interval constraint, { ; ∈ [ , ]}, with (1 − ) × 100% degree of uncertainty, where = Φ 2 ((a, b); , Σ)/(Φ(V * ( )) − Φ(V * ( ))). Here and 2 are true prior mean and variance of the parameter when the uncertain priori interval condition does not exist. Based on the marginal prior distribution of in Lemma 1, we have the joint posterior distribution of and 2 proportional to the product of likelihood and the prior distribution, is the inverse-gamma density with parameters and , and 3 ( ) is the density (22), that is, the density of ∼ WN ( , ) ( , Σ). Note that the joint posterior is not simplified in an analytic form of the known The Scientific World Journal 7 density and thus intractable for posterior inference. Instead, we derive each of the conditional posterior distributions of and 2 in an explicit form, which will be useful for posterior inference such as Gibbs sampling (e.g., [14]).