Constrained Parameters in Applications : Review of Issues and Approaches

This review article provides an introduction to statistical issues that arise when some statistical model parameters are constrained. This often happens in applications, in particular in testing for variance components (e.g. genomics) and construction of onesided confidence intervals (e.g. environmental risk analysis). Heuristic explanations are provided, and a number of general and recent statistical results that appeared in statistical literature are summarized for use in applications. Simulation results are shown for illustration of consequences of ignoring parameters on the boundary. Special attention is paid to likelihood ratio tests, but other approaches to confidence interval construction, such as Wald, bootstrap, and Bayesian are also briefly discussed. This paper presents examples from the risk assessment field and genomics, but all conclusions apply to whenever one-sided testing is conducted. Recommendations are provided for dealing with parameters on the boundary for a range of situations.


Introduction
Most standard results in statistical literature (e.g., [1]) are derived assuming that the true values of parameters are interior to the parametric space.It is well known that, under some very general conditions, the usual asymptotic theory of estimation based on the maximum likelihood estimates (MLEs) and the usual asymptotic theory of tests based on the likelihood ratio tests (LRTs) are valid and provide useful tools for meaningful statistical inference in large samples.Typically, we conclude the asymptotic normality of the MLE and the asymptotic chi-square distribution of −2 * ln (LRT) with suitable degrees of freedom (df) under a null hypothesis.This is what is generally covered in statistical textbooks.
However, the asymptotic distribution of the LRT in situations when regularity conditions do not hold, in particular when parameters of interest and/or nuisance parameters are on the boundary of a parameter space, is important in many applications, for example, in variance components testing in many natural science applications; however, statistical textbooks do not cover this subject.In the field of genetics and biology, there is indeed considerable amount of interest in statistical issues when some parameters may lie on the boundaries: in almost 1000 references to pioneering paper [2] on statistical results when parameters are on the boundary, about half come from publications in genetics and biology fields.
This review paper is organized as follows.Section 2 first introduces some examples from various applications that provide motivation.Next, it provides heuristic reasoning to demonstrate complicated statistical issues arising in boundary parameter situations using simple examples.Section 3 surveys literature on several approaches to parameters on the boundary and summarizes published results, paying most attention to the LRT.Finally, discussion section summarizes advice for scientists working in applications.

Statistical Issues due to Boundaries: Examples and Motivation.
We start with basic examples of normal distribution that we will carry through the next section.

ISRN Biomathematics
Example 1. Interior and boundary points of parametric spacefor parameters of interest and nuisance parameters.
We continue with an application from the risk assessment field.
Example 2 (constraints in multistage model).Toxicological experience and principles indicate that the response will generally be bounded and nondecreasing in the dose level [3].Monotonicity may require constraints on the parameters of a dose response model, or constraints also may be needed to ensure that the probability lies within [0; 1].Any of these constraints on a parameter leads to a parametric space that includes boundary.Quantal response models represent the probability P(d; θ) of a quantal response, like presence or absence of a particular type of cancer, in relation to dose (d) in a model with parameter vector θ.The experimental data consist of counts of animals exposed to a chemical, the numbers exhibiting the response, and the dose levels (e.g., n i , x i , d i , i = 0, 1, . . ., m) from a bioassay conducted with mice or rats, typically in 2-5 dose groups and the control group, each having 10-50 animals.Commonly used model for modeling bioassay data is called multistage All parameters of the multistage model are nonnegative, resulting in parameter space [0, +∞), with true or estimated values of parameters may be 0 or be on the boundary, and k is called the order of the multistage model, for example, model with k = 3 is called cubic.Maximum likelihood estimation for the parameters θ employs the binomial likelihood where B(n i , x i ) is the binomial coefficient.The benchmark dose method [4,5] consists of estimating a lower one-sided confidence limit for the dose associated with a specified increase in adverse response (i.e., increased risk) above the background level.
In the context of dose-response modeling, the statistical problems arising when true values of parameters of a doseresponse model may be on the boundary were acknowledged long ago [5] but have not been resolved clearly for the practitioners of dose-response modeling.Molenberghs and Verbeke [6] provide a nice discussion of these issues in the risk assessment context.
Our final example is variance component testing in genetics.
Example 3 (ACE model in twin studies).The ACE model used in twin studies consists of the three components: A (additive genetics), C (common environment), and E (unique environment).Given the ACE model, researchers can test which of the three variance components are present.The likelihood ratio statistic is generally used as the basis for these tests [7].The statistical test of AE against ACE alternative is the test of the hypothesis that there is no influence of environmental factors common among twins.This corresponds to the statistical test that variance component corresponding to C is 0. The test of E against ACE is the test of no familial correlation, that is, of the hypothesis that there is no influence of genetic or shared environmental factors.This corresponds to the statistical test of the null hypothesis that variance components corresponding to A and C are both 0. As variance is always nonnegative, the null hypothesis that variance component(s) is 0 is on the boundary of that parameter space.
What follows from the examples above is that most common causes of boundaries are one-sided testing and a dual problem of one-sided confidence interval construction.Suppose that we are testing the null hypothesis for a point θ 0 .The two-sided test H 0 : θ = θ 0 versus H 1 : θ / = θ 0 , and corresponding parameter space is [−∞, +∞].However, for onesided test: H 0 : θ = θ 0 , versus H 1 : θ > θ 0 the corresponding parameter space is [θ 0 , +∞], and θ 0 is necessarily a boundary point!Therefore, in one-sided testing, under null hypothesis, the parameter of interest is always on the boundary of the parametric space, and standard results, like one-sided t-test, are not applicable [6].
(a) μ-unrestricted.In this case, μ is estimated by which is a mixture of point mass at 0 and half-normal (Figure 1(b)).
Example 2 (continued).parameter is on the boundary, it is half-normal, but the other parameters are affected and are further from normality.
In the simple univariate example above, the distribution of restricted MLE is easily described.However, in a general multivariate case, the joint distribution of MLE is generally not multivariate Normal when just a single parameter has true value on the boundary (Section 3).

Likelihood Ratio Tests. The following example is continuation of Example 1(b).
Example 4. Likelihood ratio test (LRT) for a single normal random variable as supremum in the denominator is achieved when μ is equal to X.The distribution of −2 * log (LR) is a familiar χ 2 with 1 degree of freedom.
(b) One-sided test H 0 : μ = 0 versus H 1 : μ > 0. The likelihood ratio is In this situation, supremum in the denominator depends on whether X is positive or negative, and distribution of −2 * log (LR) is a mixture of point mass at 0 and χ 2 with 1df.
Example 4 demonstrates that when the LRT is used for one-sided tests, the distribution is not χ 2 with 1 degree of freedom anymore and can be fairly complicated.Various cases are described in the next section.

Likelihood Ratio Tests.
A pioneering theoretical article [2], following mainly on earlier work [8][9][10][11], proposed a general approach to deriving distribution of the LRT when some parameters of interest and/or nuisance parameters may be on the boundary of a parameter space.They showed that, under fairly general regularity conditions, the distribution of the LRT depends only on number of parameters of interest and nuisance parameters on the boundary and off the boundary.They also explicitly derived the LRT for several particular situations but were not completely correct for some of these.Sinha et al. [12] and Kopylev and Sinha [13] solved some of the cases in [2] and derived asymptotic distribution of the LRT for some other important special cases.All these results are summarized below.
(i) When one or several parameters of interest not on the boundary and some nuisance parameters are on the boundary, the asymptotic distribution of the LRT remains χ 2 with corresponding degrees of freedom [12].
(ii) When one parameter of interest is on the boundary and no nuisance parameters are on the boundary (e.g., Example 3 above with testing AE versus ACE), the asymptotic distribution of the LRT is a 50 : 50 mixture of point mass at 0 and χ 2 with 1 df [2].
(iii) When one parameter of interest is on the boundary and one parameter of interest not on the boundary and no nuisance parameters are on the boundary, the asymptotic distribution of the LRT is 50 : 50 mixture of χ 2 with 1df and χ 2 with 2 df [2].
(iv) When two parameters of interest are on the boundary and no nuisance parameters are on the boundary (e.g., Example 3 above), the asymptotic distribution of the LRT is a mixture of point mass at 0, χ 2 with 1 df and χ 2 with 2 df with mixing parameters p, 1/2 and 1/2 − p, where p = cos −1 (ρ)/2π, ρ = I 12 /sqrt(I 11 * I 22 ), and I is an information matrix (e.g., [13] for derivation of ρ).Unfortunately, this example in [2] has probabilities of the mixture components inverted, most likely a misprint.A lucky erroneous inversion of the sign in calculations [7] allowed them to make the correct recommendation.
(v) When one parameter of interest is on the boundary and one nuisance parameter is on the boundary, the asymptotic distribution of the LRT is a mixture of point mass at 0, χ 2 with 1df and χ 2 with 2df with mixing parameters 1/2 − p, 1/2 and p, where p = sin −1 (ρ)/2π, ρ = I 12 /sqrt(I 11 * I 22 ) ≥ 0, and I is an information matrix.When ρ < 0, asymptotic distribution is not a mixture of chi-squares but is explicitly derived [13].
(vi) When one parameter of interest and two nuisance parameters are on the boundary or two parameters of interest and one nuisance parameter are on the boundary, Kopylev and Sinha [13] derive a straightforward way for simulation of the asymptotic distribution of the LRT in these cases.
3.1.1.Simulations.Table 1 (adapted from [13]) demonstrates that, in case of one parameter of interest and two nuisance parameters on the boundary, distribution of the LRT can be very different from χ 2 with 1 df and use of χ 2 with 1 df can be anticonservative.

Wald.
The correct joint asymptotic distribution of the MLE in case of some parameters on the boundary is described in [2], following earlier work [8].It turns out that the asymptotic distribution of the MLE is not multivariate normal distribution anymore (cf.Example 1).Sinha et al. [12] adapted general statistical theory [2] for the multistage model and showed that asymptotic joint distribution of the MLE can be easily simulated starting from the standard multivariate Normal distribution.They also demonstrated how to construct confidence intervals following [2] and investigated the coverage of Wald-like confidence intervals for the multistage model.For sample sizes common for bioassays, the two-sided coverage of Wald-like intervals was comparable with that of the profile likelihood method for a wide range of scenarios.However, one-sided coverage by Wald-like intervals was found to be different from the nominal coverage [12].
3.3.Bootstrap.Bootstrap approaches have been suggested by many authors for use in the field of risk assessment.Bailer and Smith [14] investigated coverage of one-sided confidence intervals for the risk based upon parametric and nonparametric approaches to bootstrap, for a number of scenarios including true values of parameters on or near the boundary.They found that both parametric and nonparametric bootstraps may be anticonservative for some scenarios and very conservative for others.

Bayesian.
There is only limited literature on Bayesian approaches to boundary parameter problems, especially when nuisance parameters on the boundary are involved.
Gelfand et al. [15] recommended use of Bayesian methods when a parameter is constrained.However, this approach had been shown not to work well when distribution of the parameter has a nonzero mass concentrated on a boundary (e.g., [16]).For situations when there is a mass on the boundary, publications [16,17] suggest a Bayesian approach.

Discussion
In the field of science, there is indeed considerable amount of interest in statistical issues when some parameters may lie on the boundaries.However, a number of publications suggest, if one is willing to use a conservative test, to ignore the possibility of parameters on the boundary as these publications claim that using the usual chi-square distribution with appropriate degrees of freedom (as when no parameters are on the boundary) will result in a conservative test (e.g., [18,19] in the field of genetics, and [20] in the field of psychology).Nevertheless, results in [13] demonstrate that when one or more nuisance parameters are on the boundary, the suggested strategy of ignoring parameters on the boundary may result in anticonservative tests for many different correlation structures between parameters that are on the boundary, as seen in Table 1 illustrating what happens with two nuisance parameters on the boundary.Therefore, correct asymptotic distribution of the likelihood ratio test must be used whenever a parameter of interest on the boundary, for example, one-sided tests, is encountered, especially if nuisance parameters on the boundary are also present.Asymptotic results for many situations that are routine in applications are summarized in Section 3.1.However, in case a different situation is encountered in applications, the theory [2] and explanations and clarifications [12] can be used to derive correct asymptotic distribution of the LRT for that particular situation.In some cases, that asymptotic distribution may be explicit, but it is likely that in most cases the asymptotic distribution will have to be simulated.
From other approaches to confidence interval construction described in Section 3, the Bayesian approach (e.g., [16,17]) is the most interesting, but more research is required especially when there are constrained nuisance parameters.Another approach useful in applications would be to try to use a different model without boundary parameters or, if possible, reparameterize existing model so that to avoid situations with parameters on the boundary.
Figure 2  shows joint distribution of the parameters of cubic multistage model when no true parameters are on the boundary (top panel) and when a true quadratic term = 0 (bottom panel).When no parameters are on the boundary, the joint distribution of parameters is multivariate normal, as expected.When the quadratic

Table 1 :
Upper percentiles of the asymptotic distribution of the LRT in case of one parameter of interest and two nuisance parameters on the boundary, calculated by simulations.Correlations are between parameter of interest and nuisance parameter 1, parameter of interest and nuisance parameter 2, and nuisance parameters 1 and 2. The last row shows corresponding percentiles of the chisquare with one degree of freedom-the asymptotic distribution of the LRT when complications due to boundary parameters are ignored.Percentiles exceeding corresponding percentiles of χ 2 (1), that is, situations when use of χ 2 (1) is anticonservative, are bolded.