Improved Inequalities for the Poisson and Binomial Distribution and Upper Tail Quantile Functions

The exact evaluation of the Poisson and Binomial cumulative distribution and inverse (quantile) functions may be too challenging or unnecessary for some applications, and simpler solutions (typically obtained by applying Normal approximations or exponential inequalities) may be desired in some situations. Although Normal distribution approximations are easy to apply and potentially very accurate, error signs are typically unknown; error signs are typically known for exponential inequalities at the expense of some pessimism. In this paper, recent work describing universal inequalities relating the Normal and Binomial distribution functions is extended to cover the Poisson distribution function; new quantile function inequalities are then obtained for both distributions. Exponential bounds—which improve upon the Chernoff-Hoeffding inequalities by a factor of at least two—are also obtained for both distributions.

Due to numerical and complexity issues, evaluation of the exponential and Binomial summations in (1) and ( 2) through recursive operations is only practical for small values of the input parameters ( or  and ).Instead, a better solution is to evaluate the CDFs directly through either their incomplete Beta/Gamma function representations which can be approximated to high precision by continued fractions or asymptotic expansions [3].With respect to the quantiles of the distributions given by ( 3) and (4), no methods to exactly evaluate these functions without iterating the exponential/Binomial sums-or alternately employing a search until the required conditions are satisfied-seem to be known.Typically, a binary search to determine the smallest  satisfying (3) or (4) evaluating the respective CDF at each step would be a better general solution, given some initial upper bound for .Such methods (and related variants) are now employed very effectively in modern commercial and research-based statistical packages.
In some situations, one may desire simpler solutions to either approximate or bound these quantities.Typically an approximation can be obtained via the standard Normal distribution; the work of Mollenaar [4] contains a good description of several applicable variants.Although quickly applied and potentially very accurate for large input parameters (due to the central limit theorem), the sign of the approximation errors is typically unknown.Exceptions are the inequality of Bohman (see [1, page 169]), which always overestimates the true Poisson probability, and the expressions recently proposed by Zubkov and Serov for the Binomial distribution [5].Methods to obtain provable bounds with known error signs (typically one would require to underestimate (1) and ( 2), whilst overestimating (3) and ( 4) in most engineering and computer science applications) principally include the Bernstein/Chernoff/Hoeffding-type exponential probability inequalities and their close variants [1,2,6,7].Although effective, one has to accept the unavoidable loss of accuracy with these bounds.
Although the need for provable, accurate bounds has been well documented in computer science, information engineering, and reliability analysis applications ( [1,2] provide such discussions), the motivation for the current work arose from a recent application in probabilistic schedulability analysis for real-time systems described by Short and Proenza [8].In this work, the authors consider efficient admission controls for providing probabilistic schedulability guarantees for real-time messages traversing communication channels with error arrival characteristics that can be approximated by Poisson or Binomial distributions.Ultimately, it is required in this application to evaluate many upper tail quantiles (directly corresponding to (3) or ( 4)) in a very short space of time by a (possibly resource-constrained) embedded microcontroller or microcomputer.Clearly the use of a commercial statistical package is not possible; several logarithmic inequalities were instead developed for these purposes.Although the bounds were shown to be tight in terms of relative errors (which become vanishingly small as the input magnitude becomes large), the absolute errors on the other hand become increasingly large as the input parameters increase.Therefore, one of the motivations for the current work was to tighten these quantile inequalities, with the goal of making them asymptotically near-exact in the input parameters.In this paper such tighter bounds are obtained, along with several other inequalities which may have a more general interest.
The remainder of the article is organized as follows.In Section 2, the recent work on the categorization of the Binomial with respect to the Normal distribution in [5] is first extended to obtain universal inequalities (with known error signs) relating the Poisson and Normal distribution functions.Section 3 obtains asymptotically near-exact analytic inequalities relating both the Poisson and Binomial upper tail quantiles to the Normal quantiles.Improved Chernoff/ Hoeffding-type exponential inequalities are then obtained for both distributions in Section 4. A brief summary is then given in Section 5.

Distribution Function Inequalities
Consider the following recently proven universal inequality on the distribution function of a Binomially distributed random variable.Theorem 1.Let  ∼ Bin(, ) be a random variable with parameters  ∈ {1, 2, 3, 4, . ..} and  ∈ (0, 1), where the integer  represents the number of trials and  the probability of success in each trial.Denoting the CDF of  as { ≤ } as per (2), then, for  = 0 and  = , one has the exact equalities { ≤ 0} = (1 − )  and { ≤ } = 1, and for all  > 0,  ∈ (0, 1), and  ∈ {1, 2, . . .,  − 2} the following inequalities hold: ( And for  =  − 1 one also has: Proof.Zubkov and Serov [5]. Although it was not explicitly denoted as such in [5], it is easy to see that (, ) represents the Kullback-Leibler ( KL ) divergence between two Bernoulli variables with respective probabilities of success  and ; hence, (, ) represents the KL divergence of  summed pairs of such variables.This observation allows the relatively straightforward extension of the above result to the case of a Poisson distributed random variable, which is given in Theorem 2.
Theorem 2. Let  ∼ Poi() be a Poisson distributed random variable with mean .Let the distribution function { ≤ } be defined as in (1), with integer support  ∈ {0, 1, . . ., ∞}.For  = 0 and  = ∞, one has { ≤ 0} =  − and { ≤ ∞} = 1.For every other  = {1, 2, 3, . ..} one has the following inequalities: where (, ) is the  divergence between two Poisson distributed random variables with respective means  and : Proof.The cases  = 0 and  = ∞ are exact equalities which are easily derived from the distribution function (1).For the other cases, first we form the variable  ∼ Bin(, ) with some finite integer  and  ∈ (0, 1), such that  = .Clearly, this variable satisfies inequality (5) for any choice of  − 2 >  > 0. Now, suppose we increase  by one and reduce  such that the constraint  =  still holds.Again, this variable still satisfies inequality (5).Now, incrementally repeat this procedure for increasing  under the constraint that  =  indefinitely; in the limit as  → ∞ one has the following: With the identity above being the famous limit theorem of Poisson, a contemporary description of which may be found in [1].Now consider the limit of the argument to Φ(); observe that as  → ∞ as the Bernoulli variables with success probability  = / become Poisson distributed with mean  = / = , then it must also follow that as  → ∞ the infinite number of Bernoulli variables with success probability / must also become Poisson distributed with mean / = .Thus, the KL divergence of this infinite number of Bernoulli variable pairs with respective success probabilities / and / becomes the KL divergence of two Poisson distributions with respective means  and , which is given by the function (, ) =  −  +  ⋅ ln (/) [9].Thus, the following limit must hold: Noting that, as (5) holds at each step as  is increased to infinity, it must also hold in this limit for any  = {1, 2, 3, . ..}, which implies the inequalities stated in (7).
As in the Binomial case, these relationships may be used to bound the quantile of a Poisson random variable to a pair of adjacent integers.However, given a desired , no analytical expression for a corresponding  may be obtained due to the presence of the term  ln(/) in the expression for (, ).A similar restriction occurs with (, ) due to the presence of multiple such terms involving  and  and their natural logarithm.Slightly weaker inequalities having an analytical form for the upper tail quantiles are thus obtained in the next section.

Upper Tail Quantile Inequalities
The first step in the Poisson quantile inequality is to obtain an expressive bound on the function (, ) defined by (8).Lemma 3.For any  > 0 and 0 <  < ∞, one has.
Proof.Observe that, for  = , (, ) =   (, ) = 0. Elementary calculations yield the first partial derivatives of both functions with respect to : And observe again that, when  = , (, )  =   (, )  = 0. Now considering the sign of the derivatives, when  >  we have that (, )  > 0 since / > 1; we also have that   (, )  > 0 as the quantity  2 +2 > 3  2 .Similarly, for  < , (, )  < 0 since / < 1 and   (, )  < 0 as  2 + 2 < 3 2 .Thus, both functions are monotonically decreasing in  over the interval (0, ) and monotonically increasing in  over the interval (, ∞]. A further application of the calculus yields the second partial derivatives of (, ) and   (, ) with respect to : And it is easy to verify that both functions are positive for all positive nonzero  and .Now, the objective is to show that (, ) ≥   (, ) for the specified ranges of  and .
Remarks.  (, ) provides a good quality bound over all ranges of  and , but it is at its tightest when  ≈ ; as the mean  increases and the quantile clusters around the mean ( ≈ ), the bound can be expected to be asymptotically very tight.Although the right-hand side of (11) seems to have a strange form, it allows one to obtain a tail quantile bound with a simple structure.Let Φ −1 () for  ∈ (0, 1) be the inverse of the standard normal CDF Φ() (i.e., the "probit" function), such that Φ(Φ −1 ()) = .The inequality can be stated.
Theorem 4. Let  ∼ Poi() be a Poisson distributed random variable with mean , and let 1 >  > 0.5.Then, one has the following bound on the th quantile of : Proof.Consider the lower inequality in (7), and suppose that the left-hand side evaluates to ; this clearly implies  > { ≤ }.Since  > 0.5, we are working in the upper tail ( > ) and sign( − ) = 1; thus, we seek an integer  such that the following holds true: Applying the probit function to both sides of the equation, then squaring both sides and dividing through by 2 isolates  on the left hand side: ( Recalling that (, ) ≥   (, ), an integer  which satisfies   (, ) ≥ Φ −1 () 2 /2 also clearly guarantees that { ≤ } > .At this point, let us substitute into (17) the label  = Φ −1 () 2 /2, let  be real valued, assume an equality, and replace (, ) with the definition of   (, ) from ( 11): Simple rearrangement and squaring eliminate the root: Expanding the right hand side and then gathering terms leads to a quadratic in : Taking the principle root (to ensure  > ) gives: from which (15) is recovered by employing the ceiling function to make  integer and then substituting  = Φ −1 () 2 /2 back into the resulting expression.
For the Binomial quantile, below is presented a corresponding bound for the Binomial upper tail quantile   (, , ).The inequality is very similar to that of Theorem 4, but skewness correction terms now also appear.Theorem 5. Let  ∼ Bin(, ) be a Binomial distributed random variable with parameters  and , and let 1 >  > 0.5.The following bound holds on the th quantile of : Proof.The proof proceeds upon almost identical lines to that of Theorem 4, except that the lower inequality in ( 5) provides the starting point.The inequality on (, ) given in (23) below, which as shown by Janson [10] is valid for  ≥  provides the analytical relaxation allowing the quantile to be explicitly solved for: Simplifying the resulting expression leads to (22).
Observe that the Poisson upper tail quantile inequality (15) is virtually identical to the expression obtained from applying a Cornish-Fisher expansion with a continuity correction to the Poisson quantile [4]: It is immediately seen from Theorem 4 that, if overestimation of the true quantile by a small factor is always desired, it suffices to simply neglect the 4/6 correction and the contribution of the asymptotic term in the above expansion.The sharpness of (15) is evident as for sufficiently large  the contribution of the term ( −0.5 ) → 0 and the gap between (15) and ( 29) is always ≤1.A similar relationship holds between expression (22) and the asymptotic expansion of the Binomial quantile [4].Practical experience with both quantile bounds indicates that equalities can be achieved even for very small values of standard deviation: Figure 1 gives an illustrative comparison between the exact and bounded th quantiles for a Poisson process having a mean of () = 0.1 for increasing values of .
The sharpness of the bound is evident, and it may be observed that the gap between the bound and the exact quantile quickly reduces as  (and hence the mean and standard deviation) increases in both cases.This illustrates the quick convergence to the observation that the asymptotic gap is always ≤1.

Exponential and Logarithmic Inequalities
It is easy to verify the sharpness of the Poisson CDF universal inequality presented in Section 2 (the sharpness of the inequality for the Binomial distribution follows from the discussions in [5]), and its form makes it relatively easy to compute and implement; both the standard Normal CDF Φ() and its inverse Φ −1 () for the quantile inequalities can be quickly calculated to machine precision by simple rational approximations [3,11].It may still, however, be desired to have simpler bounds that have closed forms (e.g., it may be needed to algebraically manipulate a probability expression for the probabilistic analysis of an algorithm).Firstly, observe that the trivial inequality 1 − exp(− 2 /2) ≤ Φ() for  > 0 can be used to recover the Chernoff exponential bound for the Poisson upper tail CDF by substitution of  with √2(, ).The same inequality may be used to recover the bound of Hoeffding [7] by replacing  with √2(, /).This leads to consider the possibility of recovering tighter exponential bounds which retain simple forms for these quantities, employing known inequalities on the standard normal CDF.To achieve this, consider first the quantity known as Mills' ratio (), which is defined for real arguments  in the usual way: where () =  −0.5 2 / √ 2 is the standard normal density function.For nonnegative , it is known that () is monotonically decreasing in  and achieves a maximum of √ 0.5 at  = 0. Various simple inequalities are known for this function; consider the simple upper bound () ≤  −1 for  ≥ 0 proved by Gordon [12].For small ,  −1 can become larger than √ 0.5 and the basic bound can be improved by selecting the smaller of these two values: Expression (26) may be employed to sharpen the Chernoff/Hoeffding bounds considerably via the inequalities of Theorems 1 and 2.
Expressions (28) and (29) are tighter than the corresponding Chernoff/Hoeffding bounds by a factor of at least 2, due to the presence of the additional denominator terms; in fact, they are much tighter for larger deviations as the denominators under the exponential become ≫ 2.An illustration of the improvement that is obtained by adopting expression (29) over Hoeffding's original bound is given in Figure 2. The figure illustrates the improvement in the lower bound on the distribution function that is obtained for increasing values of the normalized argument (, /) in the range from 0 to 6.
Moving deeper into the tail, it is easy to verify that the improvement is continually increasing; this can also be illustrated by way of a simple example.Suppose  = 1000,  = 0.1, and  = 150.Then the true Binomial distribution function has a value of 0.999999723 and Hoeffding's inequality gives a lower bound of 0.999995143.Application of (29) gives a value of 0.999999608 which is an order of magnitude closer to the true Binomial function; the complimentary probability has been reduced by a factor of 12.399. .., which is in fact the value of the function √4(, /) at this point.Extension of ( 29 ) is obtained by employing (29) with  = √2(, /), as per Hoeffding [7].Sharper inequalities for () can be used to improve these exponential bounds further, at the expense of increased complexity.

Conclusions
In this paper, some improved inequalities with a relatively simple form have been developed for the Poisson and Binomial distribution and quantile functions.Analysis and observations have helped to illustrate some improvements over previous work and related bounds.The obtained expressions should prove to be most useful in situations where provable and accurate bounds having analytic or closed forms are required and/or situations in which the use of commercial statistical software packages is not possible (see [8] for an example application arising in probabilistic real-time analysis).As a final remark, it seems that an interesting bound on the natural logarithm which to the authors' knowledge does not seem to have been previously described can also be obtained as a direct corollary of Lemma 3, which slightly sharpens similar bounds such as ln() ≥ (2 − 2)/( + 1) (see [14, page 160]).
with equality occurring only for  = 1 and the sign of inequality reversed if 0 <  < 1.
Proof.Replace / by  in (12); the corollary follows directly from Lemma 3 and some algebraic simplifications.
Form the function (, ) = ((, )  −   (, ))  .Standard analytic techniques yield the two roots of  as  =  and  = −/8.As only the former root lies in the interval of positive  and , (, )  and   (, )  can intersect only at this root which implies that the sign of the function  can only (potentially) change once at this location.Verification that (, )  dominates   (, )  reduces to demonstrating that, for arbitrary positive non-zero ,  > 0 for some  satisfying 0 <  <  and also that  > 0 for some  > .For simplicity, let us choose  = /2 and  = 2 :

ISRNFigure 2 :
Figure 2: Comparison of the improvement of the distribution function lower bound in terms of the normalized argument (, /) between expression (29) (green) and Hoeffding's inequality (red).