On the approximation of an integral by a sum of random variables

We approximate the integral of a smooth function on [0,1], where values are only known at n random points (i.e., a random sample from the uniform-(0,1) distribution), and at 0 and 1. Our approximations are based on the trapezoidal rule and Simpson's rule (generalized to the non-equidistant case), respectively. In the first case, we obtain an n2-rate of convergence with a degenerate limiting distribution; in the second case, the rate of con-vergence is as fast as n3½, whereas the limiting distribution is Gaussian then.

estimator we will employ is constructed by using the 'trapezoidal rule' on each sub- b interval [Ui_l,Ui] i-1,...,n+ 1.This rule approximates an integral f g(x)dx simply by 1/2(b-a)(g(a)+ g(b)) and it can easily be shown (see, e.g., Isaacson and   Keller [2], p. 304)that b 1/2(b-a)(g(a) $ g(b)) / g(x)dx 2(b a)3g"(r/), (2) a where z/E (a,b).Writing D Ui-Ui_l, 1,...,n+ 1, for the spacings of the Ui's our estimator of I becomes n+l In: E 1/2Di(f(Ui-1) + f(Ui))" (3) i=1 Using (2), we will prove the following limiting result for the standardized difference of I n and I: A much better and probabilistically more interesting estimator is obtained by b applying a 3-points formula, i.e., for a given c e (a,b), we approximate f g(x)dx by a wlg(a + w2g(c + w3g(b in such a way that the approximation error is zero in the case g is a polynomial of second degree.If the 3 points are equidistant, this approxi- mation is known as Simpson's rule.It is not hard to show that (5)   and it follows (see again  wlg(a)+w2g(c)+w3g(b ]" g(x)dx--/ (x-a)(x-c)(x-b)g(3)(rl)dx, (6)   a a where r/-r/(x) (a,b).Hence, our estimator of I in (1), again denoted by In, be- where, for convenience, n is taken to be odd.Formula (6) will be used to prove our main result: Theorem 2-Let n be odd.If If(5) is bounded, then n 2,( where Z is a standard normal random variable.Remark 1: The present techniques can be easily adapted to cover the situation where the Ui's are the order statistics of n independent random variables with com- mon distribution function G (on (0, 1)) having a smooth density g.The adaptation is based on the quantile transform, transforming a uniform random variable V into a random variable G-I(V) with distribution function G.In this case, under regularity conditions on g, we obtain that the weak limit in Theorem 1 becomes 1 1 lf2 (f"(x)/g2(x))dx instead of 1/2f f"(x)dx --}(if(l)-f'(0)).In Theorem 2, the 0 0 limiting random variable is again centered normal but now the standard deviation becomes On the other hand, the uniform distribution seems very relevant because of the 1 following.Since f f(x)dx can be considered as the mean 'output', given that the x- 0 1 values are 'equally important', it seems desirable to estimate f f(x)g(x)dx- )dy in the case the random variables are distributed according to G.But 0 if G is known, we can replace the pairs (Ui, f(Ui) (just below (1)), with Ui's being the order statistics from G, by (G . This brings us back to the 'uniform distribution setup' with f replaced by f o G-, but that is just the function whose integral we wanted to estimate as argued above!This idea leads to possible ways of applying the results.Suppose U represents some uncontrollable physical random quantity, like temperature, humidity or light intensity with a known distribution function G having density g.Suppose also that we can measure f (the output or yield) only at the U and that we are interested in 1 the mean output I g-f f(z)g(z)dx.Then one can use our theorems to obtain rapid- 0 ly converging estimators of Ig.In particular, when measuring the f-values is hard or expensive, one can get good estimators based on a few observations.Also note that for the trapezoidal rule in Theorem 1 and f" being constant, the 1 1 uniform distribution is optimal, since f g-2(x)dx >_ f ldx-1. (This can be easily seen by using Jensen's inequality" where X is a random variable with density g.)A similar remark applies to Theorem 2 with f(3) being constant.
Remark 2: There are various other ways to extend our results, which we will not pursue here, e.g., applying m-points formulas for m > 3 (Simpson's rule is 'by far the most frequently used in obtaining approximate integrals', Davis and Rabinowitz [1], p. 45), combining trapezoidal rules to eliminate the bias (fl'(1)-f'(0)), proving a 'second order' limit result for n2(In I) 1 -(f (1)-if(0))in Theorem 1, or treating the case n 'even' in Theorem 2. We are not pursuing these extensions because we believe they are not very interesting and/or they do not give good results.
Remark 3: We briefly compare our results with the deterministic, equidistant i-0,1 n/l.It is well-known that the limit in Theorem 1 case, i.e., U n -1' is 1 (f (1)-f'(0)) in that case, which means that we loose a factor of 6 by having random Ui's. (Essentially, this 6 is coming from the third moment of a standard ex- ponential random variable.)From Theorem 2, it is well-known that in the equidis- tant case (Simpson's rule), the rate is n4.So, there our loss is of order n 1/.Never- 1 theless, from statistical point of view, n 3 is a remarkably fast rate of convergence.

Proofs
The following well-known lemma will be used frequently; it can be found in, e.g., Shorack and Wellner [3], p. 721.
Lemma 1: Let E1,...,E n+ 1 be independent exponential random variables with mean 1 and S n + 1 be their sum.With Di, i-1,...,n + 1, as before, we have (DI," nn+i )d( E1 En+i) " Sn + 1 Sn-t-1 Proof of Theorem 1" Using (3), ( 1)and ( 2) we see that _rt2 n-bl i--1 for some U (Ui_ 1,Ui), and hence, n "+ ( ) n2 n-bl n2(In-I) -l DT" n + l + D(yi )T'"(Ui), ( 9) From the boundedness of f'" (by M, say) and the with U between U and n + 1" weak convergence (to a Brownian bridge) of the uniform quantile process (see, e.g., Shorack and Wellner [3]), it is readily seen that by Lemma 1, and by two applications of the weak law of large numbers, this last ex- pression is Op(n-2).Combining this with ( 10) and ( 11) yields that the second term on the right in (9) converges to zero in probability.Hence, it remains to consider the first term The proof is complete by noting that n-bl The proof of Theorem 2 is heavily based on the following two lemmas.
Proof: By ( 7), ( 1)and ( 6)we have 31 n 2(I n I) for somme U2i-g2i(x 6_ (U2i_2, g2i) and hence for some (U2i-2, U2i), the right-hand side of ( 12) is equal to Let M be a bound on f(5) and all lower order derivatives of f.Then the abso- lute value of this last term is bounded from above by _< --ff-n i=IE (D2i-1 + D2i 6 Sn + 1 1 due to Lemma 1 and two applications of the weak law of large numbers.
So, it suffices to show the convergence to zero in probability of T1, n -}-T2, n' for some 2i 2 between U2i 2 and 2i-,,,2 By the weak convergence of the uniform n4-1" quantile process, By Lemma 1 and twice the weak law of large numbers, this last expression is easily seen to be Op(1).Hence, the proof of Lemma 3 is complete if we show T1, n Op(1).