Multiple Hypotheses LAO Testing for Many Independent Objects

The procedure of many hypotheses logarithmically asymptotically optimal (LAO) testing for a model consisting of three or more independent objects is analyzed. It is supposed that M probability distributions are known and each object follows one of them independently of others. The matrix of asymptotic interdependencies (reliability-reliability functions) of all possible pairs of the error probability exponents (reliabilities) in optimal testing for this model is studied. This problem was introduced (and solved for the case of two objects and two given probability distributions) by Ahlswede and Haroutunian; it is a generalization of two hypotheses LAO testing problem for one object investigated by Hoeffding, Csiszár and Longo, Tusnády, Longo and Sgarro, Birgé, and others.


Introduction
In [1][2][3] Ahlswede and Haroutunian formulated an ensemble of new problems on multiple hypotheses testing for many objects and on identification of hypotheses.These problems are extensions of those investigated in the books mentioned in [4,5].Problems of distribution identification and distributions ranking for one object were solved in [2].Also the problem of hypotheses testing for the model consisting of two independent or two strictly dependent objects (when they cannot admit the same distribution) with two possible hypothetical distributions was investigated in [2].In this paper we study the specific characteristics of the model consisting of K(≥ 3) objects which independently of others follow one of given M(≥ 2) probability distributions.The study concerns certain number K of similar objects (cities, institutions, schools, hospitals, factories, etc.), or one object in a series of K different periods of time.The problem is a generalization of two hypotheses testing investigated in [6][7][8][9][10] and of testing of many hypotheses concerning one object solved in [11].The case of two independent objects with three hypotheses was examined in advanced edition [12], in a local publication of small circulation.
Investigation of testing of the distributions of many uniform objects is an interesting not yet fulfilled task.It is natural to begin this study with the simplest case of statistically independent objects.Let P (X) be the space of all probability distributions (PDs) on finite set X.There are given M distinct PDs G m ∈ P (X), m = 1, M, which are known as possible distributions of the objects.
Let us recall main definitions from [11] for the case of one object.The random variable (RV) X, which is a characteristic of the studied object, takes values on X and follows unknown PD G which is one of M given PDs G m , m = 1, M. The statistician have to accept one of M hypotheses observations of the object.The procedure of the decision making is a nonrandomized test ϕ N , which can be defined by division of the sample space The set A N l contains all vectors x for which the hypothesis H l is adopted.The probability α l|m (ϕ N ) of the erroneous acceptance of hypothesis We define the probability to reject H m , when it is true, as ( The exponential decrease of the error probabilities as N → ∞ is studied.The error probability exponents which is pertinent to call reliabilities, of the sequence of tests ϕ, are defined as follows: From ( 1) and ( 2) we see that The matrix we call it the reliability matrix of the sequence ϕ of tests.It was studied in [11].The question is values of which number of elements of E(ϕ) can be given in advance and which optimal values can be guarantied by the best test for the others.
The sequence of tests ϕ * is called logarithmically asymptotically optimal (LAO) if for given positive values of first M − 1 diagonal elements of the matrix E(ϕ * ) maximum possible values are provided to all other elements of it.The concept of LAO test was introduced by Birgé [10] and also elaborated in [11,12].
Let us now consider the model with three objects.Let X 1 , X 2 , and X 3 be independent RVs taking values in the same finite set X with one of M PDs, this RVs are the characteristics of the corresponding independent objects.The random vector (X 1 , X 2 , X 3 ) assumes values (x 1 , , N, be a sequence of results of N independent observations of the vector (X 1 , X 2 , X 3 ).The test have to determine unknown PDs of the objects on the base of observed data.The selection for each object should be made from the same set of hypotheses: We call this procedure the compound test for three objects and denote it by Φ N , it can be composed of three individual tests ϕ 1 N , ϕ 2 N , ϕ 3 N for each of three objects.We denote the infinite sequence of compound tests by Φ.When we have K independent objects the test Φ is composed of tests ϕ 1 , ϕ 2 , . . ., ϕ K .
Let α l1,l2,l3|m1,m2,m3 (Φ N ) be the probability of the erroneous acceptance of the hypotheses triple (H l1 , H l2 , H l3 ) by the test Φ N provided that the triple of hypotheses The probability to reject a true triple of hypotheses (H m1 , H m2 , H m3 ) by analogy with (1) is the following: We study corresponding reliabilities E l1,l2,l3|m1,m2,m3 (Φ) of the sequence of tests Φ E l1,l2,l3|m1,m2,m3 (Φ) lim Definitions ( 5) and ( 6) imply (cf.( 3)) that We call the test sequence Φ * LAO for the model with K objects if for given positive values of certain part of elements of the reliability matrix E(Φ * ) the procedure provides maximal values for all other elements of it.
We consider the problem for three objects for brevity; the generalization of the problem for K independent objects will be discussed hereafter along the text and in Section 4, but before that we recall the results for one object.The generalization of the problem for cases when RVs X i take values in different sets X i and have hypothetical PDs G i m , m = 1, M, i = 1, 3, will be only more complicated in notations.

LAO Testing of Many Hypotheses for One Object
We define the divergence (Kullback-Leibler distance) D(Q||G) for PDs Q and G from P (X) as usual: For given positive diagonal elements E 1|1 , E 2|2 , . . ., E M−1|M−1 of the reliability matrix we consider sets of PDs and the following values for elements of the future reliability matrix of the LAO tests sequence: We recall the theorem concerning one object.
Theorem 1 (see [11]).If the distributions G m are different, that is, all divergences D(G l ||G m ), l / = m, l, m = 1, M, are strictly positive, then two statements hold: (a)when the given numbers E 1|1 , E 2|2 , . . ., E M−1|M−1 satisfy the conditions then there exists an LAO sequence of tests ϕ * , the elements of the reliability matrix of which E(ϕ * ) = {E * l|m } are defined in (10)-( 13) and all of them are strictly positive; (b) even if one of the conditions (14) or (15) is violated, then the reliability matrix of any such test includes at least one element equal to zero.

Corollary 1. The diagonal elements of the reliability matrix of the LAO test in each row are equal only to the element in the same row and in the last column:
That is, the elements of the last column are equal to the diagonal elements of the same row and due to (3) are minimal in this row.Consequently first M −1 elements of the last column also can be considered as given parameters for construction of the LAO test.
Proof.For m = M (16) is the sequence of (3).From the conditions ( 14) and (15) we see that . Applying Kuhn-Tucker theorem for (11) we can derive (the proof is not difficult, but long, so we avoid the exposition) that the elements E * l|l , l = 1, M − 1 may be determined by elements E * l|m , m / = l, m = 1, M, with the following inverse functions: Then it follows from (11) and our provisional supposition that but one can see from conditions ( 14) and ( 15) that Our assumption is not correct, hence ( 16) is valid and equality (3) implies E * m|m = E * M|m .

LAO Testing of Hypotheses for Three Independent Objects
Now let us consider the model of three independent objects and M hypotheses.It was noted that the compound test Φ N may be composed from separate tests ϕ 1 N , ϕ 2 N , ϕ 3 N .Let us denote by E(ϕ i ) the reliability matrices of the sequences of tests ϕ i , i = 1, 3, for each of the objects.The following lemma is a generalization of lemmas from [2,12].Lemma 1.If the elements E l|m (ϕ i ), m, l = 1, M, i = 1, 3, are strictly positive, then the following equalities hold for Φ = (ϕ 1 , ϕ 2 , ϕ 3 ): Equalities (19) are valid also if E li|mi (ϕ i ) = 0 for several pairs (m i , l i ) and several i's.
Proof.It follows from the independence of the objects that Remark that here we consider also the probabilities of right (not erroneous) decisions.Because E l|m (ϕ i ) are strictly positive then the error probability α l|m (ϕ i N ) tends to zero, when N → ∞.According to this fact we have lim From definitions ( 5) and ( 6), equalities (22), and applying (23), we obtain relations ( 19)-( 21).Now we will show how we can erect the LAO test from the set of compound tests when 3(M − 1) strictly positive elements of the last column of the reliability matrix E M,M,M|m,M,M , E M,M,M|M,m,M and E M,M,M|M,M,m , m = 1, M − 1, are preliminarily given.
The following subset of tests: is distinguished by the property that when , and E M,M,M|M,M,m (Φ), m = 1, M − 1, of the reliability matrix are strictly positive.Really, because E m|m (ϕ i ) > 0, then in view of (3) E M|m (ϕ i ) are also strictly positive.From equalities (23) keeping in mind ( 6), (16), and ( 22) we obtain that the noted elements are strictly positive for Φ ∈ D and E M,M,M|m,M,M , E M,M,M|M,m,M , and  E M,M,M|M,M,m , m = 1, M − 1:

Define the following family of decision sets of PDs for given positive elements
Define also the values of the reliability matrix of the LAO test for three objects: The following theorem is the main result of the present paper.It is a generalization and improvement of the corresponding theorem proved in [2] for the cases K=2, M=2.
(b) When even one of the inequalities (31)-( 34) is violated, then at least one of inequalities (36) is violated.Then from Theorem 1 one of elements E m|l (ϕ i, * ) is equal to zero.Suppose E 3|2 (ϕ 1, * ) = 0, then the elements Theorem 2 is proved.

On the Case of K(> 3) Objects
When we consider the model with K independent objects the generalization of Lemma 1 will take the following form.

Conclusion
We exposed a solution of multiple hypothesis LAO testing problem for many objects.The first idea may be to study matrix E(Φ) by renumbering K-vectors of PDs from 1 to M K as PDs of one complex object.We can give M K − 1 diagonal elements of such matrix E(Φ) and apply Theorem 1 concerning one object.In this case the number of the preliminarily given elements of the matrix E(Φ) would be greater (because M K − 1 > K(M − 1), M ≥ 2, K ≥ 2), and the procedure of calculations would be longer than in our algorithm presented in Section 3.
Proposed approach to the problem gives also the possibility to define the LAO tests for each of the separate objects.It must be noted that the approach with renumbering of the triples of hypotheses does not have this opportunity.
In applications one of two approaches may be used in conformity with preferences of the investigator.

Theorem 2 .
If all distributions G m , m = 1, M, are different, (and equivalently D(G l ||G m ) > 0, l / = m, l, m = 1, M), then the following statements are valid: (a) when given strictly positive elements E M,M,M|m,M,M , E M,M,M|M,m,M , and E M,M,M|M,M,m , m = 1, M − 1, meet the following conditions: