Minimax Robust Optimal Estimation Fusion for Distributed Multisensor Systems with a Relative Entropy Uncertainty

This paper considers the robust estimation fusion problem for distributed multisensor systems with uncertain correlations of local estimation errors. For an uncertain class characterized by the Kullback-Leibler (KL) divergence from the actual model to nominal model of local estimation error covariance, the robust estimation fusion problem is formulated to find a linear minimum variance unbiased estimator for the least favorablemodel. It is proved that the optimal fuser under nominal correlationmodel is robust while the estimation error has a relative entropy uncertainty.


Introduction
During the past decades, multisensor systems have received much attention in many applications, such as signal processing, communication, target tracking, and remote sensing [1][2][3][4][5][6][7].In distributed estimation fusion frame, each sensor observes some local information of the physical environment, estimates the state or parameter of system in terms of some optimization criterion, and then transmits it to fusion center.The fusion center fuses the data from local sensors and obtains a better estimate.Generally, distributed fusion may have a less computational burden, lower communication rates, and higher survivability and be more flexible and reliable than centralized fusion which processes directly observation data from local sensors.
There is much work on developing distributed estimation fusion algorithms.In [8,9], a distributed Kalman filtering fusion formula was presented which has the same estimation performance as centralized Kalman filtering fusion.For the general systems with known auto-and cross-correlations of estimation errors from local sensors, in [6,[10][11][12], the optimal linear estimation fusion formulas were proposed in the sense of linear minimum variance (LMV).In practice, the crosscorrelations of estimation errors among the sensors may be completely or partially unknown.Some fusion methods have been developed for the systems with various uncertain correlations.The simple convex combination approach ignores the cross-correlations.The covariance intersection (CI) method linearly combines the local estimates and considers a conservative estimate of the estimation error covariance matrix (see [13][14][15][16]).An information-theoretic justification for CI was presented in [16] to minimize the Chernoff information.A robust estimation fuser was developed in [17] by finding the Chebyshev center, that is, center of the minimum radius ball enclosing the intersection of ellipsoid sets.For a class of estimation error covariance matrices with norm-bounded uncertainties, a robust fusion model to minimize the worstcase mean square error (MSE) was proposed in [18].In [19,20], a generalized convex combination method was discussed.Recently, by employing random matrix to describe the uncertainty of cross-correlation, a chance-constrained programming approach was presented in [21].In [22], a robust estimation problem was addressed for a linear model in which the unknown parameter vector is norm-bounded and noise covariance matrix is uncertain with a special structure.It was to get a linear estimator which minimizes the worst-case MSE over all vectors and noise covariance matrices in a specified uncertain region.
This paper considers the robust estimation fusion approach for distributed multisensor systems with uncertain correlations among local estimation errors.Assuming the true joint probability density of local estimators belonging to a neighborhood of the nominal model, the robust estimation fusion can be modeled as a minimax problem that minimizes the variance for the least favorable model in an uncertain class.In this paper, the neighborhood is specified by the Kullback-Leibler (KL) divergence.It is worth noting that the use of KL divergence is rather natural as a metric for model mismatch in the statistics and information societies.From an information geometric viewpoint, it was argued in [23] that the KL divergence is a natural geometric "distance" between systems.
The remainder of the paper is organized as follows.In Section 2, we formulate the robust distributed estimation fusion as a min-max optimization problem.Then, we convert it equivalently into a min-min optimization problem by Lagrangian duality method in Section 3.Moreover, in Section 4, we give the analytical solution of original minmax problem and conclude that the linear minimum variance unbiased estimation (LMVUE) fusion is robust for the least favorable statistical model.Section 5 gives a conclusion.
Notations R  and R × stand for the set of all dimensional real vectors and the set of all -by- real matrices, respectively.For a matrix ,   ,  † , and ‖‖ denote the transpose, Moore-Penrose generalized inverse, and spectral norm of , respectively. ≻ 0 means  is symmetric and positive definite .The symbol  represents the identity matrix with appropriate dimension.The expectation of a random variable (or random vector) is denoted by E[⋅].

Problem Formulation
Consider an -sensor distributed system in which  ∈ R  is the state or parameter to be estimated, and x (1) , . . ., x () are local unbiased estimates of  available at fusion center.The relationship between the local unbiased estimates and the state or parameter can be formulated as follows: where . . .
is augmented local unbiased estimates and can be viewed as the observation at fusion center,  = [, . . ., ]  ∈ R × , and  ∈ R  is the augmented error vectors of all local estimates.It is clear that  has mean zero and covariance matrix  = (  (11) ⋅ ⋅ ⋅  (1) . . . . . .
) , where In order to find an unbiased linear estimation fusion x =  +   , where  and  are compatible vector and matrix, respectively, a necessary and sufficient condition is  = 0, and    = .Therefore, the fused estimate becomes x =   .
When the error covariance matrix  is accurately given, a general linear fusion result is presented in [6,11], which is optimal in the sense of LMV; that is, it has the LMV among all linear unbiased estimation fusion rules.In this case, the LMVUE fusion is uniquely given with probability one as follows: where  = − † is an orthogonal projector (i.e., symmetric and idempotent matrix).Moreover, if  is nonsingular, then the optimal estimation fusion becomes However, the estimation error covariance matrix cannot be accurately obtained in practice.Denote the actual and nominal error covariance matrices by  and V, respectively.For measurement model (1), we assume that the noise  has the -dimensional Gaussian distribution with zero mean.It is well known that the Gaussianity assumption is reasonable in many practical applications.The actual and nominal probability density functions of observation  are (, ) and (, V), respectively, where the matrices  and V are assumed to be invertible.Therefore, the probability density functions of the actual and nominal models are, respectively, given by In order to measure the "distance" between the two models, we adopt the KL divergence from the actual model to nominal model: Although it is not a conventional distance, since it is not symmetric and does not obey the triangle inequality, the KL divergence satisfies (, V) ≥ 0 with equality if and only if denote the set of all possible estimation error covariance matrices such that the KL divergence from corresponding model to actual model is less than or equal to  ≥ 0. Note that C is a convex set since (, V) is a convex function of .The minimax robust estimation fusion problem is to seek for an optimal weighting matrix  * to minimize the MSE in the worst case.It can be formulated as the following minimax problem: where the optimization problem ( 9) is equivalent to the following optimization problem:

Problem Conversion
Consider the inner maximization problem in (11); that is, max ≻0 tr (  ) To find the optimal solution, we construct the Lagrangian function where  ≥ 0 is the Lagrange multiplier associated with the inequality constraint.Let (, ) be the Lagrange dual function sup ≻0 (, , ) while the latter is finite.
Since the primal problem (12) involves the maximization of a linear function tr(  ) over a bounded convex set C, the optimal objective value is finite.In addition, there exists  = V ≻ 0 with (, V) = 0 < .Therefore, the Slater conditions are satisfied.From Proposition 6.4.3 in [25], strong Lagrangian duality holds.
From Lemma 1, the original min-max problem ( 9) or ( 11) is converted equivalently into the following min-min problem: Furthermore, from ( 21), we have Then,

Solution of the Robust Optimization Problem
The objective function (, ) of problem ( 23) is convex in (, ) since (, , ) is convex in (, ) and (, ) is obtained by maximizing (, , ) over  (see [25, Proposition 1.2.4c]).In addition, the set is convex because it is the intersection of a linear constraint and a semidefinite programming (SDP) constraint.Consequently, the problem ( 23) is a convex optimization problem.Next, we pursue the analytical solution of problem (23).
Lemma 2. If  ≻ 0 and  is an orthogonal projector with compatible dimensions, then Proof.Using the properties of Moore-Penrose generalized inverse (see, e.g., [26]), we have Equation ( 27) thus holds, and then (28) is direct.
Theorem 3. The optimal weighting matrix of the robust fusion problem (9) or (11) is given by Proof.The derivative of (, ) given by (25) From    =  and the result on derivatives of determinants of matrices (see, e.g., [24, p. 182]), the derivative of function (, ) with respect to  is The solution of the matrix equation    =  given by equality constraint in the optimization problem ( 23) can be expressed as where  = − † and  ∈ R × is any matrix.Next, we seek the matrix  to optimize the weighting matrix .Redenote the objective function (, ) given by (25) That is, Because V is invertible, from (27), the solution of matrix equation ( 42) is where  ∈ R × is any matrix.Substituting  * into (38) and using ( − ) = 0, we have From the convexity of problem (23) and Proposition 4.7.1 in [25], the pair ( * ,  * ) ∈ D is the global minimum of (23).Moreover, from the definition of  and (28), we can obtain As a result , by the invertibility of   V−1 , we have The proof of this theorem is completed.
Remark 4. The optimal weighting matrix  * given in Theorem 3 does not depend on the parameter .However, it is worth noting that the optimum value of the objective function is increasing as  increases.In fact, the optimal Lagrange multiplier  * depends on .
From Theorem 3, although the least favorable distribution given by covariance  * is not the nominal distribution, the robust estimation fusion formula ( * ) is the same as the fusion formulas given by (4) and ( 5) which are derived from the nominal distribution.It concludes that the usual LMVUE fusion for a Gaussian nominal model is robust.

Conclusion
A minimax optimization problem is formulated to find the robust estimation fusion for multisensor systems with uncertain correlations of local estimation errors characterized by the relative entropy.By rigorous mathematical deduction, it has been proved that the usual LMVUE fusion for a Gaussian nominal model is robust no matter how far the KL divergence is from the actual model to nominal model.It is an interesting and significant discovery.