Covariance-Based Estimation from Multisensor Delayed Measurements with Random Parameter Matrices and Correlated Noises

The optimal least-squares linear estimation problem is addressed for a class of discrete-time multisensor linear stochastic systems subject to randomly delayed measurements with different delay rates. For each sensor, a different binary sequence is used to model the delay process. The measured outputs are perturbed by both random parameter matrices and one-step autocorrelated and cross correlated noises. Using an innovation approach, computationally simple recursive algorithms are obtained for the prediction, filtering, and smoothing problems, without requiring full knowledge of the state-space model generating the signal process, but only the information provided by the delay probabilities and the mean and covariance functions of the processes (signal, random parameter matrices, and noises) involved in the observation model. The accuracy of the estimators is measured by their error covariance matrices, which allow us to analyze the estimator performance in a numerical simulation example that illustrates the feasibility of the proposed algorithms.


Introduction
In the past decades, the development of network technologies has promoted the study of the estimation problem in multisensor systems, where the observations provided by all the sensor networks are transmitted to a fusion center for being processed, thus obtaining the whole available information on the signal.This kind of systems with multiple sensors is becoming an interesting research topic due to its broad scope of application as they can provide more information than traditional communication systems with a single sensor.This form of transmission has several advantages, such as low cost or simple installation and maintenance; however, due to the imperfection of the communication channels, during the transmission process, there exist often random sensor delays and/or multiple packet dropouts.Standard observation models are not appropriate under these random uncertainties, and classical estimation algorithms, where the measurements generated by the system are available in real time, cannot be applied directly.Therefore, new algorithms are needed and, recently, the estimation problem in multisensor systems with some of the aforementioned random uncertainties has become a research topic of growing interest (see, e.g., [1][2][3][4][5][6] and references therein).
There are many current applications, for example, networked multiple sensor systems with measurement-based output feedback, where the measurements may be randomly delayed due to network congestion or random failures in the transmission mechanism.Several modifications of the standard estimation algorithms have been proposed to incorporate the effects of randomly delayed measurements, in both linear and nonlinear systems.Assuming full knowledge of the state-space model of the signal process to be estimated we can mention [7][8][9][10] and using covariance information, [11,12], among others.Although all papers above mentioned involve systems with randomly delayed sensors, their major handicap is that all the sensors are assumed to have the same delay characteristics.Nevertheless, such an assumption is not realistic in many practical situations, where the information is gathered by an array of heterogeneous sensors, and the delay probability at each individual sensor can be different from the others.In recent years, this approach has been generalized considering multiple delayed sensors with different delay characteristics (see, e.g., [13,14], using the state-space model, and [15,16], using covariance information).
Furthermore, in many sensor network applications the measured outputs present uncertainties which cannot be described only by the usual additive disturbances, and multiplicative noises must be included in the observation equations to model such uncertainties (see, e.g., [17,18]).Also, in the context of missing and fading measurements, the observation equations include multiplicative noises described by scalar random variables with arbitrary discrete probability distribution over the interval [0, 1] (see, e.g., [19][20][21]).The above systems are a special case of systems with random parameter matrices, which have important practical significance and arise in areas such as digital control of chemical processes, systems with human operators, economic systems, and stochastically sampled digital control systems [22].
In [22,23], the optimal linear filtering problem in systems with independent random state transition and measurement matrices is addressed by transforming the original system into one with deterministic parameter matrices and statedependent process and measurement noises, to which the Kalman filter is applied.Although in [22] the Kalman filter is applied without providing any theoretical justification, in [23] it is shown that, under mild conditions, the transformed system satisfies the Kalman filter requirements and, hence, optimal linear estimators are obtained for systems with independent random parameter matrices.In [24], systems with deterministic transition matrices and one-step correlated measurement matrices are considered, and the optimal recursive state estimation problem is addressed by converting the observation equation into one with deterministic measurement matrices and applying the optimal Kalman filter for the case of one-step correlated measurement noise.In the abovementioned papers, although the noises of the transformed system with deterministic matrices depend on the system state and therefore can be correlated, the original system noises are assumed to be independent white processes.This assumption can be restrictive in many real world problems in which correlation and cross-correlation of the noises may be present.For this reason, the estimation problem in systems with correlated and cross-correlated noises is becoming an active research topic (see [25][26][27][28][29] for systems with deterministic matrices and [30,31] for systems with random parameter matrices, among others).In [30] a locally optimal filter in the class of Kalman-type recursive filters is presented and, in [31], the optimal least-squares linear filter is derived.
Motivated by the above analysis, in this paper we address the signal estimation problem from measurements coming from multiple sensors which are randomly delayed by one sampling time with different delay characteristics, under the assumption that the measured outputs are perturbed by both random parameter matrices and one-step autocorrelated and cross-correlated observation noises.The main contributions of this paper can be highlighted as follows: (1) the observation model considers simultaneously random delayed measurements and both random parameter matrices and correlated noises (one-step autocorrelation and also one-step crosscorrelations between different sensor noises are considered) in the measured outputs; (2) optimal LS linear recursive filtering and smoothing algorithms are obtained without requiring signal augmentation approach thus avoiding the expensive computational cost; (3) the proposed algorithms are obtained without requiring full knowledge of the statespace model generating the signal process; and (4) the innovation technique is used, simplifying substantially the derivation of the algorithms since the innovation process is a white noise.
The rest of the paper is organized as follows.In Section 2, we present the delayed measurement model to be considered and the assumptions and properties under which the LS linear estimation problem is addressed.The innovation approach which, as mentioned above, yields straightforward derivation of the estimation algorithms is given in Section 3. The recursive filtering and smoothing algorithms are derived in Sections 4 and 5, respectively.In Section 6, the performance of the proposed filtering algorithms is illustrated by a numerical simulation example where the signal of a first-order autoregressive model is estimated from delayed observations coming from two sensors with different delay characteristics, considering two kinds of measured outputs with correlated noises.The paper concludes with some final comments in Section 7.
Notation.The notation used throughout the paper is standard.R  denotes the -dimensional Euclidean space and R × is the set of all  ×  real matrices.  and  −1 denote the transpose and inverse of a matrix , respectively.The shorthand Diag( 1 , . . .,   ) denotes a diagonal matrix whose diagonal entries are  1 , . . .,   . 1 = (1, . . ., 1)  denotes the all-ones vector and  the identity matrix.If the dimensions of matrices are not explicitly stated, they are assumed to be compatible with algebraic operations.The notation ∘ denotes the Hadamard product ([ ∘ ]  =     ). − represents the Kronecker delta function, which is equal to one if  =  and zero otherwise.Moreover, for arbitrary random vectors,  and , we will denote Cov where [⋅] stands for the mathematical expectation operator.

Problem Formulation
The aim of this paper is to find recursive algorithms for the optimal least-squares (LS) linear filtering and smoothing problems of an -dimensional discrete-time random signal   using measurements perturbed by random observation matrices and correlated additive noises, which are transmitted by multiple sensors where one-step random delays with different rates may occur during the transmission process.
The estimation problem is addressed under the assumption that the evolution model of the signal to be estimated is unknown and only information about its mean and covariance functions is available; this information is specified in the following assumption.Next, the observation model with one-step random delays is described and the assumptions under which the LS linear estimation problem will be addressed are presented.It is assumed that, at any sampling time, the outputs are transmitted from the  different sensors to a data processing center producing the signal estimation and, as a consequence of possible failures during the transmission process, onestep delays may occur randomly in the measurements used for estimation.These measurement delays are modelled by introducing different sequences of Bernoulli variables whose values, zero or one, indicate whether the current measurement is up-to-date or delayed, respectively.Specifically, assume that, at initial time  = 1, the actual outputs, ỹ 1 , are always available for the estimation but, at any time  > 1, the available measurements coming from each sensor may be randomly delayed by one sampling time according to different delay rates.Therefore, if {   ;  > 1},  = 1, 2, . . ., , denote sequences of Bernoulli random variables, the available measurements from the th sensor are described by

Delayed Observation
Remark 6. Model ( 2) is commonly used to describe measurements coming from multiple sensors which are onestep randomly delayed with different delay rates (see, e.g., [13] using the state-space model and [15] using covariance information).From ( 2) it is clear that if    = 1, which occurs with a certain probability    , then    = ỹ −1 and the measurement from the th sensor is delayed by one sampling period; otherwise,    = 0 and    = ỹ  , which means that the measurement is up-to-date with probability 1−   .Therefore, the variables {   ;  > 1} model the random delays of the th sensor and the following assumption is made.Note that this assumption is more general than that considered in [13,15] where the processes {   ;  > 1}, for  = 1, 2, . . ., , are assumed to be mutually independent.
Finally, the following independence hypothesis is also assumed.
Remark 9. From the above properties, the following ones, which will be frequently used in the derivation of the algorithms, are obtained.
(a) The covariances of vectors H   , with H =   −   , are given by This identity is easily obtained from the conditional expectation properties, using the independence of   and H , and Assumption 1: From ( 5), these matrices are known and their entries are given by (b) The covariances of the vectors     are given by (c) If  is a × random matrix independent of {Γ  ,  > 1}, the Hadamard product properties and (iii) lead to To simplify future formulas and expressions, the observation model (3) will be written equivalently as follows: Taking into account the model properties and those specified in Remark 9, the first-and second-order properties of the processes {  ;  ≥ 1} and {V  ;  ≥ 1} and, consequently, those of the observation process, {  ;  ≥ 1}, are easily obtained.They are established in the following lemmas.

Innovation Approach to the LS Linear Estimation Problem
To obtain a recursive algorithm for the LS linear estimator, ẑ/ , of the signal,   , based on the randomly delayed observations, { 1 , . . .,   }, an innovation approach will be used [32].This approach consists of transforming the observation process {  ;  ≥ 1} into an equivalent one (innovation process) of orthogonal vectors {  ;  ≥ 1}, defined by   =   − ŷ/−1 , where ŷ/−1 is the orthogonal projection of   into the linear space generated by { 1 , . . .,  −1 }.The orthogonality property of the new process allows us to simplify the estimators' expressions (which also simplifies the algorithms derivation) in comparison to those obtained when the estimators are expressed directly as linear combination of the observations.Specifically, if   denotes a random vector to be estimated, the LS linear estimator of   based on the observations { 1 , . . .,   } (which will be denoted as ŵ/ ) agrees with that based on the innovations { 1 , . . .,   } or, equivalently, with the orthogonal projection of   onto the linear space generated by { 1 , . . .,   }.Hence, Consequently, by denoting Π  = [     ], the following general expression for the LS linear estimators of   is obtained: 3.1.Innovation Process.As indicated above, the innovation at time  is defined as   =   − ŷ/−1 , where ŷ/−1 , the orthogonal projection of   onto the linear space generated by { 1 , . . .,  −1 }, is the LS one-stage linear predictor of   .From (10) and the orthogonal projection lemma, this estimator can be expressed by so we need the one-stage predictors x/−1 and V/−1 which, by using the general expression ( 16) for the LS linear estimators, are given by (1) From the independence property (iv), it is clear that and hence, for then, from (16) for ẑ/−2 and ẑ−1/−2 , we obtain (2) The uncorrelation of Ṽ and Ṽ−1 with Ṽ1 , . . ., Ṽ−3 and the independence property (iv) guarantee that and this last expectation is equal to [V  V  −2 ] from the uncorrelation of V  and  −2 ; hence, Now, from ( 21) and (22), by denoting  ,−1 = [    −1 ], it is immediately clear that the innovation at time  can be expressed as and, hence, its determination requires that of the linear signal predictors, ẑ/ ,  =  − 1,  − 2. The derivation of the linear predictors is analogous to that of the filter, ẑ/ , so both are obtained simultaneously in the following section.

Prediction and Filtering Recursive Algorithm
The following theorem presents a recursive algorithm for the signal LS linear predictor and filter based on the delayed observation model given in Section 2.
Theorem 13.The signal predictors ẑ/ ,  < , and the signal filter, ẑ/ , are obtained as ẑ/ =     ,  < ; ẑ/ =     , where the vectors   are recursively calculated from The matrix function  is given by with The innovation,   , satisfies where The innovation covariance matrix, Π  , is given by The matrices  H  and    are given in (5) and (13), respectively. , , for  = , −1, −2, are given in (12).Finally, the matrices    and    are defined by Proof.From the general expression ( 16), ẑ/ = ∑  =1 [     ] Π −1    , for  ≤ , and the coefficients  , = [     ],  ≤ , must be calculated in order to determine the predictors and filter of   .Using expression (23) for   , we obtain (a) On the one hand, from (10) and independence hypotheses, we have where Assumption 1 and expression (31) for    have been used.
To prove recursive expression (29) for ,  ≥ 2, we calculate both expectations as follows.
(1) From expression (10) for   , using the independence properties and and since it is clear that Finally, formula (30) for the innovation covariance matrices is obtained by writing , using the expression for the observation predictor, ŷ/−1 , and taking into account that [     ] =   and [     ] =   .

Filtering Error Covariance Matrices.
The performance of the LS estimators ẑ/ ,  ≤ , is measured by the covariance matrices of the estimation errors, Σ / = [(  − ẑ/ )(  − ẑ/ )  ].Since the error of a LS linear estimator is orthogonal to the estimator, using Assumption 1, these matrices are given by Then, by using expression (24) and taking into account that   = [     ], we obtain the following expressions for the prediction and filtering error covariance matrices: Note that the computation of the prediction and filtering error covariance matrices does not depend on the current set of observations, as it only needs the matrices   and   , which are known, and the matrices   , which are recursively calculated from (27); hence, the error covariance matrices provide a measure of the estimator performance even before we get any observed data.

Fixed-Point Smoothing Algorithm
In this section, we present a recursive algorithm for the LS linear fixed-point smoothers, ẑ/ ,  > , where  is fixed and recursions for increasing  are proposed.By starting from the general expression for the LS linear estimator of the signal, ẑ/ = ∑  =1  , Π −1    , where  , = [     ] and it is clear that the linear fixed-point smoothers, ẑ/ ,  > , can be recursively calculated as with the linear filter, ẑ/ , as initial condition.Hence, to calculate the fixed-point smoothing estimators, ẑ/ , for  >  ( fixed), we need a recursive relation in  for ,  > .On the one hand, as in the proof of Theorem 13, using (10) and taking into account the independence hypotheses, together with Assumption 1 and expression (31) for    , we have On the other hand, the expression of ŷ /−1 obtained from (28) for  =  yields Therefore, defining the function  , = [     ], the following expression holds: with initial conditions given by  ,−1 =    −1 and  , =     , from (36).Finally, we need a recursive expression for  , ,  >  − 2. Taking into account that, from the orthogonality property, , using (24), and that   = [     ], we have that  ,−1 =    −1 and  , =     .Now, using (25) for   , the following formula is immediately deduced: Summarizing these results, the following recursive fixedpoint smoothing algorithm is obtained.
Theorem 14.The fixed-point smoother ẑ/ , with,  > , of the signal   is calculated as with initial condition given by the filter, ẑ/ , and with  ,−1 =    −1 and  , =     .The matrices  , satisfy the following recursive formula: The filter ẑ/ , the matrices    ,  ,−1 , and   , and the innovations ]  and their covariance matrices Π  are obtained from the linear filtering algorithm given in Theorem 13.

Numerical Simulation Example
In this section, the applicability of the proposed prediction, filtering, and fixed-point smoothing algorithms is shown by a numerical simulation example with two kinds of measured outputs.For this purpose, the signal values and their observations have been simulated in MATLAB and the signal estimates have been calculated, as well as the corresponding error variances, which provide a measure of the estimation accuracy.
Measurements coming from two sensors are considered and, according to the proposed observation model, it is assumed that, at any sampling time  ≥ 2, the measured output from the th sensor, ỹ  , can be randomly delayed by one sampling period during network transmission; that is, where {   ;  > 1},  = 1, 2, are independent sequences of independent Bernoulli random variables with [ 1  = 1] =   , ∀ > 1.
Firstly, to compare the performance of the predictor, ẑ/−1 , filter, ẑ/ , and fixed-point smoothers, ẑ/ , with  =  + 1,  + 2,  + 3, the corresponding error variances are calculated considering constant delay probabilities,  1 = 0.1 and  2 = 0.3.The results are displayed in Figure 1 which shows that the error variances corresponding to the fixedpoint smoother are less than those of the filter and the filtering error variances are smaller than the prediction ones, thus confirming that the smoother has the best performance while the predictor has the worst performance.This figure also shows that the performance of the fixed-point smoothers improves as the number of available observations increases.Analogous results are obtained for other values of the probabilities   ,  = 1, 2.
Next, we study the filtering error variances, Σ / , when the delay probabilities  1 and  2 are varied from 0.1 to 0.9.In all the cases, the filtering error variances present insignificant variation from the 10th iteration onwards and, consequently, only the error variances at a specific iteration are shown here.From these figures it is concluded that the performance of the filter improves as the delay probabilities,   ,  = 1, 2, decrease.Consequently, more accurate estimations are obtained as   comes nearer to 0, a case in which all the observations arrive on time.
Case 2 (systems with missing measurements).As in [28], consider missing measurements from two sensors, with different missing characteristics and noise correlation: For different values of the missing probability  and the delay probabilities  1 and  2 , a comparative analysis, similar to that carried out in Case 1, based on the estimation error variances of the predictor, filter, and smoother was performed.For all values, the results were similar to those given in Figure 1, showing that the fixed-point smoothing error variances are less than the filtering ones which, in turn, are smaller than the prediction error variances, thus confirming the comments on Figure 1.
Next, considering a fixed value of , namely,  = 0.5, the filtering error variances have been calculated for different values of the delay probabilities  1 and  2 .Specifically, the values  1 = 0.1, 0.3, 0.4 and  2 = 0.1, 0.3, 0.4, 0.5 have been used.The results are given in Figure 3 which shows that, as the delay probability  1 or  2 increases, the filtering error variances become greater and, consequently, worse estimations are obtained.Also, a similar study to that performed in Figure 2 has been carried out in this case; specifically, for fixed values of  and fixed delay probability in one of the sensors, the filtering error variances have been analyzed for different delay probabilities in the other sensor.The results are omitted as they are completely analogous to those displayed in Figure 2.
Also, to analyze the performance of the proposed estimators versus the probability  that the signal is present in the measurements of the second sensor, the filtering error variances have been calculated for  1 = 0.1,  2 = 0.3, and  varying from  = 0.3 to  = 0.8.The results are displayed in Figure 4; this figure shows that, as  increases, the filtering error variances become smaller and, hence, better estimations are obtained.Analogous conclusions are deduced for other values of  1 ,  2 , and .
Finally, we present a comparative analysis of the proposed filter and the following filters: (a) the suboptimal Kalman-type filter [13] for systems with uncorrelated white noises and one-step random delays, (b) the optimal linear filter based on covariance information [15] for the same class of systems considered in [13], (c) the centralized Kalman-type filter [26] for systems with correlated and cross-correlated noises, (d) the optimal centralized filter [28] for systems with missing measurements and correlated and crosscorrelated noises.

2
, where { ()  ; 1 ≤  ≤ 50} denote the th set of artificially simulated data and x() / is the filter at the sampling time  in the th simulation run.The results are displayed in Figure 5, which shows that (a) the proposed filtering algorithm provides better estimations than the other four filtering algorithms; (b) the performance of the optimal filter [15] is better than that of the suboptimal filter [13]; (c) the performance of the filters [13,15] is better than that of the filters [26,28] since these filters ignore any delay assumption; (d) the filtering algorithm in [26] provides the worst estimations as this filter considers correlated and cross-correlated noises, but neither missing observations nor delayed measurements are taken into account.

Conclusions
The optimal least-squares linear estimation problem from randomly delayed measurements has been investigated for discrete-time multisensor linear stochastic systems with both random parameter matrices and correlated noises in the measured outputs.The main contributions are summarized as follows.
(1) The current multisensor observation model considers simultaneously one-step random delayed measurements with different delay rates and both random parameter matrices and correlated noises in the measured outputs.This observation model covers those situations where the sensor noises are one-step autocorrelated and also one-step cross-correlations between different sensor noises are considered.This correlation assumption is valid in a wide spectrum of applications, for example, in target tracking systems where a target is observed by multiple sensors and all of them operate in the same noisy environment.
A similar study to that performed in this paper would allow us to generalize the current results by considering more general situations in which the signal and the observation noises are correlated.This extension would cover systems where the sensor and process noises are correlated and would constitute an interesting research topic.
(2) The random delay in each sensor is modelled by a sequence of independent Bernoulli random variables, whose parameters represent the delay probabilities.Another interesting future direction would be to complement the current study considering randomly delayed measurements correlated at consecutive sampling times, thus covering situations where two successive observations cannot be delayed.This kind of delay is usual in situations such as network congestion, random failures in the transmission mechanism, or data inaccessibility at certain times.
(3) Using covariance information, recursive optimal LS linear prediction, filtering, and smoothing algorithms, with a simple computational procedure, are derived by an innovation approach without requiring full knowledge of the state-space model generating the signal process.
(4) The applicability of the proposed algorithms is illustrated by a numerical simulation example, where a scalar state process generated by a first-order autoregressive model is estimated from delayed measurements coming from two sensors, in the following cases: (1) systems with observation multiplicative noises and (2) systems with missing measurements, both with correlated observation noises.
Assumption 1.The -dimensional signal process {  ;  ≥ 1} has zero mean and its autocovariance function is expressed in a separable form, [     ] =      ,  ≤ , where  and  are known  ×  matrix functions.