A Novel Bayesian Method for Calculating Circular Error Probability with Systematic-Biased Prior Information

Circular Error Probability (CEP) is defined as the radius of a circle where the probability of an impact point being inside is 50%, which is also widely used as a measure of the guidance weapon systems’ precision. In order to achieve a fusion of various test information, Bayesianmethods and improved Bayesianmethods have been extensively studied in calculating theCEP.Nevertheless, these methods could fail when there exists unknown systematic bias in the prior information. Therefore, a novel method called Bayesian estimation based on representative points (BERP) with an optimization procedure for determining the optimal number of representative points is proposed in this paper. Explicit theoretical analyses demonstrate that the BERP outperforms the classical Bayesian methods when fusing the slightly biased prior information and also give the bound of the systematic bias for stopping using the heavily biased prior information. Moreover, when the systematic bias is within the bound, simulation results indicate that our method is credible and outperforms the classical Bayesian method in calculating the CEP of guidance weapon systems.


Introduction
Performance evaluation of complex equipment, such as the guidance weapon systems, is very important before application.In the assessment of impact accuracy of the guidance weapon systems, CEP is the most commonly used as a measure, which can integrate the precision and dispersion to assess the impact accuracy [1][2][3].Usually, the tighter the pattern of impact point errors, the smaller the CEP we can get; i.e., there is higher impact accuracy of guidance weapon systems.Let (, ) be the impact point errors of the projectile, where  and  are downrange and cross-range misses, respectively; then the CEP can be defined as follows [4,5]: (, )  = 0.5 (1) It is shown in the above equation that the CEP can be calculated by numerical integrations when giving the probability density function (, ).In practice, (, ) are usually assumed to follow the bivariate normal distribution, i.e., (, ) ∼ (, Σ),where  and Σ are the mean and the covariance, respectively.Then the problem of calculating CEP rests on estimating  and Σ, whose accuracy will determine the precision of the CEP calculation.
In the performance evaluation of guidance weapon systems, the impact points used to calculate the CEP are collected from various tests.The realistic tests of guidance weapon systems are usually extremely expensive and timeconsuming, so generally the sample size of impact points is very small.Moreover, the calculation of CEP based on the small sized data would lead to unreliable results, so it is reasonable to introduce the prior information to refine the results.Therefore, Bayesian estimation is widely used to achieve the fusion of the prior information and the realistic test information [6,7] and hence to increase the reliability and the accuracy of the estimation of (, Σ), as well as the CEP.In fact, the most important prior information is provided by the substitute tests in the development processes.The performance evaluation is a sequential process, so the data collected from substitute tests are usually regarded as prior samples.In order to use the prior information more reasonably, several strategies are introduced in bringing the credibility of the prior information into Bayesian estimation, such as data compatibility tests, the information divergence, and the theory of fuzzy operators [8,9].In the CEP calculation, Huang comes up with a measure of credibility from physics resources of data [10], which has a good estimation accuracy if we have a clear understanding of the physical background.Duan suggested a method with all prior information normalized into one test sample [11], which can reduce the deviation when prior information got distorted, but more theoretical discussion is needed for this method when considering the information loss and fusion efficiency of the normalization.
However, the substitute tests for the guidance weapon systems may be systematically biased compared with realistic tests.Since the pattern of systematic bias is unknown, we cannot estimate the bias but can only give a rough range of it.Simulation results demonstrate that the systematically biased prior information will cause serious impact on the mean estimation  when applying classical Bayesian estimation directly.The estimation of the parameter Σ is slightly affected by the unknown systematic bias because of the same guidance system of these tests.The improved methods considering the normalization of prior information could reduce the estimate bias of , but this will cause much loss of prior information and inevitably leads to an unreliable calculation of CEP.One possible way to solve the problem is by choosing appropriate samples, such as representative points to generate new prior information.
Representative points (RPs), also known as Principal Points [12], are a group of points that could represent a distribution with the least mean squared error (MSE).RPs' theory is brought up in 1990s and now widely applied in clustering analyses [13], statistical simulations [14], image processing [15], and so on.In this paper, we resample samples from the systematically biased data to get the RPs which are regarded as the new prior information, expecting to reduce the estimate bias of  substantially.This new estimation method is called Bayesian estimation based on representative points (BERP).Meanwhile, we propose an optimization procedure which balances the effects of estimate bias and information loss to determine the optimal number of RPs.In addition, two theorems are proposed to prove that the estimate bias and MSE of  with RPs are smaller than those with the raw data.Furthermore, we also analyze the bound of the systematic bias for stopping using the heavily biased prior information.Within the bound, both the simulations and authentic experiments show that our BERP outperforms the classical Bayesian methods in estimating the parameters.In the performance evaluation of guidance weapon systems, via BERP it is better to choose RPs from the raw prior samples as new prior information when calculating the CEP.On the whole, by using the theory of RPs, our works enrich the Bayesian methods especially on the prior information fusion patterns theoretically and provide a possible way to solve the engineering problem in performance evaluation of guidance weapon systems.
The rest of this paper is organized as follows.Section 2 introduces the classical Bayesian estimation applied in CEP calculation.Section 3 describes some notations and preliminaries of RPs, the process of BERP, and the optimization procedure for determining the optimal number of RPs.In Section 4, we propose two theorems to compare the estimation performances by BERP and classical Bayesian estimation and analyze the bound of systematic bias for stopping using the biased prior information.Section 5 shows three numerical experiments about our new method.The conclusion is given in Section 6.

Bayesian Estimation in CEP Calculation
The parameters   ,   are standard deviations of impact point errors for downrange direction and cross-range direction, respectively;   ,   are means of impact point errors for each direction;  (0 ≤ || < 1) is the correlation coefficient of  and .In most cases,  and  are independent of each other.Nevertheless, even if  ̸ = 0, we can use the orthogonal transformation to achieve the decorrelation of , .So we assume that  = 0 in the rest of the paper.Under the assumption that  = 0, the CEP  satisfies the equation After estimating the parameters   ,   ,   , and   , the CEP  can be calculated by numerical integrations.More details about calculating CEP are given in [16].

Classical Bayesian Estimation.
As described in the Introduction, the performance evaluation of guidance weapon systems is a sequential process; the distribution parameters would change when fusing test data from different stage.The realistic tests are conducted to refine the previous evaluation results based on the substitute tests.Therefore, the data collected from substitute tests are usually regarded as prior samples.
Because the procedures of estimating the parameters (  ,   ) and (  ,   ) are similar, we take the downrange direction of impact point errors as an example to introduce the classical Bayesian estimation, where  follows the normal distribution (  ,   2 ).For convenience, we drop the subscripts of (  ,   2 ) as (,  2 ) and let  1 (,  2 ) be the joint prior distribution of (,  2 ).In Bayesian theory, the conjugate prior distributions of  and  2 are normal distribution ( 1 ,  2 / 1 ) and inverse Gamma distribution Mathematical Problems in Engineering 3 ( 1 ,  1 ), respectively.As for the distribution parameters  1 ,  1 ,  1 , and  1 , they are determined by (7).The probability density function of ( 1 ,  1 ) is where Γ(⋅) means the Gamma function.Moreover, the joint prior distribution  1 (,  2 ) is the normal-inverse Gamma distribution, so we have Suppose the prior samples for downrange direction are X (1) = { (1)  1 ,  (1)  2 ⋅ ⋅ ⋅ ,  (1)   1 },  1 is the size of the samples; let Then the estimates of the parameters of the joint prior distribution are Similarly, when the realistic test samples X (2) = { (2)  1 ,  (2)  2 , ⋅ ⋅ ⋅ ,  (2)   2 } are obtained,  2 is the size of the samples; let Because of the property of conjugate prior distribution, the posterior distribution of (,  2 ) is also a normal-inverse Gamma distribution: where  2 ,  2 ,  2 , and  2 are the parameters of the normalinverse Gamma distribution, and the estimates of the parameters are So the estimates of (,  2 ) by classical Bayesian estimation are As for the cross-range direction of impact point errors, the estimates of   and   can also be calculated by (11).After obtaining the estimates of   ,   ,   , and   , we can calculate the CEP of impact points by (3).

Bayesian Estimation Based on Representative Points
In this section, we propose the novel method Bayesian estimation based on representative points and the procedure of determining the optimal number of RPs.RPs can not only optimally represent the distribution of prior information in terms of MSE principle, but also have smaller sample size compared with raw prior samples.Therefore, RPs can retain the useful information of prior samples and reduce the estimate bias of   ,   .If we search RPs as new prior information, we may get more accurate and reliable CEP of guidance weapon systems.

Methods for Searching Representative Points.
In this subsection, we will give a brief introduction of RPs and methods for searching RPs.Assume that X ∈   is a  dimensional random vector, and the probability density function of X is ().Define the mean squared error for a set of points { 1 ,  2 , ⋅ ⋅ ⋅ ,   } ⊆   of the random vector X as follows: where ‖ ⋅ ‖ stands for  2 -norm.The vectors { 1 , ⋅ ⋅ ⋅   } ∈   are called  representative points of a random vector X if for all sets { 1 ,  2 , ⋅ ⋅ ⋅ ,   } ⊆   .
From the definition above, it is obvious that when  = 1, the single RP is the mean of X. Searching the RPs equals doing the optimal grouping [17]; it is difficult to derive the concrete RPs theoretically even if the number of RPs is given.Flury has proved that there is no theoretical derivation of RPs when  > 2 [18].So some approximation algorithms have been proposed to search RPs including k-means methods [19], parametric k-means methods [20], and nonparametric methods [21].The k-means methods are searching the clustering centers as the RPs.Moreover, the parametric k-means methods resample large samples from a specific distribution whose parameters are estimated by maximum likelihood, then searching the RPs from the resampled samples by k-means methods.The main idea of nonparametric methods is to build the empirical distribution function of X and resample large samples from this empirical function, after which the RPs are chosen from the resampled samples.In most cases, nonparametric methods have better performance to represent a distribution in terms of MSE than the k-means methods and parametric k-means methods.The main steps of the nonparametric method introduced in [21] are shown as follows: use k-means algorithm to obtain  points P (1) = { (1)  1 ,  (1)  2 , ⋅ ⋅ ⋅ ,  (1)   } from X (0) as an initial solution.(ii) Step2.Use the kernel estimation method to estimate the density function of original samples, denoted as f().(iv) Step4.Based on the  training data Z and the  starting points P (1) , use k-means algorithm to obtain  RPs P (2) = { (2)  1 ,  (2)  2 ⋅ ⋅ ⋅ ,  (2)   }.
Following the four steps above, we can get the RPs from prior samples.After that, we can use the RPs as new prior information to estimate the population parameters.More details about BERP will be introduced in the next subsection.

The Procedure of BERP.
In this subsection, we will introduce the procedure of BERP which is similar to the classical Bayesian estimation in Section 2.2.We also take the downrange direction of impact point errors as an example, suppose the original prior samples are X (1) = { (1)  1 ,  (1)  2 , ⋅ ⋅ ⋅ ,  (1)   1 }.Choose   RPs from X (1) by the nonparametric method introduced in Section 3.1 and denote X () = { }, and let Similar to Section 2.2, let   ,   ,   , and   be the parameters of the joint prior distribution.By searching RPs as new prior information, the estimates of these parameters are Moreover, let  2 ,  2 ,  2 , and  2 be the parameters of posterior distribution.So the estimates of these parameters are where  (2) and  2 2 are calculated by (8).So the estimates of (,  2 ) by BERP are If we know the exact number of RPs, it is easy to estimate the parameters   ,   ,   , and   by (17).However, it is difficult to determine the optimal number of RPs because there are no theoretical methods about this.Therefore, we propose an optimization procedure to determine the optimal number of RPs when there exists unknown systematic bias in prior samples.

Optimal Number of Representative Points.
The approach to determining the optimal number of RPs varies with the background of practical problem.On the one hand, the RPs are closer to the original prior samples as the number grows.Therefore, the more the RPs chosen, the larger bias they may bring to the estimate of , which will also be validated in Theorem 1, Section 4. On the other hand, similar to the methods of normalizing prior information, choosing small number of RPs will cause great information loss.Therefore, we consider two factors when determining the optimal number of RPs: the estimate bias and the information loss.We still take the downrange direction of the impact point errors as an example to describe the optimization procedure.   and    stand for the estimate bias and the information In fact, it is hard to quantify the estimate bias    without the true values of the population parameters.But there is a principle in the performance evaluation of guidance weapon systems that every realistic test sample should be used.So we can use the mean of realistic test samples to approximate the true value of parameter , denoted as  true .The estimate of  by BERP is denoted as  BERP .So the approximated    is As for    , the information loss could be estimated by Cox (1957) [17]: where  2 is the variance of all prior samples;  2  is the variance of samples in  ℎ class which is classified in Step 4 for searching RPs.As the number of RPs increases, the estimate bias would increase and the information loss would decrease.The optimal number of RPs is determined by the minimal value of the objective function Algorithm 1 describes the optimization procedure of determining the optimal number of RPs.

Theoretical Analysis of BERP
Estimate bias and MSE are the common measures to evaluate the quality of an estimator.So we will use them to analyse the theoretical performance of BERP and classical Bayesian estimation in this section.Suppose the posterior distribution of parameter  is ( | X),  is the posterior expectation of , and  true is the true value of .The estimate bias and MSE of the estimator θ are where Var( θ | ) is the variance of θ.
Let realistic test samples follow the normal distribution (,  2 ), and the prior samples follow the normal distribution ( + ,  2 ), where  is the systematic bias.Let  1 ,  2 be the sample sizes of prior samples and realistic test samples, respectively.Suppose the posterior estimate of  by classical Bayesian estimation is μBayes ,  Bayes is its posterior expectation.The posterior estimate of  by BERP is μBERP ,  BERP is its posterior expectation, and   is the number of RPs.Let  true be the true value of parameter .There are two theorems to compare the estimate bias and MSE of the two estimators μBayes and μBERP .Theorem 1.In the case that there exists systematic bias in prior samples, one has Proof.From ( 11) and ( 17 where Because prior samples and realistic test samples both follow the normal distribution (,  2 ) and there exists systematic bias  in prior samples, we have the approximated results: Let  1 =  1 +  2 and   =   +  2 , and Δ 1 is determined as follows: when MSE( μBERP | ) ≤ MSE( μBayes | ), if and only if To sum up, when  2 ≥  2 / 2 , there is The estimate bias of μBERP is smaller than that of μBayes when there exists unknown systematic bias in prior samples.Moreover, the MSE of μBERP is also smaller than that of μBayes when  2 ≥  2 / 2 .In most cases,  2 / 2 is smaller than  2 .Therefore, it can be concluded that BERP has better accuracy for estimating the parameter  of normal distribution than classical Bayesian estimation when there exists unknown systematic bias in prior samples.
However, when the systematic bias is beyond a certain bound, it may be better to stop fusing the biased prior samples even if they may provide some useful information.Suppose μ is estimated by maximum likelihood estimation without using the prior information, it is obviously an unbiased estimate.The MSE of μ is Moreover, compared with (34), there is Δ 1 < Δ 2 , which means that when  2 ⩾  2 / 2 ⋅ Δ 2 , MSE( μMLE | ) is also smaller than MSE( μBayes | ).Therefore, if  2 is larger than  2 / 2 ⋅ Δ 2 , we should stop using the prior information; √ 2 / 2 ⋅ Δ 2 is the bound of the systematic bias.

Numerical Experiments
In this section, three numerical experiments are provided to validate that BERP can help to get more accurate and reliable calculation of CEP when the systematic bias is within the bound.The first one is to show the performance of optimization procedure when determining the optimal number of RPs; the second one is to compare the estimation accuracy of BERP with that of classical Bayesian estimation; the third one is to compare the calculation of CEP based on BERP with classical Bayesian estimation.
Example 1.In this example, we will analyze the optimization procedure for determining the optimal number of RPs.Let  1 = 200 be the sample size of the prior samples and  2 = 5 be the sample size of realistic test samples.The prior samples and the realistic test samples are generated from normal distributions ( + ,  2 ) and (,  2 ), respectively, where  = 0,  2 = 10,   the nonparametric method.In order to reduce the influence of random factors on simulation result, resample samples 100 times under the same circumstance and get the average results.Figure 1 shows the variation of the information loss, the estimate bias, and the objective function for different numbers of RPs, where the estimate bias and information loss are both calculated after normalization.
As displayed in Figure 1, the estimate bias approximated by (19) is very close to the theoretical one, which means that using    to quantify the estimate bias is reasonable.In addition, when the number of RPs increases, the information loss decreases while the estimate bias increases, so the objective function can balance the two factors well when determining the optimal number of RPs.From Figure 1(b), when the number of RPs is 9, the objective function reaches the minimal value, so the optimal number of RPs is 9 in this example.
Example 2. In this example, we will compare the estimation performance of the two methods.Let  1 = 200 be the sample size of the prior samples,  2 = 5 be the sample size of the realistic test samples, and  = 20 be the predetermined maximum number of RPs.The prior samples and realistic test samples are generated from normal distributions ( + ,  2 ) and (,  2 ), respectively, where  = 0 and  is the systematic bias in prior samples.We set different values for  in the simulations to investigate the estimation performance when estimating the parameter .Similar to Example 1, we resample samples 100 times under the same circumstance and get the average results in order to reduce the influence of random factors.Table 1 shows the simulation results.
From Table 1, we can summarize two conclusions.(1) Within the bound of the systematic bias shown in Section 4, if there exists slight systematic bias in prior samples, μBERP is much more closer to the true value of the parameter  than μBayes , and MSE( μBERP ) is also smaller than MSE( μBayes ) in most cases.Therefore, BERP has higher accuracy to estimate the parameter  than classical Bayesian estimation when there exists slight systematic bias in prior information.(2) If there is no systematic bias in prior samples, μBERP is very close to μBayes , and MSE( μBERP ) is a little larger than MSE( μBayes ).There is no obvious difference between the estimation accuracies of the two methods.Moreover, without the systematic bias in prior samples, the optimal number of RPs is close to the predetermined maximum number 20.In this case, BERP is degenerated into the classical Bayesian estimation to some extent.Example 3. When there is systematic bias in prior information, the CEP calculation based on BERP and classical Bayesian estimation is discussed in this example.We simulate  1 = 200 prior samples (X (1) , Z (1) )  from the bivariate normal distribution ( + , Σ) and  2 = 5 realistic test samples (X (2) , Z (2) )  from the bivariate normal distribution (, Σ), where  = ( √ 10, √ 10)  ,  = (0, 0)  , and Σ = diag(40, 40).The parameters   ,   ,   , and   are estimated by BERP and classical Bayesian estimation, respectively.Based on the estimates of the population parameters, we use the numerical integration to get the CEP. Figure 2 shows the simulation results about CEP calculation.
As shown in Figure 2, the true CEP is CEP true = 7.4466, the CEP calculation based on BERP is CEP BERP = 7.6532, and the CEP calculation based on classical Bayesian estimation is CEP Bayes = 8.4402.It is easy to conclude that the CEP calculation based on BERP is much closer to the true CEP when the systematic bias is within the bound.In addition, the CEP calculation without using the prior information is defined as CEP noprior .If the prior information were effectively ignored, the CEP noprior would be unreliable and unstable because of the large MSE of the estimation of  without  using prior information.Therefore, using BERP to estimate the parameters   ,   will get more accurate calculation of CEP than classical Bayesian estimation when there is slight systematic bias in prior information.In addition, CEP calculation by BERP also outperforms the method with the prior information ignored.

Conclusions
In this paper, we have investigated the methods for performance evaluation of guidance weapon systems.Because of the small sample size of the realistic test data, we would fuse the prior information in the evaluation.However, by classical Bayesian estimation, the unknown systematic bias in prior information may cause large deviation for CEP calculation.
For purpose of addressing it, a novel Bayesian method called BERP is proposed in this paper, and the corresponding optimization procedure is designed.In addition, we also give the bound of systematic bias for stopping using the heavily biased prior information.
Within the bound of the systematic bias, theoretical analysis and simulation results prove that our new method has smaller estimate bias and MSE for estimating the mean of normal distribution than classical Bayesian estimation when there exists slight systematic bias in prior information.As for CEP calculation, the simulation results also validate that the CEP calculated by BERP is more accurate and reliable than the CEP calculated by classical Bayesian estimation.It can be concluded that a more accurate and reliable estimation of the CEP can be obtained via the BERP when the unknown systematic bias is within the bound.
There is no obvious difference of the estimation accuracy between the two methods; BERP also has a good estimation performance when there is no systematic bias.Therefore, in order to get accurate and reliable evaluation results of guidance weapon systems, it is better to calculate the CEP via BERP than classical Bayesian estimation if the systematic bias is within the bound.In contrast, if the systematic bias is beyond the bound, we should stop fusing the biased prior information and evaluate the performance only by realistic test samples even if the sample size is small.

Figure 1 :
Figure 1: Performance for different representative points.
Define  : Predetermined maximum number of RPs   : The current number of RPs  *  : Optimal number of RPs    : Estimate bias    : Information loss for 1 ⩽   ⩽  do Choose   RPs from the prior samples and calculate    and    Update   =   + 1 , respectively, where   stands for the number of RPs.The objective function    is to achieve a balance between    and    .So we have    =    + loss ), the estimate bias of μBERP and μBayes is The estimate bias of   increases with the number of RPs.Since the size of prior samples is much larger than the size of RPs, we have  1 ≫   .Therefore there is 2= 50.The RPs are searched by

Table 1 :
Comparison of the simulation result of the two methods.