Best Linear Unbiased Prediction for Multifidelity Computer Experiments

Recently it becomes a growing trend to study complex systems which contain multiple computer codes with different levels of accuracy, and a number of hierarchical Gaussian process models are proposed to handle such multiple-fidelity codes. This paper derives the best linear unbiased prediction for three popular classes of multiple-level Gaussian process models. The predictors all have explicit expressions at each untried point. Empirical best linear unbiased predictors are also provided by plug-in methods with generalized maximum likelihood estimators of unknown parameters.

It is a growing trend to study the complex system which contains multifidelity computer codes with different levels of accuracy.For example, a Bayesian approach is described to predict and analyze complex computer codes which can be run at different levels of sophistication [2].A novel approach is taken to integrate data from approximate and detailed simulations to build a surrogate model that describes the relationship between output and input parameters [3].The Bayesian hierarchical Gaussian process (BHGP) models are introduced to integrate low-accuracy and high-accuracy [4].A class of nonstationary Gaussian process models are proposed to link the computer outputs of different mesh densities [5].However, there are few papers that provide BLUPs for multifidelity computer experiments.
The purpose of this article is to find BLUPs for multifidelity computer experiments.The structure of the article is as follows.In Section 2, BLUPs for two levels of accuracy [4] are discussed.In Section 3, BLUPs for general -level cases in autoregressive model described by Kennedy and O'Hagan [2] are illustrated.In Section 4, BLUPs for continuous level in nonstationary Gaussian process model described by Tuo, Wu, and Yu [5] are demonstrated.We present a real application in Section 5. Concluding remarks are given in Section 6.

BLUPs for Two-Level Cases
The two experiments considered in this section are named as the low-accuracy experiment (LE) and high-accuracy experiment (HE).Let GP(,  2 , ) denote the Gaussian process with mean , variance  2 , and correlation parameters .Let   = (x 1 , . . ., x  ) and  ℎ = (s 1 , . . ., s  ) denote the design set for the LE and HE, respectively.Following Qian and Wu [4], for any x  ∈   , the LE is described by where (⋅) ∼ GP(0,  2  ,  1 ) and f(x) = [ 1 (x), . . .,   (x)]  is a set of prespecified regression functions.For any s  ∈  ℎ , the HE can be described by where the scale changes from LE to HE (⋅) ∼ GP( 0 ,  2  ,  3 ), the location adjustment (⋅) ∼ GP( 0 ,  2  ,  2 ), the measurement error  ∼ (0,  2  ) and (⋅), (⋅), and  are jointly independent.Let Y  = (  (x 1 ), . . .,   (x  ))  and Y ℎ = (  (s 1 ), . . .,   (s  ))  . and Proof.The linear predictor Ŷℎ ( For any linear unbiased predictor (LUP) of where Thus the Lagrange multipliers can be used to solve BLUP corresponding to a 1 and a 2 that minimize (5) subject to The Lagrange function is Let the gradient with respect to a 1 , a 2 ,  1 ,  2 be zero, and we have where After a small amount of algebra, the a solution gives Thus the BLUP of  ℎ (x 0 ), the computer output at an untried point x 0 , is a  Y.
Note that the parameters  and  0 are often unknown in (10).We can use their estimators to replace them and get the empirical BLUP.For , we use its maximum likelihood estimate (MLE) based on the data To estimate  0 , we first predict the computer output at   = (s 1 , . . ., s  ), say Ŷ *  , based on the assumption  1 (⋅) ∼ GP(h  (⋅) 1 ,  2  1 ,  1 ) and training data ( 1 , Z 1 ).We then estimate by MLE based on the model Y ℎ (x  ) =  0 Ŷ (x  ) + (x  )+ (x  ).
The log likelihood, up to an additive constant, can be written as where and then the MLE of  0 is
The autoregressive model [2] can be described by where where the notation is given in the proof.
Proof.The linear predictor a  Z +  is unbiased for   (x 0 ) provided where ) ) , ) ) , For any LUP of Ẑ (x 0 ), say a  Z, the MSPE of a  Z is Let A  (  ,   ) denote the matrix of correlations between points in   and   ; A  (  ) is the abbreviation for A  (  ,   ).
The notation in ( 22) is as follows: The Lagrange function is where ) , . . .
) , Let the gradient with respect to a,  be zero, and we have After a small amount of algebra, the a solution gives Note that the parameters  −1 are often unknown in (27).We estimate them as follows: (a) Predict the computer output at 1 , based on the assumption  1 (⋅) ∼ GP(h  (⋅) 1 ,

Ẑ(𝑖)
−1 +   , the log likelihood (up to an additive constant) is The empirical BLUP can be obtained by using (29).

BLUPs for Continuous Levels
For continuous levels, we assume that as the tuning parameter  gets closer to zero, the output of the computer experiment gets closer to the truth-value.Model [5] is where (x) is a stationary Gaussian process with mean function and covariance function cov (x, ) is a nonstationary Gaussian process with mean structure and covariance structures are and  and  are mutually independent.It follows that the covariance function of  has the form Theorem 3. The BLUP of  0 = (x 0 , 0) based on training data W = ((x 1 ,  1 ), . . ., (x  ,   ))  is a  W, where where the notation is given in the proof.
Proof.Since the linear predictor a  W +  is unbiased for  0 , The Lagrange function is where Let the gradient with respect to a,  1 ,  2 be zero, and we have After a small amount of algebra, the a solution gives where , and k = (f  0 , 0)  .This completes the proof.

A Real Application
Casting is one of the most important manufacturing processes for making metal products.Nowadays computer simulations for casting processes are commonly used.In the project we consider a low-pressure die-casting process that produces a certain component of satellites and study how the process parameters influence the temperature difference at a key location on the casting product.In the simulations the input factors are four-process parameters shown in Table 1.We conduct the experiment with two levels of accuracy and specify the run sizes of the high-accuracy and low-accuracy simulations as 16 and 32, respectively.The design and its corresponding responses are showed in Table 2.We then use the BLUP (5) to predict the high-accuracy responses at the points where we have only low-accuracy responses.The results are also reported in Table 2; see asterisked values.It can be seen that the prediction results are satisfactory.

Concluding Remarks
We have provided computable BLUPs for three classes of Gaussian process models, which are commonly used in modeling multifidelity computer experiments.A real engineering application is presented to show the effectiveness of our method.An advantage of our method is its explicit forms.The predictors by our method are consistent (or close to) with Bayesian predictors in the literature.Furthermore, our method does not require nested structure of the design [6,7].A limitation of our method lies in that its expression involves some unknown parameters which we need to estimate in advance.The MLE method or modified MLE methods can be used to solve this problem.Our method to derive BLUPs can also be used for other related cases, including sequential settings [8], computer experiments with both qualitative and quantitative factors [9], and sensitivity analysis [10].

Table 1 :
Input factors in Section 5.