An Efficient Reliability Analysis Method Combining Improved EIF Active Learning Mechanism and Kriging Metamodel

Complex implicit performance functions widely exist in many engineering problems. The reliability analysis of these problems has always been a challenge. Using surrogate model instead of real performance function is one of the methods to solve this kind of problem. Kriging is one of the surrogate models with precise interpolation technique. In order to make the kriging model achieve higher accuracy using a small number of samples, i.e., improve its practicability and feasibility in practical engineering problems, some active learning equations are wildly studied. Expected improvement function (EIF) is one of them. However, the EIF has a great disadvantage in selecting the added sample point. Therefore, a joint active learning mechanism, J-EIF, is proposed to obtain the ideal added point. The J-EIF active learning mechanism combines the two active learning mechanisms and makes full use of the characters of kriging model. It overcomes the shortcoming of EIF active learning mechanism in the selection of added sample points. Then, using Monte Carlo Simulation (MCS) results as a reference, the reliability of two examples is estimated. The results are discussed showing that the learning efficiency and accuracy of the improved EIF are both higher than those of the traditional EIF.


Introduction
Reliability analysis plays an increasingly important role in practical engineering problems [1].However, most engineering problems are the complex implicit nonlinear problems.Therefore, the reliability analysis of these problems is a timeconsuming process, and sometimes it is unacceptable or impossible.Simultaneously, the application of auxiliary tools in the large-scale projects is more extensive because it is difficult or impossible to obtain the explicit performance function of the systems involving implicit performance functions.However, in most cases, many complex engineering problems may need a long time to complete one time deterministic analysis with those auxiliary tools, not to mention the reliability analysis, of which the repeated calls of the performance functions are necessary.Therefore, how to use these auxiliary tools to efficiently calculate the reliability of these kinds of implicit practical engineering problems is always a challenge for engineers.So far, an effective way to solve the reliability of these problems is to establish the input-output relationship of these implicit problems, i.e., surrogate model, to simulate its behavior [2][3][4][5].Then, combining the established surrogate model with the Monte Carlo Simulation (MCS) [6], the reliability of these complex problems can be efficiently calculated [7].
Several surrogate models, for instance, neural network [8][9][10], support vector machine [11][12][13], response surface [14][15][16], and kriging model [17][18][19][20], can be used to approximate the actual performance function for the implicit problems.Among them, kriging model has been received more attention in recent years due to its flexible application.Furthermore, the result of kriging's prediction at a new sample point can be considered as a normal random variable, of which the mean value and variance are the kriging predictive value and the so-called kriging variance, respectively.Kaymaz [21] combined the kriging method with the response surface method to estimate the reliability of the system.Sakata et al. [22] used the kriging model approximation to deal with the problems involving system optimization and obtained good results.
For improving the approximating ability of the kriging model, some elaborate active learning functions are constructed and employed to update the kriging model in an iterative way.An expected improvement function (EIF) [23] is one of the active learning functions and widely used to improve the accuracy of the kriging model.Hu [24] employed the EIF-kriging model to estimate the reliability of the timedependent problems.Yang [25] was inspired by the EIF active learning mechanism and then proposed a hybrid reliability analysis method considering both random and interval variables.After obtaining the high-precision kriging model, the reliability of the systems involving implicit problems can be easily estimated by the MCS method.Echard et al. [26] combined the kriging model and the MCS method and proposed an AK-MCS method; the high accuracy of the proposed method is demonstrated by several examples.Other studies on the kriging model can be found in [27][28][29][30].
For the EIF active learning mechanism, firstly it is needed to use the current best solution, usually the minimum value of the real system at current training sample points, to construct a learning function.Secondly, find the sample point that maximizes the EIF in a large number of samples; notice that this process is performed with the kriging model rather than real objective function.Finally, add this sample and its corresponding real function value to training samples to update the kriging model.Meanwhile, the above active learning will terminate when the iteration process satisfies a given convergence criterion.This paper studied the EIF active learning mechanism and found some shortcomings in the process of screening the added sample point.The improvement is made for EIF to optimize its learning process and then improve the prediction ability of the kriging model.In particular, a joint active learning mechanism, J-EIF, is proposed to obtain the ideal added point.The J-EIF active learning mechanism combines the two active learning mechanisms which have the same principle but different integral area.It overcomes the shortcoming of EIF active learning mechanism in the selection of added sample points.
It is well known that three key points should be considered in a surrogate model with high accuracy and efficiency.The first point is the accuracy of the metamodel itself, the second point is the performance of the active learning mechanisms, and the third is the selection of appropriate experimental design points.In this paper, the target of the research is to improve the performance of the active learning mechanisms to make the accuracy and efficiency of the surrogate model better, rather than the metamodel or experimental design points.
The remaining of this paper is organized as follows.A brief introduction of MCS procedure for reliability analysis and kriging metamodel is given in Section 2. In Section 3, a brief introduction and some discussion on the EIF are given and a J-EIF active learning mechanism is proposed in order to overcome the shortcoming of EIF active learning mechanism in the selection of added sample points.In Section 4, using the MCS procedure as the reference, the accuracy of the proposed method is fully illustrated with a numerical example in example 1.At the same time, its practicability and feasibility in practical engineering problems are demonstrated in example 2. The conclusions of this study are drawn in Section 5.

Fundamental Theory
2.1.MCS Procedure for Reliability Analysis.Suppose that  is an -dimension input variables vector and ŷ() is an approximate surrogate model of the implicit performance function().Then, the reliability   can be expressed as follows: where  is the given security threshold, which is used as a criterion for evaluating whether the system fails.If the response of the system exceeds the given security threshold, the system is invalid; otherwise, it is safe.Therefore, it plays an important role in reliability analysis.  denotes the dimension variable space,   () is the joint probability density function of , and   () is an indicator function and is given as follows: It can be seen from ( 1) that the reliability   is the expectation of the indicator function   ().Then, (1) can be rewritten as follows: where (•) denotes the expectation operator and  () represents the th input sample.

Kriging Metamodel.
Use  = [ 1 ,  2 , ⋅ ⋅ ⋅ ,   ]  and () to denote the input random vector and the output response, respectively.Then, the specific kriging model can be expressed as follows: where the first part (, ) is the parameter distribution of linear regression and it can be specifically expressed as   ();   () represents the global trend of the model, and it is usually a constant value because it does not play a decisive role for the accuracy of model fitting [26].In this paper, it is assumed that   () is equal to 1.  denotes the trend coefficients vector.() is a stationary Gaussian process with the following statistic characteristics: where Cov[•] denotes standard covariance operator.( 1 ,  2 ) represents the spatial correlation function of any two sample points in sample space.There are many forms for correlation function; the Gaussian correlation function is employed in this paper and is expressed as follows: where  is the number of design variables;    and    represent the th component of the vectors   and   , respectively; and   is a correlation parameter that reflects the correlation between the samples.
Given  group of experimental sample points  = [ 1 ,  2 , ⋅ ⋅ ⋅ ,   ]  and that the correspond function response vector is  = [ 1 ,  2 , ⋅ ⋅ ⋅ ,   ]  , then the unknown parameters β and σ2 can be estimated by the following equations, respectively: where  is an -dimensional unit column vector. is a correlation matrix containing the correlation between two experimental design samples; i.e.,  , = (  ,   ).
It can be seen from ( 6), (7), and ( 8) that both β and σ2 are the functions of   .Therefore,   needs to be estimated firstly.It can be obtained by optimizing a maximum the following equation: where | • | denotes the determinant value.Then, the liner unbiased prediction of an unknown sample vector   and the corresponding prediction variance can be expressed as follows, respectively: where  =    −1 (  ) − (  ) and   (  ) is an dimension column vector revealing the correlative relations between new sample vector   and experimental sample vector It can be expressed as follows: It can be seen clearly that the prediction of the kriging model at a new sample vector   can be considered as the mean value of a random variable following normal distribution with mean value ŷ(  ) and variance  2 ŷ(  ).

Improved EIF Learning Mechanism
It can be seen from the above analysis that a good kriging prediction model is a guarantee of reliability estimations.The EIF active learning mechanism is a function which can improve the prediction ability of the kriging model by adding new sample to training samples.Next, a brief introduction is given on the EIF.Then, some discussion on the EIF is given and a J-EIF active learning mechanism is proposed.

Review of EIF Active Learning Mechanism.
The EIF is an active learning function used to select the location at which the value of EIF is the largest.Then, the corresponding sample point and its objective function value should be added to the training sample points to improve the prediction ability of the kriging model.Its learning mechanism is given as follows [31]: where (•) is the expectation operator and  min is the minimum value chosen from the true objective function values at the training sample points.For simplicity, use ŷ to represent ŷ(  ).The above expectation can be solved by where  is a realization value of prediction ŷ and  ŷ is the probability density function of the prediction ŷ.The final analytical solution of the above integral equation is given as where Φ(•) is the standard normal cumulative distribution function and (•) is the standard normal probability density function.Then, the added point  *  can be calculated by This indicates that  *  is a sample point at which the EIF is the maximum.

Some Discussion and Improvement on the EIF.
For simplicity, a single variable function  = () is used in the following discussion.Assume that six experimental sample points  1 ,  2 ,  3 ,  4 ,  5 ,  6 are used to construct a kriging predictor for () as shown in Figure 1.It can be seen from Figure 1 that the standard error of the predictor at above six experimental samples does go to zero, which indicates that the variance  2 ŷ at these sample points is close to zero and the prediction ŷ is close to a determined value, i.e., the corresponding objective function value.Standard error of the kriging predictor at sample points   and   is large because there are no sample points around them.It can be found that the larger the standard error of the predictor, the greater the 'uncertainty' of the predicted.Simultaneously,

Kriging Predictor
Standard error of the predictor Objective function the corresponding PDFs at these sample points are more decentralized; i.e., the kriging variance is larger.
It can be seen from ( 14) that there is a relationship between the EIF learning mechanism and the shaded area shown in Figure 1.To illustrate this relationship intuitively, another form of ( 13) is derived, and the derivation process is as follows.
is used to denote a normal random variable with mean value  and standard deviation .By probabilistic knowledge it can be known that where (•) is the PDF of ; then Let us suppose that the minimum value of objective function  min is reached at the training point  3 .For a new point   , the following equation is established: where  2   denotes the 2nd shaded area of the PDF  ŷ(  ), and it is shown in Figure 1.From ( 15) ∼ (20), another form of the EIF active learning mechanism is obtained and expressed as follows: It can be seen from ( 21) that there is a relationship between the EIF value and the shaded area  2   ; the larger the area  2   , the greater the EIF value.It is worth noting that although  2   has a positive effect on EIF value, it is not determinative as the minimum objective function value,  min , the prediction ŷ, the predicted variance  2 ŷ , and the PDF value  ŷ( min ) at point   , all affecting the EIF value.
Similar to EIF active learning function, a new EIF active learning function is introduced and expressed as follows: where  max is the maximum value chosen from the true objective function values at the training sample points, and it is shown in Figure 1.Similarly, for a new sample point   , the value of the  max [ ŷ] is also the function of the shaded area  1   , and it is expressed as Comparing ( 21) and ( 23), it can be noticed that the equations are highly similar; however, the integral regions of the above two active learning mechanisms are completely different.For  min [ ŷ], the integral region is  2  1 , and for  max [ ŷ], its integral region is  1  1 .Nevertheless, they all represent the probability average of the functions max ( min − ŷ(  ), 0) and max ( ŷ(  ) −  max , 0) in their respective integration regions.
Suppose that   is the sample point at which ( 15) is maximum; i.e.,   = argmax( min [ ŷ]).Equation ( 22) is maximum at point   ; i.e.,   = argmax( max [ ŷ]).Obviously, it would be problematic if we just added  min -selected point   to the training point because it has some doubts about then the optimal added point is   rather than   .It is clear that the EIF active learning function has shortcoming which may miss the optimal candidate point; this disadvantage may add undesirable candidate points to the training points and lead to an increase in the number of iterations and a decrease in the prediction accuracy of the kriging model.
In view of the above problems, a joint active learning mechanism, J-EIF, is proposed to obtain the ideal added point.It is expressed as follows: which indicates that the ideal added point  *  is the inverse of the maximum of the two maximum values max( min [ ŷ]) and max( max [ ŷ]).It is clear that the J-EIF active learning mechanism combines the two active learning mechanisms which have the same principle but different integral areas.It is a simple improvement; however, it overcomes the shortcoming of EIF active learning mechanism in the selection of added sample points.
In the following sections, a numerical example is studied to illustrate the accuracy of the proposed J-EIF.The result of the MCS procedure with 10 6 samples is used as a reference.Then, a certain system in example 2 demonstrates the practicality and feasibility of kriging model which combined the J-EIF active learning mechanisms.The computational efficiency and accuracy of the J-EIF and the EIF active learning equation are also compared in example 2.

Examples and Discussions
Example 1 (a numerical example).In order to illustrate the accuracy of the proposed J-EIF, a numerical example is studied in this section with the reference results of MCS.Assume the performance function is In this example, variables  = [ 1 ,  2 ,  3 ] are normal distribution variables with means  = [ 1 ,  2 ,  3 ] = [1.5, 2.5, 2.25] and standard deviations  = , where  is the variation coefficient.The threshold is given as .In addition to the accuracy and the number of the function calls, the results under different variation coefficients or security thresholds can reflect the robustness of the proposed method.Therefore, comparing the results of calculations under multiple different variation coefficients or security thresholds can rule out the contingency caused by single variation coefficient or security threshold.The analysis results are listed in Tables 1 and 2.
It can be seen from Tables 1 and 2 that the results obtained by the proposed J-EIF match well with those obtained by MCS procedure.Simultaneously, the error of the proposed method is controlled within 0.1%, illustrating that the proposed method has high accuracy.In order to intuitively compare the results of the two methods, the reliability listed in Tables 1 and 2 is drawn in Figures 2 and 3, respectively.
It can be seen from Figure 2 that the reliability of this numerical example decreases with the increases of the variation coefficient .It is worth noting that the relationship between the reliability and the variation coefficient  is nonlinear rather than linear, and the larger the variation coefficient, the greater the reduction in reliability.Comparing Figures 2 and 3, it can be seen that the influence of variation coefficient  on reliability is opposite to the influence of threshold  on reliability.Reliability increases in logarithmic form with threshold  increase; however, the larger the threshold, the smaller the reliability.
Numerical example 1 fully demonstrates the high accuracy of the proposed J-EIF.Next, a certain system example is studied and used to verify the superiority of the proposed J-EIF method.Example 2 (a certain system).Assume the limit state function of a certain system is In (26),  = ( 1 ,  2 ,  3 ,  4 ,  5 ,  6 ,  7 ) is the input random vector obeying normal distribution, and the corresponding mean parameters are listed in Table 3.
According the above information, the reliability results of this example at different variation coefficient  are listed in Table 4.
The computational efficiency and accuracy of improved method J-EIF are estimated by two indices, i.e., the number of the function calls N call and the error e(%).The results show that, at any variation coefficient level, the above two indices of the improved method J-EIF are both smaller than that of the EIF.For example, it can be seen from the second column of Table 4 that, when fixing the variation coefficient  at 0.003, the errors of J-EIF and EIF are 0.01% and 0.03% and the numbers of function calls of J-EIF and EIF are 12+5 and 12+8, respectively.12 is the number of initial training sample  points; 5 and 8 are the numbers of added sample points by the J-EIF and the EIF, respectively.Apparently, the improved active learning mechanism, J-EIF, undoubtedly has greater advantages than EIF as the J-EIF uses smaller number of samples but has higher calculation accuracy.
In order to observe how reliability changes when variation coefficient  varies and more intuitively to see the accuracy of the J-EIF, the results in Table 4 can be reflected by Figure 4.It can been seen clearly from Figure 4 that there is a great gap between the reliability curve estimated by the kriging model using the EIF active learning mechanism and that estimated by the MCS procedure.However, the reliability result curves estimated by the kriging model using the J-EIF match well with that estimated by the MCS.
It can be seen from the above analysis that the improved J-EIF active learning mechanism performs well with the variety of variation coefficient.Its performance at different thresholds is listed in Table 5.
It can be seen from Table 5 that the largest error of J-EIF method is 0.02% and the smallest error of EIF is 0.06%.The mean errors of the J-EIF and the EIF are calculated as 0.0117% and 0.19%, respectively.This indicates that the kriging model enhanced by the improved J-EIF active learning mechanism has higher accuracy than that enhanced by the EIF active learning mechanism.In order to intuitively reflect the superiority of the J-EIF active learning mechanism, the results in Table 5 can be described by Figure 5.It can be intuitively seen from Figure 5 that the J-EIF reliability curve is almost coincident with the MCS reliability curve.Simultaneously, it can be found that there is a big gap between the EIF reliability curve and the MCS reliability curve.This indicates that the J-EIF active learning mechanism is more helpful to improve the accuracy of the kriging model than the EIF active learning mechanism.
The accuracy and the number of the function calls are two important indicators that reflect the performance of the proposed method.In this paper, the first example just reflects the accuracy of the proposed method, and the second example not only reflects the accuracy of the proposed method, but also illustrates the two important indicators in detail.It is worth noting that example 2 verifies the accuracy and the number of the function calls of the proposed method from different angles; i.e., the two important indicators of the proposed method are illustrated by the different variation coefficients or different thresholds.Although there are only two examples, the sufficient calculations have already illustrated the accuracy and the number of the function calls of the proposed method in detail.

Conclusions
The selection of a new added sample point by the EIF has some shortcomings; i.e., the added sample point selected by the EIF may not be the best solution.In view of the above problems, this paper proposes a J-EIF active learning function, which extends the unilateral EIF active learning mechanism to a bilateral active learning mechanism and makes full use of the characters of kriging model.The high accuracy of the proposed J-EIF can be seen from numerical example 1.In addition, the results show that the relationship between the reliability and variation coefficient or threshold is nonlinear rather than linear.To fully verify the superiority of the proposed method, a certain system is studied by the proposed J-EIF and original EIF method.The results show that the proposed J-EIF reaches the convergence criterion by using fewer samples than EIF, indicating that the J-EIF active learning function has a higher convergence speed than the EIF.Moreover, using MCS results as a reference, it can be found that the kriging model based on the J-EIF has higher accuracy than that based on the EIF.

Figure 1 :
Figure 1: Kriging predictor and its standard error.

Figure 4 :
Figure 4: The reliability at different levels of variation coefficient .

Table 1 :
Reliability changes when variation coefficient  varies, fixing  at 4.8.

Table 2 :
Reliability changes when threshold  varies, fixing  at 0.5.

Table 3 :
Random variables and their distribution parameters of a certain system.

Table 4 :
Reliability results at different levels of variation coefficient .

Table 5 :
Reliability results with the changes of threshold  when fixing the variation coefficient  at 0.045.