MPE Mathematical Problems in Engineering 1563-5147 1024-123X Hindawi 10.1155/2018/8156390 8156390 Research Article Novel Two-Stage Method for Low-Order Polynomial Model Yan Cheng 1 2 http://orcid.org/0000-0002-3434-3373 Shen Xiuli 1 Guo Fushui 3 Castillo Carmen 1 School of Energy and Power Engineering Beihang University 100191 Beijing China buaa.edu.cn 2 Shen Yuan Honors College Beihang University 100191 Beijing China buaa.edu.cn 3 Research and Development Center AECC Commercial Aircraft Engine Co. Ltd. 200240 Shanghai China 2018 472018 2018 03 12 2017 09 05 2018 14 06 2018 472018 2018 Copyright © 2018 Cheng Yan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

One of the most popular statistical models is a low-order polynomial response surface model, i.e., a polynomial of first order or second order. These polynomials can be used for global metamodels in weakly nonlinear simulation to approximate their global tendency and local metamodels in response surface methodology (RSM), which has been studied in various applications in engineering design and analysis. The order of the selected polynomial determines the number of sampling points (input combinations) and the resulting accuracy (validity, adequacy). This paper derives a novel method to obtain an accurate high-order polynomial while requiring fewer sampling points. This method uses a two-stage procedure such that the second stage modifies the low-order polynomial estimated in the first stage; this second stage does not require new points. This paper evaluates the performance of the method numerically by using several test functions. These numerical results show that the proposed method can provide more accurate predictions than the traditional method.

1. Introduction

Metamodels are essentially the simple approximation functions of simulation models of real systems. The currently most common usage of metamodels is to reduce the computational cost significantly by substituting the time-consuming evaluations in some computationally intensive tasks . The second usage of metamodels is to smooth objective-function surfaces and deal with noisy data, which can facilitate the use of gradient-based methods for optimization problems [2, 3]. Another increasingly popular usage for metamodels is to deal with missing data and gain insight into the contributions of each input variable to associated output variables . Besides, metamodels can also be used to reduce numerical instability . Moreover, metamodels can be utilized as calibration methods for low-fidelity simulations of limited accuracy . Because of these benefits, metamodels have been extensively researched and employed in various applications including engineering design, analysis, and optimization .

Up to now, various types of metamodeling techniques have been proposed, such as radial basis function (RBF) , polynomial response surface , Kriging [13, 14], support vector regression (SVR) [15, 16], multivariate adaptive regression splines (MARS) [17, 18], and artificial neural networks (ANN) . Particularly, one of the most popular metamodeling techniques is the polynomial response surface, which was first proposed by Box and Wilson  and later discussed in detail by Myers, Montgomery, and Anderson-Cook . The polynomials have great advantages in efficiency of model construction as well as transparency, which means the functional relationships between the input variables and associated output variables can be easily obtained . These polynomials can be not only used for local metamodels in response surface methodology (RSM) but also used for global metamodels in weakly nonlinear simulation to approximate their global tendency. Therefore, they have been widely utilized in both academic research and engineering applications .

Generally speaking, the order of the selected polynomial has significant influences on the number of sampling points as well as the resulting accuracy (validity, adequacy). Namely, as the order of the polynomial increases, the polynomial response surface becomes more accurate in approximating higher nonlinear problems. However, the number of sampling points need to increase sharply, which may be impractical for these high-fidelity simulations. We should note that the mean squared error (MSE) may also increase.

This paper derives a novel method to obtain an accurate high-order polynomial while requiring fewer sampling points. The proposed approach is based on a two-stage procedure. The second stage modifies the low-order polynomial constructed in the first stage by utilizing its feedback. It is noted that the second stage does not require new sampling points.

The remaining sections of this paper are organized as follows. Firstly, we analyze the basic theory and characteristics of the polynomial response surface. Secondly, we introduce the modeling process of the improved polynomial response surface. Thirdly, we present the detailed scheme of the numerical experiments. Then, we analyze the results and discuss the performance of the proposed method. Finally, we conclude our paper with a summary and suggestions for future work.

2. Polynomial Response Surface

The polynomial response surface is mainly used to develop an approximate functional relationship between a number of input variables and an associated response. The relationship can usually be written as follows:(1)y=y^x+ϵ=zTβ+ϵwhere y denotes the true response, x=(x1,x2,,xk)T denotes the vector of input variables, y^(x) denotes the approximation response, ϵ denotes a random error, β=(β0,β1,,βp-1)T denotes a vector of p constant coefficients, and z=(1,z1,z2,,zp-1)T denotes a polynomial basis-function vector with p elements that consists of constant term, terms of x1,x2,,xk as well as cross-products of these terms up to a certain order.

The key step of the polynomial response surface is to estimate β by using the least-square method. In detail, an appropriate design of experiment (DOE) should first be chosen and a series of n(≥p) samples can therefore be obtained. The totality of these samples can be represented by a n×k design matrix, which is denoted by D as follows:(2)D=x1,x2,,xnTwhere xi=(xi,1,xi,2,,xi,k)T,(i=1,2,,n) denotes the ith sampling point.

Second, the values of the polynomial basis-function vectors z1,z2,,zn corresponding to all the sampling points x1,x2,,xn should be calculated. The totality of these vectors can be represented by a n×p matrix, which is denoted by Z as follows:(3)Z=z1,z2,,znT

Third, the true response yi should be observed or measured for each sampling point xi by conducting numerical simulations or physical experiments. The totality of these responses can be represented by a n×1 vector, which is denoted by y as follows:(4)y=y1,y2,,ynT

Then, from (1), we can have(5)yi=β0+β1zi,1+β2zi,2++βp-1zi,p-1+ϵii=1,2,,nwhere ϵi denotes the random error term at the ith sampling point. Equation (5) can be expressed in matrix form as follows:(6)y=Zβ+ϵwhere ϵ=(ϵ1,ϵ2,,ϵn)T.

Next, the sum of squared residuals (SSR) L should be calculated.(7)L=i=1nϵi2=ϵTϵ=y-ZβTy-Zβ

Besides, L need to be minimized; therefore the following equation must be satisfied:(8)Lβ=-2ZTy+2ZTZβ=0

Finally, β can be estimated from (8).(9)β=ZTZ-1ZTy

In this way, the polynomial response surface has been constructed.

3. Improved Polynomial Response Surface

The first-order and second-order polynomial models are the two most popular polynomial response surfaces. Although polynomial with order higher than two can also be employed, the number of sampling points needs to increase sharply with the order increasing. In order to overcome this difficulty, we propose an improved method. The core idea is to start with a low-order polynomial and refit it to obtain high-order polynomial in a second successive fitting by using the feedback of the initial simple fitting. No new sampling point is needed in the second fitting.

In detail, the improved method involves the following steps:

Choose an appropriate DOE and generate a series of samples, namely, the design matrix D. Further information about DOE can be found in a large number of references [12, 2630].

Conduct numerical simulations or physical experiments to observe or measure the true response vector y for all the sampling points obtained from step (1).

Construct the initial low-order polynomial response surface y^. The first-order and second-order polynomial models are selected in this paper and denoted by 1RS and 2RS, respectively.

Choose an appropriate method to modify the initial polynomial response surface y^, and construct the improved model y^imp. The improved polynomial response surfaces corresponding to the initial model 1RS and 2RS are denoted by 1IRS and 2IRS, respectively.

Use the improved model y^imp to predict the response.

3.1. Model Construction

This paper propose a method to correct the initial low-order polynomial and construct the corresponding high-order polynomial. The method can be expressed as follows:(10)y^imp=C0x,α+C1x,γy^C0x,α=α0+α1x1++αkxk=cTαC1x,γ=γ0+γ1x1++γkxk=cTγwhere α and γ are two vectors of k+1 constant coefficients, respectively, and c is a first-order polynomial basis-function vector with k+1 elements. They can be written as follows:(11)c=1x1xkTα=α0α1αkTγ=γ0γ1γkT

The idea of the method is to treat the response of the low-order polynomial as feedback and then multiply a linear regression model and add another different linear regression model. Essentially, a second-order polynomial can be obtained when applying the method to 1RS. A third-order polynomial can be obtained when applying the method to 2RS. We can see that 2RS has (k+1)(k+2)/2 different coefficients and the correction method has 2(k+1) different coefficients. Therefore, (k+1)(k+2)/2 sampling points are needed at least to obtain a second-order polynomial for the traditional method, while (k+1)(k+2)/2 (when k>1) sampling points are needed at least to obtain a third-order polynomial for the improved method. It is noted that (k+1)(k+2)(k+3)/6 sampling points are needed at least to obtain a third-order polynomial model for the traditional method. Obviously, the improved method can obtain high-order polynomial with fewer sampling points.

We begin to construct the improved model y^imp. It should be noted at first that there is no need to generate new sampling points because the design matrix D expressed in (2) as well as the true response vector y expressed in (4) can be used to construct not only the initial model y^ but also the improved model y^imp.

From (1) and (10), we can get(12)yi=y^imp,i+ϵimp,i=ciTα+ciTγy^i+ϵimp,ii=1,2,,nwhere y^imp,i denotes the response of the improved model at the ith sampling point, ϵimp,i denotes the error term of the improved model at the ith sampling point, and y^i denotes the response of the initial model at the i-th sampling point.

Equation (12) can be transformed as follows:(13)yi=uiTλ+ϵimp,ii=1,2,,nwhere(14)ui=1xi,1xi,ky^ixi,1y^ixi,ky^iTλ=α0α1αkγ0γ1γkT

The totality of these equations represented by (13) can also be expressed in matrix form as follows:(15)y=Uλ+ϵimpwhere(16)y=y1y2ynϵimp=ϵimp,1ϵimp,2ϵimp,nU=1x1,1x1,2x1,ky^1x1,1y^1x1,2y^1x1,ky^11x2,1x2,2x2,ky^2x2,1y^2x2,2y^2x2,ky^21xn,1xn,2xn,ky^nxn,1y^nxn,2y^nxn,ky^n

The least-square method is employed to estimate the values of λ. Similar to (9), we can get(17)λ=UTU-1UTy

In this way, the improved model has been constructed. It can be expressed as follows:(18)y^imp=uTλ

4. Numerical Experiments 4.1. Benchmark Problems for Global Performance

To test the global performance of the proposed method, we employ nine benchmark problems which are often used in relevant literature. The dimensions of these problems range from 2 to 20.

(1 ) 2-Variable Goldstein-Price Function. This 2-variable benchmark problem is taken from Acar . It can be written as(19)fx=1+x1+x2+1219-14x1+3x12-14x2+6x1x2+3x22×30+2x1-3x2218-32x1+12x12+48x2-36x1x2+27x22where x1[-2,2] and x2[-2,2].

(2 ) 2-Variable Branin-Hoo Function. This 2-variable benchmark problem is taken from Acar . It can be written as(20)fx=x2-5.1x124π2+5x1π-62+101-18πcosx1+10where x1[-5,15] and x2[-5,15].

(3 ) 3-Variable Perm Function. This 3-variable benchmark problem is taken from a website (http://www.sfu.ca/~ssurjano///perm0db.html). It can be written as(21)fx=i=13j=13j+2xji-1ji2where xj[0,1] for all j=1,2,3.

(4 ) 3-Variable Cubic-Polynomial Function. This 3-variable benchmark problem is a common function. It can be written as(22)fx=x1+x2+x33where x1[0,3], x2[0,3], and x3[0,3].

(5 ) 4-Variable Power-Sum Function. This benchmark problem is taken from a website (http://www.sfu.ca/~ssurjano///powersum.html). It can be written as(23)fx=i=1d-j=1dxji-bi2where xj[0,d], for all j=1,,d. The 4-variable model (d=4) of this problem is considered. And the function parameters b=(13,13,13,13).

(6 ) 4-Variable Hartmann Function. This 4-variable benchmark problem is taken from a website (http://www.sfu.ca/~ssurjano///hart4.html). It can be written as(24)fx=-i=14ciexp-j=14aijxj-pij2where xj[0,1] for all j=1,2,3,4, c=1.01.23.03.2T, and A and P are expressed as follows:(25)A=103.0173.50.0510170.13.03.51.710178.00.0510P=0.13120.16960.55690.1240.23290.41350.83070.37360.23480.14510.35220.28830.40470.88280.87320.5743

(7 ) 10-Variable Zakharov Function. This benchmark problem is taken from a website (http://www.sfu.ca/~ssurjano///zakharov.html). It can be written as(26)fx=i=1dxi2+i=1d0.5ixi2+i=1d0.5ixi4where xi[-5,10], for all i=1,,d. The 10-variable model (d=10) of this problem is considered.

(8 ) 15-Variable Dixon-Price Function. This benchmark problem is taken from Acar . It can be written as(27)fx=x1-12+i=2di2xi2-xi-12where xi[-10,10], for all i=1,,d. The 15-variable model (d=15) of this problem is considered.

(9 ) 20–Variable Welch et al. (1992) Function. This 20-variable benchmark problem is taken from a website (http://www.sfu.ca/~ssurjano///welchetal92.html). It can be written as(28)fx=5x121+x1+5x4-x202+x5+40x193-5x19+0.05x2+0.08x3-0.03x6+0.03x7-0.09x9-0.01x10-0.07x11+0.25x132-0.04x14+0.06x15-0.01x17-0.03x18where xi[-0.5,0.5], for all i=1,,20.

To facilitate the description, we use some simple marks to label these benchmark problems respectively. In detail, the 2-variable Goldstein-Price function is denoted by GP-2; the 2-variable Branin-Hoo function is denoted by BH-2; the 3-variable Perm function is denoted by PM-3; the 3-variable Cubic-Polynomial function is denoted by CP-3; the 4-variable Power-Sum function is denoted by PS-4; the 4-variable Hartmann function is denoted by HM-4; the 10-variable Zakharov function is denoted by ZH-10; the 15-variable Dixon-Price function is denoted by DP-15; and the 20–variable Welch et al. (1992) function is denoted by WE-20.

4.2. Benchmark Problems for Local Performance

To test the local performance of the proposed method, we employ three benchmark problems, which are polynomials of second-order, third-order, and fourth-order.

(1 ) Quadratic-Polynomial Function. This benchmark problem can be written as(29)fx=x1+x2+x32where x1[0,3], x2[0,3], and x3[0,3]. It is denoted by QP-3.

(2 ) Cubic-Polynomial Function. This benchmark problem is the same as CP-3, which is described in (22).

(3 ) Four-Order-Polynomial Function. This benchmark problem can be written as(30)fx=x1+x2+x34where x1[0,3], x2[0,3], and x3[0,3]. It is denoted by FP-3.

4.3. Numerical Procedure

When the low-order polynomials are used for local metamodels in response surface methodology (RSM), the resolution-III (R-III) designs, central composite designs (CCDs), and Box-Behnken designs are considered to be the most suitable DOE techniques. When the low-order polynomials are used for global metamodels in weakly nonlinear simulation to approximate its global tendency, the Latin hypercube sampling (LHS) technique is one of the most popular choices both in scientific research and engineering problems [21, 31, 33, 34]. LHS maximizes the minimum distance between the sampling points to obtain uniform designs; moreover its projections onto each variable axis give uniform points. This paper will first discuss the global performance of the improved method. Therefore, we first utilize MATLAB(2011) routine “lhsdesign” with “maximin” criterion to select the locations of the sampling points for all the benchmark problems discussed above.

The performance of metamodels may vary from DOE to DOE. To reduce the random effect, we select 1000 training sets and 1000 corresponding test sets for each benchmark problem. Particularly, for a specified training set, the number of points is chosen as triple the number of coefficients in a second-order polynomial model (namely, 3(k+1)(k+2)/2, k denotes the dimension of the benchmark problem), which refers to Jin, Chen, and Simpson . Meanwhile, 1000 test points are selected for a specified test set by using LHS. The performance of metamodels for each benchmark problem will be estimated by using the mean of the 1000 replicates.

In summary, the detailed information about the training and test data used for each benchmark problem are listed in Table 1.

Detailed information about the training and test data used for each benchmark problem.

Benchmark problems Number of variables Number of training points
2-variable Goldstein-Price 2 18
2-variable Branin-Hoo 2 18
3-variable Perm 3 30
3-variable Cubic-Polynomial 3 30
4-variable Power-Sum 4 45
4-variable Hartmann 4 45
10-variable Zakharov 10 198
15-variable Dixon-Price 15 408
20-variable Welch et al. (1992) 20 693
5. Results and Discussion 5.1. Global Performance of the Improved Method

Reviewing the relevant literature , we select root mean squared error at test points (RMSEtst) as validation metrics to measure the performance of the improved method for the benchmark problems. The definition of RMSEtst is expressed as follows:(31)RMSEtst=i=1ntstyi-y^i2ntstwhere ntst denotes the number of test points.

Our main concerns are the comparison between the traditional model and its corresponding improved model, namely, the comparison between 1RS and 1IRS, as well as the comparison between 2RS and 2IRS. Although we do not think it is necessary to compare 1RS with 2RS or to compare 1IRS with 2RS, we still want to present the fact that maybe 2RS has better performance than 1RS for all the benchmark problems, while the performance of 1IRS may be close to that of 2RS in some particular problems.

The RMSEtst of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) for different benchmark problems are shown in Figure 1. The boxplot provides a graphical depiction of how prediction errors vary over the range of 1000 training sets and test sets. The plot is composed of a box, an upper limit line with whiskers, a lower limit line also with whiskers and outliers. The box includes a top edge line representing the 75th percentile value, an interior line representing the median value, as well as a bottom edge line representing the 25th percentile value. The upper/lower limit line is extending from the top/bottom edge line of the box to the most extreme data points which are not considered to be outliers. The outliers are data with values beyond the limit line and are presented by the “+” symbols.

R M S E t s t of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) for different benchmark problems. (a) 2-variable Goldstein-Price function (GP-2), (b) 2-variable Branin-Hoo function (BH-2), (c) 3-variable Perm function (PM-3), (d) 3-variable Cubic-Polynomial function (CP-3), (e) 4-variable Power-Sum function (PS-4), (f) 4-variable Hartmann function (HM-4), (g) 10-variable Zakharov function (ZH-10), (h) 15-variable Dixon-Price function (DP-15), and (i) 20-variable Welch et al. (1992) function (WE-20).

From Figure 1 we can see the following. (1) When considered the median value of the 1000 test sets, 1IRS performs better than 1RS for seven benchmark problems. For the other two problems (HM-4 and DP-15), 1IRS has similar results with 1RS. It is noted that in this paper the results are called similar results if the difference is between -1% and 1%. (2) 2IRS has better accuracy than 2RS for seven benchmark problems. For the other two problems (DP-15 and WE-20), 2IRS has similar results with 2RS. (3) 2RS performs better than 1RS for all the benchmark problems, yet the accuracy of 1IRS is close to that of 2RS for GP-2 and PM-3. Particularly, the accuracy of 1IRS is even better than that of 2RS for CP-3 and ZH-10. (4) For BH-2, PM-3, and HM-4, 1IRS has larger outliers (or longer tails) than 1RS and 2IRS has larger outliers than 2RS. Therefore, the improved models have larger variance than the traditional models. The detailed mean and COV (coefficient of variation) values of RMSEtst over the 1000 test sets are shown in Appendix A.

Why Does 2IRS Perform Better Than 2RS and 1IRS Perform Better Than 1RS? We think the reason is that the improved method can obtain highly nonlinear terms with fewer sampling points when compared with the traditional method. For example, when the number of sampling points is less than (k+1)(k+2)(k+3)/6, it is impossible to obtain a third-order polynomial for the traditional method. However, for the improved method, a third-order polynomial can be obtained as long as the number of sampling points is more than (k+1)(k+2)/2. We should note that (k+1)(k+2)/2 is less than (k+1)(k+2)(k+3)/6 when k>1.

5.2. Effect of Validation Metrics

The choice of different validation metrics may influence the results. To reduce the source of uncertainty in the results as much as possible, we select another four commonly used validation metrics. They are root mean squared error at training points (RMSEtrn), correlation coefficient at test points (R), average absolute error at test points (AAE), and max absolute error at test points (MAE). The detailed definitions and results of these metrics are shown in Appendix B~E. Here we gather all the five validation metrics for all the nine benchmark problems and present them in Table 2.

Comparison for performance of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) among all the five different validation metrics and nine benchmark problems.

GP-2 BH-2 PM-3 CP-3 PS-4 HM-4 ZH-10 DP-15 WE-20 Total
1IRS > 1RS 5 5 5 5 5 2 5 2 5 39
IRS 1RS 1 2 3
IRS < 1RS 2 1 3

2IRS > 2RS 5 5 5 4 5 4 5 1 34
IRS 2RS 1 3 5 9
IRS < 2RS 1 1 2

From Table 2 we can see the following. (1) For GP-2, BH-2, PM-3, PS-4, and ZH-10, all the five metrics show that 1IRS is more accurate than 1RS and 2IRS is more accurate than 2RS. (2) For CP-3, all the five metrics show that 1IRS is more accurate than 1RS; meanwhile four metrics indicate that 2IRS is more accurate than 2RS. (3) For HM-4, two metrics show that 1RS is more accurate than 1IRS; meanwhile just one metric indicates that 2RS is more accurate than 2IRS. (4) For DP-15, only one metric shows that 1RS is more accurate than 1IRS; meanwhile just one metric indicates that 2RS is more accurate than 2IRS. (5) For WE-20, all the five metrics show that 1IRS is more accurate than 1RS; meanwhile all the five metrics show that 2IRS has similar accuracy with 2RS. (6) Among all the forty-five examples, thirty-nine examples show that 1IRS is more accurate than 1RS and three examples show that 1IRS has similar accuracy with 1RS; meanwhile thirty-four examples show that 2IRS is more accurate than 2RS, and nine examples show that 2IRS has similar accuracy with 2RS.

In summary, the choice of the validation metrics can slightly influence the results, but the conclusions obtained by the five metrics remain unchanged. The improved method performs better than the traditional method. Particularly, the least-square method, which is used to estimate the polynomial coefficients, implies that the most relevant metric is RMSE. And the test set (RMSEtst) is more relevant than the training set (RMSEtrn).

5.3. Effect of the Number of Sampling Points

The accuracy of metamodels may depend on DOE and vary from DOE to DOE. To reduce the random effect caused by DOE, we have selected 1000 different training sets for each benchmark problem. However, for each training set, the number of sampling points remains unchanged. Therefore, we still need to examine the effect of the number of sampling points on the performance of the improved method. Considering the length of our paper, we just select ZH-10 as the example problem, choose RMSEtst, R, and AAE as validation metrics, and make the comparison between 2RS and 2IRS.

Figure 2 shows the results with the number of sampling points varying between (k+1)(k+2)/2 and (k+1)(k+2)(k+3)/6. From it we can get the following findings:

With the number of sampling points increasing, both RMSEtst and AAE of 2RS and 2IRS are getting smaller and smaller. That is to say, the accuracy of 2RS and 2IRS increases continuously with the increase of the number of sampling points.

RMSEtst and AAE of 2IRS are always smaller than that of 2RS; meanwhile R of 2IRS are always bigger than that of 2RS. That is to say, 2IRS has an obvious accuracy improvement when compared to 2RS, even though the number of sampling points varies.

Effect of the number of sampling points on the performance of 2RS and 2IRS for 10-variable Zakharov function (ZH-10). (a) RMSEtst is selected as validation metric, (b) R is selected as validation metric, and (c) AAE is selected as validation metric.

5.4. Significance of Results

The results above have proven the effectiveness of the improved method to some extent. In order for the method to be better used by engineers, we will compare its performance with some other popular metamodels, which are Kriging with first-order polynomial regression function (KRG1), Kriging with second-order polynomial regression function (KRG2), radial based function with Gaussian-form basis (RBFG), and radial based function with multiquadric-form basis (RBFM). Considering Jin, Chen, and Simpson  have concluded that 2RS has special advantages in efficiency and transparency and some disadvantages in accuracy, we mainly focus on the accuracy improvement of the proposed method.

Figure 3 shows RMSEtst of 2RS, 2IRS, KRG1, KRG2, RBFG, and RBFM for different benchmark problems. The detailed results are shown in Appendix F. Considering the frequency of the accuracy ranking of all metamodels for the nine benchmark problems, we can see the following. (1) The frequency of 2RS that ranks 1st or 2nd is only one, while that of 2IRS is three, that of KRG1 is two, that of KRG2 is eight, that of RBFG is one, and that of RBFM is three. (2) The frequency of 2RS that ranks 5th or 6th is five, while that of 2IRS is one, that of KRG1 is three, that of KRG2 is zero, that of RBFG is eight, and that of RBFM is one. (3) Obviously, 2RS performs worse than KRG1, yet 2IRS performs better than KRG1. (4) 2RS performs worse than RBFM, yet 2IRS has similar performance with RBFM. (5) Both 2RS and 2IRS perform better than RBFG. (6) 2RS performs worse than KRG2 for all the nine benchmark problems, yet 2IRS performs better than KRG2 for PO-3 and ZH-10.

R M S E t s t of 2RS, 2IRS, KRG1, KRG2, RBFG, and RBFM for different benchmark problems. (a) 2-variable Goldstein-Price function (GP-2), (b) 2-variable Branin-Hoo function (BH-2), (c) 3-variable Perm function (PM-3), (d) 3-variable Cubic-Polynomial function (CP-3), (e) 4-variable Power-Sum function (PS-4), (f) 4-variable Hartmann function (HM-4), (g) 10-variable Zakharov function (ZH-10), (h) 15-variable Dixon-Price function (DP-15), and (i) 20-variable Welch et al. (1992) function (WE-20).

In summary, compared to the traditional method, the improved method retains the advantages in efficiency and transparency and possesses significant accuracy improvement.

5.5. Local Performance of the Improved Method

All the results above are mainly used to test the global performance of the improved method. Therefore, LHS is utilized to generate the sampling points. To test the performance of the improved method used for local metamodels in RSM, we should use simple benchmark problems (i.e., QP-3, CP-3, and FP-3) and select CCDs to generate corresponding training points. CCDs are considered to be one of the most suitable DOE techniques in response surface methodology . Considering the length of our paper, we just select RMSEtst, R, and AAE as validation metrics.

Figure 4 shows the results of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) for QP-3, CP-3, and FP-3. From it we can see the following. (1) For QP-3, 1IRS performs better than 1RS for all the validation metrics. Particularly, the errors of 1IRS, 2RS, and 2IRS are zero. This is because the function QP-3 can be fitted exactly by the three models. (2) For CP-3 and FP-3, 1IRS performs better than 1RS for all the validation metrics, while 2IRS also performs better than 2RS for all the validation metrics.

Comparison of the local performances between traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS). (a) QP-3 using RMSEtst as validation metric, (b) QP-3 using R as validation metric, (c) QP-3 using AAE as validation metric, (d) CP-3 using RMSEtst as validation metric, (e) CP-3 using R as validation metric, (f) CP-3 using AAE as validation metric, (g) FP-3 using RMSEtst as validation metric, (h) FP-3 using R as validation metric, and (i) FP-3 using AAE as validation metric.

In summary, when the polynomials are used for local metamodels in RSM, the improved method still performs better than the traditional method.

6. Conclusions

In this paper, we proposed a new method to obtain an accurate high-order polynomial while requiring fewer sampling points. The core idea of the method is to start with a low-order polynomial and refit it to obtain high-order polynomial in a second successive fitting by using the feedback of the initial simple fitting.

To test the global performance of the improved method, we employed nine example problems which are widely used as benchmark problems in relevant literature. As expected, the accuracy of the improved method is better than that of the traditional method. Analyzing the principle, we think the reason for the better performance of the improved method is that it can obtain highly nonlinear terms with fewer sampling points when compared with the traditional method.

To obtain general conclusions, we investigated the effects of validation metrics and the number of sampling points on the performance of the improved method. We found that the choice of the validation metrics and the number of sampling points can slightly influence the results, but the conclusions remain unchanged.

In order for the improved method to be better used, we compared its performance with KRG1, KRG2, RBFG, and RBFM. The results showed that the improved method retains the advantages in efficiency and transparency and possesses significant accuracy improvement when compared with the traditional polynomial response surface.

Moreover, we researched the performance of the improved method used for local metamodels in RSM. The results showed that the proposed method still performs better than the traditional method.

However, there is no single outstanding metamodel which works best for all tasks. Therefore, finding more accurate metamodels is still our future work.

Appendix A. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M172"><mml:mi>R</mml:mi><mml:mi>M</mml:mi><mml:mi>S</mml:mi><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi><mml:mi>s</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>

Table 3 shows the mean and COV values of RMSEtst of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems. The values inside parentheses are COV values of RMSEtst.

Mean and COV values of RMSEtst of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

1RS 1IRS 2RS 2IRS
GP-2 1.162E+05(0.086) 8.271E+04(0.177) 8.180E+04(0.160) 7.172E+04(0.225)
BH-2 6.611E+01(0.066) 6.352E+01(0.147) 5.862E+01(0.114) 5.106E+01(0.265)
PM-3 1.576E+01(0.060) 7.883E+00(0.416) 6.086E+00(0.118) 5.056E+00(0.175)
CP-3 4.097E+01(0.058) 7.056E+00(0.138) 7.322E+00(0.148) 1.113E+00(0.613)
PS-4 4.986E+04(0.056) 4.158E+04(0.075) 2.876E+04(0.106) 2.527E+04(0.123)
HM-4 8.414E-01(0.033) 8.479E-01(0.133) 6.623E-01(0.091) 6.484E-01(0.123)
ZH-10 1.048E+08(0.091) 5.103E+07(0.128) 5.655E+07(0.128) 2.022E+07(0.166)
DP-15 3.838E+05(0.022) 3.849E+05(0.025) 1.363E+05(0.035) 1.373E+05(0.035)
WE-20 1.365E+00(0.022) 1.297E+00(0.024) 9.506E-01(0.026) 9.505E-01(0.026)
B. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M176"><mml:mi>R</mml:mi><mml:mi>M</mml:mi><mml:mi>S</mml:mi><mml:msub><mml:mrow><mml:mi>E</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>

The definition of RMSEtrn is expressed as follows:(B.1)RMSEtrn=i=1ntrnyi-y^i2ntrnwhere ntrn denotes the number of training points.

Table 4 shows the mean values of RMSEtrn of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 training sets for nine benchmark problems.

Mean values of RMSEtrn of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 training sets for nine benchmark problems.

1RS 1IRS Difference 2RS 2IRS Difference
GP-2 9.273E+04 5.052E+04 -45.5% 4.824E+04 2.928E+04 -39.3%
BH-2 5.491E+01 4.358E+01 -20.6% 3.651E+01 2.772E+01 -24.1%
PM-3 1.380E+01 6.063E+00 -56.1% 3.748E+00 2.612E+00 -30.3%
CP-3 3.580E+01 4.259E+00 -88.1% 4.073E+00 7.076E-01 -82.6%
PS-4 4.415E+04 3.098E+04 -29.8% 1.764E+04 1.433E+04 -18.8%
HM-4 7.802E-01 7.274E-01 -6.8% 4.509E-01 4.070E-01 -9.7%
ZH-10 9.675E+07 3.771E+07 -61.0% 3.375E+07 9.383E+06 -72.2%
DP-15 3.703E+05 3.561E+05 -3.8% 8.920E+04 8.734E+04 -2.1%
WE-20 1.326E+00 1.221E+00 -7.9% 6.240E-01 6.205E-01 -0.6%
C. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M182"><mml:mrow><mml:mi>R</mml:mi></mml:mrow></mml:math></inline-formula>

The definition of R is expressed as follows:(C.1)Ry,y^=1/VVy-y¯y^-y^¯dvδyδy^where (C.2)y¯=i=1ntstyintstδy=i=1ntstyi-y¯2ntst1VVyy^dv=i=1ntstyiy^intst

Table 5 shows the mean values of R of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

Mean values of R of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

1RS 1IRS Difference 2RS 2IRS Difference
GP-2 4.505E-01 7.970E-01 76.9% 8.130E-01 8.564E-01 5.3%
BH-2 9.404E-02 3.448E-01 266.7% 5.141E-01 6.407E-01 24.6%
PM-3 2.100E-01 8.403E-01 300.2% 9.288E-01 9.479E-01 2.1%
CP-3 9.302E-01 9.981E-01 7.3% 9.979E-01 9.999E-01 0.2%
PS-4 7.574E-01 8.386E-01 10.7% 9.272E-01 9.438E-01 1.8%
HM-4 5.610E-01 5.750E-01 2.5% 7.811E-01 7.937E-01 1.6%
ZH-10 7.371E-01 9.463E-01 28.4% 9.339E-01 9.918E-01 6.2%
DP-15 2.467E-02 1.163E-01 371.4% 9.337E-01 9.327E-01 -0.1%
WE-20 7.660E-01 7.919E-01 3.4% 8.967E-01 8.967E-01 0.0%
D. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M188"><mml:mi>A</mml:mi><mml:mi>A</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula>

The definition of AAE is expressed as follows:(D.1)AAE=i=1ntstyi-y^intst

Table 6 shows the mean values of AAE of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

Mean values of AAE of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

1RS 1IRS Difference 2RS 2IRS Difference
GP-2 7.171E+04 4.979E+04 -30.6% 4.918E+04 3.980E+04 -19.1%
BH-2 4.895E+01 4.529E+01 -7.5% 4.144E+01 3.498E+01 -15.6%
PM-3 1.147E+01 5.799E+00 -49.4% 4.212E+00 3.454E+00 -18.0%
CP-3 2.999E+01 4.512E+00 -85.0% 4.762E+00 8.330E-01 -82.5%
PS-4 3.595E+04 2.879E+04 -19.9% 2.014E+04 1.678E+04 -16.7%
HM-4 6.576E-01 6.712E-01 2.1% 5.144E-01 4.918E-01 -4.4%
ZH-10 6.765E+07 3.113E+07 -54.0% 3.701E+07 1.156E+07 -68.8%
DP-15 3.076E+05 3.079E+05 0.1% 1.081E+05 1.089E+05 0.7%
WE-20 1.070E+00 1.012E+00 -5.4% 7.688E-01 7.686E-01 0.0%
E. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M193"><mml:mi>M</mml:mi><mml:mi>A</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula>

The definition of MAE is expressed as follows:(E.1)MAE=maxyi-y^ii=1,2,,ntst

Table 7 shows the mean values of MAE of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

Mean values of MAE of traditional models (1RS and 2RS) and their corresponding improved models (1IRS and 2IRS) over the 1000 test sets for nine benchmark problems.

1RS 1IRS Difference 2RS 2IRS Difference
GP-2 7.668E+05 5.407E+05 -29.5% 5.302E+05 4.735E+05 -10.7%
BH-2 3.584E+02 3.194E+02 -10.9% 2.780E+02 2.535E+02 -8.8%
PM-3 1.038E+02 5.065E+01 -51.2% 4.363E+01 3.352E+01 -23.2%
CP-3 2.319E+02 5.583E+01 -75.9% 5.588E+01 4.901E+00 -91.2%
PS-4 3.591E+05 2.688E+05 -25.2% 2.116E+05 1.823E+05 -13.8%
HM-4 2.800E+00 2.833E+00 1.2% 2.663E+00 2.928E+00 9.9%
ZH-10 1.093E+09 5.805E+08 -46.9% 5.760E+08 2.380E+08 -58.7%
DP-15 1.463E+06 1.483E+06 1.3% 4.993E+05 5.101E+05 2.2%
WE-20 5.205E+00 5.009E+00 -3.8% 3.304E+00 3.303E+00 0.0%
F. Significance

Table 8 shows the mean values of RMSEtst of 2RS, 2IRS, KRG1, KRG2, RBFG, and RBFM over the 1000 test sets for nine benchmark problems.

Mean values of RMSEtst of 2RS, 2IRS, KRG1, KRG2, RBFG, and RBFM over the 1000 test sets for nine benchmark problems.

2RS 2IRS KRG1 KRG2 RBF RBFM
GP-2 8.180E+04 7.172E+04 7.143E+04 6.551E+04 7.566E+04 6.165E+04
BH-2 5.862E+01 5.106E+01 3.168E+01 3.207E+01 8.063E+01 3.451E+01
PM-3 6.086E+00 5.056E+00 4.326E+00 3.730E+00 3.613E+00 2.666E+00
CP-3 7.322E+00 1.113E+00 6.729E+00 3.879E+00 4.927E+01 1.131E+01
PS-4 2.876E+04 2.527E+04 2.669E+04 2.411E+04 7.125E+04 2.533E+04
HM-4 6.623E-01 6.484E-01 4.182E-01 5.465E-01 7.118E-01 5.572E-01
ZH-10 5.655E+07 2.022E+07 8.331E+07 5.384E+07 1.738E+08 6.453E+07
DP-15 1.363E+05 1.373E+05 3.377E+05 1.361E+05 1.028E+06 1.804E+05
WE-20 9.506E-01 9.505E-01 1.335E+00 9.471E-01 1.206E+00 9.417E-01
Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Authors’ Contributions

Cheng Yan and Xiuli Shen conceived and designed the experiments; Fushui Guo performed the experiments; Cheng Yan analyzed the data; Fushui Guo contributed analysis tools; Cheng Yan wrote the paper.

Simpson T. Booker A. Ghosh D. Giunta A. Koch P. Yang R. Approximation methods in multidisciplinary analysis and optimization: a panel discussion Structural and Multidisciplinary Optimization 2004 27 5 10.1007/s00158-004-0389-9 Hemker T. Fowler K. R. Farthing M. . von Stryk O. A mixed-integer simulation-based optimization approach with surrogate functions in water resources management Optimization and Engineering. International Multidisciplinary Journal to Promote Optimization Theory & Applications in Engineering Sciences 2008 9 4 341 360 10.1007/s11081-008-9048-0 MR2447416 Kavetski D. Kuczera G. Model smoothing strategies to remove microscale discontinuities and spurious secondary optima im objective functions in hydrological calibration Water Resources Research 2007 43 3 2-s2.0-34247610333 Forrester A. Sobester A. Keane A. Engineering design via surrogate modelling: a practical guide 2008 New York, USA Wiley Doherty J. Christensen S. Use of paired simple and complex models to reduce predictive bias and quantify uncertainty Water Resources Research 2011 47 12 10.1029/2011WR010763 Umakant J. Sudhakar K. Mujumdar P. M. Rao C. R. Customized regression model for improving low fidelity analysis tool Proceedings of the 11th AIAA/ISSMO Multidisciplinary Analysis and Optimaztion Conference September 2006 USA 2470 2482 2-s2.0-33846482605 Viana F. A. C. Simpson T. W. Balabanov V. Toropov V. Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA Journal 2014 52 4 670 690 10.2514/1.J052375 2-s2.0-84896913144 Forrester A. I. J. Keane A. J. Recent advances in surrogate-based optimization Progress in Aerospace Sciences 2009 45 1–3 50 79 10.1016/j.paerosci.2008.11.001 2-s2.0-58549086381 Kleijnen J. P. C. Sargent R. G. A methodology for fitting and validating metamodels in simulation European Journal of Operational Research 2000 120 1 14 29 2-s2.0-0002337285 10.1016/S0377-2217(98)00392-0 Zbl0985.65007 Kleijnen J. P. Design and analysis of simulation experiments 2015 230 Second Springer, Cham International Series in Operations Research & Management Science 10.1007/978-3-319-18087-8 MR3242760 Fang K. T. Li R. Sudjianto A. Design and Modeling for Computer Experiments 2005 CRC Press Myers R. H. Montgomery D. C. Response Surface Methodology 1995 New York, NY, USA John Wiley & Sons, Inc. MR3497873 Zbl1161.62392 Toal D. J. Keane A. J. Non-stationary kriging for design optimization Engineering Optimization 2012 44 6 741 765 10.1080/0305215X.2011.607816 MR2928008 2-s2.0-84861413463 Kleijnen J. P. C. Kriging metamodeling in simulation: a review European Journal of Operational Research 2009 192 3 707 716 10.1016/j.ejor.2007.10.013 MR2457613 Zbl1157.90544 2-s2.0-51649116012 Clarke S. M. Griebsch J. H. Simpson T. W. Analysis of support vector regression for approximation of complex engineering analyses Journal of Mechanical Design 2005 127 6 1077 1087 2-s2.0-25144486629 10.1115/1.1897403 Yan C. Shen X. Guo F. An improved support vector regression using least squares method Structural and Multidisciplinary Optimization 2018 57 6 2431 2445 10.1007/s00158-017-1871-5 MR3808620 Crino S. Brown D. E. Global optimization with multivariate adaptive regression splines IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 2007 37 2 333 340 2-s2.0-34047153277 10.1109/TSMCB.2006.883430 17416161 Kooperberg Charles Multivariate adaptive regression splines The Annals of Statistics 1991 19 1 1 67 Eason J. Cremaschi S. Adaptive sequential sampling for surrogate model generation with artificial neural networks Computers & Chemical Engineering 2014 68 220 232 2-s2.0-84904622083 10.1016/j.compchemeng.2014.05.021 Box G. E. P. Wilson K. B. On the experimental attainment of optimum conditions Journal of the Royal Statistical Society: Series B (Statistical Methodology) 1951 13 1 1 45 MR0046009 Zbl0043.34402 Jin R. Chen W. Simpson T. Comparative studies of metamodeling techniques under multiple modeling criteria Proceedings of the 8th Symposium on Multidisciplinary Analysis and Optimization Long Beach,CA,U.S.A. 10.2514/6.2000-4801 González-Fernández C. Molinuevo-Salces B. García-González M. C. Evaluation of anaerobic codigestion of microalgal biomass and swine manure via response surface methodology Applied Energy 2011 88 10 3448 3453 2-s2.0-79957977782 10.1016/j.apenergy.2010.12.035 Yongjiang W. Zhong C. Jianwei M. Minger F. Xueqian W. Optimization of ultrasonic-assisted extraction process of Poria cocos polysaccharides by response surface methodology Carbohydrate Polymers 2009 77 4 713 717 2-s2.0-67349148126 10.1016/j.carbpol.2009.02.011 Zhang J. Fu D. Xu Y. Liu C. Optimization of parameters on photocatalytic degradation of chloramphenicol using tio2 as photocatalyst by response surface methodology Journal of Environmental Sciences 2010 22 8 1281 1289 10.1016/S1001-0742(09)60251-5 Wang G. G. Shan S. Review of metamodeling techniques in support of engineering design optimization Journal of Mechanical Design 2007 129 4 370 380 10.1115/1.2429697 2-s2.0-34248346299 Santner T. J. Williams B. J. Notz W. I. The Design And Analysis of Computer Experiments 2003 New York, NY, USA Springer 10.1007/978-1-4757-3799-8 MR2160708 Zbl1041.62068 Joseph V. R. Hung Y. Orthogonal-maximin Latin hypercube designs Statistica Sinica 2008 18 1 171 186 MR2416907 Zbl1137.62050 2-s2.0-43049116178 Park J.-S. Optimal Latin-hypercube designs for computer experiments Journal of Statistical Planning and Inference 1994 39 1 95 111 10.1016/0378-3758(94)90115-5 MR1266995 Zbl0803.62067 2-s2.0-38149146334 Kleijnen J. P. C. Sanchez S. M. Lucas T. W. Cioppa T. M. A users guide to the brave new world of designing simulation experiments INFORMS Journal on Computing 2003 17 3 Kleijnen J. P. C. An overview of the design and analysis of simulation experiments for sensitivity analysis European Journal of Operational Research 2005 164 2 287 300 2-s2.0-10444244845 10.1016/j.ejor.2004.02.005 Zbl1068.90104 Acar E. Various approaches for constructing an ensemble of metamodels using local measures Structural and Multidisciplinary Optimization 2010 42 6 879 896 2-s2.0-78049384518 10.1007/s00158-010-0520-z Acar E. Simultaneous optimization of shape parameters and weight factors in ensemble of radial basis functions Structural and Multidisciplinary Optimization 2014 49 6 969 978 2-s2.0-84901663867 10.1007/s00158-013-1028-0 Zhou X. Jiang T. Metamodel selection based on stepwise regression Structural and Multidisciplinary Optimization 2016 54 3 641 657 10.1007/s00158-016-1442-1 MR3529779 Goel T. Haftka R. T. Shyy W. Queipo N. V. Ensemble of surrogates Structural and Multidisciplinary Optimization 2007 33 3 199 216 2-s2.0-33846688018 10.1007/s00158-006-0051-9 Toal D. J. Keane A. J. Performance of an ensemble of ordinary, universal, non-stationary and limit Kriging predictors Structural and Multidisciplinary Optimization 2013 47 6 893 903 2-s2.0-84879111941 10.1007/s00158-012-0866-5 Zhou X. Ma Y. Tu Y. Feng Y. Ensemble of surrogates for dual response surface modeling in robust parameter design Quality and Reliability Engineering International 2013 29 2 173 197 2-s2.0-84874108748 10.1002/qre.1298 Zhou X. J. Ma Y. Z. Li X. F. Ensemble of surrogates with recursive arithmetic average Structural and Multidisciplinary Optimization 2011 44 5 651 671 2-s2.0-84855812994 10.1007/s00158-011-0655-6 Acar E. Rais-Rohani M. Ensemble of metamodels with optimized weight factors Structural and Multidisciplinary Optimization 2009 37 3 279 294 2-s2.0-56449097052 10.1007/s00158-008-0230-y Acar E. Effect of error metrics on optimum weight factor selection for ensemble of metamodels Expert Systems with Applications 2015 42 5 2703 2709 2-s2.0-84918840902 10.1016/j.eswa.2014.11.020 Goel T. Hafkta R. T. Shyy W. Comparing error estimation measures for polynomial and kriging approximation of noise-free functions Structural and Multidisciplinary Optimization 2009 38 5 429 442 2-s2.0-67349175273 10.1007/s00158-008-0290-z