A Study on the Generalized Approximation Modeling Method Based on Fitting Sensitivity for Prediction of Engine Performance

. Prediction technology for aeroengine performance is significantly important in operational maintenance and safety engineering. In thepredictionofengineperformance,toaddressoverfittingandunderfittingproblemswiththeapproximationmodelingtechnique, wederivedageneralizedapproximationmodelthatcouldbeusedtoadjustfittingprecision.Approximationprecisionwascombined withfittingsensitivitytoallowthemodeltoobtainexcellentfittingaccuracyandgeneralizationperformance.TakingtheGrey model(GM)asanexample,wediscussedthemodelingapproachofthenovelGMbasedonfittingsensitivity,analyzedthesetting methodsandoptimizationrangeofmodelparameters,andsolvedthemodelbyusingageneticalgorithm.Byinvestigatingtheeffect ofeverymodelparameteronthepredictionprecisioninexperiments,wesummarizedthechangeregularitiesoftheroot-mean-squareerrors(RMSEs)varyingwiththemodelparametersinnovelGM.Also,byanalyzingthenovelANNandANNwithBayesian regularization,itisconcludedthatthegeneralizedapproximationmodelbasedonfittingsensitivitycanachieveareasonablefitting degreeandgeneralizationability.


Introduction
Prediction technology for aeroengine performance is significantly important in operational maintenance and safety engineering.In January 2008, Australia's Qantas Airlines Boeing 747-400 power system malfunctioned in mid-flight, and four engines were all unusable [1].In America, transport airplane accidents due to mechanical malfunctions were investigated, which covered 7,571 registered airplanes from 1980 to 2001.It is concluded that landing gear and turbine engine are most likely broken.Besides that, in commercial aircraft industry, $31B is spent worldwide on aircraft maintenance in 2007, but 31% of it is on engine maintenance [2].So, prediction technology for aeroengine performance can decrease the possibility of plane crash and save the cost of maintenance.
Aeroengine is complicated, nonlinear equipment, because it includes a large number of parts and there is a high degree of coupling.Also, performance parameters of aeroengine consist of many great discrete and nonlinear points.Consequently, this complexity brings about great difficulty to the development of prediction technology.
There are many methods to predict aeroengine performance.Approximate mathematical models based on datadriven approaches are acceptable substitutes for accurate physical models that are unlikely obtained in fields of multidisciplinary design optimization, prediction, and so on.Many approximation models with random and nonlinear features have been constructed, and they include the response surface model (RS) [3,4], polynomial regression model [5,6], autoregressive moving average (ARMA) [7,8], artificial neural network model (ANN) [9,10], support vector machines (SVM) [11,12], hidden Markov model (HMM) [13,14], and Grey model (GM) [15,16].These approximation models are widely used in nonlinear simulation, classification, regression, and other domains.In the training phase, overfitting reduces the fitting errors but it causes the generalization performance of the model to decline.By contrast, underfitting causes models to veer away from true models with a relatively low fitting accuracy.
The overfitting problem has been studied by many scholars from different perspectives.To measure the fitting degree, Akaike proposed an entropy-based information criterion (i.e., Akaike information criterion or AIC) with consideration of the complexity and fitting precision of models.A small AIC is indicative of good model performance and has thus been widely studied by many authors [17,18].For the sake of decreasing the high complexity of models, the Bayesian information criterion is studied on the basis of the AIC [19,20].Utilizing existing regularization methods [21,22] is another method to avoid overfitting.With this approach, optimal parameters can be obtained to simplify models by adjusting either the L1 norm or the L2 norm of the weight coefficients [23].Also, adjusting algorithm parameters, dimensionality reduction, and cross validation experiments [24] are effective means to avoid the overfitting problem.Some examples in prediction domain are shown with the same EGT data (Exhaust Gas Temperature) as follows.Prediction results with an ANN with Bayesian regularization algorithm are seen in Figure 1, prediction results with a SVM with cross validation are seen in Figure 2, and prediction results with Grey model (GM) are seen in Figure 3.
Although regularization method is used in ANN and SVM, error precision ought to be better.GM is usually applied to predict parameters with noise, but error precision should be improved too.
To obtain high prediction precision, overfitting and underfitting problems must be addressed better.In addressing fitting problems, we set up a fitting sensitivity model to make the approximation model insensitive to samples that are far from the core of sample clusters and sensitive to samples that are close to the core of sample clusters.The fitting sensitivity model is achieved by controlling the fitting sensitivity to adjust the fitting accuracy of the approximation model.In using the fitting sensitivity model, the approximation model approaches the center of the sample clusters instead of the samples far from the center of the sample clusters.Hence, the robustness and adaptability of the model to new samples are improved.
In Section 2, an approximation modeling method based on fitting sensitivity is introduced.With this method, a generalized approximation model is built by analyzing the fitting sensitivity and its correlation with the approximating precision.In Section 3, a GM is taken as an example and the approximation modeling method based on the fitting sensitivity is employed to gain a novel GM that could avoid overfitting and underfitting problems better.This novel GM exhibits such capability because the main tendency of its training samples can be obtained through fitting, which can reduce the fluctuations in prediction results.In Section 4, values of parameters for the specific model proposed in Section 3 are set, and the optimization model solved with a genetic algorithm is established with a reasonable fitting degree.In Section 5, experimental verification with a single variable is performed to analyze the impacts of the model parameters on the accuracy of novel GM.The contrast experiment between ANN with Bayesian regularization and the novel ANN showed that the approximation model based on fitting sensitivity can yield better prediction results in comparison with traditional models.

Modeling Methods of the Generalized Approximation Model Based on Fitting Sensitivity
Firstly, fitting sensitivity is introduced and then the relationship between fitting degree and fitting sensitivity is illustrated.
Secondly, generalized approximation model is constructed based on fitting sensitivity.Also, fitting error analysis for the model is derived.Lastly, a novel GM based on fitting sensitivity is studied.The corresponding analysis is presented as follows: (1) When / → 1, there is overfitting of   to   ; that is, Δ ≈ Δ.More specifically, the changing tendency of the fitting value   is in accordance with that of   , as shown in Figure 4(a).
(2) When / > 1, there is underfitting of   to   with Δ > Δ, as shown in Figure 4(b).Moreover, the fitting value   enlarges the trend of the training sample   .In this case,   is unstable and is fluctuating along with   .Consequently, inaccurate prediction results are got.This condition is called "excessive underfitting." (3) When 0 < / < 1, there is underfitting of   to   with Δ < Δ, as shown in Figure 4(c).Moreover, the fitting value   compresses the trend of the training sample   .In this case,   is close to the main trend of   .Consequently, accurate prediction results are got.This condition is called "reasonable underfitting." In conclusion, when the initial value of  is the same as that of , the different levels of overfitting and underfitting correspond to the different values of the fitting sensitivity /.In particular, the reasonable fitting degree is obtained by setting / in the interval (0, 1), which can make the model avoid overfitting and excessive underfitting in some degree.

Generalized Approximation Model Based on Fitting Sensitivity.
Setting / in the interval (0, 1), a fitting sensitivity model is built as follows when  1 =  1 : where  is coefficient of whole compressibility to The description of (1) is as follows.
(1) |  −   | becomes large, which means the fitting value   is far from   .Because the fitting value   is required to represent the average level of the training samples, strong noise and violent fluctuations are included in   .To obtain a gentle main trend in   , the sensitivity of   to   ,   /  , should be decreased.
(2) |  −   | becomes small, which means the fitting value   is close to   .Because the fitting value   is required to represent the average level of the training samples, the changing trends of   are gentle.To maintain this gentle main trend, the changing trend of   should follow that of   .In this way, the sensitivity of   to   ,   /  , should be increased.
Equation ( 1) is transformed into the following integral equation: The integral variable is transformed into where  1 is integral offset.Equation ( 4) is the implicit expression of the approximation model based on fitting sensitivity.The generalized approximation model is obtained by transforming (4) as follows: The first item in ( 5) is the fitting value   of the approximation model to the training sample   .The second item is √/ ⋅ tan(  ⋅ √  ⋅  −  1 ), which denotes the adjustment of the traditional prediction model.
The approximation model mentioned above can avoid overfitting.However, the constraint equation needs to be added to avoid "excessive underfitting," as shown in Figure 5.
Thus, the coefficient  should be changed to control |  −  | in a range.The definition domain of ( 6) is whole time domain; that is,  = 1, . . ., .However, when the number of training samples is too large, it is difficult to maintain |  −  | less than a small number Δ at any point.As samples which are close to the prediction moment play an important role in improving forecast precision, the definition domain of ( 6) is controlled into the last  points of the training samples.That is, ( 6) is effective when  =  −  + 1,  −  + 2, . . .,  (see the following equation): where  is adjusting coefficient and  ∈ (0, 1/).
Equation ( 5) is substituted into (7) to finally obtain the generalized approximation model based on fitting sensitivity as follows: where   is fitting value at the point .

Fitting Error Analysis for the Generalized Approximation
Model Based on Fitting Sensitivity.The fitting error  is to be analyzed between the fitting values and training samples.
The following is obtained from ( 8): As shown in (11), when  1 =  2 = ⋅ ⋅ ⋅ =   =  1 / √  ⋅ ,  = 0.That is, when the fitting value   is equal to a constant  1 / √ , the fitting error is zero.However,   actually varies with   .Thus,  must be a numeric value greater than a small number ; that is,  > .
The following is obtained from (9): The lower and upper bounds of the fitting error  are obtained as follows: The model controls the fitting error  in a certain range, as shown in (13).Then, the model can avoid the overfitting and underfitting problems to some extent.

Modeling Methods of a Novel GM Based on Fitting Sensitivity.
The GM is suitable for predicting the time series of performance parameters with great randomicity.The GM can reduce accumulated errors and the fluctuations of prediction results by accumulated summation.Thus, we introduce a modeling technique for the approximation model based on fitting sensitivity by taking the GM as an example.
Overfitting and underfitting problems also exist in the traditional GM during the training phase, similar to other approximation models.As shown in (14), the developed coefficient â and Grey-controlled variable û in the GM are derived from the least squares method.
where â is developed coefficient, û is Grey-controlled variable, and  1 is initial value of the training samples.
To address the fitting problem, model parameters â and û are evaluated with the approximation model based on fitting sensitivity instead of the least squares method to effectively avoid overfitting and excessive underfitting in the training phase and ultimately gain the precise estimation parameters â and û.We can then construct the novel GM based on fitting sensitivity by using â and û, as mentioned in Section 2.2.
The novel GM based on fitting sensitivity is written as follows:

Solution of Novel GM Based on Fitting Sensitivity
The model parameters in (15) and ( 16) include the adjusting coefficients , ,  1 that can avoid the overfitting problem, the adjusting coefficients ,  that can avoid the excessive underfitting problem, and the shape parameters â and û.
Then, , , ,  are set according to their physical meaning, and  1 , â, û are set with the constraint condition of avoiding overfitting and excessive underfitting.Using these parameters, the optimization model based on fitting degree is solved with the genetic algorithm.When  → , it is difficult to make all training samples be constrained to avoid excessive underfitting.Even with the avoidance of excessive underfitting, inaccurate prediction results are still obtained because of the unrelated historical information on the training samples.
As a result,  is set up to reasonable range from 1 to .The last  training samples are then selected to be constrained, including the development tendency of the training samples instead of the unrelated historical information.When  → 0,   /  → 1/, as shown in (16), which results in high fitting precision and the high sensitivity of the fitting value   to   when  = −+1, −+2, . . ., ; when  → 1/,   /  → 0, which results in the low sensitivity of the fitting value   to   when  =  −  + 1,  −  + 2, . . .,  and the inaccurate fitting values at the last  training samples.
As a result,  is set up to reasonable range from 0 to 1/ on the basis of the sensitivity of the fitting value   to   at the last  training samples; that is,  =  −  + 1,  −  + 2, . . ., .

Setting of Parameters 𝑎, 𝑏.
The adjusting coefficients to avoid overfitting are , which makes the fitting sensitivity   /  < 1, and , which is the compressibility of   affected by |  −  | to make   approach the main trend of   and reduce the fluctuation of prediction results, as shown in (16).
The adjustment of  can make   approach the main trend of   .As mentioned in Section 2.2, the fitting value   represents the average level of the training samples.Thus, when |  −   | < Δ (Δ is a threshold),   exhibits high sensitivity to   ; when |  −   | > Δ,   exhibits low sensitivity to   .In other words,   can obtain the main trend of   with consideration of the two points above.
The lowest fitting sensitivity is 1/ − , which is obtained from the constraint in (15).When the distance between   and   reaches the maximum, that is, |  −   | = Δ, the minimum fitting sensitivity   /  can be obtained as follows: We obtain  from (17).
The parameter  can be well defined after setting the threshold Δ by taking the maximum difference between the neighboring points, that is,  = max  |  −  −1 |, in the training samples as the unit.In engineering practice, when |  −   | >  ⋅  ( is a constant), the fitting value   is far from   , with the fitting being ineffective and with   /  exhibiting low sensitivity.The study sets  = 5, which means that when the distance between   and   is over five times , it is a meaningless fitting, that is, when Δ = 5 max  |  − −1 |.

Adjustment of Parameters Based on the Genetic Algorithm.
After the setting of , , ,  based on their physical meanings, other parameters,  1 , â, û, are solved with the genetic algorithm with consideration of the constraint for overfitting and underfitting.

Establishment of the Optimization Model.
The first step in the design of the genetic algorithm is to establish the optimization model, and the key to construct the model is to build the adaptive function and nonlinear constraint.
Step 1 (build an adaptive function).The adaptive function is the objective function of the optimization model which is used to decide the seeking direction of the group.As the novel GM based on fitting sensitivity can avoid the overfitting problem, the adaptive function is transferred from (15) as follows: Step 2 (construct the constraint condition).The adaptive function considers the overfitting; it is to be supplemented with the constraint for the excessive underfitting problem.
Finally, the optimization model solved with the genetic algorithm is obtained as follows.

Design of the Initial Value of the Genetic Variable.
The design of the genetic operators is very mature and is not repeated here.On the basis of the requirements of this study, we need to set up the initial population of â, û,  1 as the optimization variables, as shown in (20).

Experimental Validation
Firstly, the experimental data is described.And then the different prediction results under different values of parameters , , ,  in novel GM are analyzed.Lastly, the prediction precision of ANN with regularization method is compared with that of approximation ANN based on fitting sensitivity.

Experimental Data.
The time series of performance parameter DEGT (the difference between the monitored Exhaust Gas Temperature and the benchmark) of an aeroengine over 200 cycles is used in the experiment.It is an important performance parameter of aeroengine.But significant randomicity and serious fluctuation are observed in it as shown in Figure 6.Randomicity and fluctuation in observed parameters bring about difficulties to predict DEGT.There are mainly four reasons which are given as follows.Firstly, random factors such as the actual working condition of equipment and human operation result in the fluctuation of the time series data, and thus the chosen approximation model cannot easily approach the original nonlinearization.Secondly, for the accumulation of errors caused by the iteration method for prediction, a long prediction phase equates to a large forecast deviation.Thirdly, when nonlinear models are aimed at high precision, overfitting occurs in the training phase with forecast deviation.Lastly, in the case involving many training samples during the training phase, the model parameters for approximating such samples are difficult to determine and model accuracy is adversely affected by the underfitting problem.
Real signals are difficult to find because of the presence of noise signals.So, the smoothed data approved by the aeroengine manufacturer are regarded as true values in evaluation of prediction accuracy in this work as shown in Figure 7.

Experiment Analysis of Parameters 𝑎, 𝑛, 𝑝, 𝜀 in Novel GM.
To analyze the effect of parameters , , ,  on novel GM precision, a single variable is maintained in turn to observe the effect of every variable on the prediction precision in novel Grey model.Values of , , ,  are set to their certain range.And then prediction errors are compared and analyzed varying with , , ,  in turn.Besides, 20 points are to be predicted in all experiments.

Effect of Parameter 𝑎 on Model Precision.
To study the effect of the single variable  on the prediction precision, we set  = 70,  = 10, and  = 0.1.Value range of  is (1, 1/) = (1, 10), and  is set to 1.25, 3, 5, 7, and 9 in five groups in novel GM. 60 experiments are done in every .The box plot of RMSEs at different values of  with the novel GM is shown in Figure 8.For comparison, RMSEs of traditional GM are drawn in Figure 8 too.There are seven indicators of RMSEs in box plot: max, maximum value; Q3, 75th percentile; median, the median of RMSEs; Q1, 25th percentile; min, minimum value; outlier, the number of outliers; and DQQ, distance between Q1 and Q3.
Values of indicators in box plot of RMSEs at different values of  with the novel GM and GM are shown in Table 1.
As shown in Table 1, the least value of RMSE median is 3.22, and the least DQQ is 2.93 with  = 3 in novel GM.That is, the prediction precision of novel GM gathers round 3.22, and the dispersion of errors is small when  = 3.In the prediction model proposed in the study, the fitting degree in the training phase is partly decided by .When  → 1, overfitting is the possible problem, while when  → +∞, underfitting is the possible problem.So, there must be an  at the highest prediction precision.Here,  = 3.
Prediction results of the novel GM are better than the results of the traditional GM, because there are smaller median and DQQ, corresponding to higher precision and lower dispersion.

Effect of Parameter 𝑛 on Model Precision.
To study the effect of the single variable  on the prediction precision, we set  = 3,  = 10, and  = 0.1.And  is set to 30, 40, 50, 60, 70, and 80 in six groups in novel GM. 60 experiments are done in every .The box plot of RMSEs at different values of  is shown in Figure 9.
Values of indicators in box plot of RMSEs at different values of  with the novel GM and GM are shown in Table 2.
As shown in Table 2, the least value of RMSE median is 3.22, and the least DQQ is 2.93 with  = 70 in novel GM.That is, the prediction precision of novel GM gathers round 3.22, and the dispersion of errors is small when  = 70.Excessively large  introduces too much unrelated information about the trend of the training samples, while small  does not include sufficient information.So, there must be an  at the highest prediction precision.Here,  = 70.3.As shown in Table 3, the least RMSE median is 3.05 with  = 40 and the least DQQ is 2.93 with  = 10 in novel GM.Prediction results of the novel GM are better than results of the traditional GM, because there are smaller median and DQQ, corresponding to higher precision and lower dispersion.
When  → 1, a few training samples which do not contain the trend of   are constrained to avoid excessive underfitting.And when  → , all training samples which include the unrelated historical information are constrained to avoid excessive underfitting.So, there must be a  at the highest prediction precision.4.
As shown in Table 4, the least value of RMSE median is 3.22, and the least DQQ is 2.93 with  = 0.1 in novel GM.
Prediction results of the novel GM are better than the results of the traditional GM, because there are smaller median and DQQ, corresponding to higher precision and lower dispersion.
When  → 0, there is a high fitting degree in novel GM, and when  → 1/ = 0.33, the low sensitivity of the fitting value   to   at the last  training samples also could affect the prediction precision.So, there must be an  at the highest prediction precision.Here,  = 0.1.

Approximation Model of ANN.
Experimental data in Section 4.1 is also used to be predicted with the approximation model of ANN with Bayesian regularization.For comparison, the ANN with Bayesian regularization algorithm is employed in the experiment.
Eight experiments are designed as shown in Table 5. Prediction errors in novel ANN and ANN with regularization are shown in Figure 12 and Table 6.
By analyzing the data, we conclude that there is a smaller median in novel ANN than in ANN with regularization, but DQQ in novel ANN is not the smallest.So, higher prediction precision is obtained, but there is little big dispersion in novel ANN than in ANN with Bayesian regularization.

Conclusion
This study established a generalized approximation model based on fitting sensitivity to solve the overfitting and underfitting problems.Taking GM as an example, a novel GM based on fitting sensitivity was proposed.Then, the novel  GM was solved with the genetic algorithm by establishing an optimization model restricted by a reasonable fitting degree.We used RMSE as criteria and compared the effects of the different model parameters of the novel model on the prediction precision in the experiments.It showed that novel GM and novel ANN can get higher prediction precision than traditional GM and ANN with regularization.Therefore, the novel model based on fitting sensitivity proposed in this work could avoid overfitting and underfitting and yield accurate prediction results in accordance with the theoretical analysis.

Figure 1 :
Figure 1: Prediction results of ANN with regularization.

Figure 2 :Figure 3 :
Figure 2: Prediction results of SVM with cross validation.

Figure 4 :
Figure 4: Position relations between fitting value   and training sample   under different fitting degrees.

Figure 5 :
Figure 5: Position relation between fitting value   and training sample   under excessive underfitting.

Figure 7 :Figure 8 :
Figure 7: True values in evaluation of DEGT prediction accuracy.

Figure 9 :
Figure 9: Box plot of RMSEs at different values of  in the novel GM and GM.

4. 2 . 3 .
Effect of Parameter  on Model Precision.To study the effect of the single variable  on the prediction precision, we set  = 3,  = 70, and  = 0.1. is set to 1, 10, 20, 30, 40, 50, and 60 in seven groups with novel GM. 60 experiments are done in every .The box plot of RMSEs at different values of  is shown in Figure 10.Values of indicators in box plot of RMSEs at different values of  with the novel GM and GM are shown in Table

Figure 10 :
Figure 10: Box plot of RMSEs at different values of  in the novel GM and GM.

Figure 11 :
Figure 11: Box plot of RMSEs at different values of  in the novel GM and GM.

4. 2 . 4 .
Effect of Parameter  on Model Precision.To study the effect of the single variable  on the prediction precision, we set  = 3,  = 70, and  = 10.And  is set to 0.1, 0.2, and 0.3 in three groups in novel GM. 60 experiments are done in every .The box plot of RMSEs at different values of  is shown in Figure 11.Values of indicators in box plot of RMSEs at different values of  with the novel GM and GM are shown in Table

Figure 12 :
Figure 12: Box plot of RMSEs in the novel ANN and ANN with Bayesian regularization.
3.1.ParameterSetting of the Novel GM. , , ,  are set to a reasonable range according to their physical meaning in the novel GM.More specifically, the last  training samples are used to satisfy the constraint condition to improve the prediction accuracy,  is employed to avoid excessive underfitting in the training phase, and ,  are adjusting coefficients to avoid overfitting in the model.3.1.1.Setting of Parameter .Excessive underfitting is challenging to be avoided during the whole training phase because of the presence of a large number of training samples.Thus, only the last  training samples are used to satisfy the constraint condition to improve the prediction accuracy, as mentioned in Section 2.2.The value of  ranges from 1 to , that is, the length of the training samples; that is,  ∈ [1, ], and  ∈ .When  → 1, a few training samples are constrained to avoid excessive underfitting, but these training samples do not contain the trend of   .Inaccurate prediction results are thus obtained.

Table 1 :
Values of indicators in box plot of RMSEs at different values of  in the novel GM and GM.

Table 2 :
Values of indicators in box plot of RMSEs at different values of  in the novel GM and GM.

Table 3 :
Values of indicators in box plot of RMSEs at different values of  in the novel GM and GM.

Table 4 :
Values of indicators in box plot of RMSEs at different values of  in the novel GM and GM.

Table 5 :
Eight experiments are designed.

Table 6 :
Values of indicators in box plot of RMSEs in novel ANN and ANN with Bayesian regularization.