With the rapid technology development and improvement, the product failure time prediction becomes an even harder task because only few failures in the product life tests are recorded. The classical statistical model relies on the asymptotic theory and cannot guarantee that the estimator has the finite sample property. To solve this problem, we apply the hierarchical Bayesian neural network (HBNN) approach to predict the failure time and utilize the Gibbs sampler of Markov chain Monte Carlo (MCMC) to estimate model parameters. In this proposed method, the hierarchical structure is specified to study the heterogeneity among products. Engineers can use the heterogeneity estimates to identify the causes of the quality differences and further enhance the product quality. In order to demonstrate the effectiveness of the proposed hierarchical Bayesian neural network model, the prediction performance of the proposed model is evaluated using multiple performance measurement criteria. Sensitivity analysis of the proposed model is also conducted using different number of hidden nodes and training sample sizes. The result shows that HBNN can provide not only the predictive distribution but also the heterogeneous parameter estimates for each path.
In this high technology era, the society operations highly depend on various machinery and equipments. Once the machinery or equipment is broken down, enormous trouble and economics cost will be brought to the entire society. To enhance the product reliability, the methodologies to assess product reliability have received much discussion in both academics and industries. Among several mature techniques, degradation testing provides an efficient way for reliability assessment when product quality is associated with a timevarying degradation process. Typically, degradation measures can provide more reliability information, particularly in modeling the failurecausing mechanism, than timetofailure data in few or no failure situation.
Predicting the remaining lifetime of a product is also an important issue in quality control. For example, knowing the remaining equipment lifetime can help in optimizing the machine maintenance schedule. The equipment lifetime is traditionally studied by fitting a statistical probability distribution, and most of these statistical models are constructed to study various degradation processes of a product. Examples include Lu and Meeker [
Most of above methods emphasize on parameter estimations or the process of hypothesis testing. Under the assumption that data follows a certain probability distribution, the statistical inference is made based on the asymptotic theory. The statistical inferences based on the asymptotic theory are valid only if the sample size is large or close to infinite. When the sample information is small or when the discrete data are provided, the finite sample property of the estimation based on the asymptotic theory is not held. Therefore, nonparametric or semiparametric statistics have been proposed to perform the reliability prediction. However, these statistical methods are far from perfect because the overfitting problem usually leads to inaccurate parameter estimates.
Due to the data limitations and the drawbacks of classical statistics approaches, Bayesian approach provides the solution from a different perspective. Unlike these frequentist’s approaches which consider the data random and the test statistics or estimators are investigated over imaginary samples
Lately Bayesian has been applied in the fatigue crack growth prediction in the literature. For example, Zheng and Ellingwood [
Neural network (NN) is the other popular prediction method. Neural network is a computerintensive, algorithmic procedure for transforming inputs into desired outputs using highly interconnected networks of relatively simple processing elements (often termed neurons, units, or nodes; we will use nodes thereafter). Neural networks are modeled following the neural activity in the human brain. The essential features of a neural network are the nodes, the network architecture describing the connections between the nodes, and the training algorithm used to find values of the network parameters (weights) for a particular network. The nodes are connected to one another in the sense that the output from one node can be served as the inputs to other nodes. Each node transforms an input to an output using some specified function that is typically monotone, but otherwise arbitrary. This function depends on parameters whose values must be determined with a training set of inputs and outputs. Network architecture is the organization of nodes and the types of connections permitted. The nodes are arranged in a series of layers with connections between nodes in different layers, but not between nodes in the same layer.
Several researchers also integrate neural network algorithm with Bayesian theory, which has been known as Bayesian neural network, in prediction. For examples, Neal [
In this paper, we conduct a hierarchical Bayesian neural network analysis with MCMC estimation procedure in the failure time prediction problem. Here, hierarchy means that the coefficients in our constructed HBNN model are specified by random effect distributions. We attempt to use this hierarchical structure to determine if the heterogeneity exists among paths. The advantage of proposed HBNN model cannot only provide a better failure time prediction by incorporating the heterogeneity of components and autocorrelated structure of error term but also provide a predictive distribution for the target value. Different from previous research, the proposed HBNN model can successfully offer the full information of parameter estimation and covariance structure. Engineers can use the heterogeneity estimates to identify the causes of the quality differences and further enhance the product quality.
The data of the fatigue crack growth from Bogdanoff and Kozin [
The rest of this paper is organized as follows: Section
To model failure time, we adapted the growthcurve equation used by Liski and Nummi [
According to the above equation, we know that there are totally
According to the above setting, the likelihood function for the data can be written as
To reduce the computational burden of posterior calculation and exploration, we define
By using the Bayes theorem with the sample information and prior distribution of each parameter, the posterior distribution of each parameter can be derived. The posterior distributions and the details of the estimation procedure can be referred to Carlin and Louis [
In addition to the posterior distribution for the estimated parameter, the predictive distribution of the unobserved cycle time,
Among these MCMC methods, Gibbs sampling algorithm is one of the best known estimation procedures that uses simulation as its basis [
We use the fatigue crack growth data from Bogdanoff and Kozin [
Thirty paths of fatigue crack growth data from Bogdanoff and Kozin [
In this data set, there are 30 sample paths in total and each sample path has 164 paired observations, cycle time, and crack length. The cycle time is observed at some fixed crack lengths. We predefined the path as “failure” as soon as its crack length reaches a particular critical level of degradation (i.e.,
Because the coefficients
Table
Estimated mean and STD for posterior parameters.
Estimated parameter  

Posterior Mean 




0.001521  0.7806321 


Posterior STD 




0.000313  0.073125 
Covariance matrix of the distribution of heterogeneity.
Estimated parameter  Posterior Mean  Posterior STD 



















The model estimation shown in Section
Performance measures and their definitions.
Metrics  Calculation 

RMSE 

MAD 

MAPE 

RMSPE 

Summary of failure time prediction results by HBNN model.
Models  RMSE  MAD  MAPE  RMSPE 

HBNN  0.37340  0.27121  1.058%  1.440% 
Prediction of failure time at 49 mm (when data collection is stopped at 40 mm).
To evaluate the sensitivity of the proposed method, the performance of the HBNN model was tested using different number of hidden nodes and training sample sizes. In this section, we set the number of hidden nodes as 3, 4, 5, and 6. And three different sizes of the training dataset (observations collected from 9 (mm) to 30 (mm), 9 (mm) to 35 (mm), and 9 (mm) to 40 (mm) resp.) were considered. The prediction results made by the HBNN model are summarized in Table
Sensitivity analysis.
# of hidden nodes  Training dataset  RMSE  MAD  MAPE  RMSPE 

3  from 9 (mm) to 30 (mm)  0.40576  0.29048  1.108%  1.543% 
from 9 (mm) to 35 (mm)  0.38022  0.27297  1.081%  1.486%  
from 9 (mm) to 40 (mm)  0.37340  0.27121  1.058%  1.440%  


4  from 9 (mm) to 30 (mm)  0.40568  0.29060  1.089%  1.558% 
from 9 (mm) to 35 (mm)  0.38070  0.27326  1.057%  1.487%  
from 9 (mm) to 40 (mm)  0.37366  0.27088  1.089%  1.467%  


5  from 9 (mm) to 30 (mm)  0.40592  0.29036  1.082%  1.520% 
from 9 (mm) to 35 (mm)  0.38076  0.27350  1.056%  1.456%  
from 9 (mm) to 40 (mm)  0.37391  0.27065  1.062%  1.435%  


6  from 9 (mm) to 30 (mm)  0.40606  0.29047  1.104%  1.536% 
from 9 (mm) to 35 (mm)  0.38051  0.27393  1.098%  1.489%  
from 9 (mm) to 40 (mm)  0.37357  0.27068  1.076%  1.461% 
According to the table, the HBNN model has a lower RMSE, MAD, MAPE, and RMSPE with observations collected from 9 (mm) to 40 (mm) than with observations collected from 9 (mm) to 30 (mm). This is because the sample size of the 9 (mm) to 30 (mm) dataset is smaller than the sample size of the 9 (mm) to 40 (mm) dataset. However, the RMSE, MAD, MAPE, and RMSPE are almost the same for the cases of hidden nodes = 3, 4, 5, or 6. This result suggests that there is no difference for the predictions when the number of hidden nodes varies.
In this paper, we applied the HBNN approach to model the degradation process and to make the failure time prediction. In the process of developing the HBNN model, the MCMC was utilized to estimate the parameters. Since the prediction of failure time made by HBNN model can sufficiently represent the actual data, the timetofailure distribution can also be obtained successfully. In order to demonstrate the effectiveness of the proposed hierarchical Bayesian neural network model, the prediction performance of the proposed model is evaluated using multiple performance measurement criteria. Sensitivity evaluation of the proposed model is also conducted using different number of hidden nodes and training sample sizes. As the results reveal, using HBNN can provide not only the predictive distribution but also accurate parameter estimate. By specifying the random effects on the coefficients
For the future research, statistical inferences of failure time based on degradation measurement, such as failure rate and tolerance limits, can be further evaluated given the predicted failure time. In addition, for some highly reliable products, it is not easy to obtain the failure data even under the elevated stresses. In such case, accelerated degradation testing (ADT) can be an alternative that provides an efficient channel for failure time prediction. The proposed HBNN approach can also be applied to depict the stressrelated degradation process by including those stress factors as covariates in the model.