^{1}

^{2}

^{1}

^{2}

^{3}

^{1}

^{2}

^{1}

^{2}

^{3}

Large-scale mechanical equipment monitoring involves various kinds and quantities of information, and the present research on multisensor information fusion may face problems of information conflicts and modeling complexity. This paper proposes an analysis method combining correlation analysis and deep learning. According to the characteristics of monitoring data, three types of correlation coefficients between sensors in different states are obtained, and a new composite correlation analytical matrix is established to fuse the multisource heterogeneous data. The matrix represents fault feature information of different equipment states and helps further image generation. Meanwhile, a convolutional neural network-based deep learning method is developed to process the matrix and to discover the relationship between results and equipment states for fault diagnosis. To verify the method of this paper, experimental and field case studies are performed. The results show that it can accurately identify fault states and has higher diagnostic efficiency and accuracy than traditional methods.

With the development of technology, mechanical equipment presents the characteristics of complexity, automation, and high speed, which greatly increases the difficulty of equipment condition monitoring and fault diagnosis. Multiple sensors can receive more fault information and improve the accuracy of fault diagnosis [

Due to many data sources and different types, data fusion is needed in order to perform the fault diagnosis. Generally, there are two kinds of fault diagnosis based on multiple sensors: the decision-level fusion diagnosis and the feature-level fusion diagnosis, respectively. For the decision-level fusion diagnosis, methods like DS evidence theory [

Most previous studies treated the information of multiple sensors as a single signal and ignored the coupling relationship between signals, resulting in the loss of effective information [

In view of this, this paper proposes a new analysis method combining composite correlation analysis and deep learning theory. According to the characteristics of the traditional correlation analysis, a kind of composite correlation coefficient is designed and calculated between sensors of different states. The correlation coefficient matrix is treated as representation of fault feature information of different equipment states and is transformed into images. Deep mining of the correlation coefficient is performed by the deep learning method with multilayer network, and thus the relationship between the correlation coefficient and equipment state is obtained for fault diagnosis. CNN is selected for fault identification in this paper because it has been successfully applied in the field of mechanical equipment fault diagnosis and has shown good performance [

Through the above analysis, the main contributions in this paper involve the following. (1) A fault information fusion method is proposed for multisource heterogeneous data, and a new correlation analysis matrix is established with the comprehensive advantages of several related analysis methods. By analyzing the correlation of multisource sensors, the changes of equipment states are represented, and the image generation of monitoring data is realized. It can reduce the data dimension, improve the computational efficiency, and prevent the fault information loss caused by the direct comparison or normalization between data of different types and different orders of magnitude. (2) Considering the characteristic of high dimension and large amount of the monitoring data, a fault diagnosis model based on correlation analysis and deep learning is built which directly trains and recognizes the correlation matrix image of a large number of monitoring data. The method avoids the issues of low efficiency and dimensionality curse by the conventional pattern recognition method for large and high-dimensional data, and the fault diagnosis accuracy is improved.

Correlation analysis generally refers to the analysis method to study the relationship between variables, that is, to study the change relationship of another variable when one changes. The value describing this relationship is called correlation coefficient. Data of sensors in mechanical equipment monitoring are continuous variables. For continuous variables correlation analysis, correlation coefficients include Pearson and Spearman correlation coefficients, and multiple correlation analysis is commonly used. Meanwhile, since this paper focuses on the relationship between different sensors, it needs the correlation analysis of signal from one-to-one and one-to-many sensors, so the above three correlation analysis methods are discussed in the paper.

Pearson correlation coefficient is a statistical index describing the degree of correlation between variables, and the value varies between −1 and 1. When the changes between two variables are consistent, the value is greater than 0, and especially, it is called complete correlation when the value is 1. On the contrary, when the changes between two variables are opposite, which is called negative correlation, the value is less than 0. When there is no correlation between the changes of two variables, the value is 0, which is called uncorrelated. Let (_{i},_{i}) (_{i}, _{i}); then, the Pearson correlation coefficient is

Spearman rank correlation is a method to study the correlation between two variables based on rank data. The data requirements of Spearman rank correlation are less strict than Pearson’s correlation. As long as the observed values of two variables are rank data in pairs or the ones transformed by continuous variable observation data, Spearman rank correlation can be used, regardless of the overall distribution of the two variables and the size of samples. Spearman correlation coefficient _{s} is calculated as follows:_{i}) and _{i}) are the rank of the two variables _{i} and _{i} in their respective column vectors and

In practical analysis, a variable is often subject to the comprehensive influence of a variety of variables. The so-called complex correlation means to study the correlation between multiple variables with one at the same time. The index to measure the degree of complex correlation is the complex correlation coefficient. The correlation between multiple variables with one at the same time cannot be directly measured, and only indirect calculation can be done. The complex correlation coefficient _{c} between a variable _{i} (_{i} and

Convolutional neural network is a kind of bionics algorithm imitating biological neural network. Being a representative algorithm in deep learning theory, the difference with the traditional neural network is that it has a deeper network structure in order to simulate biological neural networks more accurately. The traditional neural network has problems in establishing the multilayer structure, such as the complex network, too many nodes and parameters, slow convergence, and computational difficulty. Convolutional neural network avoids these problems by the approach of local connections and weight sharing [

Local connection is different from the full connection in the traditional neural network. It means that the neuron nodes in a certain layer of the neural network are not connected to all the neurons in the upper and lower adjacent layers, but only connected to part of adjacent neurons according to certain rules. In this way, the number of neurons is greatly reduced, and the size of the neural network is decreased. Especially when dealing with high-dimensional data, due to the complexity of network structure and exponentially increasing neuron data, it is difficult to apply the full connection method. However, through local connection, the neural network structure is simplified with less parameter, and the network availability is improved.

Weight sharing is another characteristic of convolutional neural networks. The concept refers to the fact that in a locally connected network, the parameters are same when different upper and lower neurons are connected. On the basis of local connection, this method greatly reduces the number of parameters and improves the generalization ability of the network.

In addition to the above two characteristics, the network structure of convolutional neural network is different from traditional one, which has more complex structure and more layers. It usually consists of input layer, multiple convolution layers, pooling layers, fully connected layer, and output layer, and the convolution and pooling layers are unique to CNN, as shown in Figure

Structure of convolutional neural network.

When training by the convolutional neural network, if the training data are too less or the data image is too large, overfitting phenomenon may occur. Although better training accuracy can be obtained, the recognition rate is lower when the trained model is applied toother data.

In order to solve this problem, Alex [

Figure

Comparison of neural network (a) and (b) neural network with dropout layer.

By using dropout, the calculation equation is [

In equation (

In fault diagnosis of mechanical equipment based on multisensor information, the key problem lies in how to make comprehensive use of all monitoring information. In order to find an effective way to realize the diagnosis by multisource heterogeneous multisensor information fusion, a method based on composite correlation analysis and deep learning is proposed in the paper, and a new correlation matrix is set up to represent the correlation changing relationship between different sensors, and the 1-dimensional data are transformed into 2-dimensional images. Then, combining the deep convolutional neural network, the fault diagnosis model is built to directly analyze the image to realize the fault diagnosis and to improve fault recognition accuracy. The framework of the method is shown in Figure

Framework of the proposed method.

Different kinds of sensors are used to collect the monitoring data of mechanical equipment, and feature extraction is performed on the data according to the characteristics of different sensors. Then, the characteristic values are used to analyze the correlation between different sensors, and the new composite correlation matrix is calculated and visualized by image generation. Then, a fault diagnosis model based on deep convolutional neural network is established to classify the fault and to identify the fault pattern.

The multisource sensor information fusion method studied in this paper makes use of the interrelationship between different sensors, which requires correlation analysis of multidimensional eigenvalues of signals collected by different sensors. According to Section _{n} is established in this paper. For _{i}, _{j} (

According to equation (_{n}(_{i}, _{j}) in the composite correlation matrix is calculated differently according to different indices; when the index

How to use the composite correlation coefficient matrix of multisource sensors to build the fault diagnosis model has great influence on diagnosis accuracy . The traditional methods need to transform the matrix into a one-dimensional vector, and then methods such as neural network, SVM, and cluster analysis are used for diagnosis. According to the characteristics of composite correlation coefficient matrix, the diagnosis model based on convolutional neural network is established, and then processing is performed in the form of a two-dimensional matrix in order to improve the calculation efficiency and accuracy.

When using the convolutional neural network for classification, the input of training and classification model is required to be images. Therefore, the composite correlation coefficient matrix is firstly visualized by image generation. Since the matrix is two-dimensional, it is transformed into an 8 bit gray image in this paper as follows:_{max} and _{min} are the maximum and minimum values of the elements, and

According to the requirements of mechanical equipment monitoring and the characteristics of correlation coefficient matrix, the CNN structure is designed for the gray image of composite correlation coefficient matrix in this paper, as illustrated in Figure

Experimental setup. 1, variable speed controller; 2, thermal camera; 3, rotor test stand; 4, GUI in computer.

At the same time, because the speed of parameter updation in the model is determined by the learning rate, the adaptive learning rate is used in the experiment to speed up model optimization, guaranteeing better performance than experience-based approaches. The initial learning rate is set to 0.001. During the training, at the end of each epoch, loss and precision of the current model are evaluated in the validation set. The loss value changes are checked every other epoch, and when it is less than 0.0001, the learning rate

A variety of different faults might occur in the runtime of large mechanical equipment; sometimes, even two or more concurrent or coupling faults occur at the same time. In order to solve such problems, the researchers introduced different monitoring tools, for example, in rotor fault diagnosis, in addition to the traditional vibration monitoring, the temperature field monitoring by infrared image can better solve the concurrent or coupling faults [

The testbed is built in the laboratory to simulate different working states of rotating machinery and to collect data. The experimental hardware includes the ZT-3 rotor test bed, FLIR E50 infrared thermal camera, and MDES fault diagnosis system. Besides, the computer and signal cables are included. Figure

The ZT-3 rotor testbed is composed of a governor, base, motor, coupling, and dual-rotor system. The rotor system consists of a rotating shaft, rotor, bearing, coupling, and bearing bracket. In the experiment, the motor speed is 6000 rpm.

The vibration signal is collected by the MDES fault diagnosis system, which includes a computer, acceleration sensor, and multichannel vibration signal acquisition instrument. As shown in Figure

The arrangement of measuring points of vibration signal.

Infrared images are collected by the FLIR E50 infrared thermal imager. During the experiment, the infrared thermal imager is fixed on a tripod to ensure that all infrared images are collected under the same condition.

7 states are simulated in the experiment, including normal state (NS), imbalance (IB), misalignment (MA), rub impact (RI), bearing set loose (BSL), coupling faults of rub impact and misalignment (CFRM), and coupling faults of bearing set loose and misalignment (CFBM). 40 images and 40 sets of vibration data in each channel are collected at each state, of which 20 datasets are used for training and the remaining data are used for testing. The experiment was conducted under the Keras deep learning framework and hardware with an I9-9700 CPU and a RTX2080ti GPU.

In the experiment, firstly, the vibration and infrared data in rotating equipment monitoring are collected. Then, feature extraction is performed on the infrared image and vibration signal, respectively. The correlation coefficients of each infrared and vibration signal are calculated, and the composite correlation coefficient matrix is constructed for information fusion and converted into gray image. Finally, the deep learning model based on CNN is used for training and classification to realize fault diagnosis.

The infrared images are processed in accordance with the method in literature [

Acquisition of sensitive areas of infrared images. (a) Original image. (b) Image segmentation. (c) Extraction of sensitive areas.

Histogram features of each ROI are calculated as the characteristic values of four temperature sensors. The calculation of gray histogram information refers to equation (

Expression of histogram features of the infrared image.

The index name | Expression |
---|---|

Mean | |

Standard deviation | |

Skewness | |

Kurtosis | |

Energy | |

Entropy |

Similar to the infrared image feature extraction method, in order to simplify the analysis and avoid the differences introduced by the complex algorithm, the vibration data feature values obtained by the 8 vibration sensors are calculated by the commonly used nondimensional indicators in the time domain, as shown in Table

List of time domain features.

The index name | Expression |
---|---|

Mean | |

Standard deviation | |

Root mean square | |

Peak | |

Skewness | |

Kurtosis |

_{i} (

A 12 × 6 eigenvalue matrix is constructed by the four temperature eigenvectors and eight vibration eigenvectors. The matrix is calculated according to equation (

Correlation coefficient matrix at normal state.

Sensor | V1X | V1Y | V2X | V2y | V3x | V3y | V4x | V4y | F1 | F2 | F3 | F4 |
---|---|---|---|---|---|---|---|---|---|---|---|---|

V1x | 0.49 | 0.97 | 0.95 | 0.47 | 0.97 | 0.43 | 0.96 | 0.40 | 0.37 | 0.45 | 0.35 | 0.31 |

V1y | 0.39 | 0.38 | 0.65 | 0.89 | 0.61 | 0.96 | 0.58 | 0.98 | 0.61 | 0.64 | 0.59 | 0.50 |

V2x | 0.74 | 0.22 | 0.64 | 0.27 | 0.95 | 0.22 | 0.89 | 0.19 | 0.72 | 0.65 | 0.55 | 0.65 |

V2y | 0.53 | 0.63 | 0.15 | 0.31 | 0.49 | 1.00 | 0.61 | 1.00 | 0.28 | 0.10 | 0.07 | 0.32 |

V3x | 0.84 | 0.36 | 0.69 | 0.54 | 0.42 | 0.46 | 0.97 | 0.42 | 0.72 | 0.52 | 0.47 | 0.65 |

V3y | 0.68 | 0.53 | 0.44 | 0.81 | 0.71 | 0.49 | 0.57 | 1.00 | 0.14 | 0.35 | 0.26 | 0.08 |

V4x | 0.71 | 0.06 | 0.71 | 0.22 | 0.76 | 0.18 | 0.46 | 0.55 | 0.81 | 0.56 | 0.51 | 0.83 |

V4y | 0.34 | 0.51 | 0.44 | 0.66 | 0.35 | 0.83 | 0.25 | 0.26 | 0.10 | 0.30 | 0.19 | 0.06 |

F1 | 0.12 | 0.05 | 0.27 | 0.07 | 0.03 | 0.15 | 0.25 | 0.10 | 0.43 | 0.88 | 0.78 | 0.97 |

F2 | 0.20 | 0.02 | 0.47 | 0.04 | 0.09 | 0.13 | 0.51 | 0.30 | 0.88 | 0.55 | 0.95 | 0.79 |

F3 | 0.29 | 0.03 | 0.57 | 0.00 | 0.22 | 0.22 | 0.63 | 0.19 | 0.78 | 0.95 | 0.63 | 0.64 |

F4 | 0.37 | 0.16 | 0.58 | 0.11 | 0.42 | 0.35 | 0.61 | 0.06 | 0.97 | 0.79 | 0.64 | 0.57 |

It can be seen from Table

Correlation coefficient matrices under different fault conditions. (a) IB. (b) MA. (c) BSL. (d) RI. (e) CFRM. (f) CFBM.

In Figure

In this classification experiment, 20 sets of composite correlation coefficient matrix images are randomly selected as the training data at each state of rotor system, and the remaining 20 sets of images are taken as test data, which means that 140 sets of images form the training set and the remaining 140 sets form the test set. According to the image resolution, the structure of CNN network is shown in Figure

Structure of convolutional neural network in this study.

The classification result after 300 times of training is shown in Figure

Classification result by the method in this paper.

In this case, based on the diagnosis model in this paper, Pearson, Spearman, and complex correlation coefficient matrices are applied for fault diagnosis simultaneously, replacing the proposed composite feature coefficient matrix. Moreover, in order to compare with the traditional methods, after feature extraction, 4 temperature feature vectors and 8 vibration feature vectors are directly combined, and the traditional BP, SVM, and KNN are used for fault diagnosis. The results are shown in Table

Comparison of test accuracy from different diagnostic models.

Methods | Accuracy (%) |
---|---|

BP | 87.14 |

SVM | 88.57 |

KNN | 93.57 |

Pearson + CNN | 96.43 |

Spearman + CNN | 95.00 |

Complex correlation coefficient + CNN | 82.85 |

The proposed method | 99.29 |

It can be seen from Table

In this paper, a new fault diagnosis method based on correlation analysis and deep learning is proposed. A new composite correlation analysis method is established to perform feature-level fusion of sensor data from different sources, and then the correlation coefficient matrix image is processed directly by CNN algorithm to complete fault diagnosis. Through case study, the following conclusions are drawn:

The changes of equipment state can be represented through the correlation analysis of multiple sensors, and multisource information fusion is carried out. It can reduce the data dimension, improve the computing efficiency, and avoid the loss of fault information caused by direct comparison or normalization between data of different data types and different orders of magnitude.

The new correlation analysis method is built, which integrates the advantages of several correlation analysis methods, is suitable for heterogeneous sensor data with different distributions, and influences relationships, so as to obtain better results in fault diagnosis.

By constructing the fault diagnosis method combining the correlation analysis and deep learning model, the images are trained and identified directly, which are transformed from the correlation matrix of the monitoring data. Compared with the traditional method, the model is simplified and the fault diagnosis accuracy is higher.

The data used to support the findings of this study are available from the corresponding author upon request.

The authors declare that there are no conflicts of interest regarding the publication of this paper.

The authors acknowledge the financial support provided by the National Natural Science Foundation of China (51975038 and 51605023), Beijing Municipal Natural Science Foundation, China (19L00001), General Project of Scientific Research Program of Beijing Education Commission (KM202010016003 and SQKM201810016015), Postdoctoral Science Foundation of Beijing, China (ZZ2019-98), Support Plan for the Construction of High-Level Teachers in Beijing Municipal Universities (CIT&TCD201904062 and CIT&TCD201704052), Scientific Research Fund of Beijing University of Civil Engineering Architecture (00331615015), BUCEA Post Graduate Innovation Project (PG2019092), and Fundamental Research Funds for Beijing University of Civil Engineering and Architecture (X18133).