Defect Detection of Pandrol Track Fastener Based on Local Depth Feature Fusion Network

*ere are three main problems in track fastener defect detection based on image: (1) *e number of abnormal fastener pictures is scarce, and supervised learning detectionmodel is difficult to establish. (2)*e potential data features obtained by different feature extraction methods are different. Some methods focus on edge features, and some methods focus on texture features. Different features have different detection capabilities, and these features are not effectively fused and utilized. (3)*e detection of the track fastener clip will be interfered by the track fastener bolt subimage. Aiming at the above three problems, a method for track fastener defects detection based on Local Deep Feature Fusion Network (LDFFN) is proposed. Firstly, the track fastener image segmentationmethod is used to obtain the track fastener clip subimage, which can effectively reduce the interference of bolt subimage features on the track fastener clip detection. Secondly, the edge features and texture features of track fastener clip subimages are extracted by Autoencoder (AE) and Restricted Boltzmann Machine (RBM), and the features are fused. Finally, the similarity measurement method Mahalanobis Distance (MD) is used to detect defects in track fasteners. *e effectiveness of the proposed method is verified by real Pandrol track fastener images.


Introduction
With the continuous growth of the world economy, railway transportation plays a more and more important role in the economic lifeline. In order to meet the growing demand for economic development, railway construction has also increased year by year. Railway track is composed of several key parts, such as sleeper, rail, and fastener. As the connecting part of fixed track and sleeper, track fastener is an important part to ensure the safety of railway operation. Huge safety risks and even major accidents were caused by the defective track fasteners, such as breaks or missing. Defective track fastener is caused by the contact friction and vibration impact between the hub and the track, as well as the impact of the natural environment [1,2].
At present, the state of track fasteners is mainly detected by manual inspection. Manual inspection method is inefficient, and many uncontrollable risk factors may be caused by manual inspection, such as missed inspection and false inspection. With the rapid development of automation technology [3][4][5][6], manual inspection has been difficult to adapt to the rapid development of track transit. e inspection of track traffic needs to be automated. Automating the track fastener detection technology [7,8] is significant to track operation and maintenance. However, the existing track fastener defect detection methods are limited, and the detection accuracy is low.
In recent years, with the rapid development of artificial intelligence technology [9,10], image processing methods based on deep learning have become a research hotspot [11][12][13]. Deep learning networks have achieved important research results in image classification [14,15]. e defect detection method of track fastener based on deep learning avoids complicated feature extraction. e manual design and selection of image features are replaced by automatic extraction of image features by computer. e deep learning network automatically learns the pattern recognition system of features from the data set, and the information in the image can be extracted more comprehensively [16]. Although the feature extraction of deep learning is shown to be far superior to traditional "artificial design features," the existing track fastener defect detection network based on deep learning is a supervised network, and a large amount of labeled sample data is required for a supervised network. For example, when using convolutional neural networks to detect defects in track fasteners [17], a large number of labeled training sample images are required. However, the actual situation is that the number of abnormal samples of track fastener is rare, which is limited for the establishment of supervised learning model.
Because the supervised network is limited by the number of abnormal fastener images, it is difficult to establish, so the unsupervised network is adopted. However, different deep learning networks have different structure and depth. With different deep learning networks, the features of track fastener image are extracted differently, and the detection ability of different features is also different. e feature information of track fastener image extracted by single network is not comprehensive enough, and the feature information of track fastener image cannot be used well. For example, the edge feature of the track fastener image is extracted [18][19][20]. Edge feature is an important feature of image, and image edge is the discontinuity of image features such as pixel gray distribution. Image edge is the set of pixels whose characteristics around the image change step or ridge. e edge extraction is mainly to retain the sharply changed area of the image, and the contour edge of the track fastener can be obtained. e edge contour feature of the track fastener image is helpful to distinguish the state of track fastener loss and track fastener clip loosening. e texture feature of track fastener is extracted [21][22][23]. e image texture feature is a global feature, and the visual features of homogeneity in the image are reflected. e arrangement properties of the surface structure of the object surface with slow or periodic changes are reflected. e texture feature can effectively represent the clip detail feature of track fastener image, and the clip detail feature of track fastener image is helpful to identify the fracture state of track fastener clip. ese features are not well integrated and utilized at the same time, and the feature information of track fastener image is not fully expressed, which leads to a high rate of missing alarm in track fastener defect detection [24][25][26].
Because, for Pandrol fasteners, the track fastener picture is composed of the track fastener clip subimage and the track fastener bolt subimage, the defect of the track fastener is mainly determined by the state of the track fastener clip subimage. When the deep learning network is used to extract the full image feature, the bolt subimage feature in the image will also be extracted into the discriminant feature, which will interfere with the detection of track fastener defects. erefore, in view of the above-mentioned problems in the detection of track fastener defects: the number of abnormal samples of track fastener images is sparse; image features are not fully utilized; and track fastener detection is interfered by the feature of track fastener bolt subimage. A method that is based on LDFFN to detect the Pandrol track fastener defects is proposed. Firstly, the Pandrol track fastener picture is segmented by the automatic segmentation algorithm of track fastener pictures. e bolt information in the picture is removed to obtain the track fastener clip subpicture, so as to avoid the interference of the bolt subgraph feature on the track fastener defect detection. Secondly, AE [27,28] and RBM [29,30] are used to extract clip subimage features of track fastener, because AE focuses on edge feature for extracting subimage features of track fastener clip [31], while RBM focuses on texture feature [32,33]. Considering that the edge feature is advantageous for detecting the state of track fastener loss and track fastener clip loosening, the texture feature is advantageous for detecting the broken state of the track fastener clip. erefore, the subimage features of the track fasteners extracted by AE and RBM are fused to obtain the locally deeply integrated track fastener features [34]. Finally, the similarity measurement method MD [35] is used to detect the status of track fasteners. e image features of track fastener can be fully extracted by unsupervised LDFFN, the interference of bolt diagram of track fastener on defect detection of track fastener can be avoided, and the problem that the number of abnormal images of track fastener is rare can be solved. e second part of this article briefly introduces two deep learning networks, AE and RBM. e third part describes the proposed defect detection method of track fasteners based on LDFFN. e fourth part verifies the effectiveness of the LDFFN-based track fastener defect detection method through experimental comparative analysis. e fifth part summarizes and prospects the full text.

AE.
AE is an unsupervised neural network model that can learn the hidden features of the input data, which is called encoding. At the same time, the original input data can be reconstructed with the learned new features, which is called decoding [36]. From an intuitive point of view, AE can be used for feature dimensionality reduction, similar to principal component analysis but more compared to the performance of principal component analysis, which is due to the fact that AE has stronger nonlinear feature extraction capabilities. e main structure of AE is shown in Figure 1.
X is represented as the input of AE encodes, H is represented as the new feature, and X ′ is represented as the output of AE decodes. e encoding process is as follows: e decoding process is as follows: Complexity where L(X, g(f(X))) is represented as the reconstruction error function and L(X, g(f(X))) is determined by functions f and g. By calculating the minimum value of J AE (θ), the parameter θ of the AE neural network can be solved.

RBM.
Boltzmann machines are a large class of neural network models, but the most used model in practical applications is RBM. e RBM model itself is very simple, just a two-layer neural network. e main structure of RBM is shown in Figure 2.
is represented as the RBM network hidden nodes. Each visible node is only related to n hidden nodes and is independent of other visible nodes. at is, the state of this visible node is only affected by the influence of nhidden nodes. For each hidden node, it is only affected by m visible nodes. a � (a 1 , a 2 , . . . , a m ) is represented as the offset of the visible node.
represented as the offset of the hidden node.

Automatic Segmentation of Track Fastener Pictures.
e Pandrol track fastener image is composed of the track fastener clip subgraph and the track fastener bolt subgraph. Normally, if the fastener bolt is damaged, it will not be reflected in the image, and the damage of the fastener clip will be reflected in the image. Track fastener defect detection will be disturbed by the track fastener bolt which is not damaged. In this paper, in order to reduce the interference of track fastener bolts, the image of track fastener will be automatically segmented.
In the first step, the input image is grayscale processed [37]. Because the color image data of track fasteners is relatively large, the calculation is time-consuming. e grading empirical formula is expressed as follows: where Red(x, y) is represented as pixels in the red channel. Green(x, y) is represented as pixels in the green channel. Blue(x, y) is represented as pixels in the blue channel.
In the second step, histogram equalization is performed on the gray image [38]. e contrast and detail of the image are improved.
where n is represented as the number of pixels in the image, n i is represented as the number of pixels in each gray level of the original image, L is represented as the number of gray levels of the original image, and p x (i) is the probability of occurrence of a pixel of gray i in the image. Equation (6) is cumulative normalization.
where cdf x (i) is represented as the cumulative distribution probability, round indicates that the number is rounded, L is represented as the number of gray levels of the original image, 255 represents the maximum gray level after histogram equalization, and g(i) represents the pixel value of the pixel histogram whose gray level is i in the original image after equalization.
In the third step, the gray image is denoised by Gaussian filtering [39]. Image noise can be removed by Gaussian filter. Gaussian filter denoising can keep the edge of the image well, and more of the overall gray distribution characteristics of the image can be preserved.
In the fourth step, a sample image is made for the subimage of track fastener clip. ereafter, grayscale, grayscale histogram equalization, and Gaussian filtering are performed on the sample picture.
In the fifth step, find the location of the track fastener clip subimage in the track fastener image. e most similar place with the track fastener clip subgraph sample is the target area. e normalized correlation coefficient [40] is used to locate the best similar region, and the subimage of track fastener clip is segmented.  MD is used to measure the distance between a sample point X and a set of data distribution D. X is represented as a sample point. D is represented as the set of data. u is represented as the mean value of D. v is represented as the covariance of D. en the MD between the sample and the set is

Track Fastener Defect Detection
e covariance matrix, inverse matrix, and mean vector of training set are calculated, and then MD is calculated. e maximum value of the training set MD is selected as the detection threshold, and the image to be tested is processed in the same way to calculate the MD. If MD is less than or equal to the threshold value, the test image is judged as normal fastener, and if MD is greater than the threshold value, the test image is judged as abnormal fastener.
Flow chart of detection of defects in Pandrol track fasteners based on LDFFN is shown in Figure 3. e first part is to import the training set of track fastener image. Firstly, the track fastener image is segmented by the automatic segmentation algorithm of track fastener image to obtain the track fastener clip subgraph. Secondly, AE and RBM are used to extract the subimage features of track fastener clip, and then the extracted subimage features of track fastener clip are fused. Finally, MD of fusion features is calculated, MD of training set is selected as threshold of detection image, and feature fusion network model is saved after training. e second part is the test part. Firstly, the test image of track fastener image is imported. Secondly, the training set image is segmented by the automatic segmentation algorithm of track fastener image, and the training set image of track fastener clip subimage is obtained. Finally, through the trained feature fusion network model, the MD of each image to be tested is calculated to determine the status of track fastener image. If the calculated MD is less than or equal to the threshold value, the tested track fastener image is judged as normal track fastener image. If the calculated MD is greater than the threshold value, the tested track fastener image is judged as abnormal track fastener image. (1) Segmentation rate (SR): N O is represented as the number of original images, and N S is represented as the number of pictures successfully segmented [41].

Experiments and Applications
(2) False alarm rate (FAR), missed alarm rate (MAR), and error rate (ER): D A A indicates that the abnormal track fastener detection result is abnormal; D N A indicates that the abnormal track fastener test result is normal; D N N indicates that the normal track fastener test result is normal; D A N indicates that the normal track fastener test result is abnormal [41].

Reconstruction of Picture Features of Track Fasteners.
Feature reconstruction images of track fastener are shown in Figure 5.

Complexity
For AE extraction of track fastener image features, two coding layers and two decoding layers are designed in AE network. When encoding, the image dimension is reduced to 256 dimensions. rough the first coding layer, the ReLU activation function is used to output 128 dimensions; through the second coding layer, the ReLU activation function is used to output 64 dimensions; and then the output of the second encoding is used as the input of decoding. e 256-dimensional image features are output by two decoding layers. Training AE model, set the training cycle to 100 and the bit size to 100 and save the model. When RBM is used to extract the image features of track fastener, RBM is called directly, and the training period is set to 100 and the bit size is 100. en, the track fastener image features extracted by AE and the track fastener image features extracted by RBM are fused in serial, and the covariance matrix, inverse matrix of covariance matrix, and mean vector of training set are calculated, and then MD is calculated.
e defects detection results of Pandrol track fasteners based on DFFN are shown in Table 1 In order to find out the reason, the automatic segmentation algorithm of track fastener image is used to segment the image, and only the track fastener clip subimage is used for detection. e result shows that the same picture of track fasteners is an abnormal picture when the full picture is used for detection. e test result is a normal picture when only the track fasteners clip subimage is used for testing. In fact, the detected picture is a normal track fastener image. e results show that the defect detection of track fasteners is interfered by the features of the bolt subgraphs of track fasteners.
e characteristic interference verification diagram of the bolt subgraph of the Pandrol track fastener is shown in Figure 6.

Experimental Results of Automatic Segmentation of Pandrol Track Fastener Pictures.
e input image of track fastener is processed by gray processing, gray histogram equalization, and Gaussian filtering, and the normalized correlation coefficient is used to locate the clip area of the track fastener and segment it. e segmentation results of the Pandrol track fastener pictures are shown in Table 2. e experiment was compared and analyzed. Experiments show that it is better to segment the picture after grayscale, histogram equalization, and Gaussian filtering. Pictures that are unsuccessfully segmented are defined as false alarm image in track fastener detection and counted in the detection false alarm rate.

Defect Detection Results of Pandrol Track Fasteners
Based on LDFFN. For the left and right fasteners, 300 normal track fastener clip subimages were used for training, and the remaining clip subimages were used for detection. e data set is the same as DFFN. e defect detection results of Pandrol track fasteners based on LDFFN are shown in Table 3. For the track fastener of Pandrol, the false alarm rate and error rate of track fastener defect detection method based on LDFFN are effectively reduced compared with the track fastener defect detection method based on DFFN. It is proved that the defect detection of track fasteners is interfered by the

Conclusion
In     When the defect detection of track fastener is carried out through the track fastener picture, the information of the bolt subpicture of track fastener is not effectively used, and whether the bolt is loose cannot be detected. In the future, the research will attempt to detect the track fastener bolts by other detection methods, so that the information of the track fastener bolts can be effectively used.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.