Wear particle image analysis is an effective method to detect wear condition of mechanical devices. However, the recognition accuracy and recognition efficiency for online wear particle automatic recognition are always mutual restricted because the online wear particle images have almost no texture information and lack clarity. Especially for confusing fatigue wear particles and sliding wear particles, the online recognition is a challenging task. Based on this requirement, a super-resolution reconstruct technique and partial hierarchical convolutional neural network, SR-PHnet, is proposed to classify wear particles in one step. The structure of this network is composed by three modules, one is super-resolution layer module, the second is convolutional neural network classifier module, and the third is support vector machine (SVM) classifier module. The classification result of the second module is partial input to the third module for precision classification of fatigue and sliding particles. In addition, a new feature of radial edge factor (REF) is put forward to target fatigue and sliding wear particles. The test result shows that the new feature has the capability to distinguish fatigue and sliding particles well and time saving. The comparison experiments of the convolution neural network (CNN) method, support vector machine method (SVM) with and without REF feature, and integrated model of back-propagation (BP) and CNN are produced. The comparison results show that the online recognition speed and online recognition rate of the proposed SR-PHnet model in this paper are both improved markedly, especially for fatigue and sliding wear particles.
National Study Abroad Fund of ChinaNational Key Research and Development Program of China2017YFB1002304Fundamental Research Funds for the Central UniversitiesFRF-GF-20-16B1. Introduction
In the mechanical system, most failures are caused by severe friction or wear. Wear particle monitoring and analysis has been deemed as a powerful technology for machine wear assessment [1, 2]. It is considered that wear particles imply important clues for wear rate and wear mechanism [3]. Many methods [4–6] are used to recognize the features of wear particles. Among these techniques, image analysis is regarded as a very effective method for its easy acquiring and rapid processing speed [7–9].
The processing methods are undergoing through two generation approaches. The first generation is denoted by feature engineering [10–13]. Yuan et al. [11] analyzed the boundary of wear particles, extracted a variety of different boundary parameters, and used four different machine learning models to explore the performance of the model. Xu et al. [13] constructed a three-level classification model including K-means and support vector machine (SVM) by extracting the color and geometric features of wear particles. The model can classify different wear particles on different classifiers. Myshkin et al. [14] analyzed the color characteristics of metallic wear debris to recognize different wear debris. Stachowiak et al. [15] statistically analyzed the surface texture parameters of wear particles and designed an automatic classification system based on the surface texture of wear particles. Wang and Wang [16] combined principal component analysis (PCA) and grey relational analysis (CPGA) to identify different wear particles. CPGA is adopted to solve the complicated interrelationships between the specified characteristics of similar wear particles with features of color, shape factor, texture, and so on. However, there are two problems in the methods based on manual feature extraction. Firstly, the selection of wear particle features requires deep insight into both domain-expert knowledge and the learning algorithms. Secondly, feature extraction is not adaptive. When the dataset is transformed, the original features may reduce the recognition rate of the model.
With the development of machine learning, the second generation is denoted by deep learning networks [17–23]. Peng and Wang [21] proposed a small sample wear particle recognition model based on CNN. Compared with the general CNN model, this model can identify overlapping particles in the case of a small number of samples and uses one-dimensional convolution to reduce the computational complexity of the model. Peng et al. [22] proposed a hybrid CNN network to automatically classify four different wear particles. The hybrid model uses the idea of transfer learning to initialize the model parameters by using the learning parameters and weights of ImageNet. Aiming at the lack of enough information in two-dimensional wear particle images, Wang et al. [24] proposed a nonparametric recognition model of three-dimensional similar particles by taking fatigue and sliding wear particles as examples. This model can transform 3D image with rich information into 2D image and uses the CNN model to optimize. Peng and Wang [25] used the Inception-v3 network structure to automatically extract the characteristics of wear particles and then used the artificial neural network to classify the types of wear particles, which can realize the classification of overlapping particles. Liu et al. [26] constructed a deep convolutional neural network with Encoder-ASPP-Decoder architecture, which can classify and segment concurrently five different wear particles in ferrography images. However, in these works, the images are high definition ferrography images which have rich texture information. It is not economical and practical by using these kinds of acquisition equipment for online wear particles monitoring and analysis.
Online wear particle images acquiring and analysis make wear condition monitoring real time and straightforward by avoiding frequent oil sampling. Optical techniques using LED or laser beam and a charge-coupled device (CCD) sensor to acquire wear particle images are not new [1, 12, 27]. When particles pass through the view of light beam, their morphology can be captured by using the camera. However, the analysis of the online images is remained on the first-generation approach—feature engineering level, that is, because the online wear particle images are of low resolution and no texture information contained. Figure 1 shows the difference of high definition ferrography images (extracted from Ref. [18]) and online wear particle images. The classification of online low-resolution wear particle images by using the deep learning method is still not adequately addressed.
This paper introduces a partial hierarchical model of deep learning convolution neural network assembling with the feature engineering method, which can process and classify online low-resolution wear particle images in one-stop manner. The proposed network structure is called SR-PHnet. It includes three parts, super-resolution layer, the first-stage classifier module, and the second-stage classifier module. Firstly, the low-resolution images are transformed into super-resolution images by super-resolution image reconstruction in super-resolution layer. Then, the super-resolution images are input to the classification module. The partial hierarchical mechanism is put forward with different classification modules to precision recognize the easy recognition particles (e.g., cutting, sphere, and nonmetallic particles) and difficult recognition particles (e.g., fatigue and sliding particles). The easily confused fatigue and sliding wear particles are identified as one class on the convolution neural network classification module and then recognized separately on the SVM classifier module of the SR-PHnet model. Firstly, convolutional neural network is adopted for classifying four categories (cutting, sphere, nonmetallic, and fatigue or sliding particles); then, fatigue and sliding particle images are inputted into the second classifier for further recognition. A new feature, called radial edge factor (REF), is proposed, which is combined with area, aspect ratio, and roundness to identify fatigue and sliding particles better. The main contributions of this work are as follows:
The super‐resolution layer, the first-stage classifier module and the second-stage classifier module are integrated together to realize unclear online wear particles images in one step.
The partial hierarchical modeling method is effective for easy recognition particles and difficult recognition particles (e.g., fatigue and sliding particles). The former is recognized by convolution neural network, and the latter is classified by the feature engineering method.
Radial edge factor (REF) feature, which can be decoupled from various features, is put forward specially for fatigue and sliding particles.
The mechanism of partial hierarchical modeling determines that the modeling scale and the number of features are smaller but more effective than recent modeling methods for online wear particle classification. It provides higher recognition rate and less computing time.
The detailed procedure of modeling is described in Section 2. The experiments and discussion are explained in Section 3 and Section 4.
2. The Proposed SR-PHnet for Online Low-Resolution Wear Particle Images Classification
The overall structure of the proposed SR-PHnet is shown in Figure 2.
The network structure of the proposed SR-PHnet model.
2.1. Super-Resolution Layers
Since the online collected wear particle image is with low resolution (LR), many details are lost compared with the ferrography wear particle image. When the low-resolution image is directly used as the input of CNN, the prediction accuracy is relatively low. Therefore, we hope to recover some image details from low-resolution wear particle images, and image super-resolution reconstruction [28] is used to achieve this effect. Image super-resolution reconstruction is a technique to reconstruct a high-resolution (HR) image by using one or more low-resolution images based on certain assumptions or prior information [29]. In this paper, the single image super-resolution reconstruction (SISR) [30] technology is used to achieve the goal of image reconstruction. SISR techniques are generally classified into three categories: interpolation-based, reconstruction-based, and sample-based learning. Bicubic interpolation [31] is one of the most common methods, but it cannot reconstruct better image details. When the magnification factor is large, the image obtained by bicubic interpolation will be blurred at the edge. The reconstruction-based method [32] focuses on reconstructing lost high-frequency signals. This method only uses some prior knowledge to regularize the reconstruction process and does not use training ideas to obtain prior information. The learning-based method [33, 34] uses machine learning or deep learning algorithm to learn the mapping relationship between the LR image and the HR image to predict the missing high-frequency information of the image to reconstruct the HR image. The learning-based SISR method is used in this paper to reconstruct the low-resolution wear particle image.
As shown in Figure 2, three convolutional layers are used to achieve super-resolution reconstruction. Each convolution layer does not change the size of the image but change the number of channels of the image. After passing three convolutional layers, the original image size and channel number are restored. The convolution kernel sizes of the three convolutional layers are 7 × 7, 5 × 5, and 3 × 3, respectively, and the zero-padding required is 3, 2, and 1, respectively. The kernel stride is 1, and ReLU activation function is used for each layer. The output size of each layer is as follows:(1)o=i−k+2ps+1.
Here, i is the size of the input image, k is the convolution kernel size of the layer, p is the zero-padding number, and s is the kernel stride.
The input of the super-resolution layer is the original low-resolution (LR) image, and the output is the changes required to convert the low-resolution (LR) image into the super-resolution (SR) image, that is, the pixel difference between the SR image and the LR image. Since the SR image needs to match the input size of the convolutional classification layer, we apply the bicubic interpolation method for each LR image to enlarge it into the matched size. Then, through the super-resolution layer, we obtain the difference image and add it to the corresponding LR image to obtain the transformed SR image. The SR image ISR is solved by the following formula:(2)ISR=ILR+sILR,where ILR represents the amplified low-resolution (LR) image through bicubic interpolation and sILR represents the output of the super-resolution layer.
The super-resolution layer learns the mapping function from the low-resolution image to the difference between the super-resolution image and the low-resolution image [35]. In order to learn the mapping function, we propose the cost function JSR for the super-resolution layer as follows:(3)JSR=12∑i=1KIHR−ISR2=12∑i=1KIHR−ILR+sILR2,where IHR is the high-resolution form of image I and ISR is the super-resolution reconstruction image obtained after image I passes through the super-resolution layer.
The optimization objective of the super-resolution layer is to minimize the cost function value JSR. We use ImageNet datasets to train the super-resolution layer. The high-resolution images IHR in the ImageNet dataset are first down-sampled to generate the low-resolution imagesILR and then amplified to the original image size to obtain the difference ΔIHR−ILR between it and the original high-resolution image.
2.2. The CNN Classifier Module of SR-PHnet Model
In view of the advantage that no need to extract features manually for images classification by the deep learning method [36–39], the proposed SR-PHnet model in this paper uses convolution neural network as the first classifier to recognize wear particles automatically. There contains convolutional layer, pooling layer, and fully connected layer [36] in first-stage classifier. The images after super-resolution layer are inputted to the convolutional layer as shown in Figure 2. The output is the classification of wear particles. Here, we use four categories, cutting, sphere, nonmetallic, and FoS particles (for convenience of expression, FoS is used to as the abbreviation of fatigue or sliding particles). The new portable convolutional neural network ShuffleNetV2 [40, 41] structure is used to classify the wear particles. This structure uses block convolution instead of traditional convolution layer, which greatly accelerates the training and testing process of the network.
2.2.1. Convolutional Layer
Convolution layer is convoluted by multiple convolution kernels. Convolution kernel is generally composed of a three-dimensional weight tensor and a bias. It is assumed that the input tensor is Tin∈RW1∗H1∗L1, where RW1∗H1∗L1represents a three-dimension real number tensor of size W1∗H1∗L1. W1,H1, and L1 are width, height, and depth (also called channel number), respectively. The output of convolution layer is also a three-dimensional tensor, which is recorded as Tout∈RW2∗H2∗L2, where RW2∗H2∗L2 represents a three-dimension real number tensor of size W2∗H2∗L2. Then, the convolutional operation is calculated as follows:(4)yi,j,k=fCWk∗X+bk=fC∑l=1LC∑i′=1WC∑j′=1HCWi′,j′,lkxi−1∗s+i′∗s+j′,l+bk,where yi,j,k is the value of Tout at position i,j,k, Wk is the weight tensor of the kth convolution kernel of the convolution layer, bk is the bias vector of the kth convolution kernel of the convolution layer, and Xis the input tensor, for the first convolution layer, X=Tin. WC, HC, and LC are the width, height, and depth of the kth convolution kernel, respectively. The total parameter of the kth convolution kernel isWC∗HC∗LC+l, Wi′,j′,lk is the value of Wk at position i′,j′,l, xi−1∗s+i′∗s+j′,l is the value of X at position i−1∗s+i′,j−1∗s+j′,l, blk is the value of bk at position l, s is the stride of moving the convolution kernel, i.e., it is calculated by adding across s positions, and fC is a function. ReLU [36] operation is adopted, and its calculation is expressed as follows:(5)fCx=max0,x,where x is the output of the upper layer network.
2.2.2. Pooling Layer
The pooling layer is a layer without parameters. The output of general pool layer is also a three-dimensional tensor. Generally, there are maximum pooling layer and average pooling layer. The maximum pooling operation is generally used in the pooling layer. The maximum pooling operation can be described by a mathematical formula as follows:(6)pp,q,k=maxysp∗p−1+1:sp∗p−1+hp,sq∗q−1+1:sq∗q−1+wp,k,where sp is span and hp and wp are width and height of neighborhood, respectively.
2.2.3. Full Connect Layer
The full connection layer is generally composed of weight matrix and offset for vector input. Suppose the input is x=x1,x2,…,xn. The weight matrix of all connected layer is W∈Rm∗m, and the offset is b∈Rm. The output is A=a1,a2,…,am, and they are described as follows:(7)a1=W11∗x1+W12∗x2+⋯+W1nxn,a2=W21∗x1+W22∗x2+⋯+W2nxn,…,am=Wm1∗x1+Wm2∗x2+⋯+Wmnxn.
2.3. The SVM Classifier Module of SR-PHnet Model
After the first classification, cutting, sphere, nonmetallic, and FoS (fatigue or sliding) particles are recognized automatically without the guidance of feature manual extraction. The objective of the SVM classifier module of the SR-PHnet model is to recognize fatigue or sliding particles. The partial hierarchical mechanism can reduce the model scale, save computing time, and enhance the recognition accuracy.
To improve the recognition accuracy of fatigue and sliding wear particle images, feature extraction is used for guidance of classification. The normal features for recognizing different wear particles are area, aspect ratio, roundness, shape factor, and so on. The recognition rate for all wear particles by only using the feature engineering method is not high. One of the reasons is that the multiple features been coupled together make it hard to distinguish all kinds of wear particles and make the model complicated. Characteristic feature is beneficial for improving recognition rate and reducing model scale. For just aiming at fatigue or sliding particles, it is possible to make features decoupling from the others and reduce the number of features to complete the identifying task. For fatigue or sliding particles, a new feature, radial edge factor (denoted as REF), is put forward in this paper. The schematic diagram is shown in Figure 3. The brown dotted circle in Figure 3 is an analytically equivalent circle (AEC) [11] with equal perimeter. It is the ratio of the characteristic radius to the minimum radius of the inscribed circle. It is denoted as follows:(8)REF=CRr.
The representation diagram of wear particle feature REF.
Here, CR is the characteristic radius of wear particles and r is the minimum radius of the inscribed circle as shown in Figure 3. The characteristic radius CR is calculated as follows:(9)CR=P2π.
Here, P is perimeter of wear particle. Eight neighborhood distances are used as the perimeter of wear particles by accumulating edge pixel points. It is slightly different from the actual perimeter drawn as the line, but it has short computing time and little influence on the value.
REF can reflect the irregularity of wear particles. When two wear particles have the same size of the inscribed circle, the longer the perimeter of the wear particles, the severe the degree of irregularity. When the perimeters of wear particles are the same, the larger the inscribed circle is, the less irregular the wear particles are. With the help of REF feature, the model can better judge the sliding and fatigue particles. The larger the value of REF feature, the more serious the irregularity of the particle.
Combine this feature REF with the other features, areaA, aspect ratioAR, and roundness α, to build the feature engineering model for recognizing fatigue or sliding particles. The feature definitions are listed in Table 1. A is the area from the number of pixels of wear particle image. AR is the ratio of length to width. It is calculated as follows:(10)AR=LW.
Feature definitions of wear particles.
Feature
Definition
Area: A
The envelope pixels of wear particle image
Aspect ratio: AR
The ratio of length to width
Roundness: α
The similarity between the shape of wear particles and the circle
Radial edge factor: REF
The ratio of characteristic radius of wear particle to the minimum radius of inscribed circle
Here, L is the length of the smallest circumscribed rectangle surrounding the wear particle and W is the width of the smallest circumscribed rectangle surrounding the wear particle.
Roundness of wear particle is the similarity between the shape of wear particles and the circle. It is denoted as follows:(11)α=4Aπ∗P2.
Here, Ais particle area and P is perimeter of wear particles.
The SVM algorithm [42, 43] is adopted as the feature engineering modeling, and the outputs are two classifications, fatigue and sliding wear particles. For the two-classifier, the kernel function used is the Gaussian kernel function RBF, and the 3-fold cross validation is used to verify the generalization ability of the model.
3. Experiments3.1. Dataset
Online images acquisition is carried out with the CCD camera. The typical labeled images are shown in Figure 4. The collected image dataset is divided into two parts. One is for offline training to develop the SR-PHnet model, and the other is as test dataset for online recognition and classification. 1408 samples are collected, including 281 nonmetallic particles, 486 sliding particles, 439 cutting particles, 459 fatigue particles, and 95 sphere particles.
For super-resolution layer, the convolution kernel sizes of the three convolutional layers are 7 × 7, 5 × 5, and 3 × 3, respectively, and the zero-padding required is 3, 2, and 1, respectively. The kernel stride is 1. After super-resolution reconstruction, the images are transformed to the resolution of 224 × 224 × 3 and are input into the CNN classifier module of the SR-PHnet model for training. The structure settings of the super-resolution layer and the convolution neural network classifier are listed in Table 2. For the convolution neural network classifier, in the training, the batch size is 64, the learning rate is 0.0004, and the iteration number is 2000. The output of the CNN classifier includes four categories, i.e., cutting, sphere, nonmetallic, and fatigue or sliding (FoS) particles. The classification result is listed in Table 3. It can be seen that the recognition accuracy of cutting, sphere, and nonmetallic particles is high. Even for FoS particles, it is high to 92.5%, that is, because the automatic feature extraction in the SR-PHnet model is suitable for distinctive wear particle images.
The structure setting of all unit layers of super-resolution layer and the CNN classifier of the SR-PHnet model.
Structure
Layer
Output size
Kernel size
Stride
Repeat
Output channels
Super resolution
Conv1
7 × 7
1
Conv2
5 × 5
1
Conv3
3 × 3
1
SR image
224 × 224
3
CNN classifier
Conv1
112 × 112
3 × 3
2
1
24
MaxPool
56 × 56
3 × 3
2
Block2
28 × 28
2
1
116
28 × 28
1
3
Block3
14 × 14
2
1
232
14 × 14
1
7
Block4
7 × 7
2
1
464
7 × 7
1
3
Conv5
7 × 7
1 × 1
1
1
1024
GlobalPool
1 × 1
7 × 7
The four-classification result of wear particles in the CNN classifier of SR-PHnet.
Total
Correct
Recognition rate (%)
Cutting
94
88
94
Sphere
15
15
100
Nonmetallic
57
54
95
FoS
186
172
92.5
After CNN classifier module of the SR-PHnet model, fatigue or sliding wear particles are transferred to the SVM classifier module of the SR-PHnet model. The SVM classifier distinguishes FoS particles with four features. There are 941 samples in use, including 483 sliding grains and 458 fatigue grains. Using 3-folded cross validation, 142 sliding wear particles and 141 fatigue wear particles were randomly selected as test set and 341 sliding wear particles and 317 fatigue wear particles as training set. The recognition accuracy is listed in Table 4. It shows that just four features, including the new feature REF proposed in this paper, have a good performance for distinguishing fatigue or sliding wear particles. The recognition accuracy of fatigue particles and sliding particles is up to 92.9% and 91.5%, respectively.
The classification result of FoS wear particles in the SVM classifier of the SR-PHnet model.
Total
Correct
Recognition rate (%)
Fatigue
141
131
92.9
Sliding
142
130
91.5
Finally, the recognition accuracy of the SR-PHnet model for all the five types of wear particles is listed in Table 5. From the recognition result, the average recognition accuracy is up to 92.7%, and the accuracy for fatigue and sliding particles of online recognition is 85.9% and 84.6%, respectively. It has great progress in online wear particles recognition. The recognition accuracy is just 85.7% and 80% for fatigue and sliding particles in the offline model of Wang et al.’s [23] with high definition ferrography images. Our model is preferable for its lightweight computing and high recognition accuracy on low-resolution images.
The five classification results of wear particles with the SR-PHnet model.
Total
Correct
Recognition rate (%)
Cutting
94
88
94
Sphere
15
15
100
Nonmetallic
57
54
95
Sliding
98
65
84.6
Fatigue
88
74
85.9
4. Discussion
For distinguishable wear particles with distinctive features (e.g., cutting, sphere, and nonmetallic wear particles), the convolutional neural network method is preferred. Because it recognizes particles directly without feature engineering, it has high recognition accuracy and short computing time. For specific particles that are difficult to identify (e.g., fatigue and sliding wear particles), feature engineering is a good choice for recognition. So, the partial hierarchic method is the optimal combination by using convolutional neural network as the first classifier module and feature engineering method SVM as the second classifier module.
To compare the recognition performance, the method of the baseline standard CNN model (AlexNet, ResNet50, and ShuffleNetV2), SVM without REF feature, SVM with REF feature, and our SR-PHnet model proposed in this paper is executed, respectively. The comparison results with AlexNet, ResNet50, and ShuffleNetV2 are listed in Table 6. The comparison of recognition precision with AlexNet, SVM without REF feature, SVM with REF feature, and our SR-PHnet is shown in Figure 5. Not only the total sample recognition precision but the fatigue and sliding particles recognition precision, the method of our SR-PHnet model has the best performance as shown in Figure 5. The SVM algorithm has better performance than the AlexNet model for sliding and fatigue particles, that is, because feature extraction manually can customize the feature parameters and it is better than feature extraction automatically for confusing recognition objects. SVM with the REF feature method proposed in this paper has better performance on recognizing FoS particles than SVM without the REF feature. It shows that REF feature has special good performance in recognizing FoS particles than the other normal features.
The comparison results between SR-PHnet, AlexNet, ResNet50, and ShuffleNetV2.
Model
Class
TP
FP
FN
Precision
Recall
F1-score
Ours
Nonmetallic
55
6
2
0.902
0.965
0.932
Sliding
78
18
20
0.813
0.796
0.804
Cutting
80
0
14
1.000
0.851
0.920
Fatigue
82
18
6
0.820
0.932
0.872
Bubble
15
0
0
1.000
1.000
1.000
Total sample
310
42
42
0.881
0.881
0.881
AlexNet
Nonmetallic
51
8
6
0.864
0.895
0.879
Sliding
69
28
29
0.711
0.704
0.708
Cutting
76
4
18
0.950
0.809
0.874
Fatigue
69
30
19
0.697
0.784
0.738
Bubble
15
2
0
0.882
1.000
0.938
Total sample
280
72
72
0.795
0.795
0.795
ResNet50
Nonmetallic
54
6
4
0.900
0.931
0.915
Sliding
75
20
23
0.789
0.765
0.777
Cutting
81
1
13
0.988
0.862
0.920
Fatigue
78
21
10
0.788
0.886
0.834
Bubble
15
2
0
0.882
1.000
0.938
Total sample
303
50
50
0.858
0.858
0.858
ShuffleNetV2
Nonmetallic
54
7
3
0.885
0.947
0.915
Sliding
73
23
25
0.760
0.745
0.753
Cutting
79
3
15
0.963
0.840
0.898
Fatigue
75
22
13
0.773
0.852
0.811
Bubble
15
1
0
0.938
1.000
0.968
Total sample
296
56
56
0.841
0.841
0.841
The recognition rate of AlexNet, SVM (no REF), SVM (REF), and the proposed SR-PHnet model.
The SR-PHnet model proposed in this paper combines the respective advantages and avoids the disadvantages of different modeling methods for feature extraction automatically and manually. Regardless of sliding and fatigue particles recognition or the average recognition accuracy, SR-PHnet has the best performance.
The comparison of recognition accuracy with iteration times of our SR-PHnet model and AlexNet is shown in Figure 6. It shows that the SR-PHnet model needs fewer iteration times to achieve high recognition accuracy and convergence to stability.
The recognition rate of the hierarchical model with iteration times.
To compare our proposed model with the other method, the same dataset of this paper is executed by the model of Ref. [23] in the comparison experiment. The BP-CNN model in Ref. [23] is constructed. Due to the lack of texture information, the feature parameters adopted in the BP-CNN model are set as the same as the SR-PHnet model. The comparison result is listed in Table 7.
Comparison result between BP-CNN (Ref. [23]]) and SR-PHnet.
Wear particle type
BP-CNN (%)
SR-PHnet (%)
Cutting
89
94
Sphere
86.05
100
Nonmetallic
90.91
95
Fatigue
82.61
84.6
Sliding
80.21
85.9
Comparing between the BP-CNN in Ref. [23] and SR-PHnet model, the proposed SR-PHnet model has a much higher recognition accuracy than BP-CNN in the three wear particles: cutting, spherical, and nonmetal. The recognition rate of fatigue and sliding particles is also improved in the SR-PHnet model, which shows the effectiveness of the SR-PHnet model. It can be seen that the SR-PHnet model is more adaptable. Because CNN is the first classifier to automatically extract image features, it is more adaptive than manual feature extraction. In the method of BP-CNN, BP neural network is as the first classifier which needs manual feature extraction. It limits the recognition accuracy due to the error of manual feature extraction in the first step. The SR-PHnet model has higher accuracy for not only cutting, spherical, and nonmetal but also for fatigue and sliding particles.
5. Conclusions
This work aims to develop a super-resolution reconstruction partial hierarchical SR-PHnet model for online wear particle recognition and classification. The low-resolution online wear particle images lacking of texture information are firstly reconstructed by super-resolution technique, and then the super-resolution images are input to the classifier of the SR-PHnet model. The types of wear particles are merged into four categories (cutting, sphere, nonmetallic, and fatigue or sliding (FoS) particles) on the first state classifier, and then fatigue and sliding (FoS) wear particle images are transferred to the second-stage classifier. On the first stage, there are characteristic patterns existing in the four categories, so convolution neural network is the best classifier for its high recognition precision and short computing time. The recognition accuracy is up to 94%. On the second stage, the feature engineering method is the best choice for recognizing the easily confused fatigue and sliding wear particles. A new feature REF is put forward to combine with the other three features to obtain a high recognition rate. The final recognition accuracy of fatigue and sliding particles after the SR-PHnet model is up to 85.9% and 84.6%, respectively. The comparison experiment results confirm the effectiveness of the SR-PHnet model in online wear particles recognition and classification for low-resolution wear particle images lacking of texture information.
Data Availability
The original data used in the study can be made available from the corresponding author upon reasonable request.
Disclosure
Xuxu Guo and Rui Tan are co-first authors.
Conflicts of Interest
The authors declare that they have no conflicts of interest to report regarding the present study.
Authors’ Contributions
Xuxu Guo and Rui Tan contributed equally to this work.
Acknowledgments
The authors would like to express their heartfelt thanks to Yue Gao and Yue Zhao for their support and assistance in this paper. This paper was sponsored by the National Study Abroad Fund of China and supported by the National Key Research and Development Program of China (2017YFB1002304) and Fundamental Research Funds for the Central Universities (FRF-GF-20-16B).
CaoW.DongG.XieY.-B.PengZ.Prediction of wear trend of engines via on-line wear debris monitoring201812051051910.1016/j.triboint.2018.01.0152-s2.0-85041468886WakiruJ. M.PintelonL.MuchiriP. N.ChemwenoP. K.A review on lubricant condition monitoring information analysis for maintenance decision support201911810813210.1016/j.ymssp.2018.08.0392-s2.0-85052249043KumarM.MukherjeeP. S.MisraN. M.Advancement and current status of wear debris analysis for machine condition monitoring: a review201365131110.1108/003687913112927562-s2.0-84873201284GonzalezC.ViafaraC.CoronadoJ.MartinezF.Automatic classification of severe and mild wear in worn surface images using histograms of oriented gradients as descriptor2019426-4271702171110.1016/j.wear.2018.11.0282-s2.0-85057957516ZhuX.DuL.ZheJ.A 3×3 wear debris sensor array for real time lubricant oil conditioning monitoring using synchronized sampling20178329630410.1016/j.ymssp.2016.06.0142-s2.0-84995616559WuT.PengY.WuH.ZhangX.WangJ.Full-life dynamic identification of wear state based on on-line wear debris image features2014421-240441410.1016/j.ymssp.2013.08.0322-s2.0-84887052346RoylanceB. J.AlbidewiI. A.LaghariM. S.Computer-aided vision engineering (CAVE) - quantification of wear particle morphology199450111116HuX.HuangP.ZhengS.Object extraction from an image of wear particles on a complex background200616464465010.1134/s10546618060401092-s2.0-33845307255RaadnuiS.Wear particle analysis-utilization of quantitative computer image analysis: a review2005381087187810.1016/j.triboint.2005.03.0132-s2.0-23844532280WuT.PengY.WangS.ChenF.KwokN.PengZ.Morphological feature extraction based on multiview images for wear debris analysis in on-line fluid monitoring201660340841810.1080/10402004.2016.11743252-s2.0-84981266985YuanW.ChinK. S.HuaM.DongG.WangC.Shape classification of wear particles by image boundary analysis using machine learning algorithms201672-7334635810.1016/j.ymssp.2015.10.0132-s2.0-84955706838CaoW.ZhangH.WangN.WangH. W.PengZ. X.The gearbox wears state monitoring and evaluation based on on-line wear debris features2019426-4271719172810.1016/j.wear.2018.12.0682-s2.0-85059457912XuB.WenG.ZhangZ.ChenF.Wear particle classification using genetic programming evolved features201830522924610.1002/ls.14112-s2.0-85040245595MyshkinN. K.KongH.GrigorievA. Y.YoonE.-S.The use of color in wear debris analysis20012511–121218122610.1016/s0043-1648(01)00751-72-s2.0-0035482706StachowiakG. P.PodsiadloP.StachowiakG. W.Shape and texture features in the automated classification of adhesive and abrasive wear particles2006241152610.1007/s11249-006-9117-52-s2.0-33750825569WangJ.WangX.A wear particle identification method by combining principal component analysis and grey relational analysis20133041-29610210.1016/j.wear.2013.04.0212-s2.0-84878397937PengY. P.WuT. H.CaoG. Z.A hybrid search-tree discriminant technique for multivariate wear debris classification2019392-39315215810.1016/j.wear.2017.09.0222-s2.0-85030717739PengY.CaiJ.WuT.CaoG.KwokN.PengZ.WP-DRnet: a novel wear particle detection and recognition network for automatic ferrograph image analysis202015110637910.1016/j.triboint.2020.106379ChenY.JiangH.LiC.JiaX.GhamisiP.Deep feature extraction and classification of hyperspectral images based on convolutional neural networks201654106232625110.1109/tgrs.2016.25841072-s2.0-84978805819ZuoZ.ShuaiB.WangG.Learning contextual dependence with convolutional hierarchical recurrent neural networks20162572983299610.1109/tip.2016.25482412-s2.0-84971671959PengP.WangJ.FECNN: a promising model for wear particle recognition2019432-43320296810.1016/j.wear.2019.2029682-s2.0-85068857843PengY.CaiJ.WuT.A hybrid convolutional neural network for intelligent wear particle classification201913816617310.1016/j.triboint.2019.05.0292-s2.0-85066504870WangS.WuT. H.ShaoT.PengZ. X.Integrated model of BP neural network and CNN algorithm for automatic wear debris classification2019426-4271761177010.1016/j.wear.2018.12.0872-s2.0-85059350260WangS.WangT.ZhengP.KwokN.Optimized CNN model for identifying similar 3D wear particles in few samples2020460-46120347710.1016/j.wear.2020.203477PengP.WangJ.Wear particle classification considering particle overlapping2019422-42311912710.1016/j.wear.2019.01.0602-s2.0-85060123122LiuX.WangJ.SunK.ChengL.WuM.WangX.Semantic segmentation of ferrography images for automatic wear particle analysis202112210526810.1016/j.engfailanal.2021.105268KrasmikV.RöbkenN.MartinC.MartinP.SchlattmannJ.Characterising the friction and wear behaviour of lubricated metal-metal pairings with an optical online particle detection system201830520722810.1002/ls.13762-s2.0-85011672094CapelD.ZissermanA.Computer vision applied to super resolution2003203758610.1109/msp.2003.12032112-s2.0-85032751770HaT.TinnefeldP.Photophysics of fluorescent probes for single-molecule biophysics and super-resolution imaging201263159510.1146/annurev-physchem-032210-1033402-s2.0-84859899534Van OuwerkerkJ. D.Image super-resolution survey200624101039105210.1016/j.imavis.2006.02.0262-s2.0-33747070740KeysR.Cubic convolution interpolation for digital image processing19812961153116010.1109/tassp.1981.11637112-s2.0-0019654284KatartzisA.PetrouM.Current trends in super-resolution image reconstruction20081New York, NY, USAAcademic PressDongC.LoyC. C.HeK.TangX.Image super-resolution using deep convolutional networks201638229530710.1109/tpami.2015.24392812-s2.0-84962128851KimJ.Kwon LeeJ.Mu LeeK.Accurate image super resolution using very deep convolutional networks1Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionJune 2016Las Vegas, NV, USA16461654SuH.ZhouJ.ZhangZ. H.Survey of super-resolution image reconstruction methods20133912021213KrizhevskyA.SutskeverI.HintonG. E.ImageNet classification with deep convolutional neural networksProceedings of the International Conference on Neural Information Processing Systems2012NV, USALake TahoeSmirnovE. A.TimoshenkoD. M.AndrianovS. N.Comparison of regularization methods for imagenet classification with deep convolutional neural networks201368994CirsteaB.-I.Likforman-SulemL.Improving a deep convolutional neural network architecture for character recognition20162016171710.2352/issn.2470-1173.2016.17.drr-060SongL. I.WeiZ.ZhangB.Target recognition using the transfer learning based deep convolutional neural networks for SAR images2018357583SzegedyC.VanhouckeV.IoffeS.ShlensJ.WojnaZ.Rethinking the inception architecture for computer vision2016128182826MaN.ZhangX.ZhengH. Y.SunJ.Shufflenet v2: practical guidelines for efficient cnn architecture design201811218116131CortesC.VapnikV.Support-vector networks199520327329710.1007/bf00994018TangW. M.ZhangY.WangP.A new fuzzy support vector machine based on the grey relational grade to solve the two class problems20124371378