Confirmation of Thermal Images and Vibration Signals for Intelligent Machine Fault Diagnostics

1 Department of Mechanical Engineering, Faculty of Engineering, Diponegoro University, Jalan Professor Sudarto, Tembalang, Semarang 50275, Indonesia 2 Department of Mechanical and Automotive Engineering, Pukyong National University, San 100, Yongdang-dong, Nam-gu, Busan 608-739, Republic of Korea 3 Department of Energy and Mechanical Engineering, Institute of Marine Industry, Gyeongsang National University, 445 Inpyeong-dong, Gyeongnam-do, Tongyoung City 650-160, Republic of Korea


Introduction
Machine condition monitoring and fault diagnostics are very important for machineries in the industry to guarantee high performance of their functionality.High reliability is needed to reduce product and profit loss because of unwanted downtime.Therefore, machine condition monitoring and fault diagnostics become a critical issue in maintenance activity to ensure availability, minimize operator hazard, and reduce economic losses [1].For this purpose, several sensors such as vibration sensors, acoustic emission sensors, and temperature sensors are used to fulfill the requirement for machine condition monitoring procedure [2].The assessment of machine components of critical rotating machines, such as bearings (journal or rolling element), using such sensors is significant for early detection of machine condition.
The application of vibration sensors as fundamental tools for machine condition monitoring has been used extensively over a period of approximately four decades [3].The reason to use these sensors was their effectiveness of measurement process and data analysis that can represent the machine conditions.Vibration signal monitoring of installed bearings in rotating machines is still effective to obtain the machine's behavior.Such monitoring could extract the machine conditions through signals analysis in time and frequency domains.Machine fault diagnostics technique using vibration sensors has been reported by many researchers including application of intelligent system in their proposed method [4][5][6][7].
International Journal of Rotating Machinery Machinery monitoring and diagnostics by means of acoustic emission (AE) sensor have also been conducted by experts with their own proposed techniques [8][9][10].Their works were mostly applied to slow speed bearings monitoring and fault diagnostics.According to published results, AE sensor was more sensitive than vibration sensor in monitoring low speed bearings.The use of AE sensor, along with intelligent technique, that is, relevance vector machine (RVM), for help in machine diagnostics process using such sensor was reported by Widodo et al. [11].Another research in this area was reported by El-Ghamry et al. [12] in which he used the automated pattern-recognition procedure to perform fault diagnostics of reciprocating machines by AE sensor.
The use of temperature monitoring for journal bearings has been reported by Glavatskih et al. [13] and Glavatskih [14].Their work did not employ any intelligent systems for monitoring and diagnostics.Temperature monitoring by means of thermocouple sensor is usually used as a secondary sensors instead of vibration and AE sensor.Therefore, the reports which employ intelligent diagnostics using such sensor are rarely found.
Another method that is recently popular in machine condition monitoring and fault diagnostics is the thermography technique.The good performance and simplicity to use have made this technique highly popular in the engineering maintenance area.This technique has been used in wide range area such as the mechanical, electrical, petrochemical, material, medical, and structure and building areas, and so forth [15].In the case of machine condition monitoring and diagnostics, the use of such method has been reported by Mazioud et al. [16] in which fault detection of rolling element bearing degradation using infrared thermography was performed.The implementation of intelligent system use thermography has been proposed by Fidali [17].He proposed a continuous monitoring using thermography camera and fault-diagnostics based statistical texture analysis.His work employed Fisher criterion and minimization of error classification for diagnostics procedure.Younus et.al.[18] reported evaluation on thermal data image for machine diagnostics.Image features are extracted by principal component and independent component analysis.Their proposed system was augmented by support vector machine (SVM) for diagnostics process.A recent publication by Younus and Yang [19] implemented feature extraction of images by two-dimensional discrete wavelet transform.The intelligent system used the SVM for machine diagnostics.
In this paper, confirmation of thermal images and vibration signals for intelligent machine fault diagnostics is studied.The aim of this work is to find a new alternative source feature and a machine diagnostics technique by utilizing thermal images and SOM.Thermal images are processed by the thresholding method for image segmentation and clustered by means of k-means algorithm.Salient features are then extracted from binary images after images processing.The extracted features used for SOM training are area, perimeter, and central moment of region of interest (ROI).After SOM diagnostics, the result will be compared with the vibration-based method for validation purposes.

Basic Theory
2.1.Thermal Image.Infrared camera does not measure directly a temperature of surface.The detectors of the camera are sensitive to the luminance emitted by the object being studied.This luminance is naturally dependent on the temperature of the surface.Thermal image is an image which is captured by infrared thermography camera.This device can capture electromagnetic spectrum within infrared bands, 0.78-1000 μm.A thermal image is a function of radiated energy on an inspected object [15].This image can be digitized and represented in matrix form for computational processing as follows: . (1)

Image Thresholding.
Image thresholding is mostly used for image segmentation [21,22].Thresholding is also common method for defect detection of material by IRT image.
Thresholding is a process to separate objects from its background in a digital image.Histogram is the main tool in this separation process.Suppose that the grey level corresponds to an image f (x, y) that is composed of lights objects on a dark background, in such a way that object and background pixels have grey levels grouped into two dominant modes.
Extracting the object from the background is performed by selecting a threshold T that separates these modes.A thresholded image g(x, y) is defined as  The method of global threshold is selected in this paper where the grey-level histogram is calculated from the entire image [22].containing one more object, where f ( j, k) = 1 if a pixel is part of the object and f ( j, k) = 0 for all nonobject or background pixels.The bit-quads [23] method is adopted to determine the perimeter and area of ROI.The moment of order (p + q) of an image [24], denoted as m pq , is defined as x p y q g x, y , where x and y are coordinates of the pixel, and g(x, y) is the grey level of the pixel.The higher order moments represent spatial distribution of the objects.The previous two characteristics make moments a reasonable and proper feature for the classification task.However, the above equation may be greatly affected by the objects shifting.Therefore, to overcome this problem central moment is used and defined as where x = m 10 /m 00 and y = m 01 /m 00 .
The relationship between moments and central moments up to order 3 are listed as follows: ( Then, central moments could be further normalized by the following formulas: where 2.3.2.Vibration Features.Statistical are extracted from time domain vibration data using formulas presented in ( 8) and ( 9) as follows where,

Self Organizing Map (SOM).
The SOM is a neural network model that implements a characteristic non-linear projection from a high dimensional space of sensory or other input signals onto a low-dimensional array of neurons.The SOM is an unsupervised learning algorithm which produces a map pattern features on its output layer [20].Input patterns with similar features are mapped onto neighbouring output nodes.The network consists of an input layer with m neurons and an output layer with n neurons as shown in Figure 1.The image of the signal space tends to manifest the clusters of input information and their relationship on the feature map.With every node j, a parametric reference vector w j ∈ R n is associated.In an abstract scheme it may be imagined that at time k the input x(k), by means of some parallel computing mechanisms, is compared with all w j (k), and the location of best match in some metric is defined as the location of the response.However, in many practical applications, the Euclidean distances x(k) − w j (k) can be made to define the best matching node (neuron) i which is signified by the index i(x), i(x) = arg min x(k) − w j (k) = arg min e j (k) , (10) where j =

Method and Material
The flowchart of diagnostics process is presented in Figure 2.
After image acquisition from the object, thermal image is cropped to select the ROI, that is, rolling element bearing image.Next step, image clustering will be performed by using k-means algorithm to cluster the hotspot area based on color palette.Image segmentation by thresholding is conducted by using the a Otsu method [16].Image thresholding will produce binary image which is useful for image feature extraction.Perimeter, area, and central moments are then calculated based on resulted binary images.After feature extraction, extracted features from normal and faulty condition are separated for SOM training purpose.The feature of normal condition will be used as baseline of the diagnostics process.Finally, SOM will make decision of whether the machine condition is normal or faulty.Data acquisition was conducted on MFS which is able to simulate abnormality in rotating machine, that is, rolling element bearing fault, unbalance, misalignment, and looseness.Thermal images and vibration signals were simultaneously acquired from the machine that has been operated for 30 minutes to generate initial heat in the bearings.The image acquisition produced 30 images per second, and the machine was operated at 1800 rpm.Figures 3 and 4   the data acquisition process in the laboratory using infrared thermography camera and accelerometer sensors.

Results and Discussion
Data acquisition of thermal image and vibration signal of faulty bearing is presented in Figures 5 and 6.FLIR SC5000 camera was used to capture thermal images and LMS Test.Xpress was used to vibration signal acquisition from MFS.The acquired images were stored in RGB format with size of 256 × 256 pixels, and vibration signals were acquired in 5 kHz sampling frequency.
After cropping, thermal images captured from right and left bearings are presented in Figures 7 and 8.The ROI was determined by red or pink colors that represent the hottest temperature area generated by bearings.Such colors were then extracted and clustered by k-means algorithm with       Figure 9 also shows the colors in ROI from which feature extraction will be performed.The selected ROI will be transformed into binary image by thresholding method.Next step is feature extraction of perimeter, area, and central moments from ROI.This process is depicted in Figure 10.
Figures 11-13 show image features of area, perimeter, and central moment η 01 , respectively.The labels U, M, L, and B represent the faults in the machine namely unbalance, misalignment, looseness, and bearing faults, respectively.Figure 11 shows random patterns for the hot area of ROI which are not good patterns because the conditions of the machine cannot be distinguished clearly.The feature representation of Figure 12 shows better patterns than the feature of area, even though some data are out from the cluster of conditions.Observing this figure, temperature is increased between normal condition and faulty condition.This plot could be a reference for determination of bearing condition.Figure 13 also shows random structure of patterns but is better than feature of area.These patterns will be then input data for SOM training for fault diagnostics process.
Feature extraction was also performed on time domain vibration signal by calculating its statistical features.The results of feature extraction are presented in Figures 14,15,16.It is clearly observed that the machine conditions are well clustered.Comparing with features from thermal images, vibration features show better performance in clustering of machine conditions.
The fault diagnostics process is started by training SOM algorithm using features from thermal images and vibration signals.SOM is trained by 20 training data which consist of five machine conditions.The results of training process are presented by using U-matrix with hexagon topology to visualize distances between neighboring map units and help in seeing the cluster structure of the map.Map units indicate cluster border between conditions in the machine which are visualized by different color in the map.The U-matrices of SOM training are shown in Figures 17 and 18.These maps will be references from which fault diagnostics procedure is performed.
After SOM training, the performance of SOM diagnostics was evaluated by testing process using 10 data from thermal images of bearing fault and unbalance case (BU label), and 8 data from vibration signals of bearing fault case (B label) respectively.In the case of thermal images data, 8 data successfully activated cluster of BU, and 2 data were misclassified in BU cluster.It means that the accuracy is 80%.In addition, testing process using vibration data gives accuracy of 100% which means that all data tests successfully activate the cluster of B. From the experiment, the performance of testing process using vibration data is better than thermal images data.It was well known that vibration data can produce good accuracy in machine fault diagnostics.However, it could be said that the results of testing process by using thermal images and vibration data show good performance.Testing process of SOM using thermal images and vibration data are visualized by presenting the topography of best matching unit (BMU) as depicted in Figures 19 and 20, respectively.

Conclusion
This paper deals with the confirmation of thermal images and vibration signals for intelligent machine fault diagnostics using SOM network.Thermal images and vibration signals were acquired from MFS that is a reliable tool for machinery simulation.Image segmentation technique was performed on thermal images to obtain binary images for image feature extraction process.The features extracted from thermal images were area, perimeter, and central moments.Statistical features, namely, RMS, means, skewness and kurtosis, were also extracted from time-domain vibration signals.SOM were then trained by using extracted features from both thermal images and vibration signals.After training, SOM was tested by feeding some data to evaluate the performance of fault diagnostics.The results show that good accuracy was reached by using both source features thermography and vibration.Moreover, it showed that the thermography method could be an alternative way for conducting intelligent machine fault diagnostics with plausible results.

Figure 2 :
Figure 2: Flow chart of SOM diagnostics process.

Figure 7 :
Figure 7: Thermal images of right bearing.

Figure 8 :
Figure 8: Thermal images of left bearing.

Figure 19 :
Figure 19: SOM test for perimeter data.

k = 6 .
Figure 9 depicts the clustering result by means of kmeans algorithm.