Defect Detection and Localization of Nonlinear System Based on Particle Filter with an Adaptive Parametric Model

A robust particle filter (PF) and its application to fault/defect detection of nonlinear system are investigated in this paper. First, an adaptive parametric model is exploited as the observation model for a nonlinear system. Second, by incorporating the parametric model, particle filter is employed to estimate more accurate hidden states for the nonlinear stochastic system.Third, by formulating the problem of defect detection within the hypothesis testing framework, the statistical properties of the proposed testing are established. Finally, experimental results demonstrate the effectiveness and robustness of the proposed detector on real defect detection and localization in images.


Introduction
Fault detection, as a subfield of control engineering, is to monitor a stochastic system and identify when a fault occurred and the information of fault such as the type and its location.Fault detection approaches play a fundamental role to improve manufacturing process and application process in a variety of industry contexts [1][2][3][4][5][6][7], for example, quality control in process monitoring [1][2][3], product manufacturing [4], medium restoration [5], and facilities maintenance [8].Advantages of low costs, high automation, and high quality of defect detection techniques resulted in growing interests in recent years.Fault/defect detection for quality inspection includes the use of cameras, eddy current, ultrasonic, X-ray sensors and other sensors, which offer the measurements to be analyzed for extraction of the information about the fault.With the high demands for quality control using cameras in industry, visual inspection systems attract more attention in recent years.In a similar vein to fault detection of linear or nonlinear systems, defect detection in visual inspection systems can be taken as monitoring the variation of the measurements from cameras.In this paper, we devote ourselves to solving the visual defect detection problems from an unusual perspective of fault detection.
Currently, two rough categories of approaches are available for defect detection, that is, signal processing-based and model-based methods.The signal processing algorithms perform mathematical and statistical analysis tools on the measurements to extract faults [8][9][10][11][12][13].The developed methods like Gabor filter and wavelet transform have been proven to be effective solutions to locating the defects with less prior information in nonlinear systems.In recent years, defect detectors on the basis of Gabor filter [10] and wavelet transform [11] are efficient to find defects for web fabrics with stable repetition of textures, since it is easier to find defects in frequency domain.Techniques of the signal processing also cover a number of computational intelligence approaches like fuzzy logic and neural network [13], which provide some effective solutions for fault detection in various industrial problems.However, the data-driven methods suffer from large computational load and storage space.
The second category is an evolving collection of methodologies aiming to exploit the models of the system in temporal and spatial space to decide the occurrence of fault/detection [3,[5][6][7]14].Taking advantages of temporal or spatial models with the prior information, the model-based method can achieve robustness even in a scene with heavy amount 2 Mathematical Problems in Engineering of noises.This paper focuses on techniques of the defect detection of this category.
Various model-based approaches for defect/fault detection have been proposed in the past decades [3,5,6,15].Since the system state completely represents the system's internal (hidden) status and condition containing the fault information, the intrinsic problem in the model-based methodologies is the estimation of the system states from measurements of sensors.One of the most popular components of modelbased algorithms is the Bayesian method [16], which yields the posterior distribution of the system state containing the hidden state such as the occurrence and the fault type.Bayesian filter and its variations have been effective solutions to visual inspection of surface defects of materials and fault detection in process control [3], for example, chemical process.For the linear Gaussian system, Kalman filter [15][16][17] has been exploited to detect surface defects in visual inspection systems.Particle filter, as an implementation algorithm of Bayesian filter for nonlinear and non-Gaussian systems, has been used for fault detection of chemical process [3,18].However, state estimation and fault detection still remain a challenge due to the absence of suitable models for the actual application context.The linear Gaussian models in [15] cannot meet the requirements of the defect detection and localization in the actual visual inspection systems, due to the complicated intensity distributions of background pixels.
In this paper, a new intelligent defect detection algorithm without prior knowledge based on PF is presented.We formulate the visual defect detection problem to estimate the hidden states using PF and decide the occurrence or locations of defects in 2D images by chi-square test.In our method, intensity of the 2D image (the same vein as the control system) along each row or column line is assumed as a time series or a random process {  }  =1 .To detect faults/defects in the image (the system), reasonable state and measurement models are proposed firstly for particle filter.Then, with the proposed models, particle filters along rows and columns are implemented to estimate states and the measurement innovations (residuals).Finally, abrupt changes in the measurement innovations are used to locate defects in the inspection images by chi-square test.Tests on the real database demonstrate the effectiveness of the proposed algorithm.

Problem Formulation of Defect Detection
The defect detection in a visual inspection system is to identify the occurrence of the defect and its location from the measurements of camera sensors, that is, the digital image of the inspected object.A measurement   in a digital image is the gray level or intensity of the pixel.Then the pixel intensity along each row or column line in an image of  ×  pixels is assumed as a time series or a random process {  }  =1 or {  }  =1 , where  is one-dimensional coordinate of the relevant pixel on the scan line (i.e., the time step of the random process).
When the inspected target is free from defects, the intensity of pixels along each scan line takes on small variations.
Therefore, the gray level along a scan line in a defect-free image can be defined as a white Gaussian state series {  }  =1 or {  }  =1 .Due to subtle intensity changes for defect-free areas, the state sequence can be modeled by the following discrete time linear Gaussian dynamical model; that is, where   is the state in the time series,   is the state transition model, and V  is the zero-mean white Gaussian noise sequences with variance  2 V, .For defect inspection problem in this paper, the dynamical mode uses the random walk model with   = 1.Each state in a Gaussian series can also be written as where N(, ) denotes a Gaussian distribution with expectation  and variance .However, due to the influence of defects, uneven illumination, and the geometrical structure of the inspected object, the actual measurements usually follow nonlinear model, which can be formulated by a transition function approximating the Markov transition relation between the state estimate   of the defect-free and the actual measurement   of a pixel (defect-free or defective).Therefore, the measurement model for an arbitrary pixel can be defined as where   (⋅) denotes the transition function from the state estimate   to the measurement   and   is zero-mean white Gaussian noise sequences with variance  2 , .In terms of the established models, Bayesian filter can be exploited to yield the state estimate of x for each pixel along a scan line of an image by Bayes recursion below [17]: where Once the state estimate x−1 at time step  − 1 (i.e., intensity of the  − 1th pixel along the scan line) is obtained, the difference between the predicted measurement  |−1 and the actual measurement   is defined as residual ] *  , which can be calculated by Here,  |−1 can be derived by ( 1) and ( 3) with the form where  |−1 is the predicted state using (1).It can be seen from ( 6) and ( 7) that a residual ] *  reveals the difference between the actual measurement of a pixel and the defectfree pixel, which can be used to decide the occurrence of a fault/defect.When the measurement   is a Gaussian variable, Then a chi-square test [17] will declare detection of a defect if where 1 −  is the level of confidence,  indicates the degrees of freedom, and  is the detection threshold.

Adaptive Parametric Measurement Model.
As explained in Section 2, it is important to warrant that the defect information is revealed by residual ] *  of the state and the measurement.To attain this purpose, the state model about the defect-free object based on random walk model is presented in ( 1) and ( 2).Furthermore, it is also necessary to design an accurate measurement model which can closely approximate the real measurement contaminated by noise, defects, and other disturbances.As shown in Figure 1, the measurements along a scan line in a complex visual inspection system follow multimodel distribution.
In order to obtain an accurate model, the Gaussian mixture model is exploited to approximate the multimodel distribution of measurements in the image of the inspected object.Assume that at time  − 1 (i.e., the  − 1th pixel in a scan line), the probability density of the measurement can be approximated by a Gaussian mixture model (GMM) as where  −1, is the weight of the th Gaussian component in the mixture with ∑  =1  −1, = 1,  −1, is the mean,  −1, is the variance, and  is the Gaussian component number with  = 3∼5.Then, the measurement likelihood function can be designed as All the parameters { , ,  , ,  2 , }  =1 of Gaussian items in (10) can be automatically updated by the model ( 9) at time  − 1 using -means approximation [19] in Algorithm 1.The number of Gaussian distributions  is manually set by making a tradeoff among the complexity of the pixel intensity model, the available memory, and the computational power.On the one hand, due to more ideal illumination and regular geometrical structure of the inspection condition, subtle changes of illumination of the desk base surface and only repetitive changes of scene elements may occur in the real inspected images (see Figure 1(a)).Thus, it is reasonable to simply model the values of a particular pixel along a scan line as a mixture of a fixed number of Gaussian distributions.On the other hand, less Gaussian distributions can alleviate the computation load.Considering the above elements, the value of  should be as small as possible, on the premise that the number of distributions is able to represent the value of each pixel along each scan line of the inspected image.For example, in this paper,  is set to be 3 in the application of desk base inspection.
In addition,  , ,  , , and  , can be computed by the -means approximation in Algorithm 1.It should be noticed that the weight  , of a Gaussian distribution is updated according to where  is the learning rate and   ,−1 is 1 for the model matched by   (i.e., |  −  , | < 2.5 , ) and 0 for the remaining models otherwise.If none of the  distributions matches   , the least probable distribution is replaced with a Step 1. Update weights of the  Gaussian items by  , = (1 − ) ,−1 + ( ,−1 ), where  is the learning rate,  ,−1 is 1 for the model matched by   and 0 for the remaining models.If none of the  distributions matches   , the least probable distribution is replaced with a distribution with   as its mean, a high variance, and low weight.
Step distribution with   as its mean, a high variance, and a low weight.
Once parameters { , ,  , ,  2 , }  =1 at time  are obtained, the likelihood function (10) can be updated and be used to implement PF or Bayes recursion instead of   (⋅).Due to the better ability of fitting the complex distribution of pixel values, the proposed adaptive measurement model can help to represent the system more correctly.

Particle Filter with the Adaptive Model.
As analyzed in Section 2, state is required to be estimated for defect detection.Since the proposed measurement model in Section 3.1 is Gaussian and nonlinear, Bayesian filter in (4) and ( 5) can be solved by particle filter [18].Particle filter is based on Monte Carlo approximation, which is able to effectively deal with nonlinear and non-Gaussian problems [18].In terms of the proposed state model in (1) and the measurement likelihood in (10), particle filter with adaptive model is summarized below.
Step 2 (important sampling).Firstly, the particles { 4) are propagated by the state model ( ()  |  () −1 ), which can be obtained using ( 1) or ( 2) with the form Secondly, given a new observation   , each sampled particle  ()  from ( 12) is weighted by the proposed adaptive likelihood in (10) with the form Finally, the weights are normalized by Step 3 (resampling).Resample new particles from the particles { ()  ,  ()  }  =1 in Step 1 according to roulette method, where the probability of choosing a particle is proportional to its weight.After resampling, the resampled particles { x() , 1/}  =1 are with equal weights, which can approximate the posterior distribution   (  |  1: ) in ( 5) at time .
Step 4 (state extraction).The state estimate x can be extracted by using either the minimum mean square error criterion: or the maximum a posteriori (MAP) criterion:

Defect Detection and Localization Based on PF
When the state estimates { x }  =1 or { x }  =1 of pixels along each row or column scan line are generated by the proposed PF in Section 3, the state estimates can be employed to establish the statistic for detecting and locating defects.To yield the statistic for chi-square test, the residual ] *  for each pixel should be computed firstly using (6).Under the Gaussian assumption, since the residual sequences {] *  }  =1 and {] *  }  =1 are Gaussian-distributed, the normalized residual squared statistic (NRSS)   is approximately chi-square distributed with the form where  −1  denotes the covariance of ] *  .In terms of the principle of a check of the goodness of fit,   is employed as the statistic and the fault/defect can be located along the scan lines by the chi-square test: where 1 −  denotes the level of confidence,  is the degrees of freedom with  = dim(  ), and the detection threshold is .

Statistic computation
Chi-square test for edge location Output: Θ(M × N) Here,  can be referred to in the chi-square distribution table in terms of  and .If the statistic   for a pixel satisfies (18), an alarm indicating a defect point is given.When the chi-square tests along all row lines are completed, the potential defect points are stored in the detection mask Θ( × ), which is an  ×  matrix with value 1 for an edge point and with value 0 for a defect-free point.Similarly, the mask Θ  ( × ) along columns can be obtained.Considering different sensitivity of both tests along row and column directions, the final mask Θ( × ) (see Figure 3(b)) can be calculated by where (, ) denotes detection result of the th row th pixel in the image and "|" denotes logical OR operator.Based on all of the above, the proposed defect detector using PF with adaptive model is described in Figure 2. In addition, in order to locate accurately the occurrence of a defect with lower false alarms, the cumulative sum of the statistic   is calculated to distinguish the defect from the image (see Figure 3(a)).

Experimental Setup.
The VACE protocol [20] is introduced here to evaluate the detection performance of the proposed algorithm, which uses two scores, namely, multiple object detection accuracy (MODA) and multiple object detection precision (MODP).The MODP and MODA measures are directly proportional to the area overlap between the system output and the ground truth.Higher scores of MODP and MODA indicate better performance in defect detection precision and accuracy, respectively.Denote  ()  and  ()  as the th ground truth object and its associated detected object (defect) in the th frame, respectively.MODP and MODA are defined as where |] quantifies the spatial overlap ratio between the ground truth and the detected object in a sequence of frames,  mapped is the number of mapped objects over the entire sequence, and   mapped refers to the number of mapped objects in the th frame.Consider where   is the number of missed detects, fp  is the total number of false positives,   (⋅) and   (⋅) are the cost functions for   and fp  , and    is the number of ground truth objects in the th frame.In our experiments, the ground truth detects are annotated manually according to annotation guidelines for VACE [15].The best possible match between ground truth and detected objects in a global sense can be solved by Hungarian algorithm.

Defect Detection Experiment.
The proposed detector is tested by the defect image database, which contains multiple kinds of defects on the surface of the desk base of hard disks.A sample result using the proposed defect detector is shown in Figure 3. Figure 3(a) plots the accumulative sum of NSSRs for row and column scan lines and marks the defect location with red circles, whose centers are calculated by abrupt changes in the response curve of NSSRs. Figure 3(b) shows another defect location result of the same inspected part by a mask image in (19) directly using NSSR of each pixel.Figure 3 indicates that both methods on the NSSRs and their accumulative sum are effective to locate defects in an image.In the following experiments, each of the location methods is chosen arbitrarily.
Figure 4 shows the detection results of various types of defects, for example, corrosion, hard blister, pin holes, and crack.It is shown that the proposed detector is able to accurately and automatically identify and locate defective areas from complex background with much less false alarms.As well known, uneven illumination (see Figures 4(a) and 4(b)) and geometrical structure (see Figure 4(a)) lead to scattered discontinuities in the defect-free areas, which make defect detection more difficult.To alleviate it, a more accurate adaptive measurement model is proposed for PF, which can inhibit the influence of discontinuities in the defect-free pixels.

Comparison Experiment.
To evaluate the performance of the proposed detector further, we compare the proposed detector (DET PF) with the detector in [15] (DET KAL) and Otsu thresholding (DET THR). Figure 5 shows the comparison result of DET PF and DET KAL.Since DET KAL employs a single Gaussian model as the measurement model, it cannot accurately approximate the measurements of multimodel distribution.It can be seen from Figure 5(b) that DET KAL is not able to locate defects in a complicated background (i.e., complicated intensity distribution of pixels in defect-free areas).Figure 5 indicates that the proposed detector based on PF with a robust model outperforms DET KAL in fault detection for nonlinear system.
To quantify the performance difference of the three methods, we compute their performance scores according to evaluation method presented in Section 5.1.Figure 6 shows the accuracy and the precision scores (i.e., MODA and MODP scores) of the three defect detectors applied to the test database.It can be observed that the proposed detector achieves higher performance scores than common thresholding DET THR method and DET KAL.

Sensitivity Analysis of Parameters.
The parameters involved in the proposed detector, such as Gaussian model noise sequences   and V  in (1), (2), and (3) and the decision threshold  in (17), can be preset manually according to the statistical characteristics of the gray level or the intensity of the inspected image.As mentioned in Section 2 of this paper,   and V  are zero-mean white Gaussian sequences and are proportional to their standard deviations   and  V .
Figure 7 illustrates the accuracy and precision transitions of defect detection performance when the parameters   ,  V , and  of the detector are changed.When the value of one parameter with log 2 scale at a time are changed and the other parameter values are fixed, then the average values of the corresponding MODP and MODA measures are obtained and plotted in Figure 7.As shown in Figures 7(a) and 7(b), the detector is sensitive to   and  V .Both   and  V represent the model errors between the chosen system models (i.e.,   for dynamical models and  V for the measurement model) and the truth.Smaller and larger   generally leads to divergence of the particle filter and a degraded detection precision, namely, decreasing values of MODA and MODP, respectively.Similarly, choices of  V depend on the measurement model error, which can be determined practically in terms of image quality, and a smaller  V would be chosen for an image of higher quality.The theoretical value of the decision threshold  can be computed according to [17].As shown in Figure 7(c), a small variation of the detection performance scores is related to theoretical value of the decision threshold .It can be seen that, in a real application of the proposed detector, quantities of   ,  V , and  can be manually heuristically chosen from several tests.

Computational Complexity Analysis and Comparison.
The computational complexity of the proposed method depends on two main steps: the -means approximation based adaptive measurement model learning and particle filter.Indeed, at time , the -means approximation step requires calculation of cost: where  is the number of Gaussian items and   is the number of pixels in the inspected image.The particle filter requires where   denotes the state dimension in the PF and   denotes the number of particles.In the application to the desk base of hard disk,   is 1 and   is 200.Thus, in the experiment, the computational load is  , ≈ 5 *   *   = 1000  .Furthermore, in our experiments, the proposed defect detection algorithm was carried out by an Intel 2.2 GHz CPU PC.Detection was done at an average rate of 20 frames per second (i.e., 50 ms per image) for images with size of 1024 * 768 without code optimization.In comparison with the similar method (MR) in [15], MR is able to process the a test image in about 60 ms.MR suffers more intensively computational load, which may be induced by the matrix inversion operation in Kalman filter with constant velocity state model and multidimensional vector of the state.From the above analysis, with code optimization, the proposed method will be a promising solution to defect detection in real time.

Conclusions
This paper presents a novel methodology for automatic fault/defect detection based on PF for nonlinear systems.The defect/fault detecting framework in a stochastic dynamical system based on PF and chi-square test is proposed first.To alleviate the scattered discontinuities from the uneven illumination and geometrical structure of the inspected objects, an adaptive measurement model is designed and estimation method of the hidden states for nonlinear system based on PF is presented in detail.Third, the statistical properties of the hidden states are employed to establish an effective statistic for defect/fault defection by chi-square test.Finally, the improved PF with adaptive parametric model and the fault detection framework are applied to identify the defects in visual inspection system.Experiments demonstrate that this proposed methodology can detect most defects efficiently with high quality in nonlinear system, which can be exploited to deal with fault detection problems in other industry fields.
is the transition density defined by (1) or (2) and   (⋅ | ⋅) is the likelihood function defined by (3).The posterior density   (  |  1: ) includes all information of the state at time  and the state estimate x of the th pixel in a scan line can be obtained by maximum a posteriori (MAP) criterion.

Figure 1 :
Figure 1: An image for inspecting the surface defect on the base desk of a hard disk and its histogram.(a) The image of the inspected object.(b) The histogram of gray levels of pixels along a column line in (a).

Figure 2 :
Figure 2: Procedure of the proposed detector.

Figure 3 :
Figure 3: A sample result of defect detection.(a) Defect locations using the accumulative sum of NSSRs.(b) Defect locations using NSSR.

Figure 7 :
Figure 7: Sensitivity analysis of parameters used in the detector.(a) Average evaluation scores for the standard deviation of the dynamical model.(b) Average evaluation scores for the standard deviation of the observation model.(c) Average evaluation scores for the decision threshold.