Removal of Salt and Pepper Noise in Corrupted Image Based on Multilevel Weighted Graphs and IGOWA Operator

This paper proposes a novel iterative two-stage method to suppress salt and pepper noise. In the first phase, a multilevel weighted graphsmodel for image representation is built to characterize the gray or color difference between the pixels and their neighbouring pixels at different scales. Then the noise detection is cast into finding the node with minimum node strength in the graphs. In the second phase, we develop a method to determine the order-inducing variables and weighted vectors of the induced generalized order weighted average (IGOWA) operator to restore the detected noise candidate. In the proposed method, the two stages are not separate, but rather alternate. Simulated experiments on gray and color images demonstrate that the proposed method can remove the noise effectively and keep the image details well in comparison to other state-of-the-art methods.


Introduction
Images are easily corrupted by impulse noise during the signal acquisition, transmission, or storage [1].There are two types of impulse noise, salt and pepper noise and random valued noise.Salt and pepper noise randomly alters a certain amount of pixels into two extremes, either 0 or 255, for an 8-bit image.The noise significantly damages the image information which leads to difficulties in succeeding image processing tasks such as edge detection or image segmentation and image recognition tasks.Because the noise pixel differs from most of its local neighbours, it has large gradient value the same as the edge pixel [2].How to effectively restore the noisy image is still a challenging problem.
Numerous techniques have been proposed to suppress the salt and pepper noise.The most simple and well-known methods are the standard median filter [3] and its variants [4][5][6] which are nonlinear filters whose responses are based on reordering the intensity values in the neighbourhood of the corrupted pixel.The nonlinear operation exhibits good denoising power [7], but when the noise level increases, the edges and other details of the image cannot be restored.This is because the median filter simply replaces every pixel's value.
Switching or decision based filters are proposed and have become a hotspot of removal of salt and pepper noise; their strategy in common is using a two-stage technique which first detects the possible noise pixels in an image and then replaces the noise pixels [8][9][10][11][12][13][14][15][16][17][18][19].For example, Chan et al. used the adaptive median filter to detect the noise pixels and an objective function with an ℓ 1 data-fidelity term and edgepreserved regularization term to denoise, while preserving the edges [9].The NAFSMF method uses the image histogram to find the salt and pepper noise pixels and computes the number of noise-free pixels to avoid selecting a noise pixel as the median pixel for restoration [10].The MDBUTMF method regards the maximum or minimum gray level as the noise pixel and replaces the noise pixel with the median of remaining elements in the selected window or with the mean of the selected window if the selected window contains all 0's or 255's or both [11].The AWMF uses a size alterable window to judge whether the given pixel is noise pixel or not and computes the weighted mean of the current window to restore the noise pixel [12].
Most recently, Veerakumar et al. used the Rank Ordered Logarithmic Difference which is the logarithmic function on the absolute difference to detect the noise and restore the noise pixels by the adaptive anisotropic diffusion filter [13].Ahmed proposed a two-stage iterative filter which detects the noisy pixels with an adaptive fuzzy detector and uses weighted mean filter on the uncorrupted pixels [14].Vijaykumar et al. developed a fast switching filter which identifies the extreme minimum value and extreme maximum value as the noise pixels and replaced the noise pixels by either median value or mean value [15].Li et al. developed an image block-based noise density estimation method to guide noise detection and noise restoration and used the global image information for noise rectification [16].Wang et al. explored the similar patches with repeat patterns in images and proposed an iterative nonlocal means filter (INLM) for salt and pepper noise removal [17].The ANCLPVMF judges the noise pixels based on the linear prediction error and uses adaptive window based vector median filtering operation to restore the noise [18].In [19], two approaches have been provided for detecting the noise pixels based on means and variance, and a novel type-1 fuzzy approach is presented for denoising.
It can be concluded that two aspects of problems in salt and pepper noise removal are how to retain the uncorrupted pixels and how to use the uncorrupted pixels to estimate the corrupted pixel precisely [20].In order to solve the two aspects of problems effectively, in this paper, we investigate the graph theory for image noise detection and information aggregation operators for image restoration.Studies indicate that the image processing and analysis performance are dependent on the choice of data representation on which they are applied [21].Recently, graph-based representation for image has been applied to image processing and recognition successfully [22][23][24][25][26]. Inspired by these, on the one hand, we build a multilevel weighted graphs model for image representation to characterize the relationship between the pixels and their neighbouring pixels at different scales.Because the gray or color difference between the noise pixel and its neighbours is large, the node strength of the weighted graphs is utilized to detect the noise candidates.On the other hand, the ordered weighted averaging (OWA) operators provide a parameterized family of mean type aggregation operators [27][28][29][30][31][32].They learn the associated weighting vector from observational data to obtain an optimal aggregation result.The generalized OWA (GOWA) [28] operator uses generalized means in the OWA operator.The induced generalized OWA (IGOWA) [29] operator is an extension of the GOWA operator, with the difference that the reordering step of the IGOWA operator is not defined by the values of the arguments, but rather by order-inducing variables, where the ordered position of the arguments depends upon the values of the order-inducing variables.It is a more general formulation of the reordering process that is able to consider more complex situations.Thus, we develop a restoration method based on the IGOWA operator to aggregate the uncorrupted pixels.It has high flexibility and could restore the noise pixel accurately even when the 0's and 255's are not balanced in the local neighbourhood of the noise pixel.The proposed noise detection and restoration are not separate, but rather alternate, and iterative, which can deal with the single-point (single-pixel) and noise patches.This is the first attempt of employing the graph theory for solving the noise detection and the information aggregation operator for removing salt and pepper noise.Hopefully, this is an initial work of graph theory and information aggregation operator for image restoration.
The rest of the paper is structured as follows.The proposed method and the flow diagram of the proposed algorithm are described in Section 2. Section 3 provides the simulation results with different images in comparison with the six state-of-the-art methods.Finally conclusions are drawn is Section 4.

Multilevel Weighted Graphs Modeling for Noise Detection.
To detect the noise in an image we model the image as multilevel weighted graphs {  = (, , )},  = 1, 2, . . ., , which are a series of graphs  1 ,  2 , . . .,   .At each level , each V  ∈  corresponds to a pixel in the image, an edge (V  , V  ) ∈  connects vertices V  and V  ,   represents the weight on that edge connecting V  and V  , and  denotes the level version.The edges are created by connecting the nodes, whose corresponding pixels are in the neighbour pixel sets.
In this paper, we define three different neighbour pixel sets given a pixel, which is illustrated in Figure 1, for a given pixel   , the pixels labeled "1" are the 8-neighbour pixels, labeled "2" are the 12-neighbour pixels, and labeled "3" are the 16-neighbour pixels.We denote the pixel set as  8 (  ),  12 (  ), and  16 (  ), respectively.For a pixel, we connect its corresponding node and the nodes corresponding to its 8-neighbour pixels to create the edges in  1 , connect its corresponding node and the nodes corresponding to its 12neighbour pixels to create the edges in  2 , and connect its corresponding node and the nodes corresponding to its 16-neighbour pixels to create the edges in  3 .The weight connecting two nodes should reflect the difference of the corresponding pixels' gray or color in  1 ,  2 , and  3 .The edge weight connecting node V  and node V  in the multilevel weighted graphs is defined as The node strength of node V  is defined by where  is node number of the graph.The node strength integrates the information on the number (degree) and the weights of links incident in a node [33].With the defined multilevel weighted graphs for image above, the noise pixels' corresponding node's strength is much smaller than the noise-free pixels in the graphs.For this reason, different shaped noise can be detected through finding the corresponding nodes with the minimum node strength in the multilevel weighted graphs.The first level weighted graph is effective in detecting the single-point noise, and when the noise density increases, the second-and third-level weighted graphs are needed to detect the noise patches or blob noise.Therefore, the proposed method can detect the single-point and irregularly shaped noise.
To illustrate, take a subimage in Lena image corrupted with 50% salt and pepper noise for example, the gray values are shown in Figure 2. Using the proposed detection method, the first noise candidate detected in the dashed-line box is the pixel in row 4, column 2 (noted in yellow), whose corresponding node strength is minimum in the first level weighted graph.Noted that the value of the detected pixel is 0, and it is exactly a noise pixel.
In our algorithm, the noise detection and restoration are alternate.We replace the noise pixel once it is detected, and to detect the next noise pixel, the weighted graph should be constructed again.This procedure is repeated till the pixel corresponding to the minimum strength node is not extreme.The flowchart of the proposed algorithm is shown in Section 2.3.

Noise
Restoration by Developed IGOWA Operator.When restoring a detected noise pixel, it is appealing to utilize the uncorrupted pixels as correctly as possible to void the information distortion since the entire algorithm is unsupervised.An IGOWA operator aggregates  inputs into one output with an associated weighting vector  of dimension  such that the sum of the weight is 1 and   ∈ [0, 1], and a set of orderinducing variables   , and a parameter  ∈ (−∞, +∞).The IGOWA operator is defined as follows: where   are the order-inducing variables,   are the argument variables, and -index() is the index of the th largest element in the { 1 ,  2 , . . .,   }.To restore a noise pixel ,   ( = 1, 2, . . ., ) are the pixel values of the neighbouring  pixels of the noise pixel .However, there are noise pixels in the  pixels, which should be excluded when restoring using (3).Firstly, we define the set of order-inducing variables   as follows: where OWA  (  4) indicates that if   approximates the value of representative value, then the order-inducing variables   will be large; on the contrary, the larger the distance between   and the representative value is, the more possibly   is noise, and the smaller the order-inducing variables   will be.Secondly, to avoid the influence of the noise when computing the representative value, the two ends of the weight vector u = { 1 ,  2 , . . .,   }  are set as zero.The nonzero elements of u are defined as follows.Assume there exist  ( ≤ ) pixels, whose noise density is , among the neighbouring  pixels of the noise pixel .Let random variable  denote the number of the noise pixels among the  pixels; then  follows binomial distribution; that is, the probability of  equals  which is { = } = (   )   (1 − ) − ,  = 1, 2, . . ., .The probability that the number of the noise pixels  less than or equal to ℎ is To denoise with the significant level , we seek the minimal value  satisfying () ≥ 1 − .The weighted vector u can be computed by where  =  if  is even and  =  + 1 if  is odd.Finally, the weighted vector  = ( 1 ,  2 , . . .,   )  is defined by basic unit-interval monotonic (BUM) function [34] () =   (0 <  < 1); here obviously, ∑  =1   = 1, and 1 ≥  1 ≥  2 ≥ ⋅ ⋅ ⋅ ≥   ≥ 0. Consequently, the original value of the corrupted pixel can be predicted by (3).
From the above analysis, algorithm of the developed IGOWA for restoration is summarized as follows.
Step 3. Based on (4), we get the order-inducing variables   .
Step 5.The restored value of the noise pixel is obtained through (3).

The Proposed Algorithm.
From the principles, in comparison with the previous methods, the proposed method has two advantages which can be concluded as follows.
(i) The proposed method utilizes both the structure features and gray level of image to detect the noise candidates.Therefore, the detection method can detect the noise accurately and keep the uncorrupted pixels which are in themselves extremely unchanged.
Whereas the previous methods either use the gray level or the statistical information of image which possibly bring detection errors.(ii) For restoration, the IGOWA operator is adopted and developed for the first time to predict the corrupted pixel using the local neighbouring pixels in our method.We design the order-inducing variables of the operators to produce the appropriate pixel value for denoising.This method is easy to understand and effective, whereas the previous denoising methods are either too easy to give a good result or ambiguous to understand and carry out.
For implementation, the images are restored from top to bottom and left to right.Initially, we denoise a 4 × 4 subimage in the top left of the image, as described in Sections 2.1 and 2.2.When the pixel corresponding to the node with minimum node strength in the multilevel weighted graphs is not extreme, the denoising procedure of the current subimage completes.Then we construct the next subimage which consists of part of the restored image and part of the contaminated image to denoise.These are repeated till the entire image is processed.The flow diagram of proposed algorithm is shown in Figure 3.

Configuration.
In this section, the performance of our proposed method is tested with different gray-scale and color images (Lena, Boat, Baboon, Cornfield, Flower, Fruits, and Pens) whose dynamic ranges are [0 255].The test images considered in this work have both low and high frequency content.We use the signal-to-noise ratio (PSNR) and the correlation factor (CF) to quantitatively measure the restoration results.
The PSNR is defined as where  , is the pixel value of the original noise-free image,  , is the pixel value of the restored image, and  ×  is the size of the input image.The MSE is the mean-square-error between the original and restored images.The correlation factor is defined as where  , is the pixel value of the original noise-free image,  is the mean pixel value of the original image,  , is the pixel value of the restored image,  is the mean pixel value of the restored image, and  ×  is the size of the input image.The correlation factor  = 1 if the two images are absolutely identical,  = 0 if they are completely uncorrelated, and  = −1 if they are anticorrelated.Our proposed method is compared to other state-of-theart salt and pepper noise filters, which include NAFSMF [10], MDBUTMF [11], AWMF [12], FSMMF [15], LGF [16], and ANCLPVMF [18].All the experiments are implemented in MATLAB and executed on a Core i7-4790 CPU@4.00GHz with 16.0 GB of memory and no effort made to optimize algorithm speed.
Parameters of the proposed method and the compared algorithms are sets as below.In the developed IGOWA for restoration, we set the significant level  = 0.05 when the noise density is lower than 0.5,  = 0.1 when the noise density is equal or higher than 0.5, the exponent of the BUM function  = 0.55, and the  = 2. Parameters of the compared methods are set as below.In method NAFSMF, the maximum In method AWMF, the  = 1, ℎ = 1, and    = 39.In method in LGF,  = 9 in the noise density estimation stage,  = 400 in the noise detection rectification stage,   = 7 and  = 1 in the first noise detection stage, and  = 7 in the second noise detection stage.We either use the recommended parameters as in the original papers or tune the parameters to obtain the optimal results for the six compared methods.

Denoising Performance.
Firstly, to verify the efficiency of the proposed denoising method, among the commonly tested 512×512 8-bit gray-scale images, the image (Lena) with homogeneous region and image (Boat) with high activity are taken as testing samples.Each sample is randomly corrupted by salt and pepper noise with varying densities from 10% to 90% with increment of 10%.The PSNR and CF values of the proposed method in comparison to the state-of-theart methods for different noise densities for Lena and Boat image are illustrated in Tables 1-4.From Tables 1 and 3, we can see that although the AWMF, ANCLPVMF, and LGF can provide a good PSNR value, the proposed method can give much better PSNR values than the compared six methods.Actually, the proposed method can significantly improve the restoration in slight noise corruptions, such as the noise density less than 40%.Moreover, it is evident that the proposed method can give much greater CF values than the compared six methods from Tables 2 and 4. For visual comparison, Figures 4 and 5 show the restored images of various methods for the Lena and Boat corrupted by 70% noise density, respectively.From Figure 4, it can be found that the AWMF, ANCLPVMF, FSMMF, and LGF reveal superior performance to the NAFSMF and MDBUTMF, which deteriorate the quality of the restored images.Our proposed method slightly outperforms the AWMF, ANCLPVMF, FSMMF, and LGF, among which the method in LGF suffers from blurred effect in the restored image.The restored images through the AWMF, ANCLPVMF, and FSMMF still have     different amounts of residual noise, and the restored image by the proposed method has better image details, where the edges and textures are preserved well.These results are consistent with that for the Boat image as shown in Figure 5.
To further demonstrate the excellent performance of the proposed method, we also use the proposed method to process color images that are corrupted by salt and pepper noise.The color images taken into account are Baboon, Cornfield, Flower, and Fruits.To perform image denoising on a color image, we respectively apply our method to the separate color channels (RGB) of noisy color images and recombining them.Visual restored Baboon images corrupted by 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, and 90% noise density are shown in Figure 6.We can see that the proposed method can obtain good visual effects and also perfectly protects the detail of the restored image.Figure 7 has shown visual observation

Figure 2 :
Figure 2: Noise detection by minimum node strength.

Figure 4 :
Figure 4: Restored images of various results for the Lena image with 70% noise density.(a) Noisy image; (b) restored image using the NAFSMF; (c) restored image using the MDBUTMF; (d) restored image using the AWMF; (e) restored image using the method in FSMMF (f) restored image using the method in LGF; (g) restored image using the ANCLPVMF; (h) restored image using the proposed image; and (i) original image.

Figure 5 :
Figure 5: Restored images of various results for the Boat image with 70% noise density.(a) Noisy image; (b) restored image using the NAFSMF; (c) restored image using the MDBUTMF; (d) restored image using the AWMF; (e) restored image using the method in FSMMF; (f) restored image using the method in LGF; (g) restored image using the ANCLPVMF; (h) restored image using the proposed image; and (i) original image.

Figure 7 :
Figure 7: Restoration results of color images with 60% noise density (left to right): original images, noisy images, and restored image used by the proposed method.

Table 2 :
Comparison of restorations in CF for Lena image.

Table 3 :
Comparison of restorations in PSNR(dB) for Boat image.

Table 4 :
of restorations in CF for Boat image.