An Efficient Universal Noise Removal Algorithm Combining Spatial Gradient and Impulse Statistic

We propose a novel universal noise removal algorithm by combining spatial gradient and a new impulse statistic into the trilateral filter. By introducing a reference image, an impulse statistic is proposed, which is called directional absolute relative differences (DARD) statistic.Operationwas carried out in two stages: getting reference image and image denoising. For denoising, we introduce the spatial gradient into the Gaussian filtering framework for Gaussian noise removal and integrate our DARD statistic for impulse noise removal, and finally we combine them together to create a new trilateral filter for mixed noise removal. Simulation results show that our noise detector has a high classification rate, especially for salt-and-pepper noise. And the proposed approach achieves great results both in terms of quantitative measures of signal restoration and qualitative judgments of image quality. In addition, the computational complexity of the proposed method is less than that of many other mixed noise filters.


Introduction
Noise can be easily introduced into digital images due to analog-to-digital conversion errors and malfunctioning pixel elements in the camera sensors [1,2].Noise can significantly degrade the image quality and increase the difficulty in subsequent processing, such as image segmentation, object recognition, and edge detection.Therefore, noise removal becomes a necessary and fundamental step in image processing.However, noise removal is a difficult task because images may be corrupted by different types of noise.Fortunately, most noise added to images can be modeled by Gaussian noise and impulse noise [2].
Gaussian noise is always introduced during acquiring images and can be characterized by adding a zero-mean Gaussian distribution value into each image pixel [2,3].Based on this distribution property, it can be removed by locally averaging operation in general [3].Classical linear filters are the common choice, such as the Gaussian filter which is a widely used method to remove Gaussian noise, however, it blurs edges and details significantly.In order to preserve edges and details in images while removing noise; Tomasi and Manducci proposed a bilateral filter that uses weights based on spatial and radiometric similarity [4][5][6].The bilateral filter has proven to be very useful; however, it is slow.To solve this problem, Paris and Durand proposed a fast approximation of the bilateral filter based on a signal processing interpretation [7].It downsamples convolution computation significantly without impacting the result accuracy.
Impulse noise can occurr in image transmission and characterized by replacing some pixels with noise while retaining the rest [8].The Gaussian noise removal methods mentioned above cannot adequately remove impulse noise.Therefore, nonlinear filters have been developed for removing impulse noise such as the traditional median filter [8,9].The median filter has been widely applied in impulse noise reduction because of its simplicity and high computational efficiency [9].Since an entire image is replaced by median values, median filter also modifies uncorrupted pixels.To improve performance, extensions of the median filter [10][11][12][13][14][15][16] and switching scheme methods are proposed.The switching scheme detects impulse noise pixels first and then replaces them with estimated values while keeping the remaining pixels unchanged [6,10,13,17].The main drawback of these filters is that they just use median values or their variations to estimate the noisy pixels, which can cause some image details Mathematical Problems in Engineering to be distorted.To overcome this, fuzzy techniques are introduced for noise removal [18][19][20].With suitable fuzzy system model, they can preserve image details during noise removal.The performances of these filters also depended on the accuracy of noise detectors.Garnett et al. proposed a rankordered absolute differences (ROAD) statistic to identify the impulse noisy pixels and incorporated it into a trilateral filter to remove impulse noise [2].It has proven to be a good impulse noise detector even with high noise level.Based on ROAD, Dong et al. proposed a rank-ordered logarithmic differences (ROLD) statistic to improve the accuracy of noise detection [21].Although it obtained better performance, its running time is significantly increased comparing with the previous mentioned filters due to the logarithmic computation.In [22], Yu et al. presented a rank-ordered relative differences (RORD) statistic through introducing a reference image and combining with a simple weighted mean filter.It can not only remove impulse noise but also preserve image details.
Mixed noise could appear during transmitting an already noise corrupted image over faulty communication lines [2].In such situation, most of the filters mentioned above will be useless.The median-based signal-dependent rank-ordered mean (SD-ROM) filter proposed by Abreu et al. can be used for mixed impulse and Gaussian noise removal [23].But it often produces visually disappointing output when applied to images with Gaussian or mixed noise [2].The trilateral filter with the ROAD noise detector [2] can remove both Gaussian and impulse noise effectively; however, it takes a long processing time due to the calculation of radiometric weighting function.In [6], Lin et al. proposed a switching bilateral filter with a texture/noise detector capable of removing mixed noise effectively, but for impulse noise it is not as good as SD-ROM, and for Gaussian noise it is not as good as bilateral filter.More recently, there are also some novel and encouraging approaches proposed [24,25].
In this paper, we first propose a directional absolute relative differences (DARD) statistic for impulse noise detection by using a reference image which is similar to RORD.Different from RORD detector, our impulse detector does not need sort operation, which can reduce computation complexity.Then, we propose an improved trilateral filter by combining spatial gradient and the DARD statistic.Instead of applying the "detect and replace" methodology of most impulse noise removal techniques, we integrate our two statistics into a filter designed to remove impulse noise, Gaussian noise, and mixed noise.Finally, a two-step iterative algorithm mentioned in [22] is adopted, which includes getting reference image and image denoising.
The remainder of the paper is arranged as follows.In Section 2, we first briefly review ROAD, ROLD, and RORD statistics and then introduce our DARD statistic.Section 3 describes how to incorporate our statistics into the bilateral filter to create two new bilateral filters for Gaussian and impulse noise, respectively, and a new trilateral filter for mixed noise.In Section 4, we provide the simulations on noise detection and noise removal with visual examples and numerical results.Finally, conclusions are drawn in Section 5.
First, let  , denote the absolute difference between  , and its neighbor  +,+ in a (2 + 1) × (2 + 1) window as Then we define where   , is the th smallest one among  , .For noisy pixels, their intensities vary greatly from their neighbors and yield large ROAD values, while noise-free pixels should have similar intensities with their neighbors and produce small ROAD values.Thus, we can identify noise by the ROAD value.However, the ROAD value of a pixel may not be large enough for it to be distinguished from noise-free pixel when the noise value is close to its neighbors [21].Then, ROLD was proposed to solve it by introducing a logarithmic function on the absolute difference ḋ, defined as Then, define ROLD as where ṙ  , is the th smallest one among ḋ, .Although ROLD is more accurate than ROAD to separate noisy pixels from noise-free pixels, it significantly increases computation complexity.
Yu et al. found that ROAD is not accurate at edge pixels because edge details in an image also cause large absolute difference values [22].Thus, they solved it by introducing a reference image, which is defined as Then, d, and RORD were defined as follows: where r  , is the th smallest one among d, .Merging (5) into (6)   +,+ may be noisy especially when the noise level is high.
In such case, RORD may be not accurate.Although it may be more robust by accumulating  smallest ones among d, , it brings errors at the same time.Take Figure 1 for example, in the 3 × 3 window of the center pixel, half or more than half of its neighbors are corrupted with impulse noise, and then RORD will falsely mark the center pixel as a noise.However, the proposed DARD can work well in this case, which will be described in the next subsection.

Definition of DARD.
Our new impulse detector is based on the following two assumptions.(1) Noise-free images contain locally smoothly varying areas separated by image edges [10].(2) Noisy pixels take gray scale values substantially smaller or larger than their neighborhoods [26,27].Here, we only focus on the edges aligned with four main directions shown in Figure 2. Let   ( = 1 to 4) denote a set of coordinates aligned with the th direction centered at (0, 0) with window size (2 + 1) × (2 + 1), that is, At first, we introduce a reference image ( ref ) that contains the original image edge information from the noisy image.In the filter window centered at (, ), for each direction, define   , as the sum of all absolute differences of gray-level values between centered pixel in  and its neighbors in  ref .Then, we have Mean DARD value Similarly, we define   , as the sum of all differences of each direction in the reference image as Finely textured or detailed regions in an image cause some kind of naturally large absolute difference values.Therefore, it is difficult to tell the difference between a texture and an impulse noise.In order to solve this case and improve the accuracy of impulse noise detection, we define an absolute relative difference statistic for each direction as follows: If  is set to 1, and the reference image is equal to the original image, then, all intensity variation caused by edges can be eliminated.However, the reference image is just generated from the noisy image by median filter (detailed generation process is shown in Section 3.2), in which only rough information of the original image is contained.In other words, the reference image can still be noisy, not only the edge details, which can also cause large intensity variation.To reduce it, we set  = 0.5 for simplicity.Finally, the directional absolute relative difference statistic is defined as Then, the decision making mechanism can be realized by employing a threshold and the impulse noise detection algorithm is shown as In general, salt-and-pepper noisy pixels have larger DARD values while random-valued impulse and Gaussian ones have smaller DARD values, which can be seen in Figure 3.

The Proposed Method
3.1.New Weighting Functions.Bilateral filter presented by Tomasi and Manduchi [4] has been proven to be very useful in removing Gaussian noise and simultaneously preserving edge details.Its main idea is combining grey levels based on both the photometric similarity and geometric closeness.It shows great results but takes a long processing time.Although some improved methods have been proposed, they are not efficient enough [7,28].Furthermore, the performance of the bilateral filter is degraded with high noise level as mentioned above.In this work, we present two new weighting functions for designing filters, as discussed below.Spatial gradient statistic is first introduced into the bilateral filtering framework through replacing the radiometric weighting function.The new created bilateral filter, named SG-BF, is capable of removing Gaussian noise while keeping edge details.The new weighting function is defined as Here, (, ) is gradient which can be generated by using Sobel operator.Let  , be the current pixel, and let  +,+ be the pixels in a (2 + 1) × (2 + 1) window that surrounds  , ; (, ) and ( + ,  + ) are the locations of  , and  +,+ .Then the new output of SG-BF is defined as where By replacing radiometric weighting function with   , our running time can be significantly reduced comparing with the original bilateral filter.In order to let bilateral filter be capable of removing impulse noise, we further incorporate the DARD statistic into the bilateral filter to create a new bilateral filter, and we name it DARD-BF.The new weighting function is defined as Then, the new output of DARD-BF bilateral filter is defined as Finally, we combine the spatial gradient with the DARD statistic to create a new trilateral filter, which can remove both Gaussian and impulse noise, and we name it SG-DARD-TRIF.
The new output of SG-DARD-TRIF is defined as In brief, our new trilateral filter can not only preserve the bilateral filter's ability to remove Gaussian noise but also work well for impulse noise.For images with no impulse noise, the value of impulsive component is nearly to one except for few points with high DARD values, and thus the impulsive component will be "shut off " and only the spatial and gradient weights are used.Essentially, the trilateral filter reverts to the bilateral filter when processing images with Gaussian noise only.For images with impulse noise only, the gradient component will also help to enhance the performance of impulse noise removal.

Denoising Algorithm.
As can be seen, the reference image  ref plays an important role in our algorithm.In order to get a satisfactory denoising result, we adopt the two-step iterative algorithm mentioned in [22].In the first step, the initial reference image is generated by using the standard median filter (SMF).If the noise ratio is high, two or more iterations are needed.In the second step, the previously generated restoration result is used as the final reference image.Then the more accurate impulse noise detection result is obtained by using the more satisfactory reference image.
Finally, the final restored image will be obtained by using our new bilateral filter (DARD-BF) or trilateral filter (SG-DARD-TRIF).Different from [22], iterative operation is only applied in the first step.Our new denoising algorithm, DARD-BF and SG-DARD-TRIF, is summarized as follows.
In general, we use 3 × 3 window median filtering in step 1, and use 5 × 5 window for calculating DARD values in step 2. If the noise ratio is higher than 25%, two or three iterations ( max = 2 or 3) and 5 × 5 window median filtering are needed in step 1.By tries and errors, threshold  is set as [0.01, 0.10], with a higher value for salt-and-pepper noise but a lower one for random-valued impulse noise and Gaussian noise.It is worth noting that we do not employ threshold  in step 1, which is different from many other methods [6,17,21,22].Although some suggestions of threshold selection are given in their literatures, it still needs trial and error.Therefore, we do not apply "detect and replace" scheme.In this work, threshold  is not as important as the other methods, which means that the proposed method can still work well when threshold  is set as 0.

Simulations
The performance of the proposed filters have been evaluated and compared with those of several existing filters for image restoration.The proposed method produced results superior to other methods in both visual image quality and quantitative measures.Simulations were made on several 512 × 512 gray scale standard test images corrupted with Gaussian noise, salt-and-pepper noise, random-valued impulse noise, and mixed noise.For illustrations, the results for images "Lena", "Boats", "Bridge", "Baboon", and "Barbara" are presented here.

Selection of Parameters.
There are three parameters in our algorithm:   (controls spatial weight),   (controls gradient weight), and   (controls DARD weight).From simulations on a large variety of images, we found that the better performance was obtained by the following settings:   should be in the interval [0.compared to ROAD, ROLD, and RORD.Here, we suppose the locations of all noisy pixels are known in advance, and then all pixels can be grouped into two sets: the noisy pixel set and the noise-free pixel set.
A good noise detector should be able to identify most of the noisy pixels and noise-free pixels, and yet its classification rate should be as high as possible.The classification rate (CR) is defined as CR = number of correctly detected noise pixels total number of pixels + number of correctly detected noise-free pixels total number of pixels . ( The results are shown in Table 1.From the experiment results, it can be seen that our DARD detector achieves significant improvement over other detectors for salt-andpepper noise, especially when the noise level is higher than 40%.For random-valued and mixed impulse noise, our DARD detector also shows better results than ROAD and ROLD detectors.Although the classification rate is slightly lower than RORD, our DARD detector has fewer edge pixels falsely detected as noise pixels.The reason is due to the fact that more pixels in subwindow are considered in RORD.
Figure 4 shows the results in detecting Lena image, which is corrupted with 20% random-valued impulse noise, where the white denotes detected noise pixels and the black denotes noise-free pixels.It is clear that the DARD detector has the fewest edge pixels falsely detected as noise pixels.

Image Quality.
To ensure that our approach provides a visually pleasing output, we make three simulations as follows.One is the Lena image contaminated by mixed saltand-pepper and Gaussian noise with  = 20% and  = 10.

Signal Restoration.
The objective quantitative measures used for comparison are the mean absolute error (MAE) [1] and the peak signal-to-noise ratio (PSNR) [1] between the original and restored images, defined by where  , and   , denote the pixel values of the original image and the restored image, respectively, and the image size is × .Larger PSNR value signifies better image restoration while lower for MAE.Our first goal is to ensure that the proposed filters can effectively restore the pixels corrupted by impulse noise.This can be justified by comparing the performance of our approach with other well-known filters for impulse noise reduction.The group of these filters consists of the standard median filter, the adaptive center-weighted median filter (ACWMF) [29], the SDROM filter, the switching bilateral filter (SBF), ROAD-TRIF, ROLD-EPR, ROLD-WMF, and DWM filter.The results of PSNR and MAE values on saltand-pepper noise are shown in Table 2.The proposed method shows significant better PSNR and MAE values than other filters both for Lena and Boats images.Table 3 shows the PSNR and MAE values for random-valued impulse noise, and the proposed filters also show better results than other filters except the RORD-WMF.It is noted that the PSNR and MAE values of our SG-DARD-TRIF are slightly better than our DARD-BF, which denotes that the use of the spatial gradient can enhance the filter's performance.
We also compared the performance of the proposed SG-BF with the performance of the previously tested filters on  images corrupted with Gaussian noise.From Table 4, we can see that SG-BF produces nearly the same results with the bilateral and trilateral filters when  ≤ 20, but it shows better results than these two filters when  = 40.Thus, our SG-BF is better than these existing filters when image is highly corrupted with Gaussian noise.Tables 5 and 6 show PSNR and MAE values corrupted with three kinds of mixed noise: salt-and-pepper and Gaussian with  = 20% and  = 10; random-valued impulse and Gaussian with  = 20% and  = 10; salt-and-pepper and random-valued impulse with

Conclusion
Many noise removal algorithms, such as the ROAD-TRIF and ROLD-EPR, tend to neglect the image edge information and, hence, end with unsatisfactory results.RORD-WMF introduces a reference image which is obtained by standard median filter to solve this problem; however, the edge direction information is still not considered.Although the DWM filter introduces the edge direction information into the median filter, the performance hardly dependeds on the accuracy of edge direction calculation.Furthermore, most of the existing filters do not have the ability of removing both impulse noise and Gaussian noise or cannot perform well.Therefore, a new trilateral filter based on DARD statistic and spatial gradient is proposed to handle these problems.The DARD statistic represents how impulse-like a particular pixel is in the sense that the larger the impulse, the greater the DARD value, while the spatial gradient is for Gaussian component weight.We incorporate the DARD statistic and spatial gradient into the Gaussian filtering framework by adding two components to the weighting functions.The weighting functions of the new trilateral filter contain spatial, gradient, and impulsive component.The gradient component combined with the spatial component smoothes away the Gaussian noise, while the impulsive component removes larger impulse noise.To demonstrate the superior performance of the proposed method, extensive experiments have been conducted on several standard test images to compare our method with many other well-known techniques.Experimental results indicate that the proposed method performs better in removing Gaussian and mixed noise as well as in removing impulse noise than many other existing techniques.

3 200Figure 1 :
Figure 1: Two failure examples of RORD compared with our DARD.Red denotes noisy pixels.

Figure 3 :
Figure 3: Error-bar charts for our DARD statistic on (a) "Lena" with salt-and-pepper impulse noise and (b) "Boats" with random-valued impulse noise.
(a) Set  = 1,  ref =   , and   = SMF().(b)Restore all pixels by DARD-BF or SG-DARD-TRIF and get the new reference image  +1 ,  =  + 1. (c) If  ≤  max , set  ref =   and then go to step 1(b); Otherwise, stop iteration and get the final reference image.
, d, can be rewritten as

Table 1 :
Comparison of noise detectors in CR.
while [0.4,1.0] for others.In addition, higher values of   and   work better with high noise level and higher   performs well in images with more textured details.4.2.Noise Detection.To demonstrate the effectiveness of our DARD statistic, we make a test on a 512 × 512 Lena image

Table 2 :
Comparative restoration results in PSNR (dB) and MAE (the second row) for salt-and-pepper noise.

Table 3 :
Comparative restoration results in PSNR (dB) and MAE (the second row) for random-valued impulse noise.

Table 4 :
Comparative restoration results in PSNR (dB) and MAE (the second row) for Gaussian noise.
= 30%.The proposed SG-DARD-TRIF consistently yields the best PSNR and MAE values for each image corrupted with the first two mixed noises.Note that it outperforms the other methods a large margin in images with salt-and-pepper and Gaussian noise due to its high salt-and-pepper noise detection rate.For mixed impulse noise, the RORD-WMF shows better results in each image and the SG-DARD-TRIF is very close to it.