Decision-Based Marginal Total Variation Diffusion for Impulsive Noise Removal in Color Images

Impulsive noise removal for color images usually employs vector median filter, switching median filter, the total variation L1 method, and variants. These approaches, however, often introduce excessive smoothing and can result in extensive visual feature blurring and thus are suitable only for images with low density noise. A marginal method to reduce impulsive noise is proposed in this paper that overcomes this limitation that is based on the following facts: (i) each channel in a color image is contaminated independently, and contaminative components are independent and identically distributed; (ii) in a natural image the gradients of different components of a pixel are similar to one another. This method divides components into different categories based on different noise characteristics. If an image is corrupted by salt-and-pepper noise, the components are divided into the corrupted and the noise-free components; if the image is corrupted by random-valued impulses, the components are divided into the corrupted, noise-free, and the possibly corrupted components. Components falling into different categories are processed differently. If a component is corrupted, modified total variation diffusion is applied; if it is possibly corrupted, scaled total variation diffusion is applied; otherwise, the component is left unchanged. Simulation results demonstrate its effectiveness.


Introduction
Noise reduction is a fundamental issue in image processing.In order to reduce noise, a great number of works have been presented over the past several decades.The overwhelming majority of these works cope with the gray-scale images.With the increase in use of color images, however, more and more works to reduce noise for color images are rapidly growing.
Indeed, color images often are corrupted by various types of noise.In this paper, impulsive noise is only considered, containing salt-and-pepper noise and random-valued impulses where they present themselves as occurring isolated chromatic points.To yield better images from their noisy versions, a series of various methods have been proposed.
Vector median filter (VMF) [1] and variants, based on reduced ordering, are popular methods.VMF calculates the cumulative sums of the Euclidean distance from every pixel to other pixels in a filtering window and outputs the pixel corresponding to the minimization sum of distances.The output pixel is called vector median, since each pixel is a vector containing multiple components.To achieve better performance in the variants, a few dissimilarity measures are used other than the Euclidean distance.For example, Basic Vector Directional Filter (BVDF) uses the angular distance [2]; Directional Distance Filter (DDF) employs the combination of the magnitude and directional processing [3].In addition, -Trimmed Vector Median Filter (T  VMF) only considers alpha pixels to calculate the dissimilarity [4], rather than all pixels within a filtering window.These methods process the noisy and noise-free pixels uniformly.Therefore they introduce extensive smoothing and blur visual features extensively and are only suited to low density noise.
To overcome this limitation, various kinds of switching vector filters have been proposed in the rich literature, which aim to only replace the corrupted pixels.This type of filters usually consists of noise detection and noise removal.The former identifies the corrupted pixels, and the latter applies some algorithm to those corrupted pixels.The noise detection is crucial.If a detector fails to identify corrupted pixels, then they will be left unchanged, resulting in poor filtered images.If the detector classifies a corrupted pixel correctly but also identifies noise-free pixels as corrupted, the tiny details may be lost.Therefore, accurate and complete identification determines the quality of filtered images.
Currently, there exist many techniques to identify noisy pixels.Peris-Fajarnes et al. [5] use the rank-ordered differences statistic method.Smolka [6] uses a soft switching technique that assigns trimmed cumulated distances to the central pixel, where the trimmed sum of distances serves as an indicator manifesting the corrupted extent of the current pixel.Morillas et al. [7] utilize the local self-adaptive fuzzy filter.Ananthi and Balasubramaniam [8] use the technique of the interval-valued fuzzy set.Jin et al. [9] employ the quaternion theory.Malinski and Smolka [10] use adaptive switching technique.Particularly, after the concept of a peer group is introduced, many schemes based on it have been presented; for example, Morillas et al. [11] use the fuzzy peer groups, and Malinski and Smolka [12] utilize the fast averaging peer group.
Apart from the filters mentioned above, there exist a family of variational and partial differential equations.This type of methods usually adopts a variational energy minimization model to obtain the solution, such as the well-known Rudin-Osher-Fatemi (ROF) model [13], the hybrid model [14], the modified total variation diffusion method [15], and the total variation  1 model [16].The main advantage for them is to use the gradient of images to represent the model, and thus they have the capability of edge-preserving.Unfortunately, this type of methods has the drawback similar to the VMF; that is, they only work well on the low density noise because they do not distinguish between the corrupted and the uncorrupted pixels.
Inspired by switching filters and variational methods, in this paper a decision-based marginal diffusion method is proposed to reduce impulsive noise for color images.The proposed method, in contrast to vectorial methods emphasizing the correlation between the channels, independently treats each channel in a color image and implements diffusion operations on every corrupted component rather than every vector.In addition, the proposed method divides components into different categories based on different noise characteristics.If an image is corrupted by salt-and-pepper noise, the components are divided into the corrupted and the noise-free components; if the image is corrupted by randomvalued impulses, the components are divided into the corrupted, noise-free, and the possibly corrupted components.Components falling into different categories are processed differently.If a component is corrupted, a modified total variation diffusion is applied; if the component is possibly corrupted, a scaled total variation diffusion is applied; otherwise, the component is left unchanged.Experimental results show that the proposed method is robust to different noise strengths and suitable for different images, with strong noise removal capability as shown by PSNR/MSSIM/FSIM results as well as the visual quality of restored images.
The rest of this paper is organized as follows.In Section 2, the details of noise removal are introduced.Section 3 reports experimental results and comparisons, and conclusions for this paper are drawn in Section 4.

The Proposed Denoising Method
In this section, the details of reducing salt-and-pepper noise and random-valued impulses are described, containing denoising structures, noise models, noise detection, and noise removal.
2.1.Noise Removal Structure.Two denoising structures are devised based on two noise characteristics, respectively.As shown in Figure 1, inset (a) is associated with salt-andpepper noise and inset (b) is associated with random-valued impulses.Both structures consist of noise detection module and noise removal module, corresponding to detection stage and removal stage.For salt-and-pepper noise, a detector divides components into the corrupted and the noise-free components; for random-valued impulses, another detector divides components into the corrupted, noise-free, and the possibly corrupted components.Based on the two detection results, two mask arrays are built where each entry is a label indicating which category the corresponding component belongs to.Components falling into different categories are processed differently in separate channels.If a component is corrupted, modified total variation (MTV) diffusion is applied; if the component is possibly corrupted, scaled total variation (STV) diffusion is applied; otherwise, the component is left unchanged.To achieve better results, diffusion operations are iteratively implemented following the corresponding mask array instructions.Especially, for random-valued impulses, a hierarchical scheme is adopted in removal stage; the corrupted components are first processed and then the possibly corrupted ones, as in inset (b).

Noise Model and Analysis
. RGB images to be processed are only considered in this paper, which contain three channels, the red, green, and blue, and a color image is defined as a two-dimensional matrix consisting of a certain number of pixels.Let u , = (  The probability of a noise-free component processed incorrectly is analyzed in the vectorial and marginal methods.Assuming detectors can correctly judge whether every component is corrupted or not, then the marginal methods can correctly process all components in the removal stage.The vectorial methods, however, may identify noise-free components as noisy to process, because the processing is based on the pixel unit rather than the component unit.If a component is corrupted with probability , then a pixel is not corrupted with probability (1 − ) 3 since a pixel may have corrupted 1, 2, or all 3 channels.Therefore, in vectorial methods a noise-free component is processed incorrectly with probability As shown in Figure 2, with the increasing probability , the probability of a noise-free component processed incorrectly is decreased.In an extreme case, only when all components are corrupted, can every component be correctly processed.Just from this viewpoint, the marginal methods are superior to the vectorial methods.The probability of processing incorrectly  be a current component centered in the sliding window; its neighborhood is defined as where  is the neighborhood radius quantified into the positive integer domain and Ω  denotes the th channel domain.Obviously, a neighborhood contains (2 + 1) × (2 + 1) samples.
The Detector Used in SPM.This detector takes components with intensity 0 or 255 as the noisy candidates.Let   , be a candidate, the absolute differences of intensities between it and every sample within the neighborhood are calculated, and then the number of samples, whose absolute difference is larger than a predefined threshold value, is determined, expressed as where  is the predefined threshold value, # denotes the cardinality of the set, and   , is the number of the samples that satisfy the threshold condition.
The number  , is used to determine if a current candidate is corrupted.The determining function is In this formula, if (, ) = 1, then candidate   , is corrupted; otherwise, it is uncorrupted.When all candidates are complete, a binary mask array can be built; each entry is either 1 or 0.
The Detector Used in RDM.This detector takes all components as the noisy candidates, which is devised based on the Rank-Ordered Absolute Differences (ROAD) [19].It assigns an index to each candidate.Based on these indices, the corresponding candidate is judged as noise-free, corrupted, or possibly corrupted.Let   , be the current candidate; the ROAD of   , is defined by where Then, the index of   , is where Moreover, components falling into different categories are processed in different diffusion ways.In SPM, the components are divided into the corrupted and noise-free components.If a component is corrupted, the MTV diffusion is applied; otherwise, the component is left unchanged.Let   be the th channel of the noise-free image,   0 be its noisy version, and Θ  ⊂ Ω  be the th distorted channel domain where Ω  ⊂ Ω is the th channel domain and Ω = {Ω 1 , Ω 2 , Ω 3 }; then the MTV diffusion is expressed as Applying Euler-Lagrange to (9), the corresponding descent flow is where  and div are the time step and the divergence operator and where  is a positive lifting parameter avoiding the |∇  | vanishing.
In RDM, the components are divided into the corrupted, noise-free, and the possibly corrupted components, and a hierarchical scheme is adopted that first processes the corrupted and then the possibly corrupted ones.As shown in inset (b) in Figure 2, in the first stage, if a component is corrupted, the MTV diffusion is applied; otherwise, the component is left unchanged.In the second stage, if a component is possibly corrupted, the STV diffusion is applied; otherwise, the component is left unchanged.Let Θ [V] denote the corrupted domain of the th channel; the descent flow of MTV in RDM is as follows: (, , 0) =   0 . ( Let Θ  [] denote the possibly corrupted domain of the th channel where } and   V denotes the th estimated channel by the MTV diffusion; then the descent of STV is where ℎ  (, ) is the scaled parameter, and the parameter value is obtained according to (8): where    and    are the first-order derivatives in the x and y directions in the th channel, respectively.
At the half-pixel (,  + 1/2) in the th channel, the following three formulas hold: In addition, Substituting ( 16) and ( 18) to ( 14 where Then, the explicit discrete schemes associated with ( 10), (11), and ( 12) are, respectively, as follows: In these equations, Δ is the time step-size, and the superscript  denotes the th iteration.

Experiments
In this section, three metrics used to evaluate denoising performance are first introduced containing Peak Signal to Noise Ratio (PSNR), Mean Structural Similarity Index Measure (MSSIM), and extended Feature Similarity index (FSIM  ), then the setting parameters are discussed, and finally the experimental results and comparisons are reported.
3.1.Three Metrics.PSNR, given in decibels (dB), is a measurement based on mean pixel intensity errors between the noisefree and the restored images.Higher PSNR means better denoising capability.Let  and û be the noise-free and the restored images, respectively,  denote the number of pixels in , and ‖ ⋅ ‖ F denote the Frobenius norm; the calculation of PSNR is MSSIM is a measure based on structural similarities [20].It is bounded in the range [0, 1], where the closer the value is to 1, the better the denoising scheme is implied.MSSIM is mean value of the structural similarities between the blocks of the noise-free image  and the restored image û.
FSIM  is a measurement based on feature similarities [21].
Similar to MSSIM, it is also bounded in the range [0, 1], where the closer the value is to 1, the better the denoising performance is implied.This index is a combination of Phase Congruency (PC) and Gradient Magnitude (GM).Let x be a pixel and  > 0 be a parameter.  (x) =  PC (x)  (x) and   (x) =   (x)  (x), where symbol  means similarity, and the subscripts  and  denote chromatic channels; the calculation of FSIM  is

Parameter Setting
The Parameters in SPM.Five parameters are set in the proposed SPM method; (, , thr) are used in salt-and-pepper noise detection, and (Δ, ) are used in diffusion operations.The parameters,  and thr, are two thresholds where the former judges the relationships between a component and its neighbors and the latter decides whether the component is corrupted or not.The influence of both on the PSNR denoising results for different intensity noise is shown in Figure 5.For all noise densities, , , thr, and Δ were set to 3, 25, 3, and 0.8, respectively.The parameter  is the desirable iterative time that depends on noise strength  and the time step-size Δ. Figure 6 illustrates that the PSNR and MSSIM results for fixed strength noise depend on the number of iterations when Δ = 0.8.From inset (a) in Figure 6, as the number of iterations  increases, the growth of PNSR tends to slow and ultimately achieves the stable value when  meets or exceeds the desirable number , for any fixed noise.The same conclusion for the MSSIM measurement can be drawn  from inset (b) in Figure 6.In our experiments, the values of  are shown in Table 1.
The Parameters in RDM.Seven parameters are set in the proposed RDM method; (, ,  1 ,  2 ) are used in randomvalued impulse detection, and (Δ,  V ,   ) are used in diffusion operations.For all the noise densities, , ,  1 ,  2 , and Δ were set to 2, 14, 150, 320, and 0.5, respectively.The parameter  V is the desirable number of iterations that is used in processing the corrupted pixels; the parameter   is the desirable number of iterations that is used in processing the possibly corrupted pixels.Both parameters depend on noise strength  and the time step-size Δ.When Δ = 0.5, the values of two parameters used in our experiments are shown in Table 2.

Experimental Results and Comparisons
Experimental Results and Comparisons in SPM.A test set, denoted by Γ, was built to validate the proposed SPM method.This set contained contaminative versions of seven images, and each image was corrupted by ten different densities of   To augment the evaluation, the proposed method is compared with other five methods including MMF [22], AMMF [23], VMF [1], TVL 1 in [18], and TVL 1 in [17].The MMF filter used the sliding window of size 3 × 3.
In the AMMF, when the noise densities were 0.05, 0.1, 0.2, 0.3, 0.4, 0.5 0.6, 0.7, 0.8, and 0.9, the filtering window radius values were set to 1, 1, 2, 3, 3, 4, 4, 5, 6, and 7, respectively.The VMF used the sliding window of size 3 × 3. The source codes of TVL 1 in [18] and TVL 1 in [17] were taken from their authors, and the parameters used in the experiments were those recommended by the authors.The five methods were also applied to the same test set Γ.The PSNR, MSSIM, and FSIM results for the Lena image are also reported in Table 3. Visual comparison results for the Pepper and Baboon images are shown in Figures 9 and  10, respectively, and the information about these images is labeled sideways.In addition, the mean PSNR, MSSIM, and FSIM results for fixed noise are calculated for noisy images, the MMF, AMMF, VMF, TVL 1 in [18], TVL 1 in [17], and the proposed SPM method, respectively.The calculations are as follows: In the above three equations, Γ  denotes the noisy images with the same density noise  in the test Γ and PSNR  (), MSSIM  (), and FSIM  () denote the PSNR, MSSIM, and FSIM values of the th image with the  density noise, respectively.For example, if the MMF is used, FSIM 0.1 denotes the mean FSIM value of all the images with the 0.1 density noise that is associated with the MMF.The mean values of the three metrics for different methods on different noise densities are plotted in Figure 11.
Experimental Results and Comparisons in RDM.By using the seven noise-free images in Figure 7, another test set was built for the proposed RDM method, denoted by Ψ, where each noise-free image was corrupted by different seven randomvalued impulses.Using the values of parameters mentioned in RDM in Section 3.2, the proposed RDM method was applied to the test Ψ.The PSNR, MSSIM, and FSIM results for the London image are reported in Table 4, and the visual results for the Pepper and Baboon images are shown in Figures 12  and 13, respectively.For comparisons, other five methods, MMF, AMMF, VMF, AVMF [24], and TVL 1 [18], were applied on the test set Ψ.Both of the MMF and VMF used the sliding window of size 3 × 3.In both of the AMMF and AVMF, when the noise densities were 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, and 0.6, the filtering window radius values were set to 1, 1, 2, 2, 3, 3, and 4, respectively.The source codes of TVL 1 [18] are taken from their authors, and the parameters used in the experiments are those recommended by the authors.The PSNR, MSSIM,  and FSIM results for the London image are also reported in Table 4. Visual comparison results for the Pepper and Baboon images are also shown in Figures 12 and 13, respectively, and the information about these images is labeled sideways.
Similar to the comparisons in SPM, the mean PSNR, MSSIM, and FSIM results for fixed noise are calculated for noisy images, MMF, AMMF, VMF, AVMF, TVL 1 [18], and the proposed RDM method, respectively, according to (28).The mean values of the three metrics for different methods on different noise densities are plotted in Figure 14.

Observations and Conclusions.
The following observations and conclusions can be drawn from the quantitative measurements and visual comparison results.First, the proposed method achieved the best results in every case tested.The proposed SPM method achieved 14.05 dB, 10.68 dB, 14.72 dB, 17.93 dB, and 8.27 dB improvements over the MMF, AMMF, VMF, TVL 1 [18], and TVL 1 [17] on average, respectively, in the PSNR results, and achieved 0.472, 0.278, 0.519, 0.554, and 0.293 improvements in the MSSIM results and achieved 0.254, 0.164, 0.274, 0.398, and 0.169 improvements in the FSIM results.The proposed RDM method outperformed MMF, AMMF, VMF, AVMF, and TVL 1 [18] by 3.56 dB, 4.16 dB, 4.80 dB, 3.73 dB, and 2.45 dB on average, respectively, in the PSNR results, and by 0.200, 0.182, 0.286, 0.231, and 0.135 in the MSSIM results and by 0.064, 0.097, 0.095, 0.102, and 0.071 in the FSIM results.Second, the proposed method has a strong capability to preserve detail.The proposed method reconstructed more image details from noisy images than the comparative methods.Third, the proposed method was more robust to different densities of noise than the comparative methods.In summary, the proposed method shows strong Zoom-in TVL [18] Zoom-in capability of reducing noise not only in SPM but also in RDM, in terms of the PSNR/MSSIM/FSIM results, in addition to producing a higher visual perception quality in restored images, in comparison with results from other methods.

Figure 1 :
Figure 1: Two structures of the noise removal.The operations surrounded by the rectangle with dashed lines are repeated.

Figure 2 :
Figure2: Illustration of the relationship between two probabilities, where a component is corrupted and a noise-free component is processed incorrectly, assuming a vectorial method is applied.

Figure 3 :
Figure 3: Illustration of RGB three channels and gradients of component in separate channels.
Discretization of the divergence div(∇  /|∇  |  ) must be discussed.As shown in Figure 4, the techniques of central finite difference and half-pixel resolution discretization are used.Let   , be a current component; the divergence at the current   , is expressed as div

Figure 4 :
Figure 4: The representation of discretization at half-pixel resolution in a channel.

Figure 5 :Figure 6 :
Figure 5: Dependence of PSNR measures on the parameters  and thr for various noise levels.

Figure 8 :
Figure 8: Visual results from the proposed SPM method for the Lena image.
1and  2 are predefined threshold parameters.The index ℎ  (, ), for (, ) ∈ Ω  , is a determining function, which divides candidates into three categories.If ℎ  (, ) = 0, then   , is declared as noisy-free; if ℎ  (, ) = 1, 2.4.Noise Removal.In the removal stage, diffusion operations are implemented in separate channels.The main reason is twofold.First, each channel in a color image is contaminated independently, and contaminative components are independent and identically distributed.Second, in a natural image the gradients of different components of a pixel are similar to one another, where Figure3illustrates this property.
), the divergence div(∇  / |∇  |  ) at the current , )] , Therefore, the computation involves two blocks, denoted by  1 and  2 , respectively.Let   1 ,   2 be the mean values of  1 and  2 , respectively,   1 and   2 the variances, and   1  2 the covariance, and let  1 and  2 denote two stabilization variables; then the calculation of SSIM is

Table 1 :
The desirable number of iterations  for different density noise when Δ = 0.8.

Table 2 :
The value of the parameter  used in RDM.

Table 3 :
Comparison of the efficiency with those of other methods for the Lena image in SPM.

Table 4 :
Comparison of the efficiency with those of other methods for the London image in RDM.