Mixed Noise Removal Algorithm Combining Adaptive Directional Weighted Mean Filter and Improved Adaptive Anisotropic Diffusion Model

Amixed noise removal algorithm combining adaptive directional weightedmean filter and improved adaptive anisotropic diffusion model is proposed. Firstly, a noise classification method is introduced to divide all pixels into two types as the pixels corrupted by impulse noise and the pixels corrupted by Gaussian noise.Then an adaptive directional weightedmean filter is developed to remove impulse noise, which can adaptively select the optimal direction template from twelve direction templates and replace the gray level of each impulse noise corrupted pixel by the weighted mean gray level of pixels on the optimal direction template. Finally, an improved adaptive anisotropic diffusion model is developed to remove Gaussian noise in the initial denoised image, which can finely classify image features as smooth regions, edges, corners, and isolated noises by characteristic parameters and variance parameter and conduct adaptive diffusion for different image features by designing reasonable eigenvalues of diffusion tensor. A large number of experimental results show that the proposed algorithm outperforms many existing main mixed noise removal methods in terms of image denoising and detail preservation.


Introduction
Noise is more or less introduced into an image in the process of image acquisition and transmission, which can significantly deteriorate image quality and increase difficulty in the subsequent image analysis [1].The task of image denoising is to remove noise from the corrupted image and meanwhile preserve edges as much as possible.The nature of image denoising depends on the types of noise added to the image.Most commonly noises are additive white Gaussian noise (AWGN), impulse noise (IN), and the mixture of AWGN and IN.
The AWGN is usually introduced into an image due to the thermal motion of electron in camera sensors and circuits, which is the most widely studied noise model in image denoising.The traditional linear filter such as mean filter can remove the AWGN but blurs edges.To overcome the disadvantage, the bilateral filter (BF) [2] uses the geometric and photometric distances based weights.The nonlocal means (NLM) filter [3] can be viewed as a significant extension of the BF based on the fact that similar pixels may not be necessarily spatial neighbors.The nonlinear diffusion filtering method such as Perona-Malik (P-M) model [4] is a well-known method based on a partial differential equation.The P-M model changes the diffusivity with the gradient during denoising, which has good capability in edge preservation.However, the P-M model has a significant shortcoming that the blocky effects exist in smooth regions after denoising.A coherence-enhancing anisotropic diffusion denoising (CAD) model [5] is developed to solve the problem by designing the diffusivity as a tensor.However, many undesired false edges appear in smooth regions after denoising.Recently, many modified models have been developed [6][7][8].
The IN is often introduced into an image by faulty memory locations or bit errors [9].Salt-and-pepper impulse noise (SPIN) and random valued impulse noise (RVIN) are the two common types of the IN.The standard median filter [10] is the most popular nonlinear filter for removing the IN, which is effective at low noise density.When the noise density is over 50%, some important edges will be blurred.To overcome the drawback, various modified median filters have been developed [11][12][13][14][15][16].The switching median filter is a popular technique to remove the IN, which firstly identifies the IN corrupted pixels and then replaces the gray level of each IN corrupted pixel by the median value of its neighborhood pixels.The switching median filter with boundary discriminative noise detection (BDND) [17] identifies the IN corrupted pixels by adaptively selecting the filtering window of suitable size.The directional weighted median (DWM) filter [18] detects the IN corrupted pixels by considering the neighborhood information of each pixel along four directions.The modified directional weighted median (MDWM) filter [19] is modified based on DWM filter, which considers the neighborhood information of each pixel on more edge directions than DWM filter.The modified directional weighted (MDW) filter [20] detects the IN corrupted pixels by combining the directional gray level difference with gray level extreme and replaces the gray level of each IN corrupted pixel by the weighted mean gray level of its neighborhood pixels.
In many cases, images may not be corrupted by only one type of noise, but mixed types of noise such as the mixture of AWGN and IN.In order to restore the image corrupted by the mixed noise, many methods have been developed [21][22][23][24][25][26].The trilateral filter (TF) [22] incorporates the rankorder absolute difference statistics into BF framework for IN detection.Switching bilateral filter (SBF) [23] introduces the sorted quadrant median vector into BF to detect IN corrupted pixels.A nonlocal mixed noise filter [24] can skillfully extend NLM into the mixed noise circumstance by combining NLM with TF.A universal denoising framework [25] can remove the mixed noise by combining a complex detection mechanism with an improved NLM.A modified two-phase filter (MTF) [26] can effectively remove the mixed noise from the corrupted image, and the computational performance of this method is further improved in [27].Recently, the low rank approximation (LRA) [28] and low rank representation (LRR) [29] have been used for removing noise.A weighted low rank approximation (WLRA) model and a weighted low rank representation (WLRR) model [30] are proposed to remove the mixed noise, which group image nonlocal similar patches as a matrix and restore the corrupted image by finding the weighted low rank approximation or representation of the matrix.
In this paper, we propose a mixed noise removal algorithm combining adaptive directional weighted mean filter and improved adaptive anisotropic diffusion model.The proposed algorithm can accurately classify IN corrupted pixels and AWGN corrupted pixels by a noise classification method.After noise classification, an adaptive directional weighted mean (ADWM) filter is developed to remove the IN based on the multidirectional image information, which can effectively remove the IN and meanwhile preserve edges and details.Then an improved adaptive anisotropic diffusion (IAD) model is introduced to remove the AWGN in the initial denoised image, which can finely classify image features as four types and control the adaptive diffusion for different image features by designing the reasonable eigenvalues of diffusion tensor.The proposed algorithm makes full use of the advantages of ADWM filter and IAD model.The experimental results show that the proposed algorithm performs better than some existing main methods in terms of image denoising and edge preservation.
The outline of this paper is as follows.Section 2 presents the mixed noise model.A noise classification framework is brought out in Section 3. In Section 4, the proposed algorithm is described in detail.The experiments and results are shown in Section 5. Conclusion is given in Section 6.

Mixed Noise Model
For a clean gray image , the gray level of the pixel at location (, ) is  , .The dynamic gray range of  is between  min and  max .If  is an 8-bit image,  min equals 0 and  max equals 255.Usually, a clean gray image is firstly contaminated by the AWGN during acquisition and then corrupted by the IN in the process of transmission.After that, an image  with mixed noise is generated.The mixed noise model is defined as follows: where  defines the proportion of the IN. is the noise value which is drawn from a Gaussian distribution with mean 0 and standard deviation . is the value of the IN which modifies the original gray level.In this paper, we mainly consider the SPIN mixed with the AWGN.

Noise Classification Method
From the mixed noise model, one can observe that the pixel in an image is corrupted by the AWGN or IN.The distribution of mixed noise cannot be described by a fixed function.Different pixels should be considered differently.Hence, a noise classification method is introduced to divide all the noise corrupted pixels into two types.Since the SPIN corrupted pixel takes the maximum or minimum gray level, its gray level is much different from the gray levels of its neighborhood pixels.Then a variance parameter is introduced to judge the gray level difference between the current pixel and its neighborhood pixels.And, for any one pixel  , in a mixed noise corrupted image, its variance parameter  , in a  ×  local window Ω is calculated as follows: (1) Calculate the gray level variance  1 , of all pixels in the local window by where    (2) Calculate the neat gray level variance  2 , of neighborhood pixels of central pixel by where , , the gray level of central pixel is much different from the gray levels of its neighborhood pixels.Then the pixel with a large variance parameter can be identified as a SPIN corrupted pixel in the high probability.A parameter  is selected as the threshold to judge the value of variance parameter.Besides, considering that the SPIN corrupted pixel takes the maximum or minimum gray level, it can be identified by combining the variance parameter and gray level extreme.Then the detailed process of noise classification is as follows: (1) For any one pixel  , in a corrupted image, establish a local window.
(3) Identify the characteristic of pixel  , in the following way: where  1 and  2 denote the pixels corrupted by the SPIN and AWGN, respectively. , is the gray level of pixel  , in the corrupted image.SPIN corrupted pixels based on the multidirectional image information.The ADWM filter firstly designs 12 direction templates in the local window, then adaptively selects the optimal direction template by calculating the sum of absolute gray level differences between the current noise corrupted pixel and its neighborhood pixels on each direction template, and finally replaces the gray level of each SPIN corrupted pixel by the weighted mean gray level of pixels on the optimal direction template.For any one pixel  , corrupted by the SPIN, the optimal direction template is selected as follows:

Proposed Denoising Algorithm
(1) Establish a  ×  local window Ω centered at the position (, ) and design 12 direction templates as shown in Figure 1.(2) Calculate the sum of absolute gray level difference between the central pixel and its neighborhood pixels on each direction template   by where (  ,   ) ̸ = (, ) and  (1 ≤  ≤ 12) is the direction index illustrated in Figure 1.The weight of each neighborhood pixel    ,  is assigned by where Ω 3 denotes a 3 × 3 local window centered at the position (, ).(3) Find out the minimum sum of absolute gray level difference among the twelve direction templates, and denote it as (4) Select the direction template with minimum sum of absolute gray level difference as the optimal direction template   * .
After selecting the optimal direction template, the gray level of central pixel will be replaced by the weighted mean gray level of the pixels on the optimal direction template.The restoration gray level of the central pixel  , can be calculated as follows: (1) Calculate the chessboard distance ℎ   ,  between the central pixel and the pixels on the optimal direction template by where (  ,   ) ∈   * and (  ,   ) ̸ = (, ).
(2) Calculate the weight   of the pixels on the optimal direction template by where  is the number of the pixels except for the central pixel on the optimal direction template.
(3) Calculate the restoration gray level  1 , of the central pixel by where    ,  denotes the gray level of pixel    ,  on the optimal direction template.
An initial denoised image  1 can be obtained by applying the ADWM filter for the mixed noise corrupted image .Then an improved adaptive anisotropic diffusion (IAD) model is introduced to remove the remaining AWGN in the initial denoised image  1 .

Improved Adaptive Anisotropic Diffusion Model.
The CAD model uses Gaussian filter twice in the diffusion process.The standard deviation in the convolution operation is hard to be determined.It takes multiple experiments to find the optimal value of standard deviation.Hence, in order to improve the calculation efficiency, the IAD model uses Wiener filter to replace Gaussian filer as follows: where (, ) denotes the position of the pixel in the image and  denotes the time.V denotes the divergence operator.∇ denotes the gradient operator. 1  = ( 1 ), and ( ) denotes Wiener filter.For any one pixel  , in the initial denoised image  1 , the gray level obtained by Wiener filter is calculated as follows:  ( 1 , ) =  + where  1 , denotes the gray level of pixel  , in the initial denoised image. denotes the mean value of the gray levels of pixels in a  ×  local window which centers at the position (, ).V denotes the standard deviation of the gray levels of pixels in the local window. denotes the standard deviation of the Gaussian noise.
The structure tensor (∇ 1  ) is defined as follows: where ⊗ denotes the Kronecker product operator. denotes the Wiener kernel.* denotes the convolution operator.The structure tensor (∇ 1  ) is a symmetric positive semidefinite matrix, whose eigenvectors are orthonormal.The eigenvectors of the structure tensor are as follows: where The diffusion tensor ((∇ 1  )) has the same eigenvectors as the structure tensor, that is, V 1 , V 2 .The corresponding eigenvalues of the diffusion tensor are  1 and  2 .Then the diffusion tensor is defined by The CAD model only divides image features into smooth regions and edges, which ignores corners and isolated noises.In order to enhance image features, the IAD model introduces an image feature classification method to classify image features as smooth regions, edges, corners, and isolated noises.Considering that the eigenvalues  1 and  2 of structure tensor can describe the changes of image gradation along directions V 1 and V 2 , two characteristic parameters are defined based on  1 and  2 as follows: Based on the above notion,  1 describes the comprehensive varying property of gray level and  2 expresses the difference of the gray varying along the two directions.Then for any one pixel  , in the initial denoised image, its feature can be identified by the characteristic parameters as follows: (1) If  1 ≈ 0 and  2 ≈ 0, it means that the changes of gray level along V 1 and V 2 are minimal.Then the pixel is located in the smooth region and  1 ≈ 0 and  2 ≈ 0. (2) If  1 ≫ 0 and  2 ≈ 0, it means that the change of gray level along V 1 is very large while the change of gray level along V 2 is minimal.Then the pixel is located in the edge and  1 ≫ 0 and  2 ≫ 0. (3) If  1 ≫ 0 and  2 ≫ 0, it means that the changes of gray level along V 1 and V 2 are very large.Then the pixel is an isolated noise pixel or located in the corner and  1 ≫ 0 and  2 ≈ 0.
From the above analysis, it can be seen that the corners and isolated noises cannot be classified by the characteristic parameters.Since the variance parameter can reflect the gray level difference between the current pixel and its neighborhood pixels, it is used to classify corners and isolated noises.For any one pixel  , in a 3×3 local window, if it is an isolated noise pixel, its gray level is much different from the gray levels of its neighborhood pixels; if it is located in the corner, its gray level is similar to the gray levels of the other pixels located in the corner.Hence, the variance parameter of isolated noise pixel is much larger than the variance parameter of pixel located in the corner.Then a parameter  * is selected as the threshold to judge the value of variance parameter  in a 3 × 3 local window.Then for any one pixel  , in the initial denoised image, its feature can be identified by combining the characteristic parameters and variance parameter as follows: (1) If  1 ≈ 0,  2 ≈ 0, the pixel is located in the smooth region.
Based on the classification result of image feature, the eigenvalues of the diffusion tensor are designed to conduct the adaptive diffusion for different image features as follows: (1) If the pixel  , is an isolated noise pixel or located in the smooth region, its eigenvalues are designed by where  ∈ (0, 1) is a large positive parameter.Both of the eigenvalues  1 and  2 are designed to be large.Then the diffusivity along the two directions V 1 and V 2 is large.Hence, the noise can be effectively removed.
(2) If the pixel  , is located in the edge, its eigenvalues are designed by where  ∈ (0, 1) is a small positive parameter.The eigenvalue  1 is designed to be small while the eigenvalue  2 is designed to be large.Then the diffusivity perpendicular to edge direction is small while the diffusivity along the edge direction is large.Hence, the noise at edges can be removed; meanwhile the edges can be preserved.
(3) If the pixel  , is located in the corner, its eigenvalues are designed by where  ∈ (0, 1) is a small positive parameter.Both of the eigenvalues  1 and  2 are designed to be small.Then the diffusivity along the two directions V 1 and V 2 is small.Hence, the corner can be preserved.
In conclusion, the IAD model can not only enhance the image information by finely classifying the image features but also remove the AWGN in the initial denoised image and meanwhile preserve edges and details.Besides, since the IAD model can identify the isolated noises in the initial denoised image, some residual SPIN can be further removed.
The pseudocodes of the proposed algorithm are provided as shown in Algorithm 1.

Experimental Results
In this section, some experiments are carried out to demonstrate the performance of the proposed algorithm.Six commonly used images are chosen as the test images: Lena, Boat, Peppers, Painting, Couple, and Hill, respectively (refer to Figure 2 for the scenes of six test images).All test images are with the size of 512 × 512.The denoising results of different methods are measured by the Peak Signal-to-Noise Ratio (PSNR) and the Feature Similarity Index Measure (FSIM).Different levels of the AWGN plus SPIN are tested to contaminate the noise-free images.The standard deviation  of the AWGN varies from 5 to 15 with the step length of 5.The ratio of the SPIN varies from 10% to 60% with the step length of 10%.The proposed algorithm is compared with different methods for mixed noise removal as TF [22], SBF [23], MNF [24], Cai [27], AMF coupled with LRA [28], AMF coupled with LRR [29], WLRA [30], and WLRR [30].We firstly compare the proposed algorithm with several classical methods as TF [22], SBF [23], and MNF [24].Figure 3 shows the denoising results of different methods for Lena image, which is corrupted by the AWGN (=10) plus SPIN (=20%).From Figure 3(c), one can observe that the TF method can remove the mixed noise but blur the edges.From Figures 3(d) and 3(e), it can be seen that the SBF and MNF methods can remove the mixed noise and meanwhile preserve edges.However, some noise corrupted pixels are residual in the denoising results of SBF and MNF methods.From Figure 3(f), one can observe that the proposed algorithm can not only effectively remove the mixed noise but also preserve edges and details well.Therefore, Figure 3 shows that the proposed algorithm can perform better than other compared methods.Figure 4 shows the denoising results of different methods for Boat image, which is corrupted by the AWGN (=10) plus SPIN (=20%).Similar conclusions to Figure 3 can be obtained from Figure 4.
Figure 5 shows the denoising results of the proposed algorithm for Lena image corrupted by different mixed noise.From Figures 5(a), 5(b), and 5(c), one can observe that the image quality is getting worse and worse with the increase of the noise density of salt-and-pepper noise and the standard deviation of Gaussian noise.From Figures 5(d), 5(e), and 5(f), it can be seen that the noise corrupted pixels are effectively restored and the edges are preserved well.Therefore, Figure 5 shows that the proposed algorithm can effectively remove noise and meanwhile preserve edges in the case of different mixed noise.Figure 6 shows the denoising results of the proposed algorithm for Boat image corrupted by different mixed noise.Similar conclusions to Figure 5 can be obtained from Figure 6.
In order to further verify the performance of the proposed algorithm, the proposed algorithm is compared with some existing main methods as Cai [27], AMF coupled with LRA [28], AMF coupled with LRR [29], WLRA [30], and WLRR [30].Tables 1 and 2 present the denoising results (PSNR) of different methods for six test images with =50% and =60%, respectively.Tables 3 and 4 present the denoising results (FSIM) of different methods for six test images with =50% and =60%, respectively.From Tables 1 and 3, one can see that the PSNR and FSIM values of the LRA and  and similar conclusions to Table 2 can be obtained from Table 4. Figures 7 and 8 show the bar plots of PSNR values of different methods for six test images with =50% and =60%, respectively.Figures 9 and 10 show the bar plots of FSIM values of different methods for six test images with =50% and =60%, respectively.From the above figures, one can observe that the proposed algorithm outperforms some existing main methods in terms of image denoising and edge preservation.
Tables 5 and 6 present the denoising results of different methods in terms of average PSNR and average FSIM for six test images, respectively.From Tables 5 and 6, one can observe that the Cai, WLRA, WLRR, and the proposed methods achieve the higher average PSNR and average FSIM values than the LRA and LRR coupled with AMF methods.This ensures that the Cai, WLRA, WLRR, and the proposed methods can perform better than the LRA and LRR coupled with AMF methods.The WLRR method can achieve the higher average PSNR and average FSIM values than the Cai and WLRA methods in most cases.It means that the WLRR methods are more effective in removing the mixed noise than the Cai and WLRA methods.The proposed algorithm obtains the highest average PSNR and average FSIM values in each case.Therefore, Tables 5 and 6 ensure that the proposed algorithm can not only effectively remove the mixed noise but also preserve edges and details well.Besides, Figures 11 and 12 show the bar plots of average PSNR and average FSIM values of different methods for six test images, respectively.From Figures 11 and 12, we can see that the proposed algorithm can perform better than other methods.
Figure 13 shows the denoising results of different methods for Lena image, which is corrupted by the AWGN (=10) plus SPIN (=50%).From Figures 13(d) and 13(e), it can be seen that the LRA and LRR coupled with AMF methods can remove the mixed noise, but they blur the edges.From Figures 13(c), 13(f), and 13(g), one can observe that the Cai, WLRR, and WLRA methods can remove the mixed noise and meanwhile preserve the edges.It means that the Cai, WLRR, and WLRA methods can perform better than the LRA and LRR coupled with AMF methods in image denoising and edge preservation.However, the Cai, WLRR, and WLRA methods cannot preserve the details well.From Figure 13(h), one can observe that the proposed algorithm obtains the more visually pleasant denoising result by reconstructing much cleaner and sharper image edges and details than other  compared methods.Hence, Figure 13 shows that the proposed algorithm outperforms other compared methods in terms of image denoising and edge preservation.Figure 14 shows the denoising results of different methods for Boat image, which is corrupted by the AWGN (=10) plus SPIN (=50%).From Figure 14, similar conclusions to Figure 13 can be obtained.
To further validate the above conclusions, some enlarged detail images of the denoising results of different methods are given.Figure 15 presents the comparisons of enlarged Lena images of different methods.The Lena image is corrupted by the AWGN (=10) plus SPIN (=50%).From Figures 15(d) and 15(e), one can see that the LRA and LRR coupled with AMF methods cannot effectively remove the mixed noise and blur the tassel.Figures 15(c), 15(f), and 15(g) show that the Cai, WLRR, and WLRA methods can remove the mixed noise, but they cannot preserve the tassel well.From Figure 15(h), it can be seen that the proposed algorithm can not only remove the mixed noise but also preserve the tassel well.Hence, Figure 15 shows that the proposed algorithm can perform better than other methods in terms of noise suppression and detail preservation.

Conclusions
In this paper, we propose an algorithm for restoring images corrupted by Gaussian noise plus salt-and-pepper noise.Firstly, a noise classification method is presented to identify different noisy pixels in the corrupted image.The noise classification method judges the gray level difference between the current pixel and its neighborhood pixels by introducing a variance parameter and identifies the salt-and-pepper noise corrupted pixels by combining the variance parameter and gray level extreme.Then, based on the result of noise classification, an adaptive directional weighted mean filter is proposed to remove the salt-and-pepper noise based on the multidirectional image information, which can preserve edges and details by adaptively selecting the optimal direction

Figure 7 :
Figure 7: The bar blots of PSNR values of different methods for six test images with =50%.(a) Lena image.(b) Boat image.(c) Peppers image.(d) Painting image.(e) Couple image.(f) Hill image.

Figure 9 :
Figure 9: The bar blots of FSIM values of different methods for six test images with =50%.(a) Lena image.(b) Boat image.(c) Peppers image.(d) Painting image.(e) Couple image.(f) Hill image.

Figure 10 :
Figure 10: The bar blots of FSIM values of different methods for six test images with =60%.(a) Lena image.(b) Boat image.(c) Peppers image.(d) Painting image.(e) Couple image.(f) Hill image.

Table 1 :
The denoising results (PSNR) of different methods for six test images with =50%.

Table 2 :
The denoising results (PSNR) of different methods for six test images with =60%.

Table 3 :
The denoising results (FSIM) of different methods for six test images with =50%.WLRA, WLRR, and the proposed method.This means that the denoising performances of the LRA and LRR coupled with AMF methods are significantly lower than the Cai, WLRA, WLRR, and the proposed method.The WLRR method achieves the higher PSNR and FSIM values than the Cai and WLRA methods in most cases.This ensures that the WLRR method can more effectively remove the mixed noise than the Cai and WLRA methods.The proposed algorithm obtains the highest PSNR and FSIM values among all compared methods in each case.With the increasing of the AWGN strength, the proposed algorithm can achieve higher PSNR and FSIM values than other methods.Hence, Tables1 and 3indicate that the proposed algorithm has stronger denoising capability than other methods.Similar conclusions to Table1can be obtained from Table3

Table 4 :
The denoising results (FSIM) of different methods for six test images with =60%.

Table 5 :
The denoising results (Average PSNR) of different methods for six test images.

Table 6 :
The denoising results (average FSIM) of different methods for six test images.