An Improved Image Denoising Model Based on Nonlocal Means Filter

The nonlocal means filter plays an important role in image denoising. We propose in this paper an image denoising model which is a suitable improvement of the nonlocal means filter. We compare this model with the nonlocal means filter, both theoretically and experimentally. Experiment results show that this new model provides good results for image denoising. Particularly, it is better than the nonlocal means filter when we consider the denoising for natural images with high textures.

Among all, nonlocal self-similarity models received recently much attention and play an important role in image processing.K. Dabov et al. [15] proposed the sparse 3D transform-domain collaborative filtering (BM3D) method, in which one exploits the inherent nonlocal self-similarity property of natural images, groups similar 2D image fragments into 3D groups, and then deals with these 3D groups by the developed collaborative filtering procedure.The BM3D and its variants proposed in [15,[21][22][23][24][25][26] have been demonstrated to achieve state-of-the-art performance.In [21,22], the BM3D is applied for coherent noise reduction in digital holography.In [23], an iterative algorithm combining BM3D technique was proposed for the salt and paper (S&P) noise removal.In [24], V. Katkovnik et al. introduced the CD-BM3D and iterative CD-BM3D algorithms based on BM3D for interferometric phase image estimation.The nonlocal means filter (NLM) proposed by Buades et al. [11], known for its simplicity and excellent ability to efficiently remove additive white Gaussian noise (AWGN) while preserving edges, has triggered an important and continuous development in image processing.Not only lots of the NLM's modified versions, the mixed schemes integrating NLM idea, and NLM-based iterative approaches are applied to remove AWGN, but also other kinds of noise, such as signal-dependent nonadditive speckle noise [16], additive mixed noise [14], and S&P noise [17].Existing NLM-based iterative approaches have also been shown to provide better results than their predecessors [17][18][19].In [18], MRI deniosing was performed by a union method (the Wiener enhanced NLM), in which the noisy MR image was deniosed successively by the traditional median filter, the Wiener filter, and the NLM filter.The experiments demonstrated that the union method obtained better results than the NLM for denoising MRI corrupted by AWGN [18].H.Y. Liu et al. [19] proposed a two-steps iterative regularized denoising approach based on the NLM and the total variation.First the prefiltered image was obtained by using the NLM with the improved similarity weights.Then, they presented a modified TV model based on the prefiltered image to achieve final denoised results.X.T. Wang et al. [17] proposed an iterative nonlocal means filter for S&P noise removal (S&P-INLM).S&P-INLM includes three stage operations.First, a spatial hard threshold method is used to mark the positions of noisy pixels and noise-free pixels of an image corrupted by S&P noise, and then the noisy pixels in the image are prefiltered by a median filter while noise-free pixels are left unchanged.The prefiltering is used to make the noise distribution in the prefiltered image close to the Gaussian distribution.Finally, the noisy pixels in the prefiltered image are filtered by the proposed NLM-based iterative algorithm [17], while noisefree pixels are not filtered.The scheme proposed in [17] is based on the detection results of noisy pixels' position and has achieved good results in removing S&P noise.However it is not suitable for the white Gaussian noise removal since the positions of the noisy pixels corrupted by white Gaussian noise are hard to detect.
As mentioned above, in image processing, nonlocal selfsimilarity based methods play an important role in image denoising.Recently, the BM3D and its modifications have been shown to outperform the NLM in many cases.Nevertheless, the NLM is still popular due to its advantages that the algorithm is simple and has good denoising performance while preserving edges.In this paper, we want to present an improved NLM-based approach for removing AWGN from noisy natural images, which is different from previous NLM-based iterative schemes and NLM modifications.Our proposed model has better denoising performance than the NLM for natural images with high textures, while its computation is almost as simple as the NLM.
Motivated by the Yaroslavsky filter [27], Buades, Coll and Morel proposed the original NLM [11].This model is given by where  ∈ Ω,  ∈ Ω, and   is defined by Here   is a Gaussian kernel with standard deviation , and () in ( 1) is defined by We define as a weight that measures the similarity of  between a neighborhood of  and a neighborhood of .The parameter ℎ in (4) controls the decay of the weight.In order to ensure that only pixels with a similar neighborhood have a large weight, ℎ usually corresponds to the standard deviation of the image noise.Formula (1) is equivalent to where  is defined by is the denoised image for .In many applications one gets a very good denoised image , though this model is very simple.This indicates that the weight  captures the feature of images very well.There are other choices of the weight .
Here, we just use this weight.Numerically the nonlocal means filter could be written as if we denote a digital image by a function  : Variational methods also work quite well for image denoising.In variational methods one considers the minimizer of a suitable functional, which usually satisfies a partial differential equation.Then we use the minimizer as the denoised image.This is one way to study the inverse problem mentioned at the beginning of this paper.The most popular functional used in image processing are the Dirichlet functional and the TV functional According to these functionals there are  1 models and TV models for image denoising.Particularly the TV model proposed by Rudin, Osher, and Fatemi [1] is very useful in image processing.Motivated by the work of Kindermann, Osher, and Jones [28], Gilboa and Osher [20,29] wanted to combine the TV model and the nonlocal means filter into a nonlocal TV model, so that the corresponding  1 model is just a regularization of the nonlocal means filter.They realized this idea successfully, by introducing suitable nonlocal operators, which was motivated by the work of Zhou-schölkopf [30,31].Having nonlocal operators, it is natural to propose a nonlocal version of the ROF model, the NLTV model [20], and other some nonlocal models.The nonlocal models they proposed are very useful in image processing.Their models have been used and extended in various problems; see, for example, [32][33][34][35][36][37][38][39][40].
Motivated by the work of Gilboa-Osher and the work in geometric analysis (see, for example, [41]), Jin-Jost-Wang proposed new nonlocal operators [42].These new nonlocal operators match the geometric structure of images well by using the weight .
Here  is defined by (5) and (()) denotes the twice use of the nonlocal means filter .
The computation of INLM is almost as simple as the NLM.Our experiments show that the INLM model is much better than (()) and also better than the nonlocal means filter (NLM) for many images with higher textures.

A New Nonlocal 𝐻 1 Model and Its Discrete Version
In [42,43] 4), which measures the similarity of images.Let  : Ω → R be defined by (6).Motivated by differential geometry, in [42] the authors used  and  to define the L 2norms for functions  : Ω → R and vector field  : Ω × Ω → R. First one defines a scalar product for functions with respect to  by Hence the  2  -norm for functions is defined by For vector fields one defines a scalar product with respect to (, ) by for any pair of vector fields ,  : Ω × Ω → R. The difference vector field of a function (an image)  : Ω → R is defined by which is different from the nonlocal gradient vector field given in [20,29,30].Here one views  as a metric.A map  : Ω × Ω → R is viewed as a vector field.Using  and  one can define the divergence operator div  : Then one can check that for any  : Ω → R and  : From this variational setting it is natural to define a (nonlocal) Dirichlet functional and a (nonlocal) Laplacian of functions which can also be defined directly from the difference operator  and the divergence operator div.This functional ( 18) is a regularization of the nonlocal means filter, which is the same as the functional obtained by Gilboa and Osher.However, the viewpoint of Jin-Jost-Wang is different from that of Gilboa and Osher.This difference leads Jin-Jost-Wang to introduce a different nonlocal TV model and  1 model in [43].Here we only consider the  1 model [43], The difference to the nonlocal setting of Gilboa or equivalently which is a discrete version of .For a digital image  we define its weight  2 -norm by This is the discrete version of (13).For a pair of , V : Ω → R we define a scalar product by Then the difference vector field defined above is given by which is a discrete version of (15).As above a map from Ω × Ω → R is considered as a vector field.For a pair of vector fields ,  : Ω × Ω → R we define a scalar product From the scalar product we have a weight  2 -norm of , which is given by Now for a vector field  : Ω × Ω → R, its divergence div  : Ω → R is defined by One can then check as above: for any  : Ω → R and  : This means that div is the adjoint operator of .Now with the gradient operator and the divergence operator we can define the (normalized) Laplacian of function  as above by ( 19) or equivalently, where  : A (discrete) function  satisfying Δ = 0, or equivalent  −  = 0, is called a harmonic function.It is easy to check that a harmonic function is a critical point of the following nonlocal Dirichlet energy: This functional is the discrete version of functional (18) One can easily check that the minimizer (critical) point of  satisfies which is the discrete version of (22).To find a minimizer , the following heat equation is considered in [43] which is the (negative)  2 gradient flow of  in the space of functions with the  2 -norm with respect to .The discrete version of ( 37) is where    =   (Δ).The initial condition is If 1 ≥ Δ(1 + ), then the coefficients in (38) are nonnegative.Therefore we have the following.
Proposition 1.This algorithm is stable, provided that

An Improved Nonlocal Means Filter Model.
In [43] Jin-Jost-Wang presented experiments to show that their new nonlocal  1 model is very good in image denoising.It was observed in [43] that with a few iterations one can already get good results.In this paper we choose  = 0 and Δ = 1/2 and iterate (38) twice.For the first iteration of (38) we have Then iterating formula (38) again we obtain This is the model we want to propose in this paper.Let us rewrite it in the following form: Recall that (()) is the twice use of nonlocal means filter (NLM2).Our experiments show that our model ( 43) is much better than (()) and also better than the nonlocal means filter for many images as shown in our experiments.
For experiments, the first task is how to compute the weight .We compute it by following the method given in [11].For a given image  : Ω → R one computes the weight by using the difference of patches around each point (pixel).The patch   () of size  ×  around  ∈ Ω is given by where  is an odd integer.Let   be the Gaussian weight Euclidean distance, i.e., where Now the weight we will use is computed by In the computation of the weight  we use  = 5, i.e., patch of size 5 × 5.The search window as mentioned above is of size 11 × 11.
Give a noisy image  =  0 + V 0 , where  0 is the ordinary true image and V 0 is a random noise (white Gaussian noise) with mean zero and standard deviation .In order to compare different models, we use the peak signal-to-noise ratio (PSNR) to measure images and compare noisy images and denoised images.The peak signal-to-noise ratio (PSNR) is defined by where  u is the maximum possible pixel value of the image .MSE is the mean squared error defined by Here  denotes the denoised image from the noisy image .

On Test
Images.The images we use are taken from USC-SIPI image database with the same name and three standard test images (Cameraman, Barbara, and Lena).The three standard test images and USC-SIPI images database are widely used in the image processing literature, for example, [2,11,20,36,40].All images in this experiment are of size 256 × 256.
The parameter ℎ in (47) usually corresponds to the standard derivation of the noise.If ℎ is small, noise remains in the denoised image, while larger ℎ will oversmooth the image.So if NLM is applied twice, ℎ should be large for the first time and much smaller for the second.In our experiments, for the first use of NLM we searched the best ℎ (i.e., the best ), denoted by ℎ 1 , so that PSNR of denoised images () has the largest value.Then we applied NLM method again to denoised images () to get denoised images (()) and used () to calculate the weight  (47) and the weighted average value (7), here using () is better than using .For the second use of NLM we also searched the best ℎ, denoted by ℎ 2 , to make PSNR of (()) be the highest.The experiment results are presented in Table 1; the corresponding denoised images are showed in Figures 1 and  2.
The experiment shows that PSNR of denoised image by INLM is much better than NLM2.It is also better than NLM for most images.INLM improves the NLM, while NLM2 gets worse results than the NLM.
In Figures 3, 4, and 5 we present local zooming images taken from the Barbara image and the 1.1.13image.From these three images one can see that the denoised images obtained     We also compared our INLM with the Wiener enhanced NLM proposed in [18].The experiments in [18] demonstrated that the Wiener enhanced NLM [18] has much better performance than the NLM for removing AWGN from MR images.We wonder how it performs for removing AWGN from noisy natural images.Based on this consideration, we applied the Wiener enhanced NLM [18] to the natural images as shown in Table 1: Cameraman, Barbara, Lena, Images 1.1.10,1.1.11,1.1.12,1.1.13,1.3.01,1.3.02,1.4.02, and 1.5.03,corrupted by AWGN with the standard derivation  = 10, respectively.The experiment results showed that the PSNRs of the denoised images by the Wiener enhanced NLM [18] are lower than NLM, NLM2, and our model INLM.It means that, for denoising natural images corrupted by AWGN, the Wiener enhanced NLM [18] does not perform as well as when denoising MRI corrupted by AWGN.

On Real Images.
In order to further illustrate the performance and the robustness of our model, we also apply our INLM model to real natural images and compare the results with those obtained by NLM and NLM2.Three real images used in the experiments are of size 512×512 and shown in Figures 6, 7, and 8.We use the blind metric BRISQUE proposed by A. Mittal et al. [44] to assess denoised images.BRISQUE is one of the state-of-the-art blind no-reference image quality metrics when noise level in the image is unknown.The smaller the BRISQUE value is, the better the performance of image restoration is [44].We calculate BRISQUE values by directly using the software of BRISQUE which is available online at http://live.ece.utexas.edu/research/quality/.In this experiment, let the decay parameter ℎ 1 = 7.0 and ℎ 2 = 3.0.The experiment results in Table 2 show that BRISQUE values of denoised image by INLM are the smallest, which means that INLM is better than other two models.The corresponding denoised images are showed in Figures 6, 7, and 8.

Conclusions
In this paper we first present a discrete version of Jin-Jost-Wang's nonlocal  1 model which was proposed in [43].Then we propose an improved nonlocal means filter (INLM) (43).This INLM is as simple as the NLM and is simpler than Jin-Jost-Wang's nonlocal  1 model proposed in [43].Our experiments show that this INLM restores the original image   from noisy image very well while preserving the image edges and details.Particularly this model is much better than the method by using the NLM model twice.It is also better than the NLM model when we consider the denoising for images with high textures.

Figure 1 :
Figure 1: Denoising of Cameraman, Barbara, and Lena images using three different methods, respectively.From top to bottom: the original images  0 , noisy images  with a white Gaussian noise ( = 10), and denoised images  by NLM, NLM2, and INLM.

Figure 2 :
Figure 2: Denoising of the images using three different methods.The images from top to bottom are 1.1.10,1.1.11,1.1.121.1.13,1.3.01,and 1.5.03,respectively.The first column: the original images  0 , the second column: noisy images  with a white Gaussian noise ( = 10), respectively, the third column: denoised images by NLM, the forth column: denoised images by NLM2, and the fifth column: denoised images by INLM.

Figure 3 :
Figure 3: A local zoomed-in zone of the Barbara image.Top left: the original image, top right: the noisy image with a white Gaussian noise ( = 10), bottom left: denoised image by NLM, bottom middle: denoised image by NLM2, and bottom right: denoised image by INLM.

Figure 4 :
Figure 4: A local zoomed-in zone of the Barbara image.Top left: the original image, top right: noisy image with a white Gaussian noise ( = 10), bottom left: denoised image by NLM, bottom middle: denoised image by NLM2, and bottom right: denoised image by INLM.

Figure 5 :
Figure 5: A local zoomed-in zone of the image 1.1.13.Top left: the original image, top right: the noisy image with a white Gaussian noise ( = 10), bottom left: denoised image by NLM, bottom middle: denoised image by NLM2, and bottom right: denoised image by INLM.

Figure 6 :
Figure 6: Denoising of the image 1 using three different methods.Top left: the real image, top right: denoised image by NLM, bottom left: denoised image by NLM2, and bottom right: denoised image by INLM.

Figure 7 :
Figure 7: Denoising of the image 2 using three different methods.Top left: the real image, top right: denoised image by NLM, bottom left: denoised image by NLM2, and bottom right: denoised image by INLM.

Figure 8 :
Figure 8: Denoising of the image 3 using three different methods.Top left: the real image, top right: denoised image by NLM, bottom left: denoised image by NLM2, and bottom right: denoised image by INLM.
With these new nonlocal operators, they proposed a new nonlocal  1 model [43].The new nonlocal  1 model provides good denoising results.By experiments we found that with only a few iterations in this nonlocal  1 model one can already obtain very good results for image denoising.This motivates us to propose in this paper an improved nonlocal means filter model (INLM) Jin-Jost-Wang introduced a new nonlocal variational setting, under which they obtained the same nonlocal Dirichlet functional, but a different nonlocal TV model and a different  1 model.In this section we first briefly recall this new nonlocal variational setting and then present its discrete version, especially the discrete version of the new nonlocal  1 model.Motivated by this model we propose at the end of this section our model as an improved nonlocal means filter.2.1.A New Nonlocal  1 Model.Let  : Ω → R be a noisy image that we want to consider and the weight  : Ω × Ω → R, computed by (

)
2.2.Its Discrete Version.In this subsection, we present the discrete version of the new nonlocal  1 model, which has its own interest.Now we represent a digital image by a (discrete) function  : Ω → R, where Ω is a set {(, ) | 1 ≤  ≤ , 1 ≤  ≤ }.Hence  is a digital image with pixels  × .For simplicity, we denote () by  i for  ∈ Ω.The weight  : Ω × Ω → R is denoted by  ij , an  ×  symmetric matrix with nonnegative entries.Then  : Ω → R is given by

Table 1 :
PSNR values of noisy images and denoised images.