Multiscale Hybrid Non-local Means Filtering Using Modified Similarity Measure

A new multiscale implementation of non-local means filtering for image denoising is proposed. The proposed algorithm also introduces a modification of similarity measure for patch comparison. The standard Euclidean norm is replaced by weighted Euclidean norm for patch based comparison. Assuming the patch as an oriented surface, notion of normal vector patch is being associated with each patch. The inner product of these normal vector patches is then used in weighted Euclidean distance of photometric patches as the weight factor. The algorithm involves two steps: The first step is multiscale implementation of an accelerated non-local means filtering in the stationary wavelet domain to obtain a refined version of the noisy patches for later comparison. This step is inspired by a preselection phase of finding similar patches in various non-local means approaches. The next step is to apply the modified non-local means filtering to the noisy image using the reference patches obtained in the first step. These refined patches contain less noise, and consequently the computation of normal vectors and partial derivatives is more accurate. Experimental results indicate equivalent or better performance of proposed algorithm as compared to various state of the art algorithms.


I. INTRODUCTION
THE phenomenon of image degradation is quite natural due to the digitization and quantization of original image. Although image denoising techniques have been extensively studied and effectively employed, the preservation of texture, edges and fine details during denoising the image, is an open problem and needs rigorous treatment.
Prior to implementation of the non-local principle, a variety of variational, PDE-based, wavelet transform-based, wavelet based and local neighborhood filtering methods were proposed relying on the local features for image denoising. A major shift in this direction was initiated by the bilateral filtering [1] which exploits spatial and intensity domain for image densoising. In this approach spatially proximate pixels are given more weights in similarity measure. The wavelet-based BLS-GSM [2] method has provided the best results in terms of PSNR measure; however, these denoised images contain ringing artifacts and have low visual quality. More recently, Baudes et al. [3] introduced non-local means filtering for image densoising. Although the PSNR of non-local means filtering was found to be less than the wavelet-based BLS-GSM [2], the notion of patch-based approach combined with the idea of non-locality led to an entirely new way of attacking the problem. Earlier, approaches similar to nonlocal means were used for image inpainting [4] and texture synthesis [5]. Since the non-local means was proposed, extensive research for better estimation of parameters or finding suitable similarity measure has improved the performance of nonlocal means filtering for a variety of noise models in various image processing applications.
Kervrann et al. [6] provided a theoretical foundation for intuitive non-local means approach using Bayesian statistics. In this approach refined adaptive dictionaries of similar patches are obtained using Bayesian estimation, while irrelevant patches are rejected. Then, these learned dictionaries are used for patch-based comparison using a modified similarity measure. Elad and Aharon [7] proposed the learned dictionaries of patches using KSVD algorithm and then employed sparse representation over the trained dictionaries for image denoising. Tasdizen [8] proposed using the similarity of patches in the principal component analysis domain. Extensive research in patchbased denoising has resulted in state-of-the-art algorithms [9], [10]. Gilboa and Osher [11] provided an elegant interpretation of non-local means approach as a generalization of variational and PDE-based formulations. Brox et al. [12] proposed a computationally very efficient algorithm for non-local filtering that arranged the data in a cluster tree. This special arrangement further helps in preselection of similar patches. Also, an iterative version of non-local filtering based on the variational framework was suggested in [12]. In [13], a specific Euclidean space of patches was elegantly defined for implementing PDE based diffusion or smoothing processes.
In addition to the photometric property is used for patch-based comparison, patches contain much more information that requires the attention of researchers. With this motivation, we propose the notion of normal vectors patch corresponding to each photometric or pixel valued patch. By employing this definition in the second phase of our algorithm, we achieve remarkably better results than most of the state-of-the-art algorithms in the presence of high noise. Inspired by the special treatment of central patch in [14], we employ a slightly higher weight than in the standard non-local means approach; the value in our approach is associated with the central patch heuristically. This modification has further improved our results.
The rest of the paper is organized as follows. A brief review of non-local means filtering is provided in section II. The new multiscale two-stage algorithm and its implementation are explained in section III. Experimental results are described in section IV, and the conclusions are drawn in section V.

II. NON-LOCAL MEANS FILTERING TECHNIQUES
Consider a gray-scale intensity image over a rectangular bounded region Ω in IR 2 with additive white Gaussian noise. Instead of non-local, it is preferable to use the notion of semi-local filtering. It was indicated in [14] that considering whole image to search for similar patches has no major benefit, with the exception of periodic or quasi-periodic images. Also searching the whole image for each pixel is computationally too expensive. Therefore, in the rest of this paper, the term non-local means refers specifically to semi-local filtering. The NL-means filtering is defined by [3] is the normalization factor. The weights are obtained using Gaussian kernel with the weighted L 2 norm as follows [3]: Discussion on the choices of these parameters can be found in [15] and [14].
To increase the computational efficiency of non-local means filtering, Baudes et al. [15] proposed an accelerated version of non-local means algorithm that, instead of using the central pixel, replaces the whole central patch around the central pixel with a weighted average of patches around the other pixels in the search window. Mathematically this process is defined as [15]: where the similarity measure is the same as that defined in eq. (2). However, the cost for the improved computational efficiency of the accelerated non-local means is a slight decrease in the PSNR value.

III. THE PROPOSED ALGORITHM
In this paper, we adopt a hybrid approach to non-local means filtering. Our scheme is comprised of two stages as shown in Fig. 1. Firstly, we apply the accelerated version in the wavelet transform domain to get a pre-denoised image. Second, we employ a modified version of conventional nonlocal means filtering on the given noisy image. In this step, the weights are computed using pre-denoised image obtained in the first step.

A. Multiscale Accelerated Non-local Means
Inspired by various approaches [6], [7], [9], based on using pre-denoised images to obtain refined patches, we seek to obtain the pre-denoised images in our approach. However, in contrast to those techniques we adopt a very simple approach. We, first decompose the image U using two dimensional discrete stationary wavelet transform (DSWT) up to the coarsest level J.
Next, the accelerated version of non-local means filtering [15] is performed on detail bands for each scale level j=1, 2, …, J. Finally, the reference pre-denoised image is obtained using the inverse stationary wavelet transform. Thanks to an overcomplete, rotationally invariant and sparse representation of a noisy image in the DSWT domain, the image can be efficiently denoised using accelerated nonlocal means filtering without much degradation of texture, edges, and fine details.

B. Normal Vector Patches and Weight Factor
Assuming the patches as surfaces, we associate normal vectors with the patches, defined as normal vector where * ⋅ denotes the point-wise multiplication of patches and Γ is the patch or matrix containing all the elementwise inner products. The weight factor is then defined as The intuitive motivation for introducing this factor is to acquire the degree of similarity of the original photometric patches, based on the similarity of mean orientation of respective normal vector patches, as shown in Fig. 2. We use this factor in the second step of our algorithm because the computation of derivatives is sensitive to noise level. In the pre-denoised image, the noise level is much lower than in the respective noisy image.

C. Modified Non-local Means Filtering
After getting pre-denoised image as defined in III-A, we perform the modified version of standard non-local means filtering on given noisy image, based on the similarity of reference patches from the pre-denoised image. The modified similarity measure is defined as It can be noticed that the convolution of the Gaussian kernel G α in eq. (2) is replaced by the weight factor ( ) i,j η obtained with eq. (6). The convolution with Gaussian kernel G α was to take into account the spatial proximity of the pixels around central pixel. However, the proposed weight factor is based upon the similarity of the mean normal orientation of central patch with other patches in the search window. Our second modification to standard non-local means filtering is made in the assignment of weight to the central patch when compared with itself. The standard non-local means assigns the maximum of all the computed weights with other patches in the search window [15].
Mathematically, this can be expressed as This choice of weight for self-comparison is intuitive and arbitrary. Salmon [14] discussed the different choices for assigning self-similarity weight. However, we set this weight based on a heuristic approach. For low or medium noise levels in the image, we assign a slightly higher self similarity weight than used in conventional non-local means filtering; our self-similarity weight is defined by The intuitive justification for assigning higher self-similarity weight is based on to the assumption that presence of low or medium levels of noise affects the self similarity slightly. However, for severe noise ( ) 20 > σ , this assumption is very weak and we use the self similarity weight as defined in eq. (8).

D. Summary of the Proposed Algorithm
The proposed denoising algorithm can be summarized as follows: 1) Construct a scale-space in wavelet domain by decomposing the noisy image using discrete stationary wavelet transform up to the coarsest scale J. 2) Perform the accelerated non-local means filtering [15], as described in III-A to obtain the denoised wavelet transform components. 3) Reconstruct the pre-denoised image using inverse DSWT. This image will be used as a reference in the next step for patch based comparison. 4) Denoise the given noisy image using the modified non-local means filtering proposed in III-C. The patch comparison is performed on the refined patches obtained in the previous step. A schematic diagram of the proposed algorithm is shown in Fig. 1.

IV. EXPERIMENTAL RESULTS
To evaluate the results of proposed method, benchmark gray-scale images of Lena, Barbara, Boats, Peppers, and House are considered. We use 9 × 9 patches, 15 × 15 search windows, and σ h 6 = as the filtering parameter in the first phase. In the second phase, we selected 7 × 7 patches with filtering parameter σ h 87 . 3 = . We can choose the same patch size for both the phases; however, choosing the different patch sizes provides a slightly better result. Also, the filtering parameter is reduced to σ 87 . 3 from σ 6 to avoid the over-smoothing phenomenon. The proposed algorithm is compared with various state-of-the-art algorithms: those by Dabov et al. [9] (BM3D), Elad and Aharon [7] (KSVD), Portilla et al. [2] (BLS-GSM), and Kervrann et al. [6] (BNLM). The PSNR value is used as a metric for comparing the denoising capability of each scheme.  The results of BM3D are always the best regardless of the noise level. For the rest of the algorithms, K-SVD performance is better when the noise level is low. However, our proposed algorithm yields better results than the others in the presence of moderate and severe noise. In particular, for Barbara image with high texture, our algorithm achieves much better results in terms of PSNR than the others. Our results can also be confirmed based on the average of the performances at various noise levels. Table II indicates that, on average, our algorithm outperforms the other three algorithms in the presence of moderate or severe noise. Fig. 3 shows the visual results for Lena and Barbara images at magnified scale. It can be seen that the texture in both the images is very much preserved, while piece-wise constant regions are not very smoothed out.

V. CONCLUSION
A new definition of normal vector patch is proposed to acquire more information about the similarity of photometric patches. The experimental results demonstrate the effectiveness of the proposed algorithm when combined with notion of normal vector patch. Patches continue to offer an undiscovered world of information. Therefore, in future, we plan to further explore their associations with other characteristics, such as coherence and curvature.