𝑙 0 Sparsity for Image Denoising with Local and Global Priors

. We propose a 𝑙 0 sparsity based approach to remove additive white Gaussian noise from a given image. To achieve this goal, we combine the local prior and global prior together to recover the noise-free values of pixels. The local prior depends on the neighborhood relationships of a search window to help maintain edges and smoothness. The global prior is generated from a hierarchical 𝑙 0 sparse representation to help eliminate the redundant information and preserve the global consistency. In addition, to make the correlations between pixels more meaningful, we adopt Principle Component Analysis to measure the similarities, which can be both propitious to reduce the computational complexity and improve the accuracies. Experiments on the benchmark image set show that the proposed approach can achieve superior performance to the state-of-the-art approaches both in accuracy and perception in removing the zero-mean additive white Gaussian noise.


Introduction
Image denoising, aiming at generating clean images by removing the noises, plays an important role in various tasks of image processing and computer vision, such as feature extraction, object detection, and pattern recognition [1][2][3].Since noise-free image can significantly improve the performance of these tasks, image denoising has drawn a growing number of attentions in recent years.Under dissimilar circumstances, images can be corrupted by different types of noises, for example, Gaussian noise, salt-and-pepper noise, and quantization noise, in the process of acquisition and transmission.Among these noises, additive white Gaussian noise, which is usually caused by high temperature or lacking of illumination, is the most conventional and has been widely studied in the past decades.In this paper, we mainly focused on removing this classic type of noise.
Given an image  corrupted by additive white Gaussian noise , we can formulate the relationships among , , and noise-free image  as follows: where the mean and standard deviation of  are 0 and , respectively.To recover  from  in (1), a number of approaches [4,5] have been proposed and achieve satisfying results.
Recently, Nonlocal Means (NLM) filtering has been rapidly developed and widely used in image processing tasks.NLM, originally introduced by Buades et al. in [6], tries to utilize all the other pixels in the image to recover the noise-free value of target pixel.NLM has achieved a superior performance [7][8][9] over previous approaches while preserving the integrity of relevant image information and has been theoretically proven to be effective in dealing with zero-mean Gaussian white noise.In order to globally generate effective correlations among pixels in the target image, the traditional NLM algorithm needs to calculate every pairwise similarity in a large search window.It causes high computing complexity and limits the application of NLM in practice.Therefore, there is an urgent requirement to develop an algorithm to reduce the search range and efficiently construct global correlations among pixels.
Recent studies about the Compressed Sensing (CS) have shown that the information of a given image is always 2 Advances in Multimedia redundant, and a specific pixel can be sparsely represented by other pixels in the image.This inspires us to reduce the search range of NLM by finding the sparse representation of the target pixel.Sparse representation has been used in many computer vision and image processing tasks, such as face recognition, motion segmentation, and image restoration.Specially,  0 sparsity based approaches have been well studied, because of its superior performance on various tasks.Xu et al. [10] have utilized  0 sparsity to solve the image smoothing and image deblurring problem.Wang et al. [11] adopt  0 sparsity in image segmentation.Nie et al. [12] propose a hierarchical  0 sparsity based approach to tackle the intrinsic image decomposition problem and achieve superior performance to the previous approaches.These successful applications of  0 sparsity inspire us to explore the potentiality of it to resolve image denoising problem.
In this paper, we propose a  0 sparsity based approach to remove the additive white Gaussian noise from a given image.To recover the noise-free value of a pixel, we combine the local and global information, together, to achieve more accurate result.The local correlation is adopted by exploring the relationships of pixels in a small search window (3 × 3 pixels in our experiments) to preserve edges and ensure local smoothness.The global correlation is generated by finding the sparse representation based on  0 sparsity to reduce the search range.Moreover, we also adopt Principal Component Analysis (PCA) to measure the similarities between features of pixels, which can both reduce the amount of calculation and improve the accuracies.Experiments on benchmark images corrupted by different level of zero-mean Gaussian white noise show that the proposed approach gains superior performance to the state-of-the-art image denoising approaches.
The rest of this paper is organised as follows: Section 2 will briefly review the NLM algorithm and  0 sparsity.Section 3 will specify the proposed  0 based nonlocal denoising algorithm in detail.In Section 4, we show the experimental results that demonstrate the effectiveness of our algorithm.Section 5 will summarise and conclude this paper.

Related Work
In this section, we will briefly review the previous studies about NLM and  0 sparsity, the two most correlated algorithms to the proposed image denoising approach.
Nonlocal Means.NLM is a popular technique developed by Buades et al. [13], which achieves unprecedented performance in image denoising task.However, the superior performance is achieved at the cost of high computational complexity.For a target pixel , the standard NLM computes the relevances based on image patches between  and all the other pixels in the image.Then, the linear combination result of similar and homologous pixel values is assigned as the noise-free value of .The high computational complexity makes standard NLM less practical in image denoising task.Salmon [14] conducted experiments to study two factors in NLM, search window size and weight of the central patch, and concluded that the performance increased when the window is larger and the way of the central weight was a crucial issue.Some approaches have been proposed to accelerate the NLM or find a more appropriate definition of weight.References [15,16] terminated the distortion computation between patches based on probabilistic early termination.But the reduction in the amount of similarity measurement resulted in the loss of image information.Darbon et al. [17] separated the pixels in neighbourhood to offer a parallel implementation in contemporary memory sharing of computer architectures, which had special requirement for hardware.In terms of weight, Zhong et al. [18] estimated the similarities between noise-free patches instead of noisy observations.The denoising scheme did secondary filtering based on the original NLM, which resulted in more calculation amount.References [19,20] built a dictionary to project the image content to the subspace and then defined a metric between a pixel and the neighbouring ones.According to region characteristics, region-based NLM [21,22] adaptively changed the similarity patch size to preserve edges.These methods did provide a more accurate metric of distance and gained a better performance than the standard NLM.However, because the search area is a region centered on the target pixel, the previous NLM based approaches are seminonlocal in essence. 0 Sparsity. 0 sparsity based approaches have been widely used in many vision and graphic tasks and acquired superior performance.Hyder and Mahata [23] proposed an iteratively approximate  0 -norm based on fixed point to reconstruct sparse signal and achieved an obvious improvement in noisy environment.In [10,24], Xu et al. developed  0 gradient minimization to control the nonzero gradients among neighborhoods, which indicated that the prominent image structures needed to be preserved in image smoothing.Later, they extended  0 sparsity to image deblurring and achieved notable improvement both in convergence speed and result quality.Mancera and Portilla [25] selected coefficients by minimizing the nonzeros with  0 -norm in inpainting problem and gained a remarkable performance compared to practical  1 -norm based methods.A Gaussian function was provided by Mohimani et al. [26] to obtain a continuous smooth estimation of  0 -norm and found the sparsest solution for linear equations of an underdetermined system.The process speed was improved by two orders of magnitude faster than  1 while retaining the same accuracy.Lopez et al. [27] adapted  0 -norm to make Least-Squares Support Vector Machines (LS-SVM) sparsity in classification and regression problems.As a result, the amount of support vectors was reduced significantly than standard LS-SVM while keeping a comparable accuracy.However, this procedure was computationally expensive.Wang et al. [11,28] built a constructed graph based on  0 representation of features, which gave a better description of superpixels, to perform image segmentation.Experimental results showed that their approach achieves competitive results compared with state-of-the-art methods.Nie et al. [12] constructed a sparse representation for pixels by solving the  0 minimization problem, then formulated a sparse prior to preserve the global consistency during the decomposition of intrinsic image, and achieved superior performance.The success of  0 sparsity in various tasks promotes us to utilize it to deal with image denoising problem.

Our Method
3.1.The Proposed Formulation.Let , , and  represent noise-free image, additive Gaussian noise, and observed image, respectively.According to (1), the value of pixel  in  can be defined as the addition of th pixel values in  and ; that is, () = () + ().We formulate the proposed denoising model as follows: where Û() is the restored value of pixel ,   () and   () represent the local prior and global prior at pixel , and () is a normalization factor.
Local prior   explores the correlations among pixels in a local window.It can preserve edges and encourage smoothness.We define the local prior at pixel  as where N  () represents the pixel set in a local search window centred at  and   (, ) represents the local similarity between pixel  and pixel .In our experiments, the size of local search window is set as 3 × 3. We will illustrate the detailed definition of   in the next section.Global prior   builds the relationships between pixels in the whole image.It can promote the global consistency and improve the denoising results.The definition of   is given as follows: where N  () represents the set of pixels globally correlated with  and   (, ) represents the nonlocal similarity.The generation of N  () and calculation of   (, ) will be specified in Section 3.3.The normalization factor () is defined as (, ) . (5)

Local Prior Acquisition.
According to (3), the local prior is defined as the summation of weighted pixel values in a local search window.In this section, we will specify the definition of similarity   (, ).
For a given pixel , we represent its feature by concatenating the gray intensity of nearby pixels within a local square patch centred at pixel , and the size  of the patch is set as 5 in our experiments.We use F = {f  } to denote the feature set of all pixels in image , where f  is the feature vector of pixel  and its dimension is  2 .For pixel  ∈ N  (), the similarity between pixels  and  is defined as follows: where ‖ ⋅ ‖ 2 represents the Euclidean distance and ℎ is the parameter which controls the decay of weights related to the level of noise.For a specific image, ℎ is a constant.The similarity definition given in ( 6) can help maintain the simplicity of algorithm; however, the computation process is time-consuming.In addition, the original feature vector always includes redundant and noisy information.To reduce the feature dimension and refine the feature representation, we project the original feature vector onto a subspace generated by Principle Component Analysis (PCA).
To achieve this goal, we first randomly select  pixels in image .We use F sub = {f  }   to denote the set of feature vectors of selected pixels.Let  be the number of principle components and let {  }  =1 be the sorted eigenvectors in descending order according to their respective eigenvalues, which are generated from the set F sub .The number  is determined by where  is a threshold which means the utilization rate of information, usually set as 0.95 in our experiments.Then, we project the original feature vector f  on the -dimensional PCA subspace as where ⟨⋅⟩ represents the distance of th original feature vector's projection on the th basis vector.Since the basis vectors {  }  =1 are orthonormal, we can calculate the distance between pixel  and pixel  on the PCA subspace as We use Dist(, ) to replace the traditional Euclidean distance.Finally, the local similarity between pixels  and  is defined as ) . (10)

Global Prior Acquisition.
Based on the mechanism of sparse coding representation, a specific signal  can be sparsely represented by the dictionary  with the coefficient vector , that is,  = , subject to || ≤ , and  is a positive integer.Accordingly, for a given pixel , we still use f  to represent its feature vector.To generate the global correlations of pixel , we solve the following minimization problem based on  0 sparsity: where   represents the sparse dictionary of pixel  and   is the corresponding coefficient vector.The construction of dictionary   for each pixel can significantly affect the accuracy and speed of the solving process.Inspired by the hierarchical  0 sparsity proposed in [12], which has shown superior performance both in effectiveness and efficiency, we also adopt a hierarchical framework in this paper.
For image , given the downsampling ratio  and layer number , we first build an image pyramid P  = {  }  =1 .  represents the image in th layer, and   corresponds to the original image .For the top layer  1 , we build the sparse dictionary  1  for pixel  by features of all the other pixels in  1 , that is, where f 1  represents the feature of th pixel in  1 and  1 is the number of pixels in  1 .By solving the coefficient vector  1  , we can find pixels which are globally correlated with pixel  through the nonzero coefficients in  1  .For layer   (1 <  ≤ ), we build sparse dictionary    for th pixel by the derivative relationships from  −1 .Specifically, we first find pixel 's corresponding pixel   in  −1 by downsampling.By  −1   , we can find the pixel set X −1   correlated with   .For each pixel  ∈ X −1   , we can get the corresponding image patch Y   in   by upsampling.Then, we use feature vectors of pixels in ⋃ ∈X −1   Y   to construct the sparse dictionary    for th pixel in layer , where ⋃ represents the union of multiple sets.
Finally, we can find the set N  () of pixels which are globally correlated to pixel  in  by nonzero coefficients in    .For pixel  ∈ N  (), the global similarity between pixels  and  is defined by the Normalized-Residual as where    () = 0, if  = , and    () =   (), otherwise.

Experimental Result
We test our approach on twelve benchmark images:  ( Û). MSE stands for the noise variance of despeckled image.SSIM is used to describe the noise removal quality and it is much closer to human vision, given by where   and   are the mean and variance of noise-free image , respectively, and the similar definition of  Û and  Û to denoising image Û.  Û is the covariance of Û and . 1 and  2 are constants, which are used to avoid instability when  2 Û +  2  and  2 Û +  2  are very close to zero.Figures 1 and 2 show the image denoising results on different levels of noise by the proposed approach.From Figures 1 and 2, we can observe that the denoising images on different noise levels achieve good visual sense.Our approach can maintain detailed structure well and has remarkable noise reduction at the flat region.Moreover, our approach can also preserve the global consistency.
We also compare the proposed approach with five NLM based image denoising approaches: NLM [6], NL-TV [7], NLMPCA [20], NLSAP [22], and R-NL [29].The proposed approach is denoted as  0 -NLM.The comparison results are shown in Table 1.From Table 1, we can observe that our approach can always achieve the highest PSNR and SSIM score in different noise levels.And our approach achieves average 0.7 PSRN improvement and 0.05 SSIM improvement to original NLM algorithm, which specifies the efficiency of proposed approach.Moreover, our approach offers superior performance on 7 test images released by state-of-the-art approach R-NL, average 0.16 PSNR improvement and 0.02 SSIM improvement, especially on the image Man and Barbara.
To further demonstrate the advantage of our approach, Figures 3 and 4 show the comparison of detailed structures of different approaches.Figure 3 3(g), we can observe that the boundary between white part and black part in the eye of image Lena is obvious; besides, the intensities of both parts in two eyes are consistent.That specifies that our model is more effective in finding the global correlations among pixels.But the results generated from other approaches have some small drawbacks; for example, the boundary is fuzzy, or the similar parts in two eyes are inconsistent.In addition, the forehead is much smoother on  0 -NLM than other models.This can also demonstrate that our approach can construct appropriately nonlocal correlations and encourage global consistency.Figure 4 4(g), we can see that the bracelet in the hand of man is much clear by  0 -NLM.The result also illustrates the superior performance of our approach.

Conclusion
This paper presents an image denoising approach based on  0 sparsity.We combine local prior and global prior together to effectively generate image denoising results.The local prior can preserve edges and encourage local smoothness.To effectively and efficiently calculate the local similarity, we adopt Principle Component Analysis to reduce the feature dimension and refine the feature representation, while the global prior encourages global consistency.The major contribution of this paper is that we utilize hierarchical  0 sparsity to construct global correlations, which is more effective and efficient to reduce the redundant information in the whole image.This differs from traditional nonlocal approaches which trivially search similar pixels in a search window.Experiments on the benchmark images show the

Figure 1 :
Figure 1: Noise-free images and denoising results with  = 20.(a), (b), (c), and (d) are Lena, Boat, Barbara, and Man, respectively.The first row is the noise-free image.The second row is the noisy image corrupted by additive zero-mean Gaussian noise with  = 20, and third row is the denoising result by the proposed approach.

Figure 2 :
Figure 2: Denoising results by the proposed approach with  = 30, 40.(a), (b), (c), and (d) are Lena, Boat, Barbara, and Man, respectively.The first two rows are the noisy images corrupted by additive Gaussian with  = 30 and the corresponding denoising result, and last two rows are noise image with  = 40 and the corresponding denoising results.

Figure 3 :
Figure 3: Detailed comparison of the proposed approach with other approaches on image Lena with  = 30: (a) is the noise-free image; (b) is the corrupted image; (c)-(f) are the denoising results by NLM, NL-TV, NLMPCA, and NLMSAP; (g) is the denoising result by the proposed  0 -NLM.
(a) is a detailed show of original image Lena and Figure 3(b) is the corresponding noisy image with  = 30.Figures 3(c)-3(g) show the detail of denoising results of NLM, NL-TV, NLMPCA, NLSAP, R-NL, and proposed model  0 -NLM.From Figure (a) is the detail of hand of noise-free image Man and Figure 4(b) is the noisy image with  = 30.Figures 4(c)-4(g) are the denoising results of different algorithms.From Figure Lena, Boat, Barbara, Man, Cameraman, House, Peppers, Couple, Hill, Fingerprint, Mandrill, and Monarch.Images are corrupted by three levels of additive zero-mean white Gaussian noise, with standard variance  = 20, 30, 40, respectively.Parameters to achieve local prior are set as illustration in Section 3.2.For parameters in global prior acquisition, we set the sparsity  = 5, the downsampling ratio  = 2, and the layer number  = 4.The parameter settings are generated by fine-tuning on the training set, including Montage, Airplane, Bridge, and Truck, to the achieve highest average accuracy.We quantitatively evaluate the denoising quality with two conventional measurements: Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure