Image Denoising Based on Dilated Singularity Prior

In order to preserve singularities in denoising, we propose a new scheme by adding dilated singularity prior to noisy images. The singularities are detected by canny operator firstly and then dilated using mathematical morphology for finding pixels “near” singularities instead of “on” singularities. The denoising results for pixels near singularities are obtained by nonlocal means in spatial domain to preserve singularities while the denoising results for pixels in smooth regions are obtained by EM algorithm constrained by a mask formed by downsampled spatial image with dilated singularity prior to suiting the sizes of the subbands of wavelets. The final denoised results are got by combining the above two results. Experimental results show that the scheme can preserve singularity well with relatively high PSNR and good visual quality.


Introduction
How to preserve edges and textures in image denoising is a very difficult and important topic.Many efforts focus on this topic for a long time.The most straight way to image denoising while preserving singularities is adaptive methods 1, 2 , which suggest that denoising should be carried on local regions.However, preserving singularities requires the size of local area to be small while denoising requires the size of local area to be large, which makes it become a trade-off problem.
Recently, Buades et al. argue that the small size of local area is not a requirement for preserving singularities in image denoising.That is, if correct similar pixels can be found through their similar local areas, singularity preserving denoising can be achieved by simple mean smoothing 3 .Nonlocal means has very good performance in maintaining the borders and textures than most of existing methods.Based on it, some improved algorithms are proposed 4-10 .However, its computation cost is very high.
Although some denoising methods on spatial domain, such as nonlocal means and its improvements, nonlinear anisotropic diffusion, and fractional calculus can provide satisfied denoising results, most of them suffer from high computation burden 9, 11-20 .Thus, some researchers suggest that wavelets can be used to reduce the computation cost and suppress noises.
Some works on wavelet domain propose that threshold values should be selected according to statistical rules; the type of these methods is called shrinkage denoising.In 21 , the threshold is determined by SURE shrinkage, which has better performance than threshold selected by Baysian rule.However, all coefficients in a subband are considered as independent identical distributed iid variables, and this is considered an unreal assumption.
In order to improve the performance of shrinkages, some schemes focus on how to describe the residual dependencies among wavelet coefficients 22-26 .In 22 , the authors present a tree structure to describe the dependency among the wavelet coefficients.Recently, 23 proposes that trivariates should be used for three subbands of wavelet coefficients.
Both methods can provide improved denoising results than independent model for wavelet coefficients, but they do not consider the difference between pixels near singularities and in smooth regions.That is, they think that the coefficients near singularities are the same as they are in smooth regions.Intuitively, it is wrong.
We suggest that the dependency among wavelet coefficients should be considered both on their local energy and their object labels 26 .However, object labels cannot be assigned correctly in serious noise especially in high-frequency subbands.
Most recently, following block similarity proposed in 3 , a denoised method based on collaborative filtering and wavelets is proposed.It obtains great success in its high value of peak signal noise ratio PSNR .However, estimating real levels both of singularity points and smooth points with large regions leads it has very high computation complexity.
In summary, there are two popular methods to preserve singularities in denoising: one is local area similarity proposed in 3 and then improved by many efforts; the other is adaptive denoising in small local regions.The former has good performance in preserving singularities but suffers from its high computation complexity while the latter has low computation complexity and good performance in smooth regions but blurs the singularities.
It reminds us that if we can distinguish points near singularities from the points in smooth regions correctly and design different denoising schemes for these two types of points separately, singularities can be preserved in denoising.Following this idea, dilated singularity prior is posed to find points "near" singularities instead of "on" singularities.After distinguishing pixels near singularities from pixels in smooth regions, the real level for pixels in smooth regions can be estimated by a smooth version of noisy image.By combining these two different denoising methods together, we can obtain good denoising results.
The detailed framework will be introduced as follows.The second section in this paper will introduce backgrounds, the third section introduces how to build mask, and the fourth section describes the denoising framework.Section 5 is the experimental results and discussion.Section 6 gives conclusions, and, finally, there is the acknowledgment part.

Backgrounds
In this section, we will introduce terms, notations, and concepts for image denoising, wavelets, and GMM.

Image Denoising
An image x is regarded as a realization of a random field X.In these terms, the image denoising problem can be rephrased as follows: given a noisy image, estimate for real gray level of each pixel.
Let the original image be {x i,j }, i, j 1, . . ., N, where N is some integer power of 2 and i, j is the pixel coordinate of the image.The image has been corrupted by additive noise and one observation y i,j x i,j n i,j , 2.1 where n i,j is an iid variable as normal N 0, σ 2 n and independent to x i,j .The goal is to remove the noise from y i,j and obtain an estimate x i,j of x i,j which minimizes the mean squared error MSE : Thus, the minimum MSE estimate would be the conditional mean estimate of a Gaussian signal with Gausssian noise where X i,j and Y i,j are the random variables of x i,j and y i,j respectively.However, only one observation a noisy image provides for us to estimate real gray levels for the image.Thus we have to share statistical information among image pixels.One common method used in adaptive denoising assuming pixels in a local region is iid variables.However, since local regions have singularities, iid variable assumption leads to blurred edges and textures.
Intuitively, pixels with more similar real gray levels have higher probability for identical distribution.Thus more plausible way for selecting iid pixels converts to find pixels with similar real gray levels in a local region.
However, finding pixels with similar real gray levels in noise is very difficult.In this paper, we use singularities detected in spatial domain combined with a mask built on LL subband of wavelets to help us find similar pixels in a local region.The whole framework will be given in Section 4.

Wavelets and Gaussian Mixture Model (GMM) [21]
Let W wx, W denote the matrix of wavelet coefficients of x, where w is the 2D dyadic orthogonal wavelet transform operator.It is convenient to group wavelet coefficients into groups or subbands of different scales and orientations, where, for example, the label HL 1 refers to those coefficients at the first scale of decomposition which are the output of the high-pass filter in the horizontal direction and the low-pass filter in the vertical direction.The subbands HH k , HL k , LH k , k 1, 2, . . ., J are called the details, where k is the scale, with J being the largest or coarsest scale in the decomposition, and a subband at scale k has size N/2k × N/2k.The subband LL J is the low-resolution residual.Wavelets lead to a natural structure of the wavelet coefficients into three subbands representing the horizontal, vertical, and diagonal edges.
The coefficients of DWTs of many real-world images tend to be sparse, with just a few nonzero coefficients containing most of the image energy.This property, combined with our view of the image as a realization from a family or distribution of signal, leads to a simple model for an individual wavelet coefficient.Hence, the marginal density of each wavelet coefficient is typically described by a peaky and heavy-tailed non-Gaussian density.
Such densities are well approximated by a two-component Gaussian mixture model 21 , and its overall pdf is given by In general, a 2-state Gaussian mixture model for a random variable W i,j consists of 1 discrete random state variable S i,j taking the value S i,j ∈ {0, 1} according to the probability mass function pmf ; 2 means and variances of Gaussian distributions.
In most applications of mixture models, the value ω i,j , can be observed, but the value of the state variable is not; we say that the value of S i,j is hidden.Although each wavelet coefficient W i,j is conditionally Gaussian given its state variable S i,j , the wavelet coefficient has an overall non-Gaussian density due to the randomness of S i,j .

EM Algorithm
EM algorithm is a kind of Max-likelihood Estimation MLE algorithm whose target is to find a set of hidden states to maximize the probability of observations.If the initial values for parameters are known, EM iterates between estimating the probability for the state Expectation and updating the model given the state probabilities Maximization .
The normal form of the E step and M step of EM algorithm is as follows.
Expectation E :

2.5
Maximization M : where N is the samples of an image.

Designing Mask
Just as discussed in Section 2.1, image denoising using statistical theory has to find similar pixels as iid variables.However, pixels near singularities have very different statistical properties to pixels in smooth regions.Thus, if two types of pixels, pixels near singularities and pixels in smooth regions, can be parted correctly, iid variables can be selected from the pixels with identical type labels.Based on this idea, we present a method to find similar pixels by posing dilated singularity prior.That is, singularity prior and its dilated version are used to find pixels near singularities while other pixels are considered as in smooth regions.However, polluted gray levels will also lead to noisy wavelet coefficients in the details of wavelets, which hampers us to detect singularities and measure the similarity among wavelet coefficients correctly.Therefore, posing singularity prior and finding similar pixels in the details of wavelets cannot be carried on directly using the details of wavelets.
One solution for these difficulties is to design a mask and then apply it to the details of wavelets for image denoising.In this section, we will discuss the method to design the mask in detail.

Dilated Singularity Prior
Pixels near singularities are different to the pixels in smooth regions.Thus if we can find the pixels near singularities correctly and handle them separately, the singularities can be preserved in denoising.On the other side, smoothing only among pixels in smooth regions, which is implemented by abandoning pixels near singularities, can help us reduce the outliers in estimate.Moreover, parting pixels to different groups can help us design different denoising methods separately according to their different statistical properties.Thus it can get satisfying denoising results for both groups.
However, in noise, singularities in the details of wavelets cannot be detected correctly.Moreover, the low-resolution residual of wavelets is an oversmoothing version for the noisy image, which blurs many important weak singularities, such as some textures and degraded edges.Therefore, singularities cannot be detected from the subbands of the wavelets directly.In this paper, the singularities are located in spatial domain using canny operator and then downsampled to suit the size of different subbands of wavelets.
It should be indicated that although some pixels near singularities have similar real gray levels to nearby pixels in smooth regions, they have different statistical natures to the pixels in nearby smooth region.Thus, we need find pixels near singularities instead of on singularities.Based on this consideration, the detected singularities by canny are dilated using mathematical morph operator.In summary, procedures for posing singularity prior are 1 detecting singularities in the noisy image using canny operator; 2 dilating singularities using mathematical morphology operator; 3 downsampling singularity image obtained in procedure 2 to form singularity prior for different subbands of wavelets.
One example for these three steps of noisy 512 × 512 barbara whose noise variance is 0.01 is shown in Figures 1 a -1 c .

Designing Mask
After parting pixels into two groups by posing dilated singularity prior, we have to measure the similarity for two groups correctly in noise.However, even having posed singularity prior, measuring similarity among pixels in smooth regions correctly in the details of wavelets is very difficult because of noise influence in high frequency.
Fortunately, the low-resolution residual of wavelets can provide a downsampled smoother version for noisy images.Thus, if the singularity prior is posed to the lowresolution residual of wavelets, the similarity among pixels in smooth regions can be measured correctly by the coefficients of the low-resolution residual.Based on this discussion, the mask for the details of wavelets is designed as posing singularity prior to the low resolution of wavelets.
Figures 1 a -1 c show the steps for building the singularity prior for pixels in smooth regions, and the final mask is shown in Figure 1 d .

The Framework
In this section, we will discuss the framework for the proposed method, whose flow chart is shown in Figure 2. In Figure 2, the squares without shadow represent the method used in this framework while the shaded squares represent the processing results for the methods, and ⊕ represents combining two squares together.The dashed frame with a title frame "Mask" is the steps for designing masks for different pixel groups presented in Section 3.
The proposed method is based on a novel idea of preserving singularities by processing pixels near singularities and in smooth regions separately.The steps for denoising to different types of pixels are shown in Figure 2 as the two dashed frames with number 1 and 2 in two triangles where number 1 indicates the denoising steps for pixels in smooth regions and number 2 indicates denoising steps for pixels near singularities.Both methods will be discussed detailedly in this section.

Denoising for Pixels Near Singularities
In order to preserve singularities, the pixels near singularities located by dilated singularity prior should be processed alone.Since LL subband of wavelets will blur the singularities while the details of wavelets will magnify the influence of noises, the denoising for pixels near singularities should be carried on spatial domain directly.In this subsection, we will present the denoising method for pixels near singularities.
For each pixel y i,j of image, after imposing dilated singularity prior, it has a one and only one label f i,j and f i,j ∈ {0, 1}, where 0 represents "non-singularity" and 1 represents "near singularity." Essentially, the most important procedure for image denoising is how to select similar points from image pixels.Similar pixels selected according to their block similarity are proposed in 3 recently.In this framework, the block is defined as a square with fixed size centered at the consideration pixel.In our method, the similarity among two pixels y i,j , y s,t with labels f i,j 1 and f s,t 1 can be measured by two 7 × 7 blocks centered at i, j and s, t : 4.1 Thus, for each pixel y i,j with f i,j 1, searches in a 21 × 21 window to find its similar pixels whose similarity between itself and y i,j is below a predefined threshold T : χ s, t ⎧ ⎨ ⎩ 1 : S y i,j , y s,t ≤ T, 0 : S y i,j , y s,t > T.

4.2
Thus the estimate of the real gray level of y i,j is where Since χ i − k, j − l and f i−k,j−l are two indicator functions, we can combine them as Therefore, 4.3 becomes where According to the above discussion, the denoising steps for pixels near singularities are as follows.
1 Initialization: give T , compute f i,j for all image pixels according to dilated singularity prior.
2 For an image pixel i, j with f i,j 1 a for k −10 : 10, l −10 : 10 compute S y i,j , y i−k,j−l using 4.1 , for all 3 Repeat step 2 a -c until all image pixels have been processed.

Denoising for Pixels in Smooth Regions
Wavelets, which are based on the idea that a linear, invertible transform will represent the image by the sparse wavelet coefficients whose structure is "simpler" to process, are a powerful tool to reduce the complexity.
However, in noise, there are two difficulties: one is the singularities cannot be located correctly in all subbands; the other is how to find similar pixels in the details of wavelets.In order to solve the above two problems, a mask will be built in the LL subbands and spatial domain together whose detailed algorithm is discussed in Section 3.That is, locate prior in spatial domain and then downsample and pose it to the LL subband to form the mask.Therefore, the real values of the details of wavelets can be estimated based on the mask.
Since the mask is built by posing singularity prior to the LL subband of each scale, we can give a label ι i,j,k for a pixel W 0 i,j,k in LL subband and ι i,j,k ∈ {0, 1}, where 0 represents "nonsingularity" and 1 represents "near singularity".Thus ι i,j,k s combined with W 0 i,j,k s forms the mask to measure the similarity between each pair of wavelet coefficients.
In our method, W t i,j,k , t 0, 1, 2, 3, is considered as a random variable with GMM distribution where i, j 1, . . ., N/2 k , t 0, 1, 2, 3. Thus if the parameters Θ {P S t i,j,k m , σ t m,i,j,k , m 0, 1} where σ t m,i,j,k , m 0, 1 are two standard deviations of Gaussian components, have known.The real value of W t i,j,k can be estimated as 4.9 where

4.10
However, only one observation is for W; thus we have to share statistical information to estimate parameter Θ.Generally, the statistical information should be shared among iid variables.Pixels with small spatial distance and similar real values are iid variables which is a plausible assumption.Then we will discuss how to find pixels with small distance and similar real values in the details of wavelets.Since the following steps are processed in a subband and one scale, we will omit index t for t 1, 2, 3, k for above notations for the sake of simple explanation.
The similar pixels of W i,j can be selected from a 2ν 1 × 2ν 1 window with measure of similarity where ω 0 i,j and ω 0 s,t are values of i, j and s, t in the LL subband, respectively; the φ ω 0 i,j , ω 0 s,t is a binary value function φ ω 0 i,j , ω 0 s,t ⎧ ⎨ ⎩ 1 : ι i,j ι s,t , 0 : otherwise.

4.13
Thus the parameter Θ can be estimated using EM algorithm in the collection Φ i,j .
Expectation E : where |Φ i,j | is the number of elements in Φ i,j .In summary, the steps of denoising for pixels in smooth regions are 1 Initialization: input scale J, dilated singularity prior F whose size is N × N, and ν.
2 Build marks for the details of different scale using downsample (see Section 3).
3 Find similar pixels for each W t i,j,k : find similar points using 4.11 and then compute φ.
4 EM algorithm, for each W t i,j,k , estimate parameter Θ using EM algorithm in 4.14 -4.15 .
5 Repeat Steps 3 -4 until all wavelet coefficients have been processed.
6 Inverse wavelet transform: obtain denoised results using inverse wavelet transform for all processed coefficients.
7 Get final denoised image: the final denoised image can be got by combining both denoising image together; that is, the real values for pixels near singularities are estimated using the method in Section 4.1, and the values for other pixels are estimated using steps 1 -6 .

Experimental Results
In order to compare our method with state-of-the-art methods and denoising methods using wavelets, the proposed method is compared with nonlocal means 3 , BM3D  similarity between a pair of pixels, which is proposed firstly in 3 .BM3D designs a complex hierarchical scheme to get very high PSNR, which is usually used as benchmark in many denoising scheme.Moreover, HMT which is presented in 22 is a famous wavelet denoising method capturing dependency among the details of wavelets using a tree structure while BLS-GSM is a wavelet denoising method using adaptive shrinkage and GMM.
Since the objective for presenting the denoised results of our method only focuses on whether adding singularity prior can improve the denoising performance, we use some simple tools for testing the potential of our new framework.The wavelet used in this paper is one scale "Haar"; similarity threshold τ, ν in Φ i,j is from 14 to 28 and 3, respectively.Even by using this simple form, the proposed method has good performance both in PSNR and visual quality see Figure 3 .That is, its average PSNR only is 0.2 db lower than the nonlocal means 3 ; the average PSNR for lena is 0.4 db lower than the BM3D, and for Barbara, it is 1.4 db lower than BM3D see Table 1 .
Although the values of PSNR for proposed method are lower than both nonlocal means and BM3D, considering both of them being state-of-art denoising methods and their computation complexity, PSNR of proposed method is an acceptable result.Moreover, the main objective for this paper only focuses on designing a new scheme to improve performance in singularity preserving by adding singularity prior in image denoising.We think the experimental results can demonstrate the potential of proposed method.Based on this framework, some different methods can be used to improve the performance further.
For example, the performance can be improved by processing pixels near singularities using BM3D, using different wavelets, designing different denoising methods for pixels in smooth regions, or by designing more suitable singularity location tools, and so forth.However, discussion about how to improve our framework further is beyond the scope of this paper.

Conclusions
In this paper, we propose a new scheme to preserve singularities in image denoising by adding dilated singularities to noisy image.This scheme even that uses simple tools can get satisfying denoising results both in PSNR and visual quality.Since the scheme is based on the fact that pixels near singularities have different statistical properties to the pixels in the smooth regions, it is designed by parting the pixels into two groups using dilated singularity prior and handled pixels with different type labels separately.Dilated singularity prior used to locate the pixels "near" singularities instead of "on" singularities can help us obtain satisfied denoising results and reduce computation complexity.Moreover, denoising in smooth regions can be carried on a smoother version of the noisy image, called mask, since the pixels near singularities have been excluded.Based on the above well-designed framework, the proposed method can obtain good denoising results.

c
Step 3: the downsampled dilated singularity prior nz = 29171 d The mask composed by LL 1 and singularity prior

Figure 1 :
Figure 1: The steps for locating singularity prior and making the mask.a -c three steps for locating singularity prior in Section 3.1; d the mask in Section 3.2.

Figure 2 :
Figure 2: The flowchart of proposed method.

Figure 3 :
Figure 3: The noisy image is shown in a whose standard deviation of noise is 25, and the denoised results using different methods are shown in b -f .