A Modified Iterative Alternating Direction Minimization Algorithm for Impulse Noise Removal in Images

Images are often corrupted by impulse noise. In this paper, an alternating direction minimization with continuation algorithm is modified and iteratively used to remove the impulse noise in images by exploring its self-similarity. A patch-based nonlocal operator and sparse representation are married in the l 1 -l 1 optimization model to be solved. Simulation results demonstrate that the proposed algorithm outperforms typical denoising methods in terms of preserving edges and textures for both salt-and-pepper noise and random-valued impulse noise. It can be also applied to suppress impulse noise-like artifacts in real mural images.


Introduction
Image denoising is a fundamental problem in image processing.Besides the commonly processed Gaussian noise, impulse noise (Figure 1(a)) is another typical type of noise generated due to noisy sensors or channel transmission errors [1].In mural images, the impulse noise-like artifacts are also observed because of the bacterial plaque (Figure 1(b)) or missing objects (Figure 1(c)).A classical impulse denoising method is median filtering, which replaces the central pixel with a median-value within a local window.Median filtering is with low computation but often loses image structures when noise is heavy.The denoising performance is expected to be greatly improved if the image structures are extracted and preserved in denoising.
In recent years, sparse representation method has attracted great attention to provide effective representation of image structures and found promising applications in image denoising [2][3][4][5], image decomposition [6,7], image fusion [8,9], pattern recognition [10][11][12][13], and so forth.In impulse noise removal, sparse representation is remarkably effective at smoothing away noise [14][15][16], but it may lose image structures which are not sparsely represented by its predefined basis such as wavelet [3,14].Although the adaptive dictionary training [15,16] improves the denoising performance, its computation cost of training dictionaries while removing noise is relatively high.Besides, the commonly used total variation [17][18][19][20] is highly related to enforcing the sparsity of finite difference of images.However, total variation assumes that images are piecewise smooth; thus, textures would be easily lost after denoising [15,16,21,22].Therefore, exploring the sparse representation for impulse noise removal is still challenging.
Similarity is often rooted in repeated patterns of images [23][24][25] and greatly improves image quality in deblurring [26] and medical image reconstruction [27].Similarity information was previously investigated in block matching 3D frames [26].Patch-based nonlocal operator (PANO) [27] is a linear operator to model the sparse representation of similar patches, which can be viewed as an alternative form of the block matching 3D frames.Unlike the traditional sparsifying methods, for example, conventional wavelets or finite difference, PANO provides an adaptive sparse representation of image with a very fast training phase.Besides, the linearity of PANO allows setting up explicitly a sparsity-based denoising model.The contributions of this paper are summarized as follows.
(1) The impulse noise is removed by making use of similarity-motivated sparse representation of images.Mathematically, we model the denoising problem as an  1 - 1 minimization problem with PANO which provides the sparse representation of similar image patches.
(2) To solve the optimization problem, a fast alternating direction minimization with continuation algorithm is developed by taking the patch-based representation property into account.
(3) Besides removing salt-and-pepper noise and random-value impulse noise in images, the proposed method is also applied in recovering mural images from corrupted data.Experimental results on simulated data and real mural images verify that the proposed method outperforms other methods in removing impulse noise or impulse noise-like artifacts.

Model
In this section, the impulse noise model and typical denoising methods are briefly summarized.Then, the proposed denoising model is presented.
2.1.Impulse Noise.Image contaminated with impulse noise is usually modeled as where u is the noiseless image, f is the observation, ⊙ denotes an element-wise product,  ∈ R  is a sample drawn from an i.i.d.multivariate Bernoulli distribution with success probability 1−, 1 ∈ R  represents a vector with all elements being equal to one,  ∈ R  is either salt-and-pepper noise or random-valued impulse noise.
For the salt-and-pepper noise, the element   of  satisfies where  can be chosen as total variation assuming that the image is piecewise constant [19,28] or ℓ 1 norm of wavelet coefficients assuming that the image is sparse in the wavelet domain [14].However, total variation is observed to easily lose textures [16] and wavelet may lose image structures that are not sparsely represented by its predefined basis.Thus, how to preserve the edges and textures remains open for the impulse denoising problem.In this paper, image self-similarity is employed to provide an optimized sparse representation of images.Assuming that the similarity information of image is known with PANO, an ℓ 1 -ℓ 1 regularization model is proposed to remove the impulse noise,
PANO is a linear operator to model the sparse representation of similar patches [27,29].For a given image u, PANO is defined as where P  denotes the patch extraction, R V  means grouping some of the patches into a 3D cube, and Ψ 3D is 3D Haar transform.Figure 2 illustrates the process of grouping patches into a cube.First, an image is extracted into patches.Then, some similar patches are grouped together and stacked into 3D cubes.Finally, a sparse representation will be achieved by performing a 3D Haar transform on each cube, where   is the sparse vector for the th cube [27,29].The adjoint operator of where O is a diagonal matrix whose th diagonal entry is a counter indicating the repetitions that the th pixel is grouped into the 3D cubes.Therefore, by taking this patchbased representation property into account, the traditional alternating direction algorithm can be derived to achieve fast computation.

Algorithm
To numerically solve the optimization problem in (4), an alternating direction minimization with continuation (ADMC) algorithm [30,31] is modified to remove the impulse noise.ADMC algorithm is also used for image deblurring [32,33].Here, by introducing auxiliary variables   = A  u and z = f − u to (6), one has As  → ∞, the solution of (7) approaches that of (6).In practice, we use the previous solution as a "warm start" for the next alternating optimization as  gradually increased. does not need to be very large for image denoising.Simulation results show that increasing it will not improve the quality of denoised image but need more computation time.Thus  ≤ 2 14 is suggested.When  is fixed, (7) can be solved in an alternating fashion between   ( = 1, . . ., ), z, and u.Consider the following.
(2) Fix u and ; solve min (3) Fix z and ; solve min u ∑  =1 ‖  − A  u‖ 2 2 + ‖z − (f − u)‖ 2  2 , whose solution is To construct PANO, similarity needs to be learnt so that some similar patches are grouped together and stacked into 3D cubes.However, the similarity is corrupted for a noisy image.To obtain a reasonable similarity, we need to have an initial denoised image as a guide image.The similarity is first learnt from a denoised image using conventional methods such as median filtering and then learnt twice from the denoised image using PANO.The process of the modified algorithm is summarized as Algorithm 1.

Results
In simulations, the proposed PANO-based denoising method is compared with median filtering [1], wavelet-based sparse representation denoising, and total variation denoising.The dual-tree complex wavelet, which was used in impulse denoising [14], is adopted as the wavelet transform.PANO is applied with the typical parameter settings: the number of similar patches is 16, the patch size is 8 × 8, and the search region is 39 × 39.Objective image quality assessments, peak signal-tonoise ratio (PSNR), and mean measure of structural similarity (MSSIM) [34] are evaluated.The PSNR is defined as where  is the number of pixels in an image and u and û are the original and denoised images, respectively.MSSIM is defined as where a  and b  are the image content at the th local window and SSIM evaluates local reconstruction errors by measuring the similarity between two images in a local window [34].
A higher MSSIM value means that structural information is better preserved and more suitable for human visual system.The experiments run on 4 Cores 2.6 GHz CPU laptop computer with 12 GB RAM.The computational time of the proposed method is approximately 25 (or 70) seconds for image with size 256 × 256 (or 512 × 512).For the simulated noisy images, the window size of the median filtering and the regularization parameters of the rest methods are optimized to maximize MSSIMs while removing most of the impulse noise.

Choose Guide Images.
The similarity is assumed to be known in the proposed denoising model.However, the similarity is also corrupted for a noisy image.To obtain a reasonable similarity, we need to have an initial denoised image as a guide image.In this section, we will discuss how to choose a guide image.
Since conventional denoising methods are available, one may consider taking the denoising result by median filtering, total variation, or wavelet denoising as the guide image to learn the similarity.When a better denoised image is obtained using the proposed approach, one may further utilize the result as an updated guide image.
As shown in Figure 3, different initial guide images lead to comparable denoising results with 30% noise.So median filtering is chosen to produce the initial guide image since its computation is the fastest.After the similarity is learnt for 3 times, both the PSNR and MSSIM of the proposed method become stable, and more similarity learning will cost more computation time.Hence, learning the similarity 3 times is sufficient.

Remove Simulated Impulse
Noise.Images (Figures 3(a), 4(a), and 4(d)), which are usually used in image processing, are treated as ground truth images; salt-and-pepper noise or random-valued impulse noise is manually added in simulations.

Salt-and-Pepper Noise
Removal.The performances of the four denoising methods for salt-and-pepper noise are compared.As shown in Figure 5, repeated image patterns, for example, textures in Barbara, are obviously lost using median filtering and total variation.Although wavelet recovers the textures much better than the above two methods, it suffers from losing edges.The proposed method preserves these features best, with the clearest textures in Barbara and edges in both House and Boat.
As shown in Figure 6, the proposed method achieves the highest PSNRs when the noise level is below 40% and the highest SSIMs for all noise levels.When the noise becomes heavier (e.g., 50%), the performance of the proposed method gradually dropped.As Figure 7 shows, the qualities of the initial guide images using median filtering, total variation, and wavelet denoising methods are all unsatisfactory; thus, similarity cannot be learnt well so that the denoising results of the proposed method are also not satisfactory.Table 1 shows the performance of the proposed method using different initial guide images under heavy noise.Suppose we have the original image as the initial guide image so that similarity information can be exactly leant; the proposed method can remove the noise very well and preserve the image structures.In summary, the proposed method leads to comparable denoising results when ground truth image is not available.

Random-Valued Impulse Noise
Removal.The performances of the four denoising methods for random-valued impulse noise are also compared.As shown in Figure 8, edges and textures are best preserved by the proposed method.Under different noise levels, proposed method achieves the highest MSSIMs (Figure 9).These observations imply that the proposed method preserves the best image structures during removing the random-valued impulse noise.(a1) (b1) ( c 1)

Remove Impulse Noise-Like Artifacts in Mural
Images.In the section, the impulse noise-like artifacts in realistic mural images are suppressed using these denoising methods.As shown in Figure 10, both median filtering and total variation will easily blur the images.On the contrary, the proposed method preserves the image details much better and suppresses most the impulse noise-like artifacts as marked.We can observe that the lost image structures are minimal among these methods (Figure 11).Therefore, the proposed method can also be used to remove these impulse noise-like artifacts of real dataset.

The Effect of Noise Detectors.
Noise detectors are very useful to improve the denoising performance of salt-andpepper noise.Chan et al. [18] proposed to use a noise detector first and apply a weighted regularization model with less penalization on the noise-free pixels.Following this idea, we use a noise detector to distinguish the noisy pixels from the rest of the pixels in the preliminary results.The locations of detected noise candidates are stored in a set Ω, whose complementary set Ω stores locations of the rest of the pixels.We tried a weighted  1 - 1 regularization model as where W is a diagonal matrix, whose th diagonal entry is   .A small weight   ( ∈ Ω) is assigned for noise candidates and a large weight   ( ∈ Ω) is set for the rest of pixels.Ψ is chosen to be total variation or PANO.Simulation results listed in Table 2 show that noise detectors can significantly improve the denoising performance.When both methods are equipped with noise detectors, the proposed method still outperforms total variation with higher MSSIMs and comparable PSNRs.This implies that the proposed method can preserve image structures better, as shown in Figure 12.
An efficient method to suppress the noise via combining PANO with noise detectors can be found in [35].However, the noise detectors proposed in [18] only work for salt-andpepper noise.How to detect the noise candidates in randomvalued noisy image is still unknown.Besides, how to solve the proposed weighted regularization model efficiently and setting the weight optimally need comprehensive investigation.We leave these as the future work.

Conclusions
Impulse noise removal with similarity-motivated sparse representation is proposed.With a patch-based nonlocal operator to model sparse representation of similar image patches, the  1 - 1 optimization problem is solved by a modified iterative alternating direction minimization algorithm.
Experiments on simulated data demonstrate that the proposed method significantly improve the recovery of textures and edges while removing both salt-and-pepper noise and random-valued impulse noise.It can be also used to remove the impulse noise-like artifacts in mural images well.Denoising performance of the proposed method may be further improved if one can learn a proper similarity from serious noisy image.

Figure 6 :
Figure 6: Denoising performance for salt-and-pepper noise.(a), (b), and (c) are the PSNRs of image Barbara, House, and Boat; (d), (e), and (f) are the MSSIMs of these images.

Figure 9 :
Figure 9: Denoising performance for random-valued impulse noise.(a), (b), and (c) are the PSNRs of image Barbara, House, and Boat; (d), (e), and (f) are the MSSIMs of these images.

Figure 12 :
Figure 12: Denoising results of Barbara image with 70% salt-and-pepper noise.(a) Ground truth image, (b) noisy image, and (c), (d), and(e) are denoised images using the proposed method, total variation with noise detectors, and the proposed method with noise detectors, respectively.

Table 1 :
Effect of initial guide image with 50% salt-and-pepper noise in Barbara.

Table 2 :
Denoising performance of Barbara at high salt-and-pepper noise levels.