Image Restoration by a Mixed High-Order Total Variation and l 1 Regularization Model

Total variation regularization is well-known for recovering sharp edges; however, it usually produces staircase artifacts. In this paper, in order to overcome the shortcoming of total variation regularization, we propose a new variational model combining high-order total variation regularization and l1 regularization.The newmodel has separable structurewhich enables us to solve the involved subproblemsmore efficiently.We propose a fast alternatingmethod by employing the fast iterative shrinkage-thresholding algorithm (FISTA) and the alternating direction method of multipliers (ADMM). Compared with some current state-of-the-art methods, numerical experiments show that our proposedmodel can significantly improve the quality of restored images and obtain higher SNR and SSIM values.


Introduction
Image restoration is an important and fundamental problem in the literature of image processing.It is widely used in many fields, such as machine identification, biomedicine, astronomy, and medical imaging [1][2][3][4].The purpose of image restoration is to get a better visual image from the degraded image.There are many factors that cause image degradation, for instance, atmospheric turbulence, camera shake, and the relative motion between an imaging device and an object [5].
The typical representation of image degradation is the blurring and additional noise of image.The original model of image degradation can generally be defined as where  is an original image,  is a blur operator,  is an additive noise, and  is a degraded image.Our aim is to recover  from , which is known as deconvolution or deblurring.
Recovering  from  is a typically ill-posed problem.So we cannot solve it simply by using the normal leastsquares method.In order to overcome the ill-condition of the problem, adding some regularization terms to the energy functional is usually adopted.One of the commendable regularization methods is Tikhonov regularization [6].However, this method often tends to make images overly smooth and fails to preserve image details.In order to overcome this shortcoming, in 1992, replacing  2 -norm of  with the TVnorm of , Rudin, Osher, and Fatemi in [7] introduced the following famous Rudin-Osher-Fatemi (ROF) model: where ‖ ⋅ ‖ 2 denotes the Euclidean norm, the first term of (2) is called fidelity term, and ‖ ⋅ ‖  is the total variation regularization term. is a positive regularization parameter which controls the trade-off between the first and second terms.The isotropic discrete total variation regularization term can be defined as where ∇ denotes the discrete gradient operator which is defined as follows: (∇) , = ((∇)  , , (∇) for ,  = 1, . . ., .Here  , refers to the (( − 1) + )ℎ entry of the vector .It is the (, )ℎ pixel location of the image; see [8].
Recently, to further study the solving methods for ROF model, Huang et al. [46] introduced a new auxiliary variable  and proposed a fast total variation minimization method as follows: min where  1 ,  2 are positive regularization parameters.The main purpose of this model ( 6) is to realize the separation of denoising and deblurring steps.The authors solved system (6) by an alternating minimization algorithm.The experimental results show the effectiveness of their method.In the process of image processing, there are some abnormal values, such as the sudden jump of pixels in a certain area.However,  1 regularization is less sensitive to abnormal values and promotes sparse solutions.In [44] where  1 ,  2 ,  3 are positive regularization parameters.Due to  1 and  norm used in model (7), the experimental results show a good recovery effect.
It has been shown in [47,48] that total variation methods can realize significantly sharper edges and overall more visually pleasing images.However, on the other hand, they tend to create piecewise-constant images even in regions with smooth transitions of grey or color values in the original image.The undesired artifact is usually called staircase effect.Staircase solution fails to satisfy the evaluation of visual quality and develops false edges that do not exist in the true image.
To overcome the adverse effects in the application of image processing, a popular method is to replace the TV norm by a higher-order TV norm [45,[49][50][51][52]. Lysaker et al. first proposed second-order total variation regularizer in [53] for additive noise removal in the process of medical image processing.Their method shows robustness in the protection of the edges.Later, a second-order model to substantially reduce the staircase effect was proposed by Chan et al. in [48].In particular, the second-order TV regularization schemes are widely studied to overcome the staircase effects while preserving the edges well in the restored image.
Recently, Lv et al. in [45] proposed a high-order total variation minimization model: min where  1 ,  2 are positive regularization parameters and ‖‖  denotes higher-order total variation.The minimization model can reduce the staircase effect in the process of image restoration.The authors adopted an alternating minimization algorithm to solve the high-order minimization problem (8).Their experimental results showed that the proposed method gave the restored images with higher quality than some existing first-order total variation restoration methods.
Motivated by the works of [44][45][46], we propose a new unconstrained minimization model: where  1 ,  2 , and  3 are positive regularization parameters.The definition of second-order total variation is similar to the definition of the first-order one: , , denote the secondorder discrete gradient of the (( − 1) + )ℎ entry of the vector .For more details about the second-order difference, refer to [54].The second-order TV regularization and  1 regularization are used; the edges in the restored image can be preserved quite well and the staircase effect is reduced simultaneously.An alternating minimization method is used to solve the new unconstrained minimization model (9).By comparing with the other existing methods about total variation, our numerical experiments show the effectiveness of the minimization model (9).
The rest of this paper is organized as follows.In Section 2, we propose our alternating minimization algorithm to solve model (9), which is based on the fast iterative shrinkagethresholding algorithm (FISTA) and the alternating direction method of multipliers (ADMM).In Section 3, we give some numerical results to demonstrate the effectiveness of the proposed algorithm.Finally, concluding remarks of this paper are given in Section 4.

Alternating Minimization Method
In this section, we apply the ADMM idea to derive an algorithm for solving problem (9).We first introduce an auxiliary variable  to transform problem (9) Then, using the alternating minimization method to solve problem (12) can be expressed as follows: Fixing   ,   , the  subproblem can be obtained by solving the following minimization problem.Recently, various acceleration techniques of iterative algorithms are proposed [55][56][57][58][59][60][61][62][63].For this subproblem, in order to accelerate the convergence of the above iteration, we adopt the acceleration technique in [55] to solve it. where is a Lipschitz constant [55], and  0 = 1.So, the minimizer of  is given by the one-dimensional shrinkage: Fixing   ,  +1 , the subproblem for  is equivalent to According to [64], the solution of  can be obtained by the two-dimensional shrinkage: where the convention 0 ⋅ (0/0) = 0 is followed.Fixing  +1 ,  +1 , the subproblem for  can be written as follows: We can get the exact minimizer solution from the optimal conditions of (19) as follows: where ∇ refers to the first-order difference operator and ∇ 2  is the adjoint of ∇ 2 . represents the identity matrix.
Finally, we update Lagrange multiplier : Our method in this paper can be described as shown in Algorithm 1.

Numerical Experiments
In this section, we present some numerical examples of image restoration to illustrate the effectiveness of our proposed approach.In our experiments, we compare the proposed algorithm with the other three state-of-the-art methods, namely, FCP [44], high-order TV [45], and SALSA [43] where  0 , , and  are the original image, the restored image, and the noise vector added in the test, respectively. is the mean intensity value of  0 ,   0 and   are the mean value of  0 and , respectively. 2  0 and  2  are the variance of  0 and , respectively.And   0  is covariance of  0 and . 1 and  2 are constants.The SSIM is an index used to measure the similarity between restored image and ideal image.The closer the SSIM value to 1, the better the restored image.

Parameters Settings.
In this subsection, we discuss the selection of the three regularization parameters  1 ,  2 , and  3 in our method.In order to explain how to choose parameters in our algorithm, we recover the "Lena" image, which is blurred by Gaussian blur with size "9× 9" and added the Gaussian white noise with BSNR=35.The influence of different  1 ,  2 , and  3 on the restoration image is explained in Figure 2. As shown in Figure 2, if  1 is too small, the restored image will be overly smoothed with smeared image details.
In contrast, the restored image will be dominated by noise component with larger  1 values.We can see from Figures 2(e) and 2(f) that when the parameter  2 changes within a certain range, there is little effect on image restoration.The parameter  3 has the same effect as the one of  1 has.The restored image will be overly smoothed with small  3 values and dominated by noise component with larger  3 values.The parameters  1 ,  2 , and  3 in our method are manually adjusted to the optimal value.Therefore, the parameters used in the tests will be illustrated in the following experiments.

Gaussian Blur.
In this example, we consider the Gaussian blur.The "Man" image shown in Figure 3(b) is degraded by Gaussian blur.The function has two parameters: bandwidth  and standard deviation .In the test, we choose  =  = 7.
We added Gaussian noise with BSNR = 35 to the test image.
The parameters  = 0.025 and  = 0.35 are selected in SALSA.The parameters  1 = 0.02,  2 = 0.001, and  3 = 0.02 are selected in our method.The parameters  1 = 0.015,  2 = 0.2, and  3 = 0.05 are used in FCP method.For the high-order TV method, the parameters  1 = 0.8 and  2 = 0.005 are selected.The restored images obtained by SALSA [43], FCP [44], high-order TV [45], and our method are shown in Figures 3(c)-3(f).The changing curve of SNR of the reconstructed image during the experiment is shown in Figure 4.It can be seen from the changing graph that our method can get higher SNR than the other three methods.
In Table 1, we list the restoration results of the four different methods in SNRs and SSIMs.From Figures 3(c)-3(f), we can see that the SALSA method can make restoration image smooth, the FCP method causes staircase effect, and our method can induce sparse structure of the image compared with the high-order TV method.The visual quality of the restored image by our method is better than the other three methods.More precisely, the same regions of image are zoomed and processed in the same way; they are presented in Figure 5.It is not difficult to see that our method has some advantages over the other three methods in eliminating the staircase effect.

Average Blur.
In this example, we choose "Cameraman" image with size 256 × 256 as an object of study.The image is degraded by average kernel with window size 11 and contaminated by the Gaussian noise with BSNR = 40, which is shown in Figure 6(b).In this example, we choose the parameters  = 0.045 and  = 0.35 for SALSA method.The parameters  1 = 0.02, 2 = 0.01, and  3 = 0.02 are used in our method.The parameters  1 = 0.008,  2 = 0.3, and  3 = 0.05 are selected in FCP method.For high-order TV, the parameters  1 = 1.5 and  2 = 0.005 are used.Figures 6(c)-6(f) represent the restored images by SALSA, FCP, highorder TV, and our method, respectively.The change of SNR of the reconstructed image during the experiment is shown in Figure 7.The experimental results on SNR and SSIM are shown in Table 1.We see that both the SNR and SSIM values of the restored image by our method are higher than those provided by the SALSA [43], FCP [44], and high-order TV [45] method.The quality of restored images by our method is better than the other three methods.
Our method has some advantages in detail processing.In Figure 6, we have enlarged some details of image.The recovery results are presented in Figures 8(b)-8(e).Obviously, our method can well overcome the staircase effect and retain more image details.    in our method.In parameters, we set  = 0.035 and  = 0.5 for SALSA method.The parameters  1 = 0.008,  2 = 0.05, and  3 = 0.015 are used in FCP method.For the highorder TV method, the parameters  1 = 1.2 and  2 = 0.005 are selected.The restored images by the SALSA, FCP, highorder TV, and our method are shown in Figures 9(c)-9(f).In Figure 10, we plot the changes of SNR value versus iteration number for the four methods.Figure 11 shows the enlarged results of a part of Figure 9.The values of SNR and SSIM are listed in Table 1.We can see that our method can reduce the staircase effect effectively while preserving the edges.
We can see from the above three experiments that the images restored by our method have better qualities with less staircase effect and more detail textures than the other three    methods.In order to further verify the effectiveness of our method, for the same experiment, we tested all the images which are listed in Figure 1, and the experimental results are shown in Table 1.We can find from the experimental results that the SNR and SSIM values obtained by our method are higher than the other three methods.

Conclusion
In this paper, by combining high-order total variation regularization and  preserving edges of the restored image.We also show that our proposed method can obtain better results than the three state-of-the-art methods with higher SSIM and SNR values.

Figure 4 :
Figure 4: SNR versus iterations with four different methods.

Figure 5 :
Figure 5: The zoomed regions of the results of Figure 3. (a) Original region; (b) blurred and noisy image; (c) restoration region of SALSA [43]; (d) restoration region of FCP[44]; (e) restoration region of high-order TV[45]; (f) restoration region of our method.

Figure 7 :
Figure 7: SNR versus iterations with four different methods.

Figure 8 :
Figure 8: The zoomed regions of the results of Figure 6.(a) Original region; (b) blurred and noisy image; (c) restoration region of SALSA [43]; (d) restoration region of FCP[44]; (e) restoration region of high-order TV[45]; (f) restoration region of our method.

Figure 11 :
Figure 11: The zoomed regions of the results of Figure 9. (a) Original region; (b) blurred and noisy image; (c) restoration region of SALSA [43]; (d) restoration region of FCP[44]; (e) restoration region of high-order TV[45]; (f) restoration region of our method.

Table 1 :
The values of SNR and SSIM for different experiments.