MPE Mathematical Problems in Engineering 1563-5147 1024-123X Hindawi Publishing Corporation 10.1155/2014/606170 606170 Research Article Semiblind Image Deconvolution with Spatially Adaptive Total Variation Regularization http://orcid.org/0000-0003-1551-9414 Ruan Yaduan 1 http://orcid.org/0000-0002-7949-8846 Fang Houzhang 2 Chen Qimei 1 Dai Binxiang 1 School of Electronics Science and Engineering Nanjing University Nanjing 210023 China nju.edu.cn 2 School of Automation Huazhong University of Science and Technology Wuhan 430074 China hust.edu.cn 2014 27 8 2014 2014 23 06 2014 13 08 2014 27 8 2014 2014 Copyright © 2014 Yaduan Ruan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

A semiblind image deconvolution algorithm with spatially adaptive total variation (SATV) regularization is introduced. The spatial information in different image regions is incorporated into regularization by using the edge indicator called difference eigenvalue to distinguish flat areas from edges. Meanwhile, the split Bregman method is used to optimize the proposed SATV model. The proposed algorithm integrates the spatial constraint and parametric blur-kernel and thus effectively reduces the noise in flat regions and preserves the edge information. Comparative results on simulated images and real passive millimeter-wave (PMMW) images are reported.

1. Introduction

Image deconvolution is a classical inverse problem that has been widely investigated in astronomical, medical, and remote sensing imaging. Over the years, many deconvolution algorithms have been developed (for reviews see [1, 2]). In this paper, we are concerned with the semiblind deconvolution, in which the point spread function (PSF) is described as a parametric Gaussian form. The degraded model can be described as (1) g = H σ u + n , where H σ denotes the convolution operator with a Gaussian kernel h σ parameterized by its width σ , that is, H σ u = h σ ( x , y ) * u , n denotes the system noise, and g and u are the observed image and the true image, respectively.

Deconvolution is an ill-posed problem and should be regularized. The TV regularization was first proposed by Rudin et al.  in image denoising. Because of its advantages in preserving edge and detailed information, it has attracted strong research interest . Nevertheless, TV model has certain shortcomings. First, TV regularization favors a piecewise constant solution, and, as a result, the processing results in the flat region are poor and some “staircase effects” are produced (Figure 1(b)). Second, TV-based method is sensitive to the regularization parameter. For instance, a large regularization parameter can help reduce the noise in flat regions but blur the edge and detailed information at the same time (Figure 1(c)).

Semiblind deconvolution with different TV regularization strengths. (a) Degraded image. (b) Best (with the highest PSNR value) result. (c) Result with large regularization strength.

In many practical situations, the PSF can be modeled by the physics/optics of the imaging set-up and described as parametric function. Then, during the deconvolution, only the PSF parameters need to be estimated. It is often called semiblind deconvolution. Some works are available in [2, 6, 7]. In , Bar et al. presented an integrated variational framework for simultaneous semiblind restoration of Gaussian blur and image segmentation. The objective functional is (2) F ε ( u , v , σ ) = 1 2 H σ u - g 2 2 + G ε ( u , v ) + γ h σ 2 2 , (3) G ε ( u , v ) = β Ω v 2 | u | 2 d x d y + α Ω ( ε | v | 2 + ( v - 1 ) 2 4 ε ) d x d y , where v denotes the edge map and Ω is the support of the image. They have demonstrated successful semiblind image restoration that is more robust than the total variation blind image restoration  where the kernel form is not restricted. However, the framework of  is very sensitive to noise. Extracting the edge map v will fail in the presence of noise. Therefore, it is limited in practical application. Further illustration can be found in .

In this paper, a semiblind image deconvolution model with spatially adaptive TV regularization (SATV)  is proposed. An effective spatial information extractor called difference eigenvalue, recently proposed by Tian et al. , is used to extract the spatial information in the image. For each pixel, a weighted parameter determined by spatial information is added to constrain the TV regularization strength. For flat area pixels, a large weight parameter is set to suppress noise. On the contrary, for nonsmooth area pixel, a small weighed parameter is set to preserve edge and detail information. To make the SATV regularization easy to optimize, the split Bregman method is used to solve the nonsmooth energy model. Our experimental results show that the proposed SATV method can automatically balance the regularized strength between different spatial property regions in the image, and thus it can effectively reduce the noise in flat regions as well as preserving the edge and detailed information.

The remainder of this paper is as follows. In Section 2, the proposed model and efficient solution are presented. In Section 3, experimental results are given. Finally, conclusions are drawn in Section 4.

2. SATV Model and Optimization Solution

In this section, our spatially adaptive TV semiblind deconvolution model is presented, and then the split Bregman method is used to solve the model.

2.1. Formulation of Our Model

In this paper, a spatially weighted TV regularization model considering the spatial dependent property of the TV regularization is introduced. The key problem is to select a good spatial information indicator, which can distinguish edges from flat areas in the image. Several alternative edge indicators have been proposed, such as gradient operator, Gaussian curvature, difference curvature , and difference eigenvalue . As shown in Figure 2, the difference eigenvalue is more robust to indicate the edge under significant noise. So we use difference eigenvalue to indicate the spatial information.

Edge map indicated by (a) gradient magnitude, (b) Gaussian curvature, (c) difference curvature, and (d) difference eigenvalue.

The difference eigenvalue  is based on the Hessian matrix of the image. We use a Gaussian filtered version of the Hessian matrix to improve the robustness to noise: (4) J η = [ j 11 j 12 j 12 j 22 ] = [ u x x * G η u x y * G η u x y * G η u y y * G η ] , where G η denotes the Gaussian kernel with the parameter η (the size is 5 × 5 and η = 0.8 in this paper). The two eigenvalues of J η , denoted by λ 1 and λ 2 , are given by (5) λ 1,2 = 1 2 [ ( j 11 + j 22 ) ± ( j 11 - j 22 ) 2 + 4 j 12 2 ] . Let λ 1 denote the larger eigenvalue and let λ 2 denote the other eigenvalue. Here λ 1 and λ 2 correspond to the maximum and minimum local variation at a pixel, respectively. The difference eigenvalue edge indicator D ( x , y ) is defined as (6) D ( x , y ) = ( λ 1 - λ 2 ) λ 1 w ( u ( x , y ) ) , where w ( u ( x , y ) ) is a weighting factor which is used to achieve balance between detail enhancement and noise suppression. Its value is estimated from the gray-level variance and defined by (7) w ( u ( x , y ) ) = ζ ( x , y ) - min ( ζ ) max ( ζ ) - min ( ζ ) , where min ( ζ ) and max ( ζ ) are minimum and maximum gray-level variances of u , respectively. For a given pixel with coordinates ( x , y ) , the gray-level variance is calculated from its 3 × 3 neighborhood: (8) ζ ( x , y ) = 1 9 i = - 1 1 j = - 1 1 [ u ( x + i , y + j ) - u ( x , y ) ] .

With the difference eigenvalue edge indicator D ( x , y ) , the SATV model proposed in this paper is defined as (9) SATV = Ω ρ | u | d x d y = Ω 1 1 + θ D ( x , y ) | u | d x d y . Here ρ = 1 / ( 1 + θ D ( x , y ) ) is the spatial weight and θ is the contrast factor. To summarize, several good features of SATV can be drawn: (1) for flat area pixels, because D ( x , y ) is close to zero, ρ is close to 1, which means that a large TV regularization strength is enforced, and then the noise will be well suppressed; (2) for edge and detailed pixels, because D ( x , y ) is very large, ρ is small and almost close to zero, and this weakens the TV regularization strength, so the edge and texture will be well preserved.

Substituting G ε ( u , v ) in (2) with SATV presented in (9), we introduce a new total bounded variation-based image restoration model as (10) min u , σ 1 2 H σ u - g 2 2 + λ Ω ρ | u | d x d y + μ 2 u 2 2 + γ h σ 2 2 .

Note that the quadratic regularization term about u provides the probability to discriminate the structure of stability results from that of the nonquadratic bounded variation term .

2.2. Numerical Algorithm

The split Bregman iteration method [5, 13] is used to solve the nonsmooth model (10). The discrete version of (10) is expressed as (11) min u , σ 1 2 H σ u - g 2 2 + λ T V ρ ( u ) + μ 2 u 2 2 + γ h σ 2 2 . The discrete spatially weighted TV model is defined by (12) T V ρ ( u ) = i j Ω ρ i j ( i j h u ) 2 + ( i j v u ) 2 , where i j h u and i j v u denote the horizontal and vertical first-order differences at pixel ( i , j ) , respectively. By letting d = u , this yields a constraint optimization problem: (13) min u , d , σ 1 2 H σ u - g 2 2 + λ i j Ω ρ i j ( d i j h ) 2 + ( d i j v ) 2 kkkkkk + μ 2 u 2 2 + γ h 2 2 s . t . k d = u .

Rather than considering (13), we will consider an unconstraint problem: (14) min u , d , σ 1 2 H σ u - g 2 2 + λ i j Ω ρ i j ( d i j h ) 2 + ( d i j v ) 2 kkkkk + β 2 d - u 2 2 + μ 2 u 2 2 + γ h σ 2 2 . Then the corresponding split Bregman algorithm is given as shown in Algorithm 1. Here, ε 1 is a small positive constant.

<bold>Algorithm 1: </bold>Algorithm (SBD-SATV).

(1) Initialization: u 0 = g , b 0 = 0 , d 0 = 0 , σ = σ 0 , σ prev > > 1

(2) While ( | σ prev - σ | > ε 1 ) repeat

u k + 1 = arg min u 1 2 H σ u - g 2 2 + β 2 d k - u - b k 2 2 + μ 2 u 2 2

d k + 1 = arg min d λ i j Ω ρ i j ( d i j h ) 2 + ( d i j v ) 2 + β 2 d - u k + 1 - b k 2 2

b k + 1 = b k + ( u k + 1 - d k + 1 )

Set σ prev = σ , and solve

σ = arg min σ 1 2 H σ u k + 1 - g 2 2 + γ h σ 2 2

End

(3) Output: u k + 1

Note that d k + 1 can be computed by direct formula and to get u k + 1 we solve a linear system of equation: (15) u k + 1 = H σ * g + β * ( d k - b k ) H σ * H σ + β * + μ , (16) d k + 1 = max ( | u k + 1 + b k | - λ β ρ , 0 ) u k + 1 + b k | u k + 1 + b k | , where H σ * is the adjoint operator of H σ . Minimizing with respect to the scalar parameter σ , one can obtain (17) i , j Ω ( H σ u - g ) ( h σ σ * u ) i , j + γ i , j Ω ( σ | h σ | 2 ) i j = 0 , where (18) h σ σ = 1 2 π σ 2 exp ( - x 2 + y 2 2 σ 2 ) ( x 2 + y 2 σ 3 - 2 σ ) , | h σ | 2 σ = 1 2 π 2 σ 4 exp ( - x 2 + y 2 σ 2 ) ( x 2 + y 2 σ 7 - 4 σ 5 ) × ( x 2 + y 2 ) . The bisection method was used to solve (17) for σ . The discrete support of the Gaussian was limited at about 21 σ × 21 σ . Equation (15) is computed in the Fourier domain, which helps accelerate the convolution operation .

3. Experiments’ Results

In this section, numerical results are presented to test the proposed method. For comparison, the semiblind restored method proposed by Bar et al.  is also tested. We call it Bar-SBD method later.

Peak-signal-to-noise ratio (PSNR) and the structural similarity (SSIM)  are used to measure the restored quality. For the simulation, the satellite, Lena, and cameraman images were blurred by Gaussian PSFs with standard deviation 2.1 of sizes 11 × 11 and 15 × 15, respectively. Then zero-mean Gaussian noise with standard deviations 0.001, 0.005, and 0.01 is added by the MATLAB command imnoise. The initial value σ 0 was 0.5 and convergence tolerance ε 1 was 10−3.

Example 1.

Figure 3 shows the restoration results for the PSF size of 11 × 11 and noise level of 0.005. From the three images, it can be seen that our method has restored much sharper results with less residual noise. Moreover, our method showed good convergence property. The estimated kernel width converged nearly to the accurate value after about 15 iterations, as illustrated in the bottom line of Figure 3.

(a) Blurred and noisy images. (b) Results by Bar-SBD. (c) Results by the proposed method. (d) Convergence curve of the estimated width σ by the proposed method.

Example 2.

We restored the images degraded by Gaussian PSF size of 15 × 15 and the three noise levels. The PSNR and the calculating times of the two methods are listed in Table 1. σ ^ is the estimated kernel width. The implementations were in MATLAB on a 2.93 GHz PC. The advantage of our method appears obvious. In every case, our method has achieved the higher PSNR and the lower computational time. Also, in every case, our method obtains more accurate estimates of the actual blur parameter. These results show that our method outperforms the Bar-SBD method at a wide range of noise levels.

PSNR (dB) and CPU time (seconds) of two methods (the numbers 1, 2, and 3 denote satellite, Lena, and cameraman image, resp.).

Noise level PSNR Bar-SBD Proposed
PSNR Time σ ^ PSNR Time σ ^
1 0.001 22.10 22.93 119.4 1.75 24.46 46.4 2.21
0.005 22.09 22.76 78.8 1.79 24.41 46.1 2.21
0.01 22.03 22.11 168.5 1.83 24.28 59.4 2.20

2 0.001 23.36 24.37 60.1 1.54 27.01 59.6 2.06
0.005 23.34 24.26 79.2 1.56 26.86 64.0 2.06
0.01 23.28 24.21 80.6 1.60 26.41 59.0 2.00

3 0.001 21.93 22.44 133.2 1.66 24.58 48.4 2.10
0.005 21.91 22.36 133.9 1.68 24.54 52.1 2.09
0.01 21.86 22.08 128.1 1.70 24.32 54.5 2.08
Example 3.

A restoration example of real passive millimeter-wave (PMMW) image is shown in Figure 4. The PMMW image was taken with a 94 GHz passive millimeter-wave imager at London Heathrow airport.

Restoration of a real PMMW image. (a) From left to right: degraded image, Bar-SBD restored result, and our restored result. (b) Close-up views.

Because of the long wavelength of millimeter-wave, the PMMW image often suffers poor spatial resolution.

Image deconvolution has been extensively researched to improve the resolution . From the results, it can be seen that sharper details are restored by our method, for instance, the number “2” and the car at right corner. Benefiting from the spatially adaptive regularization and split Bregman solution, the proposed method converges well for the practical degraded image.

Example 4.

Another restoration example of real PMMW image is shown in Figure 5. It can be observed from Figure 5(b) that the recovered image by Bar-SBD amplifies the noise obviously. As shown in Figure 5(c), the restored image by proposed method looks natural and the noise is suppressed well. The convergence process of the proposed method is illustrated in Figure 5(d).

Restoration of a real PMMW image. (a) Real PMMW image. (b) Bar-SBD restored result. (c) Our restored result. (d) Convergence curve of the estimated width σ by the proposed method.

We remark that there are several parameters to be tuned up in our proposed methods, and we have to choose optimal parameters in order to make the relatively fair comparisons. We have tried our best to determine the optimal parameters. In (9), the parameter θ in the spatial weight 1 / ( 1 + θ D ( x , y ) ) controls the ability to distinguish the edge and texture area from the flat and ramp area. To show the effect of the parameter θ on the restored result, Table 2 lists the PSNR and SSIM value when θ is selected to be 0.0005, 0.01, 0.05, 0.1, 0.5, 1, and 5, respectively. It can be seen that the PSNR and SSIM values show little change with the change of the parameter θ and achieve little higher values when θ ranges from 0.1 to 1. In this paper, the parameter θ was set to be 0.5.

PSNR and SSIM values with different theta values in Example 1.

θ 0.0005 0.01 0.05 0.1 0.5 1 5
PSNR 26.56 26.50 26.48 26.70 26.74 26.77 26.69
SSIM 0.818 0.818 0.819 0.827 0.825 0.826 0.820

The parameters λ , β , γ , and μ were selected manually to provide the best PSNR results. We first fix the parameters λ , β , γ , and μ and adjust γ such that it gives the best restoration results measured by PSNR; in our implementation, we have found that γ = 180 is a relatively fair value in most cases for different images. The remaining parameters are tuned up one by one. This procedure is repeated several times for each parameter until they become stable. In this paper, we set parameter λ = 0.0008 . The parameters β and μ were chosen by trial and error tests. In all our experiments, the parameters used were β = 0.005 and μ = 0.003 . Although the better results could be obtained with “optimal” tuning of these parameters, it was found that these heuristic values achieved good results in most cases. In fact, the same parameter λ only needs to be tuned up slightly for different images by the same blur-kernel and corrupted by the same noise level; this significantly reduces the computational time of looking for good parameters.

4. Conclusion

In this paper, we proposed a semiblind deconvolution method with spatially adaptive TV regularization. The proposed method can automatically balance the regularization strength between different spatial property regions in the image. Also, the split Bregman method is used to optimize the nonsmooth model. Comparative results on simulated and real degraded images show that the proposed method can effectively reduce the noise in flat region as well as preserving the edge and detailed information.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research work is supported by Science and Technology Department of Jiangsu Province of China (no. BE2011747, no. BE2011822) and National Natural Science Foundation of China (no. 61105015). The authors would like to thank Dr. Peter R Coward of QinetiQ Malvern, Worcs., UK, for providing the real experiment data of PMMW image.

Banham M. R. Katsaggelos A. K. Digital image restoration IEEE Signal Processing Magazine 1997 14 2 24 41 10.1109/79.581363 2-s2.0-0031101825 Almeida M. S. C. Almeida L. B. Blind and semi-blind deblurring of natural images IEEE Transactions on Image Processing 2010 19 1 36 52 10.1109/TIP.2009.2031231 MR2729956 2-s2.0-72949095927 Rudin L. I. Osher S. Fatemi E. Nonlinear total variation based noise removal algorithms Physica D 1992 60 1–4 259 268 10.1016/0167-2789(92)90242-F 2-s2.0-44049111982 Chan T. F. Wong C. Total variation blind deconvolution IEEE Transactions on Image Processing 1998 7 3 370 375 10.1109/83.661187 2-s2.0-0032028980 Li W. Li Q. Gong W. Tang S. Total variation blind deconvolution employing split Bregman iteration Journal of Visual Communication and Image Representation 2012 23 3 409 417 10.1016/j.jvcir.2011.12.003 2-s2.0-84862799078 Pankajakshan P. Zhang B. Blanc-Féraud L. Kam Z. Olivo-Marin J. Zerubia J. Blind deconvolution for thin-layered confocal imaging Applied Optics 2009 48 22 4437 4448 10.1364/AO.48.004437 2-s2.0-68749108300 Bar L. Sochen N. Kiryati N. Semi-blind image restoration via Mumford-Shah regularization IEEE Transactions on Image Processing 2006 15 2 483 493 10.1109/TIP.2005.863120 2-s2.0-31144459533 Zhao W. Pope A. Image restoration under significant additive noise IEEE Signal Processing Letters 2007 14 6 401 404 10.1109/LSP.2006.887843 2-s2.0-34249778097 Yan L. Fang H. Zhong S. Blind image deconvolution with spatially adaptive total variation regularization Optics Letters 2012 37 14 2778 2780 10.1364/OL.37.002778 2-s2.0-84864332750 Tian H. Cai H. Lai J. H. Xu X. Effective image noise removal based on difference eigenvalue Proceedings of the 18th IEEE International Conference on Image Processing (ICIP '11) September 2011 Brussels, Belgium 3357 3360 10.1109/ICIP.2011.6116392 2-s2.0-84863049927 Chen Q. Montesinos P. Sun Q. S. Heng P. A. Xia D. S. Adaptive total variation denoising based on difference curvature Image and Vision Computing 2010 28 3 298 306 10.1016/j.imavis.2009.04.012 2-s2.0-70450233723 Chavent G. Kunisch K. Regularization of linear least squares problems by total bounded variation ESAIM. Control, Optimisation and Calculus of Variations 1997 2 359 376 10.1051/cocv:1997113 MR1483764 Goldstein T. Osher S. The split Bregman method for L 1 -regularized problems SIAM Journal on Imaging Sciences 2009 2 2 323 343 10.1137/080725891 MR2496060 Hong H. Li L. Zhang T. Blind restoration of real turbulence-degraded image with complicated backgrounds using anisotropic regularization Optics Communications 2012 285 24 4977 4986 10.1016/j.optcom.2012.07.080 2-s2.0-84867580552 Yujiri L. Shoucri M. Moffa P. Passive millimeter wave imaging IEEE Microwave Magazine 2003 4 3 39 50 10.1109/MMW.2003.1237476 2-s2.0-0142027894