The denoising and deblurring of Poisson images are opposite inverse problems. Single image deblurring methods are sensitive to image noise. A single noise filter can effectively remove noise in advance, but it also damages blurred information. To simultaneously solve the denoising and deblurring of Poissonian images better, we learn the implicit deep image prior from a single degraded image and use the denoiser as a regularization term to constrain the latent clear image. Combined with the explicit
The process of image acquisition in electron microscope imaging, astronomical imaging, and medical imaging is inevitably affected by environmental factors, which often cause the captured image to be disturbed by Poisson noise and blur degradation. The image degradation process can be modeled as follows:
Poisson image denoising and blurring is an image deconvolution problem. Image noise can be divided into three categories: additive noise, multiplicative noise, and Poisson noise. The logarithm of the multiplicative noise can be processed as additive noise of Gaussian distribution. Different from the first two kinds of noise, Poisson noise generally appears in the case of very small illuminance and amplification with high power electronic circuits and obeys a Poisson distribution. Using variance stabilizing transformation [
Generally, images corrupted by Poisson noise are accompanied by blur degradation. Therefore, the deblurring problem of Poisson images is not only a denoising problem but also a deblurring problem. In the process of image restoration, the prior information of degraded image plays a very important role [
With the success of deep networks in image processing, an increasing number of scholars have begun to use neural networks to solve the restoration problem of degraded images [
The regularization term plays an important role in image deconvolution. Compared with other regularization terms, RED [
Based on the observation of blurred images [
In [
In equation (
To solve the inseparable Poisson log-likelihood problem in equation (
To avoid the differentiation of the explicit denoising function, two auxiliary variables,
According to the split Bregman method, seven unknown variables
To solve equation (
Given fixed variables
The update of variable
Given fixed variables
The update of variable
Given fixed variables
We use gradient descent to update
Given fixed variables
Given fixed variables
Given fixed variable
Algorithm
Input: degraded image Initialization: While Update Update Update Update Update Update Update End Output: recovered clear image
In this section, we carry out experiments and image quality evaluation on simulated blurred images and real blurred images that are corrupted with Poisson noise and compare the experimental results with those of four methods: PID-Split [
In this section, we compare the proposed method with the other four algorithms (PID-Split, PIDSB-FA, PIDSB-NLFA, and DeepRED) on five images with simulated Poisson image denoising and blurring experiments. The five images are Lena (size = 256 × 256), Cameraman (size = 256 × 256), Butterfly (size = 256 × 256), Zebra (size = 584 × 387), and House (size = 256 × 256), as shown in Figure
Original images. (a) Lena (size = 256 × 256). (b) Cameraman (size = 256 × 256). (c) Butterfly (size = 256 × 256). (d) Zebra (size = 584 × 387). (e) House (size = 256 × 256).
In the simulated Poisson image experiment, we applied different levels of blur and noise to different images. The Gaussian blur kernel parameters we set for the Lena image of Figure
In the experiment of deblurring simulated Poisson images with different noise levels and different blur kernel sizes, the image quality evaluation data recovered by the PID-Split, PIDSB-FA, PIDSB-NLFA, and DeepRED algorithms and the proposed method in this paper are shown in Tables
PSNR (dB) values of the five methods.
Image | Gaussian blur | Degraded image | PID-Split | PIDSB-FA | PIDSB-NLFA | DeepRED | Our method | |
---|---|---|---|---|---|---|---|---|
Lena | Size = 5, | 3,000 | 24.84 | 25.06 | 25.22 | 25.57 | 25.80 | |
4,000 | 24.87 | 25.07 | 25.25 | 25.62 | 25.82 | |||
5,000 | 24.90 | 25.11 | 25.29 | 25.68 | 25.85 | |||
Size = 15, | 3,000 | 24.36 | 25.01 | 25.12 | 25.26 | 25.81 | ||
4,000 | 24.38 | 25.03 | 25.16 | 25.28 | 25.92 | |||
5,000 | 24.40 | 25.07 | 25.20 | 25.29 | 26.09 | |||
Size = 25, | 3,000 | 22.99 | 23.70 | 24.44 | 24.55 | 24.57 | ||
4,000 | 23.00 | 23.72 | 24.49 | 24.57 | 24.59 | |||
5,000 | 23.02 | 23.75 | 24.51 | 24.59 | 24.61 | |||
Cameraman | Size = 5, | 3,000 | 23.63 | 25.38 | 26.68 | 26.73 | 27.12 | |
4,000 | 23.66 | 25.51 | 26.73 | 26.75 | 27.20 | |||
5,000 | 23.67 | 25.69 | 26.85 | 26.92 | 27.40 | |||
Size = 15, | 3,000 | 21.73 | 23.39 | 24.02 | 24.09 | 24.24 | ||
4,000 | 21.75 | 23.43 | 24.15 | 24.17 | 24.38 | |||
5,000 | 21.76 | 23.48 | 24.20 | 24.23 | 24.47 | |||
Size = 25, | 3,000 | 22.83 | 24.39 | 25.29 | 25.37 | 25.51 | ||
4,000 | 22.84 | 24.49 | 25.32 | 25.40 | 25.74 | |||
5,000 | 22.88 | 24.66 | 25.35 | 25.42 | 25.92 | |||
Butterfly | Size = 5, | 3,000 | 22.88 | 24.50 | 25.23 | 25.52 | 25.56 | |
4,000 | 22.90 | 24.63 | 25.26 | 25.55 | 25.62 | |||
5,000 | 22.91 | 24.78 | 25.27 | 25.58 | 25.66 | |||
Size = 15, | 2,000 | 20.57 | 21.96 | 22.99 | 23.29 | 23.47 | ||
3,000 | 20.60 | 22.04 | 23.03 | 23.47 | 23.59 | |||
4,000 | 20.63 | 22.22 | 23.07 | 23.57 | 23.69 | |||
Size = 25, | 3,000 | 21.81 | 23.17 | 23.96 | 24.33 | 24.46 | ||
4,000 | 21.83 | 23.33 | 23.99 | 24.36 | 24.52 | |||
5,000 | 21.84 | 23.48 | 24.00 | 24.39 | 24.59 | |||
Zebra | Size = 5, | 3,000 | 23.58 | 24.81 | 25.89 | 26.50 | 26.87 | |
4,000 | 23.61 | 24.86 | 25.96 | 26.56 | 26.96 | |||
5,000 | 23.64 | 24.92 | 26.02 | 26.62 | 27.05 | |||
Size = 15, | 3,000 | 21.69 | 23.95 | 25.05 | 25.32 | 26.38 | ||
4,000 | 21.71 | 24.03 | 25.12 | 25.43 | 26.57 | |||
5,000 | 21.73 | 24.11 | 25.20 | 25.54 | 26.69 | |||
Size = 25, | 3,000 | 22.68 | 24.41 | 25.51 | 26.04 | 26.78 | ||
4,000 | 22.71 | 24.47 | 25.57 | 26.17 | 26.90 | |||
5,000 | 22.73 | 24.54 | 25.65 | 26.29 | 27.03 | |||
House | Size = 5, | 3,000 | 28.81 | 30.08 | 30.71 | 30.43 | 32.44 | |
4,000 | 28.93 | 30.17 | 30.91 | 30.48 | 32.54 | |||
5,000 | 28.99 | 30.28 | 31.10 | 30.60 | 32.61 | |||
Size = 15, | 3,000 | 26.11 | 27.12 | 27.69 | 27.52 | 28.36 | ||
4,000 | 26.14 | 27.20 | 27.70 | 27.57 | 28.37 | |||
5,000 | 26.18 | 27.33 | 27.73 | 27.62 | 28.39 | |||
Size = 25, | 3,000 | 27.71 | 28.70 | 29.61 | 29.41 | 30.29 | ||
4,000 | 27.84 | 28.78 | 29.73 | 29.49 | 30.47 | |||
5,000 | 27.85 | 28.87 | 29.81 | 29.56 | 30.56 |
VIF values of the five methods.
Image | Gaussian blur | Degraded image | PID-Split | PIDSB-FA | PIDSB-NLFA | DeepRED | Our method | |
---|---|---|---|---|---|---|---|---|
Lena | Size = 5, | 3,000 | 0.4482 | 0.4310 | 0.4513 | 0.5147 | 0.5182 | |
4,000 | 0.4553 | 0.4327 | 0.4559 | 0.5122 | 0.5131 | |||
5,000 | 0.4603 | 0.4369 | 0.4614 | 0.5147 | 0.5136 | |||
Size = 15, | 3,000 | 0.4026 | 0.4307 | 0.4298 | 0.4938 | 0.5178 | ||
4,000 | 0.4074 | 0.4338 | 0.4346 | 0.4942 | 0.5282 | |||
5,000 | 0.4116 | 0.4393 | 0.4404 | 0.4946 | 0.5410 | |||
Size = 25, | 3,000 | 0.3484 | 0.3946 | 0.4761 | 0.4949 | 0.5130 | ||
4,000 | 0.3531 | 0.3973 | 0.4812 | 0.4962 | 0.5116 | |||
5,000 | 0.3566 | 0.4003 | 0.4842 | 0.4969 | 0.5290 | |||
Cameraman | Size = 5, | 3,000 | 0.3086 | 0.4045 | 0.4332 | 0.4336 | 0.4513 | |
4,000 | 0.3107 | 0.4133 | 0.4436 | 0.4395 | 0.4563 | |||
5,000 | 0.3140 | 0.4233 | 0.4450 | 0.4458 | 0.4573 | |||
Size = 15, | 3,000 | 0.2255 | 0.3132 | 0.3260 | 0.3860 | 0.3743 | ||
4,000 | 0.2280 | 0.3166 | 0.3347 | 0.3867 | 0.3890 | |||
5,000 | 0.2301 | 0.3195 | 0.3372 | 0.3861 | 0.3950 | |||
Size = 25, | 3,000 | 0.2705 | 0.3730 | 0.3854 | 0.3860 | 0.3996 | ||
4,000 | 0.2733 | 0.3805 | 0.3894 | 0.3876 | 0.4040 | |||
5,000 | 0.2760 | 0.3910 | 0.3903 | 0.3861 | 0.4185 | |||
Butterfly | Size = 5, | 3,000 | 0.3940 | 0.4709 | 0.5535 | 0.5597 | 0.5768 | |
4,000 | 0.3967 | 0.4815 | 0.5577 | 0.5614 | 0.5909 | |||
5,000 | 0.3982 | 0.4933 | 0.5602 | 0.5635 | 0.6074 | |||
Size = 15, | 2,000 | 0.2925 | 0.3818 | 0.4622 | 0.4717 | 0.5018 | ||
3,000 | 0.2957 | 0.3877 | 0.4661 | 0.4849 | 0.5228 | |||
4,000 | 0.2991 | 0.4002 | 0.4713 | 0.4967 | 0.5335 | |||
Size = 25, | 3,000 | 0.3471 | 0.4119 | 0.5117 | 0.5292 | 0.5478 | ||
4,000 | 0.3492 | 0.4189 | 0.5142 | 0.5325 | 0.5488 | |||
5,000 | 0.3507 | 0.4263 | 0.5163 | 0.5358 | 0.5660 | |||
Zebra | Size = 5, | 3,000 | 0.3913 | 0.4007 | 0.4905 | 0.5192 | 0.4913 | |
4,000 | 0.3979 | 0.4038 | 0.4957 | 0.5278 | 0.5106 | |||
5,000 | 0.4020 | 0.4070 | .04992 | 0.5109 | 0.5179 | |||
Size = 15, | 3,000 | 0.2832 | 0.3348 | 0.3982 | 0.4094 | 0.4295 | ||
4,000 | 0.2878 | 0.3387 | 0.4026 | 0.4155 | 0.4394 | |||
5,000 | 0.2911 | 0.3429 | 0.4071 | 0.4219 | 0.4448 | |||
Size = 25, | 3,000 | 0.3402 | 0.3697 | 0.4066 | 0.4712 | 0.4763 | ||
4,000 | 0.3463 | 0.3731 | 0.4100 | 0.4789 | 0.4850 | |||
5,000 | 0.3504 | 0.3768 | 0.4136 | 0.4857 | 0.4959 | |||
House | Size = 5, | 3,000 | 0.3842 | 0.4028 | 0.4472 | 0.5040 | 0.5062 | |
4,000 | 0.3898 | 0.4081 | 0.4563 | 0.5049 | 0.5103 | |||
5,000 | 0.3924 | 0.4127 | 0.4659 | 0.5047 | 0.5133 | |||
Size = 15, | 3,000 | 0.2914 | 0.3620 | 0.4378 | 04357 | 0.4334 | ||
4,000 | 0.2929 | 0.3688 | 0.4380 | 0.4373 | 0.4370 | |||
5,000 | 0.2946 | 0.3753 | 0.4404 | 0.4375 | 0.4389 | |||
Size = 25, | 3,000 | 0.3429 | 0.3743 | 0.4653 | 0.4719 | 0.4522 | ||
4,000 | 0.3496 | 0.3783 | 0.4741 | 0.4724 | 0.4686 | |||
5,000 | 0.3495 | 0.3823 | 0.4752 | 0.4663 | 0.4703 |
Restoration of the House image. (a) Ground truth image. (b) Degraded image blurred by Gaussian kernel of size 15 × 15,
Figure
Restoration of the Zebra image. (a) Ground truth image. (b) Degraded image blurred by Gaussian kernel of size 25 × 25,
In addition, the Butterfly image in Figure
Restoration of the Butterfly image. (a) Ground truth image. (b) Degraded image blurred by Gaussian kernel of size 5 × 5,
The explicit regularization prior term adopts the combination of the intensity-based
PSNR values and loss comparison. (a) PSNR. (b) PSNE (zoomed in). (c) Loss. (d) Loss (zoomed in).
This section is an experimental comparison of denoising and deblurring of real Poisson images. Tests were carried out on four real astronomically degraded images. To evaluate the real Poisson image quality, we used the Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) to evaluate the real data. The smaller the value of BRISQUE, the better the image quality. The image quality evaluation data recovered by the PID-Split, PIDSB-FA, PIDSB-NLFA, and DeepRED algorithms and the proposed method in this paper are shown in Table
BRISQUE values of the five methods.
Image | PID-Split | PIDSB-FA | PIDSB-NLFA | DeepRED | Our method |
---|---|---|---|---|---|
Saturn | 45.13 | 43.31 | 48.96 | 41.42 | |
Moon | 49.81 | 43.34 | 47.75 | 39.28 | |
Docking | 48.65 | 49.14 | 49.05 | 41.95 | |
Phoebe | 36.61 | 32.94 | 31.26 | 30.44 |
Figure
Restoration of real blurred image. (a) Saturn truth image. (b) Restored image by the PID-Split method. (c) Restored image by the PIDSB-FA method. (d) Restored image by the PIDSB-NLFA method. (e) Restored image by the DeepRED method. (f) Restored image by the proposed method. (g) Close-up views (close-up views of (a)–(f) image regions extracted from this example, respectively).
The second real image is a moon image, as shown in Figure
Restoration of real blurred image. (a) Moon truth image. (b) Restored image by the PID-Split method. (c) Restored image by the PIDSB-FA method. (d) Restored image by the PIDSB-NLFA method. (e) Restored image by the DeepRED method. (f) Restored image by the proposed method. (g) Close-up views (close-up views of (a)–(f) image regions extracted from this example, respectively).
Figure
Restoration of real blurred image. (a) The docking truth image of Shenzhou-9 and Tiangong-1. (b) Restored image by the PID-Split method. (c) Restored image by the PIDSB-FA method. (d) Restored image by the PIDSB-NLFA method. (e) Restored image by the DeepRED method. (f) Restored image by the proposed method. (g) Close-up views (close-up views of (a)–(f) image regions extracted from this example, respectively).
Restoration of real blurred image. (a) Phoebe truth image. (b) Restored image by the PID-Split method. (c) Restored image by the PIDSB-FA method. (d) Restored image by the PIDSB-NLFA method. (e) Restored image by the DeepRED method. (f) Restored image by the proposed method. (g) Close-up views (close-up views of (a)–(f) image regions extracted from this example, respectively).
This paper proposed an algorithm for denoising and deblurring Poisson images by using neural networks. We combine an implicit regularization prior with two explicit regularization priors that are the prior of RED and the prior of
The data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This work was supported by a project of the National Science Foundation of China (61701353, 61801337, and 61671337).