Visibility Restoration for Single Hazy Image Using Dual Prior Knowledge

Single image haze removal has been a challenging task due to its super ill-posed nature. In this paper, we propose a novel single image algorithm that improves the detail and color of such degraded images. More concretely, we redefine a more reliable atmospheric scatteringmodel (ASM) based on our previous work and the atmospheric point spread function (APSF). Further, by taking the haze density spatial feature into consideration, we design a scene-wise APSF kernel prediction mechanism to eliminate the multiplescattering effect. With the redefined ASM and designed APSF, combined with the existing prior knowledge, the complex dehazing problem can be subtly converted into one-dimensional searching problem, which allows us to directly obtain the scene transmission and thereby recover visually realistic results via the proposed ASM. Experimental results verify that our algorithm outperforms several state-of-the-art dehazing techniques in terms of robustness, effectiveness, and efficiency.


Introduction
Outdoor images captured in bad weather are prone to yield poor visibility, such as fade surface color, reduced contrast, and blurred scene details (see Figure 1(a)).Thus, removing such negative effects and recovering the realistic clear-day image have strong implications in computer vision applications, such as aerial imagery [1], image classification [2,3], remote sensing [4], and object recognition [5].
Directly employing traditional image processing techniques [6][7][8][9] to enhance the global contrast or local contrast of hazy image is probably the most intuitive and simplest way to recall the visibility of buried regions.But the dehazing effect of these techniques is somehow limited, since the degradation mechanism of hazy image is neglected.By means of the atmospheric scattering model (ASM), subsequent research works [10,11] mainly concentrate on using multiple images or external information to derive the depth map and the other unknown parameters.Although these techniques are able to achieve impressive results, the prerequisites of implementation may be difficult to fulfill in many practical cases.
Recently, single image dehazing has attracted the most research attentions, and some meaningful significant progress has been made by utilizing powerful image priors.However, these methods may fail under some particular situations.For instance, Fattal [12] presented a transmission estimation method based on the assumption that the surface shading is uncorrelated with the scene transmission in a local patch, whereas this method is unreliable for thick haze image because of the insufficient color information.Zhu et al. [13] proposed the color attenuation prior (CAP) for single image dehazing and create a linear model for modeling the scene depth under this prior, which allows us to directly obtain scene depth.But the constant scattering coefficient may lead to unstable recovery quality, especially for inhomogeneous hazy image.Yoon et al. [14] presented a spatially adaptive dehazing method that merges color histograms with consideration of the wavelength-dependent atmospheric turbidity.However, due to the insufficient wavelength information, this method is challenged when processing dense haze image.Benefiting from prevailing deep learning framework, Cai et al. [15] designed a trainable end-to-end system for the transmission estimation, but the dehazing effect is limited, which may be caused by the insufficient samples that are used to train this system.C. O. Ancuti and C.
Ancuti [16] first present a fusion-based dehazing strategy; the main advantage of this method is its linear complexity and it can be implemented in real time, yet the dehazing effect is highly dependent on the haze-feature weight maps.He et al. [17] discovered the dark channel prior (DCP) which is a kind of statistics of the haze-free outdoor images.With this prior and ASM, we can directly estimate the thickness of haze and thus recover visually pleasing result.Unfortunately, DCP approach may not well handle the brightness region where the brightness is inherently similar to the atmospheric light.
To deal with this problem, a hybrid DCP was proposed by Huang et al. [18], which also has the ability to circumvent halo effects and to avoid underexposure in the restoration output.Meanwhile, Chen and Huang [19] further developed a dual DCP that uses the big size dark channel and the small size dark channel in order to address the interference of localized light sources.Afterward, Chen et al. [20] provided the Bi-Histogram modification strategy to accurately adjust haze thickness in the transmission map (obtained using DCP) by combining the gamma correction and histogram equalization.In addition to above improved methods, some other techniques [21,22] were developed to boost the performance.Nevertheless, due to the inaccurate clustering methods, there seems to be no significant improvement on the accuracy of the estimated transmission map.Kim et al. [23] improved the visual clarity by maximizing the contrast information and minimizing information loss in the local patch.Furthermore, Meng et al. [24] proposed an image dehazing method by combining boundary constraint and contextual regularization.The method could exclude image noise and recover the interesting image details and structures.Despite the very great success for single image dehazing, as we discussed above, almost all the dehazing methods still have two common limitations.That is because the ASM is modeled under the assumption that the illumination is even distributed and the light is scattered only once during the propagation, and we will make more detailed explanation in Section 2.
In this paper, based on our previous work [25] and the multiple-scattering theory [26], we redefine a more robust ASM which is more in line with the real world.Further, based on this redefined ASM, a fast single image dehazing algorithm using dual prior knowledge is also proposed.In addition to the positive performance of the redefined ASM, the corresponding dehazing algorithm also has several advantages over the existing state-of-the-art techniques.First, due to the designed scene-wise APSF kernel prediction mechanism, our technique can well eliminate the multiple-scattering effect within each segmented scene.Second, by jointly utilizing the DCP and the maximizing contrast prior (MCP) [27], the ill-posed dehazing problem can be converted into a one-dimensional searching problem, which can effectively decrease the uncertainty of the dehazing problem.Experimental results, as shown in Figure 1, demonstrate that the proposed algorithm achieves better restoration of the edge contrast and color vividness compared with traditional image processing techniques including histogram equalization and the multiscale retinex [8].

The Redefined Atmospheric Scattering Model
In computer version and computational photography, the ASM proposed by Nayar and Narasimhan [10,11] is widely used to describe the degradation mechanism of hazy images, and it can be expressed as where (, ) is the pixel index, (, ) is the input hazy image,  ∞ is the atmospheric light which is usually assumed to be a constant [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24], (, ) is the scene albedo, and (, ) is the media transmission.In (1), the first term is named as direct attenuation, which describes how the scene radiance is attenuated by the suspended particles in the medium.The right term is called airlight, which demonstrates the influence of previously scattered light in the imaging procedure.
Although the ASM is physically valid under general conditions (see Figure 2(a)), two inner defects of this model still exist: (1) The constant- ∞ definition implies that the illuminance is even distributed throughout the entire image, and therefore ASM may not be in accordance with some of the real scenarios.(2) It is well known that the ASM is modeled under the single-scattering assumption; however, the light should be scattered multiple times by the suspended particles before reaching the camera, as can be seen in Figure 2(b).
Our previous work [25] has unveiled that the first defect can be addressed by redefining the constant atmospheric light  ∞ as the scene-wise variable, which varies spatially but is constant within each scene.To tackle the second limitation, in this paper we introduced the atmospheric point spread function (APSF) [28] to describe influence of multiple scattering and note that both the direct attenuation and airlight must be scattered multiple times during the propagation process.Consequently, a more robust ASM can be redefined as where Ω  is the th set of pixel index, * represents the convolution operator, ℎ(), (), and () are the scattering kernel, scene illuminance, and scene transmission, respectively, which are all fixed in the th index set.Obviously, the proposed ASM is more concise compared with the existing one, while the uneven-illumination problem and the multiple-scattering effect can be well physically explained by introducing scene-wise strategy and APSF.

The Proposed Algorithm
According to our previous work [25,29,30], we can segment a hazy image into  nonoverlapping scenes {Ω 1 , Ω 2 , . . ., Ω  } using -means clustering technique and further compute the corresponding haze density {(1), (2), . . ., ()} by assuming that the haze density is constant within each scene (default value of the number of scenes is  = 3).Now, the goal of dehazing is to estimate the APSF kernel, scene illuminance, and scene transmission for each scene, which is an ill-posed problem in nature.Fortunately, many image priors have been explored; this inspired us to address this issue by fully leveraging their latent statistical relationship.

A Scene-Wise APSF Kernel Prediction Mechanism. In [26],
Narasimhan and Nayar created an analytic model for the glow around a point source.However, owing to the deconvolution complexity, the convergence of the optimal solution is unreliable for all the optical thickness values.Similar to [28], we introduce the generalized Gaussian distribution model (GGDM) [31] to approximate the APSF, and the modified APSF for neighborhood can be modeled as where (, ) is the center index of neighborhood,  represents the forward scattering parameter which is correlated with the haze density,  = − ln() is the optical thickness,  is a constant that is correlated with  (default value is set as  = 0.2), Γ(⋅) is the gamma function, and (⋅) represents the scale parameter: As aforementioned, the haze density, illuminance, and transmission within each scene are approximately the same, and considering the similarity between  and  within each scene [26], we might as well assume that  =  for simplicity; thus the scattering kernel for each scene can be simplified as where  is the simplified APSF description and () and () are optical thickness and forward scattering parameter for the th scene, respectively.Accordingly, (2) can be rewritten as

Model Simplification via DCP.
To further alleviate the model complexity, we intend to leverage the relationship between the model parameters using DCP [17].Let us here recall the dark channel prior (DCP) which is a kind of statistics of the haze-free outdoor images: in most of the nonsky patches, at least one color channel has very low intensity at some pixels (even tends to be zero), and it can be described as where  is the color channel index and Ω(, ) is a local patch centered at (, ).Undoubtedly, the information contained in the th set of pixel index Ω  is more sufficient than that contained in Ω(, ), so DCP can also be applied to a scenewise strategy and become more robust.Formally, the dark channel of th scene  dark scene () can be defined as Having the scene-based DCP, we can take the min operation in each scene in (6); that is, where Putting ( 8) into ( 9), the scene illuminance () can be simply estimated as For more clarity, we substitute (11) where deconv is the deconvolution operator.Considering the constancy of transmission within each scene, the restoration function is equivalent to Since () can be determined by using our previous work [25], the scene albedo  is a function with respect to the scene transmission (), which means that the restoration function can be concisely expressed by

Scene Albedo Restoration Using MCP.
On the basis of the maximizing contrast prior (MCP) [27], the contrast for a clear-day image is higher than that of the corresponding hazy one.Therefore, we can estimate the transmission () by maximizing the contrast within each scene; that is, where ∇ is gradient operator.To obtain each scene transmission () accurately, we employ Fibonacci method [32] to solve this one-dimensional searching problem (15).If we remove the haze or fog thoroughly, the recovery results may seem unnatural.Thus, we optionally keep a small amount of haze into the dehazed image by adjusting t() =  ⋅ (), and we fix  = 1.1 in our experiments.Once the transmission map t has been determined, we can recover the haze-free image via (12).Figures 3(a)-3(c) provide two dehazing examples using the proposed technique.As we can see, due to the means clustering technique used in the segmentation process [25], the edge structure of the recovered images is not consistent with that of the input images.This is because image segmentation is inherently a scene-wise process that will blur the edges in the transmission map.Inspired by [25], we utilize the guided total variation (GTV) model to increase the edgeconsistency property, and therefore optimal t can be obtained by solving the following energy function: where guidance image  is the gray component of , edge weight  = 1 − exp(−|∇|), and  1 and  2 are regularization parameters (default setting is the same as [25]).As shown in Figures 3(d) and 3(e), the edge-inconsistency problem is well tackled after refinement, the transmission maps are more in line with our intuition, and the corresponding dehazing results are more natural and realistic.

Experiments
In order to verify the superiority of our dehazing technique, we test it on various types of hazy images and conduct the comparisons with several state-of-the-art dehazing methods, including Tarel and Hautiere [33], He et al. [17], Meng et al.
[24], Cai et al. [15], Zhu et al. [13], C. O. Ancuti and C. Ancuti [16], Choi et al. [34], and Wang et al. [35].All the methods are implemented in MATLAB 2010a on a personal computer with 2.6 GHz Intel(R) Core i5 processor and 8.0 GB RAM.The parameters in our approach are described in Section 3, and the parameters in the eight state-of-the-art dehazing methods are set to be optimal according to [13, 15-17, 24, 33-35] for fair comparison.

Qualitative Comparison.
As all the state-of-the-art dehazing methods are able to generate good results using general hazy images, it is difficult to rank all the dehazing methods visually.Therefore, we carry out these methods on the two benchmark images and five challenging hazy images, including dense haze image, the misty image, the hazy image with large sky region, the hazy image with large scale white/gray region, and the uneven-illumination hazy image.Figure 4 shows the qualitative comparison for different dehazing techniques on these challenging images.
The input hazy images are displayed in Figure 4 hazy image (the shadow regions in the seventh row of Figure 4 still look dim after restoration).In contrast, our technique successfully unveils the more scene details of the shadow regions.More importantly, our technique yields better visibility for all the experimental hazy images by eliminating the multiple-scattering effect, as shown in Figure 4(j).

Quantitative Comparison.
In order to perform quantitative evaluation and rate these algorithms, we adopt several widely used indicators, including the percentage of new visible edges , contrast restoration quality , Fog Aware Density Evaluator (FADE), Structural Similarity (SSIM), mean squares error (MSE), and color fidelity ().According to [36], indicator  measures the ratio of newly visible edges after restoration, and  is a metric that verifies the mean ratio of the gradients at visible edges.The indicator FADE is proposed by [34], which is an assessment of haze removal ability.SSIM [37] is used for evaluating the ability to maintain the edge structural information of the dehazing methods.
MSE indicates the average difference between the recovered result and the real haze-free image (or ground truth image).
[38] is used to measure the degree of color retention for the restored image.In general, higher values of , , and SSIM or the lower values of FADE, MSE, and  imply better visual improvement after restoration.Figure 5 shows the recovery results of different methods on the four synthetic hazy images which are synthesized based on the Stereo Datasets [39][40][41] and ASM.In   The computational complexity is another important evaluation factor for a dehazing method.In Table 1, we provide the time consumption comparison between Tarel and Hautiere [33], He et al. [17] (accelerated using the guided image filtering [42]), Meng et al. [24], Cai et al. [15], Zhu et al. [13], C. O. Ancuti and C. Ancuti [16], Choi et al. [34], Wang et al. [35], and our method for the images with different resolution.As we can see, the proposed technique is much faster than those of Tarel and Hautiere [33], Cai et al. [15], C. O. Ancuti and C. Ancuti [16], and He et al. [17,42].Although our method is slightly slower than Zhu et al. [13] and Wang et al. [35], the dehazing quality of our method is obviously better than that of Zhu et al. [13] and Wang et al. [35].In summary, the proposed technique has a better application prospect compared to the other methods, and we attribute these advantages to the improved ASM and the corresponding dehazing strategy.

Discussion and Conclusions
In this paper, by taking the uneven-illumination condition and the multiple-scattering effect into consideration, we have proposed a more robust ASM to overcome the inherent Mathematical Problems in Engineering limitation of the existing one.Based on this redefined ASM, a fast single image dehazing method has been further presented.In this method, we can effectively eliminate the multiple-scattering effect within each segmented scene via the proposed scene-wise APSF kernel prediction mechanism.Moreover, by sufficiently utilizing the existing prior knowledge, the dehazing problem can be subtly converted into a one-dimensional searching problem, which allows us to directly estimate the scene transmission and thereby recover a visually realistic result via the proposed ASM.The extensive experimental results have revealed the advance of the proposed technique compared with several state-of-theart alternatives.

Figure 1 :
Figure 1: Comparison with the traditional image processing techniques.(a) Hazy image.(b) The result of histogram equalization.(c) The result of multiscale retinex.(d) Our result.

Figure 2 :
Figure 2: Illustrations of single-scattering effect and multiple-scattering effect.(a) The atmospheric scattering model for single-scattering effect.(b) The atmospheric scattering model for multiple-scattering effect.

Figure 3 :
Figure 3: Comparison of dehazing quality before and after refinement via GTV.(a) Hazy images.(b) The transmission maps before edge refinement.(c) The recovered results using (b).(d) The transmission maps after edge refinement.(e) The recovered results using (d).

Figure 5 ,
from left to right, we show synthesized hazy images, the results of Tarel and Hautiere[33], He et al.[17], Meng et al.[24], Cai et al.[15], Zhu et al.[13], C. O. Ancuti and C. Ancuti[16], Choi et al.[34], Wang et al.[35], and the proposed method, as well as the ground truth images for fair comparison.The corresponding quantitative assessment results for the dehazing results shown in Figure5are depicted in Figure6.Analyzing the data of Figures6(a) and 6(b), our dehazing results achieve the top values of FADE and  for all the hazy images, which means that the images recovered using

Figure 5 :
Figure 5: Quantitative comparison of different methods on synthetic and real-world hazy images.From left to right: hazy images, Tarel and Hautiere's results, He et al. 's results, Meng et al. 's results, Cai et al. 's results, Zhu et al. 's results, C. O. Ancuti and C. Ancuti's results, Choi et al. 's results, Wang et al. 's results, and our results and the ground truth images.

Table 1 :
Time consumption comparison between Tarel and Hautiere, He et al., Meng et al., Cai et al., Zhu et al., C. O. Ancuti and C. Ancuti, Choi et al., Wang et al., and our method.Image resolutionTarel and Hautiere He et al.Meng et al.Cai et al.Zhu et al. C. O. Ancuti and C. Ancuti Choi et al.Wang et al.Ours