Visibility Video Detection with Dark Channel Prior on Highway

Dark channel prior (DCP) has advantages in image enhancement and image haze removal and is explored to detect highway visibility according to the physical relationship between transmittance and extinction coefficient. However, there are three major error sources in calculating transmittance. The first is that sky regions do not satisfy the assumptions of DCP algorithm. So the optimization algorithms combined with region growing and coefficient correction method are proposed. When extracting atmospheric brightness, different values lead to the second error.Therefore, according to different visibility conditions, amultimode classification method is designed. Image blocky effect causes the third error. Then guided image filtering is introduced to obtain accurate transmittance of each pixel of image. Next, according to the definition meteorological optical visual range and the relationship between transmittance and extinction coefficient of Lambert-Beer’s Law, accurate visibility value can be calculated. A comparative experimental system including visibility detector and video camera was set up to verify the accuracy of these optimization algorithms. Finally, a large number of highway section videos were selected to test the validity of DCP method in different models. The results indicate that these detection visibility methods are feasible and reliable for the smooth operation of highways.


Introduction
According to the road traffic accidents annual reports of China from 2007 to 2014, more than 50 percent of accidents and deaths are caused by low visibility which is below 200 meters [1].Particularly on highway, severe congestion and major accident can easily occur in low visibility conditions.Therefore, timely and accurate detection of low visibility and early warning response are important to ensure the smoothness of highway system operation.
In the present of expensive test equipment, one often can be used to obtain accurate visibility value nowadays.However, on Chinese highway, the displacement distance between these devices is very sparse, about 30 km, which results in limited detection range and failing to meet the demand in reality.In recent years, with the development of video surveillance technologies, using image processing methods to detect low visibility has attracted a vast amount of attention and obtained numerous extensive achievements.By reviewing other literatures, current video visibility detection algorithms contain four main fields: template matching, camera model calibration, dual differential luminance, and DCP method.
For template matching method, Robert et al. [2,3] comparatively analyzed the detected image and the images of which the visibility had been known in advance and then obtained the relative visibility.Bäumer et al. [4] used timeseries images and detected the farthest object to estimate meteorological visibility.Bronte et al. [5] defined three kinds of weather patterns including mist, fog, and heavy fog; moreover, they used camera projection method to estimate visibility based on the degree of fog.
For camera calibration model based on the contrast method, Steffens [6] photographed black object image and calculated visibility value according to the relative brightness ratio of object and its background.According to the contrast values of multiple targets in an image, Taek [7] fitted the nonlinear visibility curve and got daytime visibility.Babari et al. [8,9] presented a mapping model between image contrast and atmospheric visibility to estimate visibility base on the assumption that objects in the scene distributed continuously with the changing of distance.By selecting a virtual target     area in an image marked with distance information on the road, Chen et al. [10][11][12] obtained the corresponding function of spatial brightness contrast and visibility by using image matching and edge feature extraction.By extracting image feature parameters which could reflect visibility changes, An et al. [13] used the least squares method and inverse transform to find a fitting function between image features and distance to determine visibility value.
For camera model calibration method based on vanishing point, by using sky and road as references, Hautiére et al. [14] determined atmospheric extinction coefficient with camera model calibration and real-time graphics processing program and calculated atmospheric visibility through the Koschmieder's law.Li et al. [15] figured out the interested region by detecting lane boundaries and analyzed contrast values of each pixel and its neighbors.When the maximum contrast value exceeded the preset threshold, it would be considered as the distinguishable pixel by human operator.The maximum distinguishable pixel was called vanishing point; the distance between this point and camera was regarded as Mathematical Problems in Engineering the current visibility.By collecting the distinguishable pixels by human operator when the contrast value is greater than the threshold of luminance contrast, Li and Zhou [16] established a camera parameters model to calculate the distances between these points and camera.The maximum distance is visibility value.Based on region growing algorithm, Zhang et al. [17] figured out the feature points in precise road region.The contrast curve of these points could reflect road luminance variation.Moreover, they found the maximum distinguishable pixels to calculate the maximum visibility distance.
For dual differential luminance algorithm, by setting two targets with different distances from observation station, Lv et al. [18] deduced daytime visibility according to the ratio of two measured luminance differences between targets and their horizon sky background.Chang et al. [19] calculated daylight visibility by dual differential luminance algorithm and gained night time visibility with double light sources.These algorithms could solve the problem that precision was affected by uneven distribution of sky brightness.Zhao et al. [20] proposed optimization dual brightness difference algorithm with correction coefficient and multimode division method, and she developed a special visibility detection system.
Based on statistical analysis of outdoor fog-free images, the concept of DCP was put forward by He et al. [21].In the vast majority of local area of outdoor fog-free images, at least one color channel intensity value of some pixels is very low.When their minimum intensity values are close to zero, these pixels are known as the dark pixels.The DCP algorithm is widely used in image enhancement and image defogging for calculating the transmittance [22].Guo et al. [23] obtained transmittance from building to camera by using DCP and guided filter.And then, they calculated extinction coefficient based on Lambert-Beer law and got visibility according to the relationship between transmittance and extinction coefficient.
Comparative analysis of the advantages and disadvantages of the method described above is as follows: (1) The principle of template matching method is simple.However, this method requires a lot of images calibrated with visibility information as a matching database and is not suitable for practical applications.
(2) Using camera model calibration method, one can directly consider road as object to determine the extinction coefficient through camera model calibration and real-time graphics processing procedures.
Then, the atmospheric visibility can be calculated with Koschmieder's law.The advantages of this method are low cost, high stability, and high precision for moving vehicles and fixed cameras.However, selfcalibration model is complicated and easily disturbed by weather and environment.Also, this method is difficult to work at night and cannot meet the demand for all-weather real-time detection.
(3) Dual brightness difference method has great advantages in night visibility detection and meets the demand for all-weather visibility detection with better reliability, but the detection conditions of this method are high.Therefore, the additional artificial target should be set up.So it is not insufficient for highway application.
(4) DCP method combines the advantages of digital imaging method and the transmittance method.The distinguishing features are simple principle and low cost.However it is still in theoretical stage and requires higher accuracy during the calculation of transmittance.
On the majority of Chinese highways, a video surveillance camera is implemented every 2 Km along the side of the shoulder.These devices are used to monitor traffic flow and traffic environment and can provide a wealth of video resources for the development of video detection of low visibility.Taking into account the advantages of DCP method, we attempt to apply this method to detect highway visibility.According to assumptions that there are some dark pixels in local area of image, DCP calculates the transmittance.However, when this method is applied directly to the highway   visibility detection, large errors are produced due to the complexity of real highway condition.Therefore, an optimized DCP algorithm is presented which combines region growing, correction coefficient and multimode detection method to obtain more accurate transmittance.

Visibility Detection Principle of Dark Channel Prior Method
The digital image is defined as follows in computer vision and digital picture processing: where  is image intensity,  is actual scene intensity,  is atmosphere light intensity,  is the pixel of digital image, and  is transmittance describing the proportion of the light that is reflected from an object and transmitted directly to camera without scattering.

Theoretical Basis of Dark Channel Prior.
According to the literature [23], the corresponding dark channel  dark () of where  is an arbitrary channel in RGB color space,   () is pixel value of channel  at pixel , and Ω() is a local patch of which center pixel is .
Based on DCP, the transmittance of each pixel can be calculated.Main steps are shown as follows.(4) Step 3. Without considering DCP, based on the atmosphere light regularity of   , t() can be described as formula Step 4. According to DCP, formula (4) can be presented as follows: min Step 5.The estimated transmittance t() of each pixel in an image is obtained as formula

Visibility Detection Principle with Dark Channel Prior
Algorithm.The essence of DCP algorithm is to calculate the transmittance of image pixel, and based on the physical relationship between transmittance and extinction coefficient, the latter can be obtained; therefore, the visibility value is carried out easily.Bouguer-Lambert law describes the relationship between incident light flux Φ 0 and transmittance light flux Φ  , under the condition of a beam of light source transmits from a certain distance .Thus, mathematical relationship between atmospheric transmittance and meteorological optical visual range can be deduced.This law provides a theoretical basis for visibility detection with the method of DCP.
Bouguer-Lambert law is expressed as follows: where  is the transmittance with light propagation distance  and  is the atmospheric extinction coefficient.The transmittance can be derived as follows: According to the definition of  in meteorological optical range, select  = 0.05 for Bouguer-Lambert law; there is 0.05 =  − . (10) Thus, the visibility  can be obtained: It is worth noting that we should choose the objects which contain dark channel when using DCP to detect visibility.Then the extinction coefficient can be deduced based on the accurate transmittance of this region.When the transmittance is 0.05, we can calculate the meteorological optical range, namely, the atmospheric visibility.
As shown in Figure 1, video camera, variable message sign, and traffic sign are generally installed along the side of Chinese highway.Firstly, a fixed marker, such as signs, is selected as the target in the camera image collection range.Secondly, then based on DCP, camera images are used to detect region visibility.Thirdly, this visibility information will be displayed on the upstream variable message sign.Thus, the drives in the downstream section can be able to perceive the visibility information in advance.The right device in Figure 1 is the visibility detector whose data will be used to compare with the detection results of DCP method in this paper.

Error Analysis and Optimization of Dark Channel Prior.
Because of the unpredictable environment in different highway segments, when DCP method is directly used to detect visibility, great number of errors will be generated.With preliminary analysis, the main errors come from sky area transmittance calculation [24], different atmosphere brightness values [25], and image block effect [21].Therefore, we need to make the appropriate optimization algorithm for different error sources as our first step.

Error Analysis and Optimization in Sky Region.
Based on the assumption that there are dark pixels in local image region, transmittance can be calculated by DCP algorithm.Video camera images can be divided into sky region and nonsky region.However, as the sky region does not conform to this assumption, a greater deviation in the calculation of transmittance will be generated [24].
When dark channel is not considered, the actual transmittance is calculated by formula (5).In the regions without dark channel, such as sky region, there is min ∈{,,} (min ∈Ω()  c ()/  ) > 0. So the actual transmittance is larger than the estimated one calculated by formula (7).In contrast, in the dark channel area, as min ∈{,,} (min ∈Ω()  c ()/  ) is close to zero, the difference between actual transmittances and estimated one is very small.
Analyzing pixel characteristics of the sky region, the gray value changes are small and different from that of other pixels of nonsky region.By researching literatures [26][27][28], region growing method gathers the pixels based on similar characteristics in the same pixel area.One region starts from an initial pixel.And these pixels with same characteristics of the initial pixel will be incorporated into with this region.So this region gradually grows until there are no other pixels can be merged.Thus, during the growth process, region growing method requires a group of proper seed pixel in the desired area and precise guidelines for merging adjacent pixels.When seed pixel growth and appropriate guidelines are selected, region growing method has obvious advantages in computation time and segmentation accuracy.Therefore, we use region growing method to divide image into sky region and nonsky region.
(1) Seed Selection.The seed selection directly influences the effect of region growing.In the image, the brightness of sky   ())) . ( (2) Growth Rules Based on Region Gray Scale Difference.
According to the analysis of normal gray images, sky region has relatively higher brightness than nonsky region.The gray scale difference is set as growth criterion.When the gray scale difference between seed pixel and its neighbor is less than a predetermined threshold value, the latter will be merged into sky region.
where th1 is grayscale threshold in the gray image and th2 is grayscale threshold in the dark channel image.Due to the fact that white cloud regions do not meet the growth rules of sky and have no dark channels, th3 is used as threshold to distinguish these regions.Therefore, the pixels of these regions meet formula (14) and will be merged into sky region: min ∈{,,} ( min According to literatures [26][27][28], both ranges of th1 and th2 are from 0 to 0.1.The range of th3 is from 0.5 to 1.In this paper, based on a large number of video image sample tests, there are th1 = th2 = 0.5 and th3 = 0.75.By using the method described above, original image and regional segmented image are shown in Figure 2. Next, two correction coefficients  and   are introduced to realize accurate transmittance calculation in sky region, where  = 1/(1 − min ∈{,,} (min ∈Ω()   ()/  )) and   is the correction factor for atmospheric brightness.In nonsky region, considering the existence of dark channels, we directly use a correction parameter   to revise transmittance.Transmittance corrections are calculated as follows:   an inaccuracy in transmittance calculation [29].Therefore, based on the visibility level cited from the standard of "Monitoring of Visibility and Warning of Heavy Fog on Highway" [30], the visibility detection is divided into five modes as shown in Table 1.
For each mode, the difference between estimated visibility and actual visibility is considered as the objective function.By using the least square method, different correction coefficients of transmittance calculation are derived for sky region and nonsky region.Test results In sky region, the objective function and transmittance correction function in mode  are established as formula

Low visibility condition Middle visibility condition High visibility condition
in which   is actual visibility.(  ()) is estimated visibility when the transmittance is   ().  and    stand for correction parameters of sky region in mode , respectively.In nonsky region, the objective function model and transmittance correction function in mode  are set up as formula where    is the correction parameter of nonsky region in model .

Transmittance Optimization Based on Guided Filtering.
When the sizes of local region Ω() are different, the dark channel images are different.The larger the patch is, the greater the potential dark channel is, and the more obvious the block effect will be.However, significant block effect influences the accurate calculation of transmittance.As shown in Figure 3, in the dark channel image, block effect of 9 × 9 patches is more obvious than that of 3 × 3 patches.Guided filtering theory was proposed by He et al. [31] and applied to eliminate block effect of transmittance calculation in DCP method.This theory is based on local linear model.It assumes that image is a two-dimensional function, and the output and input in a two-dimensional window have a linear relation as formula where   is the value output pixel,   is the value of guidance image, and  and  are pixel indexes.  is a window which includes pixel  and  is the center of   .  and   are two coefficients of linear function in   .
In order to obtain the linear function coefficients, the following cost function is used: where  is a regularization parameter.Linear regression is used to minimize the difference between output value  and actual value .The linear function coefficients are obtained as follows: where   and  2  are the mean and variance of  in   , || is the number of pixels in   , and   is the mean of  in   .
A pixel may be contained in many windows; that is, each pixel can be described by a number of linear functions.Therefore, when calculating the linear coefficients of each window, the output value of a certain pixel is the average of all the linear functions which contains this point.There is Therefore, when Figure 3(a) is processed according to the transmittance correction functions ( 16) and ( 17), Figure 4(a) can be obtained.After Figure 4(a) is processed with guided filtering optimization method, Figure 4(b) can be obtained.Comparisons of results of estimated transmittance are shown in Figure 4.After optimization, the block effect caused by the size of window can be eliminated, and the edge feature information is well-preserved.

Visibility Detection Processes Based on Dark Channel
Prior.In summary, images captured by highway video surveillance systems are colored; the first step is to convert these images to dark channel images, which will be segmented into sky region and nonsky region based on region growing method.Secondly, in accordance with the appropriate detection mode, different factors are selected for correction.In order to improve the accuracy in transmittance calculation, guided filtering is used to eliminate block effect.Then, atmospheric brightness and target transmittance are extracted.Finally, the Lambert law is utilized to calculate extinction coefficient and visibility value.

Precision Analysis of Visibility Detection
Based on Dark Channel Prior

Contrast Test Analysis of Visibility Detection.
To verify the accuracy of the aforementioned detection algorithms, as shown in Figure 5, a comparative test system is built based on the traffic test site in Beijing, China [32].This test system includes a visibility detector, a video camera, and a traffic sign of which distance is 105 meters away from camera.The visibility detector model is VAISALA PWD 12 of which range is from 10 to 2000 meters.And this detector can not only measure accurate visibility but also display the reason for visibility decreasing.Due to the fact that traffic sign regions contain dark channel, this region is selected as the target.In addition, due to lack of light, the current highway camera images cannot be used to detect visibility at night.A light source is installed near by the traffic sign.
Based on the video image database captured from 01/09/2016 to 05/30/2016, an image was intercepted for analysis every ten minutes, and all the parameters of DCP algorithm were carefully verified and validated.By using the daytime image captured on 03/18/2016, the visibility detection process of DCP is shown as Figure 6.The test system image is displayed in Figure 6(a).When the local patch size was set to 3 × 3, the dark channel image is shown in Figure 6(b).Estimated transmittance map before optimization is shown in Figure 6(c).Estimated transmittance map after optimization is shown in Figure 6(d).The visibility detection value is 956 meters.Simultaneously, the visibility detector value is 974 meters.The relative error is 1.85%.
By using the nighttime image captured on 03/10/2016, the visibility detection process of DCP is shown as Figure 7.
The test system image is displayed in Figure 7(a).When the local patch size was set to 3 × 3, the dark channel image is shown in Figure 7(b).Estimated transmittance map before optimization is shown in Figure 7(c).Estimated transmittance map after optimization is shown in Figure 7(d).The visibility detection value is 732 meters.Simultaneously, the visibility detector value is 765 meters.The relative error is 4.31%.
One-day video image sequence was selected on 01/10/2016 as a validation.The visibility comparison results and relative errors of test system are shown in Figures 8 and  9. From two figures, the visibility results of optimization algorithm are consistent with that of the visibility detector in both trend and the numerical value.The relative error range is from 0% to 12.3% which meets the demands of highway visibility detection.

Verification of Dark Channel Prior Algorithm under
Five Models.To further verify the effectiveness of the DCP algorithm, a large number of video images, captured by the camera in different locations, were selected to verify the validation of dark channel optimization algorithm under five detection models.
(  In a certain visibility condition, the actual visibility varied slowly in a short time interval.From Figure 27, visibility is relatively stable in six minutes which is consistent with visibility change rule.Corresponding to the original images, visual visibilities of three cases are from low to high sequentially.Detection visibility by DCP optimization algorithm has the same trend.The test results show that this detecting method and system were feasible.

Verification of Dark Channel Prior Algorithm under a
Mountain Highway.The video image sequences were selected from Xuanda Highway in Hebei province, China.This is a mountain highway.As shown in Figure 28, the times of images are 13:07:17, 13:07:23, 13:26:14, and 13:57:45 on 01/25/2015.Because of the low visibility and snowy weather, an accident occurred during this time.The cabin area of the red truck is selected as target area which is 42 meters away from the camera.The dark channel images are shown in Figure 29.The estimated transmittance maps before optimization and after optimization are separately shown in Figures 30 and 31.
The eliminating degree of block effect in Figure 31 is better than the one in Figure 30.From the original images of Figure 28, fog gradually diminished over time, and the visual visibility changes from low to high.As shown in Figure 32, the detection visibility values based on DCP optimization algorithm are gradually increasing and fully agree with the actual situation.

Conclusions
Low visibility is a major factor of causing of traffic accidents, which often result in a heavy loss of human life and financial properties.The accurate detection of visibility and timely warning play important roles in assuring the highway operation smoothness.In order to make full use of the camera image resources on highways, after summarizing the advantages and disadvantages of template matching method, camera calibration method, and dual differential luminance algorithm, a new method is elucidated to detect highway visibility by using the method of DCP.
The region growing method is used to divide the sky and the nonsky region.Due to the fact that sky regions do not meet the assumptions of DCP method, different correction factors are adopted to decrease the deviations derived from transmittance calculation.According to different visibility conditions, the multimode division method is designed, which can reduce the errors from extracting atmospheric brightness values.Guided filtering method is used to eliminate the block effect and to obtain the accurate transmittance of pixels in those images.Thus, based on the theoretical relationship between transmittance and extinction coefficient in Lambert-Beer law, the accurate visibility values can be calculated.
For verifying the feasibility, a comparative system was developed to test the visibility detection accuracy of optimization algorithm.Also, a large number of highway video images were used to verify the validation of this proposed optimization algorithm under five models.The verification results of video image sequences of tunnel and mountain highway indicate that the detection values are reliable and consistent with the actual situation.Based on a number of experiments, these optimization algorithms can well satisfy the general requirements of low visibility detection on the highway.Meanwhile, we found from those researches that it still needs further analysis to select the target area precisely and automatically in complex highway environments.

Figure 6 :Figure 7 :
Figure 6: Daytime visibility detection based on comparative test system.(a) The test system image.(b) Dark channel image of test system.(c) Estimated transmittance maps of test system before optimization.(d) Estimated transmittance maps of test system after optimization.

Figure 10 :Figure 11 :
Figure 10: Verification results under the first model.(a) The original image.(b) Dark channel image of the first model.(c) Estimated transmittance maps of the first model before optimization.(d) Estimated transmittance maps of the first model after optimization.

Figure 12 :Figure 13 :
Figure 12: Verification results under the third model.(a) The original image.(b) Dark channel image of the third model.(c) Estimated transmittance maps of the third model before optimization.(d) Estimated transmittance maps of the third model after optimization.

Figure 14 :
Figure 14: Verification results under the fifth model.(a) The original image.(b) Dark channel image of the fifth model.(c) Estimated transmittance maps of the fifth model before optimization.(d) Estimated transmittance maps of the fifth model after optimization.

Figure 27 :
Figure 27: Visibility detection results of the tunnel.

Table 1 :
Visibility grade of highway and pattern classification.
1)The Verification Results under the First Model.The original image came from Jinliwen Highway of Zhejiang province, China.The time is 5:00 p.m. on 12/05/2013.The traffic sign was selected as target which is 72 meters away from the camera.The visibility detection process of the first model is shown in Figure10.The visibility detection value is 769.05 meters which are consistent with the day weather record of traffic monitoring center.
(3) The Verification Results under the Third Model.The original image came from Rongwu Highway of Shandong province, China.The time is 10:15 a.m. on 02/01/2014.The bridge was selected as target which is 109 meters away from the camera.The visibility detection process of the third model is shown in Figure12.The visibility detection value is 149.96meters which are consistent with the day weather record of traffic monitoring center.3.3.Verification of Dark ChannelPrior Algorithm under the Typical Tunnel.The original images were selected from Wangwu Tunnel of Henan province, China.Images were