A Method of Removing Reflected Highlight on Images Based on Polarimetric Imaging

A method of removing reflected highlight is proposed on polarimetric imaging. Polarization images (0, 45, 90, and 135) and the reflection angle are required in this reflected light removal algorithm. This method is based on the physical model of reflection and refraction, and no additional image processing algorithm is necessary in this algorithm. Compared to traditional polarization method with single polarizer, restricted observation angle of Brewster is not demanded and multiple reflection areas of different polarization orientations can be removed simultaneously. Experimental results, respectively, demonstrate the features of this reflected light removal algorithm, and it can be considered very suitable in polarization remote sensing.


Introduction
It is a common phenomenon that the mirror reflection of water surface produces the sun glitters; thus it heavily affects the image quality of remote sensing on the water surface and other mirror surfaces, such as ice or glass [1,2].The intensities of reflected light taken by the imagers are usually strong highlight and are much higher than the target intensities under or on the reflection surface, so we can hardly distinguish the effective information from the reflected highlight [3,4].
Over the past decades, several methods were proposed to reduce the influence of reflected highlight on images in the water quality remote sensing field.For the first method, we need to choose special time to take the remote images in order to avoid the appearance of sun glitters, but it may mismatch the essential observing time.The second method is a design of mechanical structure which can rotate the camera on the same target zone with the observing angle of forward, vertical, and backward, so we can get a final composite image without reflected highlight [5].However, changing scene or moving target is not adapted to this method.According to the Fresnel reflection theory, the glitters on mirror surface are commonly polarized, especially when the incidence angle equals the Brewster angle (normally the water's Brewster angle is about 53 ∘ ); the reflected glitters are completely linear polarization.So, for the third and mostly used method, we could use linear polarizer to get rid of polarized glitters with an observation angle at the water's Brewster angle [6].There are still some limitations of this method, such as the restricted demand of sunlight's incidence and imager's observing angles simultaneously matching water's Brewster angle and the limited ability of polarized glitter's removal at only one polarizing orientation.
Therefore, in contrast with the limitations of the traditional methods mentioned above, we proposed a novel method of removing reflected highlight on images based on polarimetric method which has more effective and adaptive performance by using the physical model of Fresnel's reflection and refraction formula and Stokes parameters.Particularly compared to the traditional polarizing way, the new method could disregard the restricted demand of Brewster angle and could remove glitters from multiple polarizing orientations from multiple reflection mirror surface.Our reflected highlight removal method could reserve the original characteristics of observed targets, so it can be easily 2 efficiently applied to the ocean remote sensing [7][8][9] and other polarimetric imaging fields [10,11].

Theoretical Background
2.1.Fresnel Formula and Stokes Parameters.The theoretical basis of the highlight removing method is based on the Fresnel reflection and refraction formula [12].In terms of surface reflection, for common situation, the refractive index of air and water, respectively, can be set as  1 and  2 .We need to focus on both the polarization properties of reflected light   and refractive light   in our method.The Fresnel reflection and fraction model is shown in Figure 1.
For the reflected light   , according to the Fresnel formula, we can get the reflected index of -wave component (the polarization direction perpendicular to the medium plane) and -wave component (parallel to the direction of polarization plane of the medium), respectively, as where  1 is the reflection angle, the same as the incidence angle, and  2 is the refraction angle.Meanwhile, we can easily get the -wave and -wave component of refracted light   , respectively, as To the polarized light, the basic definition of degree of linear polarization (DoLP) P is where  max and  min , respectively, represent the two special values of the maximum and minimum intensities measured at all polarization angles, which can be replaced by the -wave or -wave component to analyze the polarization characteristics of reflection and refraction phenomena.So taking ( 1)-( 4) into ( 5), we can calculate the respective polarization degrees of reflected and refracted lights.Considering the situation that both air and water have ideal refractive index of  1 = 1 and  2 = 1.33, we can get the simulated curves of polarization degrees of reflected and refracted lights, as shown in Figure 2. It can be clearly recognized that the reflected light has a more significant polarized feature than the refracted light.
Besides the basic definition of polarization degree, Stokes vector  = [ 0 ,  1 ,  2 ,  3 ]  is commonly used to represent the polarization characteristics of light [13], so we have another way to get the degree of polarization from the Stokes parameters.We can use the first three linear components  1 ,  2 , and  3 which are defined as where (0 ∘ ), (45) ∘ , (90 ∘ ), and (135 ∘ ), respectively, present the intensities taken by the imager at polarization orientations of 0 ∘ , 45 ∘ , 90 ∘ , and 135 ∘ .So the other expression of degree of linear polarization (DoLP) P by using Stokes parameters is given as  Figure 3, where the total image contains two parts.One part is the useful refracted light   from the light of target  target under the water, and the other part is the disturbance reflected light   from the incidence light of sun or other sources.For the mirror reflection, the incidence angle and the reflection angle have the same value of  1 .
So, the total light intensity  total received by the imager is Here, we use the basic definition of polarization degree to analyze the relations between  total and   .We can get the respective expressions of DoLP,  total and   , by using (1)-( 5) and their -wave and -wave components, shown as where  total  ,   , and   are the corresponding -wave components of total intensity, the reflected sunlight, and refracted target light and  total  ,   , and   are the corresponding wave components of them.
For common situations, the light source of sunlight and light of detected targets underwater are usually unpolarized.With the refraction index of air and water assumed to be  1 = 1 and  2 = 1.33, we can simulate the corresponding curves of DoLP of  total and   as shown in Figures 4(a) and 4(b).We can find that the DoLP of refraction of normal observing angel interval is so small that the refracted light can be usually regarded as unpolarized light.
According to the polarization characteristics and physical behaviors, we can note that refracted light of target has a much less polarization degree and a much weaker light intensity than the reflected sunlight.So a key approximate can be set up as Finally, we substitute (10) into (9) and use the results and DoLP calculated through Stokes parameters definition to get the ultimate expression of light intensity of target   , as where  total can be easily acquired through polarimetric imaging and Stokes parameters method, and the observing angle near Brewster angle is not required in our reflected highlight removal algorithm.Besides the polarimetric images, we only need to get the value of   , which is required and not difficult to measure the specific incidence or reflection angle and use the polarization characteristic in Figure 4(a).Also, we should note that the original target information is calculated by (11) in pixels, so the result of glitter removing can break the glitter removing limitation of single polarized orientation and recover the original information in pixels.
The efficiency of the highlight removing method on images will be demonstrated in the next experimental part.

Experimental Results and Discussion
Two experiments are conducted to verify this reflected highlight removing method.The first one is used to show the clear recovery result of our algorithm, and the other shows the effective result in removing highlight from multiple mirror surfaces with different reflected orientations.An 8-bit black and white industrial polarimetric CCD camera is used in our polarimetric imaging, which can take polarization image from different specified polarization orientations.Figure 5(a) shows a plastic plate filled with water, on whose ground three Chinese characters are written, and a table lamp is used as the light source in the first experiment.The mirror reflection obviously occurred on the water surface, and the actual scene is the same as the schematic diagram shown in Figure 3.With the incidence and observing angles being 34 ∘ which is far from the ideal Brewster angle 53 ∘ , the light of reflection is not completely linearly polarized.We will still get an obvious residual of reflection on our image in Figure 5(b) by using a single linear polarizer, and this limitation of the traditional polarimetric method could be noted clearly.
Then we use our method to remove this glitter.We snap four polarization images at the polarization orientations of 0 ∘ , 45 ∘ , 90 ∘ , and 135 ∘ from our polarization CCD camera, respectively.In this experiment, the measured reflection  For more details, we find that the removing result by our algorithm has weaker efficiency at the edge of reflected highlight than the center in Figure 6

Conclusions
In this paper, we demonstrate a new method of removing reflected highlight based on polarimetric imaging, and this algorithm can break the limitation of observing at Brewster angle and polarimetric method with single polarized orientation.We need to snap four polarization images at orientations of 0 ∘ , 45 ∘ , 90 ∘ , and 135 ∘ and measure the angle of reflection to fulfill the conditions of this reflected highlight removal algorithm.Two experiments of different scenes, respectively, demonstrate the advantages compared with traditional polarization methods.The first experiment clearly shows that our algorithm could effectively remove the reflected highlight without restricted observation angle of Brewster angle and reserve the original characteristics of observed targets.In the second experiment, a scene with glass of several car windows outdoors is used to verify the effect of removing multiple glitters with different polarized orientations.If we apply this reflected highlight removal algorithm to real-time polarized imaging, it might be very effective in remote sensing fields such as marine remote sensing and water quality monitoring.

Figure 1 :
Figure 1: The basic Fresnel reflection and refraction model.  and   , respectively, represent the reflected and refracted lights of incidence light  in .

Figure 2 :
Figure 2: The polarization degrees of reflected and refracted lights to the incidence angle.

Figure 3 :
Figure 3: The basic imager observing model with glitter on water surface, where   is the refracted light from target under water and   is the reflected light from sun or other light sources.

Figure 4 :
Figure 4: (a) and (b) are the simulated polarization characteristics of the reflected light   from sunlight and the transmission refracted light   from target in ideal condition, where  1 = 1 and  2 = 1.33.

Figure 5 :
Figure 5: (a) is the original image with highlight reflection on water surface and Chinese characters under water.(b) is the result of observation through a single polarizer.The observing angle mismatching the Brewster angle results in an obvious residual of reflected light on the image.
(b), because the reflection angle of glitter in our experiment is in a range near 34 ∘ , instead of a single numeric value.It can work better in the practical remote sensing with a large glitter area with uniform reflection angle and DoLP.To make a more distinct contrast of different methods, we put original image, highlight removed image by single polarizer, and highlight removed image by our algorithm together in Figures 7(a)-7(c) and then analyze the respective intensity histograms in Figures 7(d)-7(i).In Figure 7(a), the reflection highlights on the original image result in a certain amount of pixels with DN values over 150, where the DN value means the digital number of intensity.In Figures 7(b), 7(e), and 7(h), it could be clearly found that the intensity histograms compress and distort a lot because the intensities of unpolarized pixels would decline by nearly a half through the single polarizer, and there are still some reflection intensities with DN values over 60.Simply doubling the DN values for eliminating the compression will increase the highlight part and aggravate the reflection residual, and the distortion cannot be solved as well.

Figure 8 :
Figure 8: (a) is the image taken by the normal camera, and reflection occurs on the rear and side glass.(b) and (c) show clearly the results by single polarizer with polarized angle of 25 ∘ and 60 ∘ , and the reflection in the red and blue areas cannot be removed at the same time.(d) is the final reflected highlight removing result with the reflected highlight removal algorithm.