Candidate Smoke Region Segmentation of Fire Video Based on Rough Set Theory

,


Introduction
During one early fire, smoke is the main visual phenomenon of video surveillance image.Therefore, the research of smoke video recognition is particularly important to detect early fire.Most of smoke detection algorithms proceed as in the following steps: candidate smoke region segmentation, feature extraction, and smoke recognition.In this paper, we study the first step (i.e., candidate smoke region segmentation).At present, candidate smoke region segmentation methods generally can be divided into two categories: one is based on color features, such as RGB color space [1], HSV color space [2], and the combination of RGB color space and HSV color space [3][4][5].Another is based on color features and moving features.Kim et al. [6] got the counter of smoke region with two-dimensional wavelet transform.Qin and Ma [7] extracted smoke regions of interest by static saliency image and moving saliency image.Wu et al. [8] established a background model using mixed Gaussian model in RGB space and then suspected that smoke regions are detected by comparing the current frame with this reference model.However, the first kind of method cannot detect smoke of small area and low concentration because moving characteristics are not considered.And the methods cannot distinguish smoke from heavy fog and blue sky.
Rough set can realize the representation and inference of the uncertain knowledge [9].The theories of approximation representation and equivalence relation were widely used for image processing and pattern recognition [10][11][12].This paper presents a novel candidate smoke region segmentation method based on rough set theory. Figure 1 shows the basic process of the proposed scheme.Above all, video backgrounds are automatically updated based on three-frame difference approach to avoid the interference of static smoke-color objects such as blue sky.Afterwards, the upper approximation, lower approximation, and roughness of smoke-color distribution are represented using rough set theory in order to exclude moving regions of non-smokecolor objects, such as leaves and flowers.Finally, smoke regions are initially segmented by gray wave brand of , ,  basic color features.However, superfine partition can lead to small regions since excessive wave peaks are generated by initial segmentation.Thus these similar regions are merged based on equivalence relation of rough set theory.

Moving Pixels Detection
Kalman filtering is used to update video background [13].Let (, , ) and (, ,  + 1) denote the value of the pixel (, ) of th background image and ( + 1)th background image, respectively, and let (, , ) denote the value of the pixel (, ) of th frame; then the recursive formula is as follows: where (, , ) denotes gain factor and is computed using the following formula: (, , ) =  (1 −  (, , )) +  (, , ) where the parameters  and  are the moving factor and the background factor, respectively.The pixel (, ) of the th frame is marked as moving pixel if |(, , ) − (, , )| > Th 1 , and Th 1 is the threshold.The method based on Kalman filtering is a common method for moving pixel detection.The selection method of the parameters , , and Th 1 is described in [13].In this paper, the moving factor  is set to 0.001 <  < 0.01, and the background factor  is set to 0.01 <  < 0.1 [13].

Candidate Smoke Regions Segmentation
There are a mass of non-smoke-color moving objects such as trees and flowers in a surveillance video.In this section roughness measure approach is used to distinguish smokecolor moving regions from non-smoke-color moving regions.
Thus the pixels that meet Rules 1 and 2 will be detected as suspected smoke pixels.

Approximation Representation of Color Distribution.
In this section, we deal with pixels generated in suspected smoke pixels detection.That is, only suspected smoke pixels are checked in terms of rough set theory.Let the image  denote an RGB image of the size × and  denote gray level; then approximation representations of the image  can be defined based on rough set theory.The lower approximation of color distribution is defined as [11,14] where 80 ≤  ≤ 220 and  ∈ {,, }.(, , ) is the gray value of the pixel (, ) on , ,  component and (⋅) is impulse function.
The pixels that meet smoke color feature will be used to compute   ().According to the definitions, the lower approximation   () represents the number of smoke-color pixels whose gray level is ; thus   () is the certainty measure of gray distribution on , ,  component.
Let  ×  be the neighborhood of the pixel (, ) and the pixel (, ) ∈  × ; then the similarity of the pixels (, ) and (, ) is defined as [10,14] where ((, ), (, )) is the Euclidean distance of (, , ) and (, , ),  ∈ {,,}.The homogeneity degree function (, ) is defined as follows: The upper approximation of color distribution is defined as where 80 ≤  ≤ 220 and  ∈ {, , }, and   () represents the uncertainty measure of gray distribution of smoke-color pixels whose gray level is  on , ,  component.Obviously, the approximations   () and   () satisfy the properties  of rough set theory.Therefore, the roughness of smoke-color distribution is defined as [11,12] where 80 ≤  ≤ 220 and  ∈ {, , }.If the roughness   () is larger, the pixels of the gray level  locate in the regions of less pixel variety, that is, central region.Otherwise, the pixels of the gray level  locate in the edge of region.Figure 2 shows the roughness representation of one smoke video frame.

Candidate Smoke Region
Segmentation of Fire Video.The smoke region can be initially segmented by roughness of smoke-color distribution defined in Section 2. Then similarity of two small regions is defined on the basis of equivalence relation theory in order to merge similar regions.Then peak values of gray wave brand are selected as follows [12,14]:

Initial Segmentation of
where   = 10 is the threshold of peak width.The peak location can be gained on the basis of   .Wave valley location can be easily gained by computing the maximum between two adjacent peak values.Wave peaks and wave valleys come into being gray wave brands that are used to segment smoke region.The results of initial segmentation are shown in Figure 3.As seen from Figure 3, one smoke region is divided into some small clusters of smoke pixels.Smoke pixels of each cluster have similar roughness of smoke-color distribution.

Merging Small Smoke Region.
Although smoke region can be segmented using the proposed method, the method often results in superfine partition.Furthermore, the method cannot distinguish smoke images from heavy fog images.
Fortunately, smoke has one characteristic that is different from heavy fog.Smoke concentration gradually decreases from centre to edge of smoke region, which results in the fact that  component value of smoke region gradually reduces in HSV color space.Therefore, in this section small smoke regions are merged in HSV color space.Initial equivalence relation   of one small smoke region   is defined as where  is discourse domain and Let   and   be two initial equivalence relations, and then generalized equivalence relation is defined as where  We can exclude many smoke-color objects according to the characteristic that  component value of smoke pixels in HSV color space reduces from center to edge of smoke region.If one candidate smoke region is divided into 2∼3 clusters and the  component value of the smoke region decreases from center to edge, it is decided as smoke region.Otherwise, it is detected as non-smoke regions.

Experimental Result and Analysis
The proposed method of candidate smoke region segmentation is performed with an image size of 320 × 240.In the experiments, a total of 2,304 video frames from 16 fire video clips are used to verify the performance of the proposed method, including 326 positive video frames and 129 negative video frames.This section compares our method with other two methods: smoke regions detected based on RGB and HSV color space proposed in [3] (RGB-HSV for short) and smoke regions obtained using regions of interest extraction method based on vision saliency proposed in [7] (ROI for short).

Low Concentration Smoke Region Detection.
For one smoke image of low concentration, smoke region is difficultly segmented not only since it is very small, but also since the color difference between it and background is not obvious.For example, RGB-HSV method cannot detect the small smoke region of low concentration.Figure 5 shows the results of candidate smoke region extraction using our method and ROI method.As shown in Figure 5, our method and ROI method can effectively segment smoke region of low concentration.

Detecting Smoke Region of Images including Interference
Objects.Color features of some interference objects in smoke images are similar to smoke region; thus these objects may be falsely detected as smoke and result in the fact that smoke regions cannot be segmented.Figure 6 shows experimental results of candidate smoke region extraction using our method, RGB-HSV method, and ROI method.As shown in Figure 6, our method and ROI method can truly segment smoke region of images including interference objects such Figure 4: The results after merging small smoke region.
Original image ROI method Our method   as blue sky.RGB-HSV method falsely regards sky region as smoke.

Small Smoke
Journal of Electrical and Computer Engineering As shown in Figure 7, when 0.75 ≤  2 ≤ 0.85, both precision and recall are the highest.Small regions cannot be merged when the threshold  2 is too large.All the regions can be merged into one smoke cluster when the threshold  2 is too small.Therefore, too low threshold or too high threshold does not facilitate the segmentation of smoke regions using the characteristic that smoke concentration gradually decreases from centre to edge in HSV color space.

Excluding Heavy Fog Region.
Color features of dense fog are very similar to those of smoke, and fog pixels are moving pixels.Therefore, it is a difficult problem distinguishing smoke from fog. Figure 8 shows  component value variety of smoke region and heavy fog.As shown in Figure 8, the intensity of smoke region decreases from center to edge, and the intensity of fog pixels in one image is nearly the same.Therefore, we distinguish smoke from heavy fog using  component variety of smoke region.

4.3.
Performance of Smoke Region Extraction. Figure 9 shows average precision and recall of smoke region extraction using the three methods.It can be clearly seen from this figure that the proposed smoke region extraction using rough set theory outperforms the other two approaches in both precision and recall.Precision and recall of RGB-HSV method are the lowest since RGB-HSV method cannot detect some smoke images of low concentration including interference objects.Recall of ROI method is lower since ROI method falsely considers fog as smoke.

Conclusions
This paper presents a novel candidate smoke region extraction scheme of fire video, which adaptively updates video background and detects smoke regions in terms of roughness definition and equivalence relation definition of small smoke region using rough set theory.Experimental results show that the interference of static and moving smoke-color objects is excluded, and the proposed scheme outperforms the other methods in both precision and recall.
It is worth noting that although the performance of the proposed methods has been significantly improved, there are several potential works for future development.One is to amend the roughness of color distribution to effectively eliminate noise.Another is to adaptively select the threshold which contributes to merging small smoke region.

Figure 1 :
Figure 1: The basic process of the proposed scheme.

Figure 3 :
Figure 3: The results of initial segmentation ((a) and (b) show original image and the results of initial segmentation, resp.).

Figure 7 :
Figure 7: Precision and recall with variety of the threshold  2 .

Figure 8 :
Figure 8:  component value variety of smoke region and heavy fog.

Figure 9 :
Figure 9: Average precision and average recall of three methods.
Candidate Smoke Region.Smoke regions can be segmented by gray wave brands that are formed by wave peaks and valleys.Therefore, the selection of peak and valley is important to increase the performance of smoke region extraction.Let   be one peak of gray level  (80 ≤  ≤ 120); then the peak   will be reserved if it meets the condition   >   −   , where   and   are mean value and variance, respectively.Thus one new peak sequence {  1 ,   2 , . . .,    } is generated, where   denotes th gray level.
.  2 is the threshold of equivalence relation similarity.The selection of the threshold  2 is discussed in Section 4.2.1.
(  ,   ) is computed as follows:  (  ,   ) =      [  ] ∩ [  ]           [  ] ∪ [  ]      , Region Merging 4.2.1.Selection of the Threshold  2 .The threshold  2 is directly related to the performance of candidate smoke region detection.60positiveimages and 50 negative images are randomly selected from experimental dataset in order to select the threshold  2 .Figure7shows average precision and recall with variety of the threshold  2 .Precision and recall rates are computed as follows: