Forest Fire Smoke Video Detection Using Spatiotemporal and Dynamic Texture Features

. Smoke detection is a very key part of fire recognition in a forest fire surveillance video since the smoke produced by forest fires is visible much before the flames. The performance of smoke video detection algorithm is often influenced by some smoke-like objects such as heavy fog. This paper presents a novel forest fire smoke video detection based on spatiotemporal features and dynamic texture features. At first, Kalman filtering is used to segment candidate smoke regions. Then, candidate smoke region is divided into small blocks. Spatiotemporal energy feature of each block is extracted by computing the energy features of its 8-neighboring blocks in the current frame and its two adjacent frames. Flutter direction angle is computed by analyzing the centroid motion of the segmented regions in one candidate smoke video clip. Local Binary Motion Pattern (LBMP) is used to define dynamic texture features of smoke videos. Finally, smoke video is recognized by Adaboost algorithm. The experimental results show that the proposed method can effectively detect smoke image recorded from different scenes.


Introduction
Fires are a constant threat to forest ecological systems and human safety; moreover, forest fires are an important problem in regions which present hot climate.With the development of computer vision techniques, forest fire video surveillance has been one of research focuses in the field of forest fire prevention.Generally, methods for detecting fire video can be categorized as flame detection and smoke detection.However, most of the fire video systems are mainly designed for smoke detection, since the appearance of smoke is in most cases more visible than the fire itself.Smoke detection algorithms are divided into systems based on single and based on multiple frames.In the first case, smoke images are recognized by color [1,2], texture [3,4], and energy [5].In the second case, smoke features are extracted by video sequence.For example, Favorskaya and Levtin [6] extracted spatiotemporal features of smoke video by grouping moving regions with a turbulence parameter connecting with fractal properties of smoke in order to track effectively a smoke propagation.In [7], wavelet transform is used to detect high frequency information of moving pixels so that smoke flickering features are analyzed.Interesting work was presented by Wang et al. [8] who firstly detected motion regions from video frames, and then four flutter features of the motion regions are extracted over a sliding time window, including the flutter direction and three types of flutter intensities.
Smoke detection systems have made some achievements but cannot be used as a self-sufficient solution.They often have a high false rate and thus need an additional human confirmation for final decision.For bringing the performance of the detection systems closer to the results that could currently be obtained by human observers, this paper presents a novel forest fire smoke video detection based on spatiotemporal energy and dynamic texture features.First of all, candidate smoke regions are segmented and then are divided into small blocks.Afterwards, for each block, three features including two spatiotemporal features and one dynamic texture feature are extracted.Finally, Adaboost algorithm is used to recognize smoke video clips.Figure 1 shows the basic process of the proposed scheme.

Spatiotemporal Feature Extraction
Kalman filtering is firstly used to update video background [8] in order to detect motion regions.And then three features A smoke block is detected as candidate smoke block according to the following condition: where Δ 2 represents the mean of the square difference Δ 2 of the previous  − 1 frames.For a smoke block, its spatially and temporally neighboring blocks have greater possibility of being smoke blocks.We count the number of neighboring smoke blocks of the block   of the video frame at the time : where  cand () denote the number of 8-neighboring smoke blocks at the corresponding position   of the video frame at the time . Figure 2 shows the calculation process of the spatiotemporal consistency energy.

Flutter
Feature.As we know, there is massive heavy fog in the forest, so heavy fog can easily cause false alarm.

Nonsmoke block Smoke block
Current block ith previous frame i = t − 10, t − 9, . . ., t Fortunately, forest fire smoke has one characteristic that is different from heavy fog.Smoke moves from bottom to top because heat smoke has a lower density than air, which is significantly different from heavy fog.The motion direction of one block is mapped as a direction code by computing the centroid motion of one candidate smoke block.The  +  order moment of one candidate smoke block is computed by the following formula: where   and   denote starting and ending point of one smoke region width, respectively, and   and   denote starting and ending point of the region height.The centroid coordinates of one candidate smoke block are defined as where Flutter direction angle is defined as () , otherwise.The classical definition of LBP can be represented as follows: where () corresponds to the gray value of the pixel at position  and {(V  )} =1,..., to gray values of  equally spaced pixels {V  } =1,..., on a circle of radius  with the center at position .LBMP uses the basic Local Binary Pattern (LBP) to extract both dynamic and appearance features of dynamic texture of candidate smoke region.We choose one search window of the size 5 × 5.   represents the search window in the current frame   , and  0 is its central pixel.Let LBP  ( 0 ) denote the LBP descriptor of the pixel  0 computed by (8); LBP +1 (  ) denote the LBP descriptor of one arbitrary pixel   in the search window  +1 of adjacent frame  +1 .The matching point can be found by the formula where  ranges from 1 to 25 due to one search window of the size 5 × 5.For one block of the size 8 × 8, the dynamic texture is defined as where (LBMP(, , ), ) is Kronecker's delta that is defined as 3. An Adaboost Approach for Classification where  is the number of blocks and 1 ≤  ≤  and   () represents moving direction MC or energy lowering ratio ELR or color feature CF of the th block.

Adaboost Algorithm.
Adaboost is superior to tradition neural networks on learning abilities and is applied in many fields of image processing, such as car license plate detection and face recognition.Adaboost algorithm is a training procedure for a collection of weak classifiers [10].If the weak classifiers have the success rate about 0.5, they are boosted by suitable voting process to obtain a strong classifier.Because the collection of fire smoke videos is relatively difficult, the number of negative samples is much more than positive [11].
Step 1.The weight of the th sample   is initialized by formula (14): Step 2.Then, the weight of weak predictor is computed as follows: where   denotes the sum of predictive errors.
Step 3. Supposing   (),  = 1, 2, . . ., , denote the predictive sequence, the iterative formula of the weight of training sample is computed by where   denote the normalization factor.
Step 4. Let  denote the number of weak predictors.All the weak classifiers form a strong classifier using the following formula: (16)

Experimental Results
The proposed method is tested on 23 video clips of varying length and scene, including 13 positive samples and 10 negative samples, which are generally processed around 20 fps. 12 video clips (7 positive videos and 5 negative videos) are randomly chosen for training; the remaining 11 clips are used for testing.One part of the dataset is publicly available at website http://imagelab.ing.unimore.it/visor/and another part of dataset is recorded by ourselves.4.1.Segmentation of Smoke Regions.Smoke region segmentation is key part for extracting effective features of smoke region.Figure 3 shows the results of smoke region segmentation based on Kalman filter.In Figure 3, red points denote the centroids of smoke regions.

Centroid Motion of Smoke Regions.
Figure 4 shows centroid motion direction of smoke regions in one video clip.
As we see, smoke often moves from bottom to top, which is significantly different from heavy fog.the results of the video that includes obviously moving objects., respectively.To validate the performance of our method, this section compares the proposed smoke detection method with other two methods: smoke detection using image energy and color information in [5] (EN-CI for short) and smoke detection method based on mixed Gaussian model and wavelet transformation in [12] (MGM-WT for short).We evaluate the performance of smoke detection method by

Performance of Smoke
Figure 6 shows the performance of the three methods, including the proposed one, in terms of TPR and TNR, where different datasets were used.Both TPR and TNR of our  method are higher than the other two methods, as shown in Figure 6.It indicates that dynamic features extracted by the proposed method are effective in discriminating between smoke and nonsmoke video.

Conclusions
In this paper, a novel smoke detection scheme using spatiotemporal and dynamic texture features is proposed.Three dynamic features, spatiotemporal energy, flutter features, and dynamic texture feature, are extracted for recognizing forest fire smoke video effectively and efficiently.The experimental results show the proposed approach provides higher accuracy of wildfire flame detection with comparable computational time.It is worth noting that there are several potential works for future development.One is to improve background model to effectively detect candidate smoke region in smokelike scene; another is to extract more effective features for completely excluding the interference of heavy fog.

Figure 2 :
Figure 2: The calculation process of the spatiotemporal consistency energy.

( 7 ) 2 . 3 .
Dynamic Texture Feature.The Local Binary Pattern (LBP) is a powerful means of texture description.The operator labels the pixels of an image region by thresholding the neighborhood of each pixel with the center value and considering the result as a binary number (binary pattern).

Figure 3 :
Figure 3: The results of smoke region segmentation.

Figure 4 :
Figure 4: Centroid motion direction of smoke regions.

Figure 5 Figure 5 :
Figure 5: The results of several frames in three smoke videos from different scenes.
computing true positive rate (TPR) and true negative rate TNR which are, respectively, defined by TPR = Number of TP frames Number of TP frames + Number of FN frames , TNR = Number of TN frames Number of TN frames + Number of FP frames .
TNR of negative testing videos

Figure 6 :
Figure 6: Experimental results of different smoke detection methods in terms of TPR and TNR of each testing video.
=  rate ,  rate denotes one frame rate.  (, ) and   (, ) denote horizontal ordinate and vertical ordinate of the centroid of the th candidate smoke block in th time window.The moving direction angle of the th candidate smoke block