Multifeature Fusion Vehicle Detection Algorithm Based on Choquet Integral

Vision-based multivehicle detection plays an important role in Forward Collision Warning Systems (FCWS) and Blind Spot Detection Systems (BSDS). The performance of these systems depends on the real-time capability, accuracy, and robustness of vehicle detection methods. To improve the accuracy of vehicle detection algorithm, we propose a multifeature fusion vehicle detection algorithm based on Choquet integral. This algorithm divides the vehicle detection problem into two phases: feature similarity measure and multifeature fusion. In the feature similarity measure phase, we first propose a taillight-based vehicle detectionmethod, and then vehicle taillight feature similaritymeasure is defined. Second, combiningwith the definition of Choquet integral, the vehicle symmetry similarity measure and the HOG + AdaBoost feature similarity measure are defined. Finally, these three features are fused together by Choquet integral. Being evaluated on public test collections and our own test images, the experimental results show that our method has achieved effective and robust multivehicle detection in complicated environments. Ourmethod can not only improve the detection rate but also reduce the false alarm rate, whichmeets the engineering requirements of Advanced Driving Assistance Systems (ADAS).


Introduction
As an important part of the intelligent transportation system (ITS), the Advanced Driving Assistance Systems (ADAS) can significantly improve the driving safety.Forward Collision Warning Systems (FCWS) and Blind Spot Detection Systems (BSDS) are principal portions of ADAS, and their performance depends on the real-time capability, accuracy, and robustness of the vehicle detection method.Recently, with the increasing maturity of visual sensors, vision-based vehicle detection has become a hot topic in the field of intelligent vehicle.There are plenty of approaches proposed for the day time vehicle detection.These methods can be divided into the following categories: methods based on prior knowledge, such as shadow-based [1,2], taillight-based [1,2], horizontal (vertical) edge-based [2][3][4], and symmetrybased vehicle detection method [2]; methods based on stereo vision; this type of method detects vehicles by using the three-dimensional information.The most widely used methods are inverse perspective transformation (IPM) based method and disparity map based method [1]; template-based methods use predefined patterns of vehicle class and perform correlation between the image and the template [1]; the main detection steps of appearance-based methods are as follows: the appropriate descriptors are first used for representing vehicles in the image; then the machine learning methods are used to train these descriptors.Much processes have been made in appearance-based vehicle detection, such as algorithm based on HOG + AdaBoost [5], Haar + HMM [6], Haar + AdaBoost [7][8][9], HOG + SVM [10], PCA -ICA + GMM [11], and minimum Mahalanobis distance classifier [12].The method based on the motion information detects vehicles by using the motion information between vehicles and scenes, such as finding out vehicles by calculating the change of optical flow information which is caused by the relative motion of vehicles or scenes [13].

Journal of Applied Mathematics
To improve the accuracy of vehicle detection methods, several of above methods are combined together to detect vehicles.Lin et al. [3] applied the SURF and edge features to represent the vehicle and, combining with probabilistic methods, their methods have achieved vehicle detection in the blind spot area.Chen et al. [6] first used a road modelling method to confine detection regions, and then Haar-like features and eigencolours were used for detecting vehicles.Finally, a tackling method was used.Tehrani Niknejad et al. [10] proposed a deformable vehicle model based on HOG feature; the method can achieve the adaptive threshold vehicle detection under urban roads.Wang and Lien [11] proposed a vehicle detection method based on a statistical model of local feature.They applied the public dataset Caltech Cars (Rear) [16] to test their method.Alonso et al. [12] proposed a vehicle detection method based on multidimensional classification.They represented vehicles in form of rectangular subregions based on the robust classification of features vectors result of a combination of multiple morphological vehicle features.Their method can detect vehicles with very different aspectratio, color, and size.Chang and Cho [8] presented a vehicle detection algorithm based on combination of Haar feature and online boosting.Their algorithm has realized vehicle detection in various environments.Sivaraman and Trivedi [9] proposed an active-learning framework based on Haar feature and AdaBoost for vehicle detection on the highway.Jazayeri et al. [13] proposed an optical flow and hidden Markov model-based vehicle detection method which mode the locations and motion information of vehicles in the image level.Their method can deal with the vehicle identification problem under the scene of changing illumination and environment well.
Vehicle detection method based on a single feature can quickly detect vehicles in images.However, using single feature method may cause a lot of false alarms, because single feature only describes one certain characteristic of vehicles.Most of the appearance-based vehicle detection methods' performance excessively depends on the number and scale of training samples.Various samples in different situations are needed to generate more powerful classifiers.In addition, detecting vehicles in images using appearance-based methods which has to scan the whole image requires excessive calculation and cannot meet the real-time requirement of FCW.To solve the above-mentioned problem, a widely used method is multifeature fusion which combines several single feature-based algorithms together by using voting method.This can significantly reduce the false alarm rate, but the detection rate is reduced either.In recent years, mathematical theory has been widely used for improving the performance of complex vehicle systems.Much process has been made in the field of mathematical modeling and control methods [17][18][19][20][21][22][23], such as adaptive back stepping control for active suspension systems with hard constraints [17], saturated adaptive robust control for active suspension systems [18], and adaptive robust vibration control of full-car active suspensions with electrohydraulic actuators [19].Choquet integral is a widely used method in data fusion [24][25][26]; it can seek the maximum consistency of decision from the consistency and conflict detection results of multiple features.To improve the performance of vehicle detection algorithm and to solve problems above, we propose a multifeature fusion vehicle detection algorithm based on Choquet integral.Experiment results show that our multifeature fusion method will not only improve the detection rate but also reduce the false alarm rate.
Figure 1 illustrates the workflow of our approach.The rest of the paper is organized as follows.Section 2 briefly introduces the shadow-based vehicle region of interest (ROI) detection method.In Section 3, vehicle taillight feature similarity measure, vehicle symmetry feature similarity measure, and HOG + AdaBoost feature similarity measure are presented, respectively.Then our multifeature fusion vehicle detection algorithm based on Choquet integral is introduced in Section 4. Experiment results for the proposed method are shown in Section 5; finally Section 6 draws conclusions.

Shadow-Based Vehicle ROI Detection
The shadow-based vehicle detection algorithm is usually applied to extract the vehicle ROIs in the whole images for reducing computation complexity [1].We have developed a shadow-based vehicle detection method, and the basic principle of the method is that regions underneath vehicles are distinctly darker than any other regions on an asphalt road.The grayscale of pixels in shadow regions is much lower than that in any other regions in the same image.Grayscale histogram (GH) can reflect the whole image grayscale distribution well.The grayscale of vehicle shadow pixels belongs to the lower parts of GH.So we can detect the shadow regions underneath vehicles by segmenting GH with a threshold th BW. Figure 2(a) is a vehicle image from Caltech Cars (Rear) [27].Black regions in Figure 2

Feature Similarity Measure
To make full use of the Choquet integral in our multifeature fusion vehicle detection framework, each single feature should be first represented in form of fuzzification before calculating the Choquet integral.After this phase, the algorithm can fuzz the output of each single feature; then the result can be determined by using the fuzzy judgment instead of direct judgment.Therefore, in this section, we detailedly introduce three feature-based vehicle detection methods and their feature similarity measure functions.
The distance between each MCR is represented as (3).The two taillights of vehicle are usually on a horizontal line, and the thresholds th  and th  can get rid of the straight lines that are not horizontal or almost horizontal: otherwise. ( Definition 1.The taillight feature similarity measure function  tailCoeff is defined as 3.2.Vehicle Symmetry Feature Similarity Measure.The symmetry measure is a statistic to describe the symmetry of target.Vehicles are obviously symmetrical objects; therefore, we use the symmetry feature as a similarity measure in our algorithm.According to the symmetry-based method described in [28], we use the symmetry measure method based on normalized entropy to calculate the symmetry value of each vehicle ROI.The symmetry measure is described as (5), where (  ) is the symmetry measure of target.() is the information entropy, which is also the mathematical expectation of information content.  is the max value of information entropy.Consider ( Definition 2. The symmetry feature similarity measure function  symCoeff is defined as

HOG and AdaBoost Classifier Feature Similarity Measure.
The histogram of oriented gradient (HOG) is a descriptor of feature which has been widely used in object detection.Zhu et al. [29] introduced an efficient pedestrian detection method based on HOG and AdaBoost.In our previous work, we use HOG feature to detect pedestrian [14].The HOG feature is represented by calculating the histogram of oriented gradient of local region in the image.First, the image is divided into a plurality of grids according to a certain size; these grids are called BLOCK which are illustrated as in Figure 5(a).Then each BLOCK is divided into four regions which are called CELL.Each CELL projects an orientation-based histogram which includes nine bins.In this histogram, the horizontal ordinate is a range of direction angles which divide 180 ∘ into nine equal parts, and the vertical coordinates are an accumulation of each angle range.Finally, a 36D feature vector named BLOCK is formed.Due to the strong edge feature of vehicles, we employ the HOG feature to represent vehicles; then the AdaBoost-based algorithm [30] is applied to generate weak classifiers.In this paper, the training samples of generating HOG + AdaBoost classifiers are images captured from actual driving environments.Vehicle regions of these images are positive samples, and other regions of images are negative samples.The amount of positive samples and negative samples are both 10000.These samples are normalized to the same size (30 × 30).Screenshots of samples are shown in Figures 5(b) and 5(c).There are two phases to employ the HOG + AdaBoost classifier and the training and the detection phase.In the training phase, we extract HOG features by applying CELL size of 5 × 5, 10 × 10, and 15 × 15, respectively; the scanning step size is three pixels, and the weak classifiers are selected by AdaBoost algorithm.After training, we use the samples (positive 10000 and negative 10000) which are different from that of the training phase to test the weak classifiers.The ROC curves of HOG + AdaBoost algorithm under three different CELL sizes are illustrated as in Figure 6; the performance of HOG + AdaBoost classifiers whose CELL size is 15 × 15 is the best among these three types; therefore, we set the CELL size to 15 × 15 in our further experiments.
To enhance the performance of HOG + AdaBoost classifiers, inspired by method in [9], the active-learning based HOG + AdaBoost framework is used by following the steps in the Active-Learning Framework.The advantage of this framework is that you are only adding negative samples that would otherwise be causing false positives.There is no point in adding more negative samples that are handled by the original training anyways.
Active-Learning Framework.
Step 1. Train HOG + AdaBoost classifiers using the 10000 positive samples and 30000 negative samples.
Step 2. Run the algorithm by using well-trained HOG + AdaBoost classifiers on a large video set (not the training set from Step 1).
Step 3. Any false positives from the run in Step 2 can be put in the negative set.
Step 4. Retrain the algorithm using the original true positive set and the updated negative set (negatives from both Step 1 and Step 3).
Step 5.This can be repeated as many times as appropriate, using new video on each iteration.the symbol of the hogadbCoeff is positive or negative.This way is not suitable to employ the HOG and AdaBoost-based classifiers in our Choquet integral-based multifeature fusion vehicle detection framework.To represent the hogadbCoeff in form of probability, we first test the well-trained classifiers  by using the testing sample set which is different from the training sample set.And then the statistic distribution of hogadbCoeff is calculated.Finally the mapping table between the HOG and AdaBoost classifier feature similarity measure value  hogadbCoeff and the hogadbCoeff is formed.The statistic distributions of hogadbCoeff belonging to vehicles and interferences are illustrated in Figure 7; we use the algorithm precision + corresponding to interval of hogadbCoeff to be the  hogadbCoeff ; the precision is defined as (16) in this paper; the mapping table is created as Table 1.
Definition 3. The HOG + AdaBoost classifier feature similarity measure function  hogadbCoeff is defined as Table 1.

Multifeature Fusion Vehicle Detection Algorithm Based on Choquet Integral
In this paper, fuzzy integral theory is applied to vehicle detection in complex scenarios.First, the basic theory of Choquet integral is introduced here.And then the fuzzy measure of each feature is defined.Finally, the features of taillight, symmetry, and HOG + AdaBoost classifier are fused by Choquet integral of fuzzy theory.The brief concepts of Choquet integral and the fuzzy measure used in our algorithm are followed from the concepts in [24][25][26].
Definition 4. Let  be a finite set, and  is a power set which is composed of subsets of ,  :  → [0, ∞] is the mapping function from the power set  to the range of [0, ∞].If  satisfies the following three conditions,  is a fuzzy measure on .
(3) Continuity: if ∀  ∈ , and {  |  ∈ [1, +∞]} is monotonous.This is also represented in the form of The fuzzy measure which is widely applied in multifeature fusion is the regular fuzzy measure: if  ∈  and () = 1, the fuzzy measure  is regular.Definition 5.If the fuzzy measure satisfies the following conditions: ∀,  ∈ ,  ∩  = , if there exists a constant value ,  > −1 satisfying (∪) = ()+()+()(), then  is a -fuzzy measure. can be calculated by (8), where   = ({  }); it is used to indicate the importance of a single feature classifier for the final evaluation, where   ∈  = { 1 ,  2 , . . .,   }.Consider Definition 6.  :  → [0, 1] is a nonnegative function defined on ,  is a fuzzy measure defined on power set , and then Choquet integral of function  on  with respect to fuzzy measure is defined by where   = { | () ≥ ,  ∈ },  ∈ [0, ∞); the main idea of ( 9) is determining the value of Choquet integral using Riemann integral by an infinite approximation method.The definition of Choquet integral is as follows when  is a finite set: where  is a permutation of the indices such that When fuzzy measure  is a -fuzzy measure, any subset is defined by To apply the Choquet integral to detect vehicles in complex environments,  is first initialized as the vehicle ROI detected by shadow-based vehicle detection algorithm. = {vechile, interfernce} is a classification framework. = { 1 ,  2 ,  3 } is the feature set for detecting vehicle, where  1 ,  2 , and  3 represent the vehicle symmetry feature, the vehicle  taillight feature, and vehicle HOG + AdaBoost classifiers feature, respectively.Let  :  → [0, 1] be the fuzzy density of vehicle ROI  belonging to the class   ; define (  ) to be the degree of importance of the feature   in deciding whether vehicle ROI is vehicle or interference.Define ( 1 ) = ({ 1 }), ( 2 ) = ({ 2 }), and ( 3 ) = ({ 3 }); the higher the (  ) is, the more important the feature   is.The fuzzy function  is defined in [0, 1] so that ( 1 ) =  tailCoeff , ( 2 ) =  symCoeff , and ( 3 ) =  hogadbCoeff .To calculate the value of Choquet integral for each vehicle ROI, the features   in the set  are needed to be rearranged with respect to the order ( 1 ) ≤ ( 2 ) ≤ ( 3 ).
Main steps of our multifeature fusion vehicle detection algorithm based on Choquet integral are as follows.

Multifeature Fusion Vehicle Detection Algorithm Based on Choquet Integral.
Step 1. Calculate the fuzzy measure of each feature.We test each feature-based vehicle detection method on the same vehicle sample set, and, according to (16), the precision of each vehicle detection method can be acquired.Let the precision + be the fuzzy measure  corresponding to each feature-based method.
Step 4. The Choquet integral value of each ROI temp choquet can be calculated by (10) combining with three feature similarity measures.
Step 5. Decide whether the vehicle ROI is vehicle according to (13).As it is illustrated in Figure 8, the temp choquet belonging to the vehicle and that belonging to the interference are much more different; the threshold Th vehicle can be set according to Figure 8:

Experiment Results
To verify the performance of the algorithm, experimental platform has been built in c using OpenCV 1.0 library     ( Experiment 1 (calculate the fuzzy measure of each algorithm).
In our multifeature fusion vehicle detection algorithm, fuzzy measure of each feature-based algorithm is set according to the performance of its own.We test each feature-based vehicle detection method on the same vehicle sample set named JVTL.The images in JVTV are vehicle ROIs detected by shadow-based method which is introduced in Section 2.
The positive samples of JVTL are vehicles, and the negative samples are interferences in JVTL.The numbers of positive and negative samples are 3219 and 6000.According to (16), the precision of each vehicle detection method can be acquired.Let the precision + be the fuzzy measure  corresponding to each method.According to Figure 9, we can set the fuzzy measure of every algorithm.
Experiment 2 (performance of our multifeature fusion vehicle detection algorithm).After setting fuzzy measure of each feature-based algorithm, we apply the sample set JVTL to test our method and every feature-based algorithm.As shown in Figure 10, the single feature cannot meet the requirement of high detection rate and low false alarm rate.Our algorithm fuzzes the output of each single feature, and the result is determined by using the fuzzy judgment instead of direct judgment.At the same time, the use of fuzzy integral can give full consideration to the cooperation of multifeatures and the importance degree of each feature in the recognition phase.Therefore, our method outperforms each single feature.In our experiment, the average processing time (AVT) of our method can achieve 50 ms per frame when processing on the Caltech Rear public test images whose resolutions are 896 × 592, which basically achieve real-time processing.And  Experiment 3 (algorithm comparison).To verify the performance of our method, we compare our method to three feature-based methods, the voting method of these three feature-based methods, and vehicle detection methods in [11,15,16].Algorithms used for comparison are all tested on the same collection (the public test library Caltech Cars (Rear) [27]).There are two ways to get the algorithms' results.
On one hand, we download the source code from the websites which have been provided in their articles to get the testing results.On the other hand, we directly use the testing results illustrated in the articles.Comparison result is shown in Table 2; it shows that the single feature-based methods can detect vehicle better, but the false alarm rate is also the highest.Although the voting method can reduce the false alarm rate, the detection rate is reduced either.Processing time is another indicator to measure the performance of (b) are shadow regions segmented by setting th BW to 0.1.The green lines in Figure 2(c) are vehicle shadow lines detected by shadow-based vehicle detection method.

Figure 2 :Figure 4
Figure 2: Result of shadow-based vehicle ROI detection method.
Figure 4(a) is the vehicle ROI detected by the shadowbased vehicle detection method.Figure 4(b) is the binary image of taillights detected by employing (1) on the vehicle ROI.Canny-based edge detection method is used to detect the edges of taillights in Figure 4(b), and Figure 4(c) is the edge image of Figure 4(b).Then the connected domains in Figure 4(c) are extracted.The input images of connected domain extraction method are binary and edge image which are illustrated as Figures 4(b) and 4(c), respectively.

Figure 3 :Figure 4 :
Figure 3: Comparison of RGB components between taillights and the other parts of vehicles.

Figure 7 :
Figure 7: Statistic distribution of hogadbCoeff belonging to vehicles and interferences.

Figure 8 :
Figure 8: Comparison of temp choquet between the vehicle and the interference.
Number of detected vehicles × (Number of detected vehicles + Number of false alarms) −1 .
the processing time is 36 ms per frame on images whose resolutions are 640 × 480.Part of results of our algorithm are shown in Figures11 and 12. Figure11is the result of algorithm on Caltech Rear public vehicle images; we set the main thresholds as follows: th BW = 0.1 and Th vehicle = 0.9.Experimental results show that our method can detect well vehicles in different distances.The distances are different in Figures12(a) and 12(b); the distances between vehicles and camera are from 3 m to 50 m.Figure12(c) shows that our method can not only detect the single vehicle, but also handle the multivehicle detection problem.Figure12(d) is the detection result on urban road.

Figure 11 :Figure 12 :
Figure 11: Detection results on Caltech Rear public vehicle images.

Table 1 :
Mapping table between   and hogadbCoeff.
The criterion to determine a "good" detection in this paper is the overlap of the detected bounding box versus the annotated bounding box.If the overlap is larger than a certain threshold, the detection is a "good" detection.Consider datasets; the frames of our datasets are 1500, and the number of vehicles in datasets is 3219.The test images include single vehicle, multivehicle, and illumination changing in the scene.We use three indicators to measure the performance of algorithms: the detection rate +, the false alarm rate −, and the algorithm precision +.