An Adaptive Object Tracking Using Kalman Filter and Probability Product Kernel

We present a new method for object tracking; we use an efficient local search scheme based on the Kalman filter and the probability product kernel (KFPPK) to find theimage region with a histogram most similarto thehistogram ofthe tracked target. Experimental results verify the effectiveness of this proposed system.


Introduction
Real time object tracking is one of the most active research areas in computer vision.The goal of object tracking is to estimate the locations and motion parameters of the target in a video sequence given the initialized position in the first frame.Research in tracking plays a key role in understanding motion and structure of objects.It finds numerous applications including surveillance [1], human-computer interaction [2], traffic pattern analysis [3], recognition [4], and medical image processing [5], to name a few.Although object tracking has been studied for several decades and numerous tracking algorithms have been proposed for different tasks, it remains to be a very challenging problem.One of the most challenging factors in object tracking is to account for appearance variation of the target object caused by change of illumination, deformation, and pose.In addition, occlusion, motion blur, and camera view angle also pose significant difficulties for algorithms to track target objects.Furthermore, there exists no single tracking method that can be successfully applied to all tasks and situations.
In this paper we present a new method for real time tracking of complex nonrigid objects.This new method successfully coped with camera motion, partial occlusions, and target scale variations.The scale/shape of the object is approximated by an ellipse and its appearance by histogrambased features derived from local image properties.We use an efficient search scheme (Kalman filter [6,7], using the probability product kernels [8][9][10][11] and integral image [12] as a similarity measure) to find the image region with a histogram most similar to the target of the object tracker.In this work, we address the problem of scale/shape adaptation and orientation changes of the target.The proposed approach (KFPPK) is compared with MKF [13].In [13] the authors build a color histogram-based visual representation regularized by a spatially smooth isotropic kernel.Using the Bhattacharyya kernel [8] as the similarity measure, a mean shift procedure is performed for object localization by finding the basin of attraction of the local maxima.However, the tracker MKF only considers color information and therefore ignores other useful information such as target scale variations, resulting in the sensitivity to background clutters and occlusions.Extensive experiments are performed to testify the proposed method and validate its robustness and effectiveness to track the scale and orientation changes of the target in real time.
The rest of the paper is organized as follows: Section 2 presents the probability product kernels.Section 3 introduces basic Kalman filter for object tracking.Section 4 presents scale estimation.Section 5 presents the proposed approach and Section 6 the experiment result.Section 7 concludes the paper.

Probability Product Kernel
In this paper we define a class of kernels   :  ×  → R on the space of normalized discrete distributions over some index set Ω. Specifically, we define the general probability product kernel [8,9] between distributions  and  as where  is a parameter and () and () are probability distributions defined on the space Ω, since, for any  1 ,  2 ,  3 , . . .,   ∈ Ω, the Gram matrix  consisting of elements   =   (  ,   ) is positive semidefinite: for  1 ,  2 ,  3 , . . .,   ∈ R. Different  values correspond to different types of probability product kernels.For  = 1, we have We call this the probability product kernels defined by (, ) = ∑  ()().We denote the histogram of target of object tracking  as ℎ  and the number of pixels inside  as ||, which is also equal to the sum over bins || = ∑  ℎ  ().
Let  be the normalized version of ℎ  given by  = ℎ  /||, so we can consider  as a discrete distribution, with ∑  () = 1.Let  be the normalized histogram obtained in the frames of the video sequence.For the -bin of ℎ  , its value is obtained by counting the pixels that are mapped to the index : where [] is the Kronecker delta with [] = 1 if  = 0 and [] = 0 otherwise.The mapping function () maps a pixel  to its corresponding bin index.The computation of the probability product kernels can be expressed as Therefore, the computation of the probability product kernels can be done by taking the sum of values (()) within target .The output of the following algorithm is a support map using integral image to compute the similarity measure between target and candidate region from each frame of the video sequence.

Kalman Filter
The Kalman filter has many uses, including applications in control [14], navigation [15], and computer vision.One important field of computer vision is the object tracking.Different movement conditions and occlusions can hinder the vision tracking of an object.In this paper, we present the use of the Kalman filter in the object tracking.We considered using the capacity of the Kalman filter to allow small occlusions and complex movements of objects [16].Furthermore, the Kalman filter is a framework for predicting a process state and using measurements to correct or "update" these predictions.

State Prediction.
For each time step , a Kalman filter first makes a prediction ŝ of the state at this time step: where  −1 is a vector representing process state at time  − 1 and  is a process transition matrix.The Kalman filter concludes the state prediction steps by projecting estimate error covariance  −  forward one time step: where  −1 is a matrix representing error covariance in the state prediction at time  − 1 and  is the process noise covariance (or the uncertainty in our model of the process).

State Correction.
After predicting the state ŝ (and its error covariance) at time  using the state prediction steps, the Kalman filter next uses measurements to "correct" its prediction during the measurement update steps.First, the Kalman filter computes a Kalman gain   , which is later used to correct the state estimate ŝ : where  is measurement noise covariance.Determining   for a set of measurements is often difficult.In our contribution we calculated  dynamically from the measurement algorithms state.Using Kalman gain   and measurements   from time step , we can update the state estimate: Conventionally, the measurements   are often derived from sensors.In our approach, measurements   are instead the output of the tracking algorithm given the same input: one frame of a streaming video and the most likely  and  coordinates of the target object in this frame (taking the first two dimensions of ŝ ).The final step of the Kalman filter iteration is to update the error covariance  −  into   : The updated error covariance will be significantly decreased if the measurements are accurate (some entries in   are low) or only slightly decreased if the measurements are noisy (all of   is high).

Target Model Update
A target is represented by an ellipsoidal region in the image in order to eliminate the influence of different target dimensions [13].We will denote a dataset of independent samples by  = { 1 , . . .,   }.Let us assume that the probability density function Gaussian () = ℵ(  , (  ), ) is a good generative model for our data [17].Gaussian density function is proposed to automatically determine the number of density functions and their associated parameters, including the pixel locations   , covariance , and location of the target (  ).
The advantages of using the covariance matrix representation are as follows: (i) It is robust to illumination changes, occlusion, and shape deformations.(ii) It can capture the intrinsic self-correlation properties of object appearance.(iii) It is low dimensional.
Those advantages lead to the computational efficiency: We represent an object  = [ 1 , . . .,   ]  by modeling the spatial-color joint probability distribution of the corresponding region with a Gaussian density function.We define (  ):  2 → 1, . . .,  to be the function that assigns a color value of the pixel at location   .We employ the normalized RGB color space as the color feature in our case; we have used a normalized RGB histogram to provide some robustness to illumination change.We can write the features of spatial position and the color as where  is the Kronecker delta function.We use the Gaussian kernel  to rely more on the pixels in the middle of the object and to assign smaller weights to the less reliable pixels at the borders of the objects.We use only the  V 0 pixels from a finite neighborhood of the kernel and the pixels further.The color histogram that describes the appearance of the region is   ((), ) and the value of the -bins is calculated by The weights of the value of -bins in object are calculated by The new distribution of color histogram  is calculated by  The covariance matrix update can be used to approximate the new shape of the object and it is calculated by and we should use  = 1.5.The correct value for  depends on the noise that is present in the image sequence.

Proposed Approach
To ensure good organization of the progress of the work, we used the benefits of modular design in our implemented approach using MATLAB.The goal of object tracking is to generate the trajectory of an object over time by discovering its exact position in every frame of the video sequence.The steps of the object tracking system are shown in Figure 1.
The proposed approach (KFPPK) for object tracking is composed of five blocks named block processing, block prediction, block tracking, block correction, and block result.The functions of these blocks are as follows.
Block Processing.The algorithm starts the video sequence and converts it into images processing for extracting color information of images and the target of the object tracking.
Block Prediction.This step attempts to evaluate how the state of the target will change by feeding it through a state prediction of the Kalman filter.The state prediction serves two purposes: the time update equations are responsible for projecting forward (in time) the current state and the error covariance estimates for obtaining the a priori estimate for the next time step.
Block Tracking.In this block we use the probability product kernels combined with integral image as a similarity measure to find the image region with a histogram most similar to the target of the object tracker.Furthermore, we estimate the scale/shape and orientation of the object tracker using target model update.
Block Correction.The block correction update equations are responsible for the feedback.That is used for incorporating a new measurement into the a priori estimate to obtain an improved a posteriori estimate.The time update equations can also be considered as predictor equations, while the measurement update equations can be thought of as corrector equations based on block tracking.
Block Result.Tracking trajectory of the object is done on the basis of the region properties of the object such as scale/shape and centroid.
The algorithm of the proposed approach (KFPPK) can be explained as follows.
Step 1. Start video sequence and select the target of the object tracker in the first frame.
Step 2. In this stage we use state prediction of Kalman filter to estimate how the state of a target will change by feeding it through a current state and error covariance estimates to obtain a priori estimate for the next time step.This step uses (1) and (2).
Step 3. Calculate the similarity measure ((())) between the target model and candidate regions using (5) and estimate the shape and orientation of the object tracker using (13).
Step 4. Correct and update equations into a priori estimate to obtain an improved a posteriori estimate, using ( 9) and (10).
Step 5. Draw trajectory.And go to Step 2 in the next frame.

Experimental Results
This section mainly shows the tracking results using the proposed approach (KFPPK).Experiments on real video sequences have been carried out to evaluate the performance of the proposed algorithm.In these experiments, we compared our system with existing algorithms MKF [13].The proposed algorithm achieves good estimation accuracy of the scale and orientation of the object in the video sequence.We used different sequences; each has its own characteristics but the use of a single object in movement is a commonality between these different sequences.We then set up experiments to list the estimated width, height, trajectory, and orientation of the object.
In this work, we selected normalized RGB color space as the feature space and it was quantised into 16 × 16 × 16 bins to compare between algorithms.One synthetic video sequence and one real video sequence are used in the experiments.We first use a synthetic ellipse sequence (where the resolution is 352 × 240, the frame rate is 25 fps, and the number of frames is 77) which is used to verify the efficiency of the KFPPK.As shown in Figure 2, the external ellipses represent the target candidate regions, which are used to estimate the real targets, that is, the inner ellipses.The experimental results show that the KFPPK is reliable for estimating the mean position and trajectory (as shown in Figure 3) of the ellipse with scale and orientation changes.Meanwhile, the results by the MKF [13] algorithm are not as performant as the proposed approach KFPPK because of significant scale and orientation changes of the object tracker.
The second test is an occlusion sequence (where the resolution is 320 × 240, the frame rate is 25 fps, and there are 193 frames) which is used to verify the efficiency of the KFPPK on a more complex sequence.The object exhibits large scale changes with partial occlusion.We can see the results by the proposed approach (KFPPK) and the MKF [13] in Figure 4. We observe that the KFPPK works much better in estimating the scale and orientation of the target, especially when occlusion occurs.Table 1 lists the average time on the videos sequences.We notice that our proposed approach (KFPPK) has a better average time of execution than MKF [13] where TP (true positives) are examples correctly labeled as positives, FP (false positives) refer to negative examples incorrectly labeled as positive, TN (true negatives) correspond to negatives correctly labeled as negative, and FN (false negatives) refer to positive examples incorrectly labeled as negative.We studied the ROC curves of the proposed algorithm and compared them with the MKF [13] algorithm.Figure 5 compares the performance of the algorithms using ROC curves for each video sequence.The KFPPK algorithm has a higher performance in regard to the MKF [13] algorithm.
The experimental results demonstrate that the KFPPK is robust in the tracking of the trajectory of objects in different    situations (scale variation, pose, rotation, and occlusion).
It can be seen that the KFPPK achieves good estimation accuracy in real time of the scale and orientation of the target.

Conclusion
In this paper, the proposed approach (KFPPK) has been presented for tracking a single moving object in the video sequence using color information.In this approach we combine the Kalman filter and probability product kernel as a similarity measure using integral image to compute the histograms of all possible target regions of object tracking in the video sequence.The new KFPPK has been compared with the state-of-the-art algorithm on a very large dataset of tracking sequences and it exceeds in performance in the processing speed.The extensive experiments are performed

Table 1 :
The average time by different methods on the videos sequences. algorithm.