Distributed Multiagent Control Approach for Multitarget Tracking

2015 Liang Ma et al.Thisisanopen accessarticledistributedundertheCreativeCommonsAttributionLicense,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. In multiagent systems, tracking multiple targets is challenging for two reasons: firstly, it is nontrivial to dynamically deploy networked agents of different types for utility optimization; secondly, information fusion for multitarget tracking is difficult in the presenceofuncertainties,suchasdataassociation,noise,andclutter.Inthispaper,wepresentanovelcontrolapproachindistributed mannerformultitargettracking.ThecontrolproblemismodelledasapartiallyobservedMarkovdecisionprocess,whichisa NP-hardcombinatorialoptimizationproblem,byseekingallpossiblecombinationsofcontrolcommands.Tosolvethisproblem efficiently,weassumethatthemeasurementofeachagentisindependentofotheragents’behaviorandprovideasuboptimal multiagentcontrolsolutionbymaximizingthelocalR´enyidivergence.Inaddition,wealsoprovidetheSMCimplementation ofthesequentialmulti-Bernoullifiltersothateachagentcanutilizethemeasurementsfromneighbouringagentstoperform informationfusionforaccuratemultitargettracking.Numericalstudiesvalidatetheeffectivenessandefficiencyofourmultiagent controlapproachformultitargettracking.


Introduction
Advances in microelectromechanical systems have significantly boosted the development of multiagent systems in the past two decades.Low-cost agents, for example, robots, unmanned vehicles, or autonomous platforms, with high mobility, various sensing types, and powerful communication ability, are capable of different tasks in complex environment, for example, environmental monitoring, target localization and tracking, and event recognition [1,2].Multiagent coordinated control, as the fundamental problems in multiagent system, has received increasing research interest for utility optimization recently [3][4][5].An area that benefits greatly from multiagent system is target tracking [6].However, very little progress has been made in this direction since it is an extremely challenging problem in two aspects: multiagent control and information fusion for multitarget tracking.
The multiagent control problem in essence is a decision making issue which is to fulfill a given task in an optimal/ suboptimal way.In this paper, we study the multiagent control problem particularly for multitarget tracking including accurate estimation of both target number and the location of each target.In the literature, researchers and practitioners have done extensive work on the agent control problem.Olfati-Saber et al. [6] established a consensus based Kalman filter for distributed single target tracking.In [7], multiagent control is performed by path following of a virtual leader.Olfati-Saber [8] provided flocking algorithms in both theory and applications to handle large number of agents.Unfortunately, none of them fit for the multitarget tracking issue well.
Recently, random finite set (RFS) based Bayesian framework has opened doors for multisensor multiobject system and provides elegant mathematical tools to address multitarget tracking problem in multiagent systems [9].The probability hypothesis density (PHD) [10], cardinalized PHD (CPHD) [11], and multi-Bernoulli filter [12] have been developed as approximations under different posterior assumptions.Gaussian mixture and particle implementation of these filters have been shown to be effective in different tracking applications [13][14][15][16][17]. Using tools from FISST, agent control can pose as a partially observed Markov decision process (POMDP) [18], which has been shown to be effective in a single agent case in recent works [19,20].
In this paper, we will further extend the work in [19,20] to a more general problem: multiagent control for multitarget tracking, which is far more difficult than the single agent control case.We first model the multiagent coordinated control problem as a 1-step look-ahead POMDP since the multiple-step look-ahead is computationally intractable and show it is a NP combinatorial optimization problem when seeking all possible combinations of admissible control commands.Hence, we propose a suboptimal solution under the assumption that the measurements of each agent are independent of other agents' behavior.The multiagent coordinated control is decoupled into distributed control of each agent by maximizing the local Rényi divergence between prior and posterior multitarget probability density.Besides, we present the sequential Monte Carlo (SMC) implementation of the sequential multi-Bernoulli filter for each agent to utilize measurements from neighbouring agents.Numerical simulations demonstrate the effectiveness and efficiency of our approach.
The remainder of this paper is organized as follows.Section 2 presents some preliminary knowledge of RFS based Bayesian framework to lay a foundation for the rest of this paper.In Section 3, we illustrate the distributed agent control approach and present its implementation in detail.Section 4 provides the information fusion scheme for multitarget tracking for each agent to utilize measurements from other agents.Section 5 provides numerical results that verify the proposed agent control and multitarget tracking approach.

Preliminary Knowledge of RFS Based Bayesian Framework
This section provides the basic concepts and notations of RFS based Bayesian framework.Section 2.1 gives a general description of RFS and how to model multitarget by multi-Bernoulli RFS.Then, the RFS based Bayesian filtering is provided thereafter in Section 2.2.

Multi-Bernoulli RFS.
A random finite set (RFS) is a random variable that takes values as unordered finite sets.The randomness of an RFS refers to two aspects: the set cardinality (number of elements of the set) is random; each element in the set is also a random variable.The probabilistic description of RFS has been studied regarding various types of probability distributions such as multi-Bernoulli (or Bernoulli) RFS, IID (short for independent identically distributed) cluster RFS, and Poisson RFS [21].Here, we introduce the multi-Bernoulli RFS for multitarget state modelling, which offers a better alternative than the Poisson RFS and IID cluster RFS in applications with highly nonlinear model and/or nonhomogeneous sensor type [12].Assume the dimension of target state is ; then, the target state space is denoted by X ⊆ R  .A multi-Bernoulli RFS  on X is a union of a fixed number of independent Bernoulli RFSs  () with existence probability  () ∈ (0, 1) and probability density  () (defined on X),  = 1, . . ., ; that is,  = ∪  =1 { () }.
Use a Bernoulli set for modelling a single target; then, the multitarget state can be modeled as multi-Bernoulli RFS Ξ with probability density given in [12] as follows: where  () and  () , respectively, represent the existence probability and distribution of the th target and (0) = ∏  =1 (1 −  () ).It is clear that the multitarget density can be completely specified by multi-Bernoulli parameter set {( () ,  () )}  =1 .Hence, let us denote the multitarget density at time  by   = {( =1 for short in the following content.

Multisource Multiobject Bayesian
Framework.Stochastic filtering in Bayesian framework has developed in decades [22].Under the assumption of linear model and Gaussian distribution, Kalman filter is derived [23] and has been widely used for tracking since then.To extend the standard Bayesian framework to a multisource multitarget version, we need the help of RFS modelling.Let   and   denote the state set and observation set, respectively, as follows: where  is the total number of agents (we treat each agent as a single sensor) and    = {z   , . . ., z  , , } for  = 1, . . ., .   is the time-varying cardinality of targets, while  , is the cardinality of the measurement set generated by agent .
Using the RFS representation, the movement of multiobject can be described using two parts: an RFS for survival targets from previous time step   and an RFS for spontaneous birth targets at current time Γ  .Thus, at time , we have the predicted RFS   =   ∪ Γ  .The RFS for measurements    of agent  can be represented as a union of two parts: target-generated measurements Θ   and clutter    ; thus,    = Θ   ∪    .Given the specific type of RFS, the Bayesian framework for optimal estimation via RFS which is the same form as the classical Bayesian filtering is given as follows: which represent the prediction and update process of Bayesian recursion via set integrals, respectively.Under different assumptions of the RFS type, the PHD, CPHD, and multi-Bernoulli filter have been derived from (3) by finite set statistics [9].Notice that the integrals in (3) are FISST set integrals (see [9][10][11] for details).

POMDP Based Distributed Multiagent Control Approach
In this section, we first illustrate multiagent control in the framework of a POMDP in Section 3.1.Section 3.2 provides maximizing expected Rényi divergence between prior and posterior multitarget density as the objective function for the control scheme.By assuming the measurement of each agent is independent of other agents' behavior, we propose a distributed agent control approach by maximizing the local Rényi divergence in Section 3.3.

POMDP Based Multiagent Control.
We begin with the notations of using a POMDP as the solution for multiagent control.At time , denote the control command of agent  by u   ∈ U   , where U   is the set of all admissible control commands for agent .Let U  ∈ U  denote the multiagent control command, and U  is the set of all possible control command combinations.Then, U  = {u 1  , . . ., u   , . . ., u   } for  = 1, . . .,  and  is the total number of agents.
Define D(V, , ) as the objective function dependent on multiagent control command V, multitarget density , and the associated measurement set  when control command V was applied.The aim of multiagent control is to find the optimal multiagent control command U  by maximizing/minimizing the statistical expectation of predefined objective function D(V, , ) as where  −1 ( −1 |  0:−1 , U 0:−1 ) represents the multitarget posterior density after applying a sequence of multiagent control commands U 0 , . . ., U −1 .Remark that the general formulation of POMDP is a -step future decision process of which the computational cost would grow exponentially with the number of future steps.In this paper, we only consider a one-step future decision described by (4) as an approximation.

Global Objective Function.
As shown in (4), the objective function plays a crucial role in POMDP based multiagent control problem.Information theoretic method is a typical objective function for sensor management.Here, we propose maximizing the information gain of multitarget prior and posterior density as the objective function for tracking.The Rényi divergence, also known as alpha divergence, measures the information gain between any two probability densities.The objective function for multitarget tracking is given as follows: where and  is a parameter that determines how much we emphasize the tails of two densities in the metric.Notice that the Rényi divergence becomes the Kullback-Leibler discrimination and Hellinger affinity, respectively, when  → 1 and  = 0.5 [24].
To compute the expectation of ( 5), we introduce the SMC implementation of the objective function.At time , assume that the multitarget predicted density where the multisensor multitarget likelihood function dependent on multiagent control and the single-sensor multitarget likelihood function dependent on multiagent control   +1 (  +1 |   +1 , U  ) is given in [9] as follows: where   +1 (z  6) is a multisensor multitarget case, whereas the derivation is similar to the single sensor case in [19].Thus, we directly omit the tedious proof here.

Distributed Agent Control.
Even though multiagent control can be described as a POMDP given by ( 4) and ( 5), it is still intractable to achieve the global optimum of defined objective function for two reasons: firstly, searching all admissible control command combinations is a NP-hard combinatorial optimization problem [25]; secondly, global optimization requires the existence of a centralized fusion center that receives information from all agents, which is unrealistic for most large-scale multiagent systems.Hence, we propose a distributed sensor control method that compromises the local optimum of the global objective function, which is computationally tractable and convenient to implement.
Instead of computing the global information gain, we consider finding the optimal command for each individual agent.Similar to (5), the Rényi divergence for the th agent is given by Assume that measurement set generated by one agent is independent of other agents' behaviour; then, the multisensor multitarget likelihood function  +1 ( +1 |   +1 , u   ) in ( 9) can be written as follows: where ) for  ̸ = .The future measurement  +1 is generated assuming no clutter and unity detection rate as illustrated in [20].For agent ,   +1 is predicted based on the multitarget state and possible control command u   , while   +1 for  ̸ =  is predicted based on the multitarget state and current location of agent .Therefore, we can find the optimal control command for each individual agent and then combine them together to form the total control command set, which is given as follows: Remark 1. Equation ( 9) is exactly a particular case of ( 6) by assuming all agents except agent  stay still at current time.
The proposed distributed control approach is a suboptimal solution of ( 6) by seeking its local optimum, and the local optimum means that u   is optimal given all other agents keep still.
When computing the optimal command of agent , the distributed agent control given by (11) approximates the likelihood function   +1 (  +1 |   +1 ) via using current locations of agent  for  ̸ = .Hence, we refer to (9) as the "local Rényi divergence" in this paper.
The proposed distributed control approach can significantly reduce the computational cost to perform real-time agent control.For illustration, assume that all agents have the same number of possible control command denoted by ; the computation complexity of our control approach is O( ⋅ ) which is much smaller than O(  ) by searching all control combinations, especially in large-scale multiagent system.
Generally speaking, each agent can only communicate with its neighbouring    agents and obtain their current locations s   as well as their measurement model   (z | x) for  = 1, . . .,    .Here, we assume that the information received by each agent is accurate without any input saturation.For a more challenging case that there is input saturation described in [26,27], the topic is beyond the scope of this paper.The neighbouring relationship may change over time due to the relative movement of agents.Algorithm 1 provides the SMC implementation of proposed distributed control for each agent by maximizing the local Rényi divergence.

Multisensor Fusion for Multi-Target Tracking
As mentioned before, the multi-Bernoulli filter outweighs the PHD/CPHD filter in the SMC implementation for nonlinear problem since the state extraction in multi-Bernoulli filter is not dependent upon the heuristics in clustering but is dependent only on the Bernoulli parameters.Hence, the multi-Bernoulli filter has been used extensively in computer vision [16,17], robot SLAM [28], and sensor network [29].Besides, the state-of-the-art development of the multi-Bernoulli filter offers the power to directly produce tracks of individual targets, which is known as the labelled multi-Bernoulli filter in the community [30].
In this section, we first briefly review the cardinalitybalanced multi-Bernoulli filter given in [12] in Section 4.1.Then, Section 4.2 provides the sequential update scheme for information fusion of multiple sensors.

Cardinality-Balanced Multi-Bernoulli Filter
Prediction.At time , if the posterior multitarget density is multi-Bernoulli given by   = {( =1 and the density of new births is also multi-Bernoulli  Γ,+1 = {( =1 , then the predicted density is given by where for survival targets , ⟩ ⟨ =1 , the output of corrector is composed of legacy tracks and measurementupdated tracks as where and   are probability of survival and detection.The inner product ⟨⋅, ⋅⟩ is defined between two real valued functions  and  by ⟨, ⟩ = ∫ ()().Note that, without loss of generality, we refer to the cardinality-balanced multi-Bernoulli filter as "multi-Bernoulli" filter for simplicity in this paper.

Sequential Multisensor Multi-Bernoulli Filter.
In multisensor multitarget tracking scenario, there is no unified multisensor fusion method which is tractable and computationally acceptable.Sequential update has been widely used and verified to be a good approximation for information fusion of multiple sensors.Here, multitarget tracking in multiagent network is implemented via multi-Bernoulli filter with sequential update scheme based on the SMC implementation.
Suppose that, at time , the posterior multitarget density is given as , and the distribution of each target is given by a set of weighted particles . Then, we give the SMC implementation of sequential multisensor multi-Bernoulli filter in Algorithm 2. We refer the readers to Section 4.1 of [12] for detailed equations.

Simulation
In order to demonstrate the performance of proposed multiagent control approach for multitarget tracking, we present numerical results for a planar multitarget tracking scenario in multiagent system where three controllable moving observers, two equipped with range-only sensors and one with a bearing-only sensor, are placed in a specified surveillance area of size [0, 1000 m] × [0, 1000 m] to estimate the number of targets as well as their positions.Each agent shares its current locations, sensor type, and observations with the other agents.It is intuitive that the initial positions of agents have an impact on the agent control and target tracking procedure.However, we are not going to involve the network topology issue in this paper since our approach is supposed to act independently from the network topology which is different from the method described in [31,32].
There are unknown and time-varying numbers of targets observed in clutter for each agent.Assume that targets move according to the nearly constant velocity model given by where with  0 = 1 m,   = 5 × 10 −5 m −1 ,  1 = /180 rad, and   = 10 −5 rad ⋅ m −1 .Targets can appear or disappear in the scene at any time, and survival probability   = 0.95 for each existing target.New born targets appear spontaneously according to   = 0.2N(⋅; x, ).Three targets are presented as illustrated in Figure 1, where with  0 = 320 m and  = 0.002 m −1 .The clutter rate of each sensor   = 5 per scan.The standard derivation of process noise  V = 1 m/s for both V , and V , .Given the current agent location s   , the set of admissible control commands for each agent is computed as U   = {[   + Δ  cos(Δ  ),    + Δ  cos(Δ  )]} =0,...,  =0,...,  , where Δ  = 2/  and Δ  = 30 m are for angular and radial step size.  = 2 and   = 8 here.
The initial locations of agents are shown in Figure 1, and each agent runs for 30 scans with sampling period  = 1 s.Birth intensity for the multi-Bernoulli filter is approximated using adaptive target birth intensity sampling technique described in [33]. min = 300 and  max = 500 are the minimum and maximum particle numbers of each Bernoulli set for track maintenance.
Since each agent shares its location and observation with other two agents, agents have pretty much the same performance in this scene.Hence, we take agent 3 (bearing-only) as an example to illustrate the tracking procedure.Figure 2 presents three key frames of one trial run of the proposed control and tracking approach.It is clear that, with the movement of agents, scattered particles converge to the ground truth locations of targets.This is because each agent is moving forward to a more informative direction so that "ghost" particles that are not generated by actual targets can be quickly eliminated by the multisensor fusion scheme.As a result, the estimation of target positions is getting more accurate over time.Besides, it can be seen from Figure 2(c) that the trajectories of agents are different for their distinctive initial locations and sensor types.
Due to stochastic nature of our control and tracking approach, we adopt 1000 Monte Carlo runs to evaluate their performance.The optimal subpattern assignment (OSPA) metric composed of location error and cardinality error is used for tracking performance evaluation [34].Figure 3 shows the OSPA distance of our approach, ( = 100 m,  = 1) from 1000 Monte Carlo runs, from which we can see that the OSPA distance converges with the movement of agents over time.The location error is reduced gradually since each agent is obtaining more informative measurements to lower the covariance of position estimation, while the estimation of target number quickly converges to a relatively low value.The average computation time for a single run is only approximately 0.82 s (algorithm is implemented in MATLAB 2012a on a PC with 8 GB RAM and Intel Core i7-4770k CPU).Several runs have been recorded in videos attached as Supplementary Material (available online at http://dx.doi.org/10.1155/2015/903682)for demonstration.

Conclusion
In this paper, we propose a novel distributed multiagent control approach by maximizing the local Rényi divergence.The SMC implementation of the sequential multi-Bernoulli filter is provided for each agent to utilize the information from neighbouring agents.Simulation results demonstrate that sponsored by the Fundamental Research Funds for the Central Universities (HEUCF130703).
), where each multitarget particle   +1 = {x  +1,1 , . . ., x  +1,   },   +1 = |  +1 | is the number of targets, x  +1, for  = 1, . . .,   +1 represent target position in the state space, and   is the weight associated to particle   +1 .Notice that   +1 is a particle sampled from a RFS, which accounts for the randomness of both the cardinality and target positions of particle   +1 .Given the Bayesian prediction and update equations (3), we obtain (x) are prior existence probability and distribution of birth model.Update.At time  + 1, if the predicted multitarget density is multi-Bernoulli  +1| = {(

Figure 1 :
Figure 1: Target tracks.Start/stop positions for each track are shown with •/◼.Colored ⧫ are initial locations of agents.