An Unbalanced Weighted Sequential Fusing Multi-Sensor GM-PHD Algorithm

In this paper, we study the multi-sensor multi-target tracking problem in the formulation of random finite sets. The Gaussian Mixture probability hypothesis density (GM-PHD) method is employed to formulate the sequential fusing multi-sensor GM-PHD (SFMGM-PHD) algorithm. First, the GM-PHD is applied to multiple sensors to get the posterior GM estimations in a parallel way. Second, we propose the SFMGM-PHD algorithm to fuse the multi-sensor GM estimations in a sequential way. Third, the unbalanced weighted fusing and adaptive sequence ordering methods are further proposed for two improved SFMGM-PHD algorithms. At last, we analyze the proposed algorithms in four different multi-sensor multi-target tracking scenes, and the results demonstrate the efficiency.


Introduction
The Multi-sensor Multi-target Tracking (MMT) technique generally refers to the process of estimating the targets' number and dynamic states from multi-sensor observations. The MMT technique has many important applications in the fields of navigation, transportation, monitoring, tracking as well as remote sensing [1,2], etc. Although we can complete a multi-target tracking work by a single sensor, many factors may degrade the tracking results, such as detected missing, false alarm and observation deviation [3]. A reasonable way to overcome the weakness of single sensor tracking is to take the advantage of the redundancy of multi-sensor data. Therefore we can improve the tracking results by using MMT. It is worth mentioning that a MMT problem is usually more complex than a single sensor multi-target tracking (SMT) problem. Generally, for a MMT problem, we need to solve the multi-sensor data fusing problem besides the SMT problem.
Like most multi-target tracking problems, for a MMT problem, the MMT problem requires the mapping relationships between observations and targets. Traditional methods usually use the data association (DA) technique to discover the mapping relationship before getting point and track estimations. The DA based MMT algorithms include the Nearest Neighbor (NN) algorithm [3], the Probability Data Association (PDA) algorithm [4], the Joint Probability Data Association (JPDA) algorithm [5], the Multiple Hypotheses Tracking (MHT) algorithm [6] and the Probability Multiple Hypotheses Tracking (PMHT) algorithm [7], etc. Beside the DA based MMT algorithms, the Random Finite Set (RFS) theory provides other ways of solving the MMT problem [8,9]. For the RFS based MMT algorithms, there are two classes of methods for obtaining the trajectory estimates: (1) The multi-target state estimation (point estimation) can be achieved without a DA process in the first place. Then, one can apply a DA technique to further obtain the multi-target track estimation [10,11]. (2) One also can

Problem Formulation
It is assumed there are N k targets at time k, where N k is an unknown variable. The state vector of target i at time k is described by x k,i . Then, the set of N k targets at time k is denoted by X k = x k,1 , x k,2 , . . . , x k,N k . For an arbitrary target x k,i ∈ X k , if it exists at time k + 1, then, the state transition equation can be described as follows: where, f k,k+1 is the one-step state transition function, and w k,i is the un-modeled error.
It is assumed there are s sensors. For an arbitrary sensor j, its target detection rate for a particular target is 0 ≤ P d,j ≤ 1. If at time k, target x k,j is observed by sensor j, then, the observation equation can be described as follows: where, z k,j is the observation vector, g k,j is the observation function, and v k,i,j is the observation error, which is assumed to be a Gaussian white noise with zero mean and covariance R j . Except when receiving the data from targets, the sensor also receives false alarms from clutters. At time k, the received clutter number for sensor j submits to a Poisson distribution with intensity λ j . The clutters take their positions uniformly in the target space. Specifically, the clutters received by sensor j at time k are described as follows:    ρ(n k ) = (e −λ j λ n k j )/n k !n k = 0, 1, 2, . . . q(z c j,l ) = 1/Ψ(x)l = 1, 2, . . . , n k where, n k presents the observed clutter number, ρ(n k ) is the corresponding probability, q(z c j,l ) is the probability density of clutter l taking its position at z c j,l , and Ψ(x) is the volume of the observation space. The observations of sensor j at time k are denoted by Z j k = z 1 j,k , . . . , z r j,k , the observations of sensor j until time k are presented by Z j 1:k = Z j 1 , . . . , Z j k , and the accumulated observations for all s sensors are summarized by Z 1:s 1:k = Z 1 1:k , . . . , Z s 1:k . In this paper, the object of MMT is to estimate the numbers and states of the unknown targets based on Z 1:s 1:k .

Sequential Fusion Multi-Sensor Gaussian Mixture PHD Algorithm
To solve the problem described in Section 2, we propose the Sequential fusion multi-sensor Gaussian Mixture PHD (SFMGM-PHD) algorithm in this section based on the GM-PHD algorithm [33]. To facilitate discussion, we first review the basic conclusions of the single sensor PHD algorithm. Then, the details of the SFMGM-PHD algorithm will be explained.

A Briefl Review for the Single-Sensor PHD Algorithm
For sensor j, if the target motion and observation models are described by Equations (1)-(3), then, the formalized single sensor PHD algorithm can be described by Equations (4)-(6) [13,14]: where, D k|k (x|Z j 1:k ) presents the posterior PHD estimation, f k|k−1 (x|τ) is the target state transition function, P d,j is the target detection rate, g k (z|x) is the observation likelihood, and κ k (z) reflects the clutter intensity.
Generally, it is difficult to obtain the closed-form solution of Equations (4) and (5) for the integral operations that are involved. Therefore, suboptimal methods are proposed, such as the Particle Filter PHD (PF-PHD) algorithm [34] and the GM-PHD algorithm [33], where the integral operation can be replaced by the summation of the particles or Gaussian Mixtures, respectively.

The SFMGM-PHD Algorithm
In this section, the SFMGM-PHD algorithm will be proposed through the fusion of the multi-sensor posterior GM estimations. As depicted in Figure 1, we assumed that there are s sensors. First, each sensor receives its posterior GM estimation by a single sensor GM-PHD algorithm. Secord, we fuse the posterior GM estimations of sensors 1 and 2 to possess the local posterior GM estimation, based on the matching and CI fusing [35,36] operations. Third, we fuse the local posterior GM estimations with the GM estimations of the remaining sensors sequentially, to achieve the overall posterior GM estimation. At last, the fused GM estimation will be fed back to the local sensors. The details are as follows.
(1) Single Sensor Posterior GM Estimation 1 Prediction: It is assumed that the posterior GM estimation of sensor j at time k − 1 is available as . If the target survival probability is q s , then, the predictive GM estimation for the survival target can be calculated by Equation (7): is a Gaussian Mixture with a mean m j,i S,k−1|k−1 and a covariance P j,i S,k−1|k−1 . At time k, the newborn GM estimation for sensor j is described by Equation (8): Then, the total predictive GM estimation for sensor j at time k is calculated by the following equation: Here, the predictive GM number is J k/k−1 = J k−1 + J γ,k , J k−1 is the successive part, and J γ,k is the newborn part. If the observation of sensor j at time k is available as Z j k = {z 1 j,k , . . . , z r j,k }, then we can use Equation (10) to obtain the posterior GM estimation: where, ω i k (z l j,k ) can be calculated as follows: where, q i k (z l j,k ) and κ k (z l j,k ) are the likelihood functions of target and clutter, respectively.

(2) Multi-sensor GM Estimation Matching and Fusing
If the posterior GM estimations of sensor j = 1, 2, . . . , s are available at time k, then, we can construct the multi-sensor GM estimation sequential fusing algorithm according to the fusing framework described by Figure 1; the details are listed in Algorithm 1.
For convenience, we further summarize the SFMGM-PHD algorithm in Algorithm 2.

Algorithm 2 A simple list of the SFMGM-PHD algorithm
Setup 1: Use Equations (7)-(9) to calculate the multi-sensor predictive GM estimations.
Step 3: Fuse the multi-sensor posterior GM estimations by the algorithm described in Algorithm 1. The fused GM estimations are feedback to local sensors as the preliminary data in step 1.
Step 4: Some pruning, merging, clustering, and association methods [10,28] can be applied to achieve the targets' number and state estimations.

Remark 1.
In Section 3, the SFMGM-PHD algorithm is proposed for the MMT problem by fusing the multi-sensor GM estimations in a sequential way. However, two problems are not considered by the normal SFMGM-PHD algorithm. First, the normal SFMGM-PHD algorithm does not consider the problem of the multi-sensor fusing order. As mentioned in Section 1, the sequential fusing method is sensitive to the fusing order [31], so that adopting an unsuitable fusing order may degrade the fusing quality. By observing Equations (12) and (13), we can see the second problem is that the SFMGM-PHD algorithm adopts a balanced weighted fusing method. Specifically, the SFMGM-PHD tracker calculates the fusing weights for all sensors in a balanced way, and the balances method will in fact pay more attention to the GM estimations from the later fused sensors than those from the earlier fused sensors. In normal cases, there is no need for us to pay more attention to the GM estimations from the later fused sensors, so that we need to pursuit an unbalanced weighted fusing method that can fuse the multi-sensor GM estimations in a non-discriminatory way.

Unbalanced Weighted Sequential Fusion Multi-Sensor Gaussian Mixture PHD Algorithm
In this section, two USFMGM-PHD algorithms are proposed to solve the above-mentioned problems in Remark 1. As depicted in Figure 2, the USFMGM-PHD algorithm adopts a similar framework as described in Figure 1. The improvements are as follows. (1) First, the OSPA metric [34] is employed to value the quality of GM estimations of each sensor. Then, we can sort the multi-sensor data fusing sequence from the higher quality sensor to the lower quality sensor. (2) Second, an unbalanced weighted multi-sensor sequential fusing method is proposed, to eliminate the discrimination for the different sensors.

Unbalanced Weighted Multi-Sensor Sequential Fusing Method
Obviously, the sequential fusing time slots for the s sensors is 1 s . When processing the n -th fusion, 1 1 n s    , we need to fuse the GM estimations from the ( 1) n  -th sensor with the previously fused GM estimations. In order to derive non-discriminatory fusing weights, we reform Equation (13) in an unbalanced way, as Equation (18):

Sorting the Multi-Sensor Fusing Sequence Based on the OSPA Metric
It is assumed that the posterior GM estimations of all s sensors at time k are available as , j = 1, . . . , s. Then, for two sensors j 1 = j 2 , we can use the OSPA metric to measure the consistency between the two sensors, by Equation (16): where, c is the blending coefficient that can create a balance between the spacing and the quantity, and 1 ≤ p ≤ +∞ is the sensitive factor, which can be normally selected as 1 or 2. , is defined by Equation (17): Based on Definition 1, we can calculate the overall consistency values for all sensors. We assume that the majority of the sensors can provide high quality estimates, and the minority of the sensors provide inferior estimates. Therefore, we think if a sensor is consistent with the majority, there is a high probability that this sensor is a high quality one. Then, the sensor with smaller overall consistency value can provide better GM estimates. As the consequence, we sort the multi-sensor fusing sequence from small to large with respect to the overall consistency value (OCV).

Unbalanced Weighted Multi-Sensor Sequential Fusing Method
Obviously, the sequential fusing time slots for the s sensors is s − 1. When processing the n-th fusion, 1 ≤ n ≤ s − 1, we need to fuse the GM estimations from the (n + 1)-th sensor with the previously fused GM estimations. In order to derive non-discriminatory fusing weights, we reform Equation (13) in an unbalanced way, as Equation (18):

The Pseudo-Codes of Two USFMGM-PHD Algorithms
In this section, we summarize two USFMGM-PHD algorithms in Algorithms 3 and 4. We name them as USFMGM-PHDA and USFMGM-PHDB respectively. The two USFMGM-PHD algorithms have the similar frameworks. The difference is that USFMGM-PHDB adopts the adaptive sorting and unbalanced weighted fusing methods that is proposed in Sections 4.1 and 4.2, whereas USFMGM-PHDA only adopts the unbalanced weighted fusing method that is proposed in Section 4.2.
Algorithm 3 A simple list of the USFMGM-PHDA algorithms.
Step 3 Fuse the multi-sensor posterior GM estimations by the algorithm described in Algorithm 1, where Equation (13) is replaced by Equation (18) (see Section 4.2). The fused GM estimations are fed back to local sensors as the preliminary data in Step 1.
Step 4: Some pruning, merging, clustering, and association methods [10,28] can be applied here to achieve the targets' number and state estimations.
Step 3: Before fusing the multi-sensor posterior GM estimations, use Equations (16) and (17) to sort the multi-sensor fusing sequence from small to large with respect to the overall consistency value (see Section 4.1).
Step 4: Fuse the multi-sensor posterior GM estimations by the algorithm described in Algorithm 1, where Equation (13) is replaced by Equation (18) (see Section 4.2). The fused GM estimations are fed back to the local sensors as the preliminary data in step 1.
Step 5: Some pruning, merging, clustering, and association methods [10,28] can be applied to achieve the targets' number and state estimations.

Simulations
In the simulations, four multi-sensor multi-target tracking scenes are considered with different detection and false alarm settings. Three proposed multiple-sensor GM-PHD algorithms will be analyzed. The first algorithm is the SFMGM-PHD proposed in Section 3, the second and third algorithms are the USFMGM-PHDA and USFMGM-PHDB proposed in Section 4. The SGM-PHD algorithm [28] and the sequential measurement fusion multi-sensor GM-PHD (SMFMGM-PHD) algorithm [29] will also be tested, and their results will be adopted as the benchmark. All results are achieved by 1000 Monte Carlo runs.
We set three targets in the simulations and the nearly constant velocity model was used to describe the motions: where presents the state vector of target i at time k, and the components are the position and velocity along the x and y coordinates, respectively. F is the state transition matrix, and ω i k is the unmolded noise. We set the sampling period T= 1 second in the tests.
We set four sensors in the simulations, and they were independent to each other. The target-derived observation was modeled by the following equation: where z j k is the observation achieved by sensor j at time k. H is the observation matrix, and v j k is the observation error, which is Gaussian and white. For i = j, v i k is independent to v j k . The four sensors have the same observation error covariance matrixes R.
The clutter-derived observation is described by Equation (3) in Section 2. In the simulations, we set the tracking space as a [−1000, 1000] × [−1000, 1000] plane. Therefore, the space volume was Ψ(x)= 4 × 10 6 here. The clutter intensity λ j for each sensor depended on the specific scene, and this will be explained in the following part.
We set four tracking scenes with different detection rates and clutter intensities settings, as described in Table 1. Here, we used P i d and λ i to present the detection rate and clutter intensity of sensor i, respectively. We set the same detection rate and clutter intensity for all sensors in scene one. In scene two, we had different detection rates for the four sensors. Specifically, the first sensor had the highest detection rate as P 1 d = 0.9, and the fourth sensor had the lowest detection rate as P 1 d = 0.6. In scene three, the detection rate was same for all sensors, while the clutter intensities were different. Specifically, sensor one had the weakest clutter intensity as λ 1 = 20, and sensor four had the strongest clutter intensity as λ 4 = 80. In scene four, the detection rates and clutter intensities were different for the four sensors, where sensor one had the highest detection rate and the weakest clutter intensity, and sensor four had the lowest detection rate and the strongest clutter intensity. Therefore, it can be roughly considered that the four sensors had the same quality in scene one. In the other three scenes, we had an order with respect to the sensor quality (from good to bad), where "sensor one → sensor two → sensor three → sensor four". Table 1. Four tracing scenes with different detection rates and clutter intensity settings.

Detection Rate Clutter Intensity
Scene one  Figure 3, as the four sensors in scene one had the same detection rate and clutter intensity. In the other three scenes, the targets' motions were the same as in scene one, while the detection rates and clutter intensities were different. The details are shown in Table 1.   Tables 2 and 3. The target detection rates and clutter intensities were the same for the four sensors in this scene, thus the OSPA and target number estimation results for the two SGM-PHD algorithms were very similar. The proposed algorithms achieved obvious improvements over the SGM-PHD and SMFMGM-PHD algorithms. Besides, we noted that the USFMGM-PHDA and USFMGM-PHDB algorithms outperformed the SFMGM-PHD algorithm, as the first two algorithms adopt the unbalanced weighted fusing method (see Section 4.2). As the data qualities of four sensors were similar in scene one, the estimated OCVs of the four sensors were identical, and the fusing sequences with different orders led to similar results. Therefore, although the USFMGM-PHDB has the mechanism to sort the multi-sensor fusing sequence (see Section 4.1), the results of   Tables 2 and 3. The target detection rates and clutter intensities were the same for the four sensors in this scene, thus the OSPA and target number estimation results for the two SGM-PHD algorithms were very similar. The proposed algorithms achieved obvious improvements over the SGM-PHD and SMFMGM-PHD algorithms. Besides, we noted that the USFMGM-PHDA and USFMGM-PHDB algorithms outperformed the SFMGM-PHD algorithm, as the first two algorithms adopt the unbalanced weighted fusing method (see Section 4.2). As the data qualities of four sensors were similar in scene one, the estimated OCVs of the four sensors were identical, and the fusing sequences with different orders led to similar results. Therefore, although the USFMGM-PHDB has the mechanism to sort the multi-sensor fusing sequence (see Section 4.1), the results of USFMGM-PHDA and USFMGM-PHDB are quite similar. The OSPAs, OCVs, and the target number estimations of scene one are shown in Figures 4-6, and Tables 2 and 3. The target detection rates and clutter intensities were the same for the four sensors in this scene, thus the OSPA and target number estimation results for the two SGM-PHD algorithms were very similar. The proposed algorithms achieved obvious improvements over the SGM-PHD and SMFMGM-PHD algorithms. Besides, we noted that the USFMGM-PHDA and USFMGM-PHDB algorithms outperformed the SFMGM-PHD algorithm, as the first two algorithms adopt the unbalanced weighted fusing method (see Section 4.2). As the data qualities of four sensors were similar in scene one, the estimated OCVs of the four sensors were identical, and the fusing sequences with different orders led to similar results. Therefore, although the USFMGM-PHDB has the mechanism to sort the multi-sensor fusing sequence (see Section 4.1), the results of USFMGM-PHDA and USFMGM-PHDB are quite similar.          The OSPA, OCV, and the target number estimations of scene two are shown in Figures 7-9, Tables 4 and 5. The OSPA, OCV, and target number estimations of scene three are shown in Figures 10-12, and Tables 6 and 7. The OSPA, OCV, and the target number estimations of scene four are shown in Figures 13-15, and Tables 8 and 9. In the above three scenes, we set different observation capacities for the four sensors. Specifically, we set the same clutter intensities and different detection rates in scene two. In scene three, we set the same detection rate and different clutter intensities. Also, in the last scene, both the detection rates and clutter intensities were different. For the sake of convenience, we set sensor one as the best sensor, and sensor four as the worst one in the above three scenes. As shown in Figures 7-15 and Tables 4-9, the three proposed algorithms outperformed the best SGM-PHD algorithm (sensor one) and the SMFMGM-PHD in both the OSPA and target number estimations. Among the three proposed algorithms, the USFMGM-PHDB was the best one, and the worst one was SFMGM-PHD. From Figures 8, 11 and 14, we see the estimated OCV reflected the sensor quality accurately; therefore the USFMGM-PHDB could obtain an optimized fusing order, based on the estimated OCV, and provide the best tracking results. Among the above three scenes, the most challenging one was scene four, and the easiest one was scene three (shown in Tables 4, 6 and 8). Tables 6 and 8 further show that the OSPA deteriorative degree of the USFMGM-PHDB algorithm was smaller than the deteriorative degree of the SFMGM-PHD algorithm. Specifically, the average OSPAs of the SFMGM-PHD algorithm in scenes three and four were 11.9580 and 14.1943, with an increase of OSPA at 15.75%; the average OSPAs of the USFMGM-PHDB algorithm in scenes three and four were 10.2914 and 12.0674, with the increase of the OSPA at 14.71%. Comparing Table 7 with Table 9, we can see the target number estimation results of the USFMGM-PHDB algorithm in scene four were even better than the results in scene three. Specifically, the average TNE deviation of the SFMGM-PHD algorithm in scene three and four are 0.0931 and 0.0992 (Tables 7 and 9), with an increase of 6.55%. The average TNE deviations of the USFMGM-PHDB algorithm in scenes three and four were 0.0576 and 0.0527 (Tables 7 and 9), with a decrease of 8.50%. In addition, the average OSPA of the USFMGM-PHDA algorithm in scenes two to four was 11.4098. The average OSPA of the USFMGM-PHDB algorithm in scene two to four was 10.8954. As we can see, in the last three scenes, the USFMGM-PHDB algorithm obtained an average OSPA decrease, and the USFMGM-PHDA algorithm was 4.51%. The average TNE deviations of the USFMGM-PHDA and USFMGM-PHDB algorithms were 0.0827 and 0.0594, respectively. Also, the USFMGM-PHDB algorithm obtained a TNE that was lower than the USFMGM-PHDA algorithm, at 28.17%. In general, the USFMGM-PHDA&B algorithms are better than the SFMGM-PHD algorithm, as the unbalanced weighted fusing method is adopted by the former two algorithms. Besides, the USFMGM-PHDB algorithm sorts the multi-sensor fusing sequences from the best sensor to the worst sensor, based on the OSPA consistency metric. Then, the fully utilized information will come from the best sensor rather than the worst one for the USFMGM-PHDB algorithm. As a consequence, USFMGM-PHDB shows better results than USFMGM-PHDA when the multiple sensors' qualities are different.
general, the USFMGM-PHDA&B algorithms are better than the SFMGM-PHD algorithm, as the unbalanced weighted fusing method is adopted by the former two algorithms. Besides, the USFMGM-PHDB algorithm sorts the multi-sensor fusing sequences from the best sensor to the worst sensor, based on the OSPA consistency metric. Then, the fully utilized information will come from the best sensor rather than the worst one for the USFMGM-PHDB algorithm. As a consequence, USFMGM-PHDB shows better results than USFMGM-PHDA when the multiple sensors' qualities are different.                                  We give the complexity analysis for the SFMGM-PHD, USFMGM-PHDA and USFMGM-PHDB algorithms. It is assumed that there are s sensors. At a particular tracking time stamp, the average Gaussian Mixture number of one sensor is assumed to be J, and the average observation number of one sensor is assumed to be r. We assume that the average estimated target number of one sensor at a particular tracking time stamp is n. Then, the computing complexities of the above-mentioned algorithms are shown in Table 10. As we can see, the SFMGM-PHD and USFMGM-PHDA algorithms have the same computing complexity, and the complexity of the USFMGM-PHDB algorithm is a bit larger than the other two.

Conclusions
Three sequential fusing multi-sensor GM-PHD algorithms (SFMGM-PHD, USFMGM-PHDA&B) are proposed. First, we propose a multi-sensor posterior GM estimation sequential fusing framework by constructing the SFMGM-PHD algorithm. Then, we analyze the deficiencies of the standard SFMGM-PHD algorithm. (1) The multi-sensor GM estimations are fused with the balanced way that the information of the later fused sensor will be overused. (2) The problem of fusion sequence sorting is not considered and the fused results may be affected by the fusing sequence order. Therefore, the methods of unbalanced weighted fusing and fusion sequence sorting are further proposed to derive the USFMGM-PHDA&B algorithms. The simulation results show that the proposed algorithms can achieve better tracking results than the SGM-PHD algorithm of the best sensor and the SMFMGM-PHD algorithm in the testing scenes. Besides, the two USFMGM-PHD algorithms show better results than the SFMGM-PHD algorithm, and the best one is the USFMGM-PHDB algorithm.
There are two potential directions of future work. The first one is that we may calculate the unbalanced multi-sensor fusing weight by other ways. The second one is to find other metrics to sort the multi-sensor fusing sequence.

Conflicts of Interest:
The authors declare no conflict of interest.