Moving Point Density Estimation Algorithm Based on a Generated Bayesian Prior

Abstract

To improve decision making, real-time population density must be known. However, calculating the point density of a huge dataset in real time is impractical in terms of processing time. Accordingly, a fast algorithm for estimating the distribution of the density of moving points is proposed. The algorithm, which is based on variational Bayesian estimation, takes a parametric approach to speed up the estimation process. Although the parametric approach has a drawback, that is the processes to be carried out on the server are very slow, the proposed algorithm overcomes the drawback by using the result of an estimation of an adjacent past density distribution.

1. Introduction

Population density, namely the number of people per unit area, is one of the most effective factors in decision making involved in tasks, like infrastructure management, city development planning and merchandising [1]. For example, police officers can optimize their patrols in a city on the basis of high-population density locations, because accidents often occur at those locations in cities. A dataset of people’s positions, consisting of “moving points”, as the example in Figure 1a, can be used to calculate the population density. Figure 1a was drawn by plotting the results of questionnaires about people’s journeys (namely, data pairs consisting of origination and destination pairs of the travels) during a whole day on a base map [2]. The drawing thus represents people’s positions at the moment in the past. In contrast, the people’s positions measured in real time through mobile phones with GPS-like positioning functions [3,4] reflect people’s movements. In that case, the people’s positions are thus recorded in real time as moving points. Such massive data about people’s positions should be commonly used, because they should be taken into account for many decisions, such as devising patrol plans for police officers. The OGC (Open Geospatial Consortium [5]), which is an international standard organization concerned with geospatial information systems, publishes standard specifications applicable to such data. A standard for encoding such massive moving-point data is defined as “OGC Moving Features encoding” [6], which is defined for exchanging massive amounts of moving-point data easily.

As mentioned above, real-time moving-point data are useful for many applications; especially, the distribution function of the density of moving points (hereafter “point density”), expressed as a function of position (x, y) and time t, is the most effective feature for describing the tendency of the moving points. However, handling a massive point dataset itself is not so easy, even if it is exchangeable. Therefore, a conversion from massive moving-point data to easily handleable data is needed.

“Coverage data,” which describes the point density, are one of the most easily handleable kinds of data. The coverage can be encoded by various encoding standards, like OGC NetCDF [7] and GML [8]. Figure 1b shows a population density distribution visualized from the points shown in (a). Such an intelligible image colored according to density, often called a “heatmap,” can be generated automatically (Figure 1b is generated by a quadratic kernel density estimator shown in Section 2.) After visualization, the heatmaps can be encoded by general image formats, like PNG and JPG. For distributing such images, an OGC WMS (Web Map Service) [9] interface is commonly used. Accordingly, a conversion from point data to a coverage would contribute to system interoperability.

Counting the number of the points is the most naive technique for estimating the density of stationary points. Namely, a space in which points exist is separated into small cells, and the number of points in each cell is counted. Such a method is called a histogram method [10]. However, the histogram method has several problems. One of the problems is the trade-off between the size of the cell (“cell size”) and the number of data. Cell size can be adjusted in the case that the hierarchical cell structure, like a “quadtree” [11], is adapted. It should be smaller if the point density is required to be finer; however, statistical dispersion of the estimation results becomes higher if the cells are small, because smaller cells have fewer points. To estimate a finer point density, a huge dataset is therefore required. For that reason, several conventional algorithms for accurately estimating the point density, even with small cells, have been proposed. The reciprocal of the cell size is called “resolution”; that is, the resolution of a point density using small cells is “high.”

Another problem with histograms arises in practical data-providing systems. An example of a client-server system using point density is shown in Figure 2. The data source periodically provides a real-time point dataset to the analysis server. The server estimates the point density and sends it to user-client systems A and B. The user clients use the point density by associating it with other information, such as temperature distribution and map illustrations. In this case, the server is basically a “high-spec” computer; in contrast, the user clients are generally “low-spec” computers. For such systems, OGC standards are available. Namely, OGC WMS [9] is a service interface for distributing map images, and OGC WCS (Web Coverage Service) [12] is a service interface for distributing coverage data. They provide web APIs to send the point density data. WMS-request data sent from user-client systems include the area boundary of the required map and the map-image size, which determine the image resolution. The resolution required by user-client A might be different from that required by user-client B; besides, the required resolution dynamically changes according to the user requirements. The point density must thus be dynamically calculated.

The computational power of the user client system is generally weaker than that of the server. Furthermore, the point dataset itself may not be distributed, because the amount of data provided by the server should be small enough to be sent via a network with low throughput. Many calculations of point density with various resolutions should therefore be carried out on the server in real time. Accordingly, a fast on-line algorithm for calculating point densities is needed.

To satisfy the requirements mentioned above, in this work, a “point-density-estimation algorithm” for handling a moving-point dataset (described with OGC Moving Features) is proposed. The proposed algorithm, which is based on Bayesian estimation, can estimate point density (distributed with OGC WMS or WCS) by using only a partial dataset; consequently, it works faster than conventional algorithms.

2. Related Works and Contributions

2.1. Trajectory Analysis and Density Estimators

An analysis of tracks of moving points is called “trajectory analysis”, namely trajectory smoothing [13] (for reducing errors in positioning), trajectory clustering [14–17] (for retrieving similar trajectories from a database), prediction of movement [18,19], representative path extraction [15,17,20–22] and an algorithm for labeling positioning data [23].

These trajectory analyses have been developed for extracting information from stored trajectories, not points. Basically, the density of moving points does not depend on the movement of points; that is, the density can be calculated by using only a “snapshot” of the moving points. With the proposed algorithm, however, the density estimation is accelerated by using the “adjacent past density distribution”, which is based on the similarity between the current point density and the most recent one.

Kernel-density estimators [24,25] are popular methods for estimating a point density. The density calculated by a kernel-density estimator is the summation of distributions around the points. The Gaussian kernel-density estimator, which is the most general type, estimates the point densities as a summation of Gaussians. The density P(x) is given as:

$$P(\mathrm{x})=\frac{1}{N}{\displaystyle \sum _n^N\frac{1}{2\pi h^2}}\exp \left(-\frac{|\mathrm{x}-{\mathrm{x}}_n{|}^2}{2h^2}\right)$$

(1)

where N points are denoted as {x_n} and h corresponds to the radius of each Gaussian. A quadratic kernel-density estimator, which estimates points densities as a summation of quadratic functions as:

$$P(\mathrm{x})=\frac{1}{N{h}^2}{\displaystyle \sum _n^{N}Q\left(\frac{|\mathrm{x}-{\mathrm{x}}_n|}{h}\right)},Q(r)\equiv \{\begin{array}{ll}\frac{6}{\pi }{(1-r)}^2\hfill & r<1\hfill \\ 0\hfill & r\ge 1\hfill \end{array}$$

(2)

is often used. A quadratic kernel estimator works faster than a Gaussian kernel estimator. On the other hand, the accuracy of a quadratic kernel estimator is generally less than that of a Gaussian kernel estimator. Accordingly, a quadratic kernel estimator is often applied to visualize density in the form of a heatmap. Several kernel-density estimators for summarizing the tendency of moving points have been proposed; as examples, the dual kernel-density estimator [26] is designed to show differences between two different point densities; and the directional kernel-density estimator [27] visualizes the tendency of the movement itself. These kernel-density estimators use all points to estimate the point density. Thus, the calculation of P(x) in the density estimation becomes more complex in proportion to the number of points. Consequently, it is difficult to carry out the calculation in real time. This kind of estimator is called “nonparametric”, because the parameters used in the model are not determined by the estimation process.

Another approach, called “parametric”, is based on a specific functional formula. The parameters of the functional formula are optimized to make them consistent with the dataset. The calculation time taken by the parametric approach is proportional to the complexity of the function. Thus, even if the number of points is large, the density estimation does not take a long time. However, optimization of the parameters takes a very long time. For example, a standard variational Bayesian estimation, which is based on a Gaussian mixture model (GMM), is often applied as a probability distribution of the point dataset. In the case of a variational Bayesian estimation, the distribution modeling is a convergence that makes parameters consistent with all point data; that is, the calculations are iterated. Although the distribution modeling can be carried out by the server, the processing time is too long for real-time calculation.

2.2. Contributions

Even though the proposed algorithm takes a parametric approach, its parameter optimization is very fast, because the point density is generated by updating the adjacent past point density. Measurement by GPS-like positioning functions is generally periodic, so the proposed algorithm handles discrete time (a period of which equals a time discretization unit). Indices of the discrete times are denoted by integers ···t, t+1, ···. Under the assumption that the point density at time t is expressed as the sum of Gaussian components, the point density at t + 1 is considered as the sum of the Gaussian components similar to the Gaussian components at time t. The difference between the point density at t and that at t + 1 is predictable even if the point dataset is partially used.

The proposed algorithm is based on the update of the Gaussian components by using a partial point dataset. As the mechanism for updating the Gaussian components, Bayesian estimation is applied. In the case of Bayesian estimation, a probability-distribution function is assumed a priori and revised by using the point dataset. Bayesian estimation thus seems applicable for updating the probability-distribution function; however, to apply Bayesian estimation, a trick to determine parameters used by the assumed probability-distribution function is needed. In the following section, the trick named “responsibility reduction” is described.

3. Proposed Algorithm

3.1. Variational Bayesian Estimation

Variational Bayesian estimation is often applied to estimate a multivariate probability-distribution function. An argument of the multivariate probability-distribution function is replaced by a pair of coordinate values of a point as (x, y). The N-points density, which is obtained from point probability distribution P(x, y), is calculated as NP(x, y).

A probability distribution of parameters used in the probability-distribution function is defined in the variational Bayesian estimation. The probability-distribution function of the parameter set is called the “prior probability-distribution function” (“prior”). The prior and additional observed points (“observation”) derive the estimated “posterior probability distribution” (“posterior”) in the variational Bayesian estimation. The probability-distribution function of the GMM is denoted by:

$$P(\mathrm{x}|\{{\pi }_k\},\{{\mathrm{\mu }}_k\},\{{\mathrm{\Lambda }}_k\})={\displaystyle \sum _k{\pi }_k\mathcal{N}(\mathrm{x}|{\mathrm{\Lambda }}_k,{\mathrm{\mu }}_k)}$$

(3)

where x is a vector consisting of an observation’s coordinate values (i.e., (x,y)), $\mathcal{N}(\mathrm{x}|\mathrm{\Lambda },\mu )$ is a two-dimensional Gaussian function with a correlation matrix denoted by Λ_k and an average denoted by μ_k, and π_k is the mixing ratio of the k-th component to the sum of all components. The prior for {π_k}, {(μ_k} and {Λ_k} is then defined as:

$$P(\{{\pi }_k\},\{{\mathrm{\mu }}_k\},\{{\mathrm{\Lambda }}_k\}|W_0,{\upsilon }_0,{\mathrm{\beta }}_0)=\mathrm{Dir}(\{{\pi }_k\}|{\alpha }_0){\displaystyle \prod _k\mathcal{N}({\mathrm{\mu }}_k|{({\mathrm{\beta }}_0{\mathrm{\Lambda }}_k)}^{-1},{\mathrm{m}}_0)\mathcal{W}(}{\mathrm{\Lambda }}_k|W_0,{\upsilon }_0)$$

Here, α₀, W₀, υ₀ and β₀ are parameters, called “hyperparameters,” of the prior. $\mathcal{W}(\cdot |W.v)$ is the Wishart distribution with deviation W and degrees of freedom υ, and Dir(·|cα) is the Dirichlet distribution with parameter α. The prior is conjugate to the Gaussian mixture; that is, the posterior formula updated by observations is the same as that of the prior. Estimation of the posterior is therefore implemented as the update of hyperparameters α₀, β₀, m₀ and W₀ by using observations {x_n} as:

$${\alpha }_k={\alpha }_0+N_k$$

(4)

$${\mathrm{\beta }}_k={\mathrm{\beta }}_0+N_k$$

(5)

$${\mathrm{m}}_k=\frac{1}{{\mathrm{\beta }}_k}\left({\mathrm{\beta }}_0{\mathrm{m}}_0+N_k{\overline{\mathrm{x}}}_k\right)$$

(6)

$$W_k^{-1}=W_0^{-1}+S_k{N}_k+\frac{N_k{\mathrm{\beta }}_0}{N_k+{\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_k){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_k)}^T$$

(7)

$${\upsilon }_k={\upsilon }_0+N_k$$

(8)

N_k, x_k and S_k are respectively calculated as:

$$N_k={\displaystyle \sum _n{r}_{n,k,}}{\overline{\mathrm{x}}}_k=\frac{1}{N_k}{\displaystyle \sum _n{r}_{n,k}{\mathrm{x}}_{n,}{S}_k={\displaystyle \sum _n{r}_{n,k}({\mathrm{x}}_n-{\overline{\mathrm{x}}}_k){({\mathrm{x}}_n-{\overline{\mathrm{x}}}_k)}^{\mathrm{T}}}}$$

(9)

r_n,_k, “responsibility”, is calculated as:

$$r_{n,k}=\frac{{\rho }_{n,k}}{{\displaystyle \sum {}_n{\rho }_{n,k}}},{\rho }_{n,k}\equiv {\tilde{\pi }}_k|{\overline{\mathrm{\Lambda }}}_k{|}^{\frac{1}{2}}{e}^{[-\frac{{\mathrm{\beta }}_k}{2}-\frac{1}{2}{({\mathrm{x}}_n-{\mathrm{m}}_k)}^T{w}_k({\mathrm{x}}_n-{\mathrm{m}}_k)]}$$

(10)

$$\mathrm{l}{\mathrm{n}\tilde{\pi }}_k\equiv \ln \mathrm{\Psi }({\alpha }_k)-\ln \mathrm{\Psi }({\displaystyle \sum _k{\alpha }_k),\ln |{\overline{\mathrm{\Lambda }}}_k|\equiv \ln \mathrm{\Psi }(\frac{{\upsilon }_k}{2}})+\ln \mathrm{\Psi }(\frac{{\upsilon }_k-1}{2})+\mathrm{l}{\mathrm{n}4|W}_k|$$

(11)

where ψ (•) is a digamma function. According to Equation (9)r_n,_k is interpreted as the contribution coming from the n-th observation to each component labeled with k.

The calculation of the responsibility requires the hyperparameters; however, the estimation of the hyperparameters also requires the responsibility. Accordingly, to optimize the hyperparameters, the above-described processes are iterated; first, the hyperparameters are randomly initialized; second, the responsibilities are calculated with the initial hyperparameters; third, the hyperparameters are updated with the calculated responsibility. After iterating the calculation, the optimized hyperparameters are obtained.

3.2. KL Divergence as a Criterion for Convergence

In the variational Bayesian estimation, the initial hyperparameters α₀, β₀, m₀ and W₀ are given α priori. In other words, the initial hyperparameters, which specify the prior, can be determined to make the convergence of the estimation results earlier. If the prior is set to a similar function as the posterior, most of the processes of the point density estimation are unnecessary. Accordingly, most of the point dataset has little effect on the point density; that is, most of the point dataset can be ignored. Moreover, the prior is accurately predictable as the adjacent past posterior, which was estimated at the previous iteration, because the point density of moving points does not change so much in a short period. The prior should therefore be set to the same function as the adjacent past posterior in order to stop the point density estimation early.

Kullback–Leibler (KL) divergence, which behaves like the distance between distribution functions, is applicable as a criterion for judging whether the calculation can be stopped or not.

Figure 3 illustrates such a judgment process. The point dataset is separated into small blocks in advance, where the block size is a constant parameter during the process. Note that the block size should be large enough to detect statistical significance. In Step 1, the prior is generated from the adjacent past posterior. In Step 2, to estimate the posterior, one of the small blocks (labeled “1”) is input into the prior. In Step 3, the next prior, which is generated by the estimated posterior in Step 2, is updated with the block labeled “2”. In this step, KL divergence from the posterior generated in Step 2 to the posterior generated in Step 3 is calculated. If the KL divergence is small enough, additional blocks are judged unnecessary. If not, the posterior is similarly calculated, and so on. In the evaluation of KL-divergence, the number of the input data is controlled; that is, few data are used when the point density changes a little, but many data are used when the point density changes drastically.

The KL divergence from p(x) to q(x) KL(p(x)||q(x)) is defined as:

$$KL(p(x)||q(x))={\displaystyle \int p(x)\ln }\frac{q(x)}{p(x)}\mathrm{d}x$$

(12)

The proposed algorithm requires the KL-divergence from the prior to the posterior. That is,

$$K\equiv KL(p(\{{\mathrm{\theta }}_i\}|{\{{\mathrm{x}}_n\}}_{m+1})||p(\{{\mathrm{\theta }}_i\}|{\{{\mathrm{x}}_n\}}_m))$$

(13)

where {θ_i} = {{π_k}, {Λ_k}, {(μ_k}}, and {x_n}_m is the m-th data block. If K is small enough, the estimation result is considered to be converged. p({θ_i}|{x_n}_m) can be replaced by q({θ_i}|{x_n}_m) = q({π_k, Λ_k, μ_k}|{α_k, β_k, m_k, υ_k, W_k}), because it is the result of the variational Bayesian estimation. K is thus transformed to:

$$\begin{array}{l}K={\displaystyle \int q(\{{\mathrm{\theta }}_i\}|\{{\mathrm{\alpha }}_k,{\mathrm{\beta }}_k,{\mathrm{m}}_k,{\upsilon }_k,W_k\})\ln \mathrm{q}(\{{\mathrm{\theta }}_i\}|\{{\alpha }_k^{\prime },{\mathrm{\beta }}_k^{\prime },{\mathrm{m}}_k^{\prime },{\upsilon }_k,W_k^{\prime }}\}){\displaystyle \prod _i\mathrm{d}{\mathrm{\theta }}_i}\\ =\ln {\alpha }_0+{\displaystyle \sum ({\mathrm{\alpha }}_0-1)\ln {\tilde{\pi }}_k}-\ln {\mathrm{\beta }}_k^{\prime }-\frac{{\mathrm{\beta }}_k^{\prime }}{2}\left(\frac{2}{{\mathrm{\beta }}_k}+{\upsilon }_k{({\mathrm{m}}_k-{\mathrm{m}}_k^{\prime })}^{\mathrm{T}}{\mathrm{W}}_k({\mathrm{m}}_k-{\mathrm{m}}_k^{\prime })\right)\\ +\ln B(W_k^{\prime },{\upsilon }_k^{\prime })+\frac{1}{2}({\upsilon }_k^{\prime }-1)\ln |{\overline{\mathrm{\Lambda }}}_k|-{\displaystyle \sum _{i,j}\frac{W_{i,j}^{\prime -1}}{W_{i,j}^{-1}}}\end{array}$$

(14)

where {α_k, β_k, m_k, υ_k, W_k}) is the set of hyperparameters updated by {x_n}_m and { ${\alpha }_k^{\prime },{\text{β}}_k^{\prime },{\text{m}}_k^{\prime },{\upsilon }_k,W_k^{\prime }$} is the set of hyperparameters updated by ${\left\{{\mathrm{x}}_n^{\prime }\right\}}_{m+1}$. Here, B(•) is defined as:

$$B(W,\upsilon )\equiv |W{|}^{-\frac{1}{2}}{\left(2^{\upsilon }{\mathrm{\pi }}^{\frac{1}{2}}\mathrm{\Gamma }(\frac{\upsilon }{2})\mathrm{\Gamma }(\frac{\upsilon -1}{2})\right)}^{-1}$$

(15)

3.3. Responsibility Reduction

With the proposed method, a trick named “responsibility reduction” is applied to determine the hyperparameters. For example, as shown in Equation (5), β_t,k = β₀ + N_t,k, where N_t,k is the sum of the responsibilities, which are contributed by the k-th component of the GMMat time t. Thus, N_t,k corresponds to the number of points belonging to the k-th component, because responsibility r_n,k indicates the i-th point’s “belongingness” to the k-th component. Then, β_t,k is updated with N_t+1,k as β_t+1,k = β₀ + N_t,k + N_t+1,k, where β_t+1,k is a hyperparameter of a posterior at t + 1. Similarly,

$${\mathrm{\beta }}_{t+n,k}={\mathrm{\beta }}_0+N_{t,k}+N_{t+1,k}+\cdots +N_{t+n,k}$$

(16)

The hyperparameter at time t + n thus gets larger if n gets larger. Consequently, the contribution of the newest point dataset, N_t+n,_k, is relatively smaller than the contribution of the prior, β₀, because the prior includes the sum of the all past responsibilities. However, the contribution of the newest point dataset should be greater than the others.

The problem described in the preceding paragraph comes from confusion between point datasets. The new point density is different from the past point density; therefore, the past points and new points should not be equally used. Accordingly, the prior should be generated from the past posterior, but the prior should not be equal to the posterior. The effects of past points on the hyperparameters should thus be removed. If the effects from the hyperparameters are equal to the effects produced by the new points, the effects of points in the hyperparameters are kept constant. The dataset at time t is separated into M blocks containing enough points for inputting sequentially, where the number of points in the blocks are denoted by $n_t^{(1)},n_t^{(2)},\cdots ,n_t^{(i)},\cdots ,n_t^{(M)}$ These blocks are input sequentially to update the posterior. When the j-th block is input, the ratio of the number of input points to the total number of points is calculated as $\frac{{\displaystyle \sum {}_i^j{n}_t^{(i)}}}{{\displaystyle \sum {}_i^M{n}_t^{(i)}}}$. The reduction rate of responsibilities γ^(j)(t) is then defined as:

$${\mathrm{\gamma }}_t^{(j)}=1-\frac{{\displaystyle \sum {}_i^j{n}_t^{(i)}}}{{\displaystyle \sum {}_i^M{n}_t^{(i)}}}$$

(17)

The sum of the past responsibility N_t,k is multiplied by the reduction rate. By this process, the total number of points contributing to the hyperparameters is adjusted and kept constant.

The updating mechanism is figuratively drawn as an “evaporating pot” in Figure 4. In Step (a), the pot is filled by a liquid, labeled t. The content of the pot is then evaporated so that half of its capacity remains. Next, a new liquid, labeled t + 1, is poured into the pot. The volume of the poured liquid is half of the pot capacity. In step (b), the contents of the pot are again evaporated to half its capacity. The liquid labeled t and the liquid labeled t + 1 are reduced equally to half of their initial volumes. That is, the volumes of the liquids labeled t and t + 1 become 1/4 of the pot capacity. Then, repeatedly, a new liquid, labeled by t + 2, is poured into the pot, which is half full of the liquid. After repeating the process, as shown in Steps (c) and (d), the ingredients in the pot are a mixture of liquids labeled t, t + 1, …. The volume of liquid labeled t′ is therefore ${\left(\frac{1}{2}\right)}^{t^{\prime }-t+1}$ V, where V is the pot capacity. Thus, the volume of liquid will be decreased exponentially.

The liquid in Figure 4 corresponds to the sum of responsibilities; in other words, evaporation corresponds to multiplication to the sum of responsibilities by the reduction rate. Accordingly, the contribution from the old dataset is decreased exponentially by multiplying it by the reduction rate, while the sum of all responsibilities is kept constant. By the multiplication, the hyperparameters in the prior at time t + 1 given by the posterior at time t are derived as follows.

$${\mathrm{\alpha }}_{0,t+1}^{(0)}={\mathrm{\alpha }}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}$$

(18)

$${\mathrm{\beta }}_{0,t+1}^{(0)}={\mathrm{\beta }}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}$$

(19)

$${\mathrm{m}}_{0,t+1}^{(0)}=\frac{1}{{\mathrm{\beta }}_{0,t+1}}({\mathrm{\beta }}_0{\mathrm{m}}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}{\overline{\mathrm{x}}}_{t,k})$$

(20)

$$W_{0,t+1}^{-1(0)}=W_0^{-1}+{\mathrm{\gamma }}_t^{(0)}{S}_{t,k}{N}_{t,k}+\frac{{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}{\mathrm{\beta }}_0}{{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}+{\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k})}^{\mathrm{T}}$$

(21)

$${\upsilon }_{0,t+1}^{(0)}={\upsilon }_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}$$

(22)

Here, N_t,k, ${\overline{\mathrm{x}}}_{t,k}$ and S_t,k at time t are given by Equation (9). The hyperparameters in the posterior at time t + 1 are given by:

$${\mathrm{\alpha }}_{t+1,k}^{(0)}={\mathrm{\alpha }}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}+n_{t,k}^{(0)}$$

(23)

$${\mathrm{\beta }}_{t+1,k}^{(0)}={\mathrm{\beta }}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}+n_{t,k}^{(0)}$$

(24)

$${\mathrm{m}}_{t+1,k}^{(0)}=\frac{1}{{\mathrm{\beta }}_{t+1,k}^{(0)}}({\mathrm{\beta }}_0{\mathrm{m}}_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}{\overline{\mathrm{x}}}_{t,k}+n_{t,k}^{(0)}{\overline{\mathrm{x}}}_{t,k}^{(0)})$$

(25)

$$\begin{array}{l}{W}_{t+1,k}^{-1(0)}=W_0^{-1}+{\mathrm{\gamma }}_t^{(0)}{S}_{t,k}{N}_{t,k}+s_{t,k}^{(0)}{n}_{t,k}^{(0)}+\frac{{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}{\mathrm{\beta }}_0}{{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}+{\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k})}^{\mathrm{T}}\\ +\frac{n_{t,k}^{(0)}{\mathrm{\beta }}_0^{(0)}}{n_{t,k}^{(0)}+{\mathrm{\beta }}_0^{(0)}}({\mathrm{m}}_{0,t}^{(0)}-{\overline{\mathrm{x}}}_{t,k}^{(0)}){({\mathrm{m}}_{0,t}^{(0)}-{\overline{\mathrm{x}}}_{t,k}^{(0)})}^{\mathrm{T}}\end{array}$$

(26)

$${\upsilon }_{t+1,k}^{(0)}={\upsilon }_0+{\mathrm{\gamma }}_t^{(0)}{N}_{t,k}+n_{t,k}^{(0)}$$

(27)

where:

$$n_{t,k}^{(0)}={\displaystyle \sum _n{r}_{t,n,k}^{(0)}},{\overline{\mathrm{x}}}_{t,k}^{(0)}=\frac{1}{n_{t,k}^{(0)}}{\displaystyle \sum _n{r}_{t,n,k}}{\mathrm{x}}_{t,n}^{(0)},s_{t,k}^{(0)}={\displaystyle \sum _n{r}_{t,n,k}^{(0)}({\mathrm{x}}_{t,n}^{(0)}}-{\overline{\mathrm{x}}}_{t,k}^{(0)}){({\mathrm{x}}_{t,n}^{(0)}-{\overline{\mathrm{x}}}_{t,k}^{(0)})}^{\mathrm{T}}$$

(28)

${\mathrm{x}}_{t,n}^{(0)}$ and $r_{t,n,k}^{(0)}$ are points in the n-th block and their responsibilities, respectively.

Another block of the new point dataset is input if the KL-divergence between the posteriors is large. The prior is updated with the i-th block by the following formulas:

$${\mathrm{\alpha }}_{0,t+1}^{(0)}={\mathrm{\alpha }}_0+{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}+{\displaystyle \sum _j^{i-1}{n}_{t,k}^{(j)}}$$

(29)

$${\mathrm{\beta }}_{0,t+1}^{(i)}={\mathrm{\beta }}_0+{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}+{\displaystyle \sum _j^{i-1}{n}_{t,k}^{(j)}}$$

(30)

$${\mathrm{m}}_{0,t+1}^{(i)}=\frac{1}{{\mathrm{\beta }}_{0,t+1}^{(i)}}\left({\mathrm{\beta }}_0{\mathrm{m}}_0+{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}{\overline{\mathrm{x}}}_{t,k}+{\displaystyle \sum _j^{i-1}{n}_{t,k}^{(j)}{\overline{\mathrm{x}}}_{t+1,k}^{(j)}}\right)$$

(31)

$${\displaystyle \begin{array}{l}{W}_{0,t+1}^{-1(i)}=W_0^{-1}+{\mathrm{\gamma }}_t^{(i)}{S}_k{N}_k+\sum _j^{i-1}{s}_{t+1,k}^{(j)}{n}_{t+1,k}^{(j)}+\frac{{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}{\mathrm{\beta }}_0}{{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}+{\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k})}^{\mathrm{T}}\\ +{\displaystyle \sum _j^{i-1}\frac{n_{t+1,k}{\mathrm{\beta }}_{0,t+1}^{(j)}}{n_{t+1,k}+{\mathrm{\beta }}_{0,t+1}^{(j)}}({\mathrm{m}}_{0,t+1}^{(j)}-{\overline{\mathrm{x}}}_{t+1,k}){({\mathrm{m}}_{0,t+1}^{(j)}-{\overline{\mathrm{x}}}_{t+1,k})}^{\mathrm{T}}}\end{array}}$$

(32)

$${\upsilon }_{0,t+1}^{(i)}={\upsilon }_0+{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}+{\displaystyle \sum _j^{i-1}{n}_{t,k}^{(j)}}$$

(33)

To obtain the recursive formula, each side of the i + 1-th formula is subtracted by each side of the i-th formula as follows.

$${\mathrm{\beta }}_{0,t+1}^{(i+1)}=({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}+{\mathrm{\beta }}_{0,t+1}^{(i)}+n_{t,k}^{(i)}$$

(34)

Additionally, ${\mathrm{\beta }}_k^{(i)}={\mathrm{\beta }}_0^{(i)}+n_k^{(i)}$, so:

$${\mathrm{\beta }}_0^{(i+1)}={\mathrm{\beta }}_{t,k}^{(i)}+({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}$$

(35)

Similarly, the following formulas are derived.

$${\mathrm{\alpha }}_{0,t+1}^{(i)}={\mathrm{\alpha }}_{t,k}^{(i)}+({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}$$

(36)

$${\mathrm{m}}_{0,t+1}^{(i)}=\frac{1}{{\mathrm{\beta }}_{0,t+1}^{(i)}}({\mathrm{\beta }}_{t+1,k}^{(i)}{\mathrm{m}}_{t+1,k}^{(i)}+({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}{\overline{\mathrm{x}}}_{t,k})$$

(37)

$$\begin{array}{l}{W}_{0,t+1}^{-1(i)}=W_{t,k}^{-1(i)}+({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}{S}_{t,k}\\ +\left(\frac{{\mathrm{\gamma }}_t^{(i+1)}{N}_{t,k}{\mathrm{\beta }}_0}{{\mathrm{\gamma }}_t^{(i+1)}{N}_{t,k}{\mathrm{\beta }}_0}-\frac{{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}{\mathrm{\beta }}_0}{{\mathrm{\gamma }}_t^{(i)}{N}_{t,k}+{\mathrm{\beta }}_0}\right)({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k})}^{\mathrm{T}}\end{array}$$

(38)

$${\upsilon }_{0,t+1}^{(i)}={\upsilon }_{t,k}^{(i)}+({\mathrm{\gamma }}_t^{(i+1)}-{\mathrm{\gamma }}_t^{(i)})N_{t,k}$$

(39)

The i + 1-th prior can therefore be calculated from ${\mathrm{\alpha }}_{t,k}^{(i)},{\mathrm{\beta }}_{0,t+1}^{(i)},{\mathrm{m}}_{t+1,k}^{(i)},W_{t,k}^{-1(i)},{\upsilon }_{t,k}^{(i)},{\mathrm{\gamma }}_t^{(i+1)}$ and ${\mathrm{\gamma }}_t^{(i)}$. After iterating this update computation until KL divergence becomes small enough, the estimated hyperparameters at time t + 1 is obtained. When KL divergence is converged at i = l, the hyperparameters of the next prior are written as:

$${\mathrm{\alpha }}_{0,t+1,k}={\mathrm{\alpha }}_0+N_{t+1,k}$$

(40)

$${\mathrm{\beta }}_{0,t+1,k}={\mathrm{\beta }}_0+N_{t+1,k}$$

(41)

$${\mathrm{m}}_{0,t+1,k}=\frac{1}{{\mathrm{\beta }}_{0,t+1,k}}({\mathrm{\beta }}_0{\mathrm{m}}_0+N_{t+1,k}{\overline{\mathrm{x}}}_{t+1,k})$$

(42)

$$W_{0,t+1,k}^{-1}=W_0^{-1}+S_{t+1,k}{N}_{t+1,k,}+\frac{N_{t+1,k}{\mathrm{\beta }}_0}{N_{t+1,k}{+\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k})}^{\mathrm{T}}$$

(43)

$${\upsilon }_{0,t+1,k}={\upsilon }_0+N_{t+1,k}$$

(44)

by $N_{t+1,k},{\overline{\mathrm{x}}}_{t+1,k}$ and S_t+1,k, which are defined as:

$$N_{t+1,k}={\mathrm{\alpha }}_{t,k}^{(l)}-{\mathrm{\alpha }}_0={\mathrm{\beta }}_{t,k}^{(l)}-{\mathrm{\beta }}_0$$

(45)

$${\overline{\mathrm{x}}}_{t+1,k}=\frac{1}{N_{t+1,k}}({\mathrm{\beta }}_{t+1,k}^l{\mathrm{m}}_{t+1,k}^l-{\mathrm{\beta }}_0{\mathrm{m}}_0)$$

(46)

$$S_{t+1,k}=\frac{1}{N_{t+1,k}}(W_{t,k}^{-1(l)}-W_0^{-1}-\frac{N_{t+1,k}{\mathrm{\beta }}_0}{N_{t+1,k}{\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k})}^{\mathrm{T}}$$

(47)

The updated formula at t + 2 is similarly derived. Note that $N_{t+1,k},{\overline{\mathrm{x}}}_{t+1,k},S_{t+1,k}$, can be converted to the following formulas.

$$N_{t+1,k}={\mathrm{\gamma }}_t^{(l)}{N}_{t,k}+{\displaystyle \sum _j^l{n}_{l+1,k}^{(j)}}$$

(48)

$${\overline{\mathrm{x}}}_{t+1,k}={\mathrm{\gamma }}_t^{(l)}{N}_{t,k}{\overline{\mathrm{x}}}_{t,k}+{\displaystyle \sum _z^l{n}_{t+1,k}^{(j)}}{\overline{\mathrm{x}}}_{t+1,k}^{(j)}$$

(49)

$$S_{t+1,k}={\gamma }_t^{(l)}{S}_{t,k}{N}_{t,k}+{\displaystyle \sum _j^l{s}_{t+1,k}^{(j)}{n}_{t+1,k}^{(j)}+\mathrm{\delta }W}$$

(50)

δW is defined as:

$$\mathrm{\delta }W=\frac{{\gamma }_t^{(i)}{N}_{t,k}{\mathrm{\beta }}_0}{{\gamma }_t^{(i)}{N}_{t,k}{+\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t,k})}^{\mathrm{T}}-\frac{N_{t+1,k}{\mathrm{\beta }}_0}{N_{t+1,k}{+\mathrm{\beta }}_0}({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k}){({\mathrm{m}}_0-{\overline{\mathrm{x}}}_{t+1,k})}^{\mathrm{T}}+{\displaystyle \sum _j^l\frac{n_{t+1,k}{\mathrm{\beta }}_{0,t+1}^{(j)}}{n_{t+1,k}{+\mathrm{\beta }}_{0,t+1}^{(j)}}(m_{0,t+1}^{(j)}-{\overline{\mathrm{x}}}_{t+1,k}){(m_{0,t+1}^{(j)}-{\overline{\mathrm{x}}}_{t+1,k})}^{\mathrm{T}}}$$

(51)

In comparison to Equation (9), the formulas for N and $\overline{\mathrm{x}}$ are natural; that is, the stack of the contributions from the past data is reduced and added to the current contribution. However, S seems similar to the other hyperparameters, except δW. δW represents the error caused by sequential estimation, so the effect of δ W on estimation accuracy is quite small if N and n are large enough. Accordingly, in the case of the proposed algorithm, δW is ignored.

The proposed algorithm is shown in Figure 5. A normal variational Bayesian estimation is carried out first. The estimated posterior at time t is used to generate the prior at the next time t+1. Each dataset is separated into many blocks, and the blocks are sequentially input to update the prior generated by the last estimated posterior. The block intromission is stopped when the KL divergence between the prior and the posterior is small enough. When that condition is met, the estimation of the point density at time t is finished. After the estimation, time t is incremented into t+ 1, and the estimation of the point density at time t + 1 is started. These processes are carried out iteratively to estimate the point density.

The parameters used for this algorithm are hyperparameters used by the variational Bayesian estimation and the number of blocks. They are determined manually in the same manner as a general variational Bayesian estimation. Note that the number of the blocks should be much smaller than the size of the entire dataset in order to obtain blocks large enough for detecting statistical significance. Satisfying this condition is generally easy because the entire data size is extremely large.

4. Experiments

4.1. Experimental Settings and Criteria

The proposed algorithm was evaluated experimentally. The features of the point dataset used for this evaluation are listed in

Table 1. PFLOWdataset summary.

Title	Value
Date	0:00 a.m.–12:00 a.m. in a day
Sampling rate	1 point per minute
Used dataset	600,000 trajectories

. The point datasets, named “people flow data” [28], describe the positions of people during a day. The point datasets were generated from the result of questionnaires about people’s travels during a day, namely data pairs of the origination and destination of the journeys. Interpolations between the origination and the destination were carried out in order to generate point datasets every minute. Therefore, the interpolation points at the same time were collected as one dataset. The point density from 0:00 a.m. to 12:00 a.m. was estimated by the proposed algorithm and by two conventional kernel density estimators. The points, which are expressed as longitude and latitude coordinates, were converted to an orthogonal coordinate system of which the origin is the centroid of points at 0:00 a.m.. Moreover, the coordinate values of the points in the orthogonal coordinate system were scaled to adjust their maximal values to 10.0.

The dataset was separated into two blocks. One block was used to estimate the point density; the other was used for evaluating the accuracy criteria. This is generally called “holdout validation”. Five thousand points included in the second block were actually used, because it would take too much time to carry out the evaluation if all of the points in the evaluation block were used.

Two criteria were evaluated: accuracy and processing time. As the accuracy criterion, Ev based on logarithmic likelihood, defined as:

$$\mathrm{Ev}=\frac{1}{N}\ln {\displaystyle \prod _{n=0}^{N}P({\mathrm{x}}_n)}$$

(52)

was applied. Here, N is the number of points denoted by x_n. Ev indicates the reproducibility of the point density of the entire dataset. As the processing-time criterion, the time for generating a 150-pixel × 100-pixel heat-map image was used; that is, the calculation was carried out 15,000 times. To evaluate the criteria accurately, the measurement of Ev and the processing time was carried out three times, and the average of the results was used as the criterion.

As the parameters used by the proposed algorithm, the number of mixed Gaussian distributions was set to 200; the block size for sequential intromission was 1,500; the initial hyperparameters were α₀ = 2.1 and β₀ = 1.0; diagonal components of W₀ were set to 1.5; and non-diagonal components were set to 0.0 and υ₀ = 2.2. In addition, 200 points were randomly selected from the initial point dataset as the m₀. These parameters were determined manually to obtain the highest Ev.

As conventional methods, the Gaussian kernel method and the quadratic kernel method were similarly evaluated. Standard deviation h was set to 0.2 for the Gaussian kernel method; radius h was set to 3.0 for the quadratic kernel method in order to avoid the P(x_n) = 0 (which gives Ev = − ∞).

Incidentally, the program used for the evaluations was implemented with Java on a computer with an Intel Core i7-3980K 3.2 GHz and 32.0 GB RAM.

4.2. Results

The processing times for each method are plotted in Figure 6a. The people’s positions at the time indicated by “dataset time” on the horizontal axis are included in the dataset used for the evaluation. Hereafter, “dataset” time is used in the same meaning. The processing time for the proposed algorithm was less than 1.0 s, that for the Gaussian kernel method around 250 s and that for the quadratic kernel method 80 s. The proposed algorithm is thus the fastest of the three methods. The accuracy criterion for each method is plotted in Figure 6b. The accuracy criterion for the proposed algorithm is close to that for the Gaussian-kernel density estimator, and that of the quadratic-kernel density estimator is lowest. These results indicate that the proposed algorithm is the fastest and accurate enough.

Distributions obtained by the three methods are compared in Figure 7. The red regions in the images indicate high-density areas, the yellow regions medium density and the green regions sparse areas. Image (a), which represents the density distribution given by the proposed algorithm, is similar to that given by the Gaussian-kernel estimator, namely Image (b), but it is diffused a bit. Image (c), which shows the result of the quadratic kernel estimation, is diffused too much to describe the point density. This diffuseness is caused by large h; therefore, the image generated with the result of the quadratic kernel with h = 1.0 is similar to Images (a) and (b), as shown in Image (d). However, the accuracy criteria for the quadratic-kernel method with h = 1.0 is −∞; that is, the distribution is not accurate at all. Accordingly, the proposed algorithm generates the most unblurred accurate heatmap in these three algorithms.

4.3. Discussion

Sub-sampling is one of the most popular methods for speeding up density estimation processes. Accordingly, a sub-sampled dataset was also evaluated. Figure 8a plots a series of processing times after the point dataset was randomly sub-sampled to 2300 points for the Gaussian-kernel estimator (h = 3.0) and 17,000 points for the quadratic-kernel estimator. Both the Gaussian-kernel estimator and the quadratic-kernel estimator took 0.6 s under this condition. These times are comparable with that taken by the proposed algorithm; however, the accuracies of the two former estimators is lower, as shown in Figure 8b. It is thus concluded that the proposed algorithm maintains high accuracy even when the processing time was reduced.

Heatmaps generated from sub-sampled datasets are shown in Figure 9a,b. The heatmap given by the quadratic-kernel estimator (Figure 9a) is very similar to the heatmap obtained without sub-sampling, which is shown in Figure 7c, because the sub-sampled dataset is large enough. However, the heatmap given by the Gaussian-kernel estimator (Figure 9b) is quite different from Figure 7b. This implies that the sub-sampled dataset used by the Gaussian-kernel estimator is too small to generate a heatmap. As shown in Figure 10, to compare heatmaps at 7:01 a.m., 7:02 a.m., 7:03 a.m. and 7:04 a.m., a generated heatmap depends on points chosen at random, that is the heatmap is changed randomly. Because the Gaussian-kernel estimator is slower than the quadratic-kernel estimator, fewer points were used. That means the quadratic-kernel estimator is more feasible for generating heatmaps quickly than the Gaussian-kernel estimator. However, as shown in Figure 8, the proposed algorithm can generate more accurate heatmaps than that generated by the quadratic kernel estimator.

For ensuring these results, Ev’s of the Gaussian-kernel estimator and the quadratic-kernel estimator under various h’s are calculated with 10,000 points at 0:00 a.m.. In the case of the Gaussian-kernel estimator, Ev = −∞ at h = 0.05, Ev = −4.00 at h = 0.1, Ev = −4.11 at h = 0.2, Ev = −4.28 at h = 0.5 and Ev = −4.41 at h = 1.0. Similarly, in the case of the quadratic-kernel, Ev = −∞ at h = 1.0, Ev = −∞ at h = 2.0, Ev = −4.40 at h = 3.0, Ev = −4.47 at h = 4.0 and Ev = −4.53 at h = 5.0. The Ev shown in the Figure 9b are comparable with these Ev. In addition, the proposed algorithm achieved Ev = −3.79, which is also comparable with Ev shown in Figure 9b. That implies that the evaluation setting like h and the number of point data are not significant for the accuracy evaluation.

According to the results of the experiment described above, the proposed algorithm achieved faster performance than two existing algorithms, namely a quadratic-kernel density estimator and a Gaussian-kernel density estimator. Moreover, by subsampling, the kernel density estimators can be accelerated to speeds comparable with that of the proposed algorithm; however, they show lower accuracy. Therefore, it is confirmed that the proposed algorithm has advantages over the existing kernel-density estimators.

5. Concluding Remarks

A method for estimating the density of moving points, based on a variational Bayesian estimation, was proposed. The prior in the estimation is generated from the posterior given by the previous estimation. Responsibilities used in the posterior formula should be reduced in order to generate the prior; therefore, a trick named “responsibility reduction” was introduced. Moreover, the processing time and the estimation accuracy of the proposed algorithm were experimentally evaluated. The proposed algorithm was faster than two conventional methods; moreover, the estimation accuracy of the proposed algorithm is comparable with those of the conventional methods. These results demonstrate that the proposed algorithm is fast without loss of accuracy. Therefore, it is concluded that a massive moving-point dataset, which can be encoded with OGC Moving Features, can be converted to a heatmap, which is distributable with OGC WMS and WCS, in real time.

As future work, the proposed algorithm will be extended to handle the non-GMM distributions. The responsibility reduction is applicable to other distributions. The the extended algorithm will thus be further developed to speed up the point density estimation.