# Discovering Homogeneous Groups from Geo-Tagged Videos

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- We propose an algorithm to discover homogeneous groups from geo-tagged videos.
- We propose a density clustering method for geo-tagged FoV clustering based on the DBSCAN method (FDBC).
- We propose an efficient filtering algorithm to reduce the candidates using a two-level grid index (FCBG).

## 2. Related Work

#### 2.1. Travel-Pattern Mining

#### 2.2. Geo-Tagged Videos

#### 2.3. Clustering Algorithms

## 3. Preliminaries

**Definition**

**1**(FoV Scene)

**.**

**Definition**

**2**(FoV Sequence)

**.**

**Definition**

**3**(Overlap Coefficient)

**.**

**Definition**

**4**(Closeness)

**.**

**Definition**

**5**(Clusters of FoVs)

**.**

**Definition**

**6**(Homogeneous Groups)

**.**

- (1)
**FC (FoVs Cluster):**the FoVs in a set of geo-tagged videos belong to the same cluster at each timestamp of T.- (2)
**Occurrence:**k is the minimal number of occurrence for FoVs belong to the same cluster.- (3)
**Significance:**$\left|C\right|$ ≥ $mi{n}_{o}$.- (4)
**Succession:**T is s-sequential.- (5)
**Connection:**T is g-connected.

**Definition**

**7**(Grid)

**.**

**Definition**

**8**(Grid-clustering)

**.**

## 4. Proposed Approach

#### 4.1. Naive Algorithm Design

Algorithm 1: FDBC(${S}_{t}$, $\u03f5$, $mi{n}_{o}$, $\delta $). |

Algorithm 2: ExpandFoVCluster(${S}_{t}$, $\u03f5$, $mi{n}_{o}$, $\mathcal{F}$, $Visited\left[\right]$). |

Algorithm 3: BoundaryFoVBelong($S{C}_{temp}$, $mi{n}_{o}$). |

**Definition**

**9**(Groups Candidates)

**.**

Algorithm 4: MiningHomogenousGroups($\mathcal{SC}$, $mi{n}_{o}$, k, s, g). |

#### 4.2. Performance Enhancement Using a Grid Index Approach

Algorithm 5: FCBG(${S}_{t}$, $\u03f5$, $mi{n}_{o}$, c, $\delta $). |

## 5. Experimental Evaluation

#### 5.1. Experiment Setting

#### 5.2. Performance of Clustering

**Selection of $mi{n}_{o}$**Value: The rule of thumb [39] is Formula (14), where dim represents the dimension of the data to be clustered.

**The effect of $mi{n}_{o}$**: To test the influence of the $mi{n}_{o}$ threshold, (Shopping data: fixed $\u03f5$ = 3, $\delta $ = 0.5; BDD100K: $\u03f5$ = 50, $\delta $ = 0.5) we changed the minimal number of FoV threshold from 4 to 20. Figure 6 shows the clustering cost of the algorithm FDBC and the algorithm FCBG based on the $mi{n}_{o}$ threshold. According to Figure 6, we observe that the running time of FDBC becomes longer as the minimal FoV threshold increases because the first step of FDBC is to find the nearest neighbor of $\u03f5$. It calculates FoV one by one and then judges the minimal number which increases the computation. However, FCBG, according to the MBR, reduces the region of calculation. After $mi{n}_{o}$ = 12, the time cost rises smoothly in Shopping data because after the FDBC calculates the distance, if $mi{n}_{o}$ is not satisfied, the following mark calculation is required. Therefore, the time cost has a slow rise depending on the different distribution densities of the data. It can be seen that there are not many clusters that meet the 12 FoVs as a class. In addition, the BDD100K shows that FCBG also has a significant increase with the increase in $mi{n}_{o}$; but overall, the time of FCBG is lower than that of FDBC.

**The effect of $\u03f5$**: To test the influence of the distance threshold, we fixed Shopping data and BDD100K to $mi{n}_{o}=4$ and $\delta $ = 0.5, and changed the distance threshold from 3 m to 27 m. Figure 7 shows the clustering cost of the algorithm FDBC and the algorithm FCBG based on the distance threshold. According to Figure 7, we observe that the running time of FDBC becomes longer as the distance threshold increases because the first step of FDBC is to find the nearest neighbor of $mi{n}_{o}$ within the distance threshold. A more extensive calculation is needed as the distance increases because this means the number of FoVs that need to be calculated will increase. On the contrary, distance is not the primary influencing factor of FCBG. As FCBG is based on the MBR of FoV and based on the grid index, it calculates the overlap coefficient $\delta $ = 0.5 and obtains $mi{n}_{o}$ FoVs whose distance threshold is not greater than eps. Therefore, the time cost of FCBG does not fluctuate significantly with an increase in $\u03f5$, as shown in the figure. In addition, in terms of running time, the efficiency of our optimized FCBG is better than that of FDBC.

**The effect of $\delta $**: To evaluate the effect of varying $\delta $, we fixed the threshold (Shopping: $\u03f5$ = 3 m, $mi{n}_{o}$ = 4; BDD100K: $\u03f5$ = 50 m, $mi{n}_{o}$ = 4), and set $\delta $ from 0.2 to 0.4. Figure 8 illustrates the clustering time of the two algorithms. In the experiment, the running time depends on the distribution of the data. When the density that satisfies the overlap threshold is large, the amount of calculation is significant, and vice versa. Nevertheless, the overall efficiency of FCBG is still higher than that of FDBC.

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Syafrudin, M.; Alfian, G.; Fitriyani, N.L.; Rhee, J. Performance analysis of IoT-based sensor, big data processing, and machine learning model for real-time monitoring system in automotive manufacturing. Sensors
**2018**, 18, 2946. [Google Scholar] [CrossRef] [PubMed] - Judd, T.; Ehinger, K.; Durand, F.; Torralba, A. Learning to predict where humans look. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October 2009; pp. 2106–2113. [Google Scholar]
- Kyriakou, K.; Resch, B.; Sagl, G.; Petutschnig, A.; Werner, C.; Niederseer, D.; Liedlgruber, M.; Wilhelm, F.; Osborne, T.; Pykett, J. Detecting moments of stress from measurements of wearable physiological sensors. Sensors
**2019**, 19, 3805. [Google Scholar] [CrossRef] - Lu, Y.; To, H.; Alfarrarjeh, A.; Kim, S.H.; Yin, Y.; Zimmermann, R.; Shahabi, C. GeoUGV: User-generated mobile video dataset with fine granularity spatial metadata. In Proceedings of the 7th International Conference on Multimedia Systems, Klagenfurt, Austria, 10–13 May 2016; pp. 1–6. [Google Scholar]
- Ding, W.; Tian, J.; Lee, Y.; Yang, K.; Nam, K.W. VVS: Fast Similarity Measuring of FoV-Tagged Videos. IEEE Access
**2020**, 8, 190734–190745. [Google Scholar] [CrossRef] - Jalal, A.; Kamal, S.; Kim, D. A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments. Sensors
**2014**, 14, 11735–11759. [Google Scholar] [CrossRef] - Gurrin, C.; Smeaton, A.F.; Doherty, A.R. LifeLogging: Personal Big Data. Found. Trends Inf. Retr.
**2014**, 8, 1–125. [Google Scholar] [CrossRef] - Brščić, D.; Kanda, T.; Ikeda, T.; Miyashita, T. Person tracking in large public spaces using 3-D range sensors. IEEE Trans. Hum.-Mach. Syst.
**2013**, 43, 522–534. [Google Scholar] [CrossRef] - Yu, F.; Chen, H.; Wang, X.; Xian, W.; Chen, Y.; Liu, F.; Madhavan, V.; Darrell, T. BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2020; pp. 2633–2642. [Google Scholar] [CrossRef]
- Vieira, M.R.; Bakalov, P.; Tsotras, V.J. On-line discovery of flock patterns in spatio-temporal data. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA, 4–6 November 2009; pp. 286–295. [Google Scholar]
- Tanaka, P.S.; Vieira, M.R.; Kaster, D.S. Efficient Algorithms to Discover Flock Patterns in Trajectories. In Proceedings of the GeoInfo, Kuala Lumpur, Malaysia, 28–30 October 2015; pp. 56–67. [Google Scholar]
- Turdukulov, U.; Calderon Romero, A.O.; Huisman, O.; Retsios, V. Visual mining of moving flock patterns in large spatio-temporal data sets using a frequent pattern approach. Int. J. Geogr. Inf. Sci.
**2014**, 28, 2013–2029. [Google Scholar] [CrossRef] - Jeung, H.; Yiu, M.L.; Zhou, X.; Jensen, C.S.; Shen, H.T. Discovery of convoys in trajectory databases. Proc. VLDB Endow.
**2008**, 1, 1068–1080. [Google Scholar] [CrossRef] - Jeung, H.; Shen, H.T.; Zhou, X. Convoy queries in spatio-temporal databases. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, Cancun, Mexico, 7–12 April 2008; pp. 1457–1459. [Google Scholar]
- Li, Z.; Ding, B.; Han, J.; Kays, R. Swarm: Mining Relaxed Temporal Moving Object Clusters. Proc. VLDB Endow.
**2010**, 3, 723–734. [Google Scholar] [CrossRef] - Van den Bergh, F.; Engelbrecht, A.P. A study of particle swarm optimization particle trajectories. Inf. Sci.
**2006**, 176, 937–971. [Google Scholar] [CrossRef] - Kalnis, P.; Mamoulis, N.; Bakiras, S. On discovering moving clusters in spatio-temporal data. In Proceedings of the International Symposium on Spatial and Temporal Databases, Angra dos Reis, Brazil, 22–24 August 2005; pp. 364–381. [Google Scholar]
- Li, Y.; Bailey, J.; Kulik, L. Efficient mining of platoon patterns in trajectory databases. Data Knowl. Eng.
**2015**, 100, 167–187. [Google Scholar] [CrossRef] - Fan, Q.; Zhang, D.; Wu, H.; Tan, K.L. A general and parallel platform for mining co-movement patterns over large-scale trajectories. Proc. VLDB Endow.
**2016**, 10, 313–324. [Google Scholar] [CrossRef] - Chen, L.; Gao, Y.; Fang, Z.; Miao, X.; Jensen, C.S.; Guo, C. Real-time distributed co-movement pattern detection on streaming trajectories. Proc. VLDB Endow.
**2019**, 12, 1208–1220. [Google Scholar] [CrossRef] - Cho, N.; Kang, Y. Space-time density of field trip trajectory: Exploring spatio-temporal patterns in movement data. Spat. Inf. Res.
**2017**, 25, 141–150. [Google Scholar] [CrossRef] - Lee, H.; Kang, Y. Mining tourists’ destinations and preferences through LSTM-based text classification and spatial clustering using Flickr data. Spat. Inf. Res.
**2021**, 29, 825–839. [Google Scholar] [CrossRef] - Ma, H.; Ay, S.A.; Zimmermann, R.; Kim, S.H. Large-scale geo-tagged video indexing and queries. GeoInformatica
**2014**, 18, 671–697. [Google Scholar] [CrossRef] - Kim, Y.; Kim, J.; Yu, H. GeoTree: Using spatial information for georeferenced video search. Knowl.-Based Syst.
**2014**, 61, 1–12. [Google Scholar] [CrossRef] - Cai, Y.; Lu, Y.; Kim, S.H.; Nocera, L.; Shahabi, C. Querying geo-tagged videos for vision applications using spatial metadata. EURASIP J. Image Video Process.
**2017**, 2017, 19. [Google Scholar] [CrossRef] - Ay, S.A.; Zimmermann, R.; Kim, S.H. Viewable scene modeling for geospatial video search. In Proceedings of the 16th ACM international conference on Multimedia, Vancouver, BC, Canada, 27–31 October 2008; pp. 309–318. [Google Scholar]
- Constantinou, G.; Shahabi, C.; Kim, S.H. Spatial Keyframe Extraction Of Mobile Videos For Efficient Object Detection At The Edge. In Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 25–28 October 2020; pp. 1466–1470. [Google Scholar]
- Kim, S.H.; Ay, S.A.; Yu, B.; Zimmermann, R. Vector model in support of versatile georeferenced video search. In Proceedings of the First Annual ACM SIGMM Conference on Multimedia Systems, Phoenix, AZ, USA, 22–23 February 2010; pp. 235–246. [Google Scholar]
- Huang, Z. Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Min. Knowl. Discov.
**1998**, 2, 283–304. [Google Scholar] [CrossRef] - Peng, K.; Zheng, L.; Xu, X.; Lin, T.; Leung, V.C. Balanced iterative reducing and clustering using hierarchies with principal component analysis (PBIRCH) for intrusion detection over big data in mobile cloud environment. In Proceedings of the International Conference on Security, Privacy and Anonymity in Computation, Communication and Storage, Zhangjiajie, China, 16–18 November 2018; pp. 166–177. [Google Scholar]
- Ester, M.; Kriegel, H.; Sander, J.; Xu, X. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), Portland, OR, USA, 2–4 August 1996; Simoudis, E., Han, J., Fayyad, U.M., Eds.; AAAI Press: Washington, DC, USA, 1996; pp. 226–231. [Google Scholar]
- Lee, Y.; Nam, K.W.; Ryu, K.H. Fast mining of spatial frequent wordset from social database. Spat. Inf. Res.
**2017**, 25, 271–280. [Google Scholar] [CrossRef] - Mantiuk, R.K.; Denes, G.; Chapiro, A.; Kaplanyan, A.; Rufo, G.; Bachy, R.; Lian, T.; Patney, A. Fovvideovdp: A visible difference predictor for wide field-of-view video. ACM Trans. Graph. (TOG)
**2021**, 40, 1–19. [Google Scholar] [CrossRef] - Park, J.Y.; Ryu, D.J.; Nam, K.W.; Jang, I.; Jang, M.; Lee, Y. DeepDBSCAN: Deep density-based clustering for geo-tagged photos. ISPRS Int. J. Geo-Inf.
**2021**, 10, 548. [Google Scholar] [CrossRef] - Nam, K.W.; Yang, K. RealROI: Discovering Real Regions of Interest From Geotagged Photos. IEEE Access
**2022**, 10, 83489–83497. [Google Scholar] [CrossRef] - Mao, Y.; Zhong, H.; Qi, H.; Ping, P.; Li, X. An adaptive trajectory clustering method based on grid and density in mobile pattern analysis. Sensors
**2017**, 17, 2013. [Google Scholar] [CrossRef] [PubMed] - Suo, Y.; Chen, W.; Claramunt, C.; Yang, S. A ship trajectory prediction framework based on a recurrent neural network. Sensors
**2020**, 20, 5133. [Google Scholar] [CrossRef] [PubMed] - Moulton, R.; Jiang, Y. Maximally Consistent Sampling and the Jaccard Index of Probability Distributions. In Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore, 17–20 November 2018; pp. 347–356. [Google Scholar]
- Tang, J.; Bi, W.; Liu, F.; Zhang, W. Exploring urban travel patterns using density-based clustering with multi-attributes from large-scaled vehicle trajectories. Phys. A Stat. Mech. Appl.
**2021**, 561, 125301. [Google Scholar] [CrossRef] - Danielsson, P.E. Euclidean distance mapping. Comput. Graph. Image Process.
**1980**, 14, 227–248. [Google Scholar] [CrossRef]

Notation | Description |
---|---|

$\mathit{F}$ | FoV sequence in a geo-video |

$\mathcal{T}$ | Sequence of all timestamps after partition in a geo-video |

$\delta $ | Overlap coefficient of common region for two FoVS |

$\mathcal{SC}$ | FoV collection of snapshots at the timestamp |

$\mathcal{C}$ | Clusters in a snapshot ${S}_{t}$ |

k | Minimal number of occurrence for FoVs belonging to the same cluster |

$mi{n}_{o}$ | Minimal number of FoVs |

**Table 2.**The homogeneous groups corresponding to Figure 4.

Groups | ||||||
---|---|---|---|---|---|---|

GroupsID | 1 | 2 | 3 | 4 | 5 | 6 |

VideoID | 1, 2 | 3, 4 | 5, 6 | 7, 8 | 7, 8, 9 | 7, 9 |

SnapshotID | 1, 2, 4 | 1, 2, 4, 5 | 1, 2, 4, 5 | 1, 2, 3, 4, 5 | 1, 3, 5 | 1, 3, 5 |

Statistics | Shopping | BDD100K |
---|---|---|

Number of geo-videos | 4500 | 29,490 |

The longest time-domain length of video | 42,000 | 30,000 |

The total of FoVs | 135,000 | 1,238,580 |

The number of snapshots | 10 | 40 |

Cluster size | 44/70 | 412/965 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Di, X.; Lew, D.J.; Nam, K.W.
Discovering Homogeneous Groups from Geo-Tagged Videos. *Sensors* **2023**, *23*, 4443.
https://doi.org/10.3390/s23094443

**AMA Style**

Di X, Lew DJ, Nam KW.
Discovering Homogeneous Groups from Geo-Tagged Videos. *Sensors*. 2023; 23(9):4443.
https://doi.org/10.3390/s23094443

**Chicago/Turabian Style**

Di, Xuejing, Dong June Lew, and Kwang Woo Nam.
2023. "Discovering Homogeneous Groups from Geo-Tagged Videos" *Sensors* 23, no. 9: 4443.
https://doi.org/10.3390/s23094443