Next Article in Journal
Exploring Signals on L5/E5a/B2a for Dual-Frequency GNSS Precise Point Positioning
Previous Article in Journal
Experimental Verification of the Concept of Using LOFAR Radio-Telescopes as Receivers in Passive Radiolocation Systems
Previous Article in Special Issue
DDoS Flood and Destination Service Changing Sensor
Article

Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder

1
Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
2
Office of Institutional Research, Hokkaido University, N-8, W-5, Kita-ku, Sapporo 060-0808, Hokkaido, Japan
3
Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan
*
Author to whom correspondence should be addressed.
This paper is an extended version of our paper published in: Hirasawa, K.; Maeda, K.; Ogawa, T.; Haseyama, M. Important Scene Detection Based on Anomaly Detection using Long Short-Term Memory for Baseball Highlight Generation. In the Proceedings of the IEEE International Conference on Consumer Electronics—Taiwan (IEEE 2020 ICCE-TW), Taoyuan, Taiwan, 28–30 September 2020.
Academic Editor: Tsung-Han Tsai
Sensors 2021, 21(6), 2045; https://doi.org/10.3390/s21062045
Received: 28 January 2021 / Revised: 4 March 2021 / Accepted: 10 March 2021 / Published: 14 March 2021
A new method for the detection of important scenes in baseball videos via a time-lag-aware multimodal variational autoencoder (Tl-MVAE) is presented in this paper. Tl-MVAE estimates latent features calculated from tweet, video, and audio features extracted from tweets and videos. Then, important scenes are detected by estimating the probability of the scene being important from estimated latent features. It should be noted that there exist time-lags between tweets posted by users and videos. To consider the time-lags between tweet features and other features calculated from corresponding multiple previous events, the feature transformation based on feature correlation considering such time-lags is newly introduced to the encoder in MVAE in the proposed method. This is the biggest contribution of the Tl-MVAE. Experimental results obtained from actual baseball videos and their corresponding tweets show the effectiveness of the proposed method. View Full-Text
Keywords: multimodal variational autoencoder; detection of important scenes; Twitter; sports video; time-lags multimodal variational autoencoder; detection of important scenes; Twitter; sports video; time-lags
Show Figures

Figure 1

MDPI and ACS Style

Hirasawa, K.; Maeda, K.; Ogawa, T.; Haseyama, M. Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder. Sensors 2021, 21, 2045. https://doi.org/10.3390/s21062045

AMA Style

Hirasawa K, Maeda K, Ogawa T, Haseyama M. Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder. Sensors. 2021; 21(6):2045. https://doi.org/10.3390/s21062045

Chicago/Turabian Style

Hirasawa, Kaito, Keisuke Maeda, Takahiro Ogawa, and Miki Haseyama. 2021. "Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder" Sensors 21, no. 6: 2045. https://doi.org/10.3390/s21062045

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop