Skip to Content
You are currently on the new version of our website. Access the old version .
Future InternetFuture Internet
  • Article
  • Open Access

5 March 2026

24 Pages

Multi-Modal Artificial Intelligence for Smart Cities: Experimental Integration of Textual and Sensor Data

Department of Computer Science and Engineering, College of Computer Science and Engineering, University of Hafr Al Batin, Hafr Al Batin 39524, Saudi Arabia

Abstract

Smart city decision-making increasingly relies on heterogeneous urban data sources. Dense traffic sensor streams provide continuous quantitative measurements, while citizen-generated textual reports offer event-driven contextual information. However, integrating these modalities remains challenging due to temporal misalignment, textual sparsity, and semantic noise. This paper investigates multi-modal learning for traffic congestion severity prediction through an experimental integration of open traffic sensor data (METR-LA: Los Angeles, USA) and citizen-generated textual reports (NYC 311: New York City, USA). Congestion severity is formulated as a four-class classification task derived from traffic speed measurements. We propose an end-to-end framework that combines: (i) sensor time-series encoding using a GRU-based temporal encoder, (ii) textual representation learning using a BERT-based encoder, (iii) a symmetric time-window alignment strategy (±Δ) to associate irregular reports with sensor time steps, and (iv) multiple fusion architectures, including early fusion, late fusion, and a cross-attention module for cross-modal interaction modeling. Experiments on publicly available datasets show that multi-modal early fusion achieves the best overall performance (Accuracy = 0.8283, Macro-F1 = 0.8231) compared to uni-modal baselines. In the studied cross-city setting with sparse and weakly aligned textual signals, the proposed cross-attention fusion does not outperform the strong sensor-only baseline, suggesting that the sensor modality dominates when cross-modal signal strength is limited. These results highlight both the potential and the practical constraints of multi-modal fusion in heterogeneous smart-city environments, emphasizing the importance of alignment design, modality relevance, and transparent experimental validation.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.