A Survey on Video Big Data Analytics: Architecture, Technologies, and Open Research Challenges

Abstract

The exponential growth of video data across domains such as surveillance, transportation, and healthcare has raised critical challenges in scalability, real-time processing, and privacy preservation. While existing studies have addressed individual aspects of Video Big Data Analytics (VBDA), an integrated, up-to-date perspective remains limited. This paper presents a comprehensive survey of system architectures and enabling technologies in VBDA. It categorizes system architectures into four primary types as follows: centralized, cloud-based infrastructures, edge computing, and hybrid cloud–edge. It also analyzes key enabling technologies, including real-time streaming, scalable distributed processing, intelligent AI models, and advanced storage for managing large-scale multimodal video data. In addition, the study provides a functional taxonomy of core video processing tasks, including object detection, anomaly recognition, and semantic retrieval, and maps these tasks to real-world applications. Based on the survey findings, the paper proposes ViMindXAI, a hybrid AI-driven platform that combines edge and cloud orchestration, adaptive storage, and privacy-aware learning to support scalable and trustworthy video analytics. Our analysis in this survey highlights emerging trends such as the shift toward hybrid cloud–edge architectures, the growing importance of explainable AI and federated learning, and the urgent need for secure and efficient video data management. These findings highlight key directions for designing next-generation VBDA platforms that enhance real-time, data-driven decision-making in domains such as public safety, transportation, and healthcare. These platforms facilitate timely insights, rapid response, and regulatory alignment through scalable and explainable analytics. This work provides a robust conceptual foundation for future research on adaptive and efficient decision-support systems in video-intensive environments.

1. Introduction

The explosive growth of video data has transformed multiple sectors, including transportation, healthcare, surveillance, and retail. According to Statista [1], video accounted for over 80% of global internet traffic in 2023, largely driven by the increase of smart cameras, IoT devices, autonomous systems, and online video platforms. This growth demands scalable, real-time, and privacy-preserving Video Big Data Analytics (VBDA) systems to support intelligent decision-making across distributed environments.

VBDA has become an integral part of modern urban infrastructure, as seen in smart city projects such as Singapore’s traffic monitoring and electronic road pricing (ERP) systems, helping to reduce congestion and optimize signal flow [2]. In the healthcare domain, AI-driven video surveillance supports early detection of critical events in hospital environments, contributing to improved clinical outcomes [3]. In the retail sector, systems like Amazon Go leverage VBDA to monitor customer behavior, automate checkouts, and deliver personalized services [4].

Despite the promising real-world applications of VBDA, its development remains challenged by the high volume, heterogeneity, and complexity of video data. Video streams are inherently multimodal and high-dimensional, encompassing spatial, temporal, and auditory characteristics, and are often encoded in varied formats such as MP4, AVI, and MKV [5]. These characteristics introduce difficulties in decoding, indexing, and storage management [6,7]. Real-time video analytics faces latency, bandwidth, and storage limitations, particularly in safety-critical applications such as autonomous driving and emergency response [4,8]. Furthermore, privacy and data protection regulations such as GDPR and HIPAA require the incorporation of secure and explainable AI methodologies [3,9].

Although distributed data processing frameworks such as Apache Spark, Kafka, and Flink [10,11] have proven to be effective for scalability, they often fail to deliver low-latency responses and video-specific optimizations. Recent advances in edge AI and Federated Learning (FL) have led to a shift toward decentralized analytics closer to the data source [5,12,13]. FL, in particular, enables collaborative training between edge devices without transmitting raw data, which in turn improves privacy and reduces the load on the network [9,14]. However, challenges such as model synchronization, non-independent and identically distributed (non-IID) data distributions, and edge-device constraints remain open research issues [15,16].

This paper presents a comprehensive synthesis of the current landscape in VBDA, focusing on system architectures, enabling technologies, and application-oriented scenarios. Existing VBDA platforms are systematically categorized into our representative system architectures as follows: centralized, cloud-based, edge computing, and hybrid cloud–edge models. Concurrently, the study analyzes a broad spectrum of enabling technologies, including real-time stream processing frameworks, scalable distributed processing engines, deep learning, and federated learning models for intelligent analytics, and advanced storage solutions for managing multimodal, high-volume video data. Beyond architectural and technological classification, the study highlights critical challenges that persist throughout the VBDA analytical process, particularly regarding latency constraints, storage efficiency, and privacy preservation in distributed learning settings. To address these challenges, this paper proposes ViMindXAI, a hybrid AI-driven platform that incorporates edge and cloud orchestration, adaptive video storage mechanisms, privacy-aware learning processes, and intelligent semantic-level video retrieval capabilities.

Our work, as described in this paper, makes three main contributions. First, this work presents a consolidated and up-to-date evaluation of VBDA platform architectures and technologies, together with a functional taxonomy of core video analytics tasks and their real-world applications. Second, the paper identifies and synthesizes key open research challenges related to real-time responsiveness, scalability, AI integration, and privacy constraints in large-scale video analytics. Third, this study proposes ViMindXAI as an integrated architectural framework that serves as a reference design for the development of scalable, intelligent, and trustworthy next-generation VBDA platforms. It is important to note that ViMindXAI is currently introduced as a conceptual reference architecture, developed through in-depth analysis of underlying technologies and their capacity to address the practical requirements of modern video big data analytics platforms. Its prototyping and comprehensive performance evaluation are planned as part of future research efforts.

To conduct this study, we adopt a survey-based methodology, integrating insights from a broad spectrum of the scholarly literature and real-world implementations across both academic and industrial contexts. An initial corpus of over 350 publications was retrieved from reputable databases, including Scopus, Web of Science, IEEE Xplore, and the ACM Digital Library. A systematic screening process was applied to remove non-peer-reviewed works, studies not directly addressing system architectures or enabling technologies in VBDA, and publications based on outdated technological approaches no longer relevant to current developments. As a result, 157 publications were retained for comprehensive analysis. The majority of these works, focusing on VBDA architectures and core technologies, were published between 2018 and 2025. A substantial portion appeared in high-impact journals and conferences, demonstrating their technical depth, alignment with emerging research directions, and significant contributions to the advancement of VBDA platforms. This systematically selected set of studies ensures that the survey captures both foundational and cutting-edge contributions to the field.

The remainder of this paper is organized as follows. Section 2 provides a comprehensive overview of VBDA, detailing architectural paradigms, enabling technologies, and a functional taxonomy of core video processing tasks. Section 3 examines in greater depth VBDA system architectures and foundational technologies, tracing their development and emerging trends from 2018 to 2025. Section 4 introduces the proposed ViMindXAI platform, emphasizing its hybrid architecture, modular design, and AI-driven capabilities for scalable, privacy-preserving video analytics. This section also presents three representative case studies demonstrating the platform’s adaptability to diverse real-world scenarios. Section 5 discusses open challenges and outlines future research directions, followed by the conclusion in Section 6.

2. Literature Overview in VBDA

The growing complexity and scale of video data have driven significant research into scalable, low-latency, and privacy-preserving VBDA systems. While previous surveys have explored key aspects individually, such as edge computing [17], FL [9,18], and video surveillance architectures [19], these works often consider each technology in isolation and lack a unified, system-level perspective. For example, Elharrouss et al. [19] provide a detailed review of conventional surveillance architectures but do not address recent advances in FL or explainable AI. Xu et al. [17] focus on the architectural challenges of edge-based video analytics but overlook federated or collaborative learning methods. Similarly, surveys on FL [9,18] offer high-level overviews without contextualizing their insights within end-to-end video analytics pipelines or practical deployment frameworks. This section aims to bridge these gaps by presenting a unified and structured overview of the VBDA research landscape. It begins with the historical evolution of system architectures and continues with a review of enabling technologies and current architectural paradigms. This synthesis lays the groundwork for the ViMindXAI platform, which will be detailed in the following sections.

2.1. Evolution and Trends in VBDA

The evolution of VBDA represents a significant paradigm shift toward distributed, AI-driven architectures, prompted by the growing demands for scalability, responsiveness, and data privacy. As illustrated in Figure 1, this transformation has been driven by advances in networked video systems, scalable data infrastructures, and intelligent analytics. Since the early 2010s, coinciding with the growth of digital surveillance systems such as city-wide CCTV infrastructures, smart building monitoring, and big data platforms such as Apache Hadoop, Spark, VBDA has progressed through five distinct architectural stages. Each phase contributed specific capabilities to address critical challenges, including scalability, low-latency processing, data privacy, and contextual video understanding [7,20,21]. Understanding this historical progression provides essential context for the subsequent overview of the current research landscape presented in this section.

(1) 

Centralized and On-Premises VBDA (2010–2012)

The initial phase of VBDA was defined by centralized, on-premises architectures, where video acquisition, storage, and processing were handled entirely on local servers. These systems employed RAID or SAN/NAS storage and used relational databases such as MySQL v5.5 and PostgreSQL v9.0, with occasional support from early NoSQL platforms like MongoDB v2.4 for metadata handling [17,19]. With no support for distributed processing or elastic scalability, they were ill-suited for high-throughput or real-time analytics. Video understanding was based on classical machine learning techniques using handcrafted features. Among these, Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT) were the most widely adopted for object detection and recognition due to their robustness to scale and orientation changes [19]. Additionally, LBP and Haar-like features were frequently used in face detection pipelines and embedded systems, given their computational efficiency [17,19]. Although tools like OpenCV 2.0 (2010) introduced basic GPU acceleration through CUDA, performance was still constrained by hardware limitations globally, including in developed countries such as the United States, Germany, and Japan. Despite early adoption of GPU acceleration via CUDA, these regions faced practical deployment challenges due to memory bandwidth, power constraints, and the high cost of large-scale real-time video processing. A major shift occurred in 2012 with the introduction of AlexNet, which demonstrated the power of deep learning for visual recognition and revealed the shortcomings of rule-based methods. This breakthrough established the foundation for transitioning to scalable, AI-enhanced cloud architectures, which characterized the subsequent batch-processing era [22,23].

(2) 

Cloud-Based Batch Processing VBDA (2013–2016)

As video data grew in volume and complexity, VBDA systems shifted from on-premises architectures to cloud-based infrastructures. Distributed frameworks like Apache Hadoop (https://hadoop.apache.org, accessed on 12 May 2025) and Apache Spark (https://spark.apache.org, accessed on 12 May 2025) facilitated large-scale batch analytics, allowing efficient processing of video archives within elastic cloud infrastructures [24,25]. Simultaneously, cloud object storage platforms like Amazon S3 and Google Cloud Storage offered scalable, cost-effective alternatives to traditional systems, supporting flexible data ingestion and on-demand compute provisioning [26].

This period also marked the integration of deep learning models such as VGGNet and ResNet into VBDA pipelines, improving object detection and scene interpretation through cloud-native AI services [27]. Elasticsearch, widely adopted during this phase, enabled scalable indexing and fast metadata retrieval across distributed video repositories [25]. However, batch-based processing introduced significant latency and lacked responsiveness. These limitations rendered it less suitable for real-time scenarios such as surveillance and traffic analytics [24]. These limitations spurred the shift toward more agile, edge-driven architectures in the next phase.

(3) 

Streaming Video Analytics and Edge Computing (2017–2019)

To overcome the latency and scalability limitations of cloud-only batch processing, VBDA systems began incorporating edge computing and streaming analytics. This evolution led to hybrid cloud–edge architectures, where edge devices performed real-time inference, while cloud platforms handled model training and archival storage. Stream processing tools such as Apache Kafka (https://kafka.apache.org, accessed on 15 May 2025) and Apache Flink (https://flink.apache.org, accessed on 13 May 2025) became foundational for enabling low-latency video ingestion and analytics pipelines [28]. At the same time, hardware accelerators like NVIDIA Jetson (NVIDIA Corporation, Santa Clara, CA, USA) and Intel Movidius (Intel Corporation, Santa Clara, CA, USA) enabled efficient on-device video processing [2].

The introduction of YOLOv3 in 2018 further enhanced real-time object detection capabilities, reinforcing the practicality of edge-first analytics [29]. Complementary tools such as Elasticsearch for semantic indexing and graph databases for contextual correlation, advanced video content organization, and retrieval. Emerging technologies like ONNXpromoted AI model portability across heterogeneous hardware. While early 5G adoption helped reduce communication latency. Nevertheless, coordinating distributed inference, handling heterogeneous edge hardware, and achieving seamless edge–cloud orchestration continued to present formidable challenges.

(4) 

Hybrid Cloud-Edge AI and FL (2020–2022)

This phase marked a turning point in VBDA system design, defined by the rise of hybrid architectures that integrated real-time edge inference with cloud-based training and coordination [11]. FL gained traction as a privacy-preserving paradigm, enabling model training directly on distributed edge devices without transferring raw video data [14].

During this time, lakehouse platforms like Delta Lake (https://delta.io, accessed on 18 May 2025) were increasingly adopted to manage large-scale, structured video data, offering both scalability and transactional consistency. Semantic indexing and entity modeling were enhanced by the use of graph databases such as Neo4j [30], while Apache NiFi played a key role in orchestrating complex, high-throughput video processing workflows [11]. Despite these advancements, the coordination of distributed models and the optimization of video analytics across heterogeneous infrastructure continued to pose significant challenges.

(5) 

AI-Driven Autonomous Video Processing (2023–2025)

The current stage of VBDA marks a shift toward autonomous, AI-native systems capable of handling complex video analytics with minimal human intervention. Advanced generative models, including GANs, ViT, and diffusion architectures, are increasingly applied in tasks such as anomaly detection, semantic summarization, and predictive surveillance [31,32]. Although these models offer promising performance, their applicability in real-time systems is constrained by substantial computational overhead.

Concurrently, blockchain technologies are being leveraged to support secure metadata logging, transparent access control, and tamper-resistant video sharing [33]. The integration of privacy-preserving mechanisms such as Zero-Knowledge Proofs and decentralized consensus further reinforces compliance with data protection standards. Future advancements, such as 6G networks and quantum-enhanced video processing, are expected to drive the emergence of scalable, resilient, and privacy-conscious VBDA systems [5].

In summary, VBDA has evolved from centralized, rule-based systems to hybrid, AI-driven architectures that effectively address latency, scalability, and data privacy challenges. Key milestones, ranging from cloud-based batch processing to edge computing and FL, have laid the foundation for more adaptive and resilient video analytics frameworks. Modern platforms now integrate generative models, semantic indexing, and decentralized training to enable real-time, autonomous video understanding. Looking forward, advances in explainable AI, privacy-preserving computation, and next-generation infrastructure such as 6G networks and quantum-enhanced processing are expected to drive the development of secure, intelligent, and context-aware VBDA systems for critical real-time applications [9,20,32].

2.2. Evolution of Architectural Paradigms in VBDA

Over the past decade, VBDA architectures have progressively transitioned from rigid centralized pipelines to distributed and collaborative models. Traditional systems were monolithic and tightly coupled to specific applications, typically running on standalone servers and using tools such as FFmpeg or GStreamerfor basic media processing [24,34,35]. Although suitable for offline tasks, these early setups presented significant limitations in scalability, modularity, and integration with big data ecosystems, rendering them ineffective for high-throughput or real-time video analytics.

This evolution phase was marked by the introduction of distributed batch processing frameworks, including Apache Hadoop and Apache Spark, which enabled greater parallelism and scalability [36,37]. These architectures enabled large-scale video indexing and archive-based analysis, proving effective for retrospective tasks. However, their batch-oriented nature came with high latency and reliance on centralized coordination, making them unsuitable for time-sensitive domains like smart surveillance or autonomous systems.

To support real-time inference and more dynamic workloads, cloud-centric architectures emerged as a flexible and scalable alternative. These systems utilized elastic cloud resources for deep learning tasks such as object detection, summarization, and large-scale storage [11,28]. Despite offering scalability, centralized cloud designs introduce latency, bandwidth overhead, and regulatory concerns associated with transmitting raw video data [38], thereby necessitating decentralized alternatives.

Edge computing addressed these limitations by pushing inference and preprocessing closer to data sources, including IP cameras and sensors. This approach reduced network load and latency, enabling faster decision-making in critical applications like autonomous driving and real-time surveillance [4,5]. However, edge systems continue to encounter challenges related to hardware heterogeneity, constrained computational resources, and synchronization of models across distributed nodes [3,14].

In response, hybrid cloud–edge architectures have become the dominant model in modern VBDA. These systems offload real-time tasks to edge devices while leveraging the cloud for orchestration, storage, and training [4,20]. Hybrid architectures offer a balanced trade-off between latency, scalability, and data privacy, yet they introduce orchestration complexity and unresolved challenges in distributed indexing, workload balancing, and coordination [5].

Hybrid architectures are increasingly evolving to incorporate generative AI and context-aware orchestration mechanisms. Recent paradigms within the edge-to-cloud spectrum are designed to enable semantic-level video understanding through capabilities such as real-time summarization, anomaly detection, and autonomous analytics [31]. This architectural continuum facilitates seamless coordination between resource-constrained edge devices and scalable cloud infrastructures [39], thereby enabling distributed intelligence across heterogeneous computing layers. Nevertheless, the intensive computational requirements of generative and multimodal AI models introduce new challenges. These include the need for advanced task scheduling, intelligent compression techniques, and resilient synchronization frameworks to ensure efficient operation across diverse environments.

In summary, the evolution of VBDA architectures reflects ongoing efforts to balance scalability, latency, and privacy. Early monolithic and batch-based systems lacked responsiveness and flexibility. Cloud-based models improved scalability but raised concerns over latency and data privacy. Edge architectures reduced delay and preserved data locality but faced resource and coordination limits. Hybrid designs, integrating edge and cloud, describe a balanced approach with adaptive workload distribution and semantic analytics. However, challenges remain in model management, device interoperability, and orchestration. Advancing VBDA requires the integration of these architectural strategies through intelligent coordination, enabling scalable, real-time, and privacy-aware video analytics systems.

2.3. Technologies and AI Enhancements in VBDA

The foundation of VBDA has been significantly influenced by advances in distributed computing, real-time stream processing, and intelligent data infrastructure. The shift from batch-oriented processing to real-time streaming and from centralized systems to edge-based deployments reflects increasing demands for low-latency, scalable, and privacy-aware video analytics solutions.

2.3.1. Enabling Technologies in VBDA

Real-time streaming frameworks such as Apache Kafka and Apache Flink play a central role in video data ingestion and analysis. Kafka enables scalable and fault-tolerant pipelines, while Flink supports low-latency, stateful processing with fine-grained event handling [40]. Nevertheless, large-scale VBDA environments continue to face unresolved challenges in adaptive resource allocation and seamless pipeline integration.

Distributed data processing platforms continue to serve as key infrastructure for handling massive video workloads. Hadoop offers cost-effective long-term storage for archived video but lacks native support for streaming [41]. Spark improves processing speed through in-memory execution, yet is less suited for real-time video stream analytics [42]. Apache Flink, while designed for streaming, still faces deployment limitations in edge-centric and latency-sensitive contexts [11].

Video data storage systems have also evolved to manage the volume and complexity of multimedia streams. Hadoop HDFS supports durable, large-scale storage but suffers from high query latency [6]. Cloud-based solutions like Amazon S3 provide elastic storage capacity but incur latency and bandwidth costs for real-time access [5]. NoSQL and graph-based databases offer flexible schemas for representing semantic relationships among video elements such as scenes and objects, albeit with trade-offs in consistency guarantees and query performance [43,44].

To address these limitations, hybrid storage models that combine SQL, NoSQL, and data lakes are under active investigation [45]. AI-driven storage mechanisms, including self-supervised video indexing and adaptive caching, are actively investigated to enhance storage efficiency and minimize retrieval latency [46].

2.3.2. AI Capabilities and Integration

Recent advancements in artificial intelligence have substantially enhanced the functional scope of VBDA systems, particularly in tasks such as object detection, activity recognition, summarization, and semantic interpretation of large-scale video data.

AI-driven video analytics commonly employ deep learning models, including convolutional neural networks (CNNs) [47] and Transformer-based architectures [48]. State-of-the-art detectors such as YOLO, Vision Transformers (ViT), and EfficientDet have demonstrated high accuracy in dynamic and complex video environments [49]. While these models are well-suited for high-level tasks such as face recognition and behavior detection, their deployment in real-time settings is often limited by substantial computational demands [5].

Edge AI offers a promising solution for reducing latency by shifting inference closer to the data source. By processing frames locally, edge devices can minimize reliance on cloud infrastructure and reduce bandwidth usage [50]. Lightweight architectures such as TinyML enable on-device analysis for continuous video streams. Nonetheless, the performance of edge-based inference is frequently limited by hardware constraints, particularly in terms of memory capacity and computational throughput [51]. Moreover, achieving synchronized inference between edge and cloud environments remains a persistent challenge [52].

FL further supports privacy-preserving analytics by enabling decentralized model training across distributed nodes without transferring raw video data [53]. This paradigm is particularly useful in privacy-sensitive domains such as healthcare and surveillance. Nonetheless, FL introduces communication overhead, and synchronization becomes increasingly complex in the presence of non-IID data and unreliable network conditions [54].

In recent years, generative models such as GANs and diffusion-based architectures have gained traction in tasks like video augmentation, anomaly detection, and spatiotemporal prediction [55]. These models enhance semantic understanding and dataset diversity but are often computationally intensive, limiting their applicability in latency-sensitive scenarios [56].

In parallel, blockchain-based approaches have been explored to support secure provenance tracking, access control, and authentication in decentralized video systems. Smart contracts enable immutable logging of video events and transparent authorization mechanisms [57]. However, the high computational cost and latency associated with current blockchain protocols hinder their deployment in real-time video pipelines [33].

Despite notable progress, VBDA systems still face substantial barriers related to latency, model complexity, and infrastructure coordination. Edge–cloud coordination lacks robust synchronization, and privacy-preserving methods such as FL increase communication load. In addition, adaptive video indexing and semantic retrieval continue to be computationally demanding, especially in large-scale and dynamic environments.

2.4. Problem Taxonomy and Applications of Video Processing in VBDA

VBDA encompasses a wide range of processing tasks aimed at addressing the inherent complexities of large-scale video interpretation and analysis. This section introduces a problem-centric taxonomy, organized around core functions such as object detection, activity recognition, anomaly detection, and content-based retrieval. Each task is systematically aligned with practical use cases in domains such as surveillance, healthcare, transportation, and smart retail. By linking technical approaches to application needs, the taxonomy provides a structured lens to evaluate the effectiveness, scalability, and real-world applicability of existing VBDA systems.

2.4.1. Object Detection and Tracking

Object detection and tracking are foundational components of VBDA, enabling automated monitoring of entities such as people, vehicles, and objects in dynamic environments. These tasks underpin applications in surveillance, traffic control, and sports analytics [5]. Whereas early systems relied on handcrafted feature-based techniques such as HOG and SVM [58], recent approaches have embraced deep learning models like YOLO, Faster R-CNN, and SSD, which are specifically optimized for edge deployment and real-time inference [59,60]. Tracking is often handled using DeepSORT [61], FairMOT [62], or Siamese networks [63], while 3D object detectors like PointNet [64] and VoxelNet [65] are gaining traction in autonomous driving and robotics.

2.4.2. Activity and Behavior Analysis

Human activity recognition plays a pivotal role in contextual video understanding, particularly in applications such as fall detection, violence recognition, retail analytics, and interactive media [5,66]. The field has evolved from using handcrafted features to adopting deep learning architectures, including 3D CNNs [67], LSTMs [68], and Transformer-based models. Skeleton-based approaches [69] provide fine-grained motion analysis and are particularly effective in healthcare and smart home environments [70].

2.4.3. Anomaly Detection

Anomaly detection focuses on identifying irregular or unexpected events in video streams, particularly in security-critical domains such as industrial monitoring, smart surveillance, and emergency response [5]. Techniques range from unsupervised models like autoencoders [71] and isolation forests [72] to more recent methods leveraging graph neural networks (GNNs) [73] and Transformers [74]. Benchmark datasets including UCSD [75], Avenue, and ShanghaiTech [76], are commonly used for evaluation.

2.4.4. Crowd Analysis and Management

Crowd analytics addresses the need for real-time understanding of group movement, density estimation, and behavioral anomalies in densely populated areas. These techniques are critical for ensuring event safety, optimizing public space design, and managing pedestrian movement in high-density environments [77]. Approaches have evolved from traditional optical flow and density map estimation to GNN-based and Transformer-based models for dynamic prediction. Benchmark datasets such as Mall [78], UCF-QNRF [79], and ShanghaiTech support comparative evaluation.

2.4.5. Content-Based Video Retrieval and Understanding

Semantic video retrieval enables efficient access to relevant content in massive video repositories, supporting applications in surveillance triage, content moderation, and multimedia recommendation. Recent advances in cross-modal learning have led to models like Show and Tell [80], VideoBERT [81], and Frozen in Time [82], which align visual and textual modalities to support tasks such as zero-shot captioning and search. Large-scale datasets such as YouTube-8M [83], ActivityNet [84], and MSR-VTT [85] have been pivotal for training and evaluation.

2.4.6. Video Summarization and Captioning

Video summarization transforms extended footage into succinct representations, thereby facilitating efficient review, archival, and content personalization. This is particularly valuable in applications like surveillance review, broadcast editing, and education. Recent approaches go beyond keyframe extraction to include generative models such as VST [48], ViViT, and BLIP-2 for producing text summaries or abstract visual synopses. Commonly used benchmarks include SumMe [86] and TVSum [87].

A structured summary of key problem domains in VBDA is presented in

Table 1. Problem taxonomy, core technologies, and applications in VBDA.

Problem Class	Representative Technologies	Benchmark Datasets	Applications	References
Object Detection and Tracking	YOLO, Faster R-CNN, SSD; DeepSORT, FairMOT; PointNet, VoxelNet	COCO, KITTI, VisDrone, AICity	Surveillance, Traffic Monitoring, Autonomous Driving, Sports	[88,89,90,91]
Activity and Behavior Analysis	3D CNNs, LSTM, SlowFast, Transformers; OpenPose, HRNet; Multimodal Fusion	UCF-101, HMDB-51, NTU RGB+D, Kinetics	Healthcare, Human–Robot Interaction, Smart Spaces, Sports	[70,92,93,94]
Anomaly Detection	Autoencoders, Isolation Forest, GANs, GNNs, Transformers	UCSD, Avenue, ShanghaiTech	Intrusion Detection, Industrial Safety, Emergency Response	[5,74,95]
Crowd Analysis	CNN Density Maps, Optical Flow, YOLOv8, GNNs, Transformers	UCF-QNRF, ShanghaiTech, Mall	Event Safety, Urban Planning, Transit Optimization	[77,91,96]
Content-Based Retrieval and Understanding	VideoBERT, CLIP, FAISS, Pinecone, ElasticSearch	YouTube-8M, ActivityNet, MSR-VTT	Surveillance Forensics, Media Search, Smart Recommender	[82,93,97]
Video Summarization and Captioning	VST, ViViT, TimeSformer; BLIP-2, GPT-4V; Diffusion Models	TVSum, SumMe, Kinetics, Show and Tell	Surveillance Triage, Media Editing, Accessibility	[92,98,99]

, outlining core technologies, datasets, and application contexts. While these tasks have enabled practical deployments in domains such as transportation and healthcare, transitioning from research prototypes to scalable production systems remains hindered by unresolved technical challenges.

A primary barrier is scalability and system heterogeneity. Real-world VBDA must handle continuous, high-resolution video from diverse sources such as CCTV cameras, drones, and IoT devices. This introduces substantial demands on bandwidth, storage, and computational infrastructure. Coordinating processing across decentralized layers spanning edge and cloud adds complexity in terms of orchestration, resource allocation, and vendor interoperability.

Latency and responsiveness represent additional bottlenecks. Despite improvements in deep learning accuracy, many state-of-the-art models remain too resource-intensive for real-time inference on edge-constrained hardware. Meeting low-latency, high-throughput requirements calls for end-to-end optimization across both model design and system architecture.

Privacy and compliance are equally critical, especially in domains involving sensitive video data. Regulatory frameworks such as GDPR and HIPAA require secure governance practices, including FL, encrypted computation, and verifiable data workflows to prevent misuse and ensure data sovereignty.

Finally, limited labeled data remains a critical bottleneck in advancing robust VBDA systems. Most current approaches rely heavily on large-scale supervised learning, yet collecting and annotating video data is both time-consuming and costly. To address this challenge, self-supervised and semi-supervised learning techniques have been increasingly adopted as alternatives to fully supervised approaches. Nevertheless, these models frequently encounter difficulties in generalization, especially under distribution shifts or deployment in novel operational contexts [66].

3. VBDA Architectures and Core Technologies

This section offers a comprehensive analysis of VBDA system architectures and enabling technologies, synthesizing developments from 2018 to 2025. The first part provides a structured comparison of foundational components, ranging from data acquisition and storage to processing and analytics frameworks, highlighting how these elements collectively support real-time, scalable video understanding. The second part focuses on emerging trends that define the trajectory of next-generation VBDA platforms, including hybrid edge–cloud architectures, Edge AI, FL, and AI-driven semantic indexing. Together, these innovations contribute to the development of intelligent, privacy-aware, and resource-efficient systems capable of meeting the growing demands of dynamic, high-throughput video environments.

3.1. System-Level Architectural Models in VBDA

Building on the evolutionary trends outlined earlier, this section synthesizes the VBDA architecture landscape into four core system-level models: Centralized, Cloud-Centric, Edge Computing, and Hybrid Cloud–Edge. These models reflect the prevailing deployment strategies in contemporary systems, each characterized by specific trade-offs in latency, scalability, privacy, cost, and complexity. They serve as foundational blueprints for designing and optimizing VBDA systems across diverse operational contexts.

Table 2. Comparison of VBDA system architectures.

Architecture	Key Characteristics	Advantages	Limitations	VBDA Suitability	Reference
Centralized	Single-node processing; local storage; no distribution.	Low deployment cost; simple management; consistent control.	High latency; poor scalability; privacy risks; SPOF.	Limited: Suitable for small-scale or archival VBDA.	Wu et al. (2022) [6] Wang et al. (2019) [100] Chen et al. (2021) [101]
Cloud-Centric	Compute and storage offloaded to cloud; centralized orchestration.	High scalability; deep learning support; elastic compute; lower hardware burden.	Latency from data upload; privacy concerns; higher recurring cost.	Moderate to High: Batch analytics, model training.	Chen et al. (2021) [101] Xu et al. (2020) [102] Zhu et al. (2018) [103]
Edge Computing	Local inference near data source; minimal cloud dependency.	Ultra-low latency; enhanced privacy; bandwidth-efficient; real-time response.	Limited compute; synchronization overhead; hardware heterogeneity.	High: Suitable for real-time apps (e.g., smart cities, surveillance).	Liu et al. (2022) [104] Sharshar et al. (2023) [105] Fenil et al. (2019) [106]
Hybrid Cloud–Edge	Edge handles real-time inference; cloud for training, storage, orchestration.	Balanced latency and scalability; privacy-aware; supports FL; adaptive workload distribution.	Complex deployment; resource orchestration challenges; higher initial setup cost.	Very High: Best for large-scale, real-time VBDA systems.	Alam et al. (2020) [107] Gayakwad et al. (2024) [108] Mouradian et al. (2020) [109]

summarizes a structured comparison of these architectures, outlining their respective strengths and limitations in real-world application scenarios.

Centralized architectures offer simplicity and low cost, making them suitable for small-scale or archival VBDA systems. However, they rely on a monolithic design with single-node processing, resulting in poor scalability and high latency. A critical risk is the Single Point of Failure (SPOF), where the failure of a central node disrupts the entire system. These architectures are best for static environments with low-performance demands [6,100].

Cloud-Centric architectures leverage elastic computing and storage, supporting large-scale analytics and model training, while powerful, their dependency on remote data transfer leads to latency and privacy challenges, limiting their use in real-time scenarios. As shown by Chen et al. (2021) [101] and Xu et al. (2020) [102], they are most effective for batch analytics, archival processing, and AI training pipelines.

Edge computing architectures enable low-latency, privacy-preserving analytics by performing inference close to data sources. Ideal for real-time applications (e.g., smart surveillance), they reduce bandwidth usage and ensure timely responses. However, edge devices face hardware constraints and synchronization complexity [104,105], requiring lightweight models and careful orchestration.

Hybrid cloud–edge architectures integrate the advantages of both paradigms, supporting real-time processing at the edge while utilizing the cloud for training, storage, and system orchestration. They support FL and dynamic workload allocation, making them highly scalable and adaptive [107,109]. While complex to deploy, they offer the best trade-off for future-proof VBDA systems across diverse and dynamic environments. Hybrid cloud–edge architectures are best suited to meet the evolving demands of VBDA. Their adaptability, scalability, and built-in support for privacy-preserving, AI-driven video analytics make them the most promising foundation for future intelligent video systems.

3.2. VBDA Core Technologies

Next-generation VBDA systems rely on integrated technological stacks to achieve real-time performance, intelligent interpretation, and secure, scalable data management. As video data continues to grow in volume, velocity, and variety, system architectures must evolve to handle end-to-end processing needs from ingestion and distributed computation to semantic understanding and visualization. This section provides a focused analysis of five critical components underpinning modern VBDA platforms: scalable data processing frameworks, deep learning-based video analytics, FL combined with edge AI, advanced storage and retrieval infrastructures, and visualization tools for system interpretability. These technologies serve as the core foundation for intelligent VBDA pipelines, enabling low-latency inference, model coordination across devices, and privacy-conscious data management across diverse domains.

3.2.1. Scalable Big Data Processing Frameworks

The transition from batch-oriented to real-time stream processing has fundamentally reshaped the architecture and responsiveness of VBDA systems. As summarized in

Table 3. Scalable big data processing frameworks for VBDA.

Reference	Year	Hadoop (Map Reduce)	Kafka	Spark	Flink	NiFi	Hive	Impala	Pig	Storm
Gayakwad et al. [108]	2024			✓			✓
Hlaing et al. [111]	2023		✓			✓
Abusalah et al. [10]	2023		✓	✓	✓
Raptis et al. [110]	2023		✓			✓
Xu et al. [112]	2022		✓		✓
Shafiyah et al. [113]	2022		✓				✓		✓
Yang et al. [114]	2021			✓			✓	✓
Tripathi et al. [115]	2021		✓	✓						✓
Melenli et al. [116]	2021		✓	✓			✓	✓
Alam et al. [21]	2020	✓	✓	✓
Alam et al. [107]	2020		✓	✓
Alam et al. [11]	2020	✓	✓	✓	✓
Mendhe et al. [117]	2020	✓		✓
Khan et al. [118]	2020		✓	✓
Uddin et al. [119]	2019		✓	✓
Supangkat et al. [120]	2019	✓	✓	✓			✓			✓
Wang et al. [100]	2019	✓	✓	✓	✓	✓
Dai et al. [30]	2019		✓	✓
Zhang et al. [121]	2019		✓	✓						✓
Lv et al. [122]	2018		✓
Munshi et al. [123]	2018	✓		✓			✓	✓
Total		6	17	16	4	3	6	3	1	3

, Apache Kafka has established itself as a cornerstone for real-time ingestion and message queuing, offering high throughput, scalability, and robust fault tolerance through its distributed architecture [110]. Apache Spark remains a widely adopted engine, valued for its ability to unify batch and micro-batch processing, thus supporting scalable analytics across both historical and near-real-time video streams [10].

Apache Flink has gained traction for its native support of true stream processing and stateful computations, enabling low-latency analytics in applications such as anomaly detection, traffic monitoring, and intelligent surveillance [11]. Complementing these systems, Apache NiFi is increasingly used to orchestrate end-to-end VBDA pipelines, offering low-code automation for ingesting, transforming, and routing video data between edge and cloud components [113].

Although Hadoop MapReduce previously played a central role in batch video analytics, its relevance has declined in modern VBDA pipelines due to inherent latency and a lack of flexibility. It is now primarily used for back-end archival processing and long-term video storage, rather than front-line real-time inference [30,107].

More recently, Apache Pulsar (https://pulsar.apache.org, accessed on 18 May 2025) has emerged as a potential alternative or complement to Kafka. Its support for multi-tenancy, geo-replication, and a decoupled architecture for storage and computing make it suitable for hybrid cloud–edge deployments. Although its adoption in VBDA remains limited as of 2024, its architectural flexibility positions it as a promising component in next-generation, AI-integrated video analytics systems [10].

3.2.2. AI-Driven Video Analytics

AI models play a pivotal role in enabling spatial, temporal, and semantic understanding of complex video data. As detailed in

Table 4. AI and deep learning for video analytics in VBDA.

Reference	Year	YOLO	R-CNN Family	3D-CNN/ SlowFast	LSTM	Transformers	GANs	CLIP/ VLMs	SSL Methods	Anomaly Models	MOT Trackers
Yang et al. [124]	2025				✓		✓			✓
Rahman et al. [125]	2025	✓			✓	✓
Ahamad et al. [126]	2024	✓	✓
Liang et al. [32]	2024				✓	✓	✓	✓
Jung et al. [127]	2024	✓		✓						✓	✓
Zhao et al. [31]	2024				✓	✓	✓	✓	✓
Su et al. [8]	2024							✓
Sathupadi et al. [51]	2024	✓	✓		✓
Mon et al. [128]	2024	✓			✓						✓
Hassan et al. [54]	2024	✓	✓		✓					✓
Liu et al. [129]	2023		✓
Wu et al. [130]	2023				✓	✓
Alpay et al. [131]	2023	✓	✓			✓		✓	✓
Badidi et al. [5]	2023	✓	✓		✓					✓	✓
Kwon et al. [4]	2022	✓	✓
Wu et al. [6]	2022				✓
Na et al. [33]	2022		✓
Ulhaq et al. [132]	2022	✓	✓						✓
Apostolidis et al. [133]	2021				✓	✓	✓
Kul et al. [134]	2021	✓	✓
Alam et al. [107]	2020	✓
Khan et al. [118]	2020	✓
Subudhi et al. [28]	2019						✓
Dai et al. [30]	2019				✓	✓
Geng et al. [29]	2019	✓	✓	✓	✓	✓
Total		14	11	2	13	8	5	4	3	4	3

, a wide range of deep learning architectures has been employed in VBDA from 2018 to 2025, reflecting significant evolution in both model design and application scope. Among object detection models, YOLO and R-CNN variants remain the most widely adopted. YOLO is widely adopted in latency-sensitive applications owing to its real-time inference capabilities. In contrast, R-CNN models, though more computationally demanding, are favored in scenarios requiring high detection precision, such as security surveillance and edge analytics [51].

For temporal modeling, LSTM-based architectures have gradually been supplanted by Transformer models, which demonstrate superior capabilities in capturing long-range dependencies, multi-modal learning, and contextual scene understanding [130]. These models are now commonly used for video summarization, action recognition, and event detection across heterogeneous VBDA workloads. In parallel, self-supervised learning (SSL) techniques such as contrastive learning are gaining prominence for their ability to reduce reliance on labeled data [132]. SSL enables systems to extract meaningful features from raw video streams, making it highly relevant for tasks such as anomaly detection, motion prediction, and behavior recognition, especially under privacy constraints where the annotation is costly or infeasible.

Generative models, including GANs and diffusion-based architectures, contribute to simulation, rare event reconstruction, and data augmentation. For instance, retrieval-augmented generation techniques have been shown to improve the realism and contextual relevance of synthesized video sequences in simulation-based training and safety-critical environments [31]. Additionally, vision–language models (VLMs) such as CLIP, VideoBERT, and BLIP are now integral to cross-modal VBDA tasks [81,135,136]. These models enable zero-shot video retrieval, caption generation, and semantic alignment between visual and textual modalities, supporting applications in forensic search, intelligent surveillance, and human-AI interaction [32].

Beyond mainstream models, the integration of multi-object tracking (MOT), SSL-enhanced anomaly detection, and hybrid spatiotemporal architectures point to emerging directions in adaptive video intelligence [5]. These techniques provide deeper behavioral insight, enabling more robust reasoning in dynamic environments.

Overall, the AI landscape in VBDA demonstrates a clear shift from conventional deep learning models to more flexible, multi-modal, and privacy-aware architectures. This transition aligns with the design principles of the proposed ViMindXAI platform, which emphasizes scalable learning, decentralized intelligence, and explainable analytics across diverse video-driven domains.

3.2.3. Federated Intelligence and Edge AI for Distributed VBDA

The integration of FL and Edge AI has become a cornerstone for building scalable, privacy-aware intelligence in distributed VBDA systems. Rather than sending raw video to centralized servers, FL allows model training to occur locally at edge nodes where data is originally captured by sharing only model updates. This decentralized strategy minimizes bandwidth usage, enhances responsiveness, and aligns with strict data privacy regulations. As a result, it is especially well-suited for sensitive, high-volume applications like smart surveillance, healthcare monitoring, and urban mobility analytics.

As summarized in

Table 5. Federated learning and edge AI for VBDA.

Reference	Year	FL Algorithms		FL Frameworks		Privacy	Model Optimization			Edge AI Hardware			Edge-Cloud Orchestration
		FedAvg	FedProx	FedML	TFF	Techs	Prun.	Quant.	KD	Jetson	Coral	Movidius
Zeng et al. [137]	2025	✓	✓
Hridi et al. [139]	2024	✓				✓
Hakam et al. [140]	2024									✓	✓	✓
Su et al. [8]	2024	✓											✓
Wang et al. [3]	2024	✓		✓	✓	✓	✓		✓	✓		✓	✓
Prangon et al. [50]	2024	✓				✓							✓
Pandya et al. [9]	2023	✓				✓							✓
Badidi et al. [5]	2023	✓				✓				✓	✓	✓	✓
Ramamoorthi et al. [138]	2023	✓	✓	✓	✓								✓
Brecko et al. [14]	2022	✓			✓	✓	✓						✓
Das et al. [13]	2022								✓	✓			✓
Kwon et al. [4]	2022							✓	✓	✓			✓
Rocha et al. [141]	2020	✓	✓	✓	✓
Total		10	3	3	4	6	2	1	3	5	2	3	9

, FedAvg remains the most widely adopted FL paradigm due to its simplicity and communication efficiency [137]. However, it exhibits suboptimal performance in environments characterized by non-IID data, a common scenario in real-world video deployments. To mitigate this, improved variants such as FedProx have been proposed, offering better convergence and robustness under heterogeneous data distributions. Meanwhile, FL platforms such as FedML and TensorFlow Federated (TFF) provide scalable toolkits for deploying distributed VBDA applications with modular design and cross-platform compatibility [138].

Efficient inference at the edge requires advanced model compression techniques to accommodate resource-constrained hardware. Methods such as pruning, quantization, and knowledge distillation have proven effective in reducing model size while preserving inference accuracy [3]. Edge devices, including NVIDIA Jetson, Google Coral TPU, and Intel Movidius are frequently employed in VBDA systems to support real-time tasks such as object detection, anomaly recognition, and behavior analysis [5].

Nonetheless, VBDA systems continue to face persistent challenges, particularly in model synchronization, resource heterogeneity, and orchestration across distributed infrastructures. Furthermore, workload balancing is complicated by diverse computational capabilities, memory footprints, and energy constraints across nodes. To address these issues, recent developments have focused on lightweight orchestration frameworks and container-based deployment tools such as KubeEdge and NVIDIA FLARE, which enable dynamic scheduling, resource-aware task allocation, and seamless edge–cloud coordination [8].

3.2.4. Storage and Indexing in VBDA Systems

Efficient storage and indexing are critical to managing the scale, speed, and variety of video data in VBDA systems. Contemporary solutions favor hybrid architectures that blend data lakes with data warehouses, offering both flexibility and structured querying. In parallel, AI-driven indexing techniques are increasingly used to automate metadata generation, support compression, and enable scalable organization of video content. As shown in

Table 6. Technologies for data storage and management.

Reference	Year	Hadoop (HDFS)	Cloud Storages			NoSQL DBs			Graph DB	SQL DBs		Auxiliary Techs	AI-Driven Indexing and Retrieval
			Amazon S3	Google Cloud	Azure Data Lake	HBase	Mongo DB	Cassandra		MySQL	Postgre SQL
Gayakwad et al. [108]	2024		✓										AI Search
Zhang et al. [142]	2024								✓				Graph-based Retrieval
Divya et al. [143]	2024		✓	✓	✓
Zhai et al. [7]	2024	✓	✓	✓	✓							Elasticsearch, Phoenix	FAISS
Abdallah et al. [144]	2023								✓				CLIP
Raptis et al. [110]	2023	✓						✓			✓	Zookeeper, Elasticsearch
Shafiyah et al. [113]	2022	✓				✓	✓					Zookeeper, Elasticsearch
Gu et al. [145]	2021								✓			RDF	VideoBERT
Yang et al. [114]	2021	✓				✓				✓		Phoenix
Meleni et al. [116]	2021	✓				✓				✓		Zookeeper Elasticsearch
Kul et al. [134]	2021	✓									✓	Zookeeper Docker
Alam et al. [21]	2020	✓				✓						Zookeeper Phoenix
Khan et al. [118]	2020	✓				✓						Zookeeper Phoenix
Xu et al. [102]	2020	✓				✓						Thrift Elasticsearch
Mendhe et al. [117]	2020						✓					Elasticsearch
Alam et al. [107]	2020	✓				✓						RDF
Alam et al. [11]	2020	✓			✓	✓	✓			✓
Uddin et al. [119]	2019	✓	✓	✓	✓	✓						Zookeeper Elasticsearch Phoenix
Dai et al. [30]	2019	✓				✓						Thrift	BLIP
Wang et al. [100]	2019	✓	✓	✓	✓		✓				✓	Docker, Elasticsearch
Zhang et al. [121]	2019	✓				✓						Zookeeper Docker
Munshi et al. [123]	2018	✓		✓									Google Knowledge Graph
Total		16	5	5	5	11	4	1	3	3	3

, these advances are designed to provide low-latency access, reduce storage costs, and support the dynamic requirements of large-scale video analytics.

Data lakes serve as the backbone of scalable video storage in VBDA systems, offering the flexibility to ingest diverse and unstructured content, including video streams, sensor logs, and metadata. Hadoop HDFS has played a central role due to its robustness and batch-processing capabilities, cited in many of the systems surveyed [11]. However, its high-latency access and rigid structure have led to a gradual transition toward cloud-native object storage platforms such as Amazon S3, Google Cloud Storage, and Azure Data Lake. These services provide elastic scalability and seamless integration with big data pipelines and AI workloads [7]. In more localized or hybrid settings, lightweight solutions such as MinIO present S3-compatible APIs with lower overhead, making them well suited for edge-centric deployments [11].

For managing dynamic and schema-flexible metadata, NoSQL databases such as MongoDB, Cassandra, and HBase are commonly used to support distributed ingestion and real-time querying. In parallel, graph databases like Neo4j and TigerGraph are increasingly adopted for modeling semantic relationships between objects, scenes, and events. These models enable sophisticated reasoning and contextual analytics, especially in surveillance and smart city scenarios [145].

Traditional data warehouses continue to play a vital role in structured analytics and metadata management. PostgreSQL is favored for its support of JSONB, spatial queries, and compatibility with tools such as Grafana and Superset, while MySQL remains useful for lightweight, transactional operations [11,108].

Emerging data lakehouse architectures, including Delta Lake, and Apache Iceberg, bridge the gap between flexible data lakes and structured data warehouses. They support ACID transactions, schema evolution, and real-time ingestion, while integrating tightly with processing engines such as Spark and Flink. These capabilities make them well-suited for hybrid and federated VBDA systems that require both agility and consistency [7].

AI-driven indexing frameworks are transforming video data management by enabling efficient organization, retrieval, and interpretation of content. Technologies such as CLIP, VideoBERT, BLIP, and FAISS [146] support semantic tagging, cross-modal retrieval, and zero-shot learning. When combined with platforms like Elasticsearch or RDF-based engines, these tools enable scalable, multimodal analytics across a variety of domains, including healthcare monitoring, autonomous vehicles, traffic analysis, and public safety [144,145].

In summary, VBDA storage architectures have transitioned from traditional structured warehouses and standalone data lakes to integrated data lakehouse platforms supported by AI-driven indexing. Each layer, including NoSQL systems, graph databases, and cloud-native object stores, addresses specific requirements for scalability, flexibility, and semantic comprehension. This convergence equips modern VBDA systems with the capability to perform low-latency, real-time analytics while ensuring privacy-aware video intelligence across distributed and heterogeneous environments.

3.2.5. Visualization and Interpretability

Table 7. Technologies for data visualization and interpretation.

Reference	Year	Tableau	Power BI	Zeppelin	Qlik Sense	Grafana	Kibana
Gayakwad et al. [108]	2024	✓
Kadam et al. [147]	2024	✓	✓
Kumar et al. [148]	2022	✓	✓		✓
Xu et al. [112]	2022						✓
Il et al. [149]	2022	✓	✓			✓	✓
Shafiyah et al. [113]	2022	✓		✓			✓
Mendhe et al. [117]	2020						✓
Wang et al. [100]	2019			✓			✓
Munshi et al. [123]	2018	✓
Total		6	3	2	1	1	5

describes an overview of the key visualization tools adopted in VBDA systems from 2018 to 2025, highlighting their integration scopes, functional roles, and typical application contexts. These tools support tasks ranging from real-time monitoring and operational oversight to statistical analysis, model debugging, and interactive system feedback.

Among enterprise solutions, Tableau and Power BI are widely used to deliver interactive dashboards for performance tracking and system health assessment [149]. Alternatives such as Qlik Sense and Apache Zeppelin (https://zeppelin.apache.org, accessed on 14 May 2025) are favored in scenarios requiring associative exploration and research-oriented debugging [112]. For time-series data and system logs, Grafana has gained popularity due to its flexible dashboarding, alerting features, and integration with Prometheus and InfluxDB [100], while Kibana remains prevalent in Elasticsearch-based environments [113].

In experimental and academic settings, Python libraries like Matplotlib, Seaborn, and Plotly enable customizable visualization of detection outputs, performance metrics, and training dynamics. Emerging visualization interfaces, including AR dashboards and 3D spatial renderings, are also being explored in high-risk domains such as disaster response and urban surveillance [117]. Although still limited in adoption, these immersive tools hold promise for improving spatial awareness and situational interpretation. Moving forward, the integration of explainable AI (XAI) elements such as heatmaps and semantic overlays into real-time dashboards is expected to enhance interpretability, human-AI interaction, and trust in automated VBDA systems.

Figure 2 illustrates the most commonly adopted technologies across VBDA platforms between 2018 and 2025, highlighting a clear progression toward scalable and intelligent video analytics. At the data processing layer, Apache Kafka and Apache Spark stand out as dominant frameworks, each adopted by approximately 11% of of the 24 surveyed systems, owing to their robustness in managing distributed stream and batch workloads. In the AI-driven analysis layer, YOLO (9%) and LSTM (8%) are widely utilized for object detection and activity recognition, while emerging approaches such as CLIP and Transformer-based models are increasingly applied to multimodal video understanding. At the edge–cloud orchestration level, the use of FL with FedAvg (6%) and deployment via KubeEdge (6%) reflects a growing emphasis on privacy-preserving, low-latency computation at the network edge. For data management, Elasticsearch (6%) and MongoDB (3%) are frequently employed to support scalable indexing and semantic retrieval. Visualization remains a critical component, with tools like Tableau facilitating real-time dashboards for informed decision-making. Notably, the most prevalent technologies in these layers, including Kafka, Spark, YOLO, LSTM, FedAvg, and Elasticsearch, collectively appear in more than 50% of the surveyed platforms. This reinforces their foundational role in the current VBDA ecosystem.

Synthesizing the key findings from Section 2 and Section 3, we identify several persistent challenges in the VBDA landscape:

• Latency and Scalability: Centralized cloud-centric designs often struggle with real-time analytics under high video throughput and geographically dispersed deployments.
• Data Heterogeneity: ulti-source video data varies in encoding formats, resolutions, frame rates, and modalities (e.g., visual, thermal, audio), posing standardization and fusion challenges.
• Privacy and Federated Learning: FL adoption in video domains remains limited due to non-IID data distribution, communication overhead, and explainability concerns.
• Semantic Understanding: While progress has been made in video understanding using AI models, semantic indexing, multimodal fusion, and real-time reasoning remain underdeveloped in practice.
• Deployment and Resource Management: Lack of modular, open source reference platforms hinders rapid deployment, especially in resource-constrained edge settings.

These challenges are synthesized from our survey of both the academic and industry literature between 2018 and 2025. To address them holistically, the following section introduces ViMindXAI, a hybrid, modular VBDA platform designed to operationalize cloud–edge coordination, semantic video analytics, and privacy-preserving learning in real-world scenarios.

4. ViMindXAI: A Scalable and Cognitive AI Platform for VBDA

4.1. Overview of the Proposed Platform Architecture

The proposed ViMindXAI platform introduces a next-generation hybrid architecture for VBDA as illustrated in Figure 3. The name ViMindXAI reflects the platform’s core design: “Vi” for video-centric data, “Mind” for semantic analysis and adaptive intelligence, “X” for extended scalability and innovation, and “AI” for the end-to-end use of artificial intelligence, from edge inference to federated learning and explainable reasoning. It seamlessly integrates edge and cloud computing to meet critical requirements in scalability, low-latency inference, and privacy-conscious processing. Unlike conventional centralized or cloud-only approaches, ViMindXAI enables intelligent workload orchestration, executing real-time analytics at the edge, utilizing the cloud for large-scale storage and deep model training, and deploying on-premise infrastructure for security-critical and context-sensitive operations.

ViMindXAI employs a unified Hybrid AI architecture that combines Edge AI, FL, and adaptive video storage to support essential VBDA tasks such as semantic understanding, anomaly detection, and privacy-aware surveillance. The platform is organized into four layers: the Data Acquisition Layer (DAL), the Data Infrastructure Layer (DIL), the Data and Application Layer (DaAL), and the User Experience Layer (UEL). Each layer incorporates mature and widely adopted technologies that align with prevailing trends in the VBDA ecosystem. Notably, foundational components such as Apache Kafka, Spark, YOLO, LSTM, Elasticsearch, and FedAvg, which appear in more than 50% of surveyed platforms, constitute the technological core of the ViMindXAI architecture, as illustrated in Figure 2. This strategic alignment ensures compatibility with modern deployment environments, reinforcing the platform’s scalability, resilience, and applicability across diverse domains, including smart cities, healthcare, and content moderation.

4.2. Layered Architecture and Functional Components

4.2.1. Data Acquisition Layer (DAL)

The DAL serves as the foundational tier of the ViMindXAI platform. It is responsible for ingesting, integrating, and preprocessing video data from diverse sources, including surveillance cameras, drones, dashcams, IoT sensors, and streaming services. Its main objective is to support low-latency, on-device filtering before forwarding data to higher layers, thereby reducing bandwidth usage and enhancing system responsiveness.

The DAL comprises two main modules: Data Collection and Edge AI Preprocessing and Optimization. The first module aggregates multi-source video and sensor inputs into a unified stream. The second module executes real-time inference, denoising, compression, and metadata extraction using lightweight models deployed on edge hardware such as NVIDIA Jetson Nano/Xavier, Intel Neural Compute Stick (NCS2), and Google Coral TPU. These embedded devices provide sufficient GPU/TPU acceleration for on-device AI tasks, enabling scalable and cost-effective inference in resource-constrained environments.

These processes are accelerated by stream engines like Apache Flink and Spark Streaming. AI functionalities in this layer enable real-time event detection, intelligent caching, and dynamic prioritization of critical segments. FL is embedded to support local model training and inference, enabling privacy-preserving analytics at the edge.

Technologies such as Apache Kafka and NiFi manage data ingestion, while Flink and Spark handle distributed stream computation. ONNX ensures interoperability across AI models. The output of the DAL, consisting of cleaned and enriched metadata, is forwarded to the Data Infrastructure Layer for indexing and persistent storage. It also feeds the FL module in the application layer to support collaborative updates.

In summary, the DAL streamlines early-stage video analytics by enabling efficient edge inference, reducing transmission overhead, and facilitating scalable ingestion from diverse video sources.

4.2.2. Data Infrastructure Layer (DIL)

The DILconstitutes a central component of the ViMindXAI architecture, offering a scalable and intelligent foundation for storing, organizing, and retrieving large-scale video data. Tailored to the unique requirements of VBDA, this layer combines cloud and on-premises storage with real-time metadata indexing and AI-enhanced retrieval pipelines, thereby enabling high-throughput, low-latency, and semantically aware search capabilities. The hardware foundation supporting this layer includes cloud-native storage services such as AWS EC2/S3, GCP Cloud Storage, on-premises clusters with NVMe SSD arrays for high-speed access, and GPU-accelerated nodes such as NVIDIA A100, T4 for semantic indexing and batch analytics. RAID configurations and hardware-assisted encryption modules are used to ensure data redundancy, fault tolerance, and security at rest. The DIL is composed of three main modules: AI-Driven Storage Orchestration, Data Lakehouse, and Semantic Indexing.

(1) 

AI-Driven Storage Orchestration

This module is a core component of the ViMindXAI platform, designed to manage large-scale video data intelligently, efficiently, and in compliance with privacy regulations. It orchestrates data storage across edge, cloud, and on-premise systems, ensuring high availability, scalability, and regulatory alignment in dynamic VBDA environments. This module comprises three tightly integrated layers: Big Data Storage, AI-driven Storage Optimization, and Privacy-Aware Synchronization.

• Big Data Storage: establishes a hybrid, fault-tolerant infrastructure for ingesting and managing video data across diverse environments. Cloud services like Amazon S3 offer elastic scalability, while on-premise platforms such as MinIO, and Hadoop HDFS ensure secure, policy-compliant storage. At the edge, Ceph supports low-latency buffering and ingestion. Efficient video encoding formats such as MPEG-4, H.264, and H.265 are integrated into the pipeline to reduce transmission load and long-term storage costs, especially in surveillance and healthcare contexts [150]. Adaptive compression is employed to align bitrate with network and storage constraints [151]. This multi-layered storage backbone supports lifecycle management, distributed retention, and efficient data provisioning for downstream analytics within Lakehouse architectures.
• AI-Based Storage Optimization: leverages reinforcement learning to automate storage tiering decisions, guided by observed usage patterns and anticipated access frequency. This enables dynamic tiering across hot, warm, and cold layers, balancing performance and cost-efficiency. High-speed caching with Redis accelerates access to frequently queried data, while observability tools like Grafana, MLFlow, and Prometheus monitor system performance and guide optimization. To enable intelligent storage tiering, we integrate a Deep Q-Network (DQN)-based reinforcement learning agent [152]. The agent interacts with the environment to learn optimal retention policies, adapting dynamically to changes in data access frequency, latency requirements, and cost-performance trade-offs. This allows ViMindXAI to proactively allocate resources across hot, warm, and cold tiers under varying workload conditions.
• Privacy-Aware Synchronization: facilitates secure and regulation-compliant data transfers across distributed environments, maintaining confidentiality and integrity in multi-tenant deployments. It employs federated synchronization to preserve data locality and prevent unnecessary exposure of sensitive content. Containerized deployment via Docker and Kubernetes ensures portability and scalability, while Apache Zookeeper and secure API gateways manage access control and coordination in multi-tenant setups. This layer is instrumental in ensuring compliance with data protection regulations, including GDPR and HIPAA.

AI-Driven Storage Orchestration equips ViMindXAI with adaptive, intelligent, and regulation-compliant video data management capabilities, essential for large-scale operations in heterogeneous and privacy-sensitive settings.

(2) 

Data Lakehouse

This module offers a unified architectural framework that merges the flexibility of data lakes with the performance and consistency of traditional data warehouses. It enables ViMindXAI to manage and analyze large-scale, heterogeneous video and sensor data across multiple formats and modalities. This hybrid architecture enables both streaming and batch analytics, supporting structured metadata processing, semantic enrichment, and long-term data archival. Organizing data into logical zones such as raw, staging, and curated layers supports reproducible ETL workflows, governance, and traceability.

Representative technologies such as Delta Lake and Apache Iceberg are leveraged to implement this architecture. Delta Lake, optimized for Spark-based workloads, provides ACID transactions, time-travel, and schema enforcement ideal for structured processing and model training pipelines. In contrast, Apache Iceberg offers native support for multiple query engines such as Spark, Flink, and Trino and is well-suited for federated environments with complex partitioning, schema evolution, and snapshot isolation. Together, these technologies provide ViMindXAI with a scalable, interoperable, and regulation-compliant data infrastructure designed for both real-time and historical video analytics.

(3) 

Semantic Indexing

This module interfaces with Delta Lake to provide ACID-compliant storage and supports both batch and streaming workloads. In large-scale deployments, selecting technologies for semantic indexing requires balancing trade-offs among performance, scalability, and infrastructure complexity. FAISS offers fast, memory-efficient indexing for on-premise environments but lacks native support for distributed execution or elastic scaling, necessitating custom orchestration in multi-node setups. CLIP enhances semantic precision through strong vision-language alignment but incurs high computational costs for embedding generation, often requiring GPU clusters or inference acceleration frameworks in large-scale or real-time applications. Pinecone, as a cloud-native vector database, provides built-in elasticity and managed scalability, streamlining deployment. However, it introduces potential operational overhead, reliance on cloud infrastructure, and limited flexibility for hybrid or on-premise scenarios. These trade-offs inform ViMindXAI’s indexing architecture, enabling adaptation to varying workload demands, privacy constraints, and deployment environments.

4.2.3. Data and Application Layer (DaAL)

The DaAL serves as the intelligent control center of the VidMindXAI architecture, responsible for real-time inference, event summarization, anomaly detection, and FL. Positioned between the infrastructure and user interface layers, it transforms indexed multimodal data into actionable insights through AI-driven orchestration. This design facilitates secure, explainable, and scalable analytics deployment across critical sectors, including healthcare, public safety, and smart city infrastructure. The hardware foundation of the DaAL includes GPU-accelerated cloud instances such as NVIDIA A100, V100, or T4, and on-premises inference servers equipped with Tensor cores or AI inference accelerators such as Intel Habana Gaudi, AMD Instinct. Containerized orchestration is achieved via Kubernetes and KubeEdge, while distributed execution is supported through GPU clusters using Horovod or Ray. NVMe-based local storage and high-speed networking, such as RDMA, and 10/40 GbE ensure a low-latency model serving and parallel computation across modalities. The DaAL consists of two core modules: Real-Time AI Video Analytics and AI-Driven Data Services (DaaS).

(1) 

Real-Time AI Video Analytics

This module performs real-time analytics using advanced deep learning models. It includes three functional groups:

• Video Analytics Core: Executes real-time tasks such as object detection, tracking, and activity recognition using models like YOLOv11, DeepSORT, and EventNet.
• Video Understanding and XAI: Generates contextual summaries and explainable insights through captioning and attention visualization using models such as VideoBERT, BLIP, and GradCAM.
• FL and Privacy: Coordinates distributed model training and aggregation from edge devices, while preserving privacy using frameworks like FedAvg, TFF, and NVIDIA FLARE. To address non-IID data and client drift challenges in federated learning, an asynchronous FedAvg scheme enhanced with FedProx regularization is adopted as the coordination strategy. Model updates are managed through TFF, while edge orchestration is supported via KubeEdge. Privacy guarantees are reinforced using secure aggregation protocols and optional differential privacy, depending on the application context. While FedAvg and FedProx remain effective for scalable and moderately heterogeneous environments, we also evaluate more recent algorithms: MOON [153], which applies contrastive learning to enhance client representation consistency; SCAFFOLD [154], which uses control variates to mitigate client drift; and FedNova [155], which normalizes local updates to address training time imbalance. These methods are currently under consideration for integration into ViMindXAI’s FL stack, particularly in deployments with high skew or dynamic data heterogeneity.

(2) 

AI-Driven Data Services (DaaS)

This module delivers secure, intelligent, and regulation-compliant access to video big data, enabling scalable analytics and semantic interaction across the ViMindXAI platform. It includes three core components:

• Semantic Video Interaction Engine (SVIE): This module supports natural language-based interaction, semantic summarization, and retrieval using foundation models such as CLIP, Whisper, GPT, BLIP, and VideoBERT. SVIE transforms video and audio streams into deep semantic representations, enabling users to query video segments through conversational language or multimodal prompts. By integrating with the semantic index, SVIE enables contextual exploration, smart captioning, and voice-assisted scene navigation, operating as the AI-logic bridge between data and user queries.
• Semantic Index and Retrieval Manager: Responsible for embedding extraction, indexing, and query resolution, this manager leverages multimodal encoders such as CLIP, and BLIP to generate dense vector embeddings from raw data streams. Indexing frameworks such as FAISS or Elasticsearch support approximate nearest neighbor search and multimodal retrieval, which are crucial for rapid access, recommendation, and cross-modal exploration. Through standard APIs (REST, GraphQL), the module enables scalable semantic video search.
• Security and Access Control: Sensitive data is safeguarded through robust security mechanisms, including OAuth2, JWT, and RBAC, which enable fine-grained authentication and access control. It adopts a Zero Trust Security (ZTS) model and optionally uses hardware-based protections like Intel SGX for secure execution. Comprehensive audit logging ensures transparency, while access policies safeguard against misuse and ensure compliance.
• Metadata and Compliance: This submodule oversees metadata tagging, contextual annotations (e.g., object type, location, identity), and regulatory adherence. It manages sensitive content through automated tagging and enforces data privacy standards such as GDPR, HIPAA, and CCPA. Tools like Apache Atlas and policy-based engines provide lineage tracking, compliance auditing, and enforcement of fine-grained access rules.

Collectively, these components form a robust application layer that converts infrastructure-level video data into secure, explainable, and semantically enriched services, bridging low-level data orchestration with real-time user interaction and AI reasoning.

The DaAL orchestrates end-to-end video analytics by managing real-time AI workflows for tasks such as inference, anomaly detection, video summarization, and federated model training. Serving as a bridge between the infrastructure layer and user-facing components, it transforms indexed video data into actionable insights through adaptive dashboards, semantic search, and immersive AR/VR interfaces. Through the integration of XAI and privacy-preserving learning techniques, the analytics processes are rendered scalable, interpretable, and secure, aligning with the needs of high-stakes domains.

4.2.4. User Experience Layer (UEL)

The UEL serves as the principal interface between end-users and the ViMindXAI platform, facilitating access to intelligent video analytics services. It delivers multimodal, intuitive, and role-sensitive interfaces that allow users to perceive, explore, and act upon intelligent video analytics. Although the core computational logic resides in lower architectural layers, the UEL bridges these capabilities with user-facing insights, enabling informed decisions across domains such as public safety, healthcare, and operations. It is designed with modularity and cognitive adaptability in mind. To maximize usability across diverse environments, the UEL supports deployment on web, tablet, and AR/VR interfaces, requiring only minimal local inference. This ensures responsive and accessible user interaction, even in bandwidth-constrained or mobility-oriented scenarios. The UEL is structured around two core components: User Intelligence Interfaces (UII) and End-User Groups.

(1) 

User Intelligence Interfaces (UII)

This module includes three intelligent components designed to support user interaction and visualization:

• Immersive Monitoring Interface: The Immersive Monitoring Interface provides real-time AR/VR-based visualizations for operational monitoring and training, utilizing platforms such as Unity, Meta Quest, and HoloLens. It supports environments that demand high spatial and contextual awareness.
• Adaptive AI Dashboard: Presents personalized, role-specific dashboards with real-time updates and explainability features such as GradCAM and SHAP. Integrates with standard BI tools like Tableau, Grafana, Kibana, and Power BI.

Conversational access is enabled through seamless integration with the SVIE in the DaAL, users are enabled to interact with analytics through natural language queries, contextual prompts, and voice-based commands, supporting intuitive human-AI interaction. This facilitates intuitive human-AI communication across roles and contexts.

(2) 

End-User Groups

Categorizes system users into Business, Individual, and Public Sector segments. Analytics are accessed via the UII, which dynamically adapts content presentation and functionalities according to user roles, while upholding strict privacy and security policies.

By interfacing with both the DaAL and the DIL, the UEL ensures that real-time analytics are presented in an explainable, personalized, and privacy-aware manner. This design supports forward-compatible deployment in environments requiring cognitive interaction and extended reality (XR), while ensuring transparency in AI behavior aligned with user needs.

To support real-time VBDA in cloud–edge environments, the ViMindXAI platform incorporates a carefully selected set of scalable, AI-enhanced technologies. These components are distributed across four architectural layers, Data Acquisition, Infrastructure, Application, and User Experience to ensure optimal performance, security, and adaptability.

Table 8. Core technologies used across ViMindXAI layers.

Components	Applicable Technologies
Edge Inference	YOLOv8-tiny, MobileNet, TensorRT, TFLite, OpenVINO, NVIDIA Jetson
FL Frameworks	TensorFlow Federated, NVIDIA FLARE, FedAvg, Secure Multi-Party Computation (SMPC)
Stream Processing	Apache Kafka, Apache Flink, Spark Streaming, Apache NiFi
Hybrid and AI-Optimized Storage	Hadoop HDFS, MinIO, AWS S3, Ceph, Redis
Structured and Semantic Storage	Delta Lake, PostgreSQL, MongoDB, HBase, GraphDB
Search and Indexing	FAISS, OpenAI CLIP, Pinecone, Whisper, Elasticsearch, VideoBERT
Privacy and Security	Homomorphic Encryption, Differential Privacy, Zero-Knowledge Proofs (ZKP), RBAC, Intel SGX
UI/UX and Visualization	Tableau, Kibana, Grafana, Power BI, Unity, Meta Quest, GPT, HoloLens, GradCAM, SHAP

provides an overview of the representative technologies, chosen based on their technical maturity, community adoption, open source accessibility, and applicability to large-scale video data processing. The resulting technology stack balances innovation with deployment practicality, aligning with the platform’s scalability and real-world integration requirements.

4.3. Real-World Application Scenarios of ViMindXAI

This section introduces three case studies that demonstrate the real-world feasibility and domain adaptability of the ViMindXAI platform in smart buildings, healthcare, and industrial environments. Each scenario reflects practical requirements for low-latency processing, privacy preservation, and regulatory compliance. All case studies follow the four-layer architecture of ViMindXAI: edge data acquisition, hybrid data infrastructure, distributed analytics and learning, and user-facing outputs with explainability and compliance support. These examples highlight how the platform can be adapted to diverse operational contexts while maintaining modularity, scalability, and trustworthiness.

4.3.1. Case Study 1: Smart Building Surveillance and Access Management

ViMindXAI addresses the increasing demand for intelligent and privacy-aware surveillance in smart building environments. As IoT-enabled camera networks continue to expand, the resulting surge in real-time video streams necessitates analytics systems that are both scalable and compliant with privacy regulations. ViMindXAI adopts a modular edge–cloud architecture designed to support distributed threat detection, anomaly analysis, and semantic video retrieval across heterogeneous sources.

At the DAL, video streams are collected from IP cameras and processed locally on edge devices such as NVIDIA Jetson. Lightweight models like YOLOv8-tiny are used at this layer for early-stage filtering and entity detection, helping reduce transmission load. Preprocessing tasks, including frame selection and compression, further optimize bandwidth usage. Robust ingestion pipelines built on Apache Kafka and NiFi ensure reliable, fault-tolerant data flow to downstream components.

In the DIL, hybrid storage is orchestrated using MinIO and Delta Lake, organizing video data across logical zones to manage lifecycle stages. Semantic indexing is powered by CLIP and Whisper embeddings, stored in FAISS to support fast, vector-based retrieval. Federated synchronization strategies enable local storage autonomy while contributing to a shared global index under privacy-preserving constraints.

The DaAL executes core analytics workflows such as anomaly detection and person re-identification using advanced inference models. YOLOv11 is deployed here for high-precision object detection tasks in centralized pipelines, while Transformer and GNN architectures are employed for complex spatiotemporal reasoning. These models are updated through FL strategies like FedAvg. Explainable AI (XAI) modules provide transparent and interpretable outputs to support operational decisions.

Finally, the UEL delivers interactive dashboards via Grafana, enabling semantic search queries (e.g., “loitering near exit C”), spatial heatmaps, and real-time alerting. These tools support monitoring, incident response, and forensic analysis while enhancing trust through visual explanations.

By combining semantic intelligence, scalable infrastructure, and privacy-centric coordination, ViMindXAI delivers a practical and regulation-compliant platform for smart surveillance. Its architecture incorporates the most adopted technologies in modern VBDA systems, reinforcing its applicability in real-world deployments.

4.3.2. Case Study 2: Healthcare Monitoring and Patient Safety

In healthcare environments such as hospitals and long-term care facilities, continuous and intelligent video monitoring is essential for ensuring patient safety. Critical incidents, including falls, prolonged inactivity, and unsupervised movement, require immediate detection and response. At the same time, analytics systems must uphold strict privacy standards and operate efficiently in resource-constrained settings. ViMindXAI addresses these requirements by delivering real-time, federated, and privacy-aware video analytics tailored specifically for clinical deployments.

At the DAL, video streams from patient rooms and corridors are processed on edge devices such as Jetson Xavier and Coral TPU. Lightweight activity recognition models, including LSTM and 3D CNNs, are deployed via ONNX for hardware-agnostic inference, enabling the detection of behavioral indicators such as falls and wandering. In parallel, audio streams are preprocessed using Whisper to identify signs of vocal distress. Data ingestion is handled through Apache Kafka and Apache NiFi, ensuring consistent, low-latency, and fault-tolerant delivery from multiple endpoints.

The DIL establishes a secure and federated data backbone based on Delta Lake and MinIO. This hybrid architecture supports both local buffering and long-term cloud-integrated indexing, enabling scalable and privacy-conscious video storage. Semantic indexing leverages multimodal embeddings generated by CLIP and Whisper, capturing both visual and auditory context. These embeddings are indexed using FAISS to facilitate rapid, approximate similarity-based retrieval. Privacy-aware synchronization protocols govern data access and movement in alignment with HIPAA and GDPR.

Within the DaAL, ViMindXAI executes real-time anomaly detection and temporal pattern modeling to identify clinically significant risks, including prolonged inactivity, irregular movement, or repeated expressions of distress. These models are continuously refined using FL strategies such as FedAvg, allowing for localized model updates across hospital zones without exposing raw video data. XAI modules enhance interpretability by visualizing the spatial and temporal features that contribute to each decision, supporting transparent clinical validation and oversight.

The UEL provides HIPAA-compliant dashboards powered by Grafana and integrated with ViMindXAI’s semantic search engine. Clinicians can receive real-time alerts, issue natural-language video queries such as “falls inward A after midnight,” and visualize mobility trends through interactive dashboards. Annotated video summaries and XAI overlays further support clinical review and facilitate timely, evidence-based decision-making.

ViMindXAI delivers a robust, semantically enriched, and privacy-conscious video analytics platform for smart healthcare monitoring. Its modular and AI-driven architecture supports real-time event detection, regulatory compliance, and transparent system behavior across all layers, making it a comprehensive solution for next-generation clinical safety systems.

4.3.3. Case Study 3: Industrial Safety and Compliance Monitoring

Ensuring occupational safety and regulatory compliance in industrial environments such as manufacturing plants and logistics hubs requires real-time detection of safety violations, including missing personal protective equipment (PPE), unauthorized access, and unsafe machine interactions. ViMindXAI offers a scalable, AI-powered video analytics platform that supports incident prevention, policy enforcement, and post-event auditing while maintaining privacy compliance.

At the DAL, video streams from fixed and wearable cameras are processed on-site using edge devices such as NVIDIA Jetson. PPE-aware object detection models, including YOLOv8-PPE, along with tracking algorithms like FairMOT, are deployed to detect violations such as helmet absence or unauthorized entry into restricted areas. Preprocessing modules running at the edge perform event tagging and filtering before transmitting selected data via Apache Kafka and NiFi pipelines.

The DIL employs a hybrid lakehouse architecture based on Delta Lake and MinIO to manage video streams, sensor metadata, and compliance logs. Semantic indexing is achieved by combining multimodal embeddings generated by CLIP and Whisper, enriched with contextual metadata such as zone identifiers, worker IDs, and violation types. These embeddings are indexed using FAISS for fast similarity-based search. Optionally, Neo4j is used to model spatiotemporal relationships between workers, machines, and incident locations. Federated synchronization ensures that data locality and privacy regulations are respected across distributed industrial sites.

Within the DaAL, advanced analytics tasks such as cross-shift pattern detection and violation prediction are performed using Transformer-based reasoning models and anomaly detectors. FL strategies, including FedAvg, allow for site-specific model personalization without exchanging raw data. Reinforcement learning techniques are used to dynamically optimize inspection resource allocation based on incident history and risk profiles. XAI modules enhance transparency by identifying the key features influencing each safety decision.

The UEL delivers interactive dashboards through tools such as Power BI, Grafana, or Kibana. These interfaces support semantic video search (e.g., “PPE violations near conveyor belt 3”), compliance heatmaps, and worker-specific safety profiles. Real-time alerts are integrated with enterprise notification systems to enable immediate response, while visual explanations generated by XAI modules assist safety officers in reviewing and validating flagged events.

Through its modular and privacy-aware architecture, ViMindXAI provides a robust and adaptable solution for industrial safety monitoring. By integrating real-time analytics, federated coordination, semantic indexing, and explainable auditing, the platform enhances operational resilience and ensures regulatory compliance in dynamic and high-risk industrial environments.

4.4. Deployment Considerations and Solutions

As VBDA systems move from prototype to production, real-world deployment introduces challenges related to latency, scalability, and data privacy. Ensuring robust performance across heterogeneous environments requires seamless coordination between AI models, storage infrastructures, and orchestration mechanisms. This section outlines the major deployment obstacles and the optimization strategies integrated into the ViMindXAI platform.

4.4.1. Practical Deployment Challenges

Deployment of scalable video analytics systems such as ViMindXAI in real-world environments faces multiple operational and architectural challenges. These span across edge–cloud orchestration, FL, infrastructure compatibility, and data platform migration.

One of the most critical challenges is edge–cloud orchestration, where system responsiveness depends on minimizing inference latency while balancing computational load between edge devices and cloud servers. Application domains like surveillance, healthcare, and intelligent transportation operate under highly dynamic conditions, varying network bandwidth, heterogeneous device capabilities, and fluctuating workloads [108,117]. This necessitates the use of adaptive scheduling, resilient failover strategies, and real-time resource optimization mechanisms to maintain performance under variable constraints.

A second major issue is synchronization in FL, while FL is a privacy-preserving paradigm that avoids centralized raw data sharing, it introduces significant communication overhead, especially in multi-node environments. Additionally, the presence of non-IID (non-independent and identically distributed) data across devices leads to biased local model updates and degraded convergence [107,113]. Effective deployment requires asynchronous aggregation, model pruning, and gradient compression techniques to reduce communication bottlenecks and ensure scalable training.

Another challenge lies in resource and bandwidth limitations at the edge. Most edge devices are not equipped to execute large-scale transformer-based models or multimodal inference in real-time. Moreover, streaming high-resolution video data imposes heavy pressure on upstream and inter-device bandwidth [7,112]. Addressing this requires careful model optimization, such as using quantized or distilled versions of core models, efficient codec settings, and adaptive frame skipping strategies.

Finally, the shift to modern data infrastructures introduces migration complexities toward Data Lakehouse architectures. Technologies like Delta Lake and Apache Iceberg require rethinking legacy ingestion pipelines, adopting new metadata strategies, and ensuring backward compatibility with existing BI and AI workflows. These changes, while beneficial in the long run, demand strong governance, incremental rollout, and organizational readiness for transformation.

Together, these deployment challenges highlight the importance of robust system design that accommodates dynamic environments, privacy constraints, and evolving data infrastructure, while ensuring the scalability and resilience needed for mission-critical video analytics systems.

4.4.2. Proposed Optimization Strategies

ViMindXAI addresses these challenges through a multi-tiered optimization strategy designed for real-world resilience:

• Model Compression for Edge Inference: Techniques such as pruning, quantization, and knowledge distillation are applied to compress models without compromising accuracy. This enables real-time inference on edge hardware like NVIDIA Jetson and Intel Movidius for tasks such as anomaly detection and object tracking [100,134].
• Tiered and Intelligent Storage Architecture: The platform combines edge-level buffering, AI-driven caching, and cloud-based archiving. Semantic indexing using models like CLIP, FAISS, and VideoBERT enables rapid, context-aware retrieval [30,144,145], while tiered orchestration minimizes redundancy and optimizes bandwidth usage.
• Metadata Optimization and Context-Aware Allocation: Rich metadata, including object categories, spatial context, and scene attributes, is used to prioritize content for storage and retrieval. This improves responsiveness and reduces data transfer volumes, especially in bandwidth-constrained scenarios [11,142].

Together, these strategies equip ViMindXAI with the capability to operate effectively in real-world deployments, delivering scalable, low-latency, and privacy-aware analytics. The system’s design supports modular extensibility, aligning with future advancements outlined in Section 5.

4.5. Comparative Analysis with Existing VBDA Platforms

To contextualize the architectural contributions of ViMindXAI, we conducted a comparative analysis against representative VBDA platforms: FALKON [118], SIAT [119], SurveilEdge [52], and VIDEX [133]. These systems span centralized, cloud-based, edge-centric, and hybrid deployments and are widely cited in the literature. Our comparison focuses on architectural depth, semantic capability, explainability, and integration of emerging AI paradigms such as FL, XAI, and multimodal indexing.

Table 9. Comparative Analysis of ViMindXAI and Existing VBDA Platforms.

Criteria/Layer	FALKON [115]	SIAT [116]	SurveilEdge [49]	VIDEX [130]	ViMindXAI (Proposed)
Architecture Type	Centralized	Cloud-Centric	Edge Computing	Hybrid Cloud–Edge	Hybrid Cloud–Edge (Cognitive)
Edge Preprocessing (DAL)	Basic edge input; limited model support	Standard ingestion (Kafka/NiFi)	YOLO-based detection + task allocation	Parallel object/anomaly threads	Edge AI (YOLOv8-tiny), denoising, ONNX models, FL at acquisition layer
Federated Learning and Privacy (DAL/DaAL)	Minimal FL, no privacy focus	FedAvg at cloud layer only	FL partially applied	Loosely coordinated FL logic	Full FL (FedAvg/Prox), PETs, Zero Trust sync
Lakehouse and Storage Orchestration (DIL)	HDFS-based batch storage	HDFS/HBase architecture	Local DB (SQLite) only	Unspecified backend storage	Delta Lake, Iceberg, Redis, hybrid S3/Ceph
Multimodal Semantic Indexing (DIL)	Basic temporal-spatial indexing	Partial Elastic-based indexing	Edge-level metadata tagging	Metadata-based object search	CLIP, VideoBERT, FAISS, GraphDB
Graph-Based Reasoning (DIL)	Not supported	Not supported	Not supported	Lightweight reasoning	Integrated graph-based modeling
Explainable AI (DaAL)	Not integrated	Some dashboard visualizations	Cloud-side reclassification logic	Parallel logic with minimal XAI	XAI tools (GradCAM, SHAP) throughout pipeline
Compliance and Security (DaAL)	Standard encryption only	Limited PETs in FL logic	Device-local privacy only	Basic authentication	GDPR/HIPAA compliance, Apache Atlas, Zero Trust
Immersive and Explainable UI (UEL)	Basic dashboard visualization	Grafana/Kibana UI	2D dashboard UI	MVVM GUI with static views	Immersive monitoring, adaptive dashboards, XAI overlays
Natural Language Analytics (UEL)	Not supported	Not supported	Not supported	Not supported	Natural language querying via GPT and semantic video embeddings

summarizes this evaluation across ten core criteria, categorized by the functional layers of ViMindXAI (DAL, DIL, DaAL, UEL) and mapped to architectural types introduced in Section 3.

In contrast to FALKON’s centralized batch-based architecture or SurveilEdge’s edge-focused object detection, ViMindXAI adopts a layered, hybrid cognitive framework. The DAL incorporates lightweight models (e.g., YOLOv8-tiny), preprocessing, and embedded FL at the edge, surpassing the ingestion-only approach in SIAT or the static logic in VIDEX. The DIL leverages an AI-optimized Lakehouse architecture, integrating orchestration tools (e.g., Delta Lake, Apache Iceberg), semantic indexing (e.g., CLIP, VideoBERT), and GraphDB-based reasoning, features absent in the compared platforms. In the DaAL, ViMindXAI integrates explainability (e.g., GradCAM, SHAP) with coordinated model management using TFF and NVIDIA FLARE, supporting privacy-preserving learning and accountability. The UEL enables immersive, adaptive dashboards and LLM-powered interfaces for semantic video interaction, extending far beyond the static dashboards found in prior systems.

Although Table 9 offers a qualitative comparison, the architectural design of ViMindXAI is grounded in empirical evidence from recent federated VBDA systems. Kingori et al. [156] demonstrated that a hybrid VAE–SURF model, deployed using federated learning and edge computing, can achieve 25 ms latency and 40 fps throughput with only 2.5 GFLOPs per frame, highlighting its effectiveness for real-time anomaly detection on edge devices. Similarly, Li [157] introduced Dashlet, a cross-layer video streaming system that predicts user swipe behavior to optimize prefetching, achieving sub-100 ms latency improvements in live deployments.

Drawing on these insights, ViMindXAI incorporates edge-side inference using CLIP, asynchronous federated learning via FedAvg and FedProx, and orchestration through KubeEdge and TFF. These design choices enable real-time semantic search, scalable explainability, and low-latency responsiveness while maintaining practical deployability across diverse infrastructures. Although full-scale benchmarking is planned as part of future work in Section 6, the current architecture reflects a well-founded integration of proven strategies tailored to the performance and privacy demands of modern VBDA.

While ViMindXAI is in early-stage prototyping, all integrated components are mature, open source, and production-ready. The platform is designed for heterogeneous deployment environments via an edge–fog–cloud topology: edge nodes perform inference on devices like Jetson or Coral TPU; fog nodes handle caching and policy enforcement; the cloud layer oversees training and coordination. It supports microservice and master–slaver orchestration patterns, with federated agents managing updates under privacy and bandwidth constraints. Network features like 5G/6G, SDN, and QoS are embedded in task scheduling and data routing, ensuring readiness for real-world, large-scale deployments.

5. Open Research Challenges and Future Directions

While ViMindXAI provides a promising architectural foundation and illustrates its applicability through conceptual case scenarios, its transition to real-world deployment still requires addressing several unresolved challenges. These include real-time scalability, privacy-preserving learning, explainability, and the integration of emerging technologies. Overcoming these limitations is essential to enable secure, efficient, and intelligent video analytics in high-demand and sensitive domains. These challenges not only shape the roadmap of ViMindXAI but also highlight persistent research gaps across the broader landscape of video analytics.

5.1. Scalability and Real-Time Processing

The exponential rise in video data from surveillance, healthcare, and smart city infrastructure has increased the demand for scalable, low-latency analytics. Cloud-centric architectures often suffer from latency and network bottlenecks [108,112], making them less suitable for real-time inference. While hybrid edge–cloud systems offer partial solutions, key challenges remain in adaptive inference scheduling, task offloading, and dynamic resource orchestration. Emerging strategies, such as reinforcement learning-based scheduling and predictive offloading, leverage context-aware optimization using network states and device profiles, but require further refinement to meet the stringent latency demands of live video analytics.

Edge device constraints further complicate deployment. Running deep models in resource-limited environments necessitates lightweight variants. Techniques such as quantization, pruning, and knowledge distillation [30] show promise, but need to be optimized for high-dimensional video streams without sacrificing performance. While next-generation infrastructure such as 6G may alleviate bandwidth limitations, coordinated co-design of AI models and network protocols will be essential. To this end, future versions of ViMindXAI will incorporate reinforcement learning-based schedulers and predictive offloading into the DAL and DaAL layers for context-aware resource management. ONNX-compatible lightweight models enhanced with quantization and distillation will support efficient edge inference under practical constraints.

5.2. Security, Privacy, and Ethical AI

With the expansion of VBDA into privacy-sensitive sectors such as healthcare and public safety, data protection and responsible AI usage have become critical. FL offers a promising approach by enabling local model training without sharing raw video data. However, FL remains vulnerable to threats like model poisoning, gradient leakage, and degraded performance under non-IID data distributions [154]. Addressing these risks requires privacy-preserving techniques such as asynchronous FL, HE, and ZKP. While effective, these methods often introduce high computational overhead. SMPC and DP offer additional safeguards but still face challenges in scaling, particularly in real-time or bandwidth-constrained deployments.

To strengthen privacy and trust, ViMindXAI embeds asynchronous FL with FedProx regularization in the DaAL layer to reduce client drift under non-IID conditions. Privacy is further reinforced via secure aggregation protocols and optional DP, depending on the sensitivity of the application. For highly sensitive domains like clinical video analysis, we plan to extend support for HE and SMPC. Beyond security, ViMindXAI addresses fairness by integrating inherently interpretable models and fairness auditing pipelines, enabling explainability-by-design in tasks such as identity recognition and behavioral analysis.

5.3. Integration of Emerging Technologies

Breakthroughs in blockchain, quantum computing, SSL, and generative AI are reshaping the evolving VBDA landscape. These technologies hold the potential to redefine system capabilities if integrated thoughtfully.

Blockchain for Secure Access Control and Metadata Logging: Blockchain can offer immutable logging, decentralized identity management, and transparent access auditing for video streams [144]. Smart contracts can enforce granular access policies, yet the latency and storage demands of public blockchains limit their use in real-time systems. Research on hybrid blockchain–cloud architectures could balance performance with data integrity and transparency.

Quantum Computing for Video Analytics: Quantum-enhanced algorithms hold promise for accelerating complex video processing tasks, including encryption, compression, and content-based search. However, practical limitations such as decoherence, error rates, and hardware immaturity still hinder adoption. Future directions should explore hybrid quantum-classical pipelines and quantum-friendly video representations.

Self-Supervised Learning and Vision Transformers: Annotation remains a major bottleneck in training video models. SSL methods such as masked video modeling, contrastive learning, and temporal prediction can learn meaningful representations from unlabeled data [7]. Coupled with ViTs, these approaches support scalable and label-efficient learning pipelines, with strong performance across retrieval, summarization, and activity recognition.

Generative AI for Video Understanding: Generative models like GANs and diffusion architectures enable applications such as scene completion, synthetic data generation, and anomaly simulation [144]. While these tools enhance model robustness, they raise concerns around deepfake misuse, high training costs, and instability. Future research should prioritize training efficiency, transparent evaluation benchmarks, and the development of safeguards for ethical deployment.

To support a pragmatic transition from conceptual design to real-world deployment, we propose a phased research roadmap. In the short term, our development efforts will focus on investigating real-time scalability and privacy-preserving learning. Specifically, we intend to incorporate reinforcement learning-based scheduling, predictive task offloading, and self-supervised learning techniques to enable label-efficient model training under constrained cloud–edge environments. As an initial step toward implementation, we plan to prototype ViMindXAI through a smart building surveillance and access management scenario, where multiple edge cameras monitor common areas to detect anomalous events such as unauthorized access or falls (as presented in Case Study 1, Section 4.3). The system will preserve user privacy through federated learning, supported by ONNX-compatible lightweight models and adaptive task schedulers. This experimental deployment will serve to evaluate latency, inference accuracy under non-IID data distributions, privacy guarantees, and system responsiveness in resource-constrained edge settings. In the medium term, we aim to enhance the platform’s trustworthiness and accountability by integrating blockchain-based access control mechanisms and fairness auditing pipelines. In the long term, we will explore the potential of emerging paradigms such as quantum computing and generative AI, contingent upon advances in infrastructure maturity and algorithmic stability.

6. Conclusions

This paper presented a comprehensive survey of the evolution of VBDA, highlighting the shift from centralized pipelines to hybrid cloud–edge platforms enhanced by deep learning, federated learning, and explainable AI. We identified key challenges, such as latency, scalability, data heterogeneity, and privacy, and introduced ViMindXAI, a modular platform designed to address these limitations through emerging AI and infrastructure techniques. The layered design of the platform aligns closely with the architectural gaps outlined in this survey, offering a coherent response to system-level and user-centric demands in modern video analytics. By mapping critical challenges to targeted solutions, ViMindXAI contributes a unified framework for building scalable, intelligent, and privacy-aware video analytics systems. These findings point to key directions for developing next-generation VBDA platforms that support real-time, data-driven decision-making in domains such as public safety, transportation, and healthcare. They also provide a solid foundation for future research on adaptive and efficient decision-support systems in video-intensive environments.

In the future, to advance ViMindXAI from conceptual design to real-world deployment, we outline a phased research roadmap. In the short term, we first focus on improving real-time scalability and privacy-preserving learning through reinforcement learning-based scheduling, predictive task offloading, and self-supervised training. As an initial step, ViMindXAI will be prototyped in a smart building surveillance scenario to evaluate latency, accuracy under non-IID data, privacy, and responsiveness. We then plan to prioritize lightweight model optimization for edge devices, secure federated learning with differential privacy, and blockchain-based metadata governance. In the long term, our efforts will explore quantum-inspired models and self-supervised techniques for semantic video retrieval. Empirical validation will be conducted in real-world domains such as healthcare monitoring and industrial safety, which demand low-latency processing, robust privacy protection, and high model transparency.