Abstract
Existing State-of-the-Art (SOTA) methods for situational awareness typically rely on high-bandwidth transmission of raw data or computationally intensive models, which are often impractical for resource-constrained edge devices in unstable communication environments. To address these limitations, this paper introduces a comprehensive framework for Regional Risk Monitoring utilizing unmanned swarm systems. We propose an innovative knowledge distillation approach (SIKD) that leverages both soft label dark knowledge and inter-layer relationships, enabling compressed models to run in real time on edge nodes while maintaining high accuracy. Furthermore, recognition results are fused using Bayesian inference to dynamically update the regional risk level. Experimental results demonstrate the feasibility of the proposed framework. Quantitatively, the proposed SIKD algorithm reduces the model parameters by 52.34% and computational complexity to 44.21% of the original model, achieving a 3× inference speedup on edge CPUs. Furthermore, it outperforms state-of-the-art baseline methods (e.g., DKD and IRG) in terms of convergence speed and classification accuracy, ensuring robust real-time risk monitoring.
Keywords:
Bayesian inference; edge intelligence; knowledge distillation; real-time regional risk monitoring; resource constrained MSC:
90-10
1. Introduction
Due to their cost-effectiveness, flexibility, and capability to operate in hazardous environments, unmanned swarms composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) have been extensively employed in scenarios such as post-disaster search and rescue [1], wildlife conservation [2], and regional monitoring [3]. In these contexts, collaborative groups of heterogeneous unmanned platforms can provide comprehensive and continuous surveillance over large areas, significantly enhancing global situational awareness and search efficiency. However, conventional operational paradigms typically require either human operators to remotely control the unmanned platforms or the transmission of substantial volumes of raw sensor data back to a central server for processing [4]. Although standard image or video compression techniques (e.g., H.264, H.265) can reduce bandwidth consumption to some extent, they still fundamentally rely on the continuous transmission of visual data streams. Under extremely constrained communication conditions (e.g., bandwidths of a few Kbps), even highly compressed video streams suffer from severe packet loss and latency. This approach heavily depends on stable, high-bandwidth communication links. In remote regions or during emergent public incidents, these communication channels are often unavailable or unreliable due to network congestion and other factors, resulting in significant delays and potential system failures.
Enabling edge devices within a cluster to perform intelligent processing locally presents an appealing solution, as it can significantly reduce communication demands and latency. However, this approach encounters two primary and interconnected challenges. First, high-performance target recognition models are typically computationally intensive and thus incapable of real-time execution on resource-constrained edge devices [5]. While model lightweighting techniques exist, achieving a compression that balances accuracy, speed, and robustness for diverse real-world edge scenarios remains an open issue. Second, and more critically, effective swarm coordination under stringent communication constraints requires novel paradigms. Recent work emphasizes that in bandwidth-limited or stealth-conscious operations, agents must rely on sparse, intermittent data exchanges, which leads to inherent uncertainty about the environment and other agents’ states. This necessitates not only local intelligence but also an intelligent mechanism to fuse these uncertain, asynchronous local inferences into a coherent and dynamic global picture, a capability that existing systems often lack [6].
To address the primary challenge of device intelligence, knowledge distillation (KD) [7] has emerged as a promising model compression technique that bridges this gap by enabling compact student models to learn from more powerful teacher models. By transferring knowledge through softened labels or intermediate feature representations, KD allows the student models to achieve performance levels close to those of the teacher models while requiring significantly fewer computational resources. Although recent advancements in edge-based object detection [8] significantly reduce latency, optimizing these models via standard distillation remains challenging. Existing studies such as [8,9,10] contribute to the development of efficient models; many focus solely on output-level imitation or lack a comprehensive approach to transferring the structural knowledge of the teacher, which may result in suboptimal performance or overfitting.
Simultaneously, while swarm coordination under communication limits has attracted attention [3,4], generating a unified situational picture remains difficult. Most existing dynamic spatial risk models [11] either assume continuous high-bandwidth communication or rely on static updates. Critically, they fail to explicitly model the ’temporal decay’ of information confidence—the fact that a grid’s risk status becomes increasingly uncertain as time passes since the last observation. This gap necessitates a Bayesian update mechanism capable of handling sparse detections and quantifying temporal uncertainty. Bayesian inference offers a principled methodology for updating risk estimates over time and incorporating new evidence as it becomes available. Bayesian methods are widely recognized for their capability to fuse historical and novel evidence under conditions of uncertainty. Prior studies [11,12,13] have applied Bayesian networks to threat assessment and situational awareness. However, these approaches typically focus on evaluating single-point threats or do not explicitly model the temporal degradation of information validity across spatial grids, a factor critical to sustained monitoring tasks. To bridge these gaps, this paper introduces a comprehensive framework for Real-time Regional Risk Monitoring with Autonomous UAV/UGV Swarms in Weak Communication Environments. The core contributions are fourfold:
- We propose a comprehensive real-time regional risk monitoring framework for unmanned swarm systems operating in weak communication environments. Central to this framework is a novel Knowledge Distillation algorithm based on Soft Labels and Inter-module Relationships (SIKD), which effectively compresses large-scale teacher models into lightweight student models, enabling high-accuracy, real-time object recognition on resource-constrained edge devices.
- We develop a dynamic risk update mechanism based on Bayesian inference to fuse the decentralized, asynchronous recognition results from edge nodes. This mechanism introduces a time-decay factor to quantify the confidence degradation of historical information, ensuring the construction of a globally consistent and real-time situational risk map.
- We conduct extensive simulation experiments, demonstrating its feasibility and superior performance compared to existing baseline methods.
The remainder of this paper is organized as follows. Section 2 reviews related work in knowledge distillation and Bayesian risk assessment. Section 3 details the proposed SIKD method and the Bayesian update mechanism. Section 4 presents experimental results and analysis. Section 5 discusses the strengths and limitations. Finally, Section 6 concludes the paper and discusses future directions.
2. Related Work
This section provides a comprehensive review of methodologies related to model lightweighting and dynamic risk prediction and assessment.
2.1. Model Lightweighting for Edge Deployment
Deploying deep learning models on resource-constrained edge devices is a persistent challenge. Research efforts have primarily evolved along two paths: designing inherently efficient architectures and compressing large models.
In the context of efficient architecture design, MobileNet [14] decomposes standard convolutions into a combination of depthwise and pointwise convolutions, thereby reducing the number of parameters and computational cost. Similarly, SqueezeNet [15] employs pointwise convolutions to decrease the dimensionality of the input and output channels of convolutional kernels. However, manually designing network architectures that simultaneously achieve lightweight characteristics and high performance remains a significant challenge. To address this, researchers propose the use of Neural Architecture Search (NAS) to automatically generate efficient and compact model structures [16]. Although these approaches effectively reduce computational overhead, they often fail to enhance the functional capacity of the resulting compact models, frequently leading to a notable performance gap when compared to larger models.
Deep model compression algorithms can be categorized into model pruning, parameter quantization, parameter decomposition, the design of specialized network architectures, and knowledge distillation. Han et al. [17] simplify networks by pruning connections with weights below a certain threshold. Building on this, Srinivas et al. [18] directly prune redundant neurons instead of sequentially pruning individual weights. Parameter quantization reduces the computational load of neural networks by decreasing the bit-width of weights [19]. Notably, the advent of batch normalization techniques enhances the independence between layers in deep networks, thereby further advancing parameter quantization methods [20]. Parameter decomposition techniques, such as singular value decomposition (SVD), decompose weight matrices and retain only the principal components with the highest singular values, effectively reducing the number of model parameters and computational complexity. Subsequently, orthogonal-triangular decomposition and CUR matrix decomposition are also applied to parameter decomposition in neural networks [21]. However, it is important to note that for the aforementioned compression methods based on pruning, quantization, and decomposition, higher compression ratios generally lead to a degradation in model performance [22].
Since the introduction of model compression by Hinton and colleagues, KD has emerged as a prominent technique. Traditional KD methods primarily rely on aligning the softened output logits of the teacher model, thereby transferring the “dark knowledge” pertaining to inter-class relationships. However, for complex tasks such as object detection and tracking, which involve both classification and regression components, logit-based distillation often proves insufficient. This limitation has spurred the development of feature-based distillation approaches, wherein the student model learns to emulate the intermediate feature representations of the teacher. For instance, Huang et al. [23] propose a progressive distillation strategy aimed at capturing individual units within the teacher model. Similarly, Wang et al. [24] introduce an attention masking mechanism to guide the student’s focus toward critical spatial regions. Despite these advancements, certain gaps remain.
2.2. Dynamic Risk Prediction and Assessment
Dynamic risk assessment aims to fuse heterogeneous, asynchronous information to maintain an accurate and timely estimate of a system’s state. Bayesian inference provides a principled probabilistic framework for this task, allowing for the coherent integration of prior knowledge with new evidence.
In the context of situational awareness, Bayesian networks have been widely adopted. Di et al. [25] propose a method using Bayesian networks to correlate historical information with newly acquired data for situational updates. Similarly, Kavallieratos et al. [26] design a hierarchical propagation mechanism in cyberspace to assess risk, translating the impact of individual events to a regional level. These works establish the viability of Bayesian reasoning for dynamic assessment.
A key challenge in dynamic settings is the temporal validity of information. A risk assessment that is accurate at one moment may become obsolete as the situation evolves. While some studies, like Ye et al. [27], apply Dynamic Bayesian Networks (DBNs) for threat evaluation, they often focus on single targets and lack a mechanism for area-wide, continuous updates that account for information decay over time. The work of Fu et al. [28] also employs a hierarchical Bayesian model, but did not explicitly model how risk confidence deteriorates with time since the last observation.
In summary, while existing work provides strong foundations in both knowledge distillation and Bayesian risk assessment, significant gaps remain when applying them to unmanned swarms in weak communication environments. Most current distillation methods focus primarily on output probabilities, neglecting the structural constraint of intermediate features required for robust edge inference. Furthermore, standard risk assessment models often lack a mechanism to dynamically quantify the temporal decay of information confidence, which is critical when continuous monitoring of every grid is impossible. This paper addresses these specific limitations by integrating a novel Structural and Inter-module Knowledge Distillation (SIKD) algorithm with a time-decay-aware Bayesian fusion framework. This alignment ensures both the high efficiency of individual edge nodes and the spatiotemporal consistency of the global risk map.
3. Methodology
The proposed framework for real-time regional risk monitoring is designed to operate under two core constraints: limited computational capacity at the edge and unreliable communication links to a central unit. To this end, our methodology is bifurcated into, first, enabling accurate on-device inference via a novel knowledge distillation paradigm, and second, fusing the decentralized inference results into a coherent and temporally evolving risk map using a Bayesian updating mechanism. The overall architecture of our approach is depicted in Figure 1.
Figure 1.
The architecture of the real-time regional risk monitoring approach. The color represents the risk value, the higher the green, the higher the risk value, the red star position is the very high risk value, and the red dashed circle indicates the area that needs to be investigated.
3.1. Framework Overview
As illustrated in Figure 1, our system comprises a fleet of edge devices (e.g., UAVs and UGVs) deployed within a region of interest. Each device is equipped with a compact student model, derived from a powerful teacher model via our proposed knowledge distillation algorithm. These student models perform real-time object recognition on the visual data captured on-device, eliminating the need to transmit bandwidth-intensive raw video streams. Only the lightweight recognition results (e.g., target class and location) are transmitted to the central base station whenever a communication link is available. The base station aggregates these results and employs a Bayesian risk update module to maintain a dynamic, grid-based risk map of the entire region. This map provides a real-time situational overview, enabling informed decision-making despite the weak communication environment.
3.2. Knowledge Distillation Based on Soft Label and Inter-Module Relationship
To enable the real-time deployment of trained deep learning models on edge devices, we propose a knowledge distillation method based on soft labels and inter-module relationships, termed SIKD. In this approach, a pre-trained large model serves as the teacher, while the model intended for deployment on edge devices functions as the student. Unlike the standard knowledge distillation method [7] which primarily focuses on minimizing the divergence of output probabilities (soft labels) at the final layer, our SIKD approach introduces a dual-supervision mechanism. It simultaneously captures the implicit knowledge embedded in the labels (data perspective) and explicitly aligns the intermediate feature maps between the teacher and student networks (model perspective). By minimizing the Euclidean distance between corresponding modules, SIKD ensures that the student not only mimics the teacher’s final prediction but also replicates its internal feature extraction process, thereby minimizing discrepancies between corresponding modules of the teacher and student networks. This strategy significantly reduces the number of model parameters while maintaining performance, thereby empowering edge devices with enhanced intelligent capabilities.
The overall framework of the proposed algorithm is illustrated in Figure 2. The upper large-scale model represents the teacher network, while the smaller model below corresponds to the student network. Based on training the student network with the training data, two types of supervisory signals distilled from the teacher model are employed to transfer knowledge from the teacher network, thereby enabling the student model to achieve performance comparable to that of the teacher model.
Figure 2.
The architecture of knowledge distillation based on soft label and inter-module relationship (SIKD).
Let represent the discrepancy between the respective modules of the teacher network and the student network. Assuming the modules of the teacher model are and the modules of the student model are . Specifically, we align the intermediate feature maps extracted from the output of the final residual block of each stage (i.e., ) in both the ResNet-50 (Teacher) and ResNet-18 (Student). The Euclidean distance is employed as the alignment metric because it directly minimizes the pixel-wise intensity differences in feature activations, compelling the student network to preserve the spatial structural information essential for object recognition. If the input data is x, the discrepancy is calculated as:
For the output logits z of the final module of the network, which have not undergone softmax normalization, the output values corresponding to non-target classes are not suppressed. Therefore, the student model can additionally learn the probabilities assigned by the teacher model to various non-target classes, thus acquiring the dark knowledge contained within the labels. The cognitive discrepancy between the two networks can be calculated as follows:
where corresponds to the logit value for class j, and represent the perceived probabilities for class j in the teacher and student network logits, respectively, and is the temperature coefficient used to reduce the disparity between target and non-target classes. Consequently, the total loss for model distillation training is:
Here, is the direct loss produced by the training data passing through the student network. The hyperparameters and are determined via grid search on the validation set to balance the loss components.
Design Philosophy and Comparative Analysis
To facilitate a clearer understanding of SIKD’s positioning within the knowledge distillation landscape, we present a structured comparison with established methods in Table 1, including Attention Transfer (AT) [29], Decoupled Knowledge Distillation (DKD) [30], and Instance Relationship Graph (IRG) [31].
Table 1.
Comparison of Distillation Paradigms. SIKD uniquely combines Logit-level dark knowledge with Inter-module structural dependencies to suit edge detection tasks.
Motivation and Loss Rationale: The design of SIKD is continually motivated by the specific requirements of UAV edge perception: preserving spatial structure for localization while maintaining minimal distinctness for classification.
- Why Inter-module Relationship ()? Unlike DKD [30], which focuses exclusively on separating target and non-target logic classes, pure logit-based distillation loses the spatial structural formulation inherent in convolutional feature maps. For object detection and risk monitoring, spatial cues are as vital as semantic labels. While AT [29] transfers attention maps, it often overlooks the direct mapping of module-wise feature evolution. SIKD explicitly minimizes the Euclidean distance between corresponding modules (), ensuring the student mimics how the teacher extracts features, not just what it predicts.
- Why Soft Labels ()? While feature-based methods like AT maximize structural alignment, they often discard the “dark knowledge” (inter-class correlations) found in the output layer. SIKD retains this via , ensuring the student captures the teacher’s uncertainty and subtle class distinctness.
- Distinction from IRG: Although IRG [31] also considers structural relationships, it constructs complex instance relationship graphs based on samples within a batch. This introduces significant computational overhead during training and relies heavily on batch diversity. In contrast, SIKD operates on a “per-image, per-module” basis. It is computationally more efficient for edge-oriented training pipelines and focuses on the internal consistency of the network itself rather than batch-wise sample relationships.
In summary, SIKD is designed as a hybrid mechanism. It employs Equation (1) to lock in the spatial feature capabilities required for detection and Equation (4) to refine the semantic classification boundaries, offering a balanced trade-off superior to single-focus methods.
3.3. Risk Dynamic Update Mechanism Based on Bayesian Inference
To ensure clarity in the mathematical derivation and reproducibility of the proposed mechanism, we first summarize the key symbols and parameters used in the risk update process in Table 2.
Table 2.
Nomenclature for Bayesian Risk Update Mechanism.
Supported by the aforementioned SIKD method, edge devices can independently perform target recognition tasks. This section addresses how to dynamically assess and update the risk levels across different areas based on the information generated by edge devices. By designing a risk dynamic update mechanism based on Bayesian inference, the pre-reconnaissance regional risk information is treated as prior knowledge, and the reconnaissance results from edge devices are treated as posterior knowledge. This establishes a strategy for updating regional risks and incorporates a real-time update mechanism that considers the impact of time decay on risk confidence.
For a specific operational area Ω subjected to tactical reconnaissance and surveillance, the area is first divided into grids of length l and width w. The grid size defines the granularity of the area under consideration. The probability of a target being present in each grid is evaluated as the grid’s risk level, thereby forming the risk situation map of the mission area.
After division, the area forms grids . The risk values for each grid in the area are initialized under two scenarios: (1) If the mission area has never been reconnoitered, the risk value for each grid is set to 0.5; (2) If historical reconnaissance information exists for the area, the risk values from the historical results are used as the initial values, after undergoing risk confidence iteration relative to the current time offset.
Let denote the risk value of grid before reconnaissance, and denote the risk value of grid after reconnaissance. Let represent the success rate of edge reconnaissance device in detecting a target, and represent the probability that edge reconnaissance device successfully identifies a target given its presence. If a target is detected in grid , then is set to 1, and is set to . However, for cases where no target is detected, the probability of target presence must be inferred to calculate the grid risk value.
Let event denote the presence of a target in grid , and event denote that no target is identified. It follows that:
From this, we can derive:
Thus, the risk value for grid is updated as: , .
Area reconnaissance and surveillance are typically conducted by edge devices following designated routes, making real-time monitoring of every grid impossible. Therefore, the confidence in the risk value of a grid decays with the time elapsed since the last reconnaissance. This paper designs a time decay iteration mechanism for risk confidence, enabling real-time updates of the risk values for all grids.
The Bayesian update mechanism formulated above relies on several key simplifications to ensure computational tractability and real-time operation in the initial deployment. First, the detection success rate () and correct identification probability () for each edge device are treated as static, known parameters. This assumes that environmental conditions (e.g., lighting, weather, occlusion) and target characteristics remain relatively stable during a mission, or that the model’s performance is averaged over expected conditions. Second, the mechanism is class-agnostic; it does not differentiate risk based on target type. The probability encapsulates an aggregate performance across all target classes the model is trained to detect, implying uniform difficulty. Finally, device heterogeneity is simplified: while parameters (, ) can be set per device type (as in Table 1 and Table 2), they do not adapt in real-time based on individual device status or context. These assumptions allow for a lightweight, foundational framework. Their implications and pathways for relaxation are discussed in Section 5.
To mathematically model the temporal decay of information validity, we adopt a mechanism analogous to Newton’s Law of Cooling, driving the risk probability towards a state of maximum entropy (0.5). For grids where targets were previously identified, the risk value decreases more sharply the longer the time since the last reconnaissance. Conversely, for grids where no targets were identified, the risk value increases with time, and the rate of increase becomes more significant over time. In both cases, the risk value eventually stabilizes around 0.5. The iteration of the risk value in grid with respect to the time offset t since the last reconnaissance is defined as follows:
The piecewise function in Equation (8) ensures that regardless of the initial state, the system asymptotically converges to the equilibrium of uncertainty. Where is the sensitivity coefficient of the risk value to time offset, determined empirically based on the volatility of the battlefield environment (e.g., a higher for highly dynamic scenarios). The effects of the designed mechanism are illustrated in Figure 3, Figure 4 and Figure 5. These figures show the changes in risk value under different time offset sensitivities (), for initial risk values greater than 0.7, less than 0.3, and equal to 0.5, respectively.
Figure 3.
Change of risk value when the time offset sensitivity () is 0.6.
Figure 4.
Change of risk value when the time offset sensitivity () is 1.
Figure 5.
Change of risk value when the time offset sensitivity () is 3.
The justification for these trends lies in the temporal decay of information validity and the principle of maximum entropy. An initial risk value deviating from 0.5 implies high confidence in either the presence (>0.5) or absence (<0.5) of a threat. However, as the time elapsed increases without new observations, this certainty degrades. The value 0.5 represents the state of maximum uncertainty, where the presence or absence of a target is equally likely. Mathematically, the exponential decay term in Equation (8) ensures that as , the risk likelihood asymptotically converges to the equilibrium point of 0.5. The sensitivity coefficient dictates the rate of this uncertainty propagation; a higher implies a highly dynamic environment where historical data becomes obsolete more rapidly, causing the risk curve to revert to 0.5 faster. The algorithm flow is shown in Algorithm 1.
| Algorithm 1 Dynamic Regional Risk Monitoring Algorithm |
| Require: Edge devices set E, Mission Area grids G, Time sensitivity , Initial Prior Ensure: Real-time Risk Map M
|
4. Experimental Verification and Testing
To validate the feasibility of the proposed real-time regional risk monitoring method and evaluate the performance of the designed SIKD algorithm, this paper designs a regional reconnaissance and risk monitoring scenario. The algorithm’s functionality is verified by generating risk heat maps of the selected area, while the compression ratio, accuracy, and operational speed of the distilled model are tested to demonstrate the advancement and efficiency of the proposed algorithm.
4.1. Scenario Description and Experimental Configuration
This paper selects a partial map of the Changsha area (Coordinates: E– E, N– N) as the monitoring and surveillance region. The area is divided into grids with a latitude interval of 0.04 degrees and a longitude interval of 0.05 degrees. Three UAVs and three UGVs are deployed as edge devices performing patrol tasks, following random routes. Targets are placed at random coordinates. The parameters for the UAVs are set as shown in Table 3, and the parameters for the UGVs are set as shown in Table 4. The simulation parameters for the UAVs and UGVs are selected based on the performance specifications of typical commercial off-the-shelf platforms to ensure operational realism. All devices are simulated using virtual machines. The teacher model selected is ResNet-50, trained on the CIFAR-10 dataset using an NVIDIA GeForce RTX 3080Ti Laptop GPU. The student model is ResNet-18. Considering that edge devices need to run other algorithms (e.g., path planning), the model operates in single-threaded mode.
Table 3.
Parameters of UAV.
Table 4.
Parameters of UGV.
4.2. Functional Verification and Performance Analysis
Based on the scenario and experimental configuration described in the previous section, the proposed regional risk monitoring method is functionally verified to demonstrate its effectiveness. Metrics of the distilled model are tested, and the SIKD algorithm is compared with baseline algorithms. Comparisons are made from three perspectives: compression ratio, accuracy, and operational speed, proving the feasibility of the designed algorithm and its superiority over existing methods.
Figure 6 displays the risk value heat map of the monitored area at a specific moment after the initiation of the regional reconnaissance and surveillance mission. The map highlights relatively critical areas, which can be used for planning patrol routes or enhancing search efficiency, thereby confirming the feasibility of the method.
Figure 6.
The heatmap of the regional risk value. The color represents the risk value, the higher the green is, the higher the risk value is, and the red star position is the area with an extremely high risk value that needs to be investigated.
To quantitatively evaluate the accuracy of the dynamic risk assessment, we introduce the Root Mean Square Error (RMSE) as a convergence metric. The RMSE measures the deviation between the estimated risk map and the ground truth target distribution at time t. It is defined as:
where N is the total number of grids in the surveillance area, and is a binary value (1 if a target exists, 0 otherwise).
We tracked the RMSE variation throughout the simulation timeline.
- Initialization Phase (): Since the initial risk is set to 0.5 (maximum entropy), the initial RMSE is theoretically 0.5.
- Exploration Phase ( s): As the swarm conducts reconnaissance, valid updates reduce global uncertainty, causing the RMSE to drop significantly from 0.50 to 0.18.
- Stabilization Phase ( s): Despite the time-decay mechanism slightly increasing uncertainty in unvisited areas, continuous updates stabilize the RMSE around 0.15.
This convergence trend demonstrates that the proposed Bayesian update mechanism effectively correlates fragmentary detection results into an accurate global picture.
The designed SIKD algorithm is tested separately. The model size and operational speed of the distilled model and the original model deployed on edge devices are compared, and their computational complexities are calculated. The computational complexity is calculated as follows:
For a convolutional layer , its computational complexity is:
where is the computational complexity of convolutional layer , represents its number of input channels, represents its number of output channels, and represent the width and height of the convolutional kernel, and and represent the height and width of the output feature map, respectively.
For a fully connected layer , its computational complexity is:
where is the computational complexity of fully connected layer , I is the number of input neurons, and O is the number of output neurons. Therefore, the total computational complexity of the model can be derived as:
The results are shown in Table 5. It can be observed that through the SIKD algorithm designed in this paper, the distilled model reduces the number of parameters by 52.34%, achieves a model size compression rate of 43.71%, and its computational complexity is significantly reduced, being only 44.21% of the original model. Testing shows that its operational speed on the CPU is three times faster than before, meeting the requirements for real-time operation and further verifying the feasibility of the proposed method.
Table 5.
Partial comparison between the distillation model and the original model.
Having verified the feasibility of the SIKD algorithm, this paper further tests the model’s Top-1 accuracy, Top-3 accuracy, and Top-5 accuracy, and conducts a comparative analysis with the results of current state-of-the-art baseline algorithms DKD [30] and IRG [31].
Figure 7, Figure 8 and Figure 9 record the test accuracy of the aforementioned algorithms and the proposed algorithm across different training epochs. Figure 10 presents both the training and testing accuracy curves of the KD algorithms. It can be observed that the model converges rapidly within the first 20 epochs. Crucially, the gap between the training accuracy and testing accuracy remains narrow throughout the training process, and both curves stabilize simultaneously around 50 epochs. This strong correlation indicates that the model generalizes well to unseen data and effectively avoids overfitting, a common issue in distilled models that over-imitate teacher noise. Although the accuracy of the DKD algorithm is slightly better between epochs 20–40, this is because the DKD algorithm focuses solely on mimicking the soft label output of the teacher model while neglecting the inter-layer relationships within the model, making it prone to overfitting.
Figure 7.
Top-1 accuracy comparison between the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
Figure 8.
Top-3 accuracy comparison between the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
Figure 9.
Top-5 accuracy comparison between the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
Figure 10.
Comparison of top1 accuracy between training set and test set across the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
To address the limitations of the CIFAR-10 dataset and further validate the robustness of the proposed SIKD algorithm in dealing with the complexities of real-world UAV imagery (e.g., occlusion, illumination variation, and background clutter), we extended our evaluation to the task of Visual Object Tracking. Object tracking is a fundamental component of continuous regional monitoring. We applied SIKD to the SiamRPN++ tracking framework. The teacher network employs a ResNet-50 backbone, while the student network utilizes a lightweight MobileNetV2 backbone, representing a typical configuration for resource-constrained edge devices.
We utilized two large-scale, high-fidelity benchmarks: OTB100 (100 video sequences with various attributes) and LaSOT (1400 sequences with long-term tracking challenges). These datasets significantly exceed CIFAR-10 in realism and complexity. As shown in Figure 11, on the OTB100 benchmark, the proposed SIKD method achieves a Success score of 0.672 and a Precision score of 0.885. This represents a substantial improvement over the baseline Student model (MobileNetV2 without distillation), which scores only 0.657 in Success and 0.864 in Precision. Furthermore, SIKD consistently outperforms state-of-the-art distillation competitors, including DKD (0.667 Success/0.876 Precision) and IRG (0.665 Success/0.875 Precision), effectively bridging the performance gap towards the heavy Teacher model (ResNet-50).
Figure 11.
Comparison across OTB100 between the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
Similarly, on the more challenging LaSOT dataset (Figure 12), which features longer sequences and frequent disappearances, SIKD maintains its superiority. It achieves the highest Success score of 0.457 and Precision score of 0.453 among all student variants, surpassing the baseline (0.449 Success/0.449 Precision) and confirming that transferring spatial structural knowledge significantly enhances the generalization capability of lightweight backbones in complex scenarios.
Figure 12.
Comparison across LaSOT between the proposed SIKD model and state-of-the-art baseline models: DKD [30] and IRG [31].
To demonstrate capability in handling distinct surveillance challenges, Figure 13 visualizes the tracking results on the truck-16 sequence from LaSOT. As shown, the results visually validate this robustness. While all trackers perform well in the ‘Simple’ scenario, significant divergence occurs during ‘Intra-class interference’ and ‘Occlusion’. Standard lightweight trackers (Original, DKD, IRG) tend to drift to similar background vehicles or lose the target during partial occlusion. In contrast, the SIKD-optimized tracker maintains a tight bounding box closely aligned with the Teacher’s prediction, demonstrating superior resistance to visual distractors.
Figure 13.
Visualization of truck-16 sequence of LaSOT. The first row is the simple scene, the second row is the scene with intra-class interference, and the third row is the scene with intra-class interference and occlusion. The red bounding boxes are the targets’ groundtruth.
5. Discussion and Limitations
While our proposed framework demonstrates compelling performance in experiments, it is imperative to critically examine its broader implications and acknowledge its constraints to guide future research.
The success of our method hinges on the synergistic interplay between its two core components. The SIKD algorithm effectively bridges the accuracy gap between large teacher models and compact student models, making on-device intelligence feasible. This is not merely a matter of model compression; it is about preserving the teacher’s nuanced “understanding” of the data through a multi-faceted knowledge transfer. The Bayesian update mechanism, in turn, provides a principled mathematical framework to transform sporadic, localized inferences from the edge into a globally consistent and temporally evolving situational picture. The introduction of the time-offset iteration is a crucial element, as it prevents the risk assessment from becoming stale and reflects the increasing uncertainty associated with unscouted areas over time. This is a dynamic often overlooked in static fusion models.
However, several limitations warrant attention. First, we simulate weak communication by transmitting only light results. We assume packet delivery is guaranteed, which may not hold in severe jamming scenarios. Second, our current Bayesian update model operates under the assumption that the performance characteristics of the edge devices are known and static. In reality, the probability of detecting and correctly identifying a target (, ) can vary significantly depending on the target class (e.g., a vehicle vs. a person), size, or orientation. A more nuanced risk model could incorporate distinct prior distributions and likelihood functions for different target classes, updating class-specific risk layers. Furthermore, environmental dynamics such as changing illumination, fog, or clutter are not explicitly modeled. These factors effectively cause and to be dynamic variables rather than constants. A promising direction for future work is to integrate an online meta-cognition module that estimates confidence or uncertainty scores from the student model’s internal state (e.g., prediction entropy, feature variance). These scores could then be used to dynamically adjust the effective and in the Bayesian update, making the risk assessment more responsive to real-time perceptual quality. While our current simplified assumptions enable a stable and tractable proof-of-concept, evolving towards a self-aware, context-sensitive risk fusion mechanism is a critical next step.
6. Conclusions
In this paper, we have presented a comprehensive framework for real-time regional risk monitoring in weak communication environments. The core challenge of balancing computational efficiency with perceptual accuracy was addressed through a novel knowledge distillation algorithm, SIKD, which transfers knowledge via both soft labels and inter-module relationships. This enables the deployment of compact yet highly accurate recognition models on resource-constrained edge devices. The subsequent challenge of fusing decentralized, asynchronous inferences was solved by a dynamic Bayesian update mechanism. This mechanism not only integrates new evidence with prior knowledge in a probabilistically sound manner but also explicitly accounts for the temporal decay of information confidence, ensuring the risk assessment remains current and relevant.
Extensive experimental validation confirms that our approach is both feasible and effective. Specifically, the SIKD algorithm reduces the parameter count by 52.34% and floating-point operations (FLOPs) by over 55%, resulting in a 3× speedup in CPU operational speed compared to the original ResNet-50. Moreover, in terms of enhancement over existing techniques, SIKD demonstrates faster convergence and superior accuracy and stability compared to advanced distillation methods like DKD and IRG, successfully meeting real-time inference requirements. The integrated framework successfully generated dynamic risk heatmaps, demonstrating its practical utility for continuous situational awareness.
For future work, we plan to investigate adaptive confidence estimation for edge devices to enhance the robustness of the Bayesian fusion process. Furthermore, we aim to extend the distillation pipeline and risk modeling to accommodate multi-modal data inputs (e.g., thermal, depth) and multi-class risk scenarios, thereby broadening the applicability and precision of the system in increasingly complex and dynamic real-world environments.
Author Contributions
Conceptualization, T.C.; Methodology, T.C.; Data curation, Y.Z. and L.L.; Writing—original draft, T.C.; Writing—review & editing, T.C., Y.Z., L.L. and Y.P.; Visualization, Y.P. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Data Availability Statement
The original data presented in the study are openly available in huggingface at https://tensorflow.google.cn/datasets/catalog/cifar100?use_xbridge3=true&loader_name=forest&need_sec_link=1&sec_link_scene=im&theme=light (accessed on 3 January 2026).
Conflicts of Interest
The authors declare no conflict of interest.
References
- Lyu, M.; Zhao, Y.; Huang, C.; Huang, H. Unmanned aerial vehicles for search and rescue: A survey. Remote Sens. 2023, 15, 3266. [Google Scholar] [CrossRef] [Scilit]
- Aliane, N. Drones and AI-Driven Solutions for Wildlife Monitoring. Drones 2025, 9, 455. [Google Scholar] [CrossRef] [Scilit]
- Fan, J.; Lei, L.; Cai, S.; Shen, G.; Cao, P.; Zhang, L. Area Surveillance With Low Detection Probability Using UAV Swarms. IEEE Trans. Veh. Technol. 2024, 73, 1736–1752. [Google Scholar] [CrossRef] [Scilit]
- Du, Z.; Luo, C.; Min, G.; Wu, J.; Luo, C.; Pu, J.; Li, S. A Survey on Autonomous and Intelligent Swarms of Uncrewed Aerial Vehicles (UAVs). IEEE Trans. Intell. Transp. Syst. 2025, 26, 14477–14500. [Google Scholar] [CrossRef] [Scilit]
- Surantha, N.; Sutisna, N. Key Considerations for Real-Time Object Recognition on Edge Computing Devices. Appl. Sci. 2025, 15, 7533. [Google Scholar] [CrossRef] [Scilit]
- Kim, K.; Youn, J.; Yoon, S.; Kang, J.; Kim, K.; Shin, D. Study on Cyber Common Operational Picture Framework for Cyber Situational Awareness. Appl. Sci. 2023, 13, 2331. [Google Scholar] [CrossRef] [Scilit]
- Gao, P.; Qin, J.; Xiang, X.; Tan, Y. Knowledge Distillation from Relative Distribution. Expert Syst. Appl. 2025, 284, 127736. [Google Scholar] [CrossRef] [Scilit]
- Li, Z.; Xu, P.; Chang, X.; Yang, L.; Zhang, Y.; Yao, L. When Object Detection Meets Knowledge Distillation: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 10555–10579. [Google Scholar] [CrossRef] [Scilit]
- Yuan, M.; Lang, B.; Quan, F. Student-Friendly Knowledge Distillation. Knowl.-Based Syst. 2024, 296, 111915. [Google Scholar] [CrossRef] [Scilit]
- Zhang, W.; Guo, Y.; Wang, J.; Zhu, J.; Zeng, H. Collaborative knowledge distillation. IEEE Trans. Circuits Syst. Video Technol. 2024, 34, 7601–7613. [Google Scholar] [CrossRef] [Scilit]
- Kim, A.; Lee, D. Dynamic Bayesian network-based situational awareness and course of action decision-making support model. Expert Syst. Appl. 2024, 252, 124093. [Google Scholar] [CrossRef] [Scilit]
- Ying, X. Research on Network Security Situational Awareness and Risk Assessment Model Based on Bayesian Network. J. Cyber Secur. Mobil. 2025, 14, 155–179. [Google Scholar] [CrossRef] [Scilit]
- Hunte, J.L.; Neil, M.; Fenton, N.E. A hybrid Bayesian network for medical device risk assessment and management. Reliab. Eng. Syst. Saf. 2024, 241, 109630. [Google Scholar] [CrossRef] [Scilit]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv 2017, arXiv:1704.04861. [Google Scholar] [CrossRef] [Scilit]
- Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model sizes. arXiv 2016, arXiv:1602.07360. [Google Scholar]
- Ren, P.; Xiao, Y.; Chang, X.; Huang, P.Y.; Li, Z.; Chen, X.; Wang, X. A comprehensive survey of neural architecture search: Challenges and solutions. ACM Comput. Surv. (CSUR) 2021, 54, 1–34. [Google Scholar] [CrossRef] [Scilit]
- Han, S.; Mao, H.; Dally, W.J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv 2015, arXiv:1510.00149. [Google Scholar]
- Sapkota, K.R.; Roelofsen, S.; Rozantsev, A.; Lepetit, V.; Gillet, D.; Fua, P.; Martinoli, A. Vision-Based Unmanned Aerial Vehicle Detection and Tracking for Sense and Avoid Systems. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea, 9–14 October 2016. [Google Scholar] [CrossRef] [Scilit]
- Wei, L.; Ma, Z.; Yang, C.; Yao, Q. Advances in the neural network quantization: A comprehensive review. Appl. Sci. 2024, 14, 7445. [Google Scholar] [CrossRef] [Scilit]
- Pinto, V.; Severo, V.; Madeiro, F. Optimizing Multi-View CNN for CAD Mechanical Model Classification: An Evaluation of Pruning and Quantization Techniques. Electronics 2025, 14, 1013. [Google Scholar] [CrossRef] [Scilit]
- Cai, G.; Li, J.; Liu, X.; Chen, Z.; Zhang, H. Learning and Compressing: Low-Rank Matrix Factorization for Deep Neural Network Compression. Appl. Sci. 2023, 13, 2704. [Google Scholar] [CrossRef] [Scilit]
- Li, Z.; Li, H.; Meng, L. Model Compression for Deep Neural Networks: A Survey. Computers 2023, 12, 60. [Google Scholar] [CrossRef] [Scilit]
- Huang, H.; Liu, G.; Zhang, Y.; Xiong, R. Feature Distillation Siamese Networks for Object Tracking. Appl. Soft Comput. 2023, 132, 109912. [Google Scholar] [CrossRef] [Scilit]
- Sun, H.; Bai, Y.; Zhang, W. Distilling Siamese Trackers with Attention Mask. In Proceedings of the 2022 41st Chinese Control Conference (CCC), Hefei, China, 25–27 July 2022; pp. 6622–6627. [Google Scholar] [CrossRef] [Scilit]
- Di, R.; Gao, X.; Guo, Z.; Wan, K. A Threat Assessment Method for Unmanned Aerial Vehicle Based on Bayesian Networks under the Condition of Small Data Sets. Math. Probl. Eng. 2018, 2018, 8484358. [Google Scholar] [CrossRef] [Scilit]
- Kavallieratos, G.; Spathoulas, G.; Katsikas, S. Cyber Risk Propagation and Optimal Selection of Cybersecurity Controls for Complex Cyberphysical Systems. Sensors 2021, 21, 1691. [Google Scholar] [CrossRef] [Scilit]
- Ye, F.; Mao, Y.; Li, Y.; Liu, X. Target Threat Estimation Based on Discrete Dynamic Bayesian Networks with Small Samples. J. Syst. Eng. Electron. 2022, 33, 1135–1142. [Google Scholar] [CrossRef] [Scilit]
- Fu, C.; Sayed, T. Identification of Adequate Sample Size for Conflict-Based Crash Risk Evaluation: An Investigation Using Bayesian Hierarchical Extreme Value Theory Models. Anal. Methods Accid. Res. 2023, 39, 100281. [Google Scholar] [CrossRef] [Scilit]
- Zagoruyko, S.; Komodakis, N. Paying more attention to attention: Improving the performance of convolutional neural networks via Attention Transfer. In Proceedings of the International Conference on Learning Representations, Toulon, France, 24–26 April 2017. [Google Scholar] [CrossRef] [Scilit]
- Zhao, B.; Cui, Q.; Song, R.; Qiu, Y.; Liang, J. Decoupled knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–22 June 2022; pp. 11953–11962. [Google Scholar] [CrossRef] [Scilit]
- Liu, Y.; Cao, J.; Li, B.; Yuan, C.; Hu, W.; Li, Y.; Duan, Y. Knowledge distillation via instance relationship graph. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 7096–7104. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.












