Search Results (84)

Most image forgery localization methods rely on supervised learning, requiring large labeled datasets for training. Recently, several unsupervised approaches based on the variational autoencoder (VAE) framework have been proposed for forged pixel detection. In these approaches, the latent space is built by a simple Gaussian distribution or a Gaussian Mixture Model. Despite their success, there are still some limitations: (1) A simple Gaussian distribution assumption in the latent space constrains performance due to the diverse distribution of forged images. (2) Gaussian Mixture Models (GMMs) introduce non-convex log-sum-exp functions in the Kullback–Leibler (KL) divergence term, leading to gradient instability and convergence issues during training. (3) Estimating GMM mixing coefficients typically involves either the expectation-maximization (EM) algorithm before VAE training or a multilayer perceptron (MLP), both of which increase computational complexity. To address these limitations, we propose the Deep ViT-VAE-GMM (DVVG) framework. First, we employ Jensen’s inequality to simplify the KL divergence computation, reducing gradient instability and improving training stability. Second, we introduce convolutional neural networks (CNNs) to adaptively estimate the mixing coefficients, enabling an end-to-end architecture while significantly lowering computational costs. Experimental results on benchmark datasets demonstrate that DVVG not only enhances VAE performance but also improves efficiency in modeling complex latent distributions. Our method effectively balances performance and computational feasibility, making it a practical solution for real-world image forgery localization. Full article

To address the challenges brought by the high penetration of renewable energy in power systems, such as multi-scale dynamic interactions, high feature dimensionality, and limited model generalization, this paper proposes a transient stability assessment (TSA) method that combines temporal feature selection with deep learning-based modeling. First, a two-stage feature selection strategy is designed using the inter-class Mahalanobis distance and Spearman rank correlation. This helps extract highly discriminative and low-redundancy features from wide-area measurement system (WAMS) time-series data. Then, a parallel LSTM-Transformer architecture is constructed to capture both short-term local fluctuations and long-term global dependencies. A variational inference mechanism based on a Gaussian mixture model (GMM) is introduced to enable dynamic representations fusion and uncertainty modeling. A composite loss function combining improved focal loss and Kullback–Leibler (KL) divergence regularization is designed to enhance model robustness and training stability under complex disturbances. The proposed method is validated on a modified IEEE 39-bus system. Results show that it outperforms existing models in accuracy, robustness, interpretability, and other aspects. This provides an effective solution for TSA in power systems with high renewable energy integration. Full article

The detection of weak moving targets in heterogeneous clutter backgrounds is a significant challenge in radar systems. In this paper, we propose a track-before-detect (TBD) method based on information geometry (IG) theory applied to range-azimuth measurements, which extends the IG detectors to multi-frame detection through inter-frame information integration. The approach capitalizes on the distinctive benefits of the information geometry detection framework in scenarios with strong clutter, while enhancing the integration of information across multiple frames within the TBD approach. Specifically, target and clutter trajectories in multi-frame range-azimuth measurements are modeled on the Hermitian positive definite (HPD) and power spectrum (PS) manifolds. A scoring function based on information geometry, which uses Kullback–Leibler (KL) divergence as a geometric metric, is then devised to assess these motion trajectories. Moreover, this study devises a solution framework employing dynamic programming (DP) with constraints on state transitions, culminating in an integrated merit function. This algorithm identifies target trajectories by maximizing the integrated merit function. Experimental validation using real-recorded sea clutter datasets showcases the effectiveness of the proposed algorithm, yielding a minimum 3 dB enhancement in signal-to-clutter ratio (SCR) compared to traditional approaches. Full article

Stable Q-value estimation is critical for effective policy learning in deep reinforcement learning (DRL), especially continuous control tasks. Traditional algorithms like Soft Actor-Critic (SAC) and Twin Delayed Deep Deterministic (TD3) policy gradients rely on Mean Squared Error (MSE) loss for Q-value approximation, which may cause instability due to misestimation and overestimation biases. Although distributional reinforcement learning (RL) algorithms like C51 have improved robustness in discrete action spaces, their application to continuous control remains computationally expensive owing to distribution projection needs. To address this, we propose a classification-based Q-value learning method that reformulates Q-value estimation as a classification problem rather than a regression task. Replacing MSE loss with cross-entropy (CE) and Kullback–Leibler (KL) divergence losses, the proposed method improves learning stability and mitigates overestimation errors. Our statistical analysis across 30 independent runs shows that the approach achieves an approximately 10% lower Q-value estimation error in the pendulum environment and a 40–60% reduced training time compared to SAC and Continuous Twin Delayed Distributed Deep Deterministic (CTD4) Policy Gradient. Experimental results on OpenAI Gym benchmark environments demonstrate that our approach, with up to 77% fewer parameters, outperforms the SAC and CTD4 policy gradients regarding training stability and convergence speed, while maintaining a competitive final policy performance. Full article

Hierarchical Federated Learning (HFL) for the Internet of Vehicles (IoV) leverages roadside units (RSU) to construct a low-latency, highly scalable multilayer cooperative training framework. However, with the rapid growth in the number of vehicle nodes, this framework faces two major challenges: (i) communication inefficiency under bandwidth-constrained conditions, where uplink congestion imposes significant burden on intra-framework communication; and (ii) interference from untrustworthy vehicle nodes, which disrupts model training and affects convergence. Therefore, in order to achieve secure aggregation while alleviating the communication bottleneck problem, we design a hierarchical three-layer federated learning framework with Gradient Quantization (GQ) and secure aggregation, called FedHSQA, which further integrates anomaly scoring to enhance robustness against untrustworthy vehicle nodes. Specifically, FedHSQA organizes IoV devices into three layers based on their respective roles: the cloud service layer, the RSU layer, and the vehicle node layer. During each non-initial communication round, the cloud server at the cloud layer computes anomaly scores for vehicle nodes using a Kullback–Leibler (KL) divergence-based multilayer perceptron (MLP) model. These anomaly scores are used to design a secure aggregation algorithm (ASA) that is robust to anomalous behavior. The anomaly scores and the aggregated global model are then transmitted to the RSU. To further reduce communication overhead and maintain model utility, FedHSQA introduces an adaptive GQ method based on the anomaly scores (ASQ). Unlike conventional vehicle node-side quantization, ASQ is performed at the RSU layer. It calculates the Jensen–Shannon (JS) distance between each vehicle node’s anomaly distribution and the target distribution, and adaptively adjusts the quantization level to minimize redundant gradient transmission. We validate the robustness of FedHSQA against anomalous nodes through extensive experiments on three real-world datasets. Compared to classical aggregation algorithms and GQ methods, FedHSQA reduced the average network traffic consumption by approximately 30 times while improving the average accuracy of the aggregation model by about 5.3%. Full article

This paper presents a novel unsupervised domain adaptation (UDA) framework that integrates information-theoretic principles to mitigate distributional discrepancies between source and target domains. The proposed method incorporates two key components: (1) relative entropy regularization, which leverages Kullback–Leibler (KL) divergence to align the predicted label distribution of the target domain with a reference distribution derived from the source domain, thereby reducing prediction uncertainty; and (2) measure propagation, a technique that transfers probability mass from the source domain to generate pseudo-measures—estimated probabilistic representations—for the unlabeled target domain. This dual mechanism enhances both global feature alignment and semantic consistency across domains. Extensive experiments on benchmark datasets (OfficeHome and DomainNet) demonstrate that the proposed approach consistently outperforms State-of-the-Art methods, particularly in scenarios with significant domain shifts. These results confirm the robustness, scalability, and theoretical grounding of our framework, offering a new perspective on the fusion of information theory and domain adaptation. Full article

The complex and variable operating conditions of sucker-rod pumping wells pose a significant challenge for the timely and accurate identification of oil well operating conditions. Effective deep learning based on measured multi-source data obtained from the sucker-rod pumping well production site offers a promising solution to the challenge. However, existing deep learning-based operating condition recognition methods are constrained by several factors: the limitations of traditional operating condition recognition methods based on single-source and multi-source data, the need for large amounts of labeled data for training, and the high robustness requirement for recognizing complex and variable data. Therefore, we propose a semi-supervised class-incremental sucker-rod pumping well operating condition recognition method based on measured multi-source data distillation. Firstly, we select measured ground dynamometer cards and measured electrical power cards as information sources, and construct the graph neural network teacher models for data sources, and dynamically fuse the prediction probability of each teacher model through the Squeeze-and-Excitation attention mechanism. Then, we introduce a multi-source data distillation loss. It uses Kullback-Leibler (KL) divergence to measure the difference between the output logic of the teacher and student models. This helps reduce the forgetting of old operating condition category knowledge during class-incremental learning. Finally, we employ a multi-source semi-supervised graph classification method based on enhanced label propagation, which improves the label propagation method through a logistic regression classifier. This method can deeply explore the potential relationship between labeled and unlabeled samples, so as to further enhance the classification performance. Extensive experimental results show that the proposed method achieves superior recognition performance and enhanced engineering practicality in real-world class-incremental oil extraction production scenarios with complex and variable operating conditions. Full article

Gamma-ray spectroscopy is essential in nuclear science, enabling the identification of radioactive materials through energy spectrum analysis. However, high count rates lead to pile-up effects, resulting in spectral distortions that hinder accurate isotope identification and activity estimation. This phenomenon highlights the need for automated and precise approaches to pile-up correction. We propose a novel deep learning (DL) framework plugging count rate information of pile-up signals with a 2D attention U-Net for energy spectrum recovery. The input to the model is an Energy–Duration matrix constructed from preprocessed pulse signals. Temporal and spatial features are jointly extracted, with count rate information embedded to enhance robustness under high count rate conditions. Training data were generated using an open-source simulator based on a public gamma spectrum database. The model’s performance was evaluated using Kullback–Leibler (KL) divergence, Mean Squared Error (MSE) Energy Resolution (ER), and Full Width at Half Maximum (FWHM). Results indicate that the proposed framework effectively predicts accurate spectra, minimizing errors even under severe pile-up effects. This work provides a robust framework for addressing pile-up effects in gamma-ray spectroscopy, presenting a practical solution for automated, high-accuracy spectrum estimation. The integration of temporal and spatial learning techniques offers promising prospects for advancing high-activity nuclear analysis applications. Full article

This paper addresses the challenge of assessing photovoltaic (PV) hosting capacity in distribution networks while accounting for the uncertainty of PV output, a critical step toward achieving sustainable energy transitions. Traditional optimization methods for dealing with uncertainty, including robust optimization (RO) and stochastic optimization (SO), often result in overly conservative or optimistic assessments, hindering the efficient integration of renewable energy. To overcome these limitations, this paper proposes a novel distributionally robust chance-constrained (DRCC) assessment method based on Kullback–Leibler (KL) divergence. First, the time-segment adaptive bandwidth kernel density estimation (KDE) combined with Copula theory is employed to model the conditional probability density of PV forecasting errors, capturing temporal and output-dependent correlations. The KL divergence is then used to construct a fuzzy set for PV output, quantifying its uncertainty within specified confidence levels. Finally, the assessment results are derived by integrating the fuzzy set into the optimization model. Case studies demonstrate its effectiveness of the method. Key findings indicate that higher confidence levels reduce PV hosting capacities due to broader uncertainty ranges, while increased historical sample sizes enhance the accuracy of distribution estimates, thereby increasing assessed capacities. By balancing conservatism and optimism, this method enables safer and more efficient PV integration, directly supporting sustainability goals such as reducing fossil fuel dependence and lowering carbon emissions. The findings provide actionable insights for grid operators to maximize renewable energy utilization while maintaining grid stability, advancing global efforts toward sustainable energy infrastructure. Full article

As a pivotal area of research in the field of computer vision, the technology for food identification has become indispensable across diverse domains including dietary nutrition monitoring, intelligent service provision in restaurants, and ensuring quality control within the food industry. However, recognizing food images falls within the domain of Fine-Grained Visual Classification (FGVC), which presents challenges such as inter-class similarity, intra-class variability, and the complexity of capturing intricate local features. Researchers have primarily focused on deep information in deep convolutional neural networks for fine-grained visual classification, often neglecting shallow and detailed information. Taking these factors into account, we propose a Multi-level Attention Feature Fusion Network (MAF-Net). Specifically, we use feature maps generated by the Convolutional Neural Networks (CNNs) backbone network at different stages as inputs. We apply a self-attention mechanism to identify local features on these feature maps and then stack them together. The feature vectors obtained through the attention mechanism are then integrated with the original input to enhance data augmentation. Simultaneously, to capture as many local features as possible, we encourage multi-scale features to concentrate on distinct local regions at each stage by maximizing the Kullback-Leibler Divergence (KL-divergence) between the different stages. Additionally, we present a novel approach called subclass center loss (SCloss) to implement label smoothing, minimize intra-class feature distribution differences, and enhance the model’s generalization capability. Experiments conducted on three food image datasets—CETH Food-101, Vireo Food-172, and UEC Food-100—demonstrated the superiority of the proposed model. The model achieved Top-1 accuracies of 90.22%, 89.86%, and 90.61% on CETH Food-101, Vireo Food-172, and UEC Food-100, respectively. Notably, our method not only outperformed other methods in terms of the Top-5 accuracy of Vireo Food-172 but also achieved the highest performance in the Top-1 accuracies of UEC Food-100. Full article

This study examines how ten different audio features, including MFCC, mel-spectrogram, chroma, and spectral contrast etc., influence speaker change detection (SCD) performance. The analysis is conducted using two unsupervised methods: Bayesian information criterion with Gaussian mixture model (BIC-GMM), a model-based approach, and Kullback-Leibler divergence with Gaussian Mixture Model (KL-GMM), a metric-based approach. Evaluation involved statistical analysis of feature changes in relation to speaker changes (vice versa), supported by comprehensive experimental validation. Experimental results show MFCC as the most effective feature, demonstrating consistently good performance across both methods. Features such as zero crossing rate, chroma, and spectral contrast also showed notable effectiveness within the BIC-GMM framework, while mel-spectrogram consistently ranked as the least influential feature in both approaches. Further analysis revealed that BIC-GMM exhibits greater stability in managing variations in feature performance, whereas KL-GMM is more sensitive to threshold optimization. Nevertheless, KL-GMM achieved competitive results when paired with specific features, such as MFCC and zero crossing rate. These findings offer valuable insights into the impact of feature selection on unsupervised SCD, providing guidance for the development of more robust and accurate algorithms for practical applications. Full article

Sentiment analysis is a vital component of natural language processing (NLP), enabling the classification of text into positive, negative, or neutral sentiments. It is widely used in customer feedback analysis and social media monitoring but faces a significant challenge: bias. Biases, often introduced through imbalanced training datasets, can distort model predictions and result in unfair outcomes. To address this, we propose a bias-aware sentiment analysis framework leveraging Bias-BERT (Bidirectional Encoder Representations from Transformers), a customized classifier designed to balance accuracy and fairness. Our approach begins with adapting the Jigsaw Unintended Bias in Toxicity Classification dataset by converting toxicity scores into sentiment labels, making it suitable for sentiment analysis. This process includes data preparation steps like cleaning, tokenization, and feature extraction, all aimed at reducing bias. At the heart of our method is a novel loss function incorporating a bias-aware term based on the Kullback–Leibler (KL) divergence. This term guides the model toward fair predictions by penalizing biased outputs while maintaining robust classification performance. Ethical considerations are integral to our framework, ensuring the responsible deployment of AI models. This methodology highlights a pathway to equitable sentiment analysis by actively mitigating dataset biases and promoting fairness in NLP applications. Full article

A method for evaluating Kullback–Leibler (KL) divergence and Squared Hellinger (SH) distance between empirical data and a model distribution is proposed. This method exclusively utilises the empirical Cumulative Distribution Function (CDF) of the data and the CDF of the model, avoiding data processing such as histogram binning. The proposed method converges almost surely, with the proof based on the use of exponentially distributed waiting times. An example demonstrates convergence of the KL divergence and SH distance to their true values when utilising the Generalised Pareto (GP) distribution as empirical data and the K distribution as the model. Another example illustrates the goodness of fit of these (GP and K-distribution) models to real sea clutter data from the widely used Intelligent PIxel processing X-band (IPIX) measurements. The proposed method can be applied to assess the goodness of fit of various models (not limited to GP or K distribution) to clutter measurement data such as those from the Adriatic Sea. Distinctive features of this small and immature sea, like the presence of over 1300 islands that affect local wind and wave patterns, are likely to result in an amplitude distribution of sea clutter returns that differs from predictions of models designed for oceans or open seas. However, to the author’s knowledge, no data on this specific topic are currently available in the open literature, and such measurements have yet to be conducted. Full article

Enabling a robot to learn skills from a human and adapt to different task scenarios will enable the use of robots in manufacturing to improve efficiency. Movement Primitives (MPs) are prominent tools for encoding skills. This paper investigates how to learn MPs from a small number of human demonstrations and adapt to different task constraints, including waypoints, joint limits, virtual walls, and obstacles. Probabilistic Movement Primitives (ProMPs) model movements with distributions, thus providing the robot with additional freedom for task execution. We provide the robot with three modes to move, with only one human demonstration required for each mode. We propose an improved via-point generalization method to generalize smooth trajectories with encoded ProMPs. In addition, we present an effective task-constrained optimization method that incorporates all task constraints analytically into a probabilistic framework. We separate ProMPs as Gaussians at each timestep and minimize Kullback–Leibler (KL) divergence, with a gradient ascent–descent algorithm performed to obtain optimized ProMPs. Given optimized ProMPs, we outline a unified robot movement adaptation method for extending from a single obstacle to multiple obstacles. We validated our approach with a 7-DOF Xarm robot using a series of movement adaptation experiments. Full article

The Kullback–Leibler divergence (KL divergence) is a statistical measure that quantifies the difference between two probability distributions. Specifically, it assesses the amount of information that is lost when one distribution is used to approximate another. This concept is crucial in various fields, including information theory, statistics, and machine learning, as it helps in understanding how well a model represents the underlying data. In a recent study by Nawa and Nadarajah, a comprehensive collection of exact expressions for the Kullback–Leibler divergence was derived for both multivariate and matrix-variate distributions. This work is significant as it expands on our existing knowledge of KL divergence by providing precise formulations for over sixty univariate distributions. The authors also ensured the accuracy of these expressions through numerical checks, which adds a layer of validation to their findings. The derived expressions incorporate various special functions, highlighting the mathematical complexity and richness of the topic. This research contributes to a deeper understanding of KL divergence and its applications in statistical analysis and modeling. Full article