Search Results (41)

Generative AI has been gaining attention across various creative domains. In particular, MusicGen stands out as a representative approach capable of generating music based on text or audio inputs. However, it has limitations in producing high-quality outputs for specific genres and fully reflecting user intentions. This paper proposes a prompt tuning technique that effectively adjusts the output quality of MusicGen without modifying its original parameters and optimizes its ability to generate music tailored to specific genres and styles. Experiments were conducted to compare the performance of the traditional MusicGen with the proposed method and evaluate the quality of generated music using the Contrastive Language-Audio Pretraining (CLAP) and Kullback–Leibler Divergence (KLD) scoring approaches. The results demonstrated that the proposed method significantly improved the output quality and musical coherence, particularly for specific genres and styles. Compared with the traditional model, the CLAP score was increased by 0.1270, and the KLD score was increased by 0.00403 on average. The effectiveness of prompt tuning in optimizing the performance of MusicGen validated the proposed method and highlighted its potential for advancing generative AI-based music generation tools. Full article

This study suggests a hybrid approach that combines improved feature selection and intelligent diagnosis to increase the operational safety and intelligent diagnosis capabilities of hydropower units. In order to handle the vibration data, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) is used initially. A novel comprehensive index is constructed by combining the Pearson correlation coefficient, mutual information (MI), and Kullback–Leibler divergence (KLD) to select intrinsic mode functions (IMFs). Next, feature extraction is performed on the selected IMFs using Refined Composite Multiscale Fluctuation Dispersion Entropy (RCMFDE). Then, time and frequency domain features are screened by calculating dispersion and combined with IMF features to build a hybrid feature vector. The vector is then fed into a CNN-GRU-Attention model for intelligent diagnosis. The improved slime mold algorithm (ISMA) is employed for the first time to optimize the hyperparameters of the CNN-GRU-Attention model. The experimental results show that the classification accuracy reaches 96.79% for raw signals and 93.33% for noisy signals, significantly outperforming traditional methods. This study incorporates entropy-based feature extraction, combines hyperparameter optimization with the classification model, and addresses the limitations of single feature selection methods for non-stationary and nonlinear signals. The proposed approach provides an excellent solution for intelligent optimized diagnosis of hydropower units. Full article

The absence of skin color information in skin cancer datasets poses a significant challenge for accurate diagnosis using artificial intelligence models, particularly for non-white populations. In this paper, based on the Monk Skin Tone (MST) scale, which is less biased than the Fitzpatrick scale, we propose MST-AI, a novel method for detecting skin color in images of large datasets, such as the International Skin Imaging Collaboration (ISIC) archive. The approach includes automatic frame, lesion removal, and lesion segmentation using convolutional neural networks, and modeling normal skin tones with a Variational Bayesian Gaussian Mixture Model (VB-GMM). The distribution of skin color predictions was compared with MST scale probability distribution functions (PDFs) using the Kullback-Leibler Divergence (KLD) metric. Validation against manual annotations and comparison with K-means clustering of image and skin mean RGBs demonstrated the superior performance of the MST-AI, with Kendall’s Tau, Spearman’s Rho, and Normalized Discounted Cumulative Gain (NDGC) of 0.68, 0.69, and 1.00, respectively. This research lays the groundwork for developing unbiased AI models for early skin cancer diagnosis by addressing skin color imbalances in large datasets. Full article

This paper addresses multisensor multitarget tracking where the sensor network can potentially be compromised by false data injection (FDI) attacks. The existence of the targets is not known and time-varying. A tracking algorithm is proposed that can detect the possible FDI attacks over the networks. First, a local estimate is generated from the measurements of each sensor based on the labeled multi-Bernoulli (LMB) filter. Then, a detection method for FDI attacks is derived based on the Kullback–Leibler divergence (KLD) between LMB random finite set (RFS) densities. The statistical characteristics of the KLD are analyzed when the measurements are secure or compromised by FDI attacks, from which the value of the threshold is selected. Finally, the global estimate is obtained by minimizing the weighted sum of the information gains from all secure local estimates to itself. A set of suitable weight parameters is selected for the information fusion of LMB densities. An efficient Gaussian implementation of the proposed algorithm is also presented for the linear Gaussian state evolution and measurement model. Experimental results illustrate that the proposed algorithm can provide reliable tracking performance against the FDI attacks. Full article

Since the 2008 global economic crisis, the detection of financial instabilities has garnered extensive research attention, particularly through the application of time-series analysis. In this study, a novel time-series analysis method, integrating the Kullback–Leibler Divergence (KLD) metric with a sliding window technique, is proposed to detect instabilities in time-series data, especially in financial markets. Global financial time series from 2004 to 2022 were analyzed. The raw time series were preprocessed into return rate series and transformed into complex networks using the directed horizontal visibility graph (DHVG) algorithm, effectively preserving temporal variabilities in network topologies. The KLD method was evaluated through both retrospective analysis and real-time monitoring. It successfully identified idiosyncratic incidents in the financial market, correlating them with specific economic events. Compared to traditional metrics (e.g., moments) and econometric methods, KLD demonstrated superior performance in capturing sequence information and detecting anomalies without requiring linear regression models. Although initially designed for financial data, the KLD method is versatile and can be applied to other types of time series as well. Full article

Chemical process control systems are complex, and modeling the controlled object is the first task in automatic control and optimal design. Most chemical process modeling experiments require test signals to be applied to the process, which may lead to production interruptions or cause safety accidents. Therefore, this paper proposes an improved transformer model based on a self-attention mechanism for modeling industrial processes. Then, an evaluation mechanism based on root mean square error (RMSE) and Kullback–Leibler divergence (KLD) metrics is designed to obtain more appropriate model parameters. The Variational Auto-Encoder (VAE) network is used to compute the associated KLD. Finally, a real nonlinear dynamic process in the petrochemical industry is modeled and evaluated using the proposed methodology to predict the time series data of the process. This study demonstrates the validity of the proposed transformer model and illustrates the versatility of using an integrated modeling, evaluation, and prediction scheme for nonlinear dynamic processes in process industries. The scheme is of great importance for the field of industrial soft measurements as well as for deep learning-based time series prediction. In addition, the issue of a suitable time domain for the prediction is discussed. Full article

In natural language processing (NLP) tasks, computing semantic textual similarity (STS) is crucial for capturing nuanced semantic differences in text. Traditional word vector methods, such as Word2Vec and GloVe, as well as deep learning models like BERT, face limitations in handling context dependency and polysemy and present challenges in computational resources and real-time processing. To address these issues, this paper introduces two novel methods. First, a sentence embedding generation method based on Kullback–Leibler Divergence (KLD) optimization is proposed, which enhances semantic differentiation between sentence vectors, thereby improving the accuracy of textual similarity computation. Second, this study proposes a framework incorporating RoBERTa knowledge distillation, which integrates the deep semantic insights of the RoBERTa model with prior methodologies to enhance sentence embeddings while preserving computational efficiency. Additionally, the study extends its contributions to sentiment analysis tasks by leveraging the enhanced embeddings for classification. The sentiment analysis experiments, conducted using a Stochastic Gradient Descent (SGD) classifier on the ACL IMDB dataset, demonstrate the effectiveness of the proposed methods, achieving high precision, recall, and F1 score metrics. To further augment model accuracy and efficacy, a feature selection approach is introduced, specifically through the Dynamic Principal Component Selection (DPCS) algorithm. The DPCS method autonomously identifies and prioritizes critical features, thus enriching the expressive capacity of sentence vectors and significantly advancing the accuracy of similarity computations. Experimental results demonstrate that our method outperforms existing methods in semantic similarity computation on the SemEval-2016 dataset. When evaluated using cosine similarity of average vectors, our model achieved a Pearson correlation coefficient (τ) of 0.470, a Spearman correlation coefficient (ρ) of 0.481, and a mean absolute error (MAE) of 2.100. Compared to traditional methods such as Word2Vec, GloVe, and FastText, our method significantly enhances similarity computation accuracy. Using TF-IDF-weighted cosine similarity evaluation, our model achieved a τ of 0.528, ρ of 0.518, and an MAE of 1.343. Additionally, in the cosine similarity assessment leveraging the Dynamic Principal Component Smoothing (DPCS) algorithm, our model achieved a τ of 0.530, ρ of 0.518, and an MAE of 1.320, further demonstrating the method’s effectiveness and precision in handling semantic similarity. These results indicate that our proposed method has high relevance and low error in semantic textual similarity tasks, thereby better capturing subtle semantic differences between texts. Full article

Synthetic aperture radar (SAR) automatic target recognition (ATR) technology has seen significant advancements. Despite these advancements, the majority of research still operates under the closed-set assumption, wherein all test samples belong to classes seen during the training phase. In real-world applications, however, it is common to encounter targets not previously seen during training, posing a significant challenge to the existing methods. Ideally, an ATR system should not only accurately identify known target classes but also effectively reject those belonging to unknown classes, giving rise to the concept of open set recognition (OSR). To address this challenge, we propose a novel approach that leverages the unique capabilities of the Capsule Network and the Kullback-Leibler divergence (KLD) to distinguish unknown classes. This method begins by deeply mining the features of SAR targets using the Capsule Network and enhancing the separability between different features through a specially designed loss function. Subsequently, the KLD of features between a testing sample and the center of each known class is calculated. If the testing sample exhibits a significantly larger KLD compared to all known classes, it is classified as an unknown target. The experimental results of the SAR-ACD dataset demonstrate that our method can maintain a correct identification rate of over 95% for known classes while effectively recognizing unknown classes. Compared to existing techniques, our method exhibits significant improvements. Full article

In any Bayesian computations, the first step is to derive the joint distribution of all the unknown variables given the observed data. Then, we have to do the computations. There are four general methods for performing computations: Joint MAP optimization; Posterior expectation computations that require integration methods; Sampling-based methods, such as MCMC, slice sampling, nested sampling, etc., for generating samples and numerically computing expectations; and finally, Variational Bayesian Approximation (VBA). In this last method, which is the focus of this paper, the objective is to search for an approximation for the joint posterior with a simpler one that allows for analytical computations. The main tool in VBA is to use the Kullback–Leibler Divergence (KLD) as a criterion to obtain that approximation. Even if, theoretically, this can be conducted formally, for practical reasons, we consider the case where the joint distribution is in the exponential family, and so is its approximation. In this case, the KLD becomes a function of the usual parameters or the natural parameters of the exponential family, where the problem becomes parametric optimization. Thus, we compare four optimization algorithms: general alternate functional optimization; parametric gradient-based with the normal and natural parameters; and the natural gradient algorithm. We then study their relative performances on three examples to demonstrate the implementation of each algorithm and their efficiency performance. Full article

Differential privacy, a cornerstone of privacy-preserving techniques, plays an indispensable role in ensuring the secure handling and sharing of sensitive data analysis across domains such as in census, healthcare, and social networks. Histograms, serving as a visually compelling tool for presenting analytical outcomes, are widely employed in these sectors. Currently, numerous algorithms for publishing histograms under differential privacy have been developed, striving to balance privacy protection with the provision of useful data. Nonetheless, the pivotal challenge concerning the effective enhancement of precision for small bins (those intervals that are narrowly defined or contain a relatively small number of data points) within histograms has yet to receive adequate attention and in-depth investigation from experts. In standard DP histogram publishing, adding noise without regard for bin size can result in small data bins being disproportionately influenced by noise, potentially severely impairing the overall accuracy of the histogram. In response to this challenge, this paper introduces the SReB_GCA sanitization algorithm designed to enhance the accuracy of small bins in DP histograms. The SReB_GCA approach involves sorting the bins from smallest to largest and applying a greedy grouping strategy, with a predefined lower bound on the mean relative error required for a bin to be included in a group. Our theoretical analysis reveals that sorting bins in ascending order prior to grouping effectively prioritizes the accuracy of smaller bins. SReB_GCA ensures strict $ϵ$-DP compliance and strikes a careful balance between reconstruction error and noise error, thereby not only initially improving the accuracy of small bins but also approximately optimizing the mean relative error of the entire histogram. To validate the efficiency of our proposed SReB_GCA method, we conducted extensive experiments using four diverse datasets, including two real-life datasets and two synthetic ones. The experimental results, quantified by the Kullback–Leibler Divergence (KLD), show that the SReB_GCA algorithm achieves substantial performance enhancement compared to the baseline method (DP_BASE) and several other established approaches for differential privacy histogram publication. Full article

Federated learning (FL) in mobile edge computing has emerged as a promising machine-learning paradigm in the Internet of Things, enabling distributed training without exposing private data. It allows multiple mobile devices (MDs) to collaboratively create a global model. FL not only addresses the issue of private data exposure but also alleviates the burden on a centralized server, which is common in conventional centralized learning. However, a critical issue in FL is the imposed computing for local training on multiple MDs, which often have limited computing capabilities. This limitation poses a challenge for MDs to actively contribute to the training process. To tackle this problem, this paper proposes an adaptive dataset management (ADM) scheme, aiming to reduce the burden of local training on MDs. Through an empirical study on the influence of dataset size on accuracy improvement over communication rounds, we confirm that the amount of dataset has a reduced impact on accuracy gain. Based on this finding, we introduce a discount factor that represents the reduced impact of the size of the dataset on the accuracy gain over communication rounds. To address the ADM problem, which involves determining how much the dataset should be reduced over classes while considering both the proposed discounting factor and Kullback–Leibler divergence (KLD), a theoretical framework is presented. The ADM problem is a non-convex optimization problem. To solve it, we propose a greedy-based heuristic algorithm that determines a suboptimal solution with low complexity. Simulation results demonstrate that our proposed scheme effectively alleviates the training burden on MDs while maintaining acceptable training accuracy. Full article

In the cooperative multi-sensor multi-vehicle (MSMV) localization domain, the data incest problem yields inconsistent data fusion results, thereby reducing the accuracy of vehicle localization. In order to address this problem, we propose the interval split covariance intersection filter (ISCIF). At first, the proposed ISCIF method is applied to the absolute positioning step. Then, we combine the interval constraint propagation (ICP) method and the proposed ISCIF method to realize relative positioning. Additionally, in order to enhance the robustness of the MSMV localization system, a Kullback–Leibler divergence (KLD)-based fault detection and exclusion (FDE) method is implemented in our system. Three simulations were carried out: Simulation scenarios 1 and 2 aimed to assess the accuracy of the proposed ISCIF with various capabilities of absolute vehicle positioning, while simulation scenario 3 was designed to evaluate the localization performance when faults were present. The simulation results of scenarios 1 and 2 demonstrated that our proposed vehicle localization method reduced the root mean square error (RMSE) by 8.9% and 15.5%, respectively, compared to the conventional split covariance intersection filter (SCIF) method. The simulation results of scenario 3 indicated that the implemented FDE method could effectively reduce the RMSE of vehicles (by about 55%) when faults were present in the system. Full article

Cranes are pivotal heavy equipment used in the construction of transmission line scenarios. Accurately identifying these cranes and monitoring their status is pressing. The rapid development of computer vision brings new ideas to solve these challenges. Since cranes have a high aspect ratio, conventional horizontal bounding boxes contain a large number of redundant objects, which deteriorates the accuracy of object detection. In this study, we use a rotating target detection paradigm to detect cranes. We propose the YOLOv8-Crane model, where YOLOv8 serves as a detection network for rotating targets, and we incorporate Transformers in the backbone to improve global context modeling. The Kullback–Leibler divergence (KLD) with excellent scale invariance is used as a loss function to measure the distance between predicted and true distribution. Finally, we validate the superiority of YOLOv8-Crane on 1405 real-scene data collected by ourselves. Our approach demonstrates a significant improvement in crane detection and offers a new solution for enhancing safety monitoring. Full article

In machine learning and data analysis, dimensionality reduction and high-dimensional data visualization can be accomplished by manifold learning using a t-Distributed Stochastic Neighbor Embedding (t-SNE) algorithm. We significantly improve this manifold learning scheme by introducing a preprocessing strategy for the t-SNE algorithm. In our preprocessing, we exploit Laplacian eigenmaps to reduce the high-dimensional data first, which can aggregate each data cluster and reduce the Kullback–Leibler divergence (KLD) remarkably. Moreover, the k-nearest-neighbor (KNN) algorithm is also involved in our preprocessing to enhance the visualization performance and reduce the computation and space complexity. We compare the performance of our strategy with that of the standard t-SNE on the MNIST dataset. The experiment results show that our strategy exhibits a stronger ability to separate different clusters as well as keep data of the same kind much closer to each other. Moreover, the KLD can be reduced by about $30\%$ at the cost of increasing the complexity in terms of runtime by only 1–2%. Full article

Machining is a crucial constituent of the manufacturing industry, which has begun to transition from precision machinery to smart machinery. Particularly, the introduction of artificial intelligence into computer numerically controlled (CNC) machine tools will enable machine tools to self-diagnose during operation, improving the quality of finished products. In this study, feature engineering and principal component analysis were combined with the online and real-time Gaussian mixture model (GMM) based on the Kullback–Leibler divergence’s measure to achieve the real-time monitoring of changes in manufacturing parameters. Based on the attached accelerometer device’s vibration signals and current sensing of the spindle, the developed GMM unsupervised learning was successfully used to diagnose the spindle speed changes of a CNC machine tool during milling. The F1-scores with improved experimental results for X, Y, and Z axes were 0.95, 0.88, and 0.93, respectively. The established FE-PCA-GMM/KLD method can be applied to issue warnings when it predicts a change in the manufacturing process parameter. A smart sensing device for diagnosing the machining status can be fabricated for implementation. The effectiveness of the developed method for determining the manufacturing parameter changes was successfully verified by experiments. Full article