Search Results (81)

Anomaly detection methods for industrial control networks using multivariate time series usually adopt deep learning-based prediction models. However, most of the existing anomaly detection research only focuses on evaluating detection performance and rarely explains why data is marked as abnormal and which physical components have been attacked. Yet, in many scenarios, it is necessary to explain the decision-making process of detection. To address this concern, we propose an interpretable method for an anomaly detection model based on gradient optimization, which can perform batch interpretation of data without affecting model performance. Our method transforms the interpretation of anomalous features into solving an optimization problem in a normal “reference” state. In the selection of important features, we adopt the method of multiplying the absolute gradient by the input to measure the independent effects of different dimensions of data. At the same time, we use KSG mutual information estimation and multivariate cross-correlation to evaluate the relationship and mutual influence between different dimensional data within the same sliding window. By accumulating gradient changes, the interpreter can identify the attacked features. Comparative experiments were conducted on the SWAT and WADI datasets, demonstrating that our method can effectively identify the physical components that have experienced anomalies and their changing trends. Full article

The existing body of scholarly work on the adoption of Climate-Smart Agriculture (CSA) in Africa and Nigeria has predominantly concentrated on the experiences and practices of smallholder farmers. While these studies offer valuable insights into the general factors that influence the adoption of CSA practices, their findings may not be fully applicable to the burgeoning agritourism farmers in Nigeria. This study presents a novel perspective on the socio-economic determinants of CSA adoption among the nascent agritourism farmers in Nigeria. The data were collected through a well-structured questionnaire administered to 436 agritourism farmers in Nigeria. The five mutually inclusive endogenous variables that capture the impact of CSA practices were agroforestry system, improved livestock management, organic farming, crop rotation/intercropping, and farmer field schools. While the agritourism farmers possess moderate experience and education, significant gaps exist in access to critical resources like credit, climate information, extension services, and membership in agritourism cooperatives/associations. The multivariate probit (MVP) model revealed that agritourism farming experience significantly boosts crop rotation/intercropping adoption. Education enhances organic farming uptake but negatively impacts improved livestock management. Similarly, extension services access promotes farmer field schools while discouraging organic farming. Significant negative covariance matrix between CSA practices suggests overlapping demands for limited farm resources. Full article

The integrity and reliability of wind turbine electrical data (such as active power, voltage, current, etc.) are crucial for operational monitoring, fault diagnosis, and predictive analysis in wind energy systems. However, due to various reasons such as hardware failures, network communication issues, environmental interference, and human errors, data gaps still exist in the Supervisory Control and Data Acquisition (SCADA) systems. Existing multivariate wind power time series imputation methods face two main limitations: (1) inadequate handling of continuous missing patterns (band missing and feature missing) and (2) insufficient utilization of spatiotemporal and feature correlations among wind turbines. To address these shortcomings, this study proposes an imputation framework that includes two types of SCADA data missing scenarios in wind turbines. For band missing, the framework leverages similar wind turbine data matching to explore spatiotemporal correlations in wind power data. For feature missing, the framework focuses on feature correlations in wind power data using Pearson coefficients and normalized mutual information. Additionally, we designed a novel Dual-Type Deep Convolutional Generative Adversarial Imputation Network (DT-DCGAIN) model within this framework to impute different types of missing data. Finally, by evaluating the proposed method on real-world wind farm SCADA datasets, it achieved a 13.91% to 28.32% improvement in Root Mean Square Error (RMSE). Ablation experiments on the model further validated the contributions of each correlation extraction module. Full article

Aging and poor sleep quality are associated with altered brain dynamics, yet current electroencephalography (EEG) analyses often overlook regional complexity. This study addresses this gap by introducing a novel integration of intra- and inter-regional complexity analysis using multivariate multiscale dispersion entropy (mvMDE) from awake resting-state EEG for the first time. Moreover, assessing both intra- and inter-regional complexity provides a comprehensive perspective on the dynamic interplay between localized neural activity and its coordination across brain regions, which is essential for understanding the neural substrates of aging and sleep quality. Data from 58 participants—24 young adults (mean age = 24.7 ± 3.4) and 34 older adults (mean age = 72.9 ± 4.2)—were analyzed, with each age group further divided based on Pittsburgh Sleep Quality Index (PSQI) scores. To capture inter-regional complexity, mvMDE was applied to the most informative group of sensors, with one sensor selected from each brain region using four methods: highest average correlation, highest entropy, highest mutual information, and highest principal component loading. This targeted approach reduced computational cost and enhanced the effect sizes (ESs), particularly at large scale factors (e.g., 25) linked to delta-band activity, with the PCA-based method achieving the highest ESs (1.043 for sleep quality in older adults). Overall, we expect that both inter- and intra-regional complexity will play a pivotal role in elucidating neural mechanisms as captured by various physiological data modalities—such as EEG, magnetoencephalography, and magnetic resonance imaging—thereby offering promising insights for a range of biomedical applications. Full article

This study proposes a novel modelling approach that integrates mutual information (MI)-based parameter screening with random forest (RF) modelling to achieve an accurate quantitative prediction of surface hardness and impact energy in two martensitic stainless steels (1Cr13 and 2Cr13). Preliminary analyses indicated that the magnetic parameters derived from Barkhausen noise (MBN), and the incremental permeability (IP) measurements showed limited linear correlations with the target properties (surface hardness and impact energy). To address this challenge, an MI feature screening method has been developed to identify both the linear and non-linear parameter dependencies that are critical for predicting target mechanical properties. The selected features were then fed into an RF model, which outperformed traditional multiple linear regression in handling the complex, non-monotonic relationships between magnetic signatures and mechanical performance. A key advantage of the proposed MI-RF framework lies in its robustness to small sample sizes, where it achieved high prediction accuracy (e.g., R² > 0.97 for hardness, and R² > 0.86 for impact energy) using limited experimental data. By leveraging MI’s ability to capture multivariate dependencies and RF’s ensemble learning power, it effectively mitigates overfitting and improves generalisation. In addition to demonstrating a promising tool for the non-destructive evaluation of martensitic steels, this study also provides a transferable paradigm for the quantitative assessment of other mechanical properties by magnetic feature fusion. Full article

Background: In the realm of system biology, it is a challenging endeavor to infer a gene regulatory network from time-series gene expression data. Numerous Boolean network inference techniques have emerged for reconstructing a gene regulatory network from a time-series gene expression dataset. However, most of these techniques pose scalability concerns given their capability to consider only two to three regulatory genes over a specific target gene. Methods: To overcome this limitation, a novel inference method, LBF-MI, has been proposed in this research. This two-phase method utilizes limited Boolean functions and multivariate mutual information to reconstruct a Boolean gene regulatory network from time-series gene expression data. Initially, Boolean functions are applied to determine the optimum solutions. In case of failure, multivariate mutual information is applied to obtain the optimum solutions. Results: This research conducted a performance-comparison experiment between LBF-MI and three other methods: mutual information-based Boolean network inference, context likelihood relatedness, and relevance network. When examined on artificial as well as real-time-series gene expression data, the outcomes exhibited that the proposed LBF-MI method outperformed mutual information-based Boolean network inference, context likelihood relatedness, and relevance network on artificial datasets, and two real Escherichia coli datasets (E. coli gene regulatory network, and SOS response of E. coli regulatory network). Conclusions: LBF-MI’s superior performance in gene regulatory network inference enables researchers to uncover the regulatory mechanisms and cellular behaviors of various organisms. Full article

The precise detection of effluent biological oxygen demand (BOD) is crucial for the stable operation of wastewater treatment plants (WWTPs). However, existing detection methods struggle to meet the evolving drainage standards and management requirements. To address this issue, this paper proposed a multivariable probability density-based auto-reconstruction bidirectional long short-term memory (MPDAR-Bi-LSTM) soft sensor for predicting effluent BOD, enhancing the prediction accuracy and efficiency. Firstly, the selection of appropriate auxiliary variables for soft-sensor modeling is determined through the calculation of k-nearest-neighbor mutual information (KNN-MI) values between the global process variables and effluent BOD. Subsequently, considering the existence of strong interactions among different reaction tanks, a Bi-LSTM neural network prediction model is constructed with historical data. Then, a multivariate probability density-based auto-reconstruction (MPDAR) strategy is developed for adaptive updating of the prediction model, thereby enhancing its robustness. Finally, the effectiveness of the proposed soft sensor is demonstrated through experiments using the dataset from Benchmark Simulation Model No.1 (BSM1). The experimental results indicate that the proposed soft sensor not only outperforms some traditional models in terms of prediction performance but also excels in avoiding ineffective model reconstructions in scenarios involving complex dynamic wastewater treatment conditions. Full article

The symmetric Kullback–Leibler centroid, also called the Jeffreys centroid, of a set of mutually absolutely continuous probability distributions on a measure space provides a notion of centrality which has proven useful in many tasks, including information retrieval, information fusion, and clustering. However, the Jeffreys centroid is not available in closed form for sets of categorical or multivariate normal distributions, two widely used statistical models, and thus needs to be approximated numerically in practice. In this paper, we first propose the new Jeffreys–Fisher–Rao center defined as the Fisher–Rao midpoint of the sided Kullback–Leibler centroids as a plug-in replacement of the Jeffreys centroid. This Jeffreys–Fisher–Rao center admits a generic formula for uni-parameter exponential family distributions and a closed-form formula for categorical and multivariate normal distributions; it matches exactly the Jeffreys centroid for same-mean normal distributions and is experimentally observed in practice to be close to the Jeffreys centroid. Second, we define a new type of inductive center generalizing the principle of the Gauss arithmetic–geometric double sequence mean for pairs of densities of any given exponential family. This new Gauss–Bregman center is shown experimentally to approximate very well the Jeffreys centroid and is suggested to be used as a replacement for the Jeffreys centroid when the Jeffreys–Fisher–Rao center is not available in closed form. Furthermore, this inductive center always converges and matches the Jeffreys centroid for sets of same-mean normal distributions. We report on our experiments, which first demonstrate how well the closed-form formula of the Jeffreys–Fisher–Rao center for categorical distributions approximates the costly numerical Jeffreys centroid, which relies on the Lambert W function, and second show the fast convergence of the Gauss–Bregman double sequences, which can approximate closely the Jeffreys centroid when truncated to a first few iterations. Finally, we conclude this work by reinterpreting these fast proxy Jeffreys–Fisher–Rao and Gauss–Bregman centers of Jeffreys centroids under the lens of dually flat spaces in information geometry. Full article

Investigating the stability of and fluctuations in the energy market has long been of interest to researchers and financial market participants. This study aimed to analyze the Chinese energy market, focusing on its volatility and response to financial tensions. For this purpose, data from eight major financial companies, which were selected based on their market share in Shanghai’s and Shenzhen’s financial markets, were collected from January 2014 to December 2023. In this study, stock prices and trading volumes were used as the key variables to build bootstrap-based minimum spanning trees (BMSTs) using ratio mutual information (RMI). Then, using the sliding window procedure, the major network characteristics were derived to create an anomaly-detection tool using the multivariate exponentially weighted moving average (MEWMA), along with the Brent crude oil price index as a benchmark and a global oil price indicator. This framework’s stability was evaluated through stress testing with five scenarios designed for this purpose. The results demonstrate that during periods of high oil price volatility, such as during the turbulence in the stock market in 2015 and the COVID-19 pandemic in 2020, the network topologies became more centralized, which shows that the market’s instability increased. This framework successfully identifies anomalies and proves to be a valuable tool for market players and policymakers in evaluating companies that are active in the energy sector and predicting possible instabilities, which could be useful in monitoring financial markets and improving decision-making processes in the energy sector. In addition, the integration of other macroeconomic factors into this field could strengthen the identification of anomalies and be considered a field for possible research. Full article

In recent years, there has been a notably increased interest in the study of multivariate interactions and emergent higher-order dependencies. This is particularly evident in the context of identifying synergistic sets, which are defined as combinations of elements whose joint interactions result in the emergence of information that is not present in any individual subset of those elements. The scalability of frameworks such as partial information decomposition (PID) and those based on multivariate extensions of mutual information, such as O-information, is limited by combinational explosion in the number of sets that must be assessed. In order to address these challenges, we propose a novel approach that utilises stochastic search strategies in order to identify synergistic triplets within datasets. Furthermore, the methodology is extensible to larger sets and various synergy measures. By employing stochastic search, our approach circumvents the constraints of exhaustive enumeration, offering a scalable and efficient means to uncover intricate dependencies. The flexibility of our method is illustrated through its application to two epidemiological datasets: The Young Finns Study and the UK Biobank Nuclear Magnetic Resonance (NMR) data. Additionally, we present a heuristic for reducing the number of synergistic sets to analyse in large datasets by excluding sets with overlapping information. We also illustrate the risks of performing a feature selection before assessing synergistic information in the system. Full article

Solar flares are significant occurrences in solar physics, impacting space weather and terrestrial technologies. Accurate classification of solar flares is essential for predicting space weather and minimizing potential disruptions to communication, navigation, and power systems. This study addresses the challenge of selecting the most relevant features from multivariate time-series data, specifically focusing on solar flares. We employ methods such as Mutual Information (MI), Minimum Redundancy Maximum Relevance (mRMR), and Euclidean Distance to identify key features for classification. Recognizing the performance variability of different feature selection techniques, we introduce an ensemble approach to compute feature weights. By combining outputs from multiple methods, our ensemble method provides a more comprehensive understanding of the importance of features. Our results show that the ensemble approach significantly improves classification performance, achieving values 0.15 higher in True Skill Statistic (TSS) values compared to individual feature selection methods. Additionally, our method offers valuable insights into the underlying physical processes of solar flares, leading to more effective space weather forecasting and enhanced mitigation strategies for communication, navigation, and power system disruptions. Full article

Multivariate pattern analysis (MVPA) has played an extensive role in interpreting brain activity, which has been applied in studies with modalities such as functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG) and Electroencephalography (EEG). The advent of wearable MEG systems based on optically pumped magnetometers (OPMs), i.e., OP-MEG, has broadened the application of bio-magnetism in the realm of neuroscience. Nonetheless, it also raises challenges in temporal decoding analysis due to the unique attributes of OP-MEG itself. The efficacy of decoding performance utilizing multimodal fusion, such as MEG-EEG, also remains to be elucidated. In this regard, we investigated the impact of several factors, such as processing methods, models and modalities, on the decoding outcomes of OP-MEG. Our findings indicate that the number of averaged trials, dimensionality reduction (DR) methods, and the number of cross-validation folds significantly affect the decoding performance of OP-MEG data. Additionally, decoding results vary across modalities and fusion strategy. In contrast, decoder type, resampling frequency, and sliding window length exert marginal effects. Furthermore, we introduced mutual information (MI) to investigate how information loss due to OP-MEG data processing affect decoding accuracy. Our study offers insights for linear decoding research using OP-MEG and expand its application in the fields of cognitive neuroscience. Full article

Along with the popularity of mobile Internet and smart applications, more and more high-dimensional sensor data have appeared, and these high-dimensional sensor data have hidden information about system performance degradation, system failure, etc., and how to mine them to obtain such information is a very difficult problem. This challenge can be solved by anomaly detection techniques, which is an important field of research in data mining, especially in the domains of network security, credit card fraud detection, industrial fault identification, etc. However, there are many difficulties in anomaly detection in multivariate time-series data, including poor accuracy, fast data generation, lack of labeled data, and how to capture information between sensors. To address these issues, we present a mutual information and graph embedding based anomaly detection algorithm in multivariate time series, called MGAD (mutual information and graph embedding based anomaly detection). The MGAD algorithm consists of four steps: (1) Embedding of sensor data, where heterogeneous sensor data become different vectors in the same vector space; (2) Constructing a relationship graph between sensors using their mutual information about each other; (3) Learning the relationship graph between sensors using a graph attention mechanism, to predict the sensor data at the next moment; (4) Compare the predicted values with the real sensor data to detect potential outliers. Our contributions are as follows: (1) we propose an unsupervised outlier detection called MGAD with a high interpretability and accuracy; (2) massive experiments on benchmark datasets have demonstrated the superior performance of the MGAD algorithm, compared with state-of-the-art baselines in terms of ROC, F1, and AP. Full article

The introduction of exoskeletons in industry has focused on improving worker safety. Exoskeletons have the objective of decreasing the risk of injury or fatigue when performing physically demanding tasks. Exoskeletons’ effect on the muscles is one of the most common focuses of their assessment. The present study aimed to analyze the muscle interactions generated during load-handling tasks in laboratory conditions with and without a passive lumbar exoskeleton. The electromyographic data of the muscles involved in the task were recorded from twelve participants performing load-handling tasks. The correlation coefficient, coherence coefficient, mutual information, and multivariate sample entropy were calculated to determine if there were significant differences in muscle interactions between the two test conditions. The results showed that muscle coordination was affected by the use of the exoskeleton. In some cases, the exoskeleton prevented changes in muscle coordination throughout the execution of the task, suggesting a more stable strategy. Additionally, according to the directed Granger causality, a trend of increasing bottom-up activation was found throughout the task when the participant was not using the exoskeleton. Among the different variables analyzed for coordination, the most sensitive to changes was the multivariate sample entropy. Full article

In this paper, we provide geometric insights with visualization into the multivariate Gaussian distribution and its entropy and mutual information. In order to develop the multivariate Gaussian distribution with entropy and mutual information, several significant methodologies are presented through the discussion, supported by illustrations, both technically and statistically. The paper examines broad measurements of structure for the Gaussian distributions, which show that they can be described in terms of the information theory between the given covariance matrix and correlated random variables (in terms of relative entropy). The content obtained allows readers to better perceive concepts, comprehend techniques, and properly execute software programs for future study on the topic’s science and implementations. It also helps readers grasp the themes’ fundamental concepts to study the application of multivariate sets of data in Gaussian distribution. The simulation results also convey the behavior of different elliptical interpretations based on the multivariate Gaussian distribution with entropy for real-world applications in our daily lives, including information coding, nonlinear signal detection, etc. Involving the relative entropy and mutual information as well as the potential correlated covariance analysis, a wide range of information is addressed, including basic application concerns as well as clinical diagnostics to detect the multi-disease effects. Full article