Search Results (27)

Accurate short-term load forecasting is essential for reliable power system operation under increasingly nonlinear, volatile, and multi-scale load patterns. This study proposes a VMD–BayesXGB–BiLSTM hybrid forecasting framework that integrates time-series-cross-validation-based variational mode decomposition (VMD), Bayesian-optimized XGBoost (BayesXGB), and BiLSTM residual correction. First, abnormal values in the raw load and explanatory variables are detected using the $3\sigma$ criterion and corrected by cubic spline interpolation. Then, VMD parameters are selected only within the training sequence, and leakage-free VMD features are generated from historical input windows, avoiding the use of future information. BayesXGB is employed as the primary forecasting model to capture nonlinear relationships between historical load, VMD-derived multi-scale features, and external variables. Finally, a stacked BiLSTM module learns temporal patterns from historical BayesXGB predictions and residuals, and the predicted residual correction is added to the preliminary forecast. Experiments on an Australian electricity load dataset show that the proposed model achieves an RMSE of 122.1003, an MAE of 90.7386, a MAPE of 1.0269%, and an R² of 0.9921, outperforming all compared baseline models while maintaining sub-millisecond inference per sample. Full article

Rapid advancements in the field of deep learning have had a profound impact on a wide range of scientific studies. This paper incorporates the power of deep neural networks to learn complex relationships in longitudinal data. The novel generative approach, Variational Deep Alliance (VaDA), is established, where an “alliance” is formed across repeated measurements via the strength of Variational Auto-Encoder. VaDA models the generating process of longitudinal data with a unified and well-structured latent space, allowing outcomes prediction, subjects clustering and representation learning simultaneously. The integrated model can be inferred efficiently within a stochastic Auto-Encoding Variational Bayes framework, which is scalable to large datasets and can accommodate variables of mixed type. Quantitative comparisons to those baseline methods are considered. VaDA shows high robustness and generalization capability across various synthetic scenarios. Moreover, a longitudinal study based on the well-known CelebFaces Attributes dataset is carried out, where we show its usefulness in detecting meaningful latent clusters and generating high-quality face images. Full article

We compare three modern Bayesian approaches, Hamiltonian Monte Carlo (HMC), Variational Bayes (VB), and Integrated Nested Laplace Approximation (INLA), for two classic spatial econometric specifications: the spatial lag model and spatial error model. Our Monte Carlo experiments span a range of sample sizes and spatial neighborhood structures to assess accuracy and computational efficiency. Overall, posterior means exhibit minimal bias for most parameters, with precision improving as sample size grows. VB and INLA deliver substantial computational gains over HMC, with VB typically fastest at small and moderate samples and INLA showing excellent scalability at larger samples. However, INLA can be sensitive to dense spatial weight matrices, showing elevated bias and error dispersion for variance and some regression parameters. Two empirical illustrations underscore these findings: a municipal expenditure reaction function for Île-de-France and a hedonic price for housing in Ames, Iowa. Our results yield actionable guidance. HMC remains a gold standard for accuracy when computation permits; VB is a strong, scalable default; and INLA is attractive for large samples provided the weight matrix is not overly dense. These insights help practitioners select Bayesian tools aligned with data size, spatial neighborhood structure, and time constraints. Full article

Groundwater vulnerability assessment (GVA) is critical for understanding contaminant migration into groundwater systems, yet conventional methods often overlook its probabilistic nature. Bayesian inference offers a robust framework using Bayes’ rule to enhance decision-making through posterior probability calculations. This study introduces inverse Bayesian methods for GVA using spatial-series data, focusing on nitrate concentrations in groundwater as an indicator of groundwater vulnerability in agricultural catchments. Using the joint maximum a-posteriori (JMAP) and variational Bayesian approximation (VBA) algorithms, the advantages of the Bayesian framework over traditional index-based methods are demonstrated for GVA of the Burdekin Basin, Queensland, Australia. This provides an evidence-based methodology for GVA which enables model ranking, parameter estimation, and uncertainty quantification. Full article

Friston proposed the Minimum Free Energy Principle (FEP) based on the Variational Bayesian (VB) method. This principle emphasizes that the brain and behavior coordinate with the environment, promoting self-organization. However, it has a theoretical flaw, a possibility of being misunderstood, and a limitation (only likelihood functions are used as constraints). This paper first introduces the semantic information G theory and the R(G) function (where R is the minimum mutual information for the given semantic mutual information G). The G theory is based on the P-T probability framework and, therefore, allows for the use of truth, membership, similarity, and distortion functions (related to semantics) as constraints. Based on the study of the R(G) function and logical Bayesian Inference, this paper proposes the Semantic Variational Bayesian (SVB) and the Maximum Information Efficiency (MIE) principle. Theoretic analysis and computing experiments prove that R − G = F − H(X|Y) (where F denotes VFE, and H(X|Y) is Shannon conditional entropy) instead of F continues to decrease when optimizing latent variables; SVB is a reliable and straightforward approach for latent variables and active inference. This paper also explains the relationship between information, entropy, free energy, and VFE in local non-equilibrium and equilibrium systems, concluding that Shannon information, semantic information, and VFE are analogous to the increment of free energy, the increment of exergy, and physical conditional entropy. The MIE principle builds upon the fundamental ideas of the FEP, making them easier to understand and apply. It needs to combine deep learning methods for wider applications. Full article

We consider the two-sample problem involving a new class of angle-based models for ranking data. These models are functions of the cosine of the angle between a ranking and a consensus vector. A Bayesian approach is employed to determine the corresponding predictive densities. Two competing hypotheses are considered, and we compute the Bayes factor to quantify the evidence provided by the observed data under each hypothesis. We apply the results to a real data set. Full article

We define an evolving in-time Bayesian neural network called a Hidden Markov Neural Network, which addresses the crucial challenge in time-series forecasting and continual learning: striking a balance between adapting to new data and appropriately forgetting outdated information. This is achieved by modelling the weights of a neural network as the hidden states of a Hidden Markov model, with the observed process defined by the available data. A filtering algorithm is employed to learn a variational approximation of the evolving-in-time posterior distribution over the weights. By leveraging a sequential variant of Bayes by Backprop, enriched with a stronger regularization technique called variational DropConnect, Hidden Markov Neural Networks achieve robust regularization and scalable inference. Experiments on MNIST, dynamic classification tasks, and next-frame forecasting in videos demonstrate that Hidden Markov Neural Networks provide strong predictive performance while enabling effective uncertainty quantification. Full article

This paper proposes the “ultimate compression” method as a solution to the expansive computation and high storage costs required by state-of-the-art neural network models in inference. Our approach uniquely combines tensor decomposition techniques with binary neural networks to create efficient deep neural network models optimized for edge inference. The process includes training floating-point models, applying tensor decomposition algorithms, binarizing the decomposed layers, and fine tuning the resulting models. We evaluated our approach in various state-of-the-art deep neural network architectures on multiple datasets, such as MNIST, CIFAR-10, CIFAR-100, and ImageNet. Our results demonstrate compression ratios of up to 169×, with only a small degradation in accuracy (1–2%) compared to binary models. We employed different optimizers for training and fine tuning, including Adam and AdamW, and used norm grad clipping to address the exploding gradient problem in decomposed binary models. A key contribution of this work is a novel layer sensitivity-based rank selection algorithm for tensor decomposition, which outperforms existing methods such as random selection and Variational Bayes Matrix Factorization (VBMF). We conducted comprehensive experiments using six different models and present a case study on crowd-counting applications, demonstrating the practical applicability of our method. The ultimate compression method outperforms binary neural networks and tensor decomposition when applied individually in terms of storage and computation costs. This positions it as one of the most effective options for deploying compact and efficient models in edge devices with limited computational resources and energy constraints. Full article

Finite Gaussian mixture models are powerful tools for modeling distributions of random phenomena and are widely used for clustering tasks. However, their interpretability and efficiency are often degraded by the impact of redundancy and noise, especially on high-dimensional datasets. In this work, we propose a generative graphical model for parsimonious modeling of the Gaussian mixtures and robust unsupervised learning. The model assumes that the data are generated independently and identically from a finite mixture of robust factor analyzers, where the features’ salience is adjusted by an active set of latent factors to allow a violation of the local independence assumption. For the model inference, we propose a structured variational Bayes inference framework to realize simultaneous clustering, model selection and outlier processing. Performance of the proposed algorithm is evaluated by conducting experiments on artificial and real-world datasets. Moreover, an application on the high-dimensional machine learning task of handwritten alphabet recognition is introduced. Full article

Pulse noise (such as glacier fracturing and offshore pile driving), commonly seen in the marine environment, seriously affects the performance of Direction-of-Arrival (DOA) estimation methods in sonar systems. To address this issue, this paper proposes a high robustness underwater target estimation method based on variational sparse Bayesian inference by studying and analyzing the sparse prior assumption characteristics of signals. This method models pulse noise to build an observation signal, completes the derivation of the conditional distribution of the observed variables and the prior distribution of the sparse signals, and combines Variational Bayes (VB) theory to approximate the posterior distribution, thereby obtaining the recovered signal of the sparse signals and reducing the impact of pulse noise on the estimation system. Our simulation results showed that the proposed method achieved higher estimation accuracy than traditional methods in both single and multiple snapshot scenarios and has practical potential. Full article

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model. Full article

Renewable energy sources are constantly increasing in the modern power systems. Due to their intermittent and uncertain potential, increased spinning reserve requirements are needed to conserve the reliability. On the other hand, each action towards efficiency improvement and cost reduction contradicts the participation of variable resources in the energy mix, requiring more accurate tools for optimal unit commitment. By increasing the renewable contribution, not only does the overall system inertia decrease with the decreasing conventional generation, but more generators that are expensive are also introduced. This work provides a radically different approach towards a tractable optimization task based on the framework of Lagrange relaxation and variational Bayes. Following a dual formulation of reliability and cost, the Lagrange multipliers are accelerated via a machine learning mechanism, namely, variational Bayesian inference. The novelty in the proposed approach stems from the employed acquisition function and the effect of the Gaussian process. The obtained results show great improvements compared with the Lagrange relaxation alternative, which can reach over USD 1 M in production cost credits at the least number of function evaluations. The proposed hybrid method promises global solutions relying on a proper acquisition function that is able to move towards regions with minimum objective value and maximum uncertainty. Full article

The ability to track the changes of the surrounding environment is critical for humans and animals to adapt their behaviors. In high-dimensional environments, the interactions between each dimension need to be estimated for better perception and decision making, for example in volatile or social cognition tasks. We develop a hierarchical Bayesian model for inferring and decision making in multi-dimensional volatile environments. The hierarchical Bayesian model is composed of a hierarchical perceptual model and a response model. Using the variational Bayes method, we derived closed-form update rules. These update rules also constitute a complete predictive coding scheme. To validate the effectiveness of the model in multi-dimensional volatile environments, we defined a probabilistic gambling task modified from a two-armed bandit. Simulation results demonstrated that an agent endowed with the proposed hierarchical Bayesian model is able to infer and to update its internal belief on the tendency and volatility of the sensory inputs. Based on the internal belief of the sensory inputs, the agent yielded near-optimal behavior following its response model. Our results pointed this model a viable framework to explain the temporal dynamics of human decision behavior in complex and high dimensional environments. Full article

Censored data play a pivotal role in life testing experiments since they significantly reduce cost and testing time. Hence, this paper investigates the problem of statistical inference for a system of progressive first-failure censoring data for a new Weibull–Pareto distribution. Maximum likelihood estimates for the parameters as well as some lifetime indices such as reliability, hazard rate functions, and coefficient of variation are derived. Lindley approximation and the Markov chain Monte Carlo technique are applied to obtain the Bayes estimates relative to two different loss functions: balanced linear exponential and general entropy loss functions. The results of the Bayes estimate are computed under the consideration of informative prior function. A real-life example "the survival times in years of a group of patients given chemotherapy treatment" is presented to illustrate the proposed methods. Finally, a simulation study is carried out to determine the performance of the maximum likelihood and Bayes estimates and compare the performance of different corresponding confidence intervals. Full article

In micro-electro-mechanical systems (MEMS) testing high overall precision and reliability are essential. Due to the additional requirement of runtime efficiency, machine learning methods have been investigated in recent years. However, these methods are often associated with inherent challenges concerning uncertainty quantification and guarantees of reliability. The goal of this paper is therefore to present a new machine learning approach in MEMS testing based on Bayesian inference to determine whether the estimation is trustworthy. The overall predictive performance as well as the uncertainty quantification are evaluated with four methods: Bayesian neural network, mixture density network, probabilistic Bayesian neural network and BayesFlow. They are investigated under the variation in training set size, different additive noise levels, and an out-of-distribution condition, namely the variation in the damping factor of the MEMS device. Furthermore, epistemic and aleatoric uncertainties are evaluated and discussed to encourage thorough inspection of models before deployment striving for reliable and efficient parameter estimation during final module testing of MEMS devices. BayesFlow consistently outperformed the other methods in terms of the predictive performance. As the probabilistic Bayesian neural network enables the distinction between epistemic and aleatoric uncertainty, their share of the total uncertainty has been intensively studied. Full article