Entropy

Journal Browser

► Journal Browser

Bridging Bayesian and Information-Theoretic Approaches in Earth and Environmental System Modeling: Theory and Practical Applications

Share This Special Issue

Special Issue Editors

Dr. Anneli Guthke

E-Mail Website
Guest Editor

Stuttgart Center for Simulation Science, Cluster of Excellence EXC 2075, University of Stuttgart, 70569 Stuttgart, Germany
Interests: Bayesian model assessment; information-theoretic concepts; model diagnostics; hybrid modeling; model choice uncertainty; data-integrated modeling; surrogate modeling
Special Issues, Collections and Topics in MDPI journals

Dr. Jasper A. Vrugt

E-Mail Website
Guest Editor

Department of Civil and Environmental Engineering, University of California Irvine, Irvine, CA 92697, USA
Interests: complex systems; data analysis; model evaluation; model diagnostics, probabilistic evaluation, uncertainty quantification; Bayesian estimation; statistical inference; information theory, sensitivity analysis

Dr. Steven V. Weijs

E-Mail Website
Guest Editor

Department of Civil Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada
Interests: information theory; algorithmic information theory; hydrology; monitoring network design; probabilistic forecasting; forecast evaluation

Special Issue Information

Dear Colleagues,

Information theory has gained interest in the past decade in numerous scientific disciplines. Although Shannon information theory is rooted in probability theory, its connection with Bayesian theory is rarely explored, established or demonstrated in practical applications. We invite contributions that link information and Bayesian theory using theoretical examples or cross-cutting applications. We are especially interested in manuscripts with the following characteristics:

Manuscripts that demonstrate the practical usefulness and power of information-theoretic approaches for Earth and environmental system modeling, learning and prediction;
Manuscripts that highlight computational and/or implementation challenges;
Manuscripts that address theoretical limitations, pitfalls or shortcomings.

Topics may include, but are not limited to, the following: uncertainty quantification, hypothesis testing, (probabilistic) model evaluation, compression/learning, causal inference and decision-making under uncertainty.

Dr. Anneli Guthke
Dr. Jasper A. Vrugt
Dr. Steven V. Weijs
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Benefits of Publishing in a Special Issue

Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.

Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.

Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.

External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.

Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (2 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

31 pages, 1868 KB

Open AccessArticle

Information Content and Maximum Entropy of Compartmental Systems in Equilibrium

by Holger Metzler and Carlos A. Sierra

Entropy 2025, 27(10), 1085; https://doi.org/10.3390/e27101085 - 21 Oct 2025

Viewed by 272

Abstract

Mass-balanced compartmental systems defy classical deterministic entropy measures since both metric and topological entropy vanish in dissipative dynamics. By interpreting open compartmental systems as absorbing continuous-time Markov chains that describe the random journey of a single representative particle, we allow established information-theoretic principles to be applied to this particular type of deterministic dynamical system. In particular, path entropy quantifies the uncertainty of complete trajectories, while entropy rates measure the average uncertainty of instantaneous transitions. Using Shannon’s information entropy, we derive closed-form expressions for these quantities in equilibrium and extend the maximum entropy principle (MaxEnt) to the problem of model selection in compartmental dynamics. This information-theoretic framework not only provides a systematic way to address equifinality but also reveals hidden structural properties of complex systems such as the global carbon cycle. Full article

(This article belongs to the Special Issue Bridging Bayesian and Information-Theoretic Approaches in Earth and Environmental System Modeling: Theory and Practical Applications)

► Show Figures

Figure 1

57 pages, 12419 KB

Open AccessArticle

The Learning Rate Is Not a Constant: Sandwich-Adjusted Markov Chain Monte Carlo Simulation

by Jasper A. Vrugt and Cees G. H. Diks

Entropy 2025, 27(10), 999; https://doi.org/10.3390/e27100999 - 25 Sep 2025

Viewed by 540

Abstract

A fundamental limitation of maximum likelihood and Bayesian methods under model misspecification is that the asymptotic covariance matrix of the pseudo-true parameter vector

θ_{*}

is not the inverse of the Fisher information, but rather the sandwich covariance matrix [...] Read more.

A fundamental limitation of maximum likelihood and Bayesian methods under model misspecification is that the asymptotic covariance matrix of the pseudo-true parameter vector

θ_{*}

is not the inverse of the Fisher information, but rather the sandwich covariance matrix

\frac{1}{n} A_{*}^{- 1} B_{*}^{} A_{*}^{- 1}

, where

A_{*}

and

B_{*}

are the sensitivity and variability matrices, respectively, evaluated at

θ_{*}

for training data record

ω_{1}, \dots, ω_{n}

. This paper makes three contributions. First, we review existing approaches to robust posterior sampling, including the open-faced sandwich adjustment and magnitude- and curvature-adjusted Markov chain Monte Carlo (MCMC) simulation. Second, we introduce a new sandwich-adjusted MCMC method. Unlike existing approaches that rely on arbitrary matrix square roots, eigendecompositions or a single scaling factor applied uniformly across the parameter space, our method employs a parameter-dependent learning rate

λ (θ)

that enables direction-specific tempering of the likelihood. This allows the sampler to capture directional asymmetries in the sandwich distribution, particularly under model misspecification or in small-sample regimes, and yields credible regions that remain valid when standard Bayesian inference underestimates uncertainty. Third, we propose information-theoretic diagnostics for quantifying model misspecification, including a strictly proper divergence score and scalar summaries based on the Frobenius norm, Earth mover’s distance, and the Herfindahl index. These principled diagnostics complement residual-based metrics for model evaluation by directly assessing the degree of misalignment between the sensitivity and variability matrices,

A_{*}

and

B_{*}

. Applications to two parametric distributions and a rainfall-runoff case study with the Xinanjiang watershed model show that conventional Bayesian methods systematically underestimate uncertainty, while the proposed method yields asymptotically valid and robust uncertainty estimates. Together, these findings advocate for sandwich-based adjustments in Bayesian practice and workflows. Full article

(This article belongs to the Special Issue Bridging Bayesian and Information-Theoretic Approaches in Earth and Environmental System Modeling: Theory and Practical Applications)

► Show Figures

Journal Menu

Journal Browser

Bridging Bayesian and Information-Theoretic Approaches in Earth and Environmental System Modeling: Theory and Practical Applications

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (2 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI