Entropy

Research

27 pages, 392 KB

Open AccessArticle

Divergence and Model Adequacy, a Semiparametric Case Study

by Michel Broniatowski and Justin Moutsouka

Entropy 2026, 28(7), 758; https://doi.org/10.3390/e28070758 - 2 Jul 2026

Viewed by 168

Adequacy for estimation between an inferential method and a model can be defined through two main requirements: firstly the inferential tool should define a well posed problem when applied to the model; secondly the resulting statistical procedure should produce consistent estimators. Conditions which [...] Read more.

Adequacy for estimation between an inferential method and a model can be defined through two main requirements: firstly the inferential tool should define a well posed problem when applied to the model; secondly the resulting statistical procedure should produce consistent estimators. Conditions which entail these analytical and statistical issues are considered in the context when divergence based inference is applied for smooth semiparametric models under moment restrictions. A discussion is also held on the choice of the divergence, extending the classical parametric inference to the estimation of both parameters of interest and of nuisance.Classical arguments in favor of the omnibus choice of the

L^{2}

and Kullback Leibler divergences are discussed and motivation for the class of power divergences is presented in the context of the present semi parametric smooth models. A short simulation study illustrates the method. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

22 pages, 679 KB

Open AccessArticle

Asymptotic Normality and Convergence Rates for Tsallis Entropy Estimators via Stabilization Techniques

by Mehmet Sıddık Çadırcı and Martin Singull

Entropy 2026, 28(6), 619; https://doi.org/10.3390/e28060619 - 31 May 2026

Viewed by 232

Abstract

We study nearest-neighbor-based estimators of Tsallis entropy associated with Poisson and binomial point processes on general metric measure spaces. In this study, by combining existing stabilization methods with the validation of the estimator’s local k-nearest-neighbor structure, we investigate nearest-neighbor-based Tsallis entropy estimators [...] Read more.

We study nearest-neighbor-based estimators of Tsallis entropy associated with Poisson and binomial point processes on general metric measure spaces. In this study, by combining existing stabilization methods with the validation of the estimator’s local k-nearest-neighbor structure, we investigate nearest-neighbor-based Tsallis entropy estimators under Poisson and binomial distributed input data. Rather than proposing a new second-order Poincaré inequality, this paper details and clearly presents stabilization-based normal approximation bounds for Tsallis-type k-NN functionals. We establish asymptotic normality and derive explicit convergence rates for the Kolmogorov distance. Our analysis avoids explicit score-function decompositions and instead relies on flexible localizations of add-one costs, which simplify the treatment of higher-order terms. Under natural stabilization and moment conditions, the resulting bounds recover the classical normal approximation rates

s^{- 1 / 2}

and

n^{- 1 / 2}

and extend corresponding results for Shannon and Rényi entropy estimators. We further illustrate the scope of the framework through examples involving Tsallis entropy functionals, weighted k-NN Shannon entropy estimators. The examples provided highlight the benefits of stabilization-based normal approximations for non-parametric statistical inference in complex spatial and high-dimensional settings. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

29 pages, 489 KB

Open AccessArticle

A Sequential Design for Extreme Quantile Estimation Under Binary Sampling

by Michel Broniatowski and Emilie Miranda

Entropy 2026, 28(4), 479; https://doi.org/10.3390/e28040479 - 21 Apr 2026

Viewed by 461

Abstract

We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of binary data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists of estimating [...] Read more.

We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of binary data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists of estimating a failure quantile from trials whose outcomes are reduced to indicators of whether the specimen has failed at the tested stress levels. The proposed approach relies on a splitting strategy that decomposes the target extreme probability into a product of higher-order conditional probabilities, enabling a progressive exploration of the tail of the distribution through sampling under truncated laws. We consider GEV and Weibull models for the underlying distribution, and the sequential estimation of their parameters is carried out using an enhanced maximum likelihood procedure specifically adapted to binary data, addressing the substantial uncertainty inherent to such limited information. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

25 pages, 712 KB

Open AccessEditor’s ChoiceArticle

Decision-Making Under Model Misspecification: DRO with Robust Bayesian Ambiguity Sets

by Charita Dellaporta, Patrick O’Hara and Theodoros Damoulas

Entropy 2026, 28(4), 430; https://doi.org/10.3390/e28040430 - 11 Apr 2026

Viewed by 704

Abstract

Distributionally Robust Optimisation (DRO) protects risk-averse decision-makers by considering the worst-case risk within an ambiguity set of distributions based on the empirical distribution or a model. To further guard against finite, noisy data, model-based approaches admit Bayesian formulations that propagate uncertainty from the [...] Read more.

Distributionally Robust Optimisation (DRO) protects risk-averse decision-makers by considering the worst-case risk within an ambiguity set of distributions based on the empirical distribution or a model. To further guard against finite, noisy data, model-based approaches admit Bayesian formulations that propagate uncertainty from the posterior to the decision-making problem. However, when the model is misspecified, the decision-maker must stretch the ambiguity set to contain the data-generating process (DGP), leading to overly conservative decisions. We address this challenge by introducing DRO with Robust ayesian Ambiguity Sets (DRO-RoBAS) to model misspecification. These are Maximum Mean Discrepancy ambiguity sets centred at a robust posterior predictive distribution that incorporates beliefs about the DGP. We show that the resulting optimisation problem obtains a dual formulation in the Reproducing Kernel Hilbert Space and we give probabilistic guarantees on the tolerance level of the ambiguity set. Our method outperforms other Bayesian and empirical DRO approaches in out-of-sample performance on the Newsvendor and Portfolio problems with various cases of model misspecification. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

27 pages, 536 KB

Open AccessArticle

Efficient EM Estimation for the Pogit Model via Polya-Gamma Augmentation

by Iván Gutiérrez, Sandra Ramírez and Leonardo Jofré

Entropy 2026, 28(2), 207; https://doi.org/10.3390/e28020207 - 11 Feb 2026

Viewed by 849

Abstract

The Poisson-logistic (pogit) model is widely used for count data with latent intensities, with applications including under-reporting correction and share-of-wallet estimation, yet existing estimation methods do not scale well to large datasets. We propose a new expectation-maximization (EM) algorithm for the standard pogit [...] Read more.

The Poisson-logistic (pogit) model is widely used for count data with latent intensities, with applications including under-reporting correction and share-of-wallet estimation, yet existing estimation methods do not scale well to large datasets. We propose a new expectation-maximization (EM) algorithm for the standard pogit model based on Polya-Gamma data augmentation, which yields a conditionally Gaussian complete-data likelihood with closed-form EM-updates. The resulting EM algorithm has low per-iteration cost and naturally accommodates computational enhancements, including quasi-Newton acceleration and mini-batch implementations. These features enable efficient inference on datasets with millions of observations. Simulation studies and real-data applications demonstrate substantial computational improvements without loss of statistical accuracy, and comparisons with direct maximum-likelihood optimization routines show that the proposed method provides a scalable and competitive alternative for large-scale pogit estimation. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

26 pages, 877 KB

Open AccessArticle

New Modified Generalized Inverted Exponential Distribution and Its Applications

by Zakeia A. Al-Saiary and Hana H. Al-Jammaz

Entropy 2026, 28(2), 161; https://doi.org/10.3390/e28020161 - 31 Jan 2026

Viewed by 437

Abstract

In this paper, a statistical model with three parameters is proposed which is called New Modified Generalized Inverted Exponential Distribution (MGIE). In addition, several statistical characteristics of the MGIE distribution are obtained, including quantile function, median, moments, mode, mean deviation, harmonic mean, reliability, [...] Read more.

In this paper, a statistical model with three parameters is proposed which is called New Modified Generalized Inverted Exponential Distribution (MGIE). In addition, several statistical characteristics of the MGIE distribution are obtained, including quantile function, median, moments, mode, mean deviation, harmonic mean, reliability, hazard and odds functions and Rényi entropy. Moreover, the estimators of parameters are found using the maximum likelihood estimation method. A simulation study using the Monte Carlo method is performed to assess the behavior of the parameters. Finally, three real data sets are applied to demonstrate the importance of the proposed distribution. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

15 pages, 486 KB

Open AccessArticle

Tight Bounds for Joint Distribution Functions of Order Statistics Under k-Independence

by Andrzej Okolewski and Barbara Blazejczyk-Okolewska

Entropy 2025, 27(12), 1250; https://doi.org/10.3390/e27121250 - 11 Dec 2025

Viewed by 615

Abstract

The present study investigates the problem of determining sharp bounds for key reliability and distributional characteristics associated with order statistics. We establish pointwise sharp two-sided bounds for linear combinations of joint distribution functions and joint reliability functions of selected order statistics based on [...] Read more.

The present study investigates the problem of determining sharp bounds for key reliability and distributional characteristics associated with order statistics. We establish pointwise sharp two-sided bounds for linear combinations of joint distribution functions and joint reliability functions of selected order statistics based on k-independent and identically distributed random variables. The proposed framework is general and also applies to arbitrarily dependent observations. The obtained results provide exact bounds for the expected values of functions of order statistics corresponding to finite-valued random variables. Furthermore, the study yields the best possible upper and lower bounds for the joint reliability function of semicoherent systems with shared exchangeable k-independent components. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

26 pages, 457 KB

Open AccessArticle

Statistical Inference for High-Dimensional Heteroscedastic Partially Single-Index Models

by Jianglin Fang and Zhikun Tian

Entropy 2025, 27(9), 964; https://doi.org/10.3390/e27090964 - 16 Sep 2025

Viewed by 949

Abstract

In this study, we propose a novel penalized empirical likelihood approach that simultaneously performs parameter estimation and variable selection in heteroscedastic partially linear single-index models with a diverging number of parameters. It is rigorously proved that the proposed method possesses the oracle property: [...] Read more.

In this study, we propose a novel penalized empirical likelihood approach that simultaneously performs parameter estimation and variable selection in heteroscedastic partially linear single-index models with a diverging number of parameters. It is rigorously proved that the proposed method possesses the oracle property: (i) with probability tending to 1, the zero components are consistently estimated as zero; (ii) the estimators for nonzero coefficients achieve asymptotic efficiency. Furthermore, the penalized empirical log-likelihood ratio statistic is shown to asymptotically follow a standard chi-squared distribution under the null hypothesis. This methodology can be naturally applied to pure partially linear models and single-index models in high-dimensional settings. Simulation studies and real-world data analysis are conducted to examine the properties of the presented approach. Full article

(This article belongs to the Special Issue Statistical Inference: Theory and Methods)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Statistical Inference: Theory and Methods

Share This Special Issue

Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI