Special Issue "Robust Procedures for Estimating and Testing in the Framework of Divergence Measures"

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (30 November 2020).

Special Issue Editors

Prof. Dr. Leandro Pardo
E-Mail Website
Guest Editor
Department of Statistics and O.R., Complutense University of Madrid, 28040 Madrid, Spain
Interests: minimum divergence estimators: robustness and efficiency; robust test procedures based on minimum divergence estimators; robust test procedures in composite likelihood, empirical likelihood, change point, and time series
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Nirian Martin
E-Mail Website
Guest Editor
Department of Financial and Actuarial Economics & Statistics, Complutense University of Madrid, 28040 Madrid, Spain
Interests: categorical data analysis; change points; empirical likelihood; generalized linear models; order restricted statistical inference; overdispersion models; regression diagnostics; robust statistics; small area estimation; statistical methods in public health for cancer surveillance

Special Issue Information

Dear Colleagues,

The approach for estimating and testing based on suitable divergence measures has become, in the last 30 years, a very popular technique not only in the field of statistics but also in other areas, such as machine learning, pattern recognition, etc. In relation to the estimation problem, it is necessary to minimize a suitable divergence measure between the data and the model under consideration. Some interesting examples of those estimators are the minimum phi-divergence estimators (MPHIE), in particular, the minimum Hellinger distance (MHD) and the minimum density power divergence estimators (MDPDE). The MPHIE are characterized by asymptotic efficiency (BAN estimators), the MHE by asymptotic efficiency and robustness inside the family of the MPHIE, and the MDPD by their robustness without a significant loss of efficiency as well as by the simplicity of getting them, because it is not necessary to use a nonparametric estimator of the true density function.

Based on these estimators of minima divergence or distance, many people have studied the possibility to use them in order to obtain statistics for testing hypotheses. There are some possibilities to use them with that objective: (i) Plugging them in a divergence measure in order to obtain the estimated distance (divergence) between the model, whose parameters have been estimated under the null hypothesis and the model evaluated in all the parameter space; and (ii) extending the concept of the Wald test in the sense of considering MDPDE instead of maximum likelihood estimators. These test statistics have been considered in many different statistical problems: Censoring, equality of means in normal and lognormal models, logistic regression model, multinomial regression in particular and GLM models in general, etc.

The scope of the contributions to this Special Issue will be to present new and original research papers based on MPHIE, MHD, and MDPDE as well as test statistics based on these estimators from a theoretical and applied point of view in different statistical problems with special emphasis on robustness. Manuscripts given solutions to different statistical problems as model selection criteria based on divergence measures or in statistics for high-dimensional data with divergence measures as loss function are welcome. Reviews making emphasis in the most recent state-of-the art in relation to the solution of statistical problems base on divergence measures are also welcome. 

Prof. Dr. Leandro Pardo
Prof. Dr. Nirian Martin
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • divergence measures
  • minimum divergence estimators
  • test statistics based on minimum divergence estimators
  • Wald-type tests based on minimum divergence estimators
  • efficiency; robustness
  • model selection based on minimum distance estimators

Related Special Issue

Published Papers (10 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Editorial

Jump to: Research

Editorial
Robust Procedures for Estimating and Testing in the Framework of Divergence Measures
Entropy 2021, 23(4), 430; https://doi.org/10.3390/e23040430 - 06 Apr 2021
Viewed by 426
Abstract
The approach for estimating and testing based on divergence measures has become, in the last 30 years, a very popular technique not only in the field of statistics, but also in other areas, such as machine learning, pattern recognition, etc [...] Full article

Research

Jump to: Editorial

Article
Rare Event Analysis for Minimum Hellinger Distance Estimators via Large Deviation Theory
Entropy 2021, 23(4), 386; https://doi.org/10.3390/e23040386 - 24 Mar 2021
Cited by 1 | Viewed by 514
Abstract
Hellinger distance has been widely used to derive objective functions that are alternatives to maximum likelihood methods. While the asymptotic distributions of these estimators have been well investigated, the probabilities of rare events induced by them are largely unknown. In this article, we [...] Read more.
Hellinger distance has been widely used to derive objective functions that are alternatives to maximum likelihood methods. While the asymptotic distributions of these estimators have been well investigated, the probabilities of rare events induced by them are largely unknown. In this article, we analyze these rare event probabilities using large deviation theory under a potential model misspecification, in both one and higher dimensions. We show that these probabilities decay exponentially, characterizing their decay via a “rate function” which is expressed as a convex conjugate of a limiting cumulant generating function. In the analysis of the lower bound, in particular, certain geometric considerations arise that facilitate an explicit representation, also in the case when the limiting generating function is nondifferentiable. Our analysis involves the modulus of continuity properties of the affinity, which may be of independent interest. Full article
Article
Distance-Based Estimation Methods for Models for Discrete and Mixed-Scale Data
Entropy 2021, 23(1), 107; https://doi.org/10.3390/e23010107 - 14 Jan 2021
Cited by 1 | Viewed by 596
Abstract
Pearson residuals aid the task of identifying model misspecification because they compare the estimated, using data, model with the model assumed under the null hypothesis. We present different formulations of the Pearson residual system that account for the measurement scale of the data [...] Read more.
Pearson residuals aid the task of identifying model misspecification because they compare the estimated, using data, model with the model assumed under the null hypothesis. We present different formulations of the Pearson residual system that account for the measurement scale of the data and study their properties. We further concentrate on the case of mixed-scale data, that is, data measured in both categorical and interval scale. We study the asymptotic properties and the robustness of minimum disparity estimators obtained in the case of mixed-scale data and exemplify the performance of the methods via simulation. Full article
Article
Monitoring Parameter Change for Time Series Models of Counts Based on Minimum Density Power Divergence Estimator
Entropy 2020, 22(11), 1304; https://doi.org/10.3390/e22111304 - 16 Nov 2020
Cited by 4 | Viewed by 662
Abstract
In this study, we consider an online monitoring procedure to detect a parameter change for integer-valued generalized autoregressive heteroscedastic (INGARCH) models whose conditional density of present observations over past information follows one parameter exponential family distributions. For this purpose, we use the cumulative [...] Read more.
In this study, we consider an online monitoring procedure to detect a parameter change for integer-valued generalized autoregressive heteroscedastic (INGARCH) models whose conditional density of present observations over past information follows one parameter exponential family distributions. For this purpose, we use the cumulative sum (CUSUM) of score functions deduced from the objective functions, constructed for the minimum power divergence estimator (MDPDE) that includes the maximum likelihood estimator (MLE), to diminish the influence of outliers. It is well-known that compared to the MLE, the MDPDE is robust against outliers with little loss of efficiency. This robustness property is properly inherited by the proposed monitoring procedure. A simulation study and real data analysis are conducted to affirm the validity of our method. Full article
Show Figures

Figure 1

Article
Some Dissimilarity Measures of Branching Processes and Optimal Decision Making in the Presence of Potential Pandemics
Entropy 2020, 22(8), 874; https://doi.org/10.3390/e22080874 - 08 Aug 2020
Cited by 1 | Viewed by 1078
Abstract
We compute exact values respectively bounds of dissimilarity/distinguishability measures–in the sense of the Kullback-Leibler information distance (relative entropy) and some transforms of more general power divergences and Renyi divergences–between two competing discrete-time Galton-Watson branching processes with immigration GWI for which the offspring as [...] Read more.
We compute exact values respectively bounds of dissimilarity/distinguishability measures–in the sense of the Kullback-Leibler information distance (relative entropy) and some transforms of more general power divergences and Renyi divergences–between two competing discrete-time Galton-Watson branching processes with immigration GWI for which the offspring as well as the immigration (importation) is arbitrarily Poisson-distributed; especially, we allow for arbitrary type of extinction-concerning criticality and thus for non-stationarity. We apply this to optimal decision making in the context of the spread of potentially pandemic infectious diseases (such as e.g., the current COVID-19 pandemic), e.g., covering different levels of dangerousness and different kinds of intervention/mitigation strategies. Asymptotic distinguishability behaviour and diffusion limits are investigated, too. Full article
Show Figures

Figure 1

Article
Robust Change Point Test for General Integer-Valued Time Series Models Based on Density Power Divergence
Entropy 2020, 22(4), 493; https://doi.org/10.3390/e22040493 - 24 Apr 2020
Cited by 7 | Viewed by 1281
Abstract
In this study, we consider the problem of testing for a parameter change in general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family when the data are contaminated by outliers. In particular, we use a robust change point [...] Read more.
In this study, we consider the problem of testing for a parameter change in general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family when the data are contaminated by outliers. In particular, we use a robust change point test based on density power divergence (DPD) as the objective function of the minimum density power divergence estimator (MDPDE). The results show that under regularity conditions, the limiting null distribution of the DPD-based test is a function of a Brownian bridge. Monte Carlo simulations are conducted to evaluate the performance of the proposed test and show that the test inherits the robust properties of the MDPDE and DPD. Lastly, we demonstrate the proposed test using a real data analysis of the return times of extreme events related to Goldman Sachs Group stock. Full article
Show Figures

Figure 1

Article
Robust Regression with Density Power Divergence: Theory, Comparisons, and Data Analysis
Entropy 2020, 22(4), 399; https://doi.org/10.3390/e22040399 - 31 Mar 2020
Cited by 5 | Viewed by 1440
Abstract
Minimum density power divergence estimation provides a general framework for robust statistics, depending on a parameter α , which determines the robustness properties of the method. The usual estimation method is numerical minimization of the power divergence. The paper considers the special case [...] Read more.
Minimum density power divergence estimation provides a general framework for robust statistics, depending on a parameter α , which determines the robustness properties of the method. The usual estimation method is numerical minimization of the power divergence. The paper considers the special case of linear regression. We developed an alternative estimation procedure using the methods of S-estimation. The rho function so obtained is proportional to one minus a suitably scaled normal density raised to the power α . We used the theory of S-estimation to determine the asymptotic efficiency and breakdown point for this new form of S-estimation. Two sets of comparisons were made. In one, S power divergence is compared with other S-estimators using four distinct rho functions. Plots of efficiency against breakdown point show that the properties of S power divergence are close to those of Tukey’s biweight. The second set of comparisons is between S power divergence estimation and numerical minimization. Monitoring these two procedures in terms of breakdown point shows that the numerical minimization yields a procedure with larger robust residuals and a lower empirical breakdown point, thus providing an estimate of α leading to more efficient parameter estimates. Full article
Show Figures

Figure 1

Article
Robust Model Selection Criteria Based on Pseudodistances
Entropy 2020, 22(3), 304; https://doi.org/10.3390/e22030304 - 06 Mar 2020
Cited by 5 | Viewed by 953
Abstract
In this paper, we introduce a new class of robust model selection criteria. These criteria are defined by estimators of the expected overall discrepancy using pseudodistances and the minimum pseudodistance principle. Theoretical properties of these criteria are proved, namely asymptotic unbiasedness, robustness, consistency, [...] Read more.
In this paper, we introduce a new class of robust model selection criteria. These criteria are defined by estimators of the expected overall discrepancy using pseudodistances and the minimum pseudodistance principle. Theoretical properties of these criteria are proved, namely asymptotic unbiasedness, robustness, consistency, as well as the limit laws. The case of the linear regression models is studied and a specific pseudodistance based criterion is proposed. Monte Carlo simulations and applications for real data are presented in order to exemplify the performance of the new methodology. These examples show that the new selection criterion for regression models is a good competitor of some well known criteria and may have superior performance, especially in the case of small and contaminated samples. Full article
Show Figures

Figure 1

Article
Model Selection in a Composite Likelihood Framework Based on Density Power Divergence
Entropy 2020, 22(3), 270; https://doi.org/10.3390/e22030270 - 27 Feb 2020
Cited by 2 | Viewed by 930
Abstract
This paper presents a model selection criterion in a composite likelihood framework based on density power divergence measures and in the composite minimum density power divergence estimators, which depends on an tuning parameter α . After introducing such a criterion, some asymptotic properties [...] Read more.
This paper presents a model selection criterion in a composite likelihood framework based on density power divergence measures and in the composite minimum density power divergence estimators, which depends on an tuning parameter α . After introducing such a criterion, some asymptotic properties are established. We present a simulation study and two numerical examples in order to point out the robustness properties of the introduced model selection criterion. Full article
Article
Convergence Rates for Empirical Estimation of Binary Classification Bounds
Entropy 2019, 21(12), 1144; https://doi.org/10.3390/e21121144 - 23 Nov 2019
Cited by 1 | Viewed by 1259
Abstract
Bounding the best achievable error probability for binary classification problems is relevant to many applications including machine learning, signal processing, and information theory. Many bounds on the Bayes binary classification error rate depend on information divergences between the pair of class distributions. Recently, [...] Read more.
Bounding the best achievable error probability for binary classification problems is relevant to many applications including machine learning, signal processing, and information theory. Many bounds on the Bayes binary classification error rate depend on information divergences between the pair of class distributions. Recently, the Henze–Penrose (HP) divergence has been proposed for bounding classification error probability. We consider the problem of empirically estimating the HP-divergence from random samples. We derive a bound on the convergence rate for the Friedman–Rafsky (FR) estimator of the HP-divergence, which is related to a multivariate runs statistic for testing between two distributions. The FR estimator is derived from a multicolored Euclidean minimal spanning tree (MST) that spans the merged samples. We obtain a concentration inequality for the Friedman–Rafsky estimator of the Henze–Penrose divergence. We validate our results experimentally and illustrate their application to real datasets. Full article
Show Figures

Figure 1

Back to TopTop