Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 

Saved Queries

Sign in to use this feature.

Search Filter Reset All

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
Add Subjects Reset
Select Subjects

Journals

remove_circle_outline
Add Journals Reset
Select Journals

Article Types

remove_circle_outline
Add Article Types Reset
Select Article Types

Countries / Regions

remove_circle_outline
Add Countries / Regions Reset
Select Countries / Regions
Update Search
share announcement

Search Results (2)

Search Parameters:
Keywords = Hyvarinen score

Order results
Result details
Results per page
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
22 pages, 1497 KiB  
Article
Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
by Frank Nielsen
Entropy 2021, 23(11), 1417; https://doi.org/10.3390/e23111417 - 28 Oct 2021
Cited by 10 | Viewed by 4712
Abstract
The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature [...] Read more.
The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to α-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes. Full article
(This article belongs to the Special Issue Distance in Information and Statistical Physics III)
►▼ Show Figures

Graphical abstract

10 pages, 310 KiB  
Article
On Selection Criteria for the Tuning Parameter in Robust Divergence
by Shonosuke Sugasawa and Shouto Yonekura
Entropy 2021, 23(9), 1147; https://doi.org/10.3390/e23091147 - 1 Sep 2021
Cited by 14 | Viewed by 3613
Abstract
Although robust divergence, such as density power divergence and γ-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. [...] Read more.
Although robust divergence, such as density power divergence and γ-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression. Full article
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying article 1-50 on page 1 of 1.
Back to TopTop