Next Article in Journal
Capacity Bounds on the Downlink of Symmetric, Multi-Relay, Single-Receiver C-RAN Networks
Next Article in Special Issue
Robust-BD Estimation and Inference for General Partially Linear Models
Previous Article in Journal
On the Uniqueness Theorem for Pseudo-Additive Entropies
Open AccessArticle

Robust and Sparse Regression via γ-Divergence

1
Department of Statistical Science, The Graduate University for Advanced Studies, Tokyo 190-8562, Japan
2
The Institute of Statistical Mathematics, Tokyo 190-8562, Japan
3
Department of Mathematical Statistics, Nagoya University Graduate School of Medicine, Nagoya 466-8550, Japan
*
Author to whom correspondence should be addressed.
Entropy 2017, 19(11), 608; https://doi.org/10.3390/e19110608
Received: 30 September 2017 / Revised: 7 November 2017 / Accepted: 9 November 2017 / Published: 13 November 2017
In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation, and the corresponding divergences have been discussed. One such divergence is the γ -divergence, and the robust estimator using the γ -divergence is known for having a strong robustness. In this paper, we extend the γ -divergence to the regression problem, consider the robust and sparse regression based on the γ -divergence and show that it has a strong robustness under heavy contamination even when outliers are heterogeneous. The loss function is constructed by an empirical estimate of the γ -divergence with sparse regularization, and the parameter estimate is defined as the minimizer of the loss function. To obtain the robust and sparse estimate, we propose an efficient update algorithm, which has a monotone decreasing property of the loss function. Particularly, we discuss a linear regression problem with L 1 regularization in detail. In numerical experiments and real data analyses, we see that the proposed method outperforms past robust and sparse methods. View Full-Text
Keywords: sparse; robust; divergence; MM algorithm sparse; robust; divergence; MM algorithm
Show Figures

Figure 1

MDPI and ACS Style

Kawashima, T.; Fujisawa, H. Robust and Sparse Regression via γ-Divergence. Entropy 2017, 19, 608.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop