Next Article in Journal
Exergy Analysis of Advanced Adsorption Cooling Cycles
Next Article in Special Issue
A Comparison of Variational Bounds for the Information Bottleneck Functional
Previous Article in Journal
Bayesian Edge Detector Using Deformable Directivity-Aware Sampling Window
Previous Article in Special Issue
The Conditional Entropy Bottleneck
Open AccessArticle

CEB Improves Model Robustness

Google Research, Mountain View, CA 94043, USA
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(10), 1081; https://doi.org/10.3390/e22101081
Received: 31 July 2020 / Revised: 17 September 2020 / Accepted: 21 September 2020 / Published: 25 September 2020
(This article belongs to the Special Issue Information Bottleneck: Theory and Applications in Deep Learning)
Intuitively, one way to make classifiers more robust to their input is to have them depend less sensitively on their input. The Information Bottleneck (IB) tries to learn compressed representations of input that are still predictive. Scaling up IB approaches to large scale image classification tasks has proved difficult. We demonstrate that the Conditional Entropy Bottleneck (CEB) can not only scale up to large scale image classification tasks, but can additionally improve model robustness. CEB is an easy strategy to implement and works in tandem with data augmentation procedures. We report results of a large scale adversarial robustness study on CIFAR-10, as well as the ImageNet-C Common Corruptions Benchmark, ImageNet-A, and PGD attacks. View Full-Text
Keywords: information theory; information bottleneck; machine learning information theory; information bottleneck; machine learning
Show Figures

Figure 1

MDPI and ACS Style

Fischer, I.; Alemi, A.A. CEB Improves Model Robustness. Entropy 2020, 22, 1081. https://doi.org/10.3390/e22101081

AMA Style

Fischer I, Alemi AA. CEB Improves Model Robustness. Entropy. 2020; 22(10):1081. https://doi.org/10.3390/e22101081

Chicago/Turabian Style

Fischer, Ian; Alemi, Alexander A. 2020. "CEB Improves Model Robustness" Entropy 22, no. 10: 1081. https://doi.org/10.3390/e22101081

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop