The Information Bottleneck in Deep Learning

The Information Bottleneck is a principle to trade off complexity and fidelity in statistical modeling and inference. It was introduced in the 1990s and has been applied to different domains such as clustering and system identification. Most recently, it has shown to play a role in the analysis of deep neural networks. This Special Issue focuses on the role of the Information Bottleneck and related principles in the analysis and design of representation learning and optimization algorithms for training deep neural networks. For instance, connections have been established between the Information Bottleneck and Bayesian Inference, PAC-Bayes Theory, Kolmogorov Complexity, and Minimum-Description Length—all with different algorithmic instantiation. Contributions are solicited that explore both the modeling aspect, the optimization aspect, and the empirical analysis aspect of deep learning using tools from Information Theory and Statistical Theory. Application papers are also welcome that explore the use of the Information Bottleneck as a regularization method for training deep learning models. Explorations that explore connections to biological systems are also encouraged. Manuscripts will be peer-reviewed, and published work will be available through open access. Expansion of manuscripts published at conferences are welcome, so long as they include meaningful expansion of the scope of these papers. Simultaneous submission of a short conference version and the longer journal version is acceptable, so long as it is notified to the editors and a copy of the conference submission is enclosed with the journal submission.

Prof. Dr. Naftali Tishby
Prof. Dr. Stefano Soatto
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.