Special Issue "Information Theory and Machine Learning"

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: 31 December 2021.

Special Issue Editors

Prof. Dr. Lizhong Zheng
E-Mail Website
Guest Editor
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
Interests: wireless communications; space-time codes; network information theory; wireless networks
Prof. Dr. Chao Tian
E-Mail Website
Guest Editor
Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX, USA
Interests: a computational approach to information theoretic converses; coding for distributed data storage; joint source-channel coding; an approximate approach to network information theory lossy multiuser source coding problems

Special Issue Information

Dear Colleagues,

There are a number of significant steps in the development of machine learning that benefit from information theoretic analysis, as well as the insights into information processing that it brings. While we expect information theory to play an even more significant role in the next wave of growth in machine learning and artificial intelligence, we also recognize the new challenges in this task. There are indeed a set of lofty goals, where we hope to have a holistic view of data processing, to work with high-dimensional data and inaccurate statistical models, to incorporate domain knowledge, to provide performance guarantees, robustness, security, and fairness, to reduce the use of computational resources, to generate reusable and interpretable learning results, etc. Correspondingly, in theoretical studies, we shall need new formulations, new mathematical tools, new analysis techniques, and maybe even new metrics to evaluate the guidance and insights offered by theoretical studies.

The goal of this Special Issue is to collect new results in using information theoretic thinking to solve machine learning problems. We are also interested in papers presenting new methods and new concepts, even if some of these ideas might not have been fully developed, or might not have the most compelling set of supporting experimental results.

Some of the topics of interest are listed below:

  • Understanding gradient descent and general iterative algorithms;
  • Sample complexity and generalization errors;
  • Utilizing knowledge of data structure in learning;
  • Distributed learning, communication-aware learning algorithms;
  • Transfer learning;
  • Multimodal learning and information fusion;
  • Information theoretic approaches in active and reinforcement learning;
  • Representation learning and its information theoretic interpretation;
  • Method and theory for model compression.

Prof. Dr. Lizhong Zheng
Prof. Dr. Chao Tian
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (1 paper)

Order results
Result details
Select all
Export citation of selected articles as:


On Supervised Classification of Feature Vectors with Independent and Non-Identically Distributed Elements
Entropy 2021, 23(8), 1045; https://doi.org/10.3390/e23081045 - 13 Aug 2021
Viewed by 295
In this paper, we investigate the problem of classifying feature vectors with mutually independent but non-identically distributed elements that take values from a finite alphabet set. First, we show the importance of this problem. Next, we propose a classifier and derive an analytical [...] Read more.
In this paper, we investigate the problem of classifying feature vectors with mutually independent but non-identically distributed elements that take values from a finite alphabet set. First, we show the importance of this problem. Next, we propose a classifier and derive an analytical upper bound on its error probability. We show that the error probability moves to zero as the length of the feature vectors grows, even when there is only one training feature vector per label available. Thereby, we show that for this important problem at least one asymptotically optimal classifier exists. Finally, we provide numerical examples where we show that the performance of the proposed classifier outperforms conventional classification algorithms when the number of training data is small and the length of the feature vectors is sufficiently high. Full article
(This article belongs to the Special Issue Information Theory and Machine Learning)
Show Figures

Figure 1

Planned Papers

The below list represents only planned manuscripts. Some of these manuscripts have not been received by the Editorial Office yet. Papers submitted to MDPI journals are subject to peer-review.

1. Prof. Venugopal V. Veeravalli, University of Illinois at Urbana Champaign

2. Prof. Alfred O. Hero, University of Michigan

Back to TopTop