Evaluation of Machine Learning Techniques by Entropic Means

Dear Colleagues,

Task-based conceptualization and assessment are two of the main steps taken to arrive at the present machine learning practice status quo. While there are ongoing debates on whether techniques such as the information bottleneck and other information plane measures can help improve the understanding and evaluation of, e.g., deep learning techniques, information theory is ubiquitous in:

• Providing mathematical frameworks to solve tasks, e.g., information geometry;
• Exploratory analysis and the diagnosis of task data, e.g., feature selection, source entropy triangles, and entropic diagrams;
• Providing heuristic criteria to find the solutions of the task, e.g., max-ent, min-cross-ent, max-MI, AIC, and further developments thereof, e.g., the information bottleneck and the cross-entropy penalty for classification;
• The assessment of solutions and explaining the results of techniques, e.g., information bottleneck analysis, information plane visualization, and entropy triangles.

Previous Special Issues and papers in Entropy have dealt with more generic approaches, e.g.:

• https://www.mdpi.com/journal/entropy/special_issues/Entropy_Data;
• https://www.mdpi.com/journal/entropy/special_issues/modern_machine_learning;
• https://www.mdpi.com/1099-4300/22/10/1100;
• https://www.mdpi.com/1099-4300/22/2/151.

We invite specific contributions designed to advance the assessment and explanation of the effects of techniques involved in any and all of the stages in the machine learning pipeline (classifier or regressor estimation, representation transformation, including deep transformations and embeddings, feature selection, or any other form of data preparation) using entropic measures (entropy, cross-entropy, mutual information, divergences, etc.) in any of their forms.

Examples of entropy triangles from https://www.mdpi.com/1099-4300/20/7/498.

Dr. Francisco J. Valverde-Albacete
Prof. Carmen Peláez-Moreno
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.