Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.2
2.0
Journals
Entropy
Special Issues
Information Theoretic Learning with Its Applications
entropy-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information Theoretic Learning with Its Applications

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: 30 September 2025 | Viewed by 2380

Special Issue Editors

Dr. Isidoros Perikos

E-Mail Website
Guest Editor
Department of Computer Engineering & Informatics, University of Patras, 26504 Rio, Greece
Interests: artificial intelligence; learning technologies; machine learning; human–computer interaction; social media; affective computing; sentiment analysis;
Special Issues, Collections and Topics in MDPI journals
Dr. Christos Makris

E-Mail Website
Guest Editor
Department of Computer Engineering and Informatics, University of Patras, 26504 Rio Achaia, Greece
Interests: data structures; information retrieval; data mining; bioinformatics; string algorithmic; computational geometry; multimedia databases; internet technologies
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Vasileios Megalooikonomou

E-Mail Website
Guest Editor
Computer Engineering & Informatics Department, University of Patras, Panepistimioupoli Patron, 265 04 Patra, Greece
Interests: database and knowledge-based systems; intelligent information systems; data mining; pattern recognition; data compression; biomedical informatics; multimedia
Special Issues, Collections and Topics in MDPI journals
Dr. Sotiris Kotsiantis

E-Mail Website
Guest Editor
Department of Mathematics, University of Patras, 265-00 Patras, Greece
Interests: artificial intelligence; machine learning; data mining; knowledge discovery; data science
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

In the rapidly evolving field of data science, Information Theoretic Learning (ITL) emerges as a cornerstone for uncovering complex patterns in data through the lens of information theory. This Special Issue of Entropy, titled “Information Theoretic Learning with Its Applications”, aims to explore the frontier of ITL and its transformative applications across various disciplines. We seek contributions that push the boundaries of understanding, applying, and innovating with ITL to solve real-world problems. Topics of interest include but are not limited to information theory, entropy-based algorithms, mutual information in supervised and unsupervised learning, information bottleneck methods, and applications of ITL in various domains. Through this Special Issue, we invite researchers and practitioners to share their findings, methodologies, and insights, contributing to a comprehensive discourse on how information theory continues to shape the landscape of data analysis and learning.

Dr. Isidoros Perikos
Dr. Christos Makris
Prof. Dr. Vasileios Megalooikonomou
Dr. Sotiris Kotsiantis
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • information theory
  • data science
  • supervised learning
  • unsupervised learning
  • data analysis techniques
  • machine learning applications

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (2 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

26 pages, 543 KiB  
Article
Bounds on the Excess Minimum Risk via Generalized Information Divergence Measures
by Ananya Omanwar, Fady Alajaji and Tamás Linder
Entropy 2025, 27(7), 727; https://doi.org/10.3390/e27070727 - 5 Jul 2025
Viewed by 252
Abstract
Given finite-dimensional random vectors Y, X, and Z that form a Markov chain in that order (Y→X→Z), we derive the upper bounds on the excess minimum risk using generalized information divergence measures. Here, Y is [...] Read more.
Given finite-dimensional random vectors Y, X, and Z that form a Markov chain in that order (Y→X→Z), we derive the upper bounds on the excess minimum risk using generalized information divergence measures. Here, Y is a target vector to be estimated from an observed feature vector X or its stochastically degraded version Z. The excess minimum risk is defined as the difference between the minimum expected loss in estimating Y from X and from Z. We present a family of bounds that generalize a prior bound based on mutual information, using the Rényi and α-Jensen–Shannon divergences, as well as Sibson’s mutual information. Our bounds are similar to recently developed bounds for the generalization error of learning algorithms. However, unlike these works, our bounds do not require the sub-Gaussian parameter to be constant, and therefore, apply to a broader class of joint distributions over Y, X, and Z. We also provide numerical examples under both constant and non-constant sub-Gaussianity assumptions, illustrating that our generalized divergence-based bounds can be tighter than the ones based on mutual information for certain regimes of the parameter α. Full article
(This article belongs to the Special Issue Information Theoretic Learning with Its Applications)
►▼ Show Figures

Figure 1

38 pages, 1053 KiB  
Article
Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis
by Sharu Theresa Jose and Shana Moothedath
Entropy 2024, 26(7), 606; https://doi.org/10.3390/e26070606 - 17 Jul 2024
Cited by 2 | Viewed by 1511
Abstract
We study stochastic linear contextual bandits (CB) where the agent observes a noisy version of the true context through a noise channel with unknown channel parameters. Our objective is to design an action policy that can “approximate” that of a Bayesian oracle that [...] Read more.
We study stochastic linear contextual bandits (CB) where the agent observes a noisy version of the true context through a noise channel with unknown channel parameters. Our objective is to design an action policy that can “approximate” that of a Bayesian oracle that has access to the reward model and the noise channel parameter. We introduce a modified Thompson sampling algorithm and analyze its Bayesian cumulative regret with respect to the oracle action policy via information-theoretic tools. For Gaussian bandits with Gaussian context noise, our information-theoretic analysis shows that under certain conditions on the prior variance, the Bayesian cumulative regret scales as O˜(mT), where m is the dimension of the feature vector and T is the time horizon. We also consider the problem setting where the agent observes the true context with some delay after receiving the reward, and show that delayed true contexts lead to lower regret. Finally, we empirically demonstrate the performance of the proposed algorithms against baselines. Full article
(This article belongs to the Special Issue Information Theoretic Learning with Its Applications)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-2
Back to TopTop