Information Theory and Large Language Models
A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".
Deadline for manuscript submissions: 15 November 2026 | Viewed by 368
Editors
Interests: machine learning; information theory; communications; language models
Interests: information and coding theory; machine learning; wireless communications; signal and image processing
Special Issues, Collections and Topics in MDPI journals
Special Issue Information
Dear Colleagues,
The rapid evolution of large language models (LLMs) demands deeper theoretical grounding to address critical challenges in scalability, interpretability, and robustness. While information theory (IT) offers powerful frameworks for analyzing data processing, learning dynamics, and system efficiency, its integration into modern AI remains underexplored, particularly as models grow increasingly complex and opaque. We recognize the urgent need for new theoretical formulations, mathematical tools, and practical metrics to guide the development of transparent, resource-efficient, and trustworthy AI systems. This Special Issue aims to bridge this gap by inviting contributions that explore IT as a principled framework to advance both the capabilities and interpretability of LLMs.
Our goal is to catalyze interdisciplinary innovation by highlighting novel IT-driven approaches to address the most pressing challenges in this rapidly evolving field. We welcome both rigorous theoretical advances and novel algorithms that pioneer new concepts.
Topics of interest include, but are not limited to, the following:
- Information-theoretic foundations of LLMs;
- Memory and retrieval in LLMs;
- Reinforcement learning and LLM reasoning;
- Generalization bounds for LLM training;
- Uncertainty/hallucination detection in LLMs;
- Multimodal LLM information fusion;
- Energy-efficient LLMs via info-theoretic optimization;
- Watermarking for LLMs;
- Compression of LLMs.
Dr. Xueyan Niu
Prof. Dr. Jun Chen
Dr. Bo Bai
Guest Editors
Manuscript Submission Information
Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.
Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.
Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.
Keywords
- large language models
- information theory
- LLM reasoning
- generalization
- memory management
- safety and hallucination detection
- watermarking
- model compression
Benefits of Publishing in a Special Issue
- Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
- Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
- Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
- External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
- Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.
Further information on MDPI's Special Issue policies can be found here.
