The Information Bottleneck: Foundations, Algorithms, and Modern Applications

Dear Colleagues,

The Information Bottleneck (IB) formulates representation learning as a trade-off between relevance and compression. Twenty-five years after its introduction, IB informs modern information theory, statistical learning, and large-scale AI. This Special Issue invites original research and authoritative surveys that (i) advance the mathematical foundations of IB; (ii) develop scalable objectives, estimators, and algorithms suited to contemporary data and hardware; (iii) demonstrate measurable benefits of IB in real systems. Submissions should articulate explicit information-theoretic objectives and provide reproducible analytical and/or empirical evidence. Interdisciplinary work connecting IB with causal inference, control, coding theory, or statistical physics is encouraged. We also welcome IB perspectives on large language models and agentic AI (e.g., rate–distortion-guided KV cache compression, information-constrained policies, IB-grounded evaluation and reasoning).

Key topics include (but are not limited to) the following:

• Theory: Rate–distortion and sufficiency; generalization/sample complexity; information geometry and optimal transport; multivariate/multimodal/distributed IB; semantic (concept) bottlenecks; thermodynamic/nonequilibrium analyses.
• Algorithms: IB objectives with mutual information estimation (variational, contrastive, density ratio); deterministic and stochastic encoders; transformer-specific methods (KV cache compression, attention sparsification); scalable/streaming/resource-constrained settings; federated and differentially private IB under communication limits.
• Applications: LLMs/Transformers (representation and KV cache compression; information-constrained policies/evaluation); vision, speech, and multimodal learning; reinforcement learning and control; neuroscience; genomics; signal processing and communications; privacy, fairness, and security.

Dr. Behrooz Razeghi
Prof. Dr. Slava Voloshynovskiy
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.