Rethinking Representation Learning in the Age of Large Models
A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Multidisciplinary Applications".
Deadline for manuscript submissions: 31 October 2025 | Viewed by 153
Special Issue Editors
Interests: causal representation learning; causal LLMs; latent variable models
Interests: machine learning algorithms; generative AI; efficient learning algorithms
Interests: image processing; image fusion; machine learning; computer vision
Special Issues, Collections and Topics in MDPI journals
Special Issue Information
Dear Colleagues,
In recent years, with groundbreaking advancements in artificial intelligence, we have entered the era of large models. Over time, various foundational models have emerged, including, but not limited to, large language models (LLMs), multi-modal models (e.g., vision–language models), and generative AI. In this context, it is imperative to rethink the role of representation learning to ensure that these models are responsible, e.g., not only efficient but also robust, safe, and interpretable.
This rethinking should be grounded in key theories and principles, including but not limited to causality and information theory, which provide a solid framework for constructing models that are not only powerful but also controllable and understandable. In particular, principles from information theory, such as maximizing entropy or preserving mutual information, play a vital role in learning rich and robust representations. For example, contrastive learning can be viewed as implicitly maximizing entropy within the representation space, encouraging models to retain diverse and informative features from data. Meanwhile, the information bottleneck principle offers a complementary perspective, aiming to compress representations by minimizing irrelevant information while preserving task-relevant signals.
By leveraging insights from these foundational theories and principles, we can advance large language models and unlock the potential of large models, enhancing their effectiveness across a wide range of downstream tasks—from low- and high-level vision applications to synthetic data generation and creation.
This Special Issue will focus on novel ideas and methodologies in representation learning for large models, with a particular emphasis on approaches driven by information-theoretic principles—for example, entropy maximization. However, we also welcome contributions grounded in related perspectives, such as causality or other principled theoretical frameworks. We invite original, unpublished research papers and systematic reviews covering, but not limited to, the following research areas:
- Representation learning methods and applications for large language models, multi-modal models, and generative models;
- Integration of causality and chain-of-thought with large models to enhance their reasoning ability;
- Information-theoretic approaches to large model compression, generalization, and security;
- Representation space analysis and controllability in generative models;
- Applications of representation learning in synthetic data generation, visual perception, and related tasks;
- Generalization of representations across multi-scale and multi-task scenarios;
- Bridging representation learning with complex systems theory, including chaos theory and entropy measures;
- Efficient and interpretable training and inference algorithms for large models.
Dr. Yuhang Liu
Dr. Xinyu Zhang
Prof. Dr. Qingsen Yan
Guest Editors
Manuscript Submission Information
Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.
Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.
Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.
Keywords
- large language models
- multi-modal models
- generative AI
- representation learning
- causality
- entropy
- visual perception
- efficient AI training
- interpretable AI
Benefits of Publishing in a Special Issue
- Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
- Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
- Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
- External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
- Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.
Further information on MDPI's Special Issue policies can be found here.
