Abstract
Topic modeling has traditionally relied on textual data to uncover hidden themes within a corpus. However, with the increasing prevalence of multi-modal content in research and applications, there is a need to extend topic modeling to incorporate auxiliary modalities such as images, audio, and metadata. The paper introduces Multi-Modal Generative Topic Modeling with Cross-Attentive Networks (MM-GTM-CAN), a framework that uses a cross-attentive neural network to fuse embeddings from multiple modalities. Textual data is processed using a transformer-based encoder, while auxiliary data is encoded using modality-specific architectures. The fused embeddings are then passed through a Conditional Variational Autoencoder (CVAE) to generate contextually rich topics. We assessed topic models on a dataset of 6743 BMC Bioinformatics articles (2005–2015), in an 80:20 split, using a shared vocabulary and three random seeds. Latent Dirichlet Allocation (LDA) with five topics obtained a C_V coherence score of 0.560 ± 0.028, an NPMI score of 0.065 ± 0.013, a topic diversity score of 0.953 ± 0.023, and a held-out perplexity score of 351.7 ± 3.0. LDA with twenty topics achieved stronger coherence (C_V: 0.663) but lower diversity (0.847). Thus, the number of subjects is a trade-off between coherence and repetition. Overall, the suggested system increases topic coherence and allows for broader contextual analysis than prior modal techniques.
1. Introduction
The process of retrieving relevant information from a variety of modalities, including text and images, is known as cross-modal retrieval, or multi-modal retrieval. Cross-modal retrieval has attracted a lot of attention lately due to the notable increase in online multimedia data and the growing demand for effective information retrieval [1]. However, single-mode internal retrieval, like keyword-based retrieval and content-based retrieval, is the main focus of the most widely used search technologies on the market. Traditional text analysis techniques frequently fail to adequately analyze the richness of contextual information found in a variety of data sources in an era of rapidly expanding digital content. Though useful, text-based topic modeling approaches might not be able to take advantage of supplemental information like audio, video, or metadata, which can offer important contextual clues. Because of the intricate interactions between textual content and supplemental data like figures, tables, and citations, the increasing number of bioinformatics research papers makes it difficult to extract valuable insights. This rich contextual information is frequently missed by traditional topic modeling techniques that only use text. It has become more difficult to glean valuable insights from the enormous amount of research papers due to the explosive growth of the bioinformatics scientific literature. To address this limitation, we propose Cross-Attentive Multi-Modal Topic Modeling, a novel approach that integrates multiple data modalities for enhanced topic discovery, richer contextual analysis, and more contextualized analysis of bioinformatics research papers.
By dynamically coordinating and integrating textual content with auxiliary data using a cross-attentive mechanism, MM-GTM-CAN makes sure that pertinent cross-modal relationships are efficiently recorded. Our method improves interpretability, strengthens topic coherence [2,3], and offers a more thorough grasp of bioinformatics research trends by utilizing probabilistic topic modeling in conjunction with deep learning representations. This work is especially useful for automated research summarization, biomedical knowledge discovery, and scientific literature analysis, where it is essential to extract insights from multi-modal data. We show through extensive experiments on bioinformatics research papers that MM-GTM-CAN is a powerful tool for contemporary scientific text analysis, outperforming conventional topic modeling techniques by capturing deeper semantic structures and contextual dependencies. For this analysis, we have collected 6755 research articles from various repositories [4]. Figure 1 presents how many files are in different subfolders. The y-axis, labeled “Subfolder,” lists the names of the subfolders, which seem to adhere to a naming convention associated with “BMC Bioinformatics” and particular years; the x-axis, labeled “Number of Files,” shows the number of files in each subfolder; and the orange bars of variable length show variations in the number of files in each subfolder. According to the chart’s title, “Number of Files in Each Subfolder,” it shows how many files are kept in each directory for comparison. Variability in file distribution across years is highlighted by the fact that the subfolder “2010_text_BMC_Bioinformatics” has the most files, while “2005_text_BMC_Bioinformatics” has the fewest.
Figure 1.
BMC Bioinformatics: number of files in each subfolder.
The contribution of the article is summarized as follows:
- We propose Multi-Modal Generative Topic Modeling with Cross-Attentive Networks (MM-GTM-CAN), a framework that leverages an RNN model to develop the proposed model;
- By incorporating cross-attentive fusion, which successfully blends text and auxiliary data representations for better topic coherence, contextual understanding is improved;
- More varied and significant topic distributions are created by utilizing generative topic modeling via a Conditional Variational Autoencoder (CVAE);
- To ensure a more structured and comprehensible output, the model uses modality-aware clustering, which dynamically arranges topic embeddings according to dominant modalities.
The article is organized as follows: Section 2 provides a literature survey on cross-attentive multi-modeling by various researchers, in which they have used various methodologies to handle the problem. Section 3 briefly describes the methodology that has been utilized in this article. Section 4 gives detailed algorithmic model development of the proposed system. Section 5 covers the experimental analysis of the proposed work. Section 6 concludes the Conclusion and Future Work.
2. Related Work
The work associated with cross-attentive multi-models and recommendation systems is the main focus of this section. Pawan et al. [5] proposed a meta-learning model to address the missing modality problem in movie recommendation systems. The model reconstructs missing features using existing modalities and passes them to a novel trident architecture. Experiments were conducted on a multi-modal version of the MovieLens dataset, and a new modality was introduced by extracting the subtitles of movies to increase the effectiveness of the model. Wang et al. [6] explored the use of intelligent information technology for cross-modal learning analytics, focusing on learners’ emotional status in behavior. The research presents an intelligent analysis model for evaluating learners’ emotional status, enhancing cross-modal learning analytics and improving modern teaching quality through innovative evaluation methods. Hyeyoung et al. [7] examined research trends linking advanced technical aspects of recommendation systems in service areas to business aspects. The researchers analyzed over 135 Google Scholar articles and conference papers between 2010 and 2021, categorizing recommendation system models and technology, and analyzing application service fields. The study findings show that the growth of detailed studies on recommendation systems interacts with the business growth of the applied service field, providing insights for researchers interested in recommendation systems. Ma et al. explored [8] personality-aware recommendation systems, a new research field in artificial intelligence and personality psychology. The research systematically classifies these systems, comparing their design choices, modeling methods, and techniques; in addition, it presents commonly used datasets and highlights the challenges with these systems.
Roy and Dutta [9] reviewed recent contributions to recommendation systems, focusing on diverse applications like books, movies, and products. The review analyzes the various applications of each system, performs algorithmic analysis, and frames a taxonomy for developing an effective recommendation system. The paper also examines datasets, simulation platforms, and performance metrics to provide a comprehensive understanding of the challenges and benefits of these systems. Shou et al. [10] proposed the Low-Rank Matching Attention Method (LMAM) as a lightweight cross-modal feature fusion method for conversational emotion recognition (CER). It captures contextual emotional semantic information, reduces computational complexity, and can be embedded into existing CER methods. LMAM’s superiority and generalizability make it a promising solution.
Hankar et al. [11] used a hybrid model of Bidirectional Encoder Representations from Transformers (BERT) and Latent Dirichlet Allocation (LDA) for mining meaningful topics from massive text corpora. Mustak et al. [12] explored the potential of artificial intelligence in marketing and academic research, identifying ten key research themes and analyzing the existing literature, highlighting the field’s evolution and future directions. Garapati et al. [13] discussed the use of artificial intelligence (AI) in recommendation systems, highlighting its potential to improve prediction accuracy and address data sparsity while also identifying current research issues and new directions. Rahman et al. [14] proposed a novel Cross-Attention Multi-Modal (CAMM) deep neural network for classifying multi-modal disaster data on Twitter. The model uses the attention mechanism to highlight visual features, outperforming uni-modal models and bilinear multi-modal models. CAMM achieved an average F1-score of 84.08%. Wang et al. [15] proposed a novel method for Multi-Modal Sentiment Analysis, addressing inefficiency in modeling intra-modality and inter-modality dynamics and the inability to fuse multi-modal features. Experiments on three datasets confirm its effectiveness and superior performance compared to the current optimal method. Yubin et al. [16] proposed a Multi-Modal Recommendation Model with Cross-Modal Correction (CMC-MMR) to address issues like noise and user preferences. The model uses cross-modal correction, similarity, and self-supervised task mining. The experimental results showed that CMC-MMR improved performance by 6.202%, 4.975%, 6.054%, and 11.368% on real-world datasets. Kaur et al. [17] discussed the challenges of comparing input–output queries from diverse modal families in cross-modal information retrieval. The study compared various research work and methods, focusing on benchmark datasets. The paper also addresses open issues and suggests future research directions to better understand the current situation. Cheung and Lem [18] proposed a novel model, a Cross-Modal Bipolar Attention Network (CBAN), for multi-modal classification of social media content. The model fuses visual and textual information through direct and inverse semantic relationships, outperforming state-of-the-art methods in sentiment analysis, sarcasm detection, and hate-speech detection. Yang et al. (2022) [19] proposed a framework that improves End-to-End Multi-Modal Aspect-Based Sentiment Analysis (MABSA) by incorporating aspect and sentiment information in textual and visual representations, outperforming state-of-the-art methods on Twitter datasets. Guo et al. [20] collected a 3M dataset from TikTok and Weibo, with the aim of addressing misinformation in short videos on social media. Cross-Modal Attention Misinformation Detection (CAMD) was proposed, utilizing a cross-modal attention module to effectively exchange and fuse information between modalities. The experimental results show high accuracy, F1-score, precision, and recall. Han et al. [1] provided definitions, reviewed mainstream methods, listed datasets, and analyzed future research trends. They outlined that cross-modal retrieval technology is urgently needed to mine connections between different modal samples, such as images or videos. Deep learning methods have been applied to solve similarity measurement problems.
Min et al. [21] explored a cross-platform multimedia problem involving photo and venue recommendations between Flickr and Foursquare platforms. The study proposes a cross-platform multi-modal topic model that differentiates between platform-specific and shared topics and aligns multiple modalities for effective inter-platform recommendation. Meng et al. [22] proposed a novel framework, DB-MPCA, employing multi-modal raw data for pre-training and cross-modal attention. Multi-Modal Sentiment Analysis (MSA) used text, image, and audio data to recognize and assess human sentiments. Current methods train models based on pre-extracted features, causing a loss of critical information. Yanping et al. [23] proposed a hybrid cross-modal interaction learning framework for Multi-Modal Sentiment Analysis (MSA). The framework utilizes intra-modal, inter-modal, interactive-modal, and cross-modal interactions to enhance sentiment assistance. The experimental results show significant performance on MSA tasks and improved sentiment classification performance with cross-modal interaction structures. Guan et al. [24] proposed a DMCLR model to address data sparseness in recommendation systems by incorporating new features, cross-modal semantics, adversarial learning, and a visual interface. It outperforms other models, learns visual information and users’ ratings, and accurately captures implicit feeling tone. Shaina and Ding [25] discussed the challenges faced by news recommendation systems (NRSs) and explored solutions using deep neural networks. The research also discusses the effects of news recommendations on user behaviors and suggests potential remedies. The study provides current knowledge on news recommendation algorithms. Gupta et al. [26] used topic modeling techniques like LDA and BoW to predict future technology trends and paradigms in the field of applied intelligence, analyzing 3269 articles covering 30 years, outperforming the baseline model by 41%. Maria et al. [27] provided a systematic review analyzing recommendation systems in education, focusing on formal education. The review identifies collaborative, content-based, and hybrid approaches, with a growing trend towards machine learning; additionally, it identifies gaps in research and suggests future areas for improvement. Linwei et al. [28] proposed a cross-modal self-attention module for referring segmentation in images and videos with natural language. It captures long-range dependencies between linguistic and visual features, selectively integrating self-attentive features, and integrating temporal information in consecutive frames.
Camila et al. [29] analyzed topic modeling in software engineering research, focusing on four aspects: topic models, inputs, pre-processing, and topic naming. They found that LDA and LDA-based techniques are the most common, developer communication and bug reports are most modeled, and data pre-processing and modeling parameters vary. Lee et al. [30] used a machine learning approach to analyze advanced DT technology research, identifying six dominant topics and providing intelligent bibliometric analysis. Digital transformation (DT) combines digital technologies with business models to generate value. Zhou et al. [31] present ZCCR, a Zero-shot Content-based Cross-Modal Recommendation System that uses large-scale pre-trained vision-language models to perform cross-modal similarity search on an optimized index. The system recommends user assets similar to their user profile, using agglomerative clustering to identify areas of interest. The experimental results show that ZCCR outperforms baseline tagging systems and advanced tag systems, demonstrating the efficiency of these systems.
3. Methodology
To capitalize on the advantages of architectures, an RNN model is used, which is effective at handling sequential data and learning temporal dependencies.
Figure 2 depicts the Cross-Attentive Multi-Modal Topic Modeling framework that integrates text and auxiliary data (metadata) to analyze biomedical research articles. By combining textual and auxiliary data, a Cross-Attentive Multi-Modal Topic Modeling framework is proposed for biomedical research articles analysis. The process starts with data collection and processing, followed by modality-specific encoding, where RNNs capture temporal patterns from non-image data. Next, the cross-attentive fusion module aligns and integrates textual and auxiliary embeddings to form a unified multi-modal representation. This fused data is then input into a generative topic modeling process using a Conditional Variational Autoencoder (CVAE) to generate topic distributions. To ensure semantic consistency, a cross-modal coherence assessment mechanism evaluates and aligns relationships across modalities. Finally, modality-aware clustering dynamically groups topic embeddings based on the dominant modality, generating final topics with enhanced multi-modal coherence and contextual depth. This approach improves topic modeling accuracy and interpretability for bioinformatics research.
Figure 2.
Overall architecture of proposed system.
3.1. Dataset Collection
Accessing publicly accessible repositories and databases like PubMed, Elsevier, or IEEE, which contain an extensive collection of articles, abstracts, and supplemental materials, is usually necessary for gathering data for biomedical research articles. Structured and unstructured data can be retrieved via direct dataset downloads, web-scraping tools, or APIs. To facilitate multi-modal topic modeling and analysis, the gathered data is subsequently pre-processed and formatted.
Figure 3 shows the character length of various files. The y-axis is labeled “File Length,” which displays the number of characters in each file, and the x-axis is labeled “File Index,” which indicates the index of each file. The chart implies that the purpose of the visualization is to compare the character counts of various files. There is a noticeable difference in file sizes, with the tallest bar corresponding to file index 3 and the shortest corresponding to file index 1.
Figure 3.
Length of file represented in characters.
Figure 4 shows how many words are in various files. The y-axis, “Word Count,” shows the quantity of words in each file, and the x-axis, “File Index,” shows the index of each file. From outlining the word count per file, the chart’s purpose is to compare the distribution of word counts across several files. The difference in file lengths according to word count is evident in the fact that file index 3 has the most words, closely followed by file index 4, and file index 1 has the fewest words.
Figure 4.
Word count per file.
3.2. Data Pre-Processing
In order to prepare data for analysis, biomedical research articles must first be cleaned, formatted, and transformed. For textual content, this entails tokenization, the removal of special characters, punctuation, and stop words, as well as the use of stemming or lemmatization to normalize words. Text annotation is done using domain-specific vocabularies. Resizing, normalization, and noise reduction are used to convert the data into standard formats. Standardized metadata is encoded into categorical or numerical formats, and its consistency is checked. Effective downstream multi-modal analysis is made possible by the final pre-processed data from all modalities being aligned and represented in a single format.
4. Model Structure
4.1. Model-Specific Encoding—MM-GTM-CAN
Algorithm 1 efficiently captures the sequential patterns and dependencies in the text by using an RNN for topic modeling in biomedical research articles. The first step in the process is pre-processing the corpus, which involves tokenizing each article into word index sequences and representing them in a padded format for consistency. These indices are converted to dense vector representations by a word embedding layer, which produces a matrix of embeddings for every article. The RNN, which is the central component of the algorithm, sequentially processes these embeddings to calculate hidden states at each time step while obtaining contextual information about the words. The content of the article is summarized by combining these hidden states into a compact document representation.
A fully connected layer is then applied to the document representation in order to use a softmax function to predict the distribution across topics. This distribution shows how pertinent each topic is to the text. By comparing the predicted topic distribution with the observed word distribution, the algorithm minimizes a reconstruction loss, which guarantees the meaningfulness of the topic–word and document–topic relationships. Using gradient descent, model parameters such as the embedding layer, RNN weights, and topic projection matrix are updated by computing gradients of the loss function.
Following training, each topic can be interpreted as a cohesive collection of biomedical terms due to the learned topic–word matrix, which shows the relationships between topics and words. In biomedical research, document representations can be utilized for subsequent tasks like trend analysis or clustering. This method captures subtle linguistic and thematic patterns and is especially useful for modeling the structured and context-rich nature of biomedical texts.
The model for multi-modal topic modeling in bioinformatics research papers produced the topic–word distribution shown in the table below. A topic is represented by each row, and a highly likely word related to that topic is represented by each column. Words like “bioinformatics,” “gene,” “protein,” “genome,” “DNA,” and “bacteria” are presented in Figure 5, demonstrating that the model has effectively captured the field of bioinformatics.
Figure 5.
Distribution of topics vs. words.
Figure 6 presents the top words in each topic according to a word importance score. Among the terms listed vertically on the left are “gene annotation,” “rna,” “bacteria,” “protein,” “genome,” “gene,” “sequence,” “bmc,” “bioinformatics,” and “dna,” all of which are aligned with the bars that correspond to their respective importance scores. To visually represent each topic’s contribution to the word’s overall importance, each bar is divided into colored sections that correspond to various topics (Topic 1 through Topic 5). From 0 to 7000, the “Word Importance Score” is displayed on the bottom x-axis.
Figure 6.
Top words in each topic.
4.2. Cross-Attentive Fusion
Cross-attentive fusion is a mechanism that has been developed especially to combine textual information from biomedical research articles (such as abstracts and full texts) with auxiliary data (metadata). In order to produce a cohesive representation and provide richer contextual analysis and deeper insights into the relationships between modalities, the fusion process makes sure that pertinent features from various modalities are aligned and combined.
- Input Depiction
- Text Data:
Pre-trained language models, such as BioBERT, SciBERT, or transformers optimized on biomedical datasets, are used to encode textual data from research articles, such as abstracts, titles, or full-text sections, into dense representations. The text’s semantic context is captured by these embeddings.
where Dt is the dimensionality of text features; n is the number of samples; and T stands for text embeddings.
T = TextEncoder (Dt), T ε Rn×dt
- 2.
- Attention Mechanism
The cross-attention mechanism, which discovers the connection between textual and auxiliary features, is the central component of cross-attentive fusion. It guarantees that the textual representation is complemented only by the most pertinent information from auxiliary data.
- Query
Derived from the text embeddings (T), since text is usually the main source of information in biomedical articles.
Q = WqT
- Key and Value
Key and value are derived from auxiliary data embeddings,
K = WkA, V= WvA
- Attention Score
Using scaled dot-product attention, the similarity between text (Q) and auxiliary data (K) is calculated:
- Weighted Values
The value vectors (V) are weighted using the attention scores:
R = Attention (Q, K). V
4.3. Generative Topic Modeling
A key component of deriving significant topic distributions from multi-modal data is generative topic modeling. The model creates latent topic representations from text and auxiliary embeddings using a Variational Autoencoder (VAE), discussed in Algorithm 1, and a Conditional Variational Autoencoder, described in Algorithm 2. Following cross-attentive fusion, which blends various modalities (such as text or metadata), the generative topic model learns a structured latent space in which topic distributions are established according to contextual and semantic relationships between modalities. The topics that are extracted using this method are guaranteed to be more logical, contextually rich, and consistent with the underlying data structure. Additionally, the model’s generative nature aids in uncovering hidden topic patterns, facilitating more efficient classification and clustering in multi-modal bioinformatics research analysis.
| Algorithm 1: Variational Autoencoder |
| Input: X: Corpus of biomedical research articles V: Size of vocabulary dlat: Latent topic dimension η: Learning rate ηepochs: Number of training epochs |
| Output: Z: The articles’ latent topic representations X′: Reconstructed word distributions |
|
|
|
|
|
|
|
|
| Algorithm 2: Conditional Variational Autoencoder |
| Input: , and training epochs E. Output: . For each epoch e = 1,…,E and each article minibatch B, perform Steps 2–7 for its articles. Construct the conditioning vector: ci ← mi. Estimate the posterior parameters: Sample the latent representation: Compute the topic proportions: Reconstruct the word probabilities: Compute the minibatch loss and update the encoder and decoder: |
The algorithm models latent topics in biomedical research articles by utilizing a Variational Autoencoder (VAE). The articles are first pre-processed into a representation of a bag of words. Each document is compressed into a latent representation z by the encoder, which has a Gaussian distribution with a mean of μ and log-variance of log σ2. To guarantee differentiability for gradient-based optimization, the latent variable z is sampled using the reparameterization trick. The decoder creates a probability distribution across the vocabulary by reconstructing the document from z. A reconstruction loss, which guarantees that the output matches the input, and a KL divergence loss, which aligns the latent space with a standard Gaussian distribution to promote generalization, make up the composite loss function that the algorithm minimizes. Following training, the latent variables stand in for interpretable subjects like “genomics” or “cancer research,” and the decoder weights show how words and topics relate to one another. This method offers a reliable means of examining research trends, extracting relevant topics, and analyzing biomedical texts.
The heatmap presented in Figure 7 is a 5 × 5 grid with both rows and columns representing the topics (Topic 1 through Topic 5). The coherence scores’ numerical range and matching color gradient are displayed in a colorbar on the right, labeled 0.3 to 1.0. The diagonal cells are dark red and have perfect coherence scores of 1.0, signifying the coherence of each topic with itself. Different colors and numerical values within each of the remaining cells indicate differing levels of coherence.
Figure 7.
Topic coherence heatmap.
Figure 8 presents two sets of data related to topic modeling. The top section shows the top 10 most frequent words for each of the five topics (Topic 1 through Topic 5), with each word listed alongside its topic number. The bottom section, labeled “Cluster,” shows a list of the same top 10 words but categorized into three distinct clusters (0, 1, and 2), again with each word associated with its corresponding cluster number. Both sections use a tabular format to display the information clearly.
Figure 8.
Clustering of topics.
5. Experimental Analysis
5.1. Dataset and Pre-Processing
We used 6755 full-text papers from the journal BMC Bioinformatics published between 2005 and 2015 (Mendeley Data, DOI: 10.17632/3zvv8b52pj.1). Each article was lower-cased and tokenized into alphabetic words of length at least three. We eliminated English stop words, journal boilerplate terms (“BMC”, “correspondence”, “copyright”, and “licensee”), and section headers (“background”, “results”, “methods”, and “conclusions”). We filtered out articles with fewer than 50 tokens remaining, resulting in 6743 documents. The dictionary was constructed on the training split, and any terms that appeared in fewer than 10 documents or in more than 50% of documents were deleted, which resulted in a vocabulary of 24,588 terms.
5.2. Data Division and Assessment Protocol
Documents were mixed using a fixed random seed (42) and split 80:20 into 5394 training and 1349 test documents. We trained each model with the same split, the same vocabulary, and the same number of topics, K. The quality of the topics was judged on the top 10 words of each topic:
- Coherence: C_V and NPMI scores, calculated using Gensim (version 4.4)’s CoherenceModel on the training papers.
- Topic Diversity: Number of unique words in all topics divided by K × 10.
- Perplexity: Held-out perplexity per word on the test split. For LDA, this is calculated from the held-out likelihood of the model. For the conditional VAE, it is computed from the evidence lower bound (ELBO). Thus, the two numbers are not directly comparable.
We report K = 5 and K = 20. LDA and CVAE were run using seeds 42, 43, and 44; NMF is deterministic and was run once.
5.3. Models
LDA (Gensim (version 4.4) LdaModel): A total of 10 passes, chunk size 2000, document–topic and topic–word priors computed automatically.
NMF (scikit-learn): Non-negative matrix factorization of the TF-IDF matrix initialized with NNDSVD and run for a maximum of 400 iterations.
MM-GTM-CAN: A variational autoencoder that uses as its input the word counts of the document and its publication year, encoded as a one-hot vector. The decoder takes the latent variable and the year as inputs and outputs a distribution across the vocabulary. The encoder consists of two hidden layers of 256 units. Training was done with the Adam optimizer, a learning rate of 0.001, a batch size of 64, and a KL weight of 1. The primary runs were trained over 10 epochs. One more run was done using 100 epochs (as reported in the original manuscript). The words in each topic are ranked by decoder weights + bias for that topic.
5.4. Results
Table 1 reports the mean and standard deviation over three seeds.
Table 1.
Comparative Performance of Topic Modelling Methods.
Table 1 and Figure 9 compare LDA, NMF, and MM-GTM-CAN by topic coherence C_V, NPMI, and subject diversity. The results show that MM-GTM-CAN with 100 epochs and K = 5 obtains the highest coherence of 0.846, but its diversity score decreases to 0.580, which means that more topic terms overlap. The maximum diversity score of MM-GTM-CAN is 0.993 with 10 epochs and K = 5. However, the negative NPMI score (−0.164) demonstrates that the selected topic words have a bad co-occurrence link. NMF with K = 20 is a good compromise among all criteria, with a coherence score of 0.765, the highest NPMI score of 0.179, and a diversity score of 0.915. Increasing K from 5 to 20 leads to an increase in coherence and NPMI scores for both LDA and NMF, with a slight drop in the diversity score. Overall, the results suggest that there are trade-offs between semantic coherence and topic distinctiveness rather than one model being uniformly superior across all criteria.
Figure 9.
Topic model comparison for the proposed models.
6. Conclusions and Future Work
The results show that the quality of the subjects depends on the model, the number of topics, and the training time. From the presented data, MM-GTM-CAN resulted in the highest C_V coherence score of 0.846 with five topics after 100 epochs, but its lower diversity score of 0.580 implies significant overlap among topic–word lists. The 10-epoch setting with five subjects achieved the maximum diversity score of 0.993, but its negative NPMI score indicates that the words have poor co-occurrence relationships. The best balance was achieved by NMF with twenty themes, with a coherence score of 0.765, the greatest NPMI score of 0.179, and a diversity score of 0.915. The proposed model demonstrates promising results in terms of coherence and diversity under different configurations, but does not always exceed the baselines on all measures. Future studies should aim at improving subject distinctiveness while keeping semantic consistency, and investigate the stability of prolonged training over repeated runs. Future work can also explore greater advancements in multi-modal learning by incorporating more attention processes and making the topic coherence assessment more fine-grained. Such improvements could enhance both interpretability and performance.
Author Contributions
H.S. designed the algorithm, performed the simulation results and drafted the manuscript under the supervision of M.G.S. and S.S. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors would like to acknowledge the anonymous reviewers for their thoughtful comments.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Han, Z.; Azman, A.B.; Mustaffa, M.R.B.; Khalid, F.B. Cross-modal retrieval: A review of methodologies, datasets, and future perspectives. IEEE Access 2024, 12, 115716–115741. [Google Scholar] [CrossRef] [Scilit]
- Mahalakshmi, G.S.; Hemadharsana, S.; Srividhyasaradha, K.; Sendhilkumar, S.; Sushant, C. Exploring Correlation of Deep Topic Models Using Structured Topic Coherence. In Proceedings of Data Analytics and Management; Khanna, A., Polkowski, Z., Castillo, O., Eds.; Lecture Notes in Networks and Systems; Springer: Singapore, 2023; Volume 572. [Google Scholar] [CrossRef] [Scilit]
- Mahalakshmi, G.S.; Hemadharsana, S.; Muthuselvi, G.; Sendhilkumar, S. Learning Deep Topics of Interest. In New Trends in Computational Vision and Bio-Inspired Computing; Smys, S., Iliyasu, A.M., Bestak, R., Shi, F., Eds.; Springer: Cham, Switzerland, 2020. [Google Scholar] [CrossRef] [Scilit]
- Hemadharsana, S. BMC Bioinformatics 2005-2015 Dataset. Mendeley Data V1 2025. [Google Scholar] [CrossRef]
- Agrawal, P.; Raj, S.; Saha, S.; Onoe, N. A meta-learning based generative model with graph attention network for multi-modal recommender systems. Procedia Comput. Sci. 2023, 222, 581–590. [Google Scholar] [CrossRef] [Scilit]
- Wang, L.; Peng, N.; Liu, L.; Wei, S. Innovation of Multimodal Learning Paths Based on Learning Behavior and Sentiment Analysis in AI Digital Intelligence Platform. Procedia Comput. Sci. 2025, 261, 566–573. [Google Scholar] [CrossRef] [Scilit]
- Ko, H.; Lee, S.; Park, Y.; Choi, A. A survey of recommendation systems: Recommendation models, techniques, and application fields. Electronics 2022, 11, 141. [Google Scholar] [CrossRef] [Scilit]
- Ma, C.; Xu, Z.; Ren, Y.; Hettiachchi, D.; Chan, J. PUB: An LLM-enhanced personality-driven user behaviour simulator for recommender system evaluation. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, Padua, Italy, 13–18 July 2025; pp. 2690–2694. [Google Scholar]
- Roy, D.; Dutta, M. A systematic review and research perspective on recommender systems. J. Big Data 2022, 9, 59. [Google Scholar] [CrossRef] [Scilit]
- Shou, Y.; Meng, T.; Ai, W.; Fu, F.; Yin, N.; Li, K. A comprehensive survey on multi-modal conversational emotion recognition with deep learning. ACM Trans. Inf. Syst. 2026, 44, 10. [Google Scholar] [CrossRef] [Scilit]
- Hankar, M.; Kasri, M.; Beni-Hssane, A. A comprehensive overview of topic modeling: Techniques, applications and challenges. Neurocomputing 2025, 628, 129638. [Google Scholar] [CrossRef] [Scilit]
- Mustak, M.; Salminen, J.; Plé, L.; Wirtz, J. Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda. J. Bus. Res. 2021, 124, 389–404. [Google Scholar] [CrossRef] [Scilit]
- Garapati, R.; Chakraborty, M. Recommender systems in the digital age: A comprehensive review of methods, challenges, and applications. Knowl. Inf. Syst. 2025, 67, 6367–6411. [Google Scholar] [CrossRef] [Scilit]
- Rahman, K.R.; Khandakar, A.; Hossen, M.F.; Rabbani, S.G.; Saha, A.; Ahamed, M.F.; Hossain, M.M.; Ayari, M.A.; Al-Esmail, R.A.R. ReliefNet: A novel and reliable multimodal framework for real-time disaster severity classification in humanitarian operations. Prog. Disaster Sci. 2026, 29, 100528. [Google Scholar]
- Wang, P.; Liu, S.; Chen, J. CCDA: A novel method to explore the cross-correlation in dual-attention for multimodal sentiment analysis. Appl. Sci. 2024, 14, 1934. [Google Scholar] [CrossRef] [Scilit]
- Wang, Y.; Xia, H.; Liu, Y. CMC-MMR: Multi-modal recommendation model with cross-modal correction. J. Intell. Inf. Syst. 2024, 62, 1187–1211. [Google Scholar] [CrossRef] [Scilit]
- Kaur, P.; Pannu, H.S.; Malhi, A.K. Comparative analysis on cross-modal information retrieval: A review. Comput. Sci. Rev. 2021, 39, 100336. [Google Scholar] [CrossRef] [Scilit]
- Cheung, T.H.; Lam, K.M. Crossmodal bipolar attention for multimodal classification on social media. Neurocomputing 2022, 514, 1–12. [Google Scholar] [CrossRef] [Scilit]
- Yang, L.; Na, J.-C.; Yu, J. Cross-Modal Multitask Transformer for End-to-End Multimodal Aspect-Based Sentiment Analysis. Inf. Process. Manag. 2022, 59, 103038. [Google Scholar] [CrossRef] [Scilit]
- Guo, Z.; Li, Y.; Yang, Z.; Li, X.; Lee, L.K.; Li, Q.; Liu, W. Cross-modal attention network for detecting multimodal misinformation from multiple platforms. IEEE Trans. Comput. Soc. Syst. 2024, 11, 4920–4933. [Google Scholar] [CrossRef] [Scilit]
- Min, W.; Bao, B.K.; Xu, C.; Hossain, M.S. Cross-platform multi-modal topic modeling for personalized inter-platform recommendation. IEEE Trans. Multimed. 2015, 17, 1787–1801. [Google Scholar] [CrossRef] [Scilit]
- Li, M.; Zhu, Z.; Li, K.; Pei, H. Diversity and balance: Multimodal sentiment analysis using multimodal-prefixed and cross-modal attention. IEEE Trans. Affect. Comput. 2024, 16, 250–263. [Google Scholar] [CrossRef] [Scilit]
- Fu, Y.; Zhang, Z.; Yang, R.; Yao, C. Hybrid cross-modal interaction learning for multimodal sentiment analysis. Neurocomputing 2024, 571, 127201. [Google Scholar] [CrossRef] [Scilit]
- Guan, Q.; Meng, S.; Zhang, J.; Li, Q. DMCLR: Dual-Denoising Multimodal Contrastive Learning for Robust Recommendation. IEEE Trans. Big Data 2026, 12, 1965–1976. [Google Scholar] [CrossRef] [Scilit]
- Raza, S.; Ding, C. News recommender system: A review of recent progress, challenges, and opportunities. Artif. Intell. Rev. 2022, 55, 749–800. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Gupta, R.K.; Agarwalla, R.; Naik, B.H.; Evuri, J.R.; Thapa, A.; Singh, T.D. Prediction of research trends using LDA based topic modeling. Glob. Transit. Proc. 2022, 3, 298–304. [Google Scholar] [CrossRef] [Scilit]
- Urdaneta-Ponte, M.C.; Mendez-Zorrilla, A.; Oleagordia-Ruiz, I. Recommendation systems for education: Systematic review. Electronics 2021, 10, 1611. [Google Scholar] [CrossRef] [Scilit]
- Ye, L.; Rochan, M.; Liu, Z.; Zhang, X.; Wang, Y. Referring segmentation in images and videos with cross-modal self-attention network. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 3719–3732. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Silva, C.C.; Galster, M.; Gilson, F. Topic modeling in software engineering research. Empir. Softw. Eng. 2021, 26, 120. [Google Scholar] [CrossRef] [Scilit]
- Lee, C.H.; Liu, C.L.; Trappey, A.J.; Mo, J.P.; Desouza, K.C. Understanding digital transformation in advanced manufacturing and engineering: A bibliometric analysis, topic modeling and research trend discovery. Adv. Eng. Inform. 2021, 50, 101428. [Google Scholar] [CrossRef] [Scilit]
- Zhou, S.; Gan, W.; Liu, Q.; Lei, K.; Zhu, J.; Huang, H.; Xia, Y.; Tang, R.; Dong, Z.; Zhao, Z. Recbase: Generative foundation model pretraining for zero-shot recommendation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China, 4–9 November 2025; pp. 15598–15610. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.








