Comparative Study of Filtering Methods for Scientific Research Article Recommendations

Abstract

Given the daily influx of scientific publications, researchers often face challenges in identifying relevant content amid the vast volume of available information, typically resorting to conventional methods like keyword searches or manual browsing. Utilizing a dataset comprising 1895 users and 3122 articles from the CI&T Deskdrop collection, as well as 7947 users and 25,975 articles from CiteULike-t, we examine the effectiveness of collaborative filtering and content-based and hybrid recommendation approaches in scientific literature recommendations. These methods automatically generate article suggestions by analyzing user preferences and historical behavior. Our findings, evaluated based on accuracy (Precision@K), ranking quality (NDCG@K), and novelty, reveal that the hybrid approach significantly outperforms other methods, tackling some challenges such as cold starts and sparsity problems. This research offers theoretical insights into recommendation model effectiveness and practical implications for developing tools that enhance content discovery and researcher productivity.

1. Introduction

In today’s digital age, which began in the late 20th century, the volume of scientific literature published online is growing at an unprecedented rate. Between 2000 and 2020, global scientific output has seen remarkable growth across major research-producing nations. China’s scientific publication output increased from 53.88 million in 2000 to 528.54 million in 2018, demonstrating the country’s rapid emergence as a research powerhouse. The United States maintained its strong position with approximately 450.32 million publications in 2018, while the European Union collectively produced around 489.67 million publications in the same year. Other significant contributors include the United Kingdom (108.21 million), Germany (103.54 million), and Japan (95.87 million) in 2018 [1] More recent data from Scopus [2] show that in 2022, China surpassed the United States in total number of publications produced, accounting for 23.4% of global research output, followed by the United States at 17.8% and the European Union at 14.6%. This trend reflects the shifting landscape of global scientific research and knowledge production. This explosion of available information poses significant challenges for researchers, who must navigate this vast expanse to find relevant studies that inform their research, validate their hypotheses, or situate their findings within the broader scientific landscape to contrast them with existing studies, identify trends, and place their work within the context of what is already known in their field. Traditional methods of discovering pertinent literature, such as keyword searches [3,4], browsing conference proceedings, and reading journal publications, are becoming increasingly inadequate. The vast amount of information makes it difficult to efficiently locate high-quality resources, thus hindering the research process. This is where recommendation systems [5,6,7] come into play, offering a potential solution to streamline the discovery of relevant publications [8] and enhance the research process. These tools, initially designed to predict user preferences and make suggestions in domains like e-commerce, music, and movies, have proven to be invaluable in academia as well. For example, in e-commerce, recommendation systems have been instrumental in increasing sales, such as on Amazon, where it is estimated that 35% of its sales are generated by its recommendation engine. Similarly, in the academic sphere, these systems can recommend relevant articles, journals, and conferences, thereby facilitating more efficient literature reviews and helping researchers stay updated with the latest developments in their fields. By leveraging algorithms tailored for academic content, recommendation systems can significantly enhance the efficiency and effectiveness of the research process.

These systems rely on two main types of information: features and user–item interactions. Features consist of unique attributes of the items, such as keywords and categories, as well as user-specific information like preferences and profiles. User–item interactions include various types of engagement, such as views, likes, comments, and the number of purchases. Based on these factors, two various classifications of recommendation systems highlighted: classical classification and the classification of Rao and Talwar. In this study, we concentrated on classical classification. Both methods are described below:

• Classical Classification [9]: This widely known classification divides recommendation systems into three main methods (which will be discussed in detail throughout this article):

  –

  Collaborative filtering (CF) [10,11]: This method predicts user preferences based on the preferences of other users with similar tastes.

  –

  Content-based filtering (CB) [12,13,14]: This method recommends items based on the attributes of the items themselves and a user’s past interactions with similar items.

  –

  Hybrid filtering: This method combines both CF and CB techniques to capitalize on the strengths of each, resulting in more precise recommendations.

• Rao and Talwar’s Classification [15]: This method expands on classical classification by introducing additional categories and categorizing systems based on the source of information used. In addition to the collaborative filtering, content-based filtering and hybrid filtering, we find:

  –

  Demographic Filtering [16]: This method uses demographic information about users to make recommendations.

  –

  Knowledge-Based Filtering [17]: This method uses specific domain knowledge and user requirements to recommend items.

  –

  Community Filtering: Also called social RSs [18], this method enhances personalized recommendations by incorporating social relationships. However, the impact of social relationships on recommendation accuracy in niche domains is not well understood, and methods often ignore implicit social influences and their evolution over time. Key research areas include the influence of community interactions, best practices for integrating community data, balancing explicit and implicit social influences, and maintaining scalability while preserving social relationship information.

This study is structured as follows: Section 2 provides a general overview of recommendation systems (RSs), including their historical developments, areas of application, research efforts related to recommending scientific articles, and an explanation of the main recommendation models. Afterward, Section 3 describes in detail the techniques used in three recommendation algorithms that were compared in terms of performance. The experiments conducted and the results obtained, along with their interpretations, are presented in Section 4. Finally, Section 5 concludes the study and outlines directions for future work.

2. Related Works

A recommendation system [19] is a type of information filtering system designed to predict the “rating” or “preference” that a user might give to a particular item. The capability of computers to make recommendations was recognized early in computing history. In 1979, Grundy [20] introduced a librarian system as one of the first steps toward automatic RSs. This system, although primitive, classified users into “stereotypes” based on a brief interview and used these stereotypes to recommend books. Despite its limited use, this was an intriguing initial attempt in the field of recommendation systems. In the early 1990s, CF emerged as a solution to information overload. In 1992, Belkin and Croft [21] analyzed information filtering and retrieval, identifying the former as crucial for RSs and the latter for search engines. Goldberg et al. [22] introduced Tapestry, the first collaborative filtering system, inspiring Massachusetts Institute of Technology (MIT) and University of Minnesota (UMN) researchers to create GroupLens [23], a news recommendation service using user–user collaborative filtering. Prof. John Riedl’s GroupLens lab at UMN pioneered RS research. Similar technologies were applied to music and video by Ringo and Video RSs [24,25]. The commercial potential of recommendation systems led to the founding of Net Perceptions [26] in 1996, which served clients like Amazon. Schafer et al. [27] highlighted how these systems boost e-commerce sales. In 1997, the GroupLens lab launched MovieLens [28], releasing influential datasets for recommendation studies. Before 2005, collaborative filtering techniques dominated the field, including user–user [29,30], item–item [31,32], and SVD-based methods [33]. The Netflix Prize (2006–2009) spurred interest in matrix factorization models [34,35] and user-centric evaluation metrics [36]. The first ACM RSs Conference [37] was held in 2007, marking the importance of RS research. Richardson et al. [38] introduced a logistic regression model improving click-through rate estimation. Subsequent advancements included factorization machines (FMs) [39] and field-aware factorization machines (FFMs) [40]. Researchers emphasized user experience, leading to user-centric evaluation frameworks [41,42]. Since 2016, deep neural network-based recommendation models have emerged, enhancing app and video recommendations. Notable models include Wide&Deep [43], DeepFM [44], and YouTubeDNN [45]. Open benchmarks like FuxiCTR [46] and reproducible evaluation metrics [47,48] have been developed to address reproducibility issues. Recent research focuses on addressing biases in recommendation systems (RSs) through causal inference [49,50]. Courses like Thorsten’s counterfactual machine learning [51] leverage these concepts by using models that answer “what-if” questions to understand how changes in input data impact recommendations, thereby identifying and reducing biases.

Regarding some research related to scientific article recommendations, we find this table (

Table 1. Overview of significant research on scientific paper recommendation systems.

Year	Author(s)	Purpose	Sample and Methods	Key Findings
2019	Collins and Beel [52]	To evaluate different document embedding methods for paper recommendations	Used Doc2Vec and TF-IDF in Mr. DLib recommender-as-service	Demonstrated the effectiveness of different document embedding approaches for paper recommendations
2019	Chen and Ban [53]	To develop a user interest clustering model	Applied LDA and pattern equivalence class mining in CPM model	Successfully clustered user interests using topic modeling and pattern mining techniques
2020	Ali et al. [54]	To create a personalized probabilistic recommendation model	Developed PR-HNE using citations, co-authorships, and topical relevance with SBERT and LDA	Effectively integrated multiple graph information sources with semantic embeddings
2020	Du et al. [55]	To develop a heterogeneous network-based recommendation system	Created HNPR using random walks on citation and co-author networks	Demonstrated the effectiveness of using heterogeneous network structures for recommendations
2020	Nishioka et al. [56]	To incorporate users’ recent interests for serendipitous recommendations	Integrated user tweets to capture current interests	Successfully enhanced recommendation serendipity through social media integration
2020	Rahdari & Brusilovsky [57]	To develop a customizable recommendation system for conference participants	Created a system allowing users to control feature impacts	Showed the benefits of user-controlled feature weighting in recommendations
2020	Wang et al. [58]	To develop a knowledge-aware recommendation system	Implemented LSTM-based path recurrent network with TF-IDF representations	Successfully mined knowledge graph paths for enhanced recommendations
2021	Bereczki [59]	To model user–paper interactions in a bipartite graph	Used Word2Vec/BERT embeddings with graph convolution	Demonstrated effective integration of text embeddings with graph-based approaches
2021	Chaudhuri et al. [60]	To incorporate indirect features for recommendations	Developed Hybrid Topic Model combining LDA and Word2Vec	Successfully utilized keyword diversification and citation analysis for improved recommendations
2022	Kreutz et al. [61]	To review contemporary paper recommendation systems	Surveyed studies from January 2019 to October 2021	Provided comprehensive overview of methods, datasets, and challenges
2022	Aymen et al. [62]	To review academic works on paper recommendations	Analyzed content-based, CF, and hybrid methods	Compared methodologies and identified open issues in the field
2023	Zhang et al. [63]	To survey scholarly recommendation systems	Reviewed challenges and approaches in scholarly recommendations	Provided insights into broader scholarly recommendation systems

) presents a chronological overview of significant contributions in the field from 2019 to 2023. The table encompasses both comprehensive survey papers and individual research works that have advanced the state of scientific paper recommendation systems. From foundational approaches using document embeddings and topic modeling to more sophisticated methods incorporating heterogeneous networks and knowledge graphs, the table captures the evolution of recommendation techniques. Recent works have increasingly focused on hybrid approaches, combining multiple data sources and advanced machine learning methods. The table is structured to highlight each study’s year, authors, primary purpose, methodological approach, and key findings, providing a systematic view of the field’s development over the past five years.

Recommendation models encompass various approaches to predict user preferences:

2.1. Collaborative Filtering (CF)

CF methods [64] make automatic predictions about a user’s interests by collecting preference or taste information from many users. It assumes that users who have shown similar preferences or agreed on certain items in the past are likely to have similar preferences in the future. Therefore, it leverages user behavior data to make recommendations. It pioneered technology in RSs and remains simple and effective. The CF process involves three stages: collecting user information, creating a matrix to calculate user associations, and providing reliable recommendations. Major companies like YouTube, Netflix, and Spotify heavily rely on such systems. CF systems are generally divided into two subcategories: memory-based and model-based.

2.1.1. Memory-Based

Memory-based algorithms [65] utilize the entire user rating database to generate predictions by leveraging user-to-user and item-to-item similarities or correlations. These algorithms are widely employed in many commercial systems due to their straightforward implementation and ability to provide accurate recommendations efficiently. Various statistical techniques such as Pearson correlation, cosine similarity, Euclidean distance, and the Jaccard measure can be used to estimate similarities between users or items.

2.1.2. Model-Based

Model-based algorithms [65] rely on developing a predictive model from the user rating data to make recommendations. Instead of using the entire database directly, these algorithms create a model that can generalize the data and identify patterns. For instance, techniques like matrix factorization, including singular value decomposition (SVD), discern latent features such as user preferences for genres or item popularity, effectively generalizing user–item interaction data. generalize user–item interaction data. Similarly, clustering methods group users with similar behaviors, allowing the identification of common interests in niche categories. Unlike memory-based approaches, model-based CF methods often require more computational resources and training time but can be more accurate and scalable, particularly with large datasets. Common techniques used in model-based CF include the following:

• Matrix Factorization [66,67]: Involves techniques like Singular Value Decomposition (SVD) and Alternating Least Squares (ALS) to decompose the user-item interaction matrix into smaller matrices. This helps uncover hidden factors that explain observed interactions.
• Bayesian Networks [68]:These probabilistic models represent dependencies among variables (users and items) and use these relationships to make predictions.
• Clustering [69]: Users or items are grouped into clusters based on their similarities, and recommendations are made based on the preferences of the cluster members.
• Markov Decision Processe [70]: These models consider the sequences of user interactions and make recommendations by predicting the next item in a user’s sequence.
• Machine Learning Techniques [71]: Algorithms such as neural networks, decision trees, and support vector machines can be trained on user-item interaction data to predict user preferences.

In summary, CF methods predict user preferences by analyzing user data, making them foundational in recommendation systems. Their simplicity and effectiveness ensure they remain widely used on popular platforms.

2.2. Content-Based Filtering (CB)

This approach relies on comparing items with the past preferences of a specific user. Items that have similar features to those the user’s profile has previously liked or viewed are likely to be recommended. Content-based recommendation systems use various methods to analyze and compare the features of items and user profiles (see Figure 1).

2.3. Hybrid Recommendation Systems

CF and CB recommendation methods, previously discussed in detail, have strengths and weaknesses that affect their performance across different scenarios (See

Table 2. Strengths and weaknesses of recommendation systems.

Approach	Strengths	Weaknesses
Collaborative Filtering (CF)	No need for item content. Leverages community data. Captures complex preferences.	Requires large user data (Scalability). Struggles with new items or users (Cold Start). Sparse data problem (Sparsity).
Content-Based (CB)	Does not require a large user base. Can recommend new items. Effective for unique tastes.	Requires item features. Tends to recommend similar items.

). Understanding these characteristics is crucial for selecting the appropriate approach for a given application. To leverage their advantages while addressing their limitations, hybrid recommendation systems were developed. These systems integrate CF and CB methods, leading to more accurate, diverse, and robust recommendations. For instance, the cold start problem in CF can be addressed by incorporating content-based methods. In content-based filtering, recommendations rely solely on item descriptions, eliminating the need for an item to be rated by multiple users before it can be recommended, as required in CF. According to Robin D. Burke [72], there are different ways to build hybrid recommendation systems:

• Weighted: Assigns weights to each of the methods, combining their scores into a single recommendation score based on the weighted sum.
• Switching: Alternates between different recommendation methods depending on specific criteria like user type, item characteristics, or the recommendation context.
• Mixed: Independently generates recommendations using various methods and then merges the results into a unified list presented to the user.
• Cascade: Uses one recommendation method to produce an initial list and refines it with another method. For example, CF can create a broad list, which is then fine-tuned by content-based filtering.
• Feature augmentation: Enhances one recommendation method by incorporating additional features derived from another. For example, user similarity scores from CF can improve content-based filtering.
• Meta-level: Utilizes the model output from one recommendation technique as input features for another. For example, the results of a content-based model can serve as features in a CF model.

3. Proposed Methods

This study uses a quantitative research approach to evaluate and compare the performance of three recommendation algorithms: collaborative filtering (CF), content-based (CB) methods, and hybrid methods. The effectiveness of these algorithms is systematically measured and analyzed using numerical metrics like recommendation accuracy, ranking quality, and novelty. Below, we describe the specific techniques used for each algorithm:

• Collaborative Filtering: For this method, we employed a model-based approach, specifically using the singular value decomposition (SVD) algorithm [73], as it is a robust and reliable choice for building high-quality CF recommendation systems. The SVD algorithm (see Figure 2) decomposes the user-item interaction Y matrix into three matrices $U\in R^{m\times p}$, $\Sigma \in R^{p\times p}$, and $V^T\in R^{p\times n}$ that capture the latent factors representing users and items.

  $$Y=U\Sigma V^T$$

  (1)

  where U and V are the orthogonal matrices and $\Sigma$ is the singular orthogonal matrix with non-negative elements and contains all the singular values of Y. This decomposition allows the model to make predictions by approximating the original interaction matrix using these latent factors, thereby improving the accuracy and scalability of the recommendations.
• Content-Based Filtering: Since the task involves recommending scientific articles, we decided to use common techniques from the fields of information retrieval and text mining. Specifically, we used Term Frequency–Inverse Document Frequency (TF-IDF) [74] to convert text data into numerical vectors and extract features, where each word is represented by a position in the vector, and the value indicates the relevance of that word to a particular article. We calculate the similarity between the articles because all the elements will be represented in the same vector space model. The weight $W_{(t,d)}$ of term t in document d is given by:

  $$W_{t,d}=T{F}_{t,d}log{\displaystyle \frac{N}{D{F}_t}}$$

  (2)

  where $T{F}_{t,d}$ is the number of occurrences of t in document d, $D{F}_t$ is the number of articles containing the term t, and N is the total number of articles in the corpus. We used cosine similarity to measure the similarity between the vectors generated by TF-IDF.
• Hybrid Filtering: Concerning this approach, we used the dynamic weighted hybridization method (see the algorithm flowchart in Figure 3). This technique combines the recommendations from both CF and CB by assigning different weights ($w_{CF}$ and $w_{CB}$), we can adjust them based on performance evaluation until convergence, resulting in final recommendations. This flexibility is particularly useful when dealing with diverse datasets, as it enables us to tailor the hybrid model to better suit the specific characteristics of the data. We update weights using the gradient of the loss function as follows:

  $$Loss=\sum _{\left(u,i\right)\in TrainSet}{\left(r_{ui}-\left(Score\left(u,i\right)\right)\right)}^2$$

  (3)

  where $r_{ui}$ is the actual rating of user u to item i, and $Score\left(u,i\right)$ is the combined score for user u and item i generated by the hybrid RS.

4. Experimental Section

4.1. Experimental Settings and Datasets

The datasets are split into 80% for training and 20% for testing, ensuring the model learns effectively by having sufficient data to identify patterns and behaviors. This split also allows for realistic performance evaluation on unseen data. Cross-validation techniques were used to enhance robustness. For the hybrid model, a learning rate of 0.001 was selected, and the Adam optimizer was employed with a batch size of 64. Training was conducted for 100 epochs with early stopping applied if validation performance did not improve after 10 iterations.

In this study, we used two datasets: “CI&T Deskdrop” and “Citeulike-t”. The first dataset includes a real sample of logs spanning a year (from March 2016 to February 2017) derived from CI&T’s in-house communication system, known as DeskDrop. The second dataset is collected from CiteULike and Google Scholar. It includes various data files representing user–article interactions and article content.

Table 3. Statistical summary of CI&T Deskdrop and Citeulike-t Datasets.

	CI&T Deskdrop	Citeulike-t
# of Users	1895	7947
# of Articles	3122	25,975
# of Interactions	72,312	134,860
Sparsity	7.99%	99.93%

provides a statistical summary of these datasets.

4.2. Metrics of Evaluation

In this study, the recommenders use implicit feedback to estimate preferences from user interactions such as views, clicks, likes, and comments. Implicit feedback refers to data indirectly reflecting user preferences, gathered unobtrusively rather than through explicit ratings or reviews [75]. This approach focuses on ranking metrics commonly used in information retrieval settings. In the context of recommendation systems (RSs), the goal is to recommend the most relevant items based on the user’s preferences and past behavior. Therefore, it is more appropriate to compute precision and recall for the highest-ranked N items, giving rise to the concepts of precision at k and recall at k, where k is a user-defined integer aligned with the top-N recommendations objective.

It is important to note that Precision at k (Precision@k) represents the proportion of relevant items within the top-k recommendations, while Recall at k (Recall@k) measures the proportion of relevant items identified among these top-k recommendations. Mathematically, Precision@k and Recall@k are defined as follows:

$$\mathrm{Precision}@\mathrm{k}={\displaystyle \frac{\mathrm{Recommended}\mathrm{items}@\mathrm{k}\mathrm{that}\mathrm{are}\mathrm{relevant}}{\mathrm{Recommended}\mathrm{items}@\mathrm{k}}}$$

(4)

$$\mathrm{Recall}@\mathrm{k}={\displaystyle \frac{\mathrm{Recommended}\mathrm{items}@\mathrm{k}\mathrm{that}\mathrm{are}\mathrm{relevant}}{\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{relevant}\mathrm{items}}}$$

(5)

The recommendation system outputs a ranked list of articles, making it essential to consider the order in which these articles are presented. Therefore, we used NDCG at k, defined as follows:

$$NDCG@k={\displaystyle \frac{1}{IDCG}}\times \sum _{i=1}^k{\displaystyle \frac{2^{r_{i-1}}}{{log}_2(i+1)}}$$

(6)

where $r_i$ is the relevance rating of the document at position i. IDCG is set so that the perfect ranking has an NDCG value of 1. In our problem, $r_i$ is 1 if the document is recommended correctly.

Accuracy is important, but focusing solely on it often results in less satisfactory recommendations. To address this, we also use non-accuracy metrics such as the novelty of the recommended items. Novelty assesses how unfamiliar the recommended items are to a user. For a given user u, Novelty is defined as the ratio of unknown items in the list of top-N recommended items:

$$Novelty\left(u\right)={\displaystyle \frac{{\sum }_{i\in R}(1-knowns(u,i\left)\right)}{\mid R\mid }}$$

(7)

where R is the set of top-N recommended items and $knowns(u,i$) is a binary function that returns 1 if user u already knows item i, and 0 otherwise.

4.3. Results

The experiments conducted in this study yielded various results, which are summarized in three figures (Figure 4, Figure 5 and Figure 6) and two tables (

Table 4. Comparison of hybrid techniques in recommender systems.

	CI&T Deskdrop				Citeulike-t
Hybridizing Technique	NDCG@5	NDCG@10	Novelty@5	Novelty@10	NDCG@5	NDCG@10	Novelty@5	Novelty@10
Switching	0.293	0.287	0.180	0.090	0.225	0.215	0.125	0.065
Cascade	0.295	0.290	0.175	0.088	0.230	0.210	0.120	0.063
Traditional Weighted	0.305	0.300	0.183	0.094	0.240	0.230	0.138	0.074
Dynamic Weighted	0.312	0.308	0.200	0.100	0.250	0.240	0.150	0.080

and

Table 5. NDCG scores for different recommendation systems through CI&T Deskdrop and Citeulike-t datasets. The highest scores for each approach are shown in bold.

	CI&T Deskdrop		Citeulike-t
	NDCG@5	NDCG@10	NDCG@5	NDCG@10
Collaborative-filtering(CF)	0.259	0.257	0.200	0.190
Content based (CB)	0.289	0.290	0.230	0.220
Hybrid	0.312	0.308	0.250	0.240

).

4.4. Discussion

To identify the most suitable hybridization technique for recommender systems, we conducted an ablation study, summarized in Table 4. Key findings include the following:

• Switching and Cascade techniques: NDCG@5 values range from 0.180 to 0.293, showing varying performance across systems.
• Traditional Weighted technique: NDCG@5 values range from 0.183 to 0.305, indicating performance variability.
• Dynamic Weighted technique: Highest NDCG@5 values ranging from 0.200 to 0.312, suggesting superior effectiveness.

For Novelty@5 and Novelty@10, the Dynamic Weighted technique generally performs best, indicating its potential for recommending novel items. Overall, the results highlight that the choice of hybrid technique significantly impacts performance metrics, with the Dynamic Weighted technique emerging as a promising hybridizing technique for enhancing both relevance and novelty in recommender systems.

Figure 4 and Figure 5 present the Precision@k and Recall@k evaluations for three recommendation systems (collaborative filtering (CF) [64], content-based (CB) [76], and hybrid [72]) using two datasets: CI&T Deskdrop and Citeulike-t. All three methods generally exhibit higher precision values for the CI&T Deskdrop dataset than the Citeulike-t dataset. The Hybrid method has the best scores across both datasets. Moreover, CB surpasses CF in both datasets, indicating that content-based recommendations are more potent than collaborative filtering in these scenarios. There is a noticeable decrease in precision from @5 to @10 across all methods and datasets, implying that while the top 5 recommendations are relatively precise, expanding to the top 10 introduces less relevant items. Similar observations can be made for recall, with the key difference being that the recall value increases from @5 to @10.

Regarding the quality of the ranking of the recommended items (NDCG), the hybrid approach consistently delivers the best performance across both datasets (see Table 5), followed by CB. CF shows lower performance, especially in the highly sparse Citeulike-t dataset (99.93%) compared to the CI&T Deskdrop dataset (7.99%) (refer to Table 3), highlighting the limitations of CF in such scenarios, which might require more data or additional techniques to handle sparsity [77] effectively.

Figure 6 shows that novelty decreases as the number of recommended items (Top-N) increases, since including more items often add more popular ones, reducing overall novelty. Examining the impact of dataset characteristics, we found that the Citeulike-t dataset, with higher sparsity, has lower novelty scores than the CI&T Deskdrop dataset, indicating that sparsity hinders the ability to provide novel recommendations due to fewer interaction data. In addition, the hybrid approach offers the best balance between relevance and novelty, making it the most effective for diverse recommendations, including many that may be unfamiliar to the user. Collaborative filtering, which outperforms content-based filtering in novelty, is useful when novel recommendations are important. In contrast, content-based filtering is beneficial when recommendations need to align closely with the user’s past interactions.

Despite the differing densities of the two datasets, the empirical findings reveal that the hybrid recommendation method consistently surpasses both collaborative filtering (CF) and content-based (CB) methods across a variety of metrics, underscoring its proficiency in addressing sparsity and cold start [78] issues. The superior precision and recall performance of hybrid methods suggests that the integration of collaborative and content-based strategies harnesses the strengths of both, resulting in more precise and comprehensive recommendations. Additionally, the balanced performance of the hybrid approach in terms of ranking quality (NDCG), and novelty indicates its capability to provide a versatile solution that emphasizes relevance while also introducing new and diverse items to users.

On the other hand, content-based methods exhibit strong potential, particularly in sparse datasets, by utilizing specific user preferences and content attributes to generate more relevant recommendations than CF. This is evident in CB’s consistent outperformance over CF across both datasets. However, CF shows limitations in handling highly sparse data, as seen with the Citeulike-t dataset, which may require additional techniques or more data to be effective. These observations collectively emphasize the need for a nuanced approach in recommendation systems, where hybrid methods offer a robust solution for diverse and accurate recommendations, while CB methods can be particularly useful in scenarios with limited interaction data. Nonetheless, further experiments and modifications to CF can still be conducted to assess its performance, especially in such a scenario.

In summary, our study advances recommendation systems by showing how hybrid approaches effectively combine collaborative filtering and content-based methods, improving not only the accuracy, relevance, and novelty of article recommendations but also addressing common challenges such as cold start and sparsity problems. These insights guide the development of more sophisticated tools that enhance content discovery and researcher productivity, making it easier for researchers to navigate the vast array of scientific literature [61,62].

Future research can explore the computational efficiency of hybrid algorithms in real-world applications. Additionally, our study provides a foundation for developing improved hybrid solutions that can be adapted for various sectors, including scientific research and business [79], where personalized content recommendations are increasingly valuable.

5. Conclusions

In this paper, we conducted a comparative study of the three most commonly used approaches in recommendation systems, using multiple metrics. Our experiments revealed that for article recommendations, the hybrid method outperformed content-based filtering and collaborative filtering in terms of relevance, ranking quality, and novelty, recommending more unknown articles to users. The hybrid method’s effectiveness stems from its combination of the two approaches, which helps address common issues such as the cold start and sparsity problems in RSs. It is important to acknowledge that this area is in a state of constant evolution, and conducting an exhaustive examination of all recommendation algorithms is impractical. The content provided here provides an overview of several existing approaches in recommendation. While comprehensive, it is naturally constrained by the selection of methods, and datasets. Moreover, the study does not consider the computational complexity and scalability of the algorithms in real-world applications. There is ample room for further enhancements and experimentation with alternative algorithms. Future research could focus on identifying the most effective algorithms, ultimately aiming to create a more efficient hybrid solution. This study highlights the potential of hybrid methods to overcome individual limitations of traditional approaches. Practically, it offers guidance for building more effective and user-centric recommendation systems tailored to academic content.