A Multidimensional Model for Recommendation Systems Based on Classification and Entropy

Abstract

The proliferation of false and redundant information on e-commerce platforms as well as the prevalence of ineffective recommendations and other untrustworthy behaviors has seriously impeded the healthy development of these platforms. To address these issues and enhance prediction accuracy and user trust, contemporary recommendation systems often utilize additional information (i.e., side information). In this work, we propose a model to improve the recommendation quality by employing the information entropy of user-item ratings. The entropy was used as side information to reflect the global rating behavior of the user and item. We also utilized the classification of the user and item as heuristic information to improve the prediction quality. In our best result, we achieved a significant improvement of 8.2% in prediction accuracy. The model classified the items by the users’ actual preference, which is more trustworthy for users. We evaluated our model with three real-world datasets. The performance of our proposed model was significantly better than the other baseline methods. The similarity calculation method employed in the present model has the potential to mitigate the data sparsity problem associated with correlation-based similarity. The proposed weight matrix has zero sparsity. Furthermore, the proposed model has a more favorable computational complexity for prediction compared to the conventional k-nearest neighbor method.

1. Introduction

With the development of information technology and the growth of the Internet, users are continuously bombarded with an endless flood of information. As a result, people have many more options to choose from in every aspect of their lives, and the cost of obtaining accurate information also increases. Specifically, when there are more choices, it is much more difficult to find the information that best suits their needs and preferences [1,2]. The emergence of the recommendation system (RS) is mainly to solve the problem of information overload. The task of the recommendation system is to help users find information that is valuable to them [3]. These systems are now widely used to recommend different items such as goods, music, and movies. Using various data sources, it infers the interests and preferences of users [4]. For example, if a person reads a recommended article in a news aggregation app, the person will likely discover that it appeals to their interests. The recommendation engine calculates the individual’s preferences based on their past behavior, then selects the best news articles for them from many sources and presents them to the person.

Over the past few decades, the field of recommendation systems has seen significant advancements [5,6,7]. A variety of algorithms have been proposed and evaluated [8,9], which can be broadly classified into three categories: content-based, collaborative filtering (CF), and hybrid methods [10]. Content-based recommenders rely on extensive item descriptions (e.g., metadata, text descriptions, topics) to generate recommendations based on the feature vectors of the content. The effectiveness and accuracy of this type of recommendation system is directly correlated with the amount of information available about the user or item. For instance, content-based recommendation systems utilized by e-commerce platforms often utilize information such as product category, detailed product descriptions, price, and manufacturer [11,12,13]. However, this approach is prone to the “cold start” problem, whereby the system is unable to make recommendations for users or items with insufficient information, and is limited in its ability to explore the users’ potential interests [14,15]. Additionally, manual annotation is often required for media such as images and audio, which increases the workload for developers. CF, another widely used recommendation technique, involves analyzing the users’ historical feedback or behavior to identify patterns that can be used to predict the recommended items [16]. CF can be further divided into memory-based and model-based approaches. The underlying assumption of CF is that users with similar preferences will rate items similarly. This approach is commonly used by e-commerce companies to provide recommendations to customers [17,18,19]. However, CF suffers from the “cold start” problem [20] and has poor scalability, particularly when dealing with large datasets. Additionally, the “data sparsity” problem arises when a user has only rated a small number of items in the user-item matrix. Hybrid recommenders including weighted hybridization, switching hybridization, cascade hybridization, and mixed hybridization [21] are variations of content-based and CF methods. These approaches seek to combine the strengths of both methodologies in order to overcome their individual limitations.

In this paper, we focused on memory-based collaborative filtering [22]. The main challenge was to determine which group of users had a similar preference to the target users [16,23] (i.e., to find an appropriate measure of user similarity). Researchers have used different side-information [24,25,26,27] to obtain a more effective similarity to improve the prediction accuracy of recommendations [28,29]. It is natural to think that we are likely to accept suggestions from similar partners [30,31]. For this reason, many aspects of both users and items are often used as auxiliary information to find similar preferences among users in CF [17,18,19,20,21]. However, due to various reasons, it is difficult to obtain a direct similarity relationship. In contrast, implicit similarity relationships are inferred from user preferences. If a user is more knowledgeable about a given subject, we assume that they are more reliable. Information entropy is the indicator that we can use to measure the amount of information [32,33]. This is a popular indicator to measure the amount of useful information and is often used by researchers to improve accuracy [21,34,35]. By using information entropy of the ratings on items and users, our proposed method considers the rating behavior of users on each common item as well as the history of ratings made by each user. We then used the entropy to make a rating prediction. Inspired by the traditional kNN method [36,37], we proposed a novel method of choosing neighbors. We then merged the entropy with classification information. We used classification information to find a more suitable similar user in the global view.

This study proposed a recommendation system that utilizes entropy and classification information to generate personalized product recommendations based on user evaluations. The proposed multidimensional model, referred to as UITrust, enables the recommendation system to identify trustworthy and similar users. Unlike traditional similarity matrices, the proposed similarity matrix did not contain empty entries. Through the use of the proposed method, we observed significant improvements in comparison to other baseline methods across three real-world datasets. The specific contributions of this research include the following:

• We propose a new multidimensional recommender model called UITrust that utilizes classification information and entropy to improve the prediction accuracy.
• Reduce both the computational complexity of prediction and the sparsity of the weight matrix compared with the baseline methods.
• Using three real-world datasets and the experimental results, we show that the proposed model is superior to other benchmark methods including traditional kNN-based methods.
• Proposed a novel approach for explainability in the recommender.

The present study presents a framework that can be effectively implemented in online e-commerce systems to assist sellers in promoting their products and services.

The subsequent section formally defines the problem and presents the proposed model. Subsequently, we outline the experimental environment and provide a comparison of the evaluation results of the proposed model with a baseline. The final section concludes the article by briefly discussing the limitations of the proposed method and suggesting avenues for future research.

2. Proposed Model

2.1. Preliminaries

Memory-based CF methods have been adopted in different practical industrial systems because of their simplicity and effectiveness. These approaches generally use the ratings vector to indicate user preference. The rating history is used to find similar users/items, also called neighbors. These can be identified by a similarity measure that represents the differentiation between two users. With the selected neighbors, the ratings to be predicted are calculated by aggregating the ratings of neighbors. We model the prediction process as follows.

2.2. Model

The objective of this study was to predict ratings for items that have not yet been rated by a user, based on the user’s rating history data. A common approach to achieve this is to calculate the similarity between the target user and a set of $k$ neighbors, and to weight the ratings of these neighbors accordingly. As a baseline, we utilized the basic k-nearest neighbor (kNN) formula outlined below.

$$\widehat{r_{u,i}}=\frac{{\sum }_{v\in N_{i }^k\left(u\right)}Si{m}_{u,v}\times r_{v,i}}{{\sum }_{v\in N_{i }^k\left(u\right)}Si{m}_{u,v}},$$

(1)

where $r_{u,i},{ r}_{v,i} \in R_{\left|U\right|\times \left|I\right|}$ is the ratings of users $u, v \in U$ for item $i \in I$ in the rating matrix $R_{\left|U\right|\times \left|I\right|}$. In this equation, $\widehat{r_{u,i}}$ is the prediction of item $i$ for user $u$. $N_{i }^k\left(u\right)$ is the set of $k$ neighbors of user $u$ who have rated the item $i$, $r_{v,i}$ is the rating given to item $i$ by the neighbor $v$, and $Si{m}_{u,v}$ is the similarity between user $u$ and user $v$. The similarity between $u$ and $v$ is calculated using the Pearson correlation, which is given by:

$$Si{m}_{u,v}=\frac{{\sum }_{i\in I_u\cap I_v}\left(r_{u,i}-\overline{r_{u,i}}\right)\left(r_{v,i}-\overline{r_{v,i}}\right)}{\sqrt{{\sum }_{i\in I_u\cap I_v}{\left(r_{u,i}-\overline{r_{u,i}}\right)}^2{\sum }_{i\in I_u\cap I_v}{\left(r_{v,i}-\overline{r_{v,i}}\right)}^2}},$$

(2)

where $\overline{r_{u,i}}$ and $\overline{r_{v,i}}$ are the average rating of the respective user. In Equation (2), based on the correlation of user $u$, $v$’s commonly rated items, we can obtain the similarity between any two users. The most essential work for memory-based CF is the identification of neighbors and the rating prediction. In the following section, we propose two novel measures for these two points.

2.2.1. Classification-Based k-Neighbor Selection Measure

When trying to calculate the similarity of two users, in general, conventional similarity is the distance between two vectors. The size of the similarity matrix will be either the number of users by number of items in the dataset. The measure only focuses on the common item’s rating vector. However, in the case where two users have no common rating, their similarity will be zero. Here, we intend to use the classification of the item along with the taste of each user. In this way, every user for every item will have a value.

We focused on using item classifications, specifically movie genres, to pick the neighbors. We derived the user’s taste for an item using the ratings expressed. Similarly, using the collective ratings given to an item, we identified how much each item belongs to a certain genre. These two vectors are then combined to obtain a user–item–weight matrix. In contrast to the similarity-based weight matrix in memory-based collaborative filtering, we used the user–item–weight to rank the prediction contributor candidate so that we could have the most similar neighbor, and at the same time, we can assume that this neighbor is also familiar with the type of item, implying that this user’s suggestion is valuable.

Let $G=\left\{g_1,g_2, \cdots ,g_k, \cdots ,g_m\right\}$ be aa set of all classifications of the items in the dataset. Item $i\in I$ can be represented in terms of a vector of classifications $\overrightarrow{g_i}$ as in the following equation.

$$\overrightarrow{g_i}=\left\{g_{i1},g_{i2}, \dots ,g_{ik},\dots ,g_{im}\right\},$$

(3)

The $g_{ik}$ will be 1 if item $i$ is labeled classification $g_k$. If not, the value of $g_{ik}$ will be zero. For example, the dataset has three genres, horror, comedy, and romance, and G maintains that order. If movie X is a romantic comedy, then it is labeled as both comedy and romance. Therefore, movie X’s classification vector $\overrightarrow{g_i} \mathrm{will} \mathrm{be} \left\{0,1,1\right\}$. Since all movies are labeled with one or more genres by the producers, a movie always has at least one classification (i.e., $\left|g_{ik}=1\right|>0$). For all items $i\in I_u$ rated by user $u$, and if the classification $g_{ik}=1$, then $g_{ik}=r_{u\left(i\right)}$, composing a new vector ${\overrightarrow{g}}_i^{*}$ from $\overrightarrow{g_i}$. Therefore, continuing with the example of movie X with user u, if the rated movie X has three stars, then ${\overrightarrow{g}}_x^{*}$ will be {0,3,3}. The user-classification vector of user $u$ can be calculated as follows:

$$\overrightarrow{s_{u,k}}=\frac{{\sum }_{i\in I_u}\overrightarrow{g_{i,k}^{*}}}{{\sum }_{i\in I_u}\overrightarrow{g_i^{*}}},$$

(4)

where $\overrightarrow{s_{u,k}}$ indicates the user-classification for the k^th genre and the denominator is the sum of the total ratings in $\overrightarrow{g_i^{*}}$. The denominator can also be interpreted as the total collective ratings given to all genres by user $u$. Please note that this value is always greater than or equal to the user $u$’s total rating. The value of each entry in the resulting vector $\overrightarrow{s_{u,k}}$ represents user $u$’s degree of preference for the genre $k$. For a worked example, consider a horror movie Y to which the user rates five stars, thus ${\overrightarrow{g}}_y^{*}$ = {5,0,0}. The $s_{u,k}$ will be 5/11 = 0.45, where k is the index of the horror genre. One of the advantages of this metric is that it defines the user’s preference for a specific genre as a continuous value rather than a discrete value. The full $\overrightarrow{s_u}$ vector will be {0.45, 0.27, 0.27}.

As mentioned earlier, movies are classified by producers or experts. However, this classification does not consider the viewers’ opinions. Initially, we represented this as a binary vector $\overrightarrow{g_i}$ in Equation (4). Since collaborative filtering uses other users’ views while making predictions, it is intuitive that the movie recommendation using genre information in the process would increase the prediction accuracy. Additionally, it is intuitive to consider all of the user’s ratings (i.e., users’ preferences) to estimate the actual genres of the movies, rather than simply relying on the producers or experts. Let S be the set containing all of the user-classifications $\overrightarrow{s_u} .$ The shape of S would be $\left|U\right|\times \left|g\right|$. We calculated the new genre values $\overrightarrow{c_i}$ for movies as follows:

$$\overrightarrow{c_i}=\frac{{\sum }_{u\in U_i}\overrightarrow{s_u}\times r_{u,i}}{{\sum }_{\mathrm{v}\in U}\overrightarrow{s_v}\times r_{v,i}},$$

(5)

where $r_{u,i},r_{v,j} \in R_{\left|U\right|\times \left|I\right|}$ is the rating by the user $u\in U_i$, who has rated item $i$ and $\overrightarrow{s_u}, \overrightarrow{s_u}\in S$, whereas $v$ is from the set of all user $U$ who rated item $j$ in the dataset. Here in the numerator of the equation, the user-classification $\overrightarrow{s_u}$ is multiplied by the user’s rating. Similar to the elements of $\overrightarrow{s_u}$, elements of $\overrightarrow{c_i}$ range between zero and one. Let us assume there is a second user z who only rated the movie Y with three starts. Now, the $\overrightarrow{s_u}$ and $\overrightarrow{s_z}$ are {0.35, 0.21, 0.21} and {0.57, 0.0, 0.0}. The new genre value calculated using Equation (5) for the item Y will be ({0.35, 0.21, 0.21} × 5)/({0.35, 0.21, 0.21} × 5 + {0.57, 0.0, 0.0} × 4) = {0.75, 1, 1}. The final user–item–weight score is calculated by taking the dot product of both the user’s taste $\overrightarrow{s_u}$ and $\overrightarrow{c_i}$ as follows:

$$w_{u,i}=\overrightarrow{s_u}\otimes \overrightarrow{ c_i},$$

(6)

where $w_{u,i}$ is the user–item weight. Then, we used the $w_{u,i}$ to sort the pending items in the dataset. The final weight $w_{u,i}$ for user u for the item $i$ was {0.73, 1, 1} $\otimes$ {0.35,0.21,0.21} = 0.699. Since every item and user pair has a weight value, the shape of the user–item matrix will be the number of users by the number of items in the dataset. For the item-based methods, we can simply transpose the matrix.

2.2.2. Entropy-Driven User–Item Similarity

In traditional CF, the similarity between users, which serves as a weight matrix, is just one of the most important factors that affect the accuracy of the prediction. Several possible factors can be used to improve the prediction accuracy or even enhance the system’s performance.

The whole calculation process of calculating the final weight matrix is shown in Figure 1:

We formulated the similarity measure, which can obtain the information from entropy for each prediction. In Equation (7), we defined the $prob\_I_i^r$, the probability $prob\left(r_i=r\right)$ of rating $r$ the item $i$. The numerator of the equation indicates the number of ratings that is equal to the currently calibrated value. The denominator is the total number of ratings obtained from this item. Here, our model used this variable to indicate the probability of each possible rating. We can apply the same idea for the user, as shown in Equation (8).

$$prob\_I_i^r=\frac{\left|\left\{u\in U r_{u,i}=r\right\}\right|}{\left|\left\{u\in U r_{u,i}\in \left[r_{\min },r_{\max }\right]\right\}\right|},$$

(7)

$$prob\_U_u^r=\frac{\left|\left\{i\in I r_{u,i}=r\right\}\right|}{\left|\left\{i\in I{r}_{u,i}\in \left[r_{\min },r_{\max }\right]\right\}\right|},$$

(8)

By studying $prob\_I_i^r$ and $prob\_U_u^r,$ the frequency at which the current rating occurs among the user’s historical ratings, we can determine the information content of the rating. Considering the example users, items, and ratings from the previous subsection and applying Equation (7), the probability $prob\_I_y^3$ of item Y rated with three stars was 1/2. Similarly applying Equation (8), the probability $prob\_U_u^5$ of user u rating an item with five stars was 1/2. Next, we incorporated the concept of entropy into the model to help it make better predictions. We assumed that the trust could be increased by adding more information to the system. Therefore, we can use the combined entropy of the user and item. As a result, we can choose the ones that have the most information, providing a more trustworthy recommendation for users. As seen in the following equation, we simply used $H\left(i\right)$ to indicate the degree to which the content of the message is more surprising. The higher value of $H\left(i\right)$ means that the information contained in the log values of the item/user is higher, making the final prediction better. Here, we used the entropy calculation method for items and users separately, then we can obtain $HoI\left(i\right)$ and $HoU\left(u\right),$ which indicate the entropy of the item and the entropy of the user. With the higher $Hol\left(i\right)$\$HoU\left(u\right)$, we can learn which item/user’s vector contains more information. The formulas are shown below:

$$HoI\left(i\right)=-\sum _{r=r_{min}}^{r_{max}}prob\_I_i^r{\mathit{log}}_2\left(prob\_I_i^r\right),$$

(9)

$$HoU\left(u\right)=-\sum _{r=r_{min}}^{r_{max}}prob\_U_u^r{\mathit{log}}_2\left(prob\_U_U^r\right),$$

(10)

In order to make predictions, we used a new similarity to use as weights in this process. We combined the user–item weight $w_{u,i}$ mentioned in the last section and the average of the $Hol\left(i\right)$ and $HoU\left(u\right)$, which is denoted as $UITrust$. The formula is shown below:

$$UITrus{t}_{u,v}=\alpha w_{u,i}+\left(1-\alpha \right)\left(\frac{\mathit{Hol}\left(i\right)+ \mathit{HoU}\left(u\right)}{2}\right)$$

(11)

where $\alpha$ is the hyperparameter in the formula. Consider the extreme case where only one item $i$ has been rated with a single star by a single user and no other user have rated that item. The $prob\_I_i^1$ will be 1 and $HoI\left(i\right)$ will be zero. Similarly the $HoU\left(u\right)$ can also be zero. However, as long as $\alpha$ is not equal to 1 in Equation (11), there will always be a $UITrust$ value. We used this new similarity $UITrus{t}_{u,v}$, which uses information entropy and classification information, to calculate the final prediction. The final prediction $\widehat{r_{u,i}}$ is made by using the mean rating $\overline{r_u}$ of the target user $u$, adding the bias part to it as follows:

$$\widehat{r_{u,i}}=\overline{r_u}+\frac{{\sum }_{j\in N_k\left(u\right)}UITrus{t}_{u,v}\times r_{u,j}}{{\sum }_{j\in N_k\left(u\right)}UITrus{t}_{u,v}},$$

(12)

where $N_k\left(u\right)$ is the set of neighbors selected by the weighted sum of classification-based $k$-neighbor selection measure. With the help of the information entropy, we can now effectively incorporate the contribution of neighbors, reflecting the global rating behavior on the user and item rating history.

3. Results and Evaluation

3.1. Evaluation Indicators

The quality of the recommendation algorithm is determined using evaluation metrics that often provide different perspectives. They are widely used in both research and practice. One of the most common evaluation metrics used for prediction accuracy is the mean absolute error (MAE) [38,39]. The MAE measures the absolute difference between the actual value and predicted values as an error, which is given by:

$$MAE=\frac{1}{\left|\widehat{R}\right|}{\sum }_{\widehat{r_{u,i}}\in \widehat{R}}\left|r_{u,i}-\widehat{r_{u,i}}\right|,$$

(13)

where $\widehat{R}$ is the set of predictions.

In contrast to taking absolute values, RMSE squares the error and takes the square root to deal with the negative values. RMSE also punishes larger errors heavily. RMSE is calculated as follows:

$$RMSE=\sqrt{\frac{1 }{\left|\widehat{R}\right|}{\sum }_{\widehat{r_{u,i}}\in \widehat{R}}{\left(r_{u,i}-\widehat{r_{u,i}}\right)}^2},$$

(14)

3.2. Dataset

In this paper, three different public benchmark datasets from MovieLens (GroupLens, University of Minnesota, Minneapolis, MN, USA) were used to evaluate the proposed algorithm [40]: (1) ML-100k; (2) ML-latest-small; and (3) ML-1m. To better illustrate the validity of the algorithm, we defined density to describe the sparsity for each dataset, which is calculated as follows:

$$Density=\frac{\left|R\right|}{\left|U\right|\times \left|I\right|},$$

(15)

where $\left|R\right|$ refers to the number of ratings in the dataset. The details about these three datasets are provided as follows:

• The MovieLens 100K dataset was collected by the GroupLens Research Project at the University of Minnesota. This dataset consists of 100,000 ratings from 943 users on 1682 movies;
• The ML-latest-small dataset, the same as the first one, was also collected by the same project. It contains 100,836 ratings and 3683 tag applications across 9742 movies. The rating scale is 0.5 to 5;
• The MovieLens 1M dataset contains 1,000,209 anonymous ratings of 3900 movies made by 6040 MovieLens users who joined MovieLens in 2000.

Additional details about the datasets are provided in

Table 1. Details of the datasets.

Dataset	No.Ratings	No.Users	$\mathbf{N}\mathbf{o}. \mathbf{I}\mathbf{t}\mathbf{e}\mathbf{m}\mathbf{s}$	$\mathbf{R}\mathbf{a}\mathbf{t}\mathbf{i}\mathbf{n}\mathbf{g} \mathbf{S}\mathbf{c}\mathbf{a}\mathbf{l}\mathbf{e}$	Density
ML-100k	100,000	943	1682	1 to 5	6.30%
ML-latest-small	100,836	610	9742	0.5 to 5	1.70%
ML-1m	1000,209	6040	3900	1 to 5	4.25%

.

From the results, we can see that these datasets were not dense. Therefore, it was necessary to find some measures to overcome the problems caused by data sparsity. To ensure the validity of the algorithm, we divided each dataset into two parts: 80% of the data was used as the training set, and the remaining 20% was used as the test set.

3.3. Experimental Setup

In this study, we utilized the Surprise framework, a python-based library widely utilized by both researchers and practitioners, to implement our algorithm. The Surprise framework, developed by Hug et al. [41], is highly modular and features several benchmark recommendation algorithms. Due to the high memory requirements of memory-based collaborative filtering (CF) methods, which store the weight matrix in memory, we conducted our experiments on a Dell PowerEdge T630 server equipped with 2.1 Hz 32 cores and 64 GB of RAM. The data for our experiments were structured as tuples comprising the user ID, item ID, ratings, and timestamps. We employed five-fold cross-validation for all datasets and test methods and compared our results to several existing approaches:

• UITrust_C: Our proposed method, as in Equation (11), with $k$ neighbor was chosen using the classification-based k-neighbor selection measure;
• UITrust_P: Our proposed method, as in Equation (11), with k-neighbor was chosen using the Pearson similarity;
• UITrust_MSD: Our proposed method, as in Equation (11), with $k$-neighbor was chosen using MSD similarity;
• UITrust_R: Our proposed method, as in Equation (11), with $k$-neighbor was chosen by Random;
• CKNN_P: A basic collaborative filtering algorithm, taking into account the mean ratings of each user, with $k$-neighbor chosen using Pearson similarity;
• CKNN_MSD: A basic collaborative filtering algorithm, taking into account the mean ratings of each user, with $k$-neighbor chosen using MSD similarity;
• BKNN_P: A basic collaborative filtering algorithm with $k$-neighbor was chosen using Pearson similarity;
• BKNN_MSD: A basic collaborative filtering algorithm with $k$-neighbor was chosen using MSD similarity.

3.4. Experimental Results

3.4.1. Results on the ML-100k Dataset

Figure 2 illustrates the prediction accuracy results obtained on the ML-100k dataset for the various methods being studied.

As shown in Figure 2, the results indicate that the proposed method (UITrust_C) using a classification-based measure as the neighbor selection method performed the best on the 100k dataset, with a slight increase in the mean absolute error (MAE) of 0.03% compared to UITrust_P, and a slight increase in the root mean squared error (RMSE) and MAE of 0.63% and 0.59%, respectively, compared to CKNN_P (using the same neighbor selection method). In comparison to CKNN_MSD, UITrust_C showed a more significant increase in accuracy, with an increase of 1.38% in MAE. In addition, UITrust_C exhibited a significant improvement against BKNN_P, with an increase of 6.80% in RMSE and 7.21% in MAE. There was also a significant improvement against BKNN_MSD, with an increase of 3.6% in RMSE and 4.53% in MAE.

The performance of UITrust_P was similar to that of UITrust_C. However, UITrust_MSD did not show any advantage compared to CKNN_P, but exhibited an improvement of 0.07% in RMSE and 0.74% in MAE compared to CKNN_MSD, and an improvement of 6.12% in RMSE and 6.62% in MAE compared to BKNN_P as well as an improvement of 2.94% in RMSE and 3.92% in MAE compared to BKNN_MSD. As for UITrust_R, the method did not perform better than CKNN_P, CKNN_MSD, and BKNN_MSD, but still showed improvements of 2.62% in RMSE and 4.28% in MAE compared to BKNN_P.

3.4.2. Results on the ML-Latest-Small Dataset

Figure 3 illustrates the prediction accuracy results obtained for the ML-latest-small dataset for the different methods being studied.

The results for the ML-latest-small dataset showed a similar pattern to those obtained on the ML-100k dataset as shown in Figure 3. In particular, UITrust_C ranked first, with a slight increase of 0.55% in the root mean squared error (RMSE) and 0.12% in the mean absolute error (MAE) compared to UITrust_MSD. There was also an increase of 1.16% in RMSE and 0.92% in MAE compared to the official baseline method CKNN_MSD. In addition, we found that UITrust with the mean squared difference (MSD) similarity exhibited a significant improvement against BKNN_MSD, with an increase of 5.81% in RMSE and 6.30% in MAE. Similarly, UITrust_P with the Pearson similarity showed a significant improvement against BKNN_P, with an increase of 8.62% in RMSE and 8.41% in MAE. Both UITrust_P and UITrust_MSD performed roughly as well as the method using entropy-based similarity. However, UITrust_P did not outperform the CKNN methods in terms of MAE. As for UITrust_R, this method did not perform better than CKNN_P, CKNN_MSD, and BKNN_MSD, but still exhibited an increase of 2.13% in RMSE and 1.18% in MAE compared to BKNN_P.

It is important to note that this dataset had the lowest density (i.e., the highest sparsity). Since both ML-100K and ML-latest-small had roughly the same number of ratings and significant improvements in UITrust-based methods compared to the baselines, it is clear that there is a significant contribution to improvements in prediction accuracy due to improved sparsity in our weight matrix.

3.4.3. Results on the ML-1m Dataset

Figure 4 illustrates the prediction accuracy results obtained for the ML-1m dataset for the different methods being studied.

The prediction accuracy of the current dataset was found to be superior to that of the other two datasets. As the size of the dataset increased, the effectiveness of our approach became more pronounced. The performance of UITrust_C and UITrust_P was almost identical, as shown in Figure 4. UITrust_C exhibited an increase of 0.56% in the root mean squared error (RMSE) and 1.49% in the mean absolute error (MAE) compared to CKNN_P, and a more significant increase of 5.07% in RMSE and 6.70% in MAE compared to BKNN_P. There was also an improvement of 1.72% in RMSE and 3.34% in MAE compared to BKNN_MSD.

Similarly, UITrust_P showed an improvement of 1.70% in RMSE and 3.34% in MAE compared to CKNN_MSD, and a significant improvement of 5.06% in RMSE and 6.99% in MAE compared to BKNN_P.

3.4.4. Sparsity of Weight Matrix

In order to compare the sparsity of the weight matrices among different methods, in Figure 5, we present a plot showing the number of empty entries in the weight matrix. The dataset that we used here was ML-100k. For the kNN-based methods, the weight matrix was the similarity matrix. We chose two different types of similarities, MSD and Pearson, for the result comparison. For our method, we used the classification-based k-neighbor selection measure and entropy-driven user–item similarity for plotting.

As seen in Figure 5, the increase in the number of users increased the sparsity of the weight matrices for the MSD and Pearson similarity. The average rate in the increase in the sparseness of the Pearson similarity matrix was roughly 2.5 times that of the MSD similarity matrix. However, for UITrust, regardless of the classification-based k-neighbor selection measure or other measures such as Pearson or MSD, the plot of them showed a flat zero, meaning that one of the users for all items always has a weight value. As discussed earlier, decreasing the sparsity is important as it impacts the accuracy of the predictions and our proposed method addressed this issue.

4. Discussion

The proposed method had significantly better prediction accuracy compared with other traditional kNN-based methods. Traditional methods of calculating similarity include MSD similarity and the Pearson correlation method. The Pearson correlation coefficient is based on the premise that both users must rate the same items and helps to choose similar users. As for MSD, only the number of items rated jointly by users is considered, but the rating of the same item between users is not considered. None of these methods solved the problem of data sparsity in the user–item matrix. One of the advantages of our proposed method is that we can have a distinct prediction result for users who have no neighbors. The classification-based k-neighbor selection measure helps significantly improve the result’s coverage and well addresses the data sparsity issue. Additionally, during the pre-computation process, our method considers the multi-dimensional information input, so we can have a reasonable graph of the calculations and results. It proposes a novel approach for interpretability in a recommender. Compared with the correlation-based similarity, the utilization of entropy also helps our proposed method reflect the global rating behaviors of users and items. Therefore, we can take into account the analysis of the overall dataset while taking into account the influence of the neighbors of the users.

However, the disadvantages are also obvious. In real-time situations, the recommendation system has no conditions to calculate the overall fluctuation of the data in real-time. Therefore, we cannot have the entropy value at a negligible length of time. This method will reduce the recommendation efficiency in this situation. At the same time, this method is not suitable for interactive recommendation systems. Our proposed measure will be more suitable for small-scaled sparse datasets.

5. Conclusions

In this study, we proposed a recommendation system algorithm that incorporates information entropy and classification information to improve the prediction accuracy. To evaluate the efficacy of our approach, we conducted experiments on three real-world datasets. The key concept behind our algorithm is the utilization of entropy in ratings for each user and item to capture the variability within the datasets, which is then integrated into traditional similarity measures in order to identify contributors with the highest amount of information, and thus the greatest level of trustworthiness. These contributors were also identified through a classification-based k-neighbor selection process, ensuring that they were familiar with the specific item being recommended. This led to more trustworthy recommendations and improved coverage compared to kNN-based methods due to the reduced sparsity of the new similarity measure. Furthermore, our method provides explainable recommendations, providing users with multiple dimensions of information to personalize the recommendation process and enhance the prediction performance. Our experiments on the MovieLens datasets demonstrate the ability of our method to enhance the prediction quality. In future work, we plan to further examine the performance of our method on modern and dense datasets from various domains, and to address the issue of a small, static dataset size in order to apply our approach to interactive industrial environments. We also intend to assess the performance of our method using various metrics such as reliability and diversity through additional experiments. It is important to note the gain in popularity of deep learning technologies in recommendation research, and in future work, we plan to compare and incorporate our method with modern deep learning approaches.