On the Smaller Number of Inputs for Determining User Preferences in Recommender Systems

Abstract

One of the most popular applications for the recommender systems is a movie recommendation system that suggests a few movies to a user based on the user’s preferences. Although there is a wealth of available data on movies, such as their genres, directors and actors, there is little information on a new user, making it hard for the recommender system to suggest what might interest the user. Accordingly, several recommendation services explicitly ask users to evaluate a certain number of movies, which are then used to create a user profile in the system. In general, one can create a better user profile if the user evaluates many movies at the beginning. However, most users do not want to evaluate many movies when they join the service. This motivates us to examine the minimum number of inputs needed to create a reliable user preference. We call this the magic number for determining user preferences. A recommender system based on this magic number can reduce user inconvenience while also making reliable suggestions. Based on user, item and content-based filtering, we calculate the magic number by comparing the accuracy resulting from the use of different numbers for predicting user preferences.

1. Introduction

The Internet environment has changed in recent years based on the development of wireless network technology and the spreading use of mobile devices. In the past, Internet users mostly accessed content created by content providers. Currently, users not only receive content but also access, modify and even produce new content, and they often consume content through mobile devices and wireless networks. For example, on YouTube, users can upload their own video content, watch media content provided by other users, evaluate content and add comments. These user activities are all valuable in providing users with better service via user profiling [1] and content evaluations [2]. In recent years, both the content uploaded to the Internet by users and the content provided by conventional providers has steadily increased. This gives rise to a challenging problem: How can users effectively find what they really want from the vast available content?

Users can rely on recommendation systems to address this problem. A recommendation system effectively provides content, such as movies, books or songs, to users based on various methods that include clustering and profiling of content and users [3,4,5,6,7]. In most recommendation systems, users can expect better recommendations when they provide own user preferences or activity histories for specific items. Then, the recommendation systems identify similar users or items based on user information, such as preferences or purchase histories, and suggest new items based on similar user data or items that are clustered by the systems [8,9,10]. However, every recommendation system has one problem in common, which is when there is a lack of sufficient information to provide a proper recommendation; this problem is called the cold-start problem [11,12]. For instance, assume that a new user joins a movie recommendation system. User activity is invaluable for predicting the user’s preference, but there is little activity on record for a new user, and thus the recommendation system cannot provide customized suggestions. In web or mobile applications, some movie recommendation services, such as Rotten Tomatoes or Netflix, explicitly require new users to provide certain information such as gender, age, location or preferred genres to alleviate this problem. These service providers ask new users to evaluate movies, and thereby infer user preferences. Table 1 shows the number of movies that some popular services ask new users to evaluate.

Table 1. Number of movies that a new user has to evaluate.

Name	No. of Initial Inputs	Homepage URL
Jinni	10	[13]
Criticker	10	[14]
Rotten Tomatoes	10	[15]
MovieLens	15	[16]

As shown in Table 1, each site requires at least ten ratings as initial inputs by new users. This implies an agreement by these services that a certain number of initial inputs is needed to establish meaningful clusters and proper recommendations. However, there is no solid evidence that ten ratings will guarantee good recommendations. This motivates us to consider the following question: How many inputs are necessary for proper movie recommendations? Note that making the initial inputs is a burden on new users. We wonder whether it is absolutely necessary to ask users to make ten ratings since many users will want to spend as little time as possible in the initial process. Another common belief is that the recommendation systems would provide better suggestions if there were more initial inputs. Statistically speaking, when the system recommends movies to users, the more initial inputs exist, the more effectively the system analyzes user preferences.

Herein, we try to verify that requiring more than ten ratings guarantees reliable recommendations. Moreover, we aim to find the minimum number of inputs that will guarantee reliable recommendations. We expect that using the minimum number will guarantee a certain precision in the recommendations and will ease the burden on new users. We define the number of initial inputs that satisfies these conditions to be the magic number for movie recommendations. If one can identify the magic number, then we can reduce user inconvenience while ensuring a certain level of system efficiency.

The approaches for the recommender systems are largely memory-based and model-based [6,8,10,17]. First of all, memory-based approaches are to select similar users or items and use them to predict users’ preferences [6,10]. The model-based approaches are to predict preferences by applying matrix factorization techniques based on a user–item matrix [8,17]. In memory-based approaches, similar users are selected based on their preferences while, in the case of matrix factorization, which means model-based approaches, the entire matrix is then reconstructed. More intuitive comparisons are possible since the input size of user preferences increases the dimensions of vectors used to select similar users. As dimensions have increased, the dimensions of other users’ vectors for calculating similar users also increase. Thus, more intuitive memory-based methods are used in this paper to observe the consequences of changing user input size.

The rest of this paper is organized as follows. In Section 2, we briefly review related research on the cold-start problem in recommendation systems. In Section 3, we introduce the magic number and compute the magic number for movie recommendations; in Section 4, we verify this calculation. Finally, we present our conclusions in Section 5.

2. Related Work

Recent research in the area of recommendation systems has been quite active, with researchers approaching the recommendation accuracy and cold-start problems from various points of view [18,19]. The research concerned with improving accuracy has included collaborative filtering based on clustering methods [6], the roles of users [20] and matrix factorization [17]. The cold-start problem usually occurs for the user-side and the item-side in recommendation systems [21]. Previous work attempted to alleviate the problems by relying on a user’s reputation in a social network group [1,22,23], user classification based on demographic data [19,24,25], a Boltzmann machine in a neural network [26] or item attributes such as categories, creators and actors [27,28]. We can consider the magic number in recommendation systems as one of the alleviative solutions for the user-side cold-start problem. Thus, we briefly summarize recent studies on this problem in Table 2.

Table 2. Studies on the cold-start problem.

Author	Year	Summary
Deldjoo et al. [29]	2019	Propose new movie recommender system that addresses the cold-start problem in the movie domain by integrating item features, exploiting an effective data fusion method
Tu et al. [30]	2019	Propose novel generative model to transfer user interests from app usage behavior to location preference and check the proposed model can reduce user-side cold-start situations
Jazayeriy et al. [31]	2018	Propose a novel similarity measure inspired by a physical resonance phenomenon, named resonance similarity, to address user cold-start problems
Zhang et al. [32]	2017	Propose a kernel-based attribute-aware matrix factorization model to address the cold-start problem for new users by utilizing the social links between users
Zheng et al. [33]	2016	Propose a tourism destination recommender system that employs opinion-mining technology to refine user preferences and item opinion reputations and embed an artificial interactive module in the proposed recommender system to alleviate the cold-start problem
Xu et al. [34]	2016	Propose a novel rating comparison strategy (RAPARE) to learn the latent profiles of cold-start users/items to break the barrier for cold-start users/items
Xu et al. [35]	2015	Propose a novel rating comparison strategy to break user/item cold-start situations
Gogna et al. [36]	2015	Enhance prediction accuracy in cold-start situations of the recommender systems, propose a method that utilizes secondary information such as user’s demography and item categories
Lika et al. [24]	2014	Propose a classification method for demographic data and calculate user similarity.
Bobadilla et al. [37]	2012	Suggest a metric criterion of similarity between new users based on neural learning. The proposed similarity metric criterion provides more precise results than a previous metric criterion.
Zhang et al. [25]	2010	Alleviate user-side cold-start problems using collaborative tagging systems.
Lam et al. [19]	2008	Report a hybrid method that alleviates the user-side cold-start problem through user and item attributes such as year, age and gender.

All the reported methods in the studies listed in Table 2 alleviated the user-side cold-start problem by machine learning algorithms, tagging or the use of item attributes. Nevertheless, several websites, such as MovieLens and Rotten Tomatoes, still require initial inputs from users (recall Table 1), since user inputs are one of the most important features in resolving the cold-start problem. This leads us to investigate the minimum number of user inputs that can guarantee a certain level of reliability of the results.

3. Materials and Methods

3.1. Computing the Magic Number

Most personalized recommendation systems make suggestions based on user preference [38,39] and, often, initially ask users to choose a certain number of items before suggesting new items. This procedure is the first step that a new user takes in making their own user profile to receive better recommendations. A recommendation system provides better suggestions if more initial inputs are available from the user. However, we notice that it is often difficult to ask users to make many such initial inputs because they do not want to spend too much time or effort in this initial procedure to receive better recommendations. This leads us to examine the small number of initial items that can guarantee a certain degree of precision in recommendations for users. We call this number the magic number.

In particular, we consider a movie recommendation system since it is one of the most common recommendation systems based on user profiles and it provides personalized movie recommendations. Herein, we use the GroupLens [40] movie database. We use two types of GroupLens databases: 1 M and 10 M databases. Each database has three sub-datasets: movies, users and ratings. In the database, there are 18 different genres and each movie has a few associated genres. Table 3 and Table 4 summarize the GroupLens database.

Table 3. GroupLens movie database.

Dataset	Attribute	1 M Size	10 M Size
Movie Dataset	MovieID, Title, Genre	3900 movies	10,681 movies
User Dataset	UserID, Gender, Age, Occupation, ZIP-code	6040 users	69,898 users
Rating Dataset	UserID, MovieID, Rating, Timestamp	1,000,209 ratings	10,000,054 ratings

Table 4. The 18 genres in the GroupLens database.

No	Genre	No	Genre	No	Genre
G1	Action	G7	Documentary	G13	Mystery
G2	Adventure	G9	Drama	G14	Romance
G3	Animation	G9	Fantasy	G15	Sci-Fi
G4	Children’s	G10	Film-Noir	G16	Thriller
G5	Comedy	G11	Horror	G17	War
G6	Crime	G12	Musical	G18	Western

A difference between the 1 M and 10 M databases is the number of users and movies. In the 1 M database, there are 3900 movies and 6040 users and 10 M has 10,681 movies and 69,898 users. In the two databases, each user rates at least twenty movies.

We test recommendation systems based on user-based, item-based and content-based filtering. In user-based and item-based filtering, we use Pearson correlation coefficients and cosine similarity for selecting similar users or items [6,10]. In memory-based collaborative filtering approaches, the important part to determine the results of the accuracy of the recommendation is to select similar user groups. Conventionally, Pearson correlation coefficients and cosine similarity are addressed to calculate similarities between users or items. Because of this reason, we utilize those two similarity measures in our analysis to determine the magic number for the recommendation systems. We utilize these processes with the different numbers of users, items and thresholds to identify the magic number. We also apply the database to the recommendation systems based on genre correlations as content-based filtering [21]; the system takes genre combinations as inputs. In other words, new users only need to select preferred genres to receive movie recommendations in this system. In the content-based filtering, we test the error rates and the Pearson correlation coefficient based on the genres extracted from different numbers of movies randomly selected from each user’s choices to identify the magic number. We apply the 1 M database to user and item-based filtering and 10 M to content-based filtering.

Our tests for user and item-based filtering use the data from only those users who have made at least fifty-one ratings and, for content-based filtering, utilize users who have made at least fifty ratings. If we were to use all the users in the database, we would have to limit the maximum number of inputs to twenty ratings. We need to design our tests based on a large number of ratings to justify the accuracy of the magic number; hence, the fifty-one or fifty ratings minimum.

The user- and item-based filtering have similar processes, whereas the content-based filtering has different steps from these two filtering methods. In the user- and item-based filtering, fifty ratings become training data and one rating is a probe for computing the magic number, while the fifty ratings are enough to compute the magic number in the content-based filtering.

3.2. Experiments for Identifying the Magic Number Based on User-Based Filtering

We apply the database to the recommendation systems based on user-based filtering. The recommendation systems have two steps: the first step is to calculate the similarity between users to select similar users and the second step is to calculate the prediction scores based on the similar users’ ratings [6].

In the first step of the system, the similarity is calculated by utilizing the number of users’ inputs. The system conventionally uses the Pearson correlation coefficient and the cosine similarity in calculating similarity between users. After the calculation of the similarity, the system selects similar users based on a threshold that is already decided by the system designer. The second step is calculation of the prediction scores. This step is based on the ratings of similar users, thus, we can consider that the factors of the first step, such as the number of users and thresholds, decide the prediction scores. Because of this reason, we focus on the factors of the first step to identify the magic number.

We have two parameters in our tests: the numbers of inputs and thresholds. We use those who have made at least fifty-one ratings. In these ratings, fifty ratings are training data and one rating is a probe. We consider the training data as initial inputs. Namely, we calculate the similarity by changing the number of ratings. The number of ratings ranges from two to 50. For example, if the number of ratings is two, we can consider the situation that a certain user has two initial inputs. In this situation, the similarity is calculated based on these two ratings. Thus, if we use different ranges, we can observe the prediction score of an item drawn from a variety of ranges of the inputs. Why we use the minimum input as two is because the similarity measures that we used in our tests require two ratings as minimum. If we use one rating, we cannot draw the similarities. Because of this reason, we start with two as the input size. We utilize the Pearson correlation coefficient and cosine similarity to calculate the similarities between users and use the thresholds, ranging from 0.1 to 0.8.

We draw the prediction scores according to these parameters. We compute the magic number by utilizing the prediction scores. Namely, we determine that the number of inputs converged to a consistent point for the prediction scores is the magic number.

In each similarity method, we use same test processes. Figure 1 shows an example of the entire test process.

Figure 1. Entire process for tests of user-based filtering.

In Figure 1, n users are selected as target users and these users have more than fifty-one ratings. We use fifty ratings as inputs and one rating as a probe item. In this Figure, R_n,m means a rating for user n’s item m. The processes for the tests of the recommendation systems based on user-based filtering have a total four steps. The first step is to select a probe user and a probe item. In this step, we randomly select a probe user from the all target users who have fifty-one ratings. Then we pick the fifty-first item as a probe item. In Figure 1, user U₁ is selected as a probe user and item I₅₁ is selected as a probe item. Second, we calculate the similarity between the probe user and other users utilizing the Pearson correlations or cosine similarity. In the third step, we select the similar users who have higher similarity than the thresholds. Finally, we calculate the prediction scores based on similar users’ ratings of the probe item. We apply these steps to a hundred users per the number of movies and provide average prediction scores.

3.2.1. Selecting the Magic Number Based on the Pearson Correlation Coefficient

Figure 2 shows the results of the prediction scores when we use the Pearson correlation coefficient for calculating similarities and Figure 3 shows the sub-graphs in Figure 2. In each graph, the y-axis represents the prediction scores and the x-axis represents the number of ratings (i.e., the number of inputs). Figure 3a–c depict the ranges from 2 to 15, from 16 to 30, and from 31 to 50, respectively. Each graph shows the prediction scores per threshold from 0.1 to 0.8 according to the different number of ratings.

Figure 2. Results of the prediction scores based on the Pearson correlation coefficient in the user-based filtering.

Figure 3. Sub-graphs of Figure 2: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

The reason why each threshold shows different results is concerned with the size of the similar user group. If we increase the thresholds in utilizing the Pearson correlation coefficient to select similar users, the size of the similar user group decreases. Figure 4 shows this phenomenon.

Figure 4. Size of similar user group according to the different numbers of ratings based on the Pearson correlation coefficient.

The different sizes for the similar user group also affect the accuracy of the results. The high thresholds, such as 0.7 and 0.8, show a smaller size of the group than others. It means that the prediction scores are calculated based on a smaller number of ratings than other thresholds. The results of these thresholds are non-consistent, thus, the prediction scores are untrustworthy. Although the accuracy of the prediction scores and the size of the similar user group are important factors in recommendation systems, we only focus on the observations that identify the number of inputs converged to a consistent point for the prediction scores since the accuracy is not an important factor for identifying the magic number.

The results of the sub-graph (a) show larger differences than the sub-graphs (b) and (c). After the start point of the sub-graph (b), each result shows gradual changes. Table 5 shows the average of the prediction scores based on the Pearson correlation coefficient for all thresholds according to the number of movies. In this table, difference means the absolute difference between the prediction scores calculated by m and m − 1 movies. Equation (1) shows more detailed results:

$$D_m=|{Avg}_m - {Avg}_{m-1}|,$$

(1)

where Avg denotes the average for the prediction scores, D_m is the absolute difference and 3 ≤ m ≤ 50 is the range of the differences. The results of Table 5 support our observation. The differences after fifteen movies are more gradual than the range from three to 14. Thus, we can consider fifteen as the magic number.

Table 5. Changing average of prediction and difference for each range (the user-based filtering: the Pearson correlation coefficient).

Input Size	Average Predictions	Difference	No.	Average Predictions	Difference
2	3.6676	-	27	3.6906	0.0016
3	3.7087	0.0411	28	3.6893	0.0013
4	3.7175	0.0088	29	3.6904	0.0011
5	3.7156	0.0019	30	3.6924	0.0020
6	3.7010	0.0146	31	3.6959	0.0035
7	3.6940	0.0070	32	3.6946	0.0013
8	3.6965	0.0026	33	3.6960	0.0013
9	3.7038	0.0073	34	3.6924	0.0036
10	3.7134	0.0096	35	3.6923	0.0001
11	3.7053	0.0081	36	3.6957	0.0034
12	3.7006	0.0048	37	3.6964	0.0007
13	3.7025	0.0019	38	3.6960	0.0004
14	3.7021	0.0004	39	3.6971	0.0011
15	3.7001	0.0019	40	3.6978	0.0006
16	3.6982	0.0019	41	3.6951	0.0027
17	3.7014	0.0032	42	3.6918	0.0032
18	3.6991	0.0023	43	3.6927	0.0008
19	3.7022	0.0031	44	3.6856	0.0070
20	3.6991	0.0031	45	3.6873	0.0017
21	3.7010	0.0019	46	3.6822	0.0051
22	3.6969	0.0041	47	3.6847	0.0025
23	3.6940	0.0029	48	3.6854	0.0007
24	3.6927	0.0013	49	3.6871	0.0016
25	3.6891	0.0036	50	3.6896	0.0025
26	3.6923	0.0032

3.2.2. Selecting the Magic Number Based on the Cosine Similarity

Figure 5 shows the results of the prediction scores when we use the cosine similarity for calculating similarities and Figure 6 shows the sub-graphs of Figure 2. In each graph, the y-axis represents the prediction scores and the x-axis represents the number of ratings (i.e., the number of inputs). The range of the y-axis in Figure 5 and Figure 6 is the same as the one in Figure 2 and Figure 3. This implies that the results of the prediction scores based on the cosine similarity are more consistent than the Pearson correlation coefficient. It is also concerned with the size of the similar user group. Figure 7 shows the size of the similar user group according to the different numbers of ratings.

Figure 5. Results of the prediction score based on the cosine similarity in the user-based filtering.

Figure 6. Sub-graphs of Figure 5: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

Figure 7. Size of similar user group according to the different numbers of ratings based on the cosine similarity.

In Figure 7, all thresholds have similar increasing shapes for the size of the similar user group. The size of the similar user group increases according to increase in the number of inputs. All thresholds have the same trends. Because of this reason, the results are more gradual than in Figure 2.

In Figure 6a, the results ranging from two to 11 show larger differences than the range after twelve. After twelve ratings, each result shows gradual changes. Table 6 shows the average of the prediction scores based on the cosine similarity for all thresholds. We use Equation (4) to calculate the difference in Table 6. The results of Table 6 support that the magic number is twelve since we can see that the differences are gradual after twelve.

Table 6. Changing average of prediction and difference for each range (the user-based filtering: the cosine similarity).

Input Size	Average Predictions	Difference	No.	Average Predictions	Difference
2	3.7542	-	27	3.7241	0.0008
3	3.7373	0.0169	28	3.7268	0.0027
4	3.7335	0.0038	29	3.7262	0.0006
5	3.7231	0.0104	30	3.7278	0.0016
6	3.7137	0.0094	31	3.7258	0.0020
7	3.7238	0.0101	32	3.7239	0.0019
8	3.7195	0.0044	33	3.7235	0.0004
9	3.7227	0.0032	34	3.7227	0.0007
10	3.7225	0.0001	35	3.7238	0.0010
11	3.7266	0.0040	36	3.7232	0.0006
12	3.7202	0.0064	37	3.7244	0.0012
13	3.7162	0.0040	38	3.7243	0.0001
14	3.7160	0.0002	39	3.7248	0.0005
15	3.7113	0.0048	40	3.7269	0.0021
16	3.7083	0.0030	41	3.7269	0.0000
17	3.7117	0.0034	42	3.7246	0.0022
18	3.7108	0.0009	43	3.7231	0.0016
19	3.7095	0.0012	44	3.7234	0.0003
20	3.7124	0.0029	45	3.7227	0.0007
21	3.7135	0.0011	46	3.7235	0.0008
22	3.7154	0.0019	47	3.7240	0.0004
23	3.7150	0.0003	48	3.7225	0.0015
24	3.7215	0.0065	49	3.7217	0.0007
25	3.7241	0.0026	50	3.7230	0.0013
26	3.7250	0.0009

Thus, we can consider that twelve is the magic number.

3.3. Experiments for Identifying the Magic Number Based on Item-Based Filtering

The recommendation systems based on the item-based filtering have similar processes to user-based filtering. The difference between the two systems is the targets used to calculate the similarity methods. In user-based filtering, users become targets for calculating similarity, whereas items are utilized as targets in item-based filtering. Figure 8 shows the entire test process in item-based filtering.

Figure 8. Entire process for tests of the recommendation systems based on the item-based filtering.

In Figure 8, n items are selected as targets and these items have more than fifty-one ratings. Thus, there exists n items and each item has 51 ratings. R_n,m means a rating for the nth user’s mth item. We can see that the criterion for calculating similarity in the user-based filtering is a user, while the system based on item-based filtering addresses an item as the criterion in calculating similarity through Figure 1 and Figure 8.

The processes for the tests of the recommendation systems based on item-based filtering also have a total of four steps. The first step is to select a probe item and a probe user. In this step, we randomly select a probe item from all target items that have fifty-one ratings. Then we pick the fifty-first user as a probe user. In Figure 8, item I₁ is selected as a probe item and user U₅₁ is selected as a probe user. Second, we calculate the similarity between the probe item and other items using the Pearson correlations or cosine similarity. In the third step, we select the similar items that have a greater similarity than the thresholds. Finally, we calculate the prediction scores based on similar items’ ratings of the probe user’s probe item. We use same parameters as user-based filtering in this test. We apply these steps to a hundred users per the number of movies.

3.3.1. Selecting the Magic Number Based on the Pearson Correlation Coefficient

Figure 9 shows the results of the prediction scores when we use the Pearson correlation coefficient for calculating similarities and Figure 10 shows the sub-graphs in Figure 9. In each graph, the y-axis and the x-axis are same as the graphs in Section 3.2.1. Figure 3a–c depict the ranges from 2 to 15, from 16 to 30 and from 31 to 50, respectively. Each graph shows the prediction scores per threshold from 0.1 to 0.8 according to the different numbers of ratings. The reason why each threshold shows different results is the same as for the user-side filtering. Figure 11 shows the size of the similar user group.

Figure 9. Results of the prediction scores based on the Pearson correlation coefficient in the item-based filtering.

Figure 10. Sub-graphs of Figure 9: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

Figure 11. Size of similar user group according to the different numbers of ratings based on the Pearson correlation coefficient.

Compared to the results in Section 3.2.1, the prediction scores are consistent ahead of the magic number in the user-side. The user-side filtering based on the Pearson correlation coefficient has a point of fifteen as the magic number. Although the system uses the same similarity measure, the item-side filtering shows different points from the user-side. In the sub-graph (a) of Figure 10, the results ranging from two to 11 show larger differences than the range after twelve. Namely, the trends of the prediction scores are similar to the cosine similarity of the user-side. Thus, we also decide twelve is the magic number from the Pearson correlation coefficient of the item-side.

Table 7 shows the average of prediction scores and differences for each range based on the Pearson correlation coefficient. In this table, we show the gradual change of the average difference after twelve that is selected as the magic number.

Table 7. Changing average of prediction and difference for each range (the item-based filtering: the Pearson correlation coefficient).

Input Size	Average Predictions	Difference	No.	Average Predictions	Difference
2	3.0839	-	27	3.3790	0.0098
3	3.1687	0.0848	28	3.3727	0.0063
4	3.2214	0.0527	29	3.3733	0.0006
5	3.2765	0.0551	30	3.3772	0.0038
6	3.2989	0.0224	31	3.3828	0.0057
7	3.2941	0.0048	32	3.3802	0.0027
8	3.3112	0.0171	33	3.3769	0.0033
9	3.3249	0.0136	34	3.3739	0.0030
10	3.3382	0.0133	35	3.3671	0.0068
11	3.3467	0.0086	36	3.3686	0.0015
12	3.3531	0.0063	37	3.3676	0.0010
13	3.3477	0.0053	38	3.3717	0.0040
14	3.3489	0.0012	39	3.3769	0.0053
15	3.3503	0.0014	40	3.3758	0.0011
16	3.3548	0.0045	41	3.3749	0.0009
17	3.3612	0.0064	42	3.3716	0.0033
18	3.3617	0.0005	43	3.3707	0.0009
19	3.3602	0.0015	44	3.3683	0.0023
20	3.3614	0.0012	45	3.3668	0.0016
21	3.3591	0.0023	46	3.3686	0.0019
22	3.3568	0.0023	47	3.3716	0.0030
23	3.3603	0.0035	48	3.3725	0.0008
24	3.3691	0.0089	49	3.3738	0.0013
25	3.3650	0.0041	50	3.3759	0.0021
26	3.3692	0.0042

3.3.2. Selecting the Magic Number Based on the Cosine Similarity

Figure 12 shows the results of prediction scores when we use the cosine similarity for calculating similarities and Figure 13 shows the sub-graphs of Figure 12. The y-axis and x-axis in each graph are the same as in Section 3.2.1. We can show the similar trends for the size of the similar user group per the number of movies with the results of the user-based filtering using the cosine similarity in Figure 14.

Figure 12. Results of the prediction scores based on the cosine similarity in the item-based filtering.

Figure 13. Sub-graphs of Figure 12: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

Figure 14. Size of similar user group according to the different numbers of ratings based on the cosine similarity.

The converged point of each threshold is similar to the graphs in the previous section. Namely, the prediction scores show large changes before twelve. Table 8 shows changes in the average of the prediction scores based on the cosine similarity and the average difference for each range. The trends showing that the average differences are gradual after twelve in this table support our claim. Thus, the magic number is twelve.

Table 8. Changing average of prediction and difference for each range (the item-based filtering: the cosine similarity).

Input Size	Average Predictions	Difference	No.	Average Predictions	Difference
2	3.1261	-	27	3.3332	0.0013
3	3.1681	0.0420	28	3.3334	0.0003
4	3.2150	0.0469	29	3.3337	0.0003
5	3.2598	0.0449	30	3.3356	0.0018
6	3.2804	0.0206	31	3.3354	0.0002
7	3.2915	0.0111	32	3.3346	0.0008
8	3.3090	0.0174	33	3.3342	0.0004
9	3.3168	0.0078	34	3.3335	0.0007
10	3.3191	0.0024	35	3.3321	0.0014
11	3.3222	0.0030	36	3.3329	0.0009
12	3.3247	0.0025	37	3.3333	0.0004
13	3.3244	0.0003	38	3.3335	0.0002
14	3.3273	0.0029	39	3.3322	0.0013
15	3.3319	0.0046	40	3.3311	0.0011
16	3.3351	0.0032	41	3.3306	0.0005
17	3.3325	0.0026	42	3.3289	0.0017
18	3.3317	0.0008	43	3.3290	0.0001
19	3.3315	0.0002	44	3.3304	0.0013
20	3.3328	0.0014	45	3.3308	0.0005
21	3.3344	0.0016	46	3.3309	0.0001
22	3.3348	0.0004	47	3.3321	0.0012
23	3.3350	0.0002	48	3.3334	0.0013
24	3.3325	0.0025	49	3.3335	0.0001
25	3.3297	0.0028	50	3.3341	0.0006
26	3.3319	0.0022

3.4. Experiments for Identifying the Magic Number Based on Content-Based Filtering

We search for the magic number by analyzing genre preferences for each user in content-based filtering. In other words, we need to compute the number of initial inputs that can reflect the user’s preferences among movie genres. For each user, we compute the genre distribution of movies rated by the user for different numbers of movies selected from all the movies rated by that user. We consider the genre distribution of all movies evaluated by a user to be that user’s true genre preference. Once we have completed the calculation, for each user, we compute the Pearson correlation coefficient and error rates (we provide a formal definition later) between the user’s preference and the estimated preference using a limited number of movies. Based on the analysis of the computation results, we suggest the magic number.

3.4.1. Computing Genre Preferences for Users

For each user, among all the movies rated by that user, we randomly select a number of movies from one to forty, and count the total number of genre appearances among all the selected movies. We call the total number of appearances by each genre the genre frequency. We normalize the genre frequency as a percentage of all the genres’ appearances. Then, the frequency percentage is taken as the genre preference of a user. Figure 15 shows an example computation of a user’s preference.

Figure 15. Example of computing the genre frequency and frequency percentage for a user.

In the example in Figure 15, the user has rated four movies whose genres are G₁, G₂ and G₃. The genre frequencies of G₁, G₂ and G₃ are two, three and three, respectively. The ratio of the genre frequency is the frequency percentage. Thus, a large percentage or high frequency for a specific genre implies that a user strongly prefers that genre.

3.4.2. Computing Error Rates

We now calculate the error rates of genre frequency of randomly selected groups of movies with respect to the true genre preference computed from all movie genres for each user. We rely on the root mean square error (RMSE) for computing the error rates between two percentage groups. RMSE is often used for measurement of the difference between the actual values and the values predicted by a model [41]. We slightly modify the RMSE formula for computing the error rate. We study two types of frequency percentages for the eighteen genres: the first type is based on all the movies selected by a user and the second type is based on movies randomly selected from all the movies. Equation (2) is the formula for calculating the error rate between the two types:

$${\mathrm{ER}}_k=\sqrt{\frac{1}{18}{\displaystyle \sum _{n=1}^{18}{{(}_o{G}_n-k{G}_n)}^2}},$$

(2)

where _oG_n and kG_n are the nth frequency percentage among the eighteen frequency percentages extracted from all movies and extracted from the k randomly selected movies, respectively.

Figure 16 shows an example of computing the error rate for Movie₁ from Figure 15. In this case, Movie₁ is an initial input. If there is a significant difference between two percentage groups, this implies that the initial input size, which is the number of randomly selected movies in our experiments, cannot properly reflect the genre preferences of a user. On the other hand, if the error rate is not large, then we may conclude that the randomly selected movies reflect the user preference. In our experiments, we randomly select from one to forty movies and calculate the error rates for all users, repeating this process one hundred times and computing the average error rates.

Figure 16. Procedure example of calculating error rates.

Figure 17 shows the relationship between the error rate and the number of randomly selected movies from one to forty. The sub-graphs (a), (b) and (c) in Figure 17 show the error rates for the ranges from one to five, from six to ten and from eleven to fifteen, respectively. In each graph in Figure 17, the y-axis represents the error rates obtained from Equation (2) and the x-axis represents the number of randomly selected movies. Notice the decrease in the gradient in Figure 17 after five movies.

Figure 17. Error rates for different sample sizes.

We next consider the difference in error rates between two adjacent ranges in Figure 10 using Equation (3):

$$D{i}_m={\mathrm{ER}}_m-{\mathrm{ER}}_{m-1},$$

(3)

where Di_m is the difference between the error rates based on m movies and based on m − 1 movies and 2 ≤ m ≤ 40 is the number of randomly selected movies.

Figure 18 shows the rates of decrease in the error rate for the data shown in Figure 17. The y-axis represents the difference between the error rates of the two consecutive intervals and the x-axis represents the number of randomly selected movies. We observe that the rate of change in the error rate decreases as the number of randomly selected movies increases. This implies that additional inputs beyond a certain number of movies will not greatly affect the recommendation results.

Figure 18. Rates of decrease in error rates.

3.4.3. Computing the Pearson Correlation Coefficients

We determine the Pearson correlation coefficient using the genre frequency [42]. The test procedure is similar to the test procedure in Section 3.3.2; we calculate the error rates using frequency percentages, and here we use genre frequency to calculate the Pearson correlation coefficient. For example, if the randomly selected movie is Movie₁ from Figure 8, then we determine the correlations between the genre frequencies for Movie₁ and all movies. See Figure 19 for an example of the procedure.

Figure 19. Example of the procedure for calculating the Pearson correlation coefficient.

We calculate the correlations between the frequencies of the randomly selected movies and those of all movies rated by a user. For our experiments, we randomly select from one to forty movies and calculate the correlations, repeating this process one hundred times and computing the average correlations.

Figure 20 shows the trends in the correlation coefficient for different numbers of randomly selected movies. The sub-graphs (a), (b) and (c) in Figure 20 depict the ranges from 1 to 5, from 6 to 10 and from 11 to 15, respectively. In each graph in Figure 20, the y-axis represents the correlation coefficient and the x-axis represents the number of randomly selected movies. In other words, Figure 20 shows the correlations between the genre frequencies of randomly selected movies and those of all movies rated by each user.

Figure 20. Pearson correlation coefficient for different random sample sizes.

Table 9 provides the computed values shown in Figure 20. In Table 9, No. and Corr. denote the number of movies and the Pearson correlation coefficient, respectively. As shown in Table 9 and the sub-graphs in Figure 20, the rates of increase in the correlation coefficient are greater when the number of movies is below five than when there are more than five movies. This implies that the distribution of the genre frequency when the number of randomly selected movies becomes five is similar to the genre distribution for all movies.

Table 9. Correlation coefficients for each random sample size.

No.	Corr.	No.	Corr.	No.	Corr.	No.	Corr.
1	0.4311	11	0.8485	21	0.9186	31	0.9487
2	0.5527	12	0.8605	22	0.9232	32	0.9507
3	0.6305	13	0.8701	23	0.9269	33	0.9527
4	0.6833	14	0.8778	24	0.9304	34	0.9543
5	0.7263	15	0.8862	25	0.9334	35	0.9560
6	0.7565	16	0.8929	26	0.9391	36	0.9576
7	0.7830	17	0.8995	27	0.9391	37	0.9592
8	0.8027	18	0.9053	28	0.9417	38	0.9607
9	0.8206	19	0.9102	29	0.9440	39	0.9623
10	0.8359	20	0.9144	30	0.9460	40	0.9636

3.4.4. Selecting the Magic Number

Based on the two experiments in Section 3.4.2 and Section 3.4.3, we claim that five is the magic number for movie recommendations. We have two reasons for making this claim.

• After five, the rates of change decrease in all graphs in Figure 17, Figure 18 and Figure 20. This signifies that as the number of movies as initial inputs increases over five, the increase in the reliability of the recommendation results is gradual. In other words, even if a user selects more than five movies, we cannot expect a significant increase in the reliability of recommendation results; five movies are already enough to guarantee a certain level of reliability. Functional modeling focuses on what tasks are being performed, and what informational elements are involved in these tasks.
• The rate of increase in the correlation coefficient becomes small after five. The selection of more than five movies hardly affects the results of the recommendations since there are almost no decreases in error rates and the correlations.

Because of these two observations, we claim that five is the magic number.

4. Results

4.1. Verifying the Magic Number

We test the usability of the magic number by applying the magic number to a recommendation system based on user-based, item-based and content-based filtering; the recommendation system using user- or item-based filtering utilizes users’ ratings as its initial inputs, whereas using content-based filtering (the recommendation systems based on genre correlations) employs a genre.

We verify the magic number determined by Section 3 in real situations. The systems based on user- or item-based filtering consists of similar processes to recommend items, whereas the content-based filtering has a diametrically opposed process. Thus, we address similar tests for user- and item-based filtering and verify the usability of the content-based filtering by utilizing a different method from the other two filtering approaches.

We draw the mean absolute error (MAE) for the predicted items to verify the usability of user- or item-based filtering [6,10,41]. Namely, we check the difference between the predicted ratings and the real ratings found through the different numbers of inputs. We also use from two to 50 items as input sizes and repeat our test processes for one hundred users.

We carry out the tests for the content-based filtering using the characteristics of the recommendation systems based on the genre correlations. The system does not consider genre weights or input orders. Thus, if different users provide the same genre combination as the recommendation system, then the results will yield the same recommendation lists regardless of input orders. Thus, we can determine from the input genres whether the recommendation results are the same. In other words, instead of comparing the system output, we compare the inputs to verify the usability of the magic number. For experiments on content-based filtering, we randomly select from one to forty movies and apply our methods to each selected movie. We repeat this step one hundred times and provide average results.

4.2. The Verification of the Magic Number Based on User-Based Filtering

We use similar processes as in Section 3.1 to verify the magic number. The test process in this section has only one more step than the process in Section 3.1. That is the step for calculating the MAE value. The result of the MAE shows the accuracy of a recommendation score. Figure 21 shows the entire verification process of the user-based filtering.

Figure 21. Verification process of the user-based filtering.

In Figure 21, a fifth step is added to Figure 1. R_1,51 means the rating of user U₁’s item I₅₁ and P_1,51 is the predicted score for this rating. Thus, we can check the accuracy of the predicted score by calculating the absolute difference between the two values. One important factor in this test is that we calculate the MAE for the predicted scores in Section 3.1. Namely, we apply our verification tests to the same users and items as in Section 3.1. Although we can show similar results when we apply the MAE to other probe items or users followed by the central limit theorem, we can expect more precise results and comparisons for the magic number by applying the MAE to the same probe sets [42].

4.2.1. Results and Analysis of the Verification Procedure for the Pearson Correlation Coefficient

Figure 22 shows MAE values for the different numbers of inputs when we use the Pearson correlation coefficient to calculate the similarities between users and Figure 23 shows the sub-graphs of Figure 22. In each graph, the y-axis is the MAE and the x-axis is the different numbers of ratings. Figure 23a–c depict the ranges from 2 to 15, from 16 to 30 and from 31 to 50, respectively.

Figure 22. Mean absolute error (MAE) values for the different numbers of ratings based on the Pearson correlation coefficient in the user-based filtering.

Figure 23. Sub-graphs of Figure 16: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

The MAE is generally used in checking the accuracy of the recommendation results. Figure 22 shows that the MAE values have the most precise results when the threshold is 0.3. However, we do not focus on the accuracy of the results but on identifying the points that converge on a specific value. Thus, we concentrate on the points before the gradual changes of MAE values.

In Section 3.1, the prediction scores have consistency after fifteen ratings. If there are gradual changes for MAE values after fifteen ratings, we can expect that a prediction score is no longer close to a real rating. Namely, more inputs than the magic number do not guarantee an increase in the accuracy of the recommendation results.

MAE values of Figure 23a have larger changes than the other two graphs. This leads to the same conclusion as the selection in Section 3.2.1. Namely, we can guarantee that fifteen ratings can reduce user inconvenience while also providing reliable results based on the Pearson correlation coefficient as the magic number. Table 10 shows change in the average of the prediction scores based on the Pearson correlation coefficient and the average difference for each range. After the magic number, the differences show slight changes. Thus, our claim that the magic number is twelve in Section 3.2.1 is valid.

Table 10. Change in the average of MAE and average difference for each range (the user-based filtering: the Pearson correlation coefficient).

Input Size	Average MAE	Difference	No.	Average MAE	Difference
2	0.9852	-	27	0.7893	0.0018
3	0.9298	0.0554	28	0.7893	0.0000
4	0.8909	0.0389	29	0.7871	0.0021
5	0.8580	0.0329	30	0.7864	0.0007
6	0.8401	0.0179	31	0.7862	0.0003
7	0.8354	0.0047	32	0.7846	0.0016
8	0.8245	0.0109	33	0.7893	0.0047
9	0.8222	0.0024	34	0.7880	0.0013
10	0.8178	0.0044	35	0.7870	0.0010
11	0.8147	0.0031	36	0.7883	0.0013
12	0.8045	0.0102	37	0.7898	0.0014
13	0.7996	0.0049	38	0.7882	0.0016
14	0.7994	0.0002	39	0.7923	0.0041
15	0.8020	0.0025	40	0.7958	0.0035
16	0.7976	0.0044	41	0.7941	0.0017
17	0.7975	0.0001	42	0.7938	0.0003
18	0.7979	0.0004	43	0.7926	0.0012
19	0.7929	0.0049	44	0.7935	0.0009
20	0.7912	0.0018	45	0.7921	0.0014
21	0.7917	0.0005	46	0.7945	0.0024
22	0.7919	0.0002	47	0.7963	0.0019
23	0.7905	0.0014	48	0.7953	0.0010
24	0.7937	0.0033	49	0.7968	0.0015
25	0.7897	0.0041	50	0.7952	0.0017
26	0.7911	0.0015

4.2.2. Results and Analysis of the Verification Procedure for the Cosine Similarity

Figure 24 shows MAE values for the different numbers of inputs when we use the cosine similarity to calculate the similarities between users and Figure 25 shows the sub-graphs of Figure 24. All axes and ranges in both figures are same as in the graphs in Section 4.2.1.

Figure 24. MAE values for the different numbers of ratings based on the cosine similarity in the user-based filtering.

Figure 25. Sub-graphs of Figure 24: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

In Section 3.2.2, the prediction scores have consistency after twelve movies. The MAE values per the number of movies are consistent after the magic number decided on in Section 3.2.2. Table 11 shows the change in the average of the MAE and the average difference for each range when we use the cosine similarity. This also supports our decision, and thus, we can use twelve as the magic number in this similarity measure for the user-based filtering.

Table 11. Change in the average of MAE and average difference for each range (the user-based filtering: the cosine similarity).

Input Size	Average MAE	Difference	No.	Average MAE	Difference
2	0.9204	-	27	0.7661	0.0015
3	0.8700	0.0504	28	0.7662	0.0001
4	0.8195	0.0505	29	0.7670	0.0007
5	0.8127	0.0068	30	0.7662	0.0008
6	0.8198	0.0071	31	0.7645	0.0017
7	0.8075	0.0123	32	0.7661	0.0015
8	0.8041	0.0034	33	0.7658	0.0003
9	0.7987	0.0055	34	0.7649	0.0009
10	0.7972	0.0015	35	0.7655	0.0006
11	0.7909	0.0063	36	0.7646	0.0009
12	0.7878	0.0031	37	0.7631	0.0015
13	0.7818	0.0060	38	0.7627	0.0003
14	0.7772	0.0046	39	0.7635	0.0007
15	0.7722	0.0051	40	0.7637	0.0002
16	0.7745	0.0023	41	0.7646	0.0009
17	0.7765	0.0020	42	0.7629	0.0017
18	0.7726	0.0039	43	0.7620	0.0009
19	0.7730	0.0004	44	0.7614	0.0006
20	0.7686	0.0044	45	0.7609	0.0005
21	0.7683	0.0003	46	0.7613	0.0004
22	0.7673	0.0010	47	0.7616	0.0002
23	0.7650	0.0023	48	0.7612	0.0003
24	0.7642	0.0008	49	0.7604	0.0008
25	0.7676	0.0034	50	0.7607	0.0003
26	0.7676	0.0000

4.3. The Verification of the Magic Number Based on Item-Based Filtering

The test in this section is similar to that in Section 4.1. In Section 4.1, the test for the verification of the magic number based on the user-side filtering has one more step than the test in Section 3.2. The test for the verification of the magic number based on the item-side filtering also has one more step, which is the step for calculating the MAE value. Figure 26 shows the entire verification process of the item-based filtering. In Figure 26, a fifth step is added to Figure 1. R_51,1 means the rating of user U₅₁’s item I₁ and P_51,1 is the predicted score for this rating. Thus, we can check the accuracy of the predicted score by calculating the absolute difference between two values. One important factor in this test is that we calculate the MAE for the predicted scores in Section 3.2. Namely, we apply our verification tests to same users and items as in Section 3.2.

Figure 26. Verification process of the item-based filtering.

4.3.1. Results and Analysis of the Verification Procedure for the Pearson Correlation Coefficient

Figure 27 shows MAE values for the different numbers of inputs when we use the Pearson correlation coefficient to calculate the similarities between users and Figure 28 shows the sub-graphs of Figure 27. All ranges and axes in both figures are the same as in Figure 24 and Figure 25.

Figure 27. MAE values for the different numbers of ratings based on the Pearson correlation coefficient in the item-based filtering.

Figure 28. Sub-graphs in Figure 27: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

In the sub-graph (a), the MAE values sharply decrease before twelve movies. This means that twelve can assure the consistency of the results. Namely, more inputs than twelve movies no longer improve the accuracy of the results. Table 12 shows the change in the average of the MAE based on the cosine similarity and the average difference for each range. After the magic number, the differences have slight changes. Thus, the magic number is twelve.

Table 12. Change in the average of MAE and difference for each range (the item-based filtering: the Pearson correlation coefficient).

Input Size	Average MAE	Difference	No.	Average MAE	Difference
2	1.0058	-	27	0.7382	0.0025
3	0.9072	0.0986	28	0.7406	0.0024
4	0.8766	0.0306	29	0.7414	0.0008
5	0.8590	0.0177	30	0.7377	0.0037
6	0.8320	0.0270	31	0.7388	0.0012
7	0.8167	0.0152	32	0.7351	0.0038
8	0.8110	0.0057	33	0.7384	0.0034
9	0.8006	0.0104	34	0.7394	0.0009
10	0.7847	0.0159	35	0.7372	0.0022
11	0.7783	0.0063	36	0.7396	0.0024
12	0.7816	0.0032	37	0.7439	0.0043
13	0.7750	0.0066	38	0.7402	0.0037
14	0.7720	0.0029	39	0.7364	0.0038
15	0.7688	0.0033	40	0.7358	0.0006
16	0.7653	0.0035	41	0.7399	0.0041
17	0.7622	0.0031	42	0.7422	0.0023
18	0.7589	0.0033	43	0.7452	0.0030
19	0.7520	0.0069	44	0.7473	0.0021
20	0.7504	0.0016	45	0.7474	0.0001
21	0.7522	0.0018	46	0.7454	0.0020
22	0.7514	0.0008	47	0.7454	0.0001
23	0.7453	0.0061	48	0.7485	0.0031
24	0.7401	0.0051	49	0.7479	0.0006
25	0.7448	0.0047	50	0.7464	0.0015
26	0.7407	0.0041

4.3.2. Results and Analysis of the Verification Procedure for the Cosine Similarity

Figure 29 shows the MAE values for the different numbers of inputs when we use the cosine similarity and Figure 30 shows the sub-graphs of Figure 29. In each graph, all axes and the ranges are the same as in the graphs in the previous section.

Figure 29. MAE values for the different numbers of ratings based on the cosine similarity in the item-based filtering.

Figure 30. Sub-graphs in Figure 29: (a) the ranges from 2 to 15; (b) from 16 to 30; (c) from 31 to 50.

In Section 3.2.2, the prediction scores have consistency after twelve ratings. If there are gradual changes for the MAE values after twelve ratings, we can expect that a prediction score is no longer close to a real rating. Namely, more inputs than the magic number do not guarantee an increase in the accuracy of the recommendation results.

MAE values ranging from two to 11 in Figure 30a show larger changes than after the magic number decided on in Section 3.2.2. Table 13 shows the change in the average of the MAE and differences for each range based on the item-based filtering. This table also shows the validation of our decision that the magic number is twelve in the item-based filtering based on the cosine similarity. Namely, in Table 13, it was confirmed that the decreasing rate of average MAE values was reduced before the magic number.

Table 13. Change in the average MAE and average difference for each range (the item-based filtering: the cosine similarity).

Input Size	Average MAE	Difference	No.	Average MAE	Difference
2	0.9968	-	27	0.7631	0.0007
3	0.9265	0.0703	28	0.7627	0.0004
4	0.8816	0.0448	29	0.7618	0.0009
5	0.8599	0.0217	30	0.7611	0.0007
6	0.8374	0.0226	31	0.7609	0.0002
7	0.8172	0.0202	32	0.7611	0.0002
8	0.8027	0.0145	33	0.7590	0.0021
9	0.7998	0.0029	34	0.7603	0.0013
10	0.7937	0.0060	35	0.7596	0.0008
11	0.7863	0.0074	36	0.7585	0.0011
12	0.7856	0.0007	37	0.7584	0.0001
13	0.7842	0.0014	38	0.7598	0.0014
14	0.7814	0.0029	39	0.7608	0.0010
15	0.7802	0.0012	40	0.7613	0.0005
16	0.7795	0.0007	41	0.7624	0.0011
17	0.7771	0.0024	42	0.7619	0.0004
18	0.7748	0.0023	43	0.7628	0.0009
19	0.7742	0.0007	44	0.7634	0.0006
20	0.7730	0.0012	45	0.7620	0.0014
21	0.7711	0.0019	46	0.7626	0.0006
22	0.7705	0.0007	47	0.7623	0.0003
23	0.7703	0.0001	48	0.7645	0.0022
24	0.7662	0.0041	49	0.7641	0.0004
25	0.7641	0.0021	50	0.7641	0.0000
26	0.7638	0.0003

4.4. The Verification the Magic Number Based on Content-Based Filtering

4.4.1. Design of the Verification Procedure

We first determine whether the three most frequent genres in a list of randomly selected movies coincide with the most frequent genres among all the movies rated by a user. If there are ties among the most frequent genres, we randomly select three genres among them as the top three. See Figure 31 for an example. In the example shown in Figure 31, a user has selected three movies (Movie₁, Movie₂ and Movie₃), which give rise to five genres (G₁, G₂, G₃, G₄ and G₅). Note that the genres G₁, G₂, G₃, G₄ and G₅ have three, three, one, three and three as their respective genre frequencies. Since there are four genres with the same top frequency, we randomly select three genres among these four genres. In this example, genres G₁, G₂ and G₅ are randomly selected as the top three.

Figure 31. Example of selecting the top three genres.

We assume that if these top three genres include the user’s top preferred genres, then these three genres reflect the user’s preferences. Equation (4) shows how to calculate the scores for the results of comparisons between the top three genres in this sample and the top preferred genres overall:

$$\mathrm{S}=\{\begin{array}{cc}1& \mathrm{if}\left|{HP}_G\right|=1,|{HP}_G \cap {RS}_G|\ne 0\\ \frac{|{HP}_G \cap {RS}_G|}{\left|{HP}_G\right|}& \mathrm{if}\left|{HP}_G\right|>1,|{HP}_G \cap {RS}_G|\ne 0\\ 0& \mathrm{otherwise}\end{array}$$

(4)

where HP_G is the set of the top preferred genres and RS_G is a set of top three genres extracted from a randomly selected list of movies. HP_G means the genre that has appeared most frequently in user-selected films.

Figure 32 illustrates an example of calculating the score using Equation (4). In Figure 32, Case-1 and Case-2 are when |HP_G| = 1 and |HP_G| > 1, respectively. More details on each case are as follows:

Figure 32. Example of calculating the score using Equation (3).

• Case-1: A user has selected n movies in total. The top preferred genre is G₂. We randomly select three movies (Movie₂, Movie₃ and Movie₅). The top three genres calculated from these selected movies are G₁, G₂ and G₁₈. Finally, we check whether these top three genres include the top preferred genre. The three genres G₁, G₂ and G₁₈ include the top genre G₂, and thus, the score is 1.
• Case-2: A user has selected m movies in total. The top preferred genres are G₁ and G₂. We randomly select three movies (Movie₁, Movie₄ and Movie₅). The top three genres calculated from these selected movies are G₂, G₁₇ and G₁₈. Finally, we check whether these top three genres include the top preferred genres. The three genres G₂, G₁₇ and G₁₈ include only G₂ among the top preferred genres, and thus, the score is 0.5.

4.4.2. Results and Analysis of the Verification Procedure

We apply this test to all users. Note that the number of randomly selected movies is the input size. Therefore, the results of this test can show whether or not the number of selected movies reflects the user’s preferences as computed from all rated movies. Figure 33 shows the results of this test.

Figure 33. Accuracy of different random sample sizes.

Figure 33 shows the accuracy of different numbers of randomly selected movies. Figure 33a–c depict the ranges from 1 to 5, from 6 to 10 and from 11 to 15, respectively. In each graph in Figure 33, the y-axis represents the accuracy and the x-axis represents the number of randomly selected movies. We notice that the sharpest increase occurs in the range 1 to 5. This means that the increase is gradual after five, which is the magic number claimed herein. We also consider Equation (5) for more detailed analysis of the results:

$${VR}_m=|{AC}_m-{AC}_{m-1}|,$$

(5)

where AC denotes accuracy, VR_m is the absolute difference between the accuracies calculated using m and m − 1 movies and 2 ≤ m ≤ 40 is the number of randomly selected movies. Table 14 provides the computed values shown in Figure 8 and their corresponding differences with each increase in the number of movies selected.

Table 14. Accuracy change and average difference for each range.

No.	Accuracy	Difference	No.	Accuracy	Difference
1	0.5478	-	21	0.9403	0.0041
2	0.6517	0.1041	22	0.9437	0.0034
3	0.6806	0.0287	23	0.9470	0.0033
4	0.7307	0.0501	24	0.9500	0.0030
5	0.7705	0.0399	25	0.9529	0.0029
6	0.7958	0.0253	26	0.9554	0.0024
7	0.8170	0.0213	27	0.9579	0.0025
8	0.8358	0.0188	28	0.9600	0.0021
9	0.8518	0.0159	29	0.9623	0.0023
10	0.8649	0.0132	30	0.9644	0.0021
11	08762	0.0113	31	0.9662	0.0018
12	0.8866	0.0104	32	0.9678	0.0016
13	0.8954	0.0088	33	0.9694	0.0016
14	0.9034	0.0080	34	0.9710	0.0016
15	0.9102	0.0068	35	0.9724	0.0014
16	0.9164	0.0062	36	0.9738	0.0014
17	0.9222	0.0058	37	0.9750	0.0012
18	0.9275	0.0053	38	0.9736	0.0013
19	0.9321	0.0046	39	0.9772	0.0010
20	0.9363	0.0042	40	0.9782	0.0010

It can be seen in Table 14 that the accuracies increase steadily with the number of movies selected. However, the tendency of this increase is gradual after five. We can expect 55.78% accuracy in the recommendation results for the results when we input one movie, compared with 77.05% for five movies. The difference between these two accuracies is 21.27%; that is to say, a user can expect 21.27% better accuracy from the results by providing an additional four movies to the system as inputs. To obtain an equal increase in accuracy after five movies would require providing more than thirty-five extra movies, since forty movies provide 97.82% accuracy. Although the accuracy is observed to improve, when more than five movies are selected, the degree of improvement diminishes beyond the magic number. Therefore, when a user selects five movies, which is the magic number, the recommendation system can provide good recommendation results.

5. Conclusions

Recommendation systems based on collaborative filtering should analyze user preferences to provide better suggestions to users. In many applications on web or mobile devices providing recommendation or curation services to users, the services ask the user for some information, such as demographic information, preferred category and more than the number of ratings for items provided in the services. However, many users may be inconvenienced by the requirement that they provide initial inputs to define their preferences. Moreover, new users must input their preferences to receive recommendations. If a system requires large amounts of input from new users, those users may become discouraged before using the recommendation system.

The magic number proposed herein can provide technical guidance in addressing the cold-start problem between users and systems when the systems analyze user preferences. We have claimed that the magic number represents different numbers of movies according to the filtering algorithms: user-, item- and content-based. Table 15 shows the identified magic number in our experiments; namely, when a user provides fifteen, twelve or five movies as initial inputs to the system based on user-, item- or content-based filtering, the recommendation system can provide sufficiently reliable results to the user. We have justified the reliability of the magic number through statistical experiments such as analyzing the prediction scores based on the Pearson correlation coefficient and the cosine similarity, MAE and error rates with different numbers of movies.

Table 15. The magic number of each filtering algorithm.

The Filtering Algorithm	The Similarity Measure	The Magic Number
User-based	Pearson correlation coefficient	15
	cosine similarity	12
Item-based	Pearson correlation coefficient	12
	cosine similarity	12
Content-based	-	5

Based on Table 15, we can determine the magic number as 15 for user-based collaborative filtering approaches. In this case, each similarity measure has different magic numbers: 15 in Pearson correlation coefficient and 12 in cosine similarity. This means that if we utilize Pearson correlation coefficient as a similarity measure for user-based collaborative filtering approaches, we can expect reliable recommendation results by using 15 inputs. Namely, the minimum number of inputs to expect reliable recommendation results in user-based collaborative filtering based on Pearson correlation coefficient is 15. The results in Table 15 also show that if we utilize cosine similarity as a similarity measure for user-based collaborative filtering approaches, we can expect reliable recommendation results by using 12 inputs. Similarly, our results show that in the case of item-based collaborative filtering approaches, the minimum number of inputs to expect reliable recommendation results is 12 for each similarity measure.

Moreover, we have verified the magic number by applying this to a real recommendation system that requires ratings or genre combinations as initial inputs. Through a series of experiments, we conclude that the use of the magic number can provide highly accurate recommendation results and ease the burden on users.

In future works, we will apply the magic number to other domains, such as music or e-commerce, and observe the stability of the accuracy by validating the MAE. In addition, based on the insight of the magic number, we will also utilize other inputs, such as category or tag information for the contents, to alleviate user-side cold-start problems in recommender systems.