The Smart television (TV) is a connected device that provides an extended functionality in delivering digital contents, such as live channels, movies, dramas, shows, and video on demand (VOD) services [1
]. The smart TV comes with processing capabilities, third-party platforms, operating systems, and media players. The amalgamation of television with a processor, connectivity capabilities, and support for the Web 2.0 features has made this device attractive not only for viewers but also for researchers [1
]. As compared to traditional TV systems, smart TV is a computing device [1
] that can perform a variety of operations, including voice and gesture recognition. The life cycle of the smart TV is longer as compared to other smart devices, such as a smartphone, smartwatch, laptop, etc., [1
]. This motivates enterprises and companies to develop technologically-advanced hardware and software for smart TVs. However, numerous issues are associated with smart TVs, which include security and privacy, complex user interfaces, interactivity issues, bloatware [3
], the complex nature of browsing and searching, and personalized recommendation issues [1
]. In this paper, we targeted the issues and factors affecting the recommendation process in the context of smart TV watching scenarios.
The most common approaches for watching the contents on smart TV are browsing and searching for desired contents from robust and diverse data sources, such as stored videos, live channels, clips, etc. However, such rich and growing data sources make it difficult to search for the desired contents [1
]. The reasons include the (a) widely used legacy remote controls, (b) lean-back nature of the smart TV, (c) specialized entertainment device, and (d) a device for all types of viewers. The details of these reasons are (a) apart from the availability of a variety of input devices and smart remote controls, the legacy remote is still widely used with smart TVs and limits frequent interactions; (b) in the lean-back, the contents are normally enjoyed in passive mode and preferably less interactively; (c) the smart TV is a specialized entertainment device, normally used for watching video, movies, live channels, and playing games; (d) the smart TV is for every type of viewer, including senior citizens, non-technical persons, and kids. Hence, searching and browsing are among the difficult activities on smart TV [4
]. Although electronic program guides (EPGs) may help in searching the desired channel, due to a significant collection of channels and programs on EPG the searching and scrolling is a difficult task [5
]. In addition to searching and browsing, the recommender system helps in reducing information overload by helping a user to select the best items from a significant collection of items [6
A recommender system is a software tool that recommends suitable items to a user or group of users [9
]. The recommender systems infer user interests by utilizing various sources of data, such as user profiles, clicks, and feedbacks (rating, and like/dislike) [8
]. However, in a smart TV environment, such data are neither accurate nor simple to predict or calculate because the smart TV represents a set of users with diverse interests and taste. This distinct watching behavior and purpose make smart TV a unique device; however, the recommender systems consider the smart TV as an ordinarily connected device and recommend items based on activities performed by a single user or group of users. Such approaches are neither viable nor accurate to recommend items to the exact viewer(s) of smart TV. Hence, in the context of smart TV viewing scenarios, content filtering, channel recommendation, scheduling programs, and personalized viewership are challenging opportunities [11
The recommender systems use numerous approaches for recommendations. Examples of such approaches include content-based filtering, collaborative filtering, and hybrid approaches [12
]. The content-based filtering techniques rely on the user’s profile information and the item’s profile information [13
]. However, smart TV is normally enjoyed in groups, and hence, the user’ profile information cannot represent the whole group of viewers. Therefore, content-based filtering is not suitable for a smart TV watching scenario. The collaborative filtering approaches rely on user feedbacks, i.e., implicit feedbacks and explicit feedbacks. The implicit feedbacks are calculated from user activities, such as navigation, browsing, etc.; whereas explicit feedbacks are provided by the user in the form of likes/dislikes, rating, etc., [14
]. The explicit feedbacks are rarely provided by a smart TV viewer. Therefore, specifically in the context of smart TV, we are left with only a few implicit feedbacks for recommending relevant items to the viewer(s) [15
]. In hybrid filtering techniques, both approaches are combined to get better recommendations. However, it suffers from the inherent issues of both content and collaborative filtering approaches. Although some hybrid recommender systems, such as discussed in [16
], combined several techniques for achieving accuracy; yet, most of the techniques demand active feedbacks from the user, which is not welcomed by smart TV viewers due to lean-back nature of a smart TV. Moreover, we cannot expect the same interactions from viewers as they normally have with computers and smartphones [17
]. In addition to the interactive nature of smart TV, it is enjoyed as a lean-back device and passive as legacy TV systems [18
The approaches for analyzing user activities, such as data mining, clickstream analysis, and in-depth identification of a viewer(s) may lead to serious security and privacy threats [18
]. Although security and privacy issues for smart TV are in the infancy stage, they are the most important concerns for smart TV viewers [19
]. Security and privacy are usually ignored by the buyer, seller, and even by the manufacturer [1
]. Such concerns need a thorough investigation to make smart TV more user-friendly and intelligent. As discussed, a smart TV can provide rich types of entertainment on a single platform; however, smart TVs are widely used for streaming live channels and movies [20
]. This distinct nature puts a question mark on the performance of the existing recommender systems because these systems are specially designed for personalized recommendations on personalized devices, such as smartphones and personal computers. The feedback that comes from computers and smartphones can be handled by the existing recommender systems; however, the feedback that comes from smart TV needs further investigation for recommending relevant items to the viewer or group of viewers.
In the smart TV watching scenario, group recommendations play a vital role. This is because, in most of the households, the smart TV is enjoyed in a group for watching movies, dramas, news, etc., [22
]. Different approaches are used for group recommendations, such as aggregated predictions and aggregated model for preferences aggregation of individuals in a group [22
]. However, the approaches for the identification of group members is based on predictions and estimation. Furthermore, due to the diverse interest of individuals; aggregated predictions and preferences aggregation are not feasible solutions and may lead to privacy issues. Therefore, the exact identity of group members and satisfying every group member is still challenging.
This paper is an attempt to identify some overlooked factors that affect the recommendation process and recommendation results on a smart TV. The factors are validated by the results of a subjective study conducted for this research. The first factor that affects the recommendation process is the group of viewers in front of a smart TV. The recommender system considers smart TV as a single viewer; however, there may be groups having diverse interests. The second factor that affects the recommendation process is the limited provision of feedbacks by the viewer. The reasons for such limited provision of feedbacks include the lean-back nature of smart TV, legacy remote controls, and the shared nature of smart TV. The third factor that affects the recommendation results is the different watching behavior of smart TV viewers. The smart TV is normally used for watching movies, video, and live channels. The viewer rarely uses smart TV for reading news, books, articles, etc. Therefore, navigating between video, movies, and channels provide limited feedbacks to the recommender system. The app-based and complex user interfaces are the fourth factor that affects the recommendation process. The smart TV comes with an operating system and applications (apps in short). Therefore, navigating between channels means navigating between apps, which create hurdles in calculating the viewer(s) interests. The fifth factor that affects the recommendation process is the shared nature of smart TV. The focus of the recommender system is the delivery of personalized recommendations; however, smart TV is not a single user device (See Section 3
for more details).
The contributions of this study are:
To investigate the issues and challenges in existing group recommender systems for content recommendations on Smart TV.
To identify the factors that affect the performance of a recommender system in the context of smart TV watching scenarios.
A subjective study for validating the factors by analyzing the watching behavior of smart TV viewers.
Results of the subjective study show that in most of the households, smart TV is enjoyed in groups for watching live channels, movies, and videos. Moreover, the identification of viewers for the personalized recommendation may lead to privacy concerns, which is not welcomed by the smart TV viewers. The findings of this paper suggest that the recommender system should treat smart TV as a different connected device. This is because the other devices, such as computers and smartphones, are personalized devices, whereas smart TV is a shared device. Furthermore, user modeling plays an important role in recommender systems [25
] and may enhance the recommendation results. This paper further suggests an enhanced user and group modeling technique for enhancing the personalization services in general and group recommendations in specific on a smart TV.
The rest of the paper is divided into 7 sections. Section 2
is the state-of-the-art Literature on the recommender systems in the context of smart TV. Section 3
presents some Potential Factors that affect the recommendation process for smart TV viewers. Section 4
presents the Methods and Material for analyzing and validating the factors identified in Section 3
. Section 5
and Section 6
are Results and Analysis, respectively. Section 7
is Discussion and Future Research Work. Section 8
concludes the paper. References are listed at the end.
2. Literature Review
During 1950, TV was considered the primary source to influence public opinion and still plays a significant role in molding and shaping people’s perception [26
]. The legacy TV system is the most popular entertainment device still widely used [27
]. Compared with the combined daily consumption of smartphones, tablets, and PC; the time spent on watching TV is still high [28
]. Today’s TVs are not only full-duplex in nature, but also smart and come with built-in operating systems, third-party software, and sensors [1
]. Moreover, modern television, called smart TV, is significantly different from the technological point of view as well as from an entertainment point of view. The smart TV comes in traditional shapes and sizes as well as in the form of set-top-boxes (STBs) that can be connected with a variety of displays, including legacy TV systems. Smart TV has revolutionized the entertainment industries by enabling streaming of Web contents. This amalgamation opens new avenues not only for entertainment industries but also for education, health, defense, business, commerce, etc. The operating system of smart TV enables the installation of different software and for managing many distinct features related to connectivity, communication, and interaction with smart TV [29
]. Table 1
shows and compares a list of the features of smart TV and legacy TV systems.
As connected TV, a smart TV can stream a variety of contents from the Web. Because of this, the terrestrial broadcast TV channels are now shifting their TV contents to the Web. The channel owners have their own web applications, which can provide streaming services to their users. Similarly, video-sharing websites are available, such as YouTube, Netflix, etc. The videos to such video sharing websites are uploaded by the channel owner or by the third parties, including individuals.
Such a rich set of multimedia contents makes it difficult to search for the relevant content. Moreover, searching is a user-driven activity that mostly relies on user queries. In addition to searching for relevant content on a smart TV, the online recommender systems are used to overcome the cognitive overload and recommend different items to a viewer [30
]. However, the existing recommender systems are designed for the individual user who consumes content on a variety of personal devices, including computers and smartphones. Recommendations on terrestrial TV are usually based on regions. TV operators recommend items, goods, and services on a regional basis. As recommendation is on regional bases, viewers have to watch the running content, or they simply change the channel. To avoid such stereotype TV content, viewers have shifted to more advanced watching systems, i.e., smart TVs. The smart TV gives better clues to the recommender systems because of an Internet Protocol (IP) address, the log file, from which location, interests, etc., can easily be extracted. However, it suffers from numerous issues (see Section 3
for details). Figure 1
shows a common recommendation process on smart TVs.
The following are some state-of-the-art systems and literature on recommender systems in the context of smart TV, connected TV, and Internet Protocol Television (IPTV).
In Hybrid Broadcast Broadband Television (HbbTV) project, they have designed several frameworks for identifying a viewer in front of a smart TV that include multi-user identification and multi-user recommendations [17
]. HbbTV is heavily criticized in the literature for its security and privacy concerns as it is capable of capturing private data, such as picture and profile information [19
]. Smart Parental Advisory [32
] proposed a deep learning-based framework and usage control for implementing dynamic parental controls on a smart TV. The proposed work shows a camera with a TV for providing real-time parental control on smart TV content. In [33
], a face recognition system for set-top-box based intelligent TV was proposed. They used a web camera for pictures, which is connected with a set-top-box, and a server is attached for face recognition. The exact identification of the viewer(s) on server-side is time-consuming due to the frequent switching of the viewer(s) in front of a smart TV. In [34
], an enhanced recommender system was proposed by face detection and recognition system in front of a smart TV. They have used Face++ (https://www.faceplusplus.com/
) and SkyBiomerty (https://skybiometry.com/
) for face and emotion detection and recognition. Furthermore, they argued that detection of more than one person could form a group, and hence, the recommender system should recommend items to the group instead of individuals.
A TV program recommendation technique for a group of viewers was proposed in [35
]. This work proposed a TV program recommendation system for multiple viewers (group) based on merging user profiles. Similarly, the study in [11
] proposed the merging of multiple preferences to improve recommendation results. RecTime [36
], proposed a real-time recommender for an online broadcasting system, which considers a user’s preferences and time factors simultaneously. A personalized TV listings service for the digital TV [37
] is proposed. The work describes the development of personalized television (PTV (http://www.ptv.ie
)) listing system which handles the information overload, by providing an Internet-based personalized listings service. Shinjee et al. [38
], worked on the automatic and personalized recommendations of TV programs for smart TV viewers. They argued that due to the massive content available for watching on a TV, it is difficult to retrieve the desired program and hence, worked on the automation of recommending TV programs. An intelligent hybrid recommendation approach is proposed in [39
], which uses hybrid collaborating filtering with a voice recognition system for controlling the smart TV. However, the work is limited to the recommendation of already crawled TV programs. Kwon and Hong [40
], proposed a personalized program recommender system (PRS) for smart TV, which is based on a novel similarity method and collaborative filtering techniques.
Most smart TVs are equipped with automatic content recognition (ACR) system that automatically recognizes the content and recommend items to a viewer based on currently watching contents [41
]. However, frequent switching of viewers in front of smart TV create difficulties for accurate recommendations. MovieLens (http://www.movielens.org
) is an online web-based movie recommender system that invites viewer(s) to rate any movie in the list. In return, the system performs predictions and personalized recommendations. In the case of smart TV, such types of web-applications are less effective because rating and tagging are among the difficult activities to perform on the legacy remote control of a smart TV. TV-Predicator [31
], designed an application which allows personalized recommendations without disturbing the lean-back position in front of a TV. It uses the customers watching behavior and explicit feedbacks (ratings) on the server side to predict user preferences and recommend relevant items.
Different data-mining techniques are used to predict viewer preferences, such as, content-based filtering algorithms are used for related items, collaborative filtering techniques are used for ratings predictions, clustering techniques are used for increasing the performance, association rules mining approaches are used for analyzing item relations, vector space model is used for the identification of the viewer’s watching patterns. However, data-mining techniques, clickstream analytics, and in-depth analysis of the viewer’s data may lead to serious security and privacy issues [42
]. We argue that personalization and recommendations on smart TV should not compromise the security and privacy of the viewers.
], is a web-based movie recommender system that uses collaborative filtering techniques to recommend items to group users rather than individuals. PolyLens has used a group recommender extension to the MovieLens recommender system. PloyLens explores the design space of collaborative filtering recommenders for group viewers. A detailed log is maintained to measure how viewers formed a group(s). They also survey the group users and analyze their experiences of group recommendations. The issue with PolyLens is that it relies on collaborative filtering techniques. Collaborative filtering techniques suffer from data sparsity and gray sheep problems [44
]. OntoTV [45
] is developed for the management of different sources of TV contents. It uses semantic multimedia techniques by developing an ontology for different TV-related content. OntoTV is a television content management system, which retrieves content information from different sources and represents them using ontologies and knowledge engineering. A cloud-based program recommendation system (CPRS) [46
] was proposed and implemented to improve channel recommendation systems by the formation of groups that have similar taste. The system is implemented using cloud computing, which uses the map-reduce algorithm. In CPRS, the first step is to cluster the user profiles and then grouped these profiles by the K-means clustering algorithm. These techniques required a good computation power to recommend an item from very large datasets.
J. Kim et al. [47
] proposed a searching and recommendation method for TV programs based on contents and viewers ontologies. They designed the ontology model of TV programs to define the semantic structure of the content. Compared with keyword-based searching, a more precise searching was achieved on documentary programs. N. Chang et al. [28
] proposed a TV program recommender framework, which integrates the Web 2.0 features into television sets and smart TVs (set-top-boxes). A personalized TV recommendation with mixture probabilistic matrix factorization [15
] developed a two-stage framework for building a TV recommender system. First, the proposed framework automatically learns the number of watching groups, and then the mixture probabilistic matrix factorization (mPMF) model was proposed for learning the mixture preference of television programs. A TV program recommendation for multiple viewers based on users’ profile merging techniques [29
] was proposed. The work proposed a TV program recommendation for multiple viewers (group) based on merging user profiles. The profile merging is based on total distance minimization techniques that generate the results closed to most users’ preferences.
Summarizing the literature; the content recommendation on a smart TV is based on a single user profile; however, in actual scenarios, the smart TV may be enjoyed in groups. This situation creates hurdles for existing recommendation algorithms. Although different approaches are used for group recommendations, such as aggregated predictions, preferences aggregation, etc., however, these approaches are not only difficult to calculate but also may lead to privacy issues. Furthermore, the identity of a group member is based on predictions and estimations and usually performed on the server-side, which may lead to serious privacy issues. The following section presents some potential factors that affect the personalization services, including the recommendations for smart TV viewer(s).
The results of this study confirm that watching behavior on a smart TV is different from other connected devices, including computers and smartphones. Moreover, apart from the technological advancement in smart TV technologies, it is used as traditional TV. Furthermore, it is confirmed that smart TV is enjoyed in groups as a passive, lean-back device, in which the viewer(s) prefer less interactivity with a smart TV. These factors provide limited input to recommender systems. Thus, the recommendation results generated on smart TV may be irrelevant to different viewers in front of a smart TV. Rest of the findings are analyzed in the following sub-sections.
6.1. Primary Activity on Smart TV
A smart TV can stream almost every type of Web content, including live channels, movies, clips, web 2.0 features, etc. However, as discussed, the smart TV is enjoyed as traditional TV. Analyzing the collected data, we found that viewers used smart TVs for watching moving pictures, i.e., videos, clips, etc. Although the transformation of the legacy TV system to smart TV took a long time, people are enjoying smart TV as a lean-back device with lots of innovative features and options for watching their desired TV contents. This watching behavior provides very limited clues for a personalized service, including recommendations. As discussed, in most of the households, the smart TV is used as a shared device, i.e., a device for the whole family. However, the existing recommender systems are not intelligent enough to consider the whole family or closed group member for precise and relevant recommendations. Therefore, the recommended item(s) may not be relevant to all viewers in front of a smart TV. From the subjective study, it is confirmed that writing blogs, commenting, likes/dislikes, and other textual entry is a cumbersome task by using a remote control. Therefore, the recommender systems relying on user feedbacks for recommendations may not perform well for smart TV viewers.
6.2. Passive Feedbacks
The existing recommender systems take a user feedbacks, preferences, and watching history as input and recommend items to a user. The feedbacks may be explicit or implicit. The explicit feedbacks, such as commenting, likes/dislikes, login information, etc., are among difficult activities on smart TV (see Section 3
for more details). The implicit feedbacks are normally calculated from user activities, such as browsing, navigations, clicks, etc. In a smart TV watching scenario, the calculation or estimation of implicit feedbacks are not only difficult but also inaccurate results are highly probable. The reasons include the lean-back nature of smart TV. Moreover, behind a smart TV, we have multiple profiles that make it difficult to calculate and estimate the exact viewer profile. As discussed, television watching is a passive activity. The viewers prefer less interaction during content consumption on smart TV. The legacy remote controls and complex user interfaces further restrict the interaction with a smart TV. This type of passive feedbacks and watching behavior provides very limited input to recommender systems. Surfing the web contents on computers and smartphones are significantly different from a smart TV. However, the existing recommender rarely considers such factors during the recommendation process.
6.3. Personalized Recommendations and Privacy Concerns
The private data of a viewer may be captured for the delivery of personalized services, such as VOD services and recommendations. Results of the subjective study show that viewers are reluctant to allow a smart TV to capture their private data. Some recommender systems, such as used by HbbTV project of Hbb-Next, capture the private data of viewers, such as face detection and recognition for making precise recommendations. However, the HbbTV has been criticized in the literature for capturing a viewer’s private data. This study suggests a tradeoff between recommendation results and privacy concerns. We argue that the recommender systems should seamlessly recommend items without any breaches of security and privacy.
6.4. Primary Communication Device with Smart TV
Analyzing the collected data, we found that apart from technological advancement, the primary communication device with a smart TV is a legacy remote control. Apart from full support, other communication devices, such as wireless keyboards, mouse, touch pads, etc., are rarely used with a smart TV. Although smartphone-based remote controls are also available; however, more accessible and innovative smartphone-based remote controls are still to come. Moreover, this study confirms that the key press is a major activity on the remote control for retrieving desired contents. Voice-based commands are used only by 5% of viewers. Some reasons for this low usage of voice-based commands are poor multi-lingual support, noisy environment, the pronunciation of words, etc. Although advanced technologies, such as Apple’s Siri for Apple-based STBs, Google-Assistants for Android-based smart TVs, and smartphone-based universal remote controls are available; however, they are rarely used for watching smart TV contents.