Conceptualization and Survey Instrument Development for Over-the-Top Platforms’ Usability

Abstract

OTT (over-the-top) streaming is a subscription-based video service model that delivers video-on-demand content, films, and series directly to end-users over the Internet, bypassing the need for traditional satellite receiver systems. The most popular OTT service providers include Netflix, Hulu, Amazon Prime, and Disney+. During the COVID-19 pandemic, the viewership rates and subscriber numbers for OTT platforms rapidly increased. Like various other products and systems, usability problems can substantially impact user satisfaction, loyalty, and the intention to continue using OTT services. Therefore, this study aimed to conceptualize the usability of OTT platforms and develop an OTT Usability Measurement Scale for the usability evaluation of OTT platforms based on the Apple tvOS Guidelines and the literature. OTT platform usability was conceptualized with nine constructs, including Accessibility and Customization, Account Management, Data Entry and Search, Branding, Privacy, Navigation, Help, Content, and Design, and the concepts were measured with a scale including 48 items. The validity of the developed scale was tested through two separate survey studies conducted with Netflix web application users. The first survey involved 650 participants. At this stage, an exploratory factor analysis was used to evaluate the scale’s measurement properties, and the developed factor structure was confirmed. In the second stage, a survey with 600 participants was conducted, and a confirmatory factor analysis was applied to validate the scale properties. Furthermore, a nomological validation of the developed scale was performed, examining the relationship between the acquired OTT factors and elements such as continued intention to use, satisfaction, and brand loyalty. As a result of the nomological validation, it was observed that the privacy and design factors significantly affected each of the three dependent variables.

1. Introduction

Over-the-top (OTT) platforms are digital service providers that deliver streaming media content, such as movies, TV series, music, and other forms of media, directly to consumers through internet connectivity, thereby bypassing traditional broadcast distribution systems such as terrestrial, satellite, or cable television networks [1,2]. These platforms leverage Internet Protocol (IP) technology to transmit content, functioning similarly to Internet Protocol Television (IPTV) but usually providing services at higher standards [1]. While such platforms generally use ad-based video on demand, subscription-based video on demand (SVOD), transactional video on demand, and hybrid business models [3], some well-known OTT services are Netflix, Hulu, Disney+, and Amazon Prime.

OTT platforms offer numerous benefits, such as convenient access to video content anytime and anywhere, a wide variety of content options, personalized recommendations, flexible viewing hours, multi-device support, and additional advantages through subscription plans. Therefore, OTT platforms are significantly transforming traditional television viewing habits and gaining acceptance as the “new television” in society [4], with OTT services playing a central role in driving changes in video content consumption patterns [5].

The lockdowns caused by COVID-19 have considerably increased the adoption of OTT platforms, resulting in an increase in usage and a rise in subscribers. The global consumer base for OTT services is anticipated to reach approximately 4.22 billion users by 2027, with a penetration rate of 53%. In addition, the revenue generated by OTT platforms is projected to reach EUR 277 billion in 2023 [6]. Improving usability becomes crucial to maintaining customer satisfaction and fostering brand loyalty in these rapidly expanding systems.

Usability is defined as a quality feature evaluating the ease of use of user interfaces and is described with five quality components: learnability, efficiency, memorability, errors, and satisfaction [7]. The ISO defines usability as the “extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [8]. Usability is a broader concept than ease of use and user-friendliness; it refers to the effective, efficient, and satisfactory interaction with a system, product, or service, considering diverse user capabilities across all system interactions, including learning, regular, or infrequent use [9]. Furthermore, integrating usability principles at various stages of the design, development, and evaluation process enhances the quality and acceptance of a new system [7,10]. By considering usability factors early on, developers can address potential issues, resulting in a user-friendly and efficient system. This approach minimizes the need for later modifications, leading to effective and user-centric systems that improve the overall user experience, satisfaction, and brand loyalty [11].

In the literature, there is a limited number of studies investigating the usability of video streaming and OTT platforms (e.g., [12,13,14,15,16,17]). On the other hand, there is a more significant number of studies in the literature concerning the evaluation of the usability of TV and similar systems (e.g., [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]). Most existing studies have examined the usability of OTT platforms, video-on-demand (VOD) systems, TV, and similar systems through user testing, field research, and heuristic evaluation. However, a few of these studies have developed usability heuristics and principles for TV systems [19,21,23,25,32,34,35,36]. On the other hand, most of them [19,21,23,34,35] only propose high-level general heuristics or principles for TV systems and give definitions rather than propose a usability checklist for each concept. The remaining ones [25,32,36] propose heuristics and principles with their explanations and checklists. However, the developed principles and heuristics are specific to TV-related services, and not OTT platforms. Our literature review reveals that only the study by Jang and Yi [29] developed a user experience measurement scale designed for smart TVs. To our knowledge, there has not been a usability measurement scale explicitly created for OTT platforms. Therefore, in the current study, we aim to conceptualize usability for OTT platforms and develop a survey instrument to measure and evaluate their usability.

This study is organized as follows: The second section provides a literature review on the usability of OTT platforms, TV, and similar systems. The following section presents the methodology and the findings. The last section discusses managerial implications, limitations, and future research.

2. Literature Review

In the literature, there are several studies investigating the factors affecting the adoption of OTT platforms or streaming services (e.g., [37,38,39,40,41,42,43]). Furthermore, Mulla [3] reviewed the literature to determine the factors influencing the adoption of OTT platforms. The results show that content, price, flexibility, convenience (perceived ease of use), perceived usefulness, perceived enjoyment (hedonic motivation), desire to be freed from any constraint, entertainment value, socialization, culture inclusion, binge-watching, and self-efficacy are the critical factors for OTT adoption.

On the other hand, in the literature, there is a limited number of studies regarding the usability of OTT platforms. These studies are not specifically OTT-focused and generally concentrate on the usability of specific applications that provide video streaming services. Therefore, we focused our literature research on streaming services and specifically included studies related to OTT platforms like Netflix, Hulu, Amazon Prime, and others. Krishnan and Sitaraman [12] conducted an in-depth investigation and found that video streaming quality, especially factors like startup delay, significantly impacts viewer behavior. Hussain et al. [13] identified video streaming and its quality as dominant usability metrics for mobile apps. Hussain et al. [14] assessed YouTube’s usability using video recordings, heatmaps, and questionnaires to evaluate metrics like ease of use and satisfaction. Eliseo et al. [15] provided interface guidelines for OTT platforms like Netflix using Nielsen’s heuristics. Yang et al. [16] assessed the usability of multiple SVOD services, such as Netflix, Amazon Prime, and Hulu, by conducting questionnaires. Kollmorgen et al. [17] highlighted Netflix’s higher UX ratings across several questionnaires, including the UEQ (User Experience Questionnaire), SUS (System Usability Scale), and UMUX (Usability Metric for User Experience). These studies on video streaming and OTT platforms [12,13,14,15,16,17] evaluated the usability of OTT platforms using various usability evaluation methods, including usability tests, heuristic evaluation, and questionnaires. However, none of them attempted to develop a dedicated instrument for evaluating the usability of OTT platforms.

While limited studies in the literature address the usability of OTT platforms, we expanded our literature review to include studies on TV and other systems with similar features. Several studies have conducted user tests on TVs and related systems. For instance, Obrist et al. [20] utilized eye tracking to study usability challenges for users above 50 years of age with interactive TV applications. Bernhaupt et al. [18] employed a combination of field studies, heuristic evaluations, and user testing to uncover usability challenges associated with iTV, a UK-based free television channel. Lim et al. [24] assessed the usability of smart TV interfaces and remotes with diverse age groups. The research by Miesler et al. [26] and Lee and Shin [44] examined user experience and interface comparisons of smart TVs, respectively. Dou et al. [28] and Ouyang and Zhou [27] both focused on usability problems faced by older people in China when using smart TVs. Awale and Murano [30] evaluated the Apple TV interface against Nielsen’s heuristics, while Bures et al. [31] introduced an automated model-based approach for usability testing. Gumussoy et al. [33] conducted a usability test using the concurrent think-aloud method, surveys, eye tracking, expression analysis, and logging techniques, investigating the relationships between usability metrics in TV-related studies.

In the studies that conducted user tests for TV and similar systems [18,20,24,26,27,28,31,33], specific usability problems related to the tested system were identified. However, these studies did not propose any general principles for the design of TV systems. On the other hand, several studies in the literature suggest principles, heuristics (general usability principles), and checklists to analyze and improve the usability of TV-related services [19,21,23,25,29,32,34,35,36]. Kim et al. [19] identified 21 usability principles for personalized electronic program guides for digital TVs, such as controllability, feedback, error, predictability, learnability, memorability, and consistency. Chorianopoulos [34] proposed user interface design principles for interactive TVs, such as viewer as a director, participatory content authoring, diverse content sources, and infotainment. Then, the principles were applied to redesign music TV as an iTV application. Geerts and De Grooff [21] conducted user testing on social TVs, identifying participant issues during testing and converting these issues into sociability guidelines, including personal privacy and group privacy, sharing content flexibly, and minimizing distraction from TV programs. Collazos et al. [35] adapted Nielsen’s ten heuristics to suit the design of an interactive digital television (iDT), incorporating additional heuristics related to navigation, information structure, physical constraints, and considerations for extraordinary users. Solano et al. [23] defined a refined set of usability heuristics proposed by Collazos et al. [35] for iDT applications. Solano et al. [25] proposed a usability checklist for iDT applications using the heuristics proposed by Solano et al. [23]. They validated the checklist’s effectiveness through expert evaluations, enabling a comparative analysis of the number and severity of usability problems against Nielsen’s heuristics. Fernandes et al. [36] introduced graphic and interaction guidelines for iTV systems using Google Material Design and Apple tvOS guides. Kaya et al. [32] proposed 16 usability heuristics and a guideline specific to set-top box and TV systems. The proposed heuristics were validated through user testing, expert judgment, and heuristic evaluation. They determined the most critical heuristics related to catastrophic usability problems to be visibility of system status, pleasurable and respectful interaction with the user, privacy, parental control, and easy access.

Our literature review indicates that the studies [19,21,23,34,35] only proposed high-level general heuristics or principles and explained them through definitions. On the other hand, our current study offers a detailed explanation of concepts related to OTT platforms using open codes. Furthermore, several studies (e.g., [19,21,23,25,32,34,35,36]) have proposed principles, heuristics, guidelines, and checklists for TV systems. However, only Jang and Yi [29] developed a user experience measurement scale specifically for smart TVs. Their research, conducted using a three-stage method encompassing identification, integration, and verification, utilized laboratory and field studies. They developed a scale to measure user experience factors and found a significant relationship between these UX factors, user satisfaction, and usage intention. Although several studies have been conducted on TV and OTT usability, the literature lacks a measurement scale designed specifically for OTT platforms. A summary of the studies is given in the Appendix

Table A1. Literature review.

Study	Platform	Research Methodology	Usability Attributes/Metrics
Bernhaupt et al. [18]	Interactive Television (iTV)	Survey, laboratory study, field study	Navigation/the task completion rate, communication interactivity index
Kim et al. [19]	Digital TV	Prototype development, literature review, expert assessment, factor analysis, survey, user test	Responsiveness, predictability, prevention, user control, error indication, feedback, controllability, generalizability, icon, learnability, visibility, color, text, observability/task completion time, ratio of task completion time and error-free time, number of commands, frequency of errors, help frequency
Obrist et al. [20]	iTV	eye-tracking method, think-aloud method, Standard Usability Score (SUS) questionnaire	The difficulty of task completion, the number of errors, gaze plot (road maps)
Piccolo et al. [130]	Interactive Digital Television (IDTV)	Analyzing the current guidelines and recommendations	Content must be perceivable, interface components in the content must be operable, content and controls must be understandable, and content should be robust enough to work with current and future user agents
Chorianopoulos [34]	iTV	Developing new design principles and applying the principles to music TV with a case study	Viewer as a director, participatory content authoring, diverse content sources, infotainment, social viewing, relaxed navigation, multiple levels of attention, and TV grammar and aesthetics
Collazos et al. [35]	Interactive Television (iTV)	Developing new heuristics for iTV applications’ usability	Nielsen’s ten heuristics, navigation, structure of information, physical constraints, extraordinary users
Geerts and De Grooff [21]	Social TV	User test, questionnaires and interviews, open-axial coding	Offer different channels and levels for communicating freely, use awareness tools for communicating availability, allow for both synchronous and asynchronous use, exploit viewing behavior for informing and engaging other viewers, support remote as well as collocated interaction, give the user appropriate control over actions and system settings, minimize distraction from the television program, notify the user of incoming events and situation changes, guarantee both personal privacy and group privacy, adapt to appropriate television program genres, let users share content flexibly, and encourage shared activities
Silva and Nunes [22]	TV	Usability test, guideline development	User drive and control, test settings and preparation, care, communication, and listening
Solano et al. [23]	Interactive Digital Television (IDTV)	Developing new heuristics using a 6-stage methodology	Match between the system and the real world, simplicity, consistency and standards, feedback, physical constraints, extraordinary users, structure of information, navigation, recognition rather than recall, flexibility and efficiency of use, user control and freedom, error prevention, recovering from errors, help, and documentation
Krishnan and Sitaraman [12]	Streaming video	Quasi-experimental design, large-scale data analysis	Failures, startup delay, average bitrate, abandonment rate, normalized rebuffer delay, playtime, return rate
Lim et al. [24]	Smart TV	User test: eye tracking and cursor recording methods, interviews, and questionnaires	Gaze duration, task completion time, performance time, error rate
Solano et al. [25]	Interactive Digital Television (IDTV)	Testing new iDT heuristics developed by Solano et al. [23]	Match between the system and the real world, simplicity, consistency and standards, feedback, physical constraints, extraordinary users, structure of information, navigation, recognition rather than recall, flexibility and efficiency of use, user control and freedom, error prevention, recovering from errors, help, and documentation
Lee and Shin [44]	Smart TV	Emotion recognition, gesture recognition, satisfaction	Facial expressions of the participants
Miesler et al. [26]	VOD app for smart TV	Complaint analysis, customer survey, log-file analysis, usability test	Average time on site/app; average page views; clickthrough rate; jumps; site exits per process; percentage of high-, medium-, and low-frequency visitors; conversion rate (registration); conversion rate (purchase); consumption rate (average order value)
Hussain et al. [13]	Mobile video streaming apps	Literature review	Video streaming quality
Eliseo et al. [15]	Video websites (Netflix, VideoBrasil Platform, Video@RNP, Vimeo, and YouTube)	Heuristic evaluation	Nielsen’s ten heuristics
Hussain et al. [14]	YouTube	Usability test with video recordings, mouse and keyboard heatmaps, questionnaires	Ease of use, usefulness, learnability, satisfaction, task time
Fernandes et al. [36]	Interactive television (iTV)	Creating new principles based on Google Material Design and Apple tvOS guidelines, mockups, prototype development, and testing	Layout and grid, images and visual textures, navigation/satisfaction, motivation, control, pragmatic quality, hedonic quality of stimulation, hedonic quality of identification, attractiveness
Yang et al. [16]	SVOD services in Japan (Netflix, Amazon Prime, etc.)	Card sorting, questionnaires	Ease of use, ease of searching for content, content indication, genre/categorization
Dou et al. [28]	Smart TV	Physiological measurement method, video recording, eye-tracking method, interviews, user test	Search task time, skin conductance (low-frequency and high-frequency)
Jang and Yi [29]	Smart TV	Laboratory and field study, instrument development	Ease of adaptation, playfulness, perceived aesthetics, relative salience, appearance appropriateness, controllability of the remote controller, cognitive easiness, perceived responsiveness, user satisfaction, connectivity, usage intention, perceived security, content diversity, social relatedness, customization flexibility, perceived sound quality, perceived helpfulness, stability, real-life applicability, perceived quality of 3D viewing, perceived picture quality
Ouyang and Zhou [27]	Smart TV	Pilot study, think-aloud method, PSSUQ (Post Study System Usability Questionnaire), user test	The task effectiveness, completion time, the number of keystrokes metrics
Awale and Murano [30]	Apple TV	Nielsen’s heuristics and the seven universal design principles	Nielsen’s heuristics and the seven universal design principles
Bures et al. [31]	Smart TV	Automatic model-based approach, user testing	Path steps, average time needed to execute the scenario
Kaya et al. [32]	Smart TV	Five-step formal methodology to develop new heuristic	Nielsen’s ten heuristics, pleasurable and respectful interaction with the user, privacy, parental control, easy access
Gumussoy et al. [33]	Set-top box and TV	Simultaneous thinking aloud technique, surveys, eye tracking, expression analysis, logging	Completion time, success rate, task difficulty, fixation duration, task completion time, fixation count, saccade count, scanpath length, keystroke count, saccade duration, blink count, negative emotions count, average fixation duration, backspace count, PSSUQ
Kollmorgen et al. [17]	Netflix, Microsoft PowerPoint, Zoom, BBB	Questionnaires: UEQ, SUS, UMUX	User experience (UX) ratings

. Therefore, this study aims to fill that gap by developing the OTT Usability Measurement Scale (OTT-UMS) to evaluate the usability of OTT platforms. This is the first study to develop a usability measurement scale tailored for OTT platforms.

3. Methodology

In this study, a three-stage methodology, proposed by Lewis et al. [45], was used to conceptualize and develop a survey instrument to evaluate the usability of OTT platforms. We followed this methodology in the current study since it includes a comprehensive and systematic approach to define constructs and develop and validate the developed survey instrument. The detailed flow of the methodology is presented in Figure 1. In the first step, the constructs’ domain related to the usability of OTT platforms is revealed with open and axial coding procedures. In the second step, the initial survey instrument is developed and validated with several steps like pretest, pilot study, and content validity checks. In the last step, the measurement properties of the survey instrument are evaluated through exploratory and confirmatory factor analyses. Furthermore, this methodology was also successfully applied in usability-related research (e.g., [46]).

3.1. Domain Definition

The first stage in developing a construct involves establishing the domain of the idea using various sources like literature reviews, interviews, or case studies [45]. Lewis et al. [45] recommended content analysis as a systematic technique for domain definition, which is used to specify distinct aspects of the construct domain through multiple iterations. In this study, we initially examined Apple tvOS guidelines by using content analysis to conceptualize the usability of OTT platforms. Open and axial coding processes were employed to conduct content analysis.

Open coding is an analytic interpretation process based on the separation, examination, comparison, comprehension, and classification of data that identifies concepts and the properties and dimensions in the data. Events, actions, and interactions are constantly compared according to their similarities and differences, and categorically similar events are grouped into categories and subcategories. Later, axial coding is conducted. The process of relating categories to their subcategories is termed “axial” coding, as it occurs around the axis of a category, linking categories at the level of properties and dimensions; in axial coding, a researcher tries to establish a connection between the concepts and categories revealed as a result of open coding [47,48].

The Apple tvOS Guidelines [49] were used as the primary source for this study. Apple developed this guideline to guide designers who want to build games, OTT applications, and smart home services for Apple TV. The guidelines describe the characteristics that a reasonable application should have in many areas, such as application architecture, visual design, interface elements, and system capabilities. We conducted a line-by-line analysis of the Apple tvOS Guidelines, generating open codes and exploring questions related to usability categories, their potential subcategories, and their definitions. Through axial coding, we organized the content of the guidelines into main categories based on the similarities of the open codes representing subcategories. Subsequently, nine categories were developed: Accessibility and Customization, Privacy, Account Management, Content, Branding, Navigation, Design, Data Entry and Search, and Help. The coding matrix, which includes axial codes, subcategories, and open codes, is presented in Appendix

Table A2. Coding matrix: axial codes, subcategories, and open codes.

Axial Codes	Subcategory	Open Codes
Account management	Accounts	The system allows users to create/delete an account easily.
		The system offers different sign-up options like sign in with Apple or Google.
		The system lets user use another device to sign up or authenticate.
		The system automatically verifies the user’s identity when signing into the OTT platform.
		The system briefly explains the benefits of creating an account on the sign-up screen.
		The system allows user to explore contents without sign-in.
	Profiles	The system allows users to create multiple profiles.
		The system shows who is logged in clearly.
		The system allows user to switch between profiles easily.
	Parental Control	The system should allow parental controls.
		The system allows users to set up a kids profile with general restrictions based on specific maturity ratings.
		The system allows users to lock their profiles with a PIN to prevent kids or others from accessing it.
Privacy	Privacy	The system respects the user’s rights when obtaining personal information.
	Data Security	The OTT platform abides by personal data protection laws and only collects user’s personal data necessary for its activity.
Navigation	Easy Navigation	The system should be natural and intuitive to ensure that people easily know what to do and where they are at all times.
		People navigate through the system easily.
		The system should offer natural and familiar navigation to make people access content easily and quickly.
		The information structure of the system should require the fewest screens.
		While performing actions, people should use a few gestures.
	User Control and Freedom	People should navigate backward or the main menu easily.
		People should cancel their actions easily.
	Menu Bar	The system should have a neat and uncomplicated menu bar with a maximum of seven items with short names.
		The system should show all items when the menu bar is in focus.
Help	Onboarding	The system should have an intuitive design where not much guidance is required.
		If necessary, the system can guide users, but the priority is that the system has an intuitive design.
	Help	The system should provide necessary instructions when controls vary from the norm.
		The system should provide necessary and easy-to-understand information in the help section.
	Feedback and Alerts	The system should use alerts in important situations, such as confirming purchases, destructive actions, or notifying people about problems.
		Alerts should have only critical information and useful choices.
		Label destructive actions clearly. The system should allow users to identify alert buttons that cause destructive actions.
		The system should use images instead of descriptive alert text whenever possible. If necessary, the alert messages should be short and only one or two lines long.
		Alarm texts and button titles should be clear without needing an extra explanation.
		The OTT platform should provide user-friendly language in warning messages.
Content	Content	Content is the most important element for people. The system should focus on content and minimize distractions to give people an uninterrupted and enjoyable viewing experience. The system should provide high-quality video and sound to enhance cinematic experience.
		Provide useful information such as images, titles, and descriptions about content.
		Avoid using more than eight lines for additional information about the content.
		In the info panel, people should display additional information such as subtitles, chapters, audio tracks, and speaker output options.
	Easy Access	People should skip forward and backward in a video by clicking the right and left sides of the progress bar.
		The system allows people to perform actions such as watching, starting over easily, and resuming playback.
		The system should allow people to access their favorite content quickly.
		The system allows people to find content by grouping content into familiar categories such as “Movies,” “TV Shows,” “Kids,” and “Sports”.
	Top Shelf Content	The system should highlight new or featured content on the top shelf to enable people to access content easily.
		The system should feature new content instead of those users have already watched.
		The system should feature episodes or season trailers, new shows, and new seasons or shows coming soon.
		The system should personalize favorite content and show recommendations based on the user’s viewing experience on the top shelf.
	PiP Mode	The system should allow people to use an app while watching content in picture-in-picture (PiP) mode.
	Loading	The system should not wait for users to reach content using splash screens, detail screens, or intro animations.
		The system should not appear to be frozen while the contents are loading.
		The system should make loading clear by using standard progress indicators or customize loadings using different educating or entertaining hints, videos, or graphics to create immersive experience while masking loading time.
		The system should provide accurate progress information.
		The system provides visual feedback to give time streaming content to load.
		The system should prefer to use progress bars for quantifiable actions; otherwise, it should use activity indicators.
		The system should preload screens of content in the background immediately.
		The system displays launch image quickly when the app starts up.
		The system should not wait users to reach content by using splash screens, detail screens, or intro animations.
Branding	Branding	Provide enough branding without overwhelming people.
		Implement refined branding through the app’s design using custom color, font, or background.
		The branding elements should be used consistently throughout the system.
	Logo	The system should have an attractive and recognizable logo.
Design	Color	Test colors on televisions with different display settings to understand how colors look on big screens.
		Avoid using colors that make it difficult for people to perceive the content.
	Layout	The system should design a layout that looks great on various screen sizes.
		The system should adhere to the screen’s safe zone and provide primary content away from the edges.
		The system should use enough consistent spacing through the system to avoid overlapping.
		The system should use clean and consistent layouts to keep the content at the center of attention.
		The system should have background compatible with other content considering image and text colors.
	Icons and Images	The system should use simple and recognizable images as icons of buttons, segments, etc. The launch image should not include logos and other branding elements, as it is not a branding opportunity. People should interact and focus on icons and images easily.
		The system should use high-quality images appropriate for different sizes of screens.
		The icons and images should have a safe zone to prevent cropping as the icon scales and moves.
	Layering	Use standard interface elements to create layered images to create a sense of realism and vigor using the parallax effect.
	Interface Elements	The system should provide consistent interface elements through the system.
	Respectful Interaction	Consider how colors are perceived in different countries and cultures to ensure the appropriate message is conveyed.
Accessibility & Customization	Typography	The system should use legible and clear fonts suitable for different screen sizes.
		The system should use appropriate fonts that are legible at a distance.
		The system should use appropriate leading to ensure readability.
		The system should use built-in text styles whenever possible to make content visually distinct.
	Customization	The system should allow users to customize their text size.
	Language	The system should allow people to change audio and subtitle languages easily.
Data Entry & Search	Effort Minimization	The system should automatically display a virtual keyboard when people click a text field.
		The system should allow users to enter text data quickly using a linear keyboard that automatically appears when they click a text field.
		The system should provide an appropriate keyboard type based on the data being collected to make entering one’s name, e-mail address, or number easier.
		The system should request data entry in fields like search and log in.
		The system shows recently entered information if data entry is required. For example, it shows the email address keyboard and recently entered addresses.
	Search	The system should list popular or recent searches in the results area before people start typing.
		The system should simplify search results and display a short list that best matches the search performed to prevent people from scrolling through the pages.
		The system should allow users to see the search results easily.

.

3.1.1. Accessibility and Customization

Designers now have more possibilities to create various interfaces, but this has increased complexity due to factors like fonts, colors, and sizes [50]. On OTT platforms, font selection for subtitles and text size significantly impact readability. Brown et al. [51] found that positioning subtitles at the bottom of the screen according to viewing angles did not receive positive feedback, but using a dark background improved readability. Studies on Chinese characters [52,53] and Latin characters [54,55] emphasized the importance of font size, resolution, and font type in readability.

Regarding OTT platforms, offering various font options and sizes is crucial for readability, particularly for older people [56,57]. Language is another essential aspect, as subtitles in different languages are necessary for non-native content access. Pedersen [58] highlighted the significance of language norms and localization in subtitles, while Kuscu-Ozbudak’s study [59] on Netflix in Türkiye showed that subtitle quality impacts subscription continuation. In conclusion, ensuring readability and customization in both interfaces and subtitles while adapting to users’ language preferences is essential for enhancing the user experience.

3.1.2. Account Management

When users decide to use any platform, systems usually prompt them to register. Users encounter two main issues at this stage. Firstly, users want to know the intended use of their information, so platforms should provide clear information about this. For example, users may be concerned about whether their entered information will be shared with third-party applications [60]. The second issue related to usability is setting up a username and password. Users need help understanding cryptographic structures, so the password structure should be easily understandable, and data entry should be user-friendly [61].

Usability problems, such as account closure and suspension options, may also arise in account management interfaces. Platforms can offer various options for account closure, as some users may want to regain access after closing their accounts. It is crucial to clearly state what happens to the data after an account is deleted [62]. Platforms can provide authentication options, such as PINs and one-time generators [61]. SMS authentication, which sends a message to the user’s phone number, is effective [63]. However, regardless of the authentication method, designers should never display users’ private information on the interface [60].

OTT platforms like TVs are shared devices at home and thus allow users to define protected user profiles to secure personal information [32]. Popular OTT platforms like Netflix, Amazon Prime Video, and Disney+ allow users to create multiple user profiles, including children’s profiles, enhancing the personalized viewing experience. Features such as profile locking add a layer of security, ensuring that each user’s preferences and viewing patterns remain private. Kaya et al. [32] proposed a usability heuristic related to parental control for STB and TV applications. Lad et al. [64] conducted a comparative study on Amazon Prime Video and Netflix. Their findings indicated that Netflix outperformed Amazon Prime Video in areas such as the availability of movie/series trailers and parental control features. In conclusion, “account management” refers to the ease and flexibility with which users can create, modify, and delete their accounts and manage multiple profiles, including those for children.

3.1.3. Branding

Companies need to create brand values to increase sales and strengthen brand awareness. Research shows that users prefer familiar brands despite having a higher perception of quality in other brands [65]. Therefore, companies should work on brand loyalty and awareness without neglecting the quality factor and strive to improve customers’ perception of the brand. Regarding brand awareness, logo design plays a significant role as a crucial element of the brand concept. The brand logo effectively enhances the quality of customer relationships [66]. Gultom et al. [67] found that brand image significantly influences subscription decisions on Netflix. However, more is needed to increase brand awareness than relying solely on logos and slogans. Providing value and customer benefits is also essential for a brand [68]. A study by Hoehle and Venkatesh [11] emphasized the importance of brand color in brand development efforts and suggested that users should not be forced to watch brand advertisements. In conclusion, companies interact with their target audiences through slogans, logos, and offered features. Therefore, OTT platforms should appropriately use their logos, colors, and other brand elements to enhance brand loyalty and awareness.

3.1.4. Data Entry and Search

Data input and search methods on OTT platforms are significant factors that impact user experience. Research indicates that different user groups respond differently to various methods, suggesting that diversifying these methods can enhance user experience. For instance, Smith and Chaparro [69] and Dou et al. [28] found that physical QWERTY keyboards and voice input methods were more effective for younger and older users. Bernard et al. [56] noted that children found speech recognition and handwriting methods more enjoyable.

The importance of text prediction in data input was emphasized by Geleijnse et al. [70], Bernhaupt et al. [18], and Solano et al. [25]. However, when determining data input methods, multiple factors must be considered. Oliveira et al. [71] pointed out that different user groups respond differently to various data input methods, making it challenging to select a one-size-fits-all approach. Barrero et al. [72] emphasized the significance of providing users with the most comfortable method. Consequently, to improve user experience, OTT platforms should offer diverse data input methods, considering different user characteristics and needs.

Users use the search function to quickly and easily access information like movies, TV series, etc. Lamkhede and Das [73] defined the concepts of “Get,” “Find,” and “Discover” on Netflix. “Get” is for users who precisely know what they are looking for, “Find” is for those who do not have a specific preference but know what they are searching for, and “Discover” is for users who are not sure about their preferences but want to explore new content. Accordingly, the search feature should produce results based on the keywords entered and offer personalized content recommendations to the user. Therefore, the success of an OTT platform is not just reliant on user-friendly data input and search mechanisms, but also on how well they personalize content recommendations and improve user experience. For example, the research by Pattanayak and Shukla [74] highlighted that employing an algorithm that provides personalized content recommendations based on user preferences can significantly enhance user experience. In conclusion, to enhance user experience and increase platform success, OTT platforms should provide diverse and user-friendly data input methods and personalize search and recommendation mechanisms based on user preferences.

3.1.5. Design

Design plays a significant role in enhancing user experience on OTT platforms. Design involves integrating various components such as colors, layouts, graphics, icons, images, and animations. Using colors is essential for usability as it directly affects users’ perception and interactions [75,76]. Usability studies have shown that color significantly impacts ease of use, satisfaction, and overall usability (e.g., [19,20,25,27,77]). It should also be noted that the impact of colors on user perception and response may vary based on cultural differences, age, or gender, and appropriate color palettes should be used in the design while taking these factors into account [78,79].

Page layout refers to arranging elements on a web page or application, such as links, buttons, images, menus, texts, etc. Effective performance of these components is crucial to provide users with an excellent aesthetic experience and ease of use across different screen sizes. Designing consistent and aesthetically pleasing layouts while avoiding complex arrangements is essential [80]. Another important aspect of layout design is visual balance. To achieve visual balance in the layout, objects should be distributed evenly along the vertical and horizontal axes [81]. A design approach that heavily uses only a specific portion of the screen and leaves some parts empty is not favored by users [82].

Icons, images, and animations are fundamental components that form an interface and are crucial for effective interaction between users and the system. These components enable users to comprehend system messages and convey their intended actions to the system. Therefore, presenting these components complexly may blur the perception of information and confuse users on where they should focus [79]. Icons play essential roles in effectively conveying messages without the need for text [83]. When used appropriately, animations can provide users with a rich visual experience, but they should be used in suitable ways to avoid distracting users [79]. The quality of images and the emotions they evoke in users are also important. Offering customized images may positively impact user interaction [84,85]. In conclusion, color, layout, and graphics (icons, images, and animations) are critical elements in the design of OTT platforms that directly influence user experience and interactions, determining aesthetics, usability, and accessibility, and ultimately shaping user satisfaction and overall platform success.

3.1.6. Help

Users frequently need help and feedback when using systems. Help and feedback are essential factors in usability evaluation and design [19,23,25,35,86,87]. Help is crucial for users to solve their problems and access necessary information. User-friendly help content and interfaces especially play significant roles in OTT platforms. Feedback helps users understand the results of their actions. Users want to know if their commands are recognized and when the operation will be completed, and they want to receive information about the system status. Designers can enhance the user experience by providing precise and consistent feedback [88,89]. In conclusion, help and feedback are essential elements that facilitate users’ interactions with systems. User-friendly help content, interfaces, and clear feedback are necessary for a successful user experience on OTT platforms.

3.1.7. Navigation

Navigation refers to the path users follow to access the desired content. Good navigation should have fundamental characteristics such as being easy to use and understand [35]. When users interact with a website or TV application, effective usability requires them to move smoothly, easily understand their location, navigate within the system, and cancel actions when needed. Solano et al. [23,25] emphasized the importance of the “Navigation” concept in their usability studies for interactive TV, highlighting the need for users to access desired information easily and receive navigational feedback. Obrist et al. [20] confirmed, using eye tracking technology, that elderly users comprehend navigation more slowly than younger users. Hence, the navigation structure developed by OTT platforms should suit users of different age groups. Another important factor is the use of vertical, horizontal, or mixed navigation structures. Ribeiro et al. [90] found, in their study on IPTV, that users made fewer errors in vertical navigation compared to horizontal navigation. Golja et al. [91] discovered in their study on iTV that users reached their desired content with fewer clicks in horizontal navigation. Navigation is essential for OTT platforms, enabling users to access desired content easily and effectively. A good navigation system assists users in smoothly navigating through the system, understanding their location, and seamlessly performing desired actions.

3.1.8. Privacy

The widespread adoption of OTT platforms in recent years has led many people to use these platforms frequently. Profit-driven OTT platforms have started analyzing user-generated data to increase their sales. As a result, there is a need to classify the data that will be used to ensure the privacy of users’ personal information. Data such as names, addresses, dates of birth, film preferences, and IP addresses can be directly or indirectly associated with users [92]. Therefore, it is concluded that the generated data are personal, and OTT platforms cannot use them as they wish; furthermore, sharing data with third parties requires permission. Additionally, the preference for OTT platforms for cloud environments instead of data centers [93] is believed to bring new privacy-related issues.

Geerts and De Grooff [21] attempted to establish intuitive rules for social TV and emphasized the importance of personal and group privacy in the rules they defined. Mohajeri Moghaddam et al. [94] conducted a study on Amazon Fire TV and Roku TV. They discovered a security vulnerability in the Roku TV application that exposed users’ locations and watched channels without permission. This indicates that inferences about user preferences are made without their consent. Shim and Yeon [95] addressed the “Privacy Paradox” concept and conducted a survey with 618 Netflix users in South Korea. The study revealed that when users believed they would benefit from Netflix using their personal data, their privacy concerns became less significant. Although OTT platforms’ analysis of user data to enhance user experiences may not be seen as a problem, it is essential to ensure the storage and security of data that can be directly linked to users.

3.1.9. Content

One of the most appealing features of OTT platforms compared to traditional television is the content they offer to users. Traditional television operates on a fixed programming schedule, requiring viewers to passively consume content at specific periods determined by the television station, limiting users’ ability to access their desired content at any time [96]. However, OTT platforms offer users a wide range of content options, offering them greater freedom and flexibility. In a study comparing Netflix with its competitors, participants emphasized that Netflix’s most substantial aspect is its content [97]. Similarly, another study conducted in Türkiye identified content as the most influential factor for subscribing to an OTT platform [59]. Furthermore, several studies, such as those by Kim and Lee [96] and Shin and Park [5], have shown that content significantly impacts the selection of OTT services and contributes to user satisfaction. Malewar and Bajaj [39] demonstrated that content diversity is a significant factor in adopting OTT platforms, affecting users’ behavioral intentions and actual usage. In addition to content diversity, the continuous delivery of new content is highlighted as an essential factor. Additionally, OTT platforms recommend content to users based on their viewing history and preferences [25]. In conclusion, the primary reason why users prefer OTT platforms is access to high-quality and diverse content. In this context, within the scope of this study, the term “content category” refers to an OTT platform that provides users with easy access to content and delivers an enjoyable, uninterrupted viewing experience.

Based on the content analysis and literature review,

Table 1. Construct definitions.

Construct Name	Construct Definition
	The degree to which a user perceives…
Account Management	... ease in creating, managing, and securing accounts on the OTT platform.
Privacy	... that the OTT platform safeguards personal information and privacy.
Navigation	... straightforward navigation and user-friendly design on the OTT platform.
Help	... accessible and comprehensive help and guidance on the OTT platform.
Content	... quality, control, and personalization of viewing content on the OTT platform.
Branding	... consistent and recognizable branding on the OTT platform.
Design	... aesthetic and consistent design across the OTT platform.
Data Entry and Search	... efficient data entry and search functionality on the OTT platform.
Accessibility and Customization	… readability and customization options on the OTT platform.

provides a summary of construct definitions.

3.2. Survey Instrument Construction

In the second stage of the methodology, a survey instrument was developed and refined over several iterations [45]. Development of the OTT measurement usability scale from categorical structures consists of four steps: (1) development of items, (2) pre-test, (3) pilot study, and (4) content validity check.

Initially, every statement within the domain was transformed into an item for the instrument. A list of open-coded items (Appendix Table A2) was analyzed to develop the initial item pool, and 80 items were created. An initial instrument was created using these items. Then, a pre-test was conducted to get feedback from experts in human-computer interaction to check the initial instrument [45]. Experts, including one assistant professor, one associate professor, and two research assistants, were asked to review the initial instrument in terms of its design, content, and clarity. Based on their feedback, necessary modifications were made. Twenty-four items unrelated to OTT platforms or specific to television applications were eliminated, and several items were modified to increase their clarity. As a result, a refined list of 56 items was obtained after the pre-test.

In the following step, a pilot study was conducted using a small sample group of real users to test the efficacy of the OTT usability measurement scale. Lewis et al. [45] emphasized the significance of choosing pilot study participants from the target audience. Thus, this research was conducted with 26 Netflix users.

Table 2. Demographic information of the pilot test.

Demographic	Category	N	Demographic	Category	N
Age	18–24	11	Most Used OTT Platforms	Netflix	23
	25–34	10		Amazon Prime	14
	35–44	5		Blu TV	3
Gender	Male	18		Mubi	1
	Female	8		Exxen	3
Job	Student	10		Disney+	3
	Education	7		TV+	1
	Public services	5		TRT	1
	IT	4	OTT Platform Usage Frequency	Less than one day	13
Education	Middle school	1		2–3 days	7
	High school	10		4–5 days	4
	Graduate	4		6–7 days	2
	Master’s degree	6	Daily Watching Time	0–1 h	14
	PhD	5		1–3 h	10
				4–6 h	2

displays their demographic details. Participants were asked to report any challenges they experienced during the survey and to identify and propose improvements for any missing, unclear, or complex items in the scale [45]. After analyzing the feedback, four items were discarded, and ten were rephrased for clarity. As a result, the scale was refined to 52 items.

Content validity is a quantitative procedure that assesses how well an individual criterion or the entire scale represents its intended domain [45]. Anderson and Gerbing’s [98] methodology was used for content analysis. The evaluators for content validity should be representative of the primary study sample and the population of interest rather than just experts. Although there is no universally accepted number for sample size, suggestions range from 12 to 30 participants [98]. This study’s content validity was conducted with 30 users in line with these recommendations.

A matrix was used for the content validity check, with categorical structures in columns and criteria in rows. Participants selected the most suitable criterion–categorical structure combinations in this matrix. Two indices were calculated from this matrix: PSA and CSV. The PSA index is the ratio of participants assigning a criterion to the related categorical structure to the total number of participants, ranging from 0 to 1. A high value indicates that the criterion effectively expresses the definition of the categorical structure. The CSV index is calculated by subtracting the highest number of times a criterion is assigned to another categorical structure from the number of participants assigning the criterion to the related structure, then dividing that value by the total number of participants. This index ranges between −1 and +1, with positive values indicating better alignment with the related categorical structure [98]. The threshold for these indices can vary. A study with four categorical structures was set at 0.30 since the expected ratio for random assignment was 0.25 [99]. High content validity ratios aligned with the determined categorical structures, and experts carefully reviewed and rearranged items falling below the threshold for better clarity [11]. Since there are nine categories in this study, the probability of random assignment is 0.111. A value higher than this, specifically 0.25, was taken as the threshold. Items HLP6, CON8, and CON9, which had values below the threshold, were removed. Meanwhile, NVG3, NVG5, NVG6, HLP1, CON2, CON3, CON4, and DES3 were revised for clarity. The results are presented in

Table 3. PSA and CSV scores of the items.

Construct	Code	Items	PSA	CSV
Account Management	ACC1	I can easily create or delete my user account on the Netflix website.	0.97	0.93
	ACC2	Netflix’s website automatically verifies my identity when I sign in.	0.70	0.50
	ACC3	I understand the benefits of creating an account from the brief information on the registration screen.	0.53	0.43
	ACC4	I can create multiple profiles for different users, including children.	0.97	0.93
	ACC5	I can easily lock my profile.	0.67	0.40
Privacy (Adapted from [100,101])	PRV1	I think the Netflix website values my privacy.	0.97	0.93
	PRV2	I feel safe sending my personal information to the Netflix website.	0.90	0.83
	PRV3	I believe my personal information is not shared with third parties without my permission.	0.97	0.93
Navigation	NVG1	I can easily understand where I am in the system.	0.60	0.27
	NVG2	I can easily navigate through the system.	0.70	0.50
	NVG3 *	I can perform actions in a few steps quickly.	0.47	0.10
		I can quickly perform my operations in a few steps. (Modified.)
	NVG4	I can easily go back or go to the main menu.	0.60	0.30
	NVG5 *	I can easily cancel my operations.	0.37	0.20
		I can cancel my actions easily. (Modified)
	NVG6 *	I think the menu design is simple and understandable.	0.20	0.37
		The menu design is simple and understandable. (Modified.)
	HLP1 *	I do not need much help using the system because it is easy.	0.40	0.20
		I do not need much guidance as the system has an intuitive design. (Modified.)
Help	HLP2	When I need help, I can easily access the necessary instructions and information.	0.90	0.87
	HLP3	I think the information provided in the help section is sufficient and understandable.	0.73	0.57
	HLP4	I am informed about important situations (such as purchasing, deleting an account, etc.).	0.53	0.30
	HLP5	I can easily understand warning messages.	0.50	0.33
	HLP6 **	The language used in the warnings is clear and sincere.	0.27	−0.23
	CON1	I think Netflix offers a pleasant and high-quality viewing experience.	0.60	0.50
	CON2 *	I think the information given about the content is sufficient and useful.	0.40	0.23
		I think the descriptions of the content are sufficient and useful. (Modified.)
Content	CON3 *	I can easily forward, rewind, start from the beginning, and resume playback anytime.	0.47	0.03
		I can control my viewing experience on Netflix by utilizing features such as fast forwarding, rewinding, starting from the beginning, pausing, and resuming playback anytime (modified).
	CON4 *	I can access my favorite content quickly.	0.40	0.13
		I can quickly access my favorite content. (Modified.)
	CON5	I think movies, series, and TV shows are appropriately categorized.	0.73	0.53
	CON6	I can easily access new or popular content.	0.57	0.37
	CON7	I think the Netflix website suggests content that is suitable for my viewing experience.	0.70	0.53
	CON8 **	I can continue to watch content on the OTT platform using a different app.	0.20	−0.06
	CON9 **	After logging in, I do not have to wait for a login screen or explanation animations to access the content.	0.10	−0.10
Branding	BRN1	I think Netflix uses its brand colors or visuals subtly and unobtrusively.	0.70	0.57
	BRN2	I can easily recognize the brand on the Netflix website through color, font, and background.	0.53	0.37
	BRN3	I think Netflix consistently uses its brand elements on its website.	0.77	0.70
	BRN4	I can easily recognize Netflix’s logo.	0.87	0.80
	DES1	I think the colors on the screen look good in different settings and screen sizes.	0.67	0.50
	DES2	I think the Netflix website has a beautiful design and layout.	0.90	0.87
	DES3 *	Netflix prioritizes primary content in areas where user interest is concentrated on the website.	0.37	0.10
		Netflix places primary content at the center of attention. (Modified.)
	DES4	The appropriate spacing on the Netflix website helps prevent overlapping content.	0.77	0.57
	DES5	I think the Netflix website has a simple and consistent layout on all pages.	0.93	0.90
Design	DES6	I can see the icons and images clearly from a distance.	0.77	0.67
	DES7	Netflix uses lively and realistic images and animations on its website.	0.90	0.87
	DES8	The Netflix website provides consistent icons and images throughout the system.	0.70	0.63
	DES9	I think the colors used on the Netflix website are suitable for my culture and values.	0.60	0.47
Accessibility and Customization	CUS1	I can easily read the text on different screen sizes.	0.60	0.30
	CUS2	I can easily read the text on the website.	0.70	0.57
	CUS3	I can customize the text size.	0.57	0.37
	CUS4	I can easily change the audio and subtitle languages.	0.57	0.40
Data Entry and Search	SRC1	I can easily enter data on the website.	0.70	0.50
	SRC2	I only enter data in the required fields on the platform.	0.80	0.73
	SRC3	I do not need to re-enter the last information I entered.	0.63	0.47
	SRC4	I can see popular or recent searches without typing in the necessary keywords.	0.83	0.77
	SRC5	I can easily search on the Netflix website.	0.97	0.93
	SRC6	I can see the search results in a list.	0.60	0.40

* describes the modified items. ** describes the items dropped from the item pool.

.

3.3. Evaluation of Measurement Properties

We initially used exploratory factor analysis (EFA) to evaluate the scale’s measurement properties to determine the factor structure. Then, we applied confirmatory factor analysis (CFA) to validate the scale properties. During the confirmatory phase, we assessed if the theoretical structures could accurately predict the related dependent variables, ensuring the nomological validity of the scale. For both the EFA and CFA, two separate samples were employed. All criteria were measured using a 7-point Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree). Two survey studies were conducted among Netflix web application users to measure the usability of OTT platforms. We selected the Netflix platform to validate our proposed instrument due to its global prominence as an OTT platform. As of the second quarter of 2023, Netflix had approximately 238.39 million subscribers worldwide [102]. Furthermore, its annual revenue has consistently risen, reaching a record high of USD 31.6 billion in 2022 [103]. Despite its relatively recent introduction to Türkiye in 2016, Netflix has rapidly emerged as one of the country’s leading OTT platforms, gaining 3.5 million subscribers by 2022 [104].

3.3.1. Exploratory Factor Analysis

The first survey was conducted with 650 individuals who use the Netflix web application in Türkiye. An extra question, “Please answer this question as “agree” (6),” was added to the survey to filter non-attentive survey respondents. Fifty-five respondents who answered the control question incorrectly and forty-three respondents who provided the same answer to all questions were excluded from the sample. An EFA was performed using the survey results of the remaining 552 individuals. The demographics of the participants are provided in

Table 4. Demographics (EFA).

Demogr.	Category	No.	Demogr.	Category	No.
Age	18–24	85	Education	PhD	9
	25–34	332		Master’s	78
	35–44	171		Bachelor’s	319
	45–54	54		College graduate	60
	55	8		High school	162
Gender	Male	320		Secondary school	17
	Female	330		Primary school	5
Job	IT	23	Most Used OTT Platforms	Netflix	650
	Banking and Finance	26		Amazon Prime	229
	Insurance, Real Estate, and Law	8		Disney+	211
	Construction & Engineering	47		BluTV	180
	Public Services	98		PuhuTV	95
	Health Service	42		Exxen	209
	Trade and Self-Employed	45		Other	12
	Education and Training	73	OTT Platform Usage Frequency (Weekly)	Less than one day	109
	Marketing, Advertising, and Design	19		2 or 3 days	221
	Student	81		4–5 days	142
	Other	188		6–7 days	178
Daily Watching Time	0–1 h (including 1)	52
	1–3 h (including 3)	391
	4–6 h (including 5)	163
	More than 6 h	44

.

In our survey, there are 46 newly developed items, including 3 items related to the “Privacy” category that were adopted from Cheung and Lee [100] and Flavián and Guinalíu [101]. The recommended item-to-response ratio for EFA should be between 1:3 and 1:8 [105]. With 46 newly developed items, at least 368 responses would be sufficient. In total, 552 responses were collected for the study, which meets this requirement. IBM SPSS Statistics 25 software was used for EFA.

Before extracting the factors, the suitability of the data for factor analysis was assessed using the Kaiser–Meyer–Olkin (KMO) test and Bartlett’s Test of Sphericity. The KMO index was found to be 0.963, surpassing the critical value of 0.6 [106]. Additionally, Bartlett’s Test of Sphericity yielded a significant result (p < 0.001), indicating the validity of the factor analysis [105].

Next, the factor structure was obtained using the principal component analysis (PCA) method with variable rotation. The results revealed nine factors that accounted for a total variance of 74.46%. Upon inspecting the factor loadings, it was observed that the factor loading of item DES9 (0.463) was below 0.5. Consequently, DES9 was removed from the scale. The remaining items had factor loadings above the threshold of 0.5 [107].

After removing DES9, the total explained variance increased to 75.12%. In Figure 2, the scree plot suggests that retaining 4–6 factors would be appropriate; however, based on eigenvalues, there are 9 factors with values above 1.

Table 5. Exploratory study: descriptive statistics, factor loadings, explained variance, and Cronbach’s alpha values.

Construct	Item	Mean	Std.	Load.	VarExp (%)	Cronbach’s α	Construct	Item	Mean	Std.	Load.	VarExp (%)	Cronbach’s α
Account Management	ACC1	6.13	1.169	0.790	8.342	0.910	Privacy	PRV1	5.75	1.425	0.851	5.724	0.935
	ACC2	6.03	1.247	0.770				PRV2	5.67	1.456	0.893
	ACC3	5.93	1.256	0.806				PRV3	5.59	1.555	0.840
	ACC4	5.97	1.299	0.736			Design	DES1	6.16	0.781	0.640	13.245	0.939
	ACC5	6.03	1.194	0.622				DES2	6.11	0.911	0.755
Branding	BRN1	6.12	0.863	0.748	6.541	0.886		DES3	6.10	0.887	0.751
	BRN2	6.16	0.835	0.782				DES4	6.04	0.950	0.696
	BRN3	6.18	0.785	0.791				DES5	6.15	0.905	0.732
	BRN4	6.32	0.787	0.690				DES6	6.13	0.910	0.750
Navigation	NVG1	6.11	1.001	0.721	9.542	0.934		DES7	6.13	0.931	0.730
	NVG2	6.24	0.913	0.765				DES8	6.16	0.848	0.751
	NVG3	6.17	0.993	0.804			Data Entry and Search	SRC1	6.24	0.863	0.658	8.187	0.910
	NVG4	6.23	0.958	0.808				SRC2	6.27	0.832	0.719
	NVG5	6.15	0.986	0.739				SRC3	6.28	0.880	0.666
	NVG6	6.19	0.905	0.769				SRC4	6.29	0.837	0.733
Help	HLP1	6.14	1.054	0.853	8.229	0.950		SRC5	6.33	0.834	0.708
	HLP2	6.12	1.049	0.863				SRC6	6.33	0.809	0.688
	HLP3	6.15	1.038	0.859			Content	CON1	6.29	0.853	0.738	9.430	0.923
	HLP4	6.03	1.150	0.795				CON2	6.23	0.904	0.673
	HLP5	6.10	1.086	0.825				CON3	6.29	0.827	0.731
Accessibility and Customization	CUS1	6.18	0.840	0.781	5.878	0.885		CON4	6.25	0.840	0.666
	CUS2	6.18	0.874	0.829				CON5	6.23	0.820	0.717
	CUS3	6.07	0.953	0.699				CON6	6.26	0.830	0.717
	CUS4	6.24	0.833	0.746				CON7	6.22	0.847	0.666

presents the means, standard deviations, final factor loadings, variance explained, and Cronbach’s alpha values for each factor. All factor loadings exceeded the acceptable value of 0.6 [107]. Additionally, the Cronbach’s alpha values for each construct were above 0.7, indicating an adequate level of reliability. The final version of the OTT usability measurement scale obtained through EFA is provided in Appendix

Table A3. OTT usability measurement scale.

Construct	Code	Items
Account Management	ACC1	I can easily create or delete my user account on the Netflix website.
	ACC2	Netflix’s website automatically verifies my identity when I sign in.
	ACC3	I understand the benefits of creating an account from the brief information on the registration screen.
	ACC4	I can create multiple profiles for different users, including children.
	ACC5	I can easily lock my profile.
Privacy	PRV1	I think the Netflix website values my privacy.
	PRV2	I feel safe sending my personal information to the Netflix website.
	PRV3	I believe my personal information is not shared with third parties without my permission.
Navigation	NVG1	I can easily understand where I am in the system.
	NVG2	I can easily navigate through the system.
	NVG3	I can quickly perform my operations in a few steps.
	NVG4	I can easily go back or go to the main menu.
	NVG5	I can cancel my actions easily.
	NVG6	The menu design is simple and understandable.
	HLP1	I do not need much guidance as the system has an intuitive design.
Help	HLP2	When I need help, I can easily access the necessary instructions and information.
	HLP3	I think the information provided in the help section is sufficient and understandable.
	HLP4	I am informed about important situations (such as purchasing, deleting an account, etc.)
	HLP5	I can easily understand warning messages.
	CON1	I think Netflix offers a pleasant and high-quality viewing experience
	CON2	I think the descriptions of the content are sufficient and useful.
Content	CON3	I can control my viewing experience on Netflix by utilizing features such as fast forwarding, rewinding, starting from the beginning, pausing, and resuming playback anytime.
	CON4	I can quickly access my favorite content.
	CON5	I think movies, series, and TV shows are appropriately categorized.
	CON6	I can easily access new or popular content.
	CON7	I think the Netflix website suggests content that is suitable for my viewing experience.
Branding	BRN1	I think Netflix uses its brand colors or visuals subtly and unobtrusively.
	BRN2	I can easily recognize the brand on the Netflix website through color, font, and background.
	BRN3	I think Netflix consistently uses its brand elements on its website.
	BRN4	I can easily recognize Netflix’s logo.
	DES1	I think the colors on the screen look good in different settings and screen sizes.
	DES2	I think the Netflix website has a beautiful design and layout.
	DES3	Netflix places primary content at the center of attention.
	DES4	The appropriate spacing on the Netflix website helps prevent overlapping content.
	DES5	I think the Netflix website has a simple and consistent layout on all pages.
Design	DES6	I can see the icons and images clearly from a distance.
	DES7	Netflix uses lively and realistic images and animations on its website.
	DES8	The Netflix website provides consistent icons and images throughout the system.
Accessibility and Customization	CUS1	I can easily read the text on different screen sizes.
	CUS2	I can easily read the text on the website.
	CUS3	I can customize the text size.
	CUS4	I can easily change the audio and subtitle languages.
Data Entry and Search	SRC1	I can easily enter data on the website.
	SRC2	I only enter data in the required fields on the platform.
	SRC3	I do not need to re-enter the last information I entered.
	SRC4	I can see popular or recent searches without typing in the necessary keywords.
	SRC5	I can easily search on the Netflix website.
	SRC6	I can see the search results in a list.

.

3.3.2. Confirmatory Factor Analysis (CFA)

In the confirmatory assessment of the OTT usability measurement scale’s final version derived from the EFA, we ensured the scale’s nomological validity, as Lewis et al. [45] recommended. Nomological validity considers the extent to which the relationship of a measured concept with other concepts aligns with previous research findings [108]. Based on a literature review, questions regarding factors like continued intention to use, brand loyalty, and satisfaction, which are inherently linked to usability, were identified.

This approach was employed in a second survey with Netflix web application users in Türkiye, where 600 individuals participated. However, twenty respondents were excluded; thirteen provided the same responses to all questions, while seven answered the control question incorrectly, resulting in 580 valid responses for the CFA. Participant demographics are illustrated in

Table 6. Demographics (CFA).

Demographic	Category	No.	Demographic	Category	No.
Age	18–24	101	Education	PhD	10
	25–34	229		Master’s	43
	35–44	190		Bachelor’s	392
	45–54	71		College graduate	47
	55	9		High school	99
Gender	Male	300		Secondary school	9
	Female	300	Most used OTT platforms	Netflix	600
Job	IT	23		Amazon Prime	321
	Banking and Finance	24		Disney+	320
	Insurance, Real Estate, and Law	13		BluTV	250
	Construction & Engineering	33		PuhuTV	166
	Public Services	151		Exxen	322
	Health Service	36	OTT platform usage frequency (weekly)	Less than one day	62
	Trade and Self-Employed	58		2 or 3 days	222
	Education and Training	68		4–5 days	156
	Marketing, Advertising, and Design	15		6–7 days	160
	Student	84
	Other	95
Daily watching time	0–1 h (including 1)	46
	1–3 h (including 3)	355
	4–6 h (including 5)	150
	More than 6 h	49

. IBM SPSS Amos software was utilized for the CFA.

The results of the CFA showed that the model fit the data well.

Table 7. Confirmatory study: model fit indices.

Fit Index	Recommended Value	Untrimmed Original Model	Observed Value	Reference
CMIN/DF	Between 1 and 3	2.40	2.11	[113]
GFI	≥0.80	0.84	0.86	[110]
AGFI	≥0.80	0.82	0.84	[109]
NFI	≥0.90	0.89	0.90	[110]
TLI	≥0.90	0.92	0.94	[111]
CFI	≥0.95	0.93	0.95	[112]
RMSEA	≤0.06	0.049	0.044	[111]

presents the untrimmed and modified models’ fit statistics. The selected fit criteria for CFA included the Comparative Fit Index (CFI), the Tucker–Lewis Index (TLI), the Goodness of Fit Index (GFI), the Adjusted Goodness of Fit Index (AGFI), the Normed Fit Index (NFI), the ratio of Chi-squared to Degrees of Freedom (CMIN/DF), and the Root Mean Square Error of Approximation (RMSEA). As the results show, all fit indicators meet the recommended cutoff values [109,110,111,112,113], indicating a good fit between the model and the data. Moreover, compared to the original model, the modified model demonstrated better fit indicators, especially regarding the RMSEA, CFI, NFI, GFI, TLI, and AGFI.

CFA was used to assess the convergent and discriminant validities of the constructs. When two or more items measure the same concept, it is called convergent validity [114]. The convergent validity of the measurement items was evaluated by examining the t-values, factor loadings, composite reliability, and average extracted variance (AVE) values (

Table 8. Confirmatory study: descriptive statistics, factor loadings, t-statistics, AVE, CR, and Cronbach’s alpha values.

Construct	Item	Mean	Std	Load.	t Stat.	AVE	CR	Cronbach’s α	Construct	Item	Mean	Std.	Load.	t Stat.	AVE	CR	Cronbach’s α
Account Management	ACC1	6.19	0.805	0.782	21.516	0.595	0.880	0.884	Privacy	PRV1	5.91	1.104	0.900	27.426	0.804	0.925	0.924
	ACC2	6.18	0.832	0.754	20.353					PRV2	5.83	1.132	0.904	27.594
	ACC3	6.10	0.83	0.823	23.350					PRV3	5.75	1.203	0.887	26.758
	ACC4	6.15	0.83	0.719	19.210				Design	DES1	6.25	0.699	0.764	21.282	0.590	0.920	0.921
	ACC5	6.17	0.818	0.777	21.437					DES2	6.15	0.77	0.790	22.341
Branding	BRN1	6.15	0.815	0.852	24.842	0.682	0.896	0.895		DES3	6.12	0.76	0.801	22.742
	BRN2	6.15	0.813	0.839	24.260					DES4	6.05	0.817	0.744	20.463
	BRN3	6.05	0.853	0.848	24.655					DES5	6.14	0.758	0.768	21.390
	BRN4	6.36	0.74	0.763	21.055					DES6	6.18	0.681	0.750	20.527
Navigation	NVG1	6.15	0.731	0.775	21.482	0.664	0.922	0.923		DES7	6.19	0.71	0.745	20.391
	NVG2	6.24	0.715	0.805	22.570					DES8	6.16	0.702	0.784	22.080
	NVG3	6.21	0.724	0.806	22.956				Data Entry and Search	SRC1	6.17	0.750	0.779	21.543	0.585	0.894	0.893
	NVG4	6.27	0.69	0.809	22.948					SRC2	6.09	0.745	0.734	19.550
	NVG5	6.20	0.728	0.845	24.537					SRC3	6.11	0.832	0.761	20.806
	NVG6	6.22	0.72	0.846	24.650					SRC4	6.15	0.808	0.780	21.165
Help	HLP1	6.11	0.867	0.819	23.403	0.658	0.905	0.910		SRC5	6.23	0.792	0.800	22.056
	HLP2	6.06	0.882	0.876	26.035					SRC6	6.25	0.713	0.734	19.769
	HLP3	6.01	0.893	0.841	24.380				Content	CON1	6.28	0.769	0.774	21.778	0.608	0.916	0.912
	HLP4	6.02	0.879	0.762	20.976					CON2	6.19	0.723	0.784	21.886
	HLP5	6.02	0.904	0.750	20.516					CON3	6.29	0.753	0.780	21.887
Accessibility and Customization	CUS1	6.14	0.777	0.812	22.395	0.610	0.862	0.851		CON4	6.27	0.743	0.784	21.907
	CUS2	6.33	0.707	0.826	23.230					CON5	6.18	0.78	0.764	21.352
	CUS3	6.08	0.867	0.711	18.894					CON6	6.27	0.74	0.799	22.618
	CUS4	6.30	0.721	0.771	20.590					CON7	6.18	0.783	0.772	21.594

).

According to the results, at a 95% confidence level, all items’ t-values are statistically different from the critical value of 1.96 [115]. The factor loadings for each item exceed the recommended value of 0.70 [105]. Additionally, all AVE values are higher than the suggested threshold of 0.50 [116]. The composite reliability values, which assess the internal consistency of the measurement model [117], are above the threshold of 0.60 [118]. Moreover, each construct’s Cronbach’s alpha score is above 0.60. As indicated in Table 8, all of these statistics demonstrate that the convergent validity condition is satisfied.

Anderson and Gerbing’s [119] criteria were used to evaluate the discriminant validity of the measurements. χ2 difference tests were applied for each pair of constructs, and the results are shown in

Table 9. Discriminant validity.

Constructs Constrained		df	χ2	Δχ2
None		1025	2167	-
Account Management	Privacy	1026	2262	95
Account Management	Navigation	1026	2430	263
Account Management	Help	1026	2371	204
Account Management	Content	1026	2383	216
Account Management	Branding	1026	2379	212
Account Management	Design	1026	2439	272
Account Management	Accessibility and Customization	1026	2412	245
Account Management	Data Entry and Search	1026	2400	233
Privacy	Navigation	1026	2330	163
Privacy	Help	1026	2271	104
Privacy	Content	1026	2291	124
Privacy	Branding	1026	2276	109
Privacy	Design	1026	2329	162
Privacy	Accessibility and Customization	1026	2312	145
Privacy	Data Entry and Search	1026	2304	137
Navigation	Help	1026	2399	232
Navigation	Content	1026	2419	252
Navigation	Branding	1026	2426	259
Navigation	Design	1026	2441	274
Navigation	Accessibility and Customization	1026	2435	268
Navigation	Data Entry and Search	1026	2459	292
Help	Content	1026	2376	209
Help	Branding	1026	2348	181
Help	Design	1026	2406	239
Help	Accessibility and Customization	1026	2376	209
Help	Data Entry and Search	1026	2388	221
Content	Branding	1026	2363	196
Content	Design	1026	2433	266
Content	Accessibility and Customization	1026	2391	224
Content	Data Entry and Search	1026	2403	236
Branding	Design	1026	2398	231
Branding	Accessibility and Customization	1026	2382	215
Branding	Data Entry and Search	1026	2373	206
Design	Accessibility and Customization	1026	2430	263
Design	Data Entry and Search	1026	2433	266
Accessibility and Customization	Data Entry and Search	1026	2400	233

. The correlation parameter was set to 1 for each pair of constructs, and the tests were conducted for all structure pairs. When a difference in χ2 is significant, it is assumed that the structures are statistically different. All χ2 differences were greater than the critical value of 3.84 at a 95% confidence level. This result indicates that the models correlated to “1” showed a poor fit for all structure pairs (all χ2 differences > 3.841, df = 1, and p = 0.05). Therefore, discriminant validity was achieved, as Bagozzi and Philips [114] suggested.

3.3.3. Nomological Validity

After examining the measurement model, a structural model was evaluated to determine the nomological validity of the OTT usability measurement scale. Three dependent variables were included in the structural model: satisfaction, continued intention to use, and brand loyalty. Satisfaction with a system refers to users’ discrepancy between their expectations of the system and the system’s actual performance [120]. Therefore, it is related to the consumer’s feeling that the system fulfills some of the needs of the customer and leads to pleasure. When satisfaction is aggregated, this feeling may lead to brand loyalty, but not necessarily [121]. Therefore, brand loyalty is a broader concept and especially important in marketing research [11,121]. It is defined as “a deeply held commitment to rebuy a preferred product/service consistently in the future, thereby causing repetitive same-brand purchasing”. In addition, in the IS research, intention to use a system is a commonly used variable to measure the likelihood of a user to use such systems. On the other hand, continued intention to use does not only focus on initial or first-time usages, but also considers a user’s decision to continue using in the long run [120]. Therefore, factors such as satisfaction, brand loyalty, and continued intention to use become important for firms to retain customers in which acquiring new customer costs more than retaining existing ones [122].

Several studies in the literature also revealed the significant effect of usability on continued intention to use, brand loyalty, and satisfaction [46,123,124,125], as hypothesized in the nomological validation of the current study. Lee et al. [123] examined the effect of a mobile phone’s key design factors (simplicity and interactivity) on satisfaction and brand trust via perceived usability. The results show that simplicity and interactivity have significant positive effects on perceived usability, which, in turn, affect both satisfaction and the brand trust. Hoehle et al. [46] defined mobile application usability and then examined the effects of these factors on both brand loyalty and continued intention to use for mobile social media application users. The results revealed that continued intention to use is explained by the usability factors of aesthetic graphics, entry points, fingertip-size controls, gestalt, subtle animation, and transition. In contrast, brand loyalty is explained with control obviousness, fingertip-size controls, gestalt, hierarchy, subtle animation, and transition. Another study conducted by Ramadan and Aita [124] revealed the impact of user experience with a mobile payment application on perceived satisfaction, which, in turn, affects brand loyalty and intention to use. Another study [125] also revealed the significant effect of brand experience on brand loyalty to a website. In that study, brand experience is defined by several factors, and it was found that usability is one of the critical factors in explaining the brand loyalty of a website. In this context, in the current study, it is expected that the usability factors of OTT platforms may affect continued intention to use, brand loyalty, and satisfaction, as confirmed in the literature. We constructed the following hypotheses:

H1 (a–i):

Usability factors of OTT platforms (Account Management (H1a), Privacy (H1b), Navigation (H1c), Help (H1d), Content (H1e), Branding (H1f), Design (H1g), Accessibility and Customization (H1h), and Data Entry and Search (H1i)) have positive effects on satisfaction.

H2 (a–i):

Usability factors of OTT platforms (Account Management (H2a), Privacy (H2b), Navigation (H2c), Help (H2d), Content (H2e), Branding (H2f), Design (H2g), Accessibility and Customization (H2h), and Data Entry and Search (H2i)) have positive effects on continued intention to use.

H3 (a–i):

Usability factors of OTT platforms (Account Management (H3a), Privacy (H3b), Navigation (H3c), Help (H3d), Content (H3e), Branding (H3f), Design (H3g), Accessibility and Customization (H3h), and Data Entry and Search (H3i)) have positive effects on brand loyalty.

The items related to brand loyalty, satisfaction, and continued intention to use were taken from the literature and then adopted to the OTT context. The additional questions related to these dependent variables are shown in

Table 10. Dependent variables used for nomological validation.

Dependent Variable	Items Used	Reference Studies
Brand loyalty	I encourage friends and relatives to be customers of the OTT platform.	[11,120,126]
	I say positive things about the OTT platform to other people.
	I recommend the OTT platform to someone who seeks my advice.
	I consider the OTT platform to be my first choice.
Satisfaction	I am very pleased with the overall experience of using the OTT platform.	[11,127]
	My choice to use the current OTT platform’s services was wise.
	The current OTT platform meets what is needed from an OTT service.
	I think I did the right thing by subscribing to the current OTT platform.
Continued intention to use	I intend to continue using the OTT platform.	[126,128,129]
	I want to continue using the OTT platform rather than discontinue.
	I predict I will continue using the OTT platform.
	Even if other providers offer cheaper plans, I will continue to use the service of this OTT platform.

.

Nine usability factors have successfully explained 51.9% of brand loyalty, 56.4% of satisfaction, and 54.9% of the intention to continue using the product or service (

Table 11. R-squared value of the outcome variables and standardized regression weights of each factor.

	Satisfaction		Continued Intention to Use		Brand Loyalty
	β	Sig. (p)	β	Sig. (p)	β	Sig. (p)
Gender	−0.046	0.147	0.017	0.588	−0.016	0.630
Age	−0.012	0.714	−0.009	0.774	0.003	0.924
Account Management	−0.041	0.515	−0.012	0.848	−0.034	0.604
Privacy	0.161	0.002 **	0.143	0.006 **	0.325	<0.001 ***
Navigation	0.218	<0.001 ***	0.104	0.059 *	0.073	0.195
Help	−0.019	0.699	−0.056	0.265	−0.067	0.194
Content	0.160	0.035 **	0.126	0.095 *	0.276	<0.001 ***
Branding	0.011	0.851	0.018	0.758	−0.014	0.824
Design	0.174	0.011 **	0.260	<0.001 ***	0.178	0.011 **
Accessibility and Customization	0.087	0.128	0.086	0.131	0.035	0.552
Data Entry and Search	0.132	0.055 *	0.190	0.006 **	0.049	0.481

*** p < 0.001, ** p < 0.05, * p < 0.1.

). The factors that influence satisfaction are navigation (21.8%), design (17.4%), privacy (16.1%), content (16%), and data entry and search (13.2%). The influential factors of the continued intention to use the product or service are design (26%), data entry and search (19%), privacy (14.3%), and content (12.6%). Furthermore, brand loyalty is statistically explained by three variables: privacy (32.5%), content (27.6%), and design (17.8%). Furthermore, the results show that the demographic factors of gender and age do not affect satisfaction, continued intention to use, or brand loyalty.

4. Discussion and Conclusions

This study aims to conceptualize the usability of OTT platforms and propose a survey instrument to evaluate the usability of OTT platforms based on the Apple tvOS Guidelines and the literature, employing a methodology that combines three stages and six phases adopted from Lewis et al. [45]. First, the usability of OTT platforms was defined by the concepts of accessibility and customization, account management, branding, data input and search, design, help, navigation, privacy, and content. Then, a survey instrument specifically for measuring the usability concepts was developed. To evaluate the characteristics of the OTT usability measurement scale, two survey studies were conducted with 650 and 600 Netflix web application users, respectively, and the scale’s validity was tested using EFA, CFA, and nomological validation techniques.

In the literature, there are several studies on the usability of OTT platforms and TV interfaces: YouTube [14,15], Netflix [15,16,17], Apple TV [30], interactive digital TV (e.g., [23,25,34,35,36,130]), social TV [21], set-top box (e.g., [32,33]), and smart TV (e.g., [22,24,26,27,28,29,31]). These studies used various usability evaluation methods (surveys, usability tests, eye tracking, think-aloud method, etc.). However, to our knowledge, no study has developed a scale to measure the usability of OTT platforms. Therefore, this study is the first one involving the development of a comprehensive assessment tool to evaluate the usability of OTT platforms. Using the developed OTT scale, usability experts and OTT platform designers assess the usability of OTT systems. Furthermore, a large-scale survey may be conducted with users to gather customer feedback about the system.

The usability of OTT platforms is conceptualized with nine factors specific to OTT platforms, namely accessibility and customization, account management, branding, data input and search, design, help, navigation, privacy, and content. In the literature, there are various studies that highlight the significance of these factors in evaluating the usability of OTT web platforms. The accessibility and customization factor is crucial in usability evaluation, as it impacts the readability of texts and subtitles on OTT platforms. Studies have highlighted the importance of subtitle position [51], readability at different screen sizes [52], font comparison [54,55], and age differences [56,57]. The Google Material Design guide [131] and Microsoft [132] also mention “typography” in text design, and the Web Content Accessibility Guidelines 2.1 [133] suggest text design as a critical factor. Account management and user features are crucial for usability, ensuring that users can easily log in and open new accounts. Concerns about private information, password entry, and authentication options [60,61,62] are addressed, requiring designers to find simple solutions without complexity.

Branding is also essential for usability because it contributes to a consistent user experience and develops trust and familiarity with the product or service. The colors used in the logo and interface layers significantly impact brand perception. For example, Netflix utilizes red in its logo and dominates its interface layers with the same color, while Amazon Prime Video uses blue harmoniously throughout its design. The importance of brand concept is supported by various studies in the literature, including works by Phillips [134], Gultom et al. [67], Govers [68], and Japutra et al. [66]. Data entry and search involve users typing in fields using their keyboards to search for content. Researchers have compared various data entry methods and evaluated their effects on young and old users [28,69,135]. Furthermore, the fact that Microsoft [136] mentions text input and Google [131] includes the concept of search highlights the significance of this factor.

Design and layout are crucial in usability, examining colors, layout design, and graphic elements. Extensive studies have explored the impact of colors on navigation and accessibility [77,130]. Layout prevents complexity [80] and ensures compatibility across different screen sizes [137]. Microsoft [132] and Google [131] have suggested improvements in color and motion, making design and layout crucial considerations for designers. The help factor emphasizes user access to instructions and information and supportive feedback from the system. Research has shown that FAQs [86], help and documentation heuristics [25,35,138], and feedback types [88] are essential components. Microsoft [136] and Google [131] suggest system status and notification concepts related to this factor. The navigation factor refers to the user’s actions on the interface, such as moving between menus or performing activities. The objective is to achieve a natural and familiar navigation experience that does not overwhelm the user interface or distract attention from the content [131] Numerous studies have analyzed navigation concepts in terms of usability. For example, Solano et al. [25] developed a navigation heuristic for interactive digital TV; Obrist et al. [20] investigated the differences between older and younger people in navigation; and Ribeiro et al. [90] and Golja et al. [91] compared vertical/horizontal navigation designs. Google [131] highlights the value of navigation in terms of design.

There are very few studies in the literature that assess the privacy factor in usability evaluation. Geerts and De Grooff [21] identified privacy heuristics, while Shim and Yeon [95] discussed the privacy paradox, where users may compromise their privacy for perceived benefits. OTT platforms manage vast amounts of confidential information, such as user preferences, viewing habits, location, and payment details, which can raise concerns about privacy and data protection. Users may be apprehensive about sharing their personal information, leading to the privacy paradox. To address this issue, OTT platforms must balance providing personalized experiences and respecting users’ privacy preferences. Prioritizing privacy in data management builds trust, ensures compliance with regulations, mitigates data breaches, and safeguards the platform’s reputation.

Content is a crucial aspect of OTT platforms, as it forms the foundation of the platform experience. Research on the content concept has often explored aspects like the intention to subscribe again and the significance of original content [39,59]. High-quality content attracts and engages users. A diverse and engaging content library increases user satisfaction and loyalty, increasing user retention. Seamless content delivery, smooth playback, and minimal buffering are crucial for providing users with a positive viewing experience. Users who can easily access and consume content without disruptions are more likely to enjoy the platform and return for future content consumption. Personalized content recommendations based on user preferences and viewing habits enhance user satisfaction. OTT platforms can establish themselves as key players in the competitive streaming market by offering compelling content, including original and exclusive productions, and ensuring efficient content delivery.

4.1. Implications and Theoretical Contributions

This study’s most important managerial contribution is establishing a categorical framework to evaluate the usability of OTT web interfaces and the development of an instrument tailored to usability experts, software designers, and evaluators. An analysis of the nomological evaluation results revealed compelling findings. Primarily, the intention to continue usage, satisfaction, and brand loyalty are explained by 54.9%, 56.4%, and 51.9%, respectively, thereby validating the efficacy of the nine-category structure. The fact that the “privacy,” “design,” and “content” categories have significant relationships with all three independent variables is essential in guiding designers. Moreover, the “navigation” and “data entry & search” categories significantly influence user satisfaction and the continued intention to use. Thus, designers can mitigate potential usability issues by focusing on these categories.

The open codes and the survey instrument, including nine main categories and 48 associated items, will benefit OTT platform designers and usability experts. This detailed structure provides a comprehensive framework for evaluating and improving the user experience on OTT platforms. For designers, the open codes and the items of the scale serve as guides for creating user-friendly interfaces. They offer specific insights into what aspects of an interface are essential for user satisfaction, such as privacy settings, content search functionality, and content delivery. This allows for designers to proactively address these factors during the development process, enhancing the overall user experience and leading to increased user retention and brand loyalty. For usability experts, the open codes and the OTT usability measurement scale’s items provide a thorough and systematic method for evaluating the usability of OTT platforms. Using this scale as a checklist, usability experts can identify the strengths and weaknesses in an interface and provide specific, actionable feedback to designers for improvements.

Furthermore, this study provides detailed instructions on the steps to be followed in developing an instrument [45], making it a valuable guide for future researchers planning to assess usability in different domains. Consequently, this study can be utilized to evaluate an existing OTT web interface and guide designers during the interface development phase via its categorical structures.

4.2. Limitations and Suggestions for Future Research

While this study provides valuable insights into the usability evaluation of OTT platforms, several limitations may impact the interpretation of the findings. Even though the nomological evaluation yielded promising results (with the continued intention to use, satisfaction, and brand loyalty at 54.9%, 56.4%, and 51.9%, respectively), it is evident that there may be other categories relevant for assessing usability. Therefore, future studies in this field should consider exploring additional categories.

Furthermore, the fact that this study was conducted with Netflix users in Türkiye may raise questions about the global applicability of the developed instrument. Addressing these considerations in subsequent research with users of diverse demographic backgrounds from different countries and cultures on various OTT platforms could lead to a more comprehensive and universally applicable understanding of OTT platform usability. Furthermore, a comparative analysis of the proposed instrument with the existing instruments can provide valuable insights regarding its relative advantages and limitations. Future research should consider conducting such comparative analyses to further validate and refine the instrument for OTT platform usability studies.