Conceptualization and Survey Instrument Development for Website Usability

: The aim of this study is to conceptualize website usability and develop a survey instrument to measure related concepts from the perspective of end users. We designed a three-stage methodology. First, concepts related to website usability were derived using content analysis technique. A total of 16 constructs measuring website usability were deﬁned with their explanations and corresponding open codes. Second, a survey instrument was developed according to the deﬁned open codes and the literature. The instrument was ﬁrst validated using face validity, pilot testing ( n = 30), and content validity ( n = 40). Third, the survey instrument was validated using explanatory and conﬁrmatory analyses. In the explanatory analysis, 785 questionnaires were collected from e-commerce website users to validate the factor structure of website usability. For conﬁrmatory factor analysis, a new sample collected from 1086 users of e-commerce websites was used to conﬁrm the measurement model. In addition, nomological validation was conducted by analyzing the effect of website usability concepts on three key factors: “continued intention to use”, “satisfaction”, and “brand loyalty”.


Introduction
Websites act as gateways for organizations to connect with the world.Despite the increasing prevalence of mobile devices, website usage is still significant among users.According to the statistics, more than 37 million websites are active around the world [1].Most of them have a positive effect on users from the perspective of ease of life.However, many website designs with a low level of usability cause loss of time, demoralization, and disappointment for the user while using the website [2,3].Furthermore, the level of usability plays a significant role in users' adoption of a website.Therefore, usability is an important issue for organizations that develop and use websites to market their products and services [4].Companies with big brands lose significant revenue because their websites are too complex for customers to understand [5].The aim of a human-centered design approach is to develop more interactive systems, resulting in increased usage [6].Institutions and organizations in various fields such as e-commerce sites, banks, aviation, and education depend on websites to respond to changing customer needs.Websites developed within the framework of human-centered design decrease the learning time of users using the system, accelerate expert performance, and consequently minimize user errors [7,8].However, a website with low usage may direct users to use another equivalent or competitor website [9,10].
The concept of usability is based on human-computer interaction (HCI) literature.It helps designers to identify and rank user requirements by proposing guidelines, methods, and approaches [11][12][13][14].As defined by the International Standardization Organization (ISO), "Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use" [15].The specified context of use indicates that the relative importance of website usability metrics may change according to the context of the website.However, determining usability factors for a specific context may be cumbersome for both practitioners and researchers.The concepts developed for website usability in this study may be used for different types of websites in the development and evaluation phases.
Although studies [16,17] were proposed with an aim to measure the usability level of websites from the perspectives of customers or potential customers based on guidelines, such as the Microsoft Usability Guidelines [16] and Research-Based Web Design and Usability Guidelines [18], this study differs in certain aspects.First, we comprehensively conceptualized website usability based on the BS EN ISO 9241-151 guidelines [19] and the corresponding literature.Furthermore, we conducted a survey to assess website usability, which was validated based on the concepts' open codes.Second, in studies that used surveys to evaluate website usability [16,17], the survey respondents were not actual users of the website.Therefore, the websites were first introduced to the respondents.Then, they were instructed to browse and examine the websites before answering the questionnaire.In contrast, the respondents in this study were actual users of the website.This allowed us to examine the relationship between website usability factors and factors, such as continued intention to use, brand loyalty, and customer satisfaction.Therefore, this study contributes to the existing literature on IS/IT adoption by linking website usability factors with the continued intention to use a website, customer loyalty, and customer satisfaction.Third, this study differs from that by [16] in that the website usability instrument was developed with a systematic approach based on the BS EN ISO 9241-151 guidelines [19] and measured using multiple-item measures, whereas Agarwal and Venkatesh [16] used a single item to measure subcategories that lacked statistical validity.Similarly, Hoehle and Venkatesh [20] developed an instrument to measure mobile device application usability based on Apple's usability guidelines, whereas this study proposes an instrument to measure the usability level of websites.Therefore, the aim of this study is to conceptualize website usability based on the BS EN ISO-9241-151 guidelines [19] and the corresponding literature, develop a generic website usability survey instrument, and validate it using a systematic approach with the participation of experts and end users.

Literature Review on Website Usability
A literature review of website usability revealed that it can be examined in several ways, such as user testing with real or potential users [21][22][23][24], expert evaluations through heuristics [25][26][27], and reviews through usability guidelines [16,17,28].
In usability evaluation through user testing, potential users of a system perform several tasks determined by the system provider or analyst to understand the system.Based on the performance of the testers, the usability level of the system is measured using subjective or objective performance indicators.Several user-testing studies have been conducted to measure website usability [21- 24,29].Groth and Haslwanter [21] compared the usability of responsive mobile tourism websites with that of task success level, page views, task success time, and satisfaction.In addition, A/B testing experiments were performed by 20 participants who completed the given tasks, and their performances were calculated and compared.Shasha [22] assessed the usability of Cape Town hotel reservation systems through usability testing.Testing scenarios and questionnaires were given to 55 participants with prior experience in using the Internet to book online accommodation.The results revealed that most participants emphasized content simplicity and understandable product offerings, whereas a smaller percentage found the websites confusing or frustrating.Kous et al. [23] examined the usability of library websites with user testing applied to different types of user groups, such as pupils, students, seniors, and researchers.A total of 25 participants performed the tasks, and their respective performances were measured in terms of effectiveness, efficiency, and satisfaction.Then, group differences among the users were analyzed.Alexander et al. [24] analyzed the influence of user culture on website usability using unmoderated user testing.To compare the cultural differences, websites from China and Australia were translated into English and Chinese, respectively.Then, the tasks defined in user testing were conducted to measure the usability attributes of effectiveness, number of errors, efficiency, and satisfaction.Furthermore, Koonsanit et al. [29] indicated that 60 participants engaged in task-based evaluation of a travel agency website.To address variations in user behavior, the present study utilized randomly ordered tasks that must be completed by the participants in different sequences.The results demonstrate that considering the actual order of user actions over random ordering significantly influences the prediction of final user satisfaction.Although the studies that conducted user testing for website usability provided insights into the usability of the system, the problems associated with usability determined with user testing were limited to the tasks defined and did not provide a comprehensive perspective for overall website usability.However, this study provides a comprehensive way to conceptualize website usability based on several concepts and developed a survey instrument to measure website usability from the perspective of actual website users.
Some studies have also used heuristics to evaluate and identify website usability problems [25][26][27]30].Torrento et al. [30] defined usability evaluation conception based on heuristics and named the system "Sirius".Experts evaluated websites by considering the proposed evaluation criteria, and a usability metric that quantified the level of usability achieved for different types of websites was developed.Fung et al. [25] conducted a heuristic evaluation using Nielsen's ten heuristics for the mobile websites of three university libraries.These websites were examined and compared in terms of usability issues encountered by experts, and possible solutions to the usability problems were proposed.Verkijika and Wet [26] assessed the usability of 279 e-government websites in 31 countries using a heuristic evaluation method.Five experts conducted evaluations using the heuristics of online services, user help, navigation, information architecture, and accessibility.Silvis et al. [27] proposed a set of heuristics by combining Nielsen's ten heuristics [8], the golden rules of Shneiderman [31], and the dialogue principles of ISO 9241-110 [6].Each heuristic was explained with reference to the literature, and recommendations related to each heuristic were proposed.Similarly, in the present study, the open codes defined for each concept helped experts develop and evaluate website usability.
A group of studies has focused on understanding the user perspective regarding the usability of specific systems in various domains, such as tourism websites [32][33][34], government websites [35,36], e-commerce websites [37][38][39], or university-related websites [40][41][42][43].These studies provide valuable insights into the usability aspects of websites in specific domains and help improve user experience and design in these areas.However, the present study's conceptualization concerned website usability for different types of websites, thus providing a comprehensive description of website usability.
However, certain other studies conceptualized website usability based on guidelines, such as the Microsoft Usability Guidelines (MUG) [16,28], Research-Based Web Design and Usability Guidelines [17], and the literature [44].Agarwal and Venkatesh [16] described a method to determine the level of usability of websites using a heuristic evaluation.The set of heuristics was adapted from the MUG, which has five major categories: content, ease of use, promotion, made-for-the-medium, and emotion.Four of the five categories have subcategories, and each subcategory is measured using a single-item measure.The respondents first examined and browsed one of the four websites, namely airlines, bookstores, auto manufacturers, and car rental agencies, and then answered the questionnaire as a customer or investor.A total of 1475 questionnaires were collected, and a comparison was drawn across industries and roles.Venkatesh and Ramesh [28] conducted a follow-up research to the study by Agarwal and Venkatesh [16], who used the same usability model to examine the usability of multiple websites across four industries to demonstrate the generalizability of MUG conceptualization.Within the scope of [28], interviews were conducted, and questionnaires were collected from the actual users as a subset of Agarwal and Venkatesh's previous study [16] to assess wireless site usability and explore distinctions between web and wireless sites.This model was tested in Finland to apply to the airline industry that one of the sectors studied [16].First, participants operated as customers and indicated the relative importance of the MUG categories; after browsing each website, they rated the airline industry website with MUG on a 10-point scale.The study analyzed how users prioritize the assessment of usability between web and wireless sites based on weighting MUG metrics.The presented results offer robust endorsement for employing MUG as a means to comprehend site usability and usage in both web and wireless environments and demonstrate the generalizability of the MUG to Finland.Venkatesh et al. [17] developed a usability instrument based on Research-Based Web Design and Usability Guidelines [18] to evaluate the usability level of the healthcare insurance website in the U.S. The instrument consists of 16 dimensions to measure the overall usability level and 205 items representing the dimensions.A total of 374 citizens answered the questionnaires, and only 144 of them had previously used the website.Therefore, at the beginning of the survey application, the website was introduced to respondents through video clips and screenshots.According to the results, six (user experience, content organization, navigation, graphs, lists, and screens) of the sixteen usability dimensions were found to be predictors of citizen satisfaction and intention to use e-government portals.Venkatesh et al. [45] used the same dimensions as Venkatesh et al. [17] and found that five dimensions, namely hardware and software, home page, screen, scrolling and paging, and user experience, were rated low in terms of usability.Then, the usability problems for each dimension were determined, and recommendations for the problems were proposed.Lee and Kozar [44] analyzed the common dimensions of website usability through a comprehensive review of 27 academic journals and conference papers.Ten constructs, namely telepresence, supportability, consistency, content relevance, credibility, readability, learnability, simplicity, navigability, and interactivity, as well as their corresponding survey items were defined based on the literature.The causal relationships among the constructs were defined using the quantitative cognitive mapping method.Additional data were collected from 711 Amazon website users, and the relationships among the constructs and their effects on purchase intention and purchase were revealed to validate the nomological network of website usability.
A comparison of this study with the existing literature reveals that it contributes to the literature in several ways.First, it covers the points of the BS EN ISO-9241-151 guidelines [19] offered by the ISO, which aim to help website developers design websites that are usable.However, current guidelines do not provide important points, nor do they provide a way to evaluate websites in a concrete manner.For example, the widely accepted scale, The System Usability Scale (SUS) [46], commonly used for the rapid measurement of system usability, provides general usability satisfaction but does not contain information related to usability issues.Our detailed instrument addresses this gap, which provides a comprehensive evaluation that measures overall satisfaction and identifies specific usability issues.In this study, website usability was conceptualized based on the BS EN ISO-9241-151 guidelines [19], and additionally, literature was included.Subsequently, a comprehensive and context-specific survey instrument was developed and validated to help developers design and improve websites according to the defined usability criteria.Furthermore, the present study is the first to develop and validate a fine-tuned instrument for measuring website usability.Second, the concepts defined by the open codes may be used as a reference list to check the conformance of a specific website to the defined open codes from the experts' perspective.Therefore, the proposed website usability evaluation system can be used effectively by end users and experts.

Conceptualization and Instrument Development
In this section, we implemented a three-step formal methodology.First, website usability was conceptualized using the axial and open coding content analysis techniques proposed by Corbin and Strauss [47].Subsequently, based on the concepts and open codes defined in the first step, we developed a website usability survey instrument.Finally, we validated the survey instrument and proposed an instrument specifically for website usability measurements.

Conceptualization of a Website Usability
In this study, BS EN ISO-9241-151 is considered the base content for the conceptualization of website usability.We applied Corbin and Strauss's [47] open and axial coding procedures for conceptualization.Open coding is "the analytic process through which concepts are identified, and their properties and dimensions are discovered in data" [47].In general terms, data are segmented into distinct pieces during the open coding process, carefully analyzed, and assessed for similarities and differences [47].We examined the BS EN ISO-9241-151 guideline to implement the open coding procedure and coded the content using line-by-line analysis.Furthermore, the literature related to website usability was reviewed to include additional open codes [16][17][18]30,48].The final list of the open codes was examined by five experts experienced in HCI.Next, the experts grouped the open codes that were conceptually similar or related under subcategories.
In later analytic steps, axial coding was implemented.Axial coding is defined as "the process of relating categories to their subcategories, and it is termed "axial" because coding occurs around the axis of a category, linking categories at the level of properties and dimensions [47].Categories and subcategories were linked in axial coding to create more thorough and accurate conceptual explanations.For example, we identified two open codes related to error prevention on web pages: (1) The content of error messages should be shown on web pages at related positions with clear reason statements, and (2) the user should be directed to solve basic errors by oneself.Then, these two open codes were conceptualized as a single subcategory labeled "Understandable Error Messages".Then, through axial coding, the category was identified as "Error Prevention".
The open codes, their corresponding subcategories, and axial codes are given in Appendix A-Table A1.A total of 16 axial codes were defined: brand identity, content design, personalization, and customization; privacy and business policies; page design, text design, and link design; interaction objects; graphics, images, and multimedia; navigation; search; help; responsiveness; error prevention and collaboration; and cultural diversity and multilingual use design.

Brand Identity
Users' perception of a website is formed within the first few seconds of a visit, and this impression is largely based on the website's visual design and branding elements [49] because websites represent businesses [38,50,51].A memorable website URL or informative homepage affects the branding identity and initial impression of a website user [52,53].Furthermore, an identifiable and informative website consistent with its intended purpose and target audience is crucial for establishing users' trust and credibility [54,55].As a result, it is easier for users to navigate and use such websites effectively [56].

Content Design
Website content should be designed according to users' goals, knowledge, and preferences.The comprehensiveness, richness, and completeness of the content are also considered as key characteristics of a website [57].Effective website content enhances user satisfaction, reduces frustration, and improves business outcomes [58].According to Agarwal and Venkatesh [16], website content is the most crucial attribute of users' website preferences.The quality, accuracy, and reliability of information are also critical for usability, which makes a website more credible [59,60].The content must have an appropriate level of detail and organized information and must be presented in a clear and logical structure that can be understood by typical users [19,61,62].A clear, logical structure enhances user interest, whereas a mismatch between the conceptual and user's mental models results in confusion and frustration [18,63].Furthermore, time-dependent content must be up-to-date, and the users must be aware of the last update or validity period [60].

Personalization and Customization
Personalization involves dynamically adapting the content, recommendations, and user interfaces based on the specific characteristics and behaviors of individual users, whereas customization allows users to modify various aspects of a website's design and functionality.By combining these approaches, websites can create highly personalized and adaptable experiences for every user [64].Some authors have highlighted that personalization improves trust perception and usability [65,66].A personalized website provides an experience based on user-specific data, such as location, browsing history, and search queries [67].It is the ability of a website to adapt to users' actions and behavior over time [68].The dynamic nature of user interests and consumer behavior is influenced by easy access to abundant information and social factors [69].By utilizing technologies such as analytics and advanced machine learning algorithms, vast quantities of data can be analyzed to forecast user preferences and behaviors.Thus, websites should employ analytical processing to enhance learning, user profiling, and predictive models, ultimately providing users with valuable and personalized recommendations [67,[69][70][71].However, customization empowers users to personalize their own experience by giving them control over certain aspects of the website's design and functionality.Venkatesh and Ramesh [28] argued that customized websites cater to users' desires in a short period of time.However, it is important to balance personalization and customization to avoid overwhelming users with too many options or too much information.

Privacy and Business Policies
Website usability and user reliance are closely linked to clear and concise privacy and business policies outlining data collection, usage, and sharing practices [72].A website's privacy policy outlines how user data are collected, used, and shared, whereas business policies provide details on the website's terms and conditions for conducting transactions with users [73].Transparent privacy policies may increase user trust and engagement during transactions in website [74,75].When collecting personal information for personalization, website policies should establish user trust and protect user rights [19,[76][77][78][79]. Several studies have shown that usability is inversely associated with the lack of privacy and security perspectives in areas such as online learning, e-banking, and e-commerce [80][81][82].For instance, many consumers are concerned about their privacy and security during purchasing transactions, especially in the e-commerce domain [83].Confusing or difficult-to-locate policies may negatively impact user experience and lead to frustration and abandonment [84].Similar to privacy, if required, transparent business policies also can increase user satisfaction and trust.

Page Design
Page design properties are significant in accurately conveying content to the user while ensuring the visual integrity of the content.Presenting content via stable, standard webpage elements familiar to users, such as page structure, list level, and style, is important for usability.Well-designed pages improve user experience, increase engagement, and promote conversion rates [85].A study on e-commerce remarked that the conceptualization of website aesthetics affects the website context [86].Well-placed headings and short and readable paragraphs are essential for page design; therefore, it is easy for the user to scan.A clear and intuitive layout can guide users through the page and help them quickly find what they are looking for [87].Furthermore, the visual hierarchy may direct users' attention to key elements on a page and facilitate information processing [88,89].It should be ensured that the desired topic is highlighted on a page.According to the research of Leavitt and Shneiderman [18], 80% of users pre-scan when looking at a webpage and aim to find what they are looking for at a single glance.

Text Design
Consistently readable content purged of technical language increases usability.The text format of website content is one of the basic design elements of a website [90,91].The use of clear, concise, and easy-to-read text may improve comprehension and task completion [92].However, poor text design, such as small fonts or poor color contrast, results in user frustration and disengagement.Information on websites should be displayed in an appropriate, de facto font size [93][94][95][96].Research-Based Web Design and Usability Guidelines [18] and other studies advise using a font size of at least 12 points and consistent font formats in all pages [97,98].Using simple and clear language, avoiding jargon and technical terms, and breaking down information into manageable chunks may improve website usability [99].

Link Design
The role of links is to ensure that internal and external page connections on the website are directed to the correct address when an article, photograph, or object is clicked.Welldesigned links may help users to navigate websites efficiently, whereas poorly designed links can lead to confusion [100].The most important aspect of link design is that it should be distinguishable from ordinary content [101].Links can be presented in different ways via text, buttons, etc., as long as they are discriminated from other content on the website pages [19].Furthermore, the use of descriptive links rather than generic phrases improves users' understanding of the link's destination and increases their trust in the website [102][103][104][105].According to previous studies, users are more likely to click on links that are underlined and highlighted in a color different from the surrounding text [106][107][108].

Interaction Objects
Interaction objects allow users to interact with the websites.They are defined with buttons, check boxes, dropdown lists, and any data-entry fields, such as completing forms and texting into boxes that allow the user to interact with the website conversationally [19].Minimized data-entry sections and simple mechanisms for user interaction should focus on website usability.To ensure the effective use of interaction objects, designers should ensure that they are easily recognizable and that their functions are clearly labeled [58,109,110].The use of consistent interaction objects throughout a website also improves usability, as users can learn and recognize them quickly [31].Furthermore, using realistic interaction objects that are compatible with the tasks and expectations of the user enhances website usability by reducing the cognitive load and contributing to trust [111].In addition, three-dimensional (3D)-style buttons enhance the recognition of clickable buttons and their accessibility [112].Interaction objects should also be located in easily accessible and used locations, such as in the main navigation bar or header of the website, which ensures that the interaction objects are placed with the alignment of user expectations and behaviors [113,114].Moreover, each entry field should be clearly and consistently labeled in the appropriate positions to guide the user interactively [18].Manual information entered externally by users should be minimized as much as possible.

Graphics, Image, and Multimedia
When attractive content, such as images, animation, video, and audio, is used appropriately, it provides additional value to a website [115].Research has also shown that the quality of graphics and images affects the perception of the user about a website and the credibility of the information presented [116].Featuring images and graphical elements within a website transmits messages more accurately to users and improves the website's overall usability and the perceived attractiveness of websites [117,118].Moreover, promoting the use of appropriate graphics, images, and multimedia enhances website aesthetics, resulting in user satisfaction [95,119].However, there must be a balance in the use of multimedia objects.Excessive use of these elements may cause distractions and have a negative impact on website usability [120,121].Images or videos with large file sizes can significantly affect download times, particularly at slower connection speeds, owing to the substantial number of bytes required.The loading speed of graphics and multimedia should be optimized to prevent slow loading times so that users do not get frustrated and abandon the website [101,122,123].It is beneficial to introduce a medium element before initiation.By utilizing thumbnail versions of larger images, users can preview images without fully downloading them.This approach not only provides a convenient way to preview images but also helps prevent any slowdown in the download time of web pages [17,18].HHS design guidelines propose reducing image size so as not to slow down page loading [124].

Navigation
Clear navigation provides users with an understanding of the website's overall structure.Many researchers emphasize the importance of the ease of navigation for usability because effective navigation structures [125] allow users to find information easily [14], thus reducing the users' effort and time spent on navigation.The homepage serves as the website's primary navigation element.It is crucial to ensure that the main navigation links are prominently displayed and easily accessible whenever users seek to accomplish their goals on the site.According to Yu and Roh [126], different menu designs affect user perceptions of easy navigation within a website.The placement and design of navigation menus may affect website usability because users often expect to consistently find navigation menus in certain locations [102].Moreover, establishing a consistent logical mapping between the structure of a website's content and selecting a suitable placement of navigation elements is important [127].Additionally, splash screens are often used to provide an initial visual experience when a website or application is loaded.According to ISO 9241-11-2018 [15], if a splash screen is used, users must be provided with a navigation option to bypass the screen.

Search
Search functionality is an essential aspect of website usability, as it allows users to quickly and easily find the required information [128][129][130].Adequate research has been conducted to show that search functionality can significantly improve user satisfaction and task-completion rates [131][132][133].According to Spool et al. [107], searching can be a faster and more efficient way for users to find content than browsing menus and links.However, the usability of a search feature depends on factors, such as the search algorithm, which generates pertinent outcomes based on anticipated keywords commonly used by users [134], presentation of search results, and availability of advanced search options.To optimize the user experience, it is recommended to minimize the search interface design and eliminate any superfluous elements that may cause distraction [135].According to the Nielsen Norman Group, users who cannot find the desired information in their first search rarely attempt the next search, and almost half of them tend to leave the system [136].As part of the search mechanism, website designers may use advanced techniques such as auto-suggestion, faceted search, and semantic search to enhance their search experience [137,138].Furthermore, the search results shown to the user should be ordered according to the relevance of the required information.Mechanisms such as sorting and filtering can enhance the performance of a website by allowing users to arrange search results based on their specific needs.

Help
Users complete tasks successfully and express satisfaction when provided with clear and concise instructions or guidance.The use of interactive tutorials, context-sensitive assistance, and error messages with helpful hints may also improve usability.Furthermore, it is important to ensure that help features are visible and easily accessible to users without overwhelming or distracting them from the primary task.In addition, it is important to identify a contact who can obtain information or solve problems [19].Helping pages, a section on frequently asked questions (FAQ), or special assistance such as chatbots should be easily accessible.Additionally, incorporating natural language processing and machine learning techniques may improve the accuracy and relevance of help responses [139].

Responsiveness
Responsiveness is an important component of system and service quality [140].Responsive website design allows a website to adjust its layout and content according to the size and type of the device being used [141].Research has confirmed that poor responsiveness may force users to stop using websites, especially during critical system requirements [79,140,142].A website's user interface can be achieved through various input devices; therefore, a robust design that is technology-independent should be adopted.In addition, it is important to keep up with commonly used browsers and screen sizes [143][144][145], and the site should function efficiently on devices with similar designs [146].

Error Prevention
It is essential to implement techniques such as error prevention, error correction, and clear error messages so that users can navigate websites efficiently.Minimizing the effort spent on recovering errors is essential in website design.Thus, to support recovery errors, potential user errors should be minimized as well.It is advisable to use contextual information and visual cues to guide users through websites via lucid and concise messages [114,[147][148][149]. Essentially, error messages or alerts should be designed to be easily comprehensible to an average user rather than exclusively targeting IT specialists [19,150].However, an efficient system design can prevent such problems [19,151].In addition, designing error-prevention mechanisms such as automatic saving, undo and redo functions, and confirmation dialogs may significantly reduce the occurrence of errors and increase user productivity [152].

Collaboration
A system with collaborative features provides users with opportunities to interact with other users, share information, and provide feedback to the system, such as through ratings, voting, and comments.Recent technological developments have enabled websites to project a personal environment, offer support from experiences shared by other people, and provide motivation from the power of the community rather than just browsing [153].User-generated content such as comments and reviews may enhance website credibility and increase user engagement [154].Adequate research has been conducted to show that customer reviews and ratings may influence purchasing decisions, improve customer loyalty, and increase user trust in websites [155][156][157][158].However, it is important to ensure that user experience features are easy to use and understand because overly complicated systems can detract from usability [159].Recommendation systems for website collaboration mechanisms may also encourage users to use the website.

Cultural Diversity and Multilingual Use Design
The use of language, symbols, color, and layout may be considered [160] as crucial for supporting cultural diversity and multilingual use.Usability is closely linked to social and cultural characteristics, and website designers should consider the relevant characteristics of different user groups to improve usability [24,161,162].Furthermore, providing multilingual options and localized content effectively enhances usability, increases business opportunities in global markets [119,163], as well as increases user engagement and satisfaction.For instance, Tsai [164] revealed that users in the U.S. prefer more personalized pages, whereas Taiwanese users prefer collective pages with more images and icons; therefore, the design differs for the same website.Culture and language affect information processing and interaction between users and interfaces as seen in examples, such as Korean, Chinese, and Arabic web pages [124,[165][166][167].Moreover, Ying and Lee [168] revealed that users expect to browse website content along different paths related to their cultural cognitive style.To accommodate cultural diversity and multilingual use, website designers should use visual cues, offer language selection options, and consider the cultural implications of design choices such as text, interaction objects, and input elements to enhance success and user acceptance [24,169].Table 1 presents the constructs, their definitions, and the literature cited in these constructs.

Survey Instrument Development
A survey instrument specific to website usability evaluation was developed based on the concepts developed in the first phase.First, five researchers examined the open codes in detail and developed a list of possible items to explain the 16 constructs.In this phase, a total of 134 items were used to define 16 constructs.The survey instrument was validated using face validity, pilot tests, and content validity checks that were performed simultaneously.The face validity of the survey instrument was assessed with a checklist that included the following questions: "Applicability of the items", "Is the item redundant and must be deleted", and "Is this item ambiguous and must be deleted".Ten participants examined each item according to the checklist and provided relevant comments and suggestions.
The participants' reviews were evaluated by the researchers of the study, possible changes were discussed, and a decision regarding the items was made.Based on the decision made, we checked whether removing an item changed the content of the construct, and it was considered that each construct was represented by at least four items.Consequently, 91 items were available for the pilot test.
In the pilot test, the survey instrument prepared in English was translated into Turkish.A bilingual translation was conducted to ensure equivalency of the Turkish and English versions of the survey instrument.The participants answered questions on the most commonly used e-commerce websites.Accordingly, 30 users with shopping experience on e-commerce websites and representing the main population participated in the pilot test study.Thereafter, the respondents' demographic profiles were framed as shown in Appendix B, Table A2.The participants were requested to review each evaluation item in the survey and provide comments on any item.Based on participants' evaluations, complex and incomprehensible items were revised.Additionally, items deemed unsuitable for evaluation by end users that could not be evaluated by the participants were removed from the survey.Of the 91 items in the pool, 11 were marked for correction.Considering that each structure needed to be described by at least four items [20], seven items were eliminated during the pilot test, and 84 items remained for the next content validity phase.
Third, a content validity approach was used to screen the items of the survey instrument.Content validity is the degree to which a developed scale represents the domain of a relevant construct [170].To verify the items, a content validity check was applied following Andersen and Gerbing's [171] approach.In this approach, the final items are listed in the rows, and the definitions of the constructs are given in the columns.The respondents were then asked to select the most appropriate item and construct a match as shown in Table 2. Hunt et al. [172] suggested that the minimum number of participants for assessing scope validity is at least 12. Within the scope of the study, 40 participants who had experience with shopping on e-commerce websites were requested to fill out a matrix containing the evaluation items from the survey and the most appropriate categorical structure matching.Content validity of the data was assessed using the proportion of substantive agreement (P sa ) and substantive validity coefficients (C sv ) as explained by Anderson and Gerbing [171].The value of P sa was calculated as the ratio of the number of respondents who positioned items to the intended construct to the total number of respondents.The value of P sa ranges between 0 and 1, and higher values for each item indicate a higher level of content validity.C sv is the ratio at which respondents positioned items as the most chosen construct rather than the intended construct.It was calculated as the ratio of the difference between the number of assignments to the intended construct and second-highest assigned construct to the total number of respondents.The range of C sv values typically is between −1 and 1. Positive values indicated that items were assigned to their intended constructs more frequently than to any other construct [171].Negative values indicated the opposite.Higher values for both indices indicated greater substantive validity.Usually, 0.5 is the recommended threshold for sufficient validity [171,173].However, a threshold value of 0.6 was selected for P sa and C sv proportions, as suggested by Hoehle and Venkatesh [20] for more prudent results.
In this study, questionnaire items with P sa and C sv values of 0.7 and above were accepted directly, while those with values between 0.5-0.7 were revised and used.Those below 0.5 were excluded from the survey: With a content validation study conducted on 84 survey items, 9 criteria were identified.Overall, 62 items were accepted directly, and 13 items were revised and reevaluated.Consequently, 75 items remained at the end of this stage.Items were calculated above the threshold value and accepted as shown in Appendix C, Table A3.

Evaluation of Survey Instrument
In the third stage, the measurement properties of the instrument were evaluated and optimized through exploratory and confirmatory analyses.Data were collected in two stages from two independent e-commerce website users.Exploratory factor analysis (EFA) measured the factor structure in the first sample, whereas while confirmatory factor analysis (CFA) confirmed the measurement properties of the developed scale in the second example [170].
The survey instrument was designed to measure all items on a 7-point Likert-agreement scale (1 = strongly disagree to 7 = strongly agree).Participants who had shopping experience on the websites of Trendyol, Hepsiburada, and N11, which are commonly used e-commerce companies in Turkey, participated in the survey.

Exploratory Factor Analysis
EFA was used to determine the underlying dimensions of the items.In the analysis, 785 responses were collected from e-commerce website users.The demographic profiles of the respondents are given in Appendix B, Table A2.Python programming (via Jupyter Notebook IDE) was used to perform the analyses.
First, Bartlett's and Kaiser-Meyer-Olkin (KMO) tests were conducted to determine whether the data obtained from the survey were suitable for factor analysis and normality testing [174].The calculated KMO value of 0.93 was above the recommended threshold of 0.6 [175].Furthermore, the Bartlett's test chi-square value was 35,099.32 and significant at a 99% confidence interval.Therefore, the collected data were suitable for factor analysis.In this study, exploratory factor analysis with varimax rotation was conducted using the maximum likelihood (ML) method to ensure the maximum variance of factors with fewer variables.In the EFA, eigenvalues greater than 1 were considered, and a total of 16 constructs were obtained, which is the same number as decided in the conceptualization stage.The constructs explained 64.18% of the observed cumulative variance, and item loadings are presented in Table 3. Items with factor loadings greater than 0.6 were assigned to their respective factors.A total of four items, which had ('GRPH3 (0.24), 'HELP5' (0.55), 'NAVG8' (0.40), and 'PAGE1' (0.44)) low factor loadings and were unrelated to any construct, were dropped from the survey instrument.A total of 71 items representing 16 constructs remained for the CFA.Based on item loadings, each factor was labeled with construct names.Furthermore, Cronbach's alpha for the underlying factors ranged from 0.84 to 0.90, thus satisfying the suggested level of construct reliability [176].The final user-centered items list obtained according to the results is given in Appendix D, Table A4.[170] reevaluated the final construct obtained from EFA through confirmatory evaluation.CFA is a valuable analytical approach for evaluating the reliability of measurement criteria and assessing the alignment between theoretical concepts and structural equation models [177].CFA is conducted by employing an independent random sample that is different from the one used in EFA.Then, the survey items are reevaluated, and a rigorous assessment of the measurement structure is performed.Second, Lewis et al. [170] suggested that the nomological network of the scale should also be evaluated as part of the validation assessment.To assess the nomological validity of conceptualized categorical structures, it is essential to gather data that are theoretically related to these structures [178].
To perform CFA, an independent sample was taken from end users who had experience with e-commerce websites and did not participate in the first study.A total of 1086 people agreed to participate in the research, and the demographic profile of the participants is shown in Appendix B, Table A2.IBM SPSS AMOS 26 program was used to analyze the data.In the measurement model, the constructs represent latent variables and survey items represent observed variables.A total of 71 items representing 16 constructs were entered into the measurement model.
First, the skewness value of the data obtained from the end user and normality of the data were tested.Then, the skewness value of all evaluation items was calculated between ±2, and a proof was obtained that the data were in accordance with the normal distribution [176,179,180].To determine the adequacy of the model, model fit indicators were calculated as shown in Table 4. Chi-square, degrees of freedom ratio (CMIN/DF), comparative fit indices (CFI), Tucker-Lewis index (TLI), incremental fit index (IFI), root mean square error of approximation (RMSEA), and standardized root mean residual (SRMR) values were used to check the conformality of the data.The fit indices of the model from the goodness-of-fit statistics were as follows GFI = 0.938, CFI = 0.993, IFI = 0.993, and TLI = 0.992; error statistics was evaluated using RMSEA = 0.011 and SRMR = 0.0235.The chi-square and degrees of freedom ratios (CMIN/DF) were 1.150.The findings revealed that the fit of the model to the data was satisfactory [176].The convergent validity of the confirmatory model was evaluated using standardized factor loadings, t-values, Cronbach's alpha, composite reliability (CR), and average variance extracted (AVE) values [188].All items' standardized factor loadings, which were between 0.7 and 0.85, exceeded the suggested threshold value [176,181], and the t-values of items were significant at a 0.01 confidence level [189].
The internal consistency reliability of the measurement model was assessed using Cronbach's alpha and CR values.Table 5 presents the results of CR, AVE, and the correlations between each construct.The Cronbach's alpha values ranged from 0.79 to 0.91, exceeding the threshold value of 0.70 [190].The CR values were greater than 0.83, and AVE values were greater than 0.54.The minimum value of the CR values calculated for the "Error Prevention" construct was 0.79; therefore, all values were greater than the recommended value of 0.70 [191].Additionally, the AVE values of the conceptual constructs exceeded the threshold value of 0.5, indicating convergent validity for all factors, as the minimum acceptable values were encountered [176,192].
The constructs' discriminant validity was assessed using the approach suggested by Fornell and Larcker [192].In this approach, the discriminant validity of any construct was achieved if the square root of the AVE of each construct was greater than its correlation with the others [177,193].The square roots of the AVE values calculated in this study were greater than their correlation values with other structures, thus indicating that the conceptual structures achieved discriminant validity [192], as shown in Table 6.In order to evaluate the nomological validity of website usability, the impact of website usability constructs on "continued intention to use", "satisfaction", and "brand loyalty" were examined.These three structures were identified and adapted from previous studies in the literature [194][195][196][197][198]. The nomological items listed in Table 7 were included in the website usability survey.
Continued intention to use is a concept of website usability that refers to a user's intention to continue using a website after its initial use.It is a critical factor in determining the long-term success of a website, as it directly influences user loyalty and retention [60].Satisfaction is defined as the user's level of contentment or feeling pleased with a website's usability, features, and overall experience, such as repeat visits, recommendations, and purchases [54,56].Furthermore, satisfaction is influenced by many factors, including website design, usability, functionality, and content [199].Brand loyalty is defined as the commitment of a customer to a particular brand and the willingness to make repeated purchases of that brand over time [200].I am pleased with the experience of using the website.

STF3
The website information content met my expectations.

STF4
My decision to use the website was a wise one.
Continued intention to use ITU1 I intend to continue using the website rather than discontinue its use.[194,195,197,201]

ITU2
My intentions are to continue using the website rather than using any alternative means.

ITU3
It is likely that I will use the website in the near future.

ITU4
I intend to conduct my online shopping transactions via the website in the near future.
Brand Loyalty BLY 1 I prefer my website to other websites.
[198] BLY 2 I recommend my website to others (e.g., friends, family).BLY 3 I say positive things about the website to other people.BLY 4 I encourage friends and relatives to be users of the website The relationships between website usability factors and the nomological constructs were tested.Table 8 presents the results of the study.It was observed that website usability constructs have a positive effect on the following nomological factors: "continued intention to use", "satisfaction", and "brand loyalty."These significant relationships provide evidence for the applicability of the developed survey instrument [170]: A total of 11 constructs, including "brand identity", "privacy and business policies", "page design", "text design", "interaction objects", "graphics, images and multimedia", "navigation", "search", "help", "responsiveness", and "collaboration", explains the 61.3% total variance of satisfaction.

Discussion
In this study, we conceptualized website usability and using a validated construct and measured it via survey instrument.The concepts and their open codes were derived from BS EN ISO-9241-151 as a base document, and the literature was examined in detail to identify possible additional concepts for website usability.A total of 16 constructs were extracted, namely "brand identity", "content design", "personalization and customization", "privacy and business policies", "page design", "text design", "link design", "interaction objects", "graphics, images, and multimedia", "navigation", "search", "help", "responsiveness", "error prevention", "collaboration", and "cultural diversity and multilingual use design", with the objective of measuring website usability.Each construct was explained using subconstructs and their corresponding open codes, which were used as comprehensive guidelines to assist developers in enhancing website usability during the design process.
Based on these concepts, a website usability survey was developed and validated using a systematic procedure.First, a pretest, pilot test, and content validity checks were conducted to ensure the validity of the survey instrument.Then, a large-scale survey was conducted to conduct EFA of e-commerce website users.It was used to assess the factorial validity and reliability of the constructs.The results show that website usability can be measured using the 16 constructs defined in the conceptualization step.Finally, CFA was performed using the second sample to validate the results obtained from EFA and examine the generalizability of the survey instrument.It was observed that the effects of website usability factors on continued intention to use, brand loyalty, and customer satisfaction were significant.In addition, open codes defined for each construct were used by experts to evaluate websites in conformance with usability principles.The developed survey instrument was used partially or completely to measure user perceptions of a specific website.Therefore, experts' and end users' views should be more examined in the evaluation of website usability.
Website usability satisfaction refers to the users' overall positive evaluation of their experience while interacting with a website.A high level of satisfaction indicates that the website was usable, enjoyable, and fulfilled their needs, resulting in a positive affective state and favorable perception of the website.The study revealed that a total of 11 constructs, namely "brand identity", "privacy and business policies", "page design", "text design", "interaction objects", "graphics, images, and multimedia", "navigation", "search", "help", "responsiveness", and "collaboration", significantly contributed to explaining 61.3% of the total variance in satisfaction.
In terms of website usability, continued intention to use refers to users' willingness and inclination to continue using a website in the future.It is influenced by users' satisfaction with a website's usability and perception of its ongoing value and relevance.When users perceive a website to be highly usable and beneficial, they are more likely to express their intention to use it repeatedly, suggesting a desire for sustained engagement and interaction.The findings of our study demonstrated that a total of 10 constructs related to website usability, namely "brand identity", "privacy and business policies", "page design", "text design", "interaction objects", "navigation", "search", "help", "responsiveness", and "collaboration", collectively accounted for 61.7% of the total variance in continued intention to use.
Brand loyalty, within the context of website usability, represents users' commitment to and preference for a particular website over its competitors.It is characterized by users' consistent and repeated use of the website owing to their positive experiences and high satisfaction with its usability.Brand loyalty implies a strong connection between users and a website, engendering trust, loyalty, and a willingness to recommend the website to others, thereby contributing to its sustained success and market position.Ten constructs, namely "brand identity", "content design", "privacy and business policies", "page design", "text design", "link design", "graphics, images, and multimedia", "navigation", "search", and "collaboration", collectively accounted for 52.7% of the total variance in brand loyalty.

Theoretical and Practical Implications
The findings of this study highlight the significance of considering website usability during the design, development, and evaluation processes.Developers can create websites that enhance user satisfaction, encourage continued use, and foster brand loyalty by following developed website usability concepts.This study provides valuable insights and practical implications for developers and businesses seeking to optimize website usability and establish strong connections with their users.The open codes generated in the study are structured to be used as a checklist for professionals working on website development.Additionally, they can be used to understand usability issues on existing websites.The proposed instrument also allows for gathering end-user input to enhance effective usability.After applying the developed scale to end users, factors and items with lower scores that significantly impact satisfaction and brand loyalty are identified.Therefore, this scale can help website designers and developers determine which factors to prioritize, ultimately enhancing customer satisfaction.
The ISO-recommended BS EN ISO-9241-151 usability guide, serving as the primary reference point in the study, does not provide a specific classification for website usability issues.Therefore, based on the fundamental principles in existing usability guides, it is meant to quickly identify usability problems, making it adaptable to many websites.It was observed that the identified factors are common across many websites, such as easy navigation, user-friendly page design, fast loading times, and efficient search options, which are essential for most websites.However, considering that not every website has a similar development structure, the scale developed in the study is adaptable to the characteristics and needs of different websites.As highlighted in the study [30], in some cases, certain sub-scales or items may be more important for specific sites while less critical for others.The factors remain general in such cases, but the weights can vary.In its broad outline, the developed website usability guide addresses website issues and can be further developed when assessing highly specialized sites.In such evaluations, the proposed instrument can serve as a starting point, allowing additional parameters to be added while preserving the main framework.It is inherently capable of continuous improvement through ongoing feedback and updates based on this feedback.Consequently, the scale maintains a dynamic nature and can be adapted to the specific needs of different types of websites.

Limitations and Future Studies
In addition to the contributions highlighted in this study, there are still some limitations.First, the developed survey instrument was validated by applying it to e-commerce users in Turkey.In future studies, the developed survey instrument could be applied to different types of websites in different cultures for further validation.Secondly, our nomological validation indicates that the proposed usability factors significantly explain user intentions, brand loyalty, and satisfaction.Specifically, the explained variances for continued intention to use, satisfaction, and brand loyalty were 61.7%, 61.3%, and 52.7%, respectively.While these results are promising, other factors might influence satisfaction, loyalty, and intention to use.Therefore, future research should be conducted to explore additional factors that impact these outcomes.While the scale we developed can be used to evaluate the usability of various website types, it is essential to recognize that different website types may have unique requirements.As a result, the scale can be modified by introducing new items or factors specific to various websites' needs.Future researchers are encouraged to use the developed scale as a foundation and customize it for particular contexts.Furthermore, by combining the findings of the developed usability guide with web analytics methods, the content of the study can be enriched, leading to more concrete website usability criteria.

Conclusions
In this study, using the BS EN ISO-9241-151 and an extensive literature review, we conceptualized website usability and subsequently developed and validated a survey instrument.We identified 16 key constructs, including "brand identity", "navigation", "page design", and "text design", among others.These constructs were found to have significant implications on user outcomes such as brand loyalty, satisfaction, and continued intention to use.Understanding and employing these constructs can lead developers to enhance the user experience on their websites.Our research emphasizes the significance of website usability in the digital age and provides resources and insights for both researchers and developers.Especially, researchers can leverage our conceptualization to further study website usability.Meanwhile, developers can utilize our instrument to enhance the design, development, and evaluation of their websites.
Institutional Review Board Statement: The data were gathered through a two-stage process from users of e-commerce websites.The survey involved participants who had previous shopping experience on well-known Turkish e-commerce platforms.The initial sample consisted of 785 responses from e-commerce website users, and for the second part of the research, a total of 1086 individuals willingly participated.To initiate this study, ethical approval was acquired from the Istanbul Technical University Social and Human Sciences Human Ethics Committee (project number: 240, date: 14 April 2022).Prior to engaging in the questionnaire, participants were adequately informed about the rigorous confidentiality measures in place.These measures guaranteed that data collected during the study would be treated as confidential and would not be disclosed to external parties.The participants were explicitly informed that their involvement in the questionnaire was entirely voluntary, and there was no coercion or obligation to participate.Furthermore, it was emphasized that participants had right to withdraw from the study at any point without facing any consequences or losing any entitlements.
Informed Consent Statement: Informed consent was obtained online from all subjects involved in the study.

Informative homepage
Home pages should provide sufficient information about the general framework of website so that user can review and anticipate the purpose of the site.

Memorable website URL
The URL used to access the website should be crafted as a memorable tagline, thus making it easy for users to remember.

Overall identity for several websites used in an organization
If an organization disperses several websites that address the same audience, the overall design should be consistent.Brand identity elements should be placed in each website consistently.

Logical conceptual model of website
The conceptual model of website should be designed according to the user's mental model.

Relevant content according to website's purpose and target audience
The content should be relevant for the purpose of the website.The particular tasks and information should be comprised of appropriate substance according to the target audience needs.
Appropriate details about content level Content should be given in units for well-organized information flow and appropriate level of granularity.

Up-to-date content availability
In situations where the validity or relevance of content is time-sensitive, it is important to display current information and emphasize its time dependency to the user.The last update and period of validity if applicable should be available.

Personalization and Customization
Personalized access paths Different access paths or structures should be designed for the differentiated user groups.

Evident user individualization
When personalization is used according to user profiles, it should be made evident to the user.

Adjustable user profile settings
If user-specified profiles are used, users should be able to see, modify, and delete that profile on demand.

Automatic customization to user preferences
Users should be informed about automatic adaptation of the website.

User permission and accessibility
Users should have an authorization to the complete content of the website.Users should be informed about how personal information is used.It is important to seek users' approval before using personal information.

Simple and clear privacy policy
When the user inputs personal information or initiates transactions on website, the user should be aware of privacy policy statements that are simple and clear.
Availability of business policy statement If business policy statement is needed for operating, it should be available on the website.

Information on additional installations required
The policy for storing data rights or executable programs that are activated on user's local device during usage should be informed.

Page Design
Descriptive page title Every page should display a descriptive title.

Consistent layout schemes
Pages should be designed using consistent layout schemes such as consistent location for page titles.
Attract new content or temporary content If there is new or significant changing content, the user should pay attention to it.

Appropriate page lengths, including horizontal and vertical scrolling
The length of a page should be designed to support the primary usage objects of the website's pages.Vertical scrolling should be minimized, and horizontal scrolling should be avoided wherever possible.

White space design
The white space of page should be organized optimally without eye-straining and limiting the visual skimming of the page.

Home page design with all features of a homepage
The home page should show features that summarize entire scope as an introduction page to users.

Readable texts
Texts on website pages should be presented with high readability according to expected display characteristics and spatial arrangement.Text can be rescaled-to-fit according to user's preferences.

Support fast skimming
Fast skimming should be enhanced with encapsulating sentences and short and clear terms supported with comprehensible headings, links, bullets, and highlighted keywords.

Text quality
To ensure readability, the textual content should maintain a satisfactory level of spelling and grammar quality.

Identification of links
The links should be differentiated from basic content by underlining, coloring, or highlighting.Elements other than links should not look like links.All links in same website should be designed in consistent manner.

Formatting of the links in the text
When the links are combined with text on website, links should be formatted in short so that the presence of links does not block the text readability.

Descriptive link with labels
The links should be tagged with appropriate descriptive labels.

Clarify the target addresses of links
Links that direct a new opening browser or pop-up window may cause uncertainty.Thus, user should be informed about where to divert.Links guidance to special targets (such as video and large-scale downloadable files) should be clearly identified with marks.Links that have been previously clicked should be differentiated with distinct coloring, etc.

Interaction Objects
Logical and easy data-entry options Select interaction objects that align with the logical properties of user's expected input/output balance and make the user's tasks easier.
Providing keyboard shortcuts By offering keyboard shortcuts, websites can cater to users who prefer or require keyboard-based navigation.

Identifiable and understandable interaction objects
Interaction objects should be coherent to reality.The media objects should be chosen as relevant, engaging, and supporting the overall goals of the website.

Providing multi equivalent options to reach media objects
To ensure continuous accessibility, alternative access methods should be synchronized, such as audio and subtitles for multimedia elements.
Control time-dependent media objects Pause, stop, and go back and forth functions should be provided for multimedia objects when they are time dependent.
Quick view for media elements Media elements should be previewed without long loading times.

Navigation
Clear navigation structure Navigation should be designed to orient users to understand the structure of website and how to navigate within pages.When the navigation structure includes multiple levels, it is important to design the navigation component in a way that allows for the simultaneous display of more than one level of navigation.Instead of a straight level of navigation, if a task requires sequential processes, providing progressive navigation structure and its visibility is more convenient for users.Step-back function between sequence steps should be provided for integrity of navigation structure.

Making navigation links obvious
The main navigation links should be in sight properly whenever the user needs them.
Placing navigation components consistently Navigation components should be shown consistently on each page of website.

Overview navigation structure
To show the grand scheme of the website structure, a site map should be provided for websites to understand website content overall.

Avoiding opening unnecessary windows
Number of extra opening windows such as splash screens should be minimized.If they do not provide useful extra content or feedback about system, they should be removed.If they are necessary, an option to skip must be available.

Home page as a top-level node of navigation
The top-level structure of the navigation should be defined in the home page.

Search
Providing search functions Search functions should be reachable from every part of website with different searching options according to possible experiences on website, such as advanced search and search with keywords.

Search entry
User should be directed for input entry with appropriate instructions.The search techniques should be clear for the user to understand how to enter input and obtain results.

Provide user query and search results accordance
The number of results found with the entered query should be provided to the users.To handle large search result sets, displaying the result technique should be stable.

Appropriate sort and filter options of search results
Sort and filter option of search results is significant for user to find searches.Search results should be aligned with the user needs.There should be alternative sorting and filtering options.

Help
Providing help pages, FAQ section, or special assistance for user Website should provide help pages, FAQ section, or special assistance like chatbot to users whenever they have issues or problems.

Accessible communication with website owner
It is important for user to be able to easily communicate with the website owner or business.

Error Prevention
Minimizing potential user errors User errors during transactions or performing tasks needed to be minimized with some warning directives to users.

Understandable error messages
The content of error messages should be shown on web pages at related positions with clear reason statements.User should be directed to solve basic errors by oneself.

Acceptable loading times
The download times of the webpage should be in acceptable range according to user expectations.

Time-out warnings
User should be informed about scheduled time outs before being forced to log off.If user requests, additional time can be provided before time expires.

Collaboration Online feedback mechanism
A mechanism should be provided to allow users to follow-up progresses by sending comments, questions, or ratings within the system and a two-way interaction between the system and users.

Inputs and outputs of a website compatibility to international audience
The input/output of information or measurement units such as currency, time zone, temperature, phone numbers, address, or postal codes should be designed obviously for inquiring international audience.

Information about geographical context
Arranging context according to information about geographical location provide users with the information whether the website is proper for their intended tasks.

Supporting different languages and customize accordingly
Support website for different languages and, if possible, identify the user's language.Additionally, customize the website according to the characteristics of different languages.

Construct Code Item
Personalization and Customization

USER1
The website emphasizes the content users care about.

USER2
The website adapts its content according to user's previous preferences.

USER3
The website allows users to view and change profile information easily.

USER4
The website offers alternatives that fit user's preferences.
Privacy and Business Policies

PRVC1
The website asks for permission before using user's personal data.

PRVC2
The website clearly states why it needs user's personal information.

PRVC3
The website provides a simple and clear privacy policy.

PRVC4
The website informs user about cookies and executable programs on user's local machine.
Page Design

PAGE2
The website provides a consistent page layout design.

PAGE4
The website allows user to view the page without scrolling horizontally.

PAGE5
The website allows user to scroll fast throughout the pages.

PAGE8
The website uses the white space on the pages in a balanced way.

PAGE9
The website's home page is professionally designed.
Text Design

TEXT1
The website offers easy-to-read texts on the pages.

TEXT2
The website allows user to resize text for reading comfort.

TEXT3
The website allows user to easily skim of the text.

TEXT4
The website uses text that complies with the terms of grammar and spelling.

TEXT5
The website uses an easily understandable terminology without jargon.
Link Design

LINK1
The website provides easy-to-distinguish links.

LINK2
The website uses standard links' format.

LINK3
The website provides link texts describing the link's destination accurately.

LINK4
The website shows the status of visited or active links with some signs.
Interaction Objects

INTR1
The website allows user to easily enter data without dealing with different input methods.

INTR2
The website uses field labels on data-entry forms that clearly indicate the data formats required to enter.

INTR3
The website presents the clickable areas (buttons, menus, etc.) in a way that can be easily distinguished.

INTR4
The website has labels that accurately represent the functions of the controls.
Graphics, Images, and Multimedia

GRPH1
Website uses images for which user can easily understand their messages.

GRPH2
Website avoids excessive use of animation.

GRPH4
The website allows user to easily pause or stop animations or videos.

GRPH5
The website allows user to preview multimedia objects (images, videos, etc.) before viewing them.

NAVG2
The website allows the user to easily understand their current location.

NAVG3
The website allows the user to see the main navigation menu even if they scroll down the pages.

NAVG4
The website provides navigation options consistently across all pages.

NAVG5
The website allows the user to skip the pop-up screens.

NAVG6
The website shows the number of steps left to complete a task.

NAVG7
The website orders navigation options in a logical way.

SRCH1
The website allows user to make advanced or simple search.

SRCH2
The website allows user to access search option in every page.

SRCH3
The website allows user to find search options easily.

SRCH4
The website provides helpful hints to support user's search.

SRCH5
The website assists to improve user's search with helpful hints.

SRCH6
The website orders search results according to their relevance.

HELP1
The website provides help when user has an issue.

HELP2
The website provides frequently asked questions to solve users' problems.

HELP3
The website provides easy to understand help content with step-by-step instructions.

HELP4
The website provides contact mechanisms such as real time chatbot, customer service, message box, or WhatsApp service.

RSPN1
The website allows users to view the content effectively on their own device.

RSPN2
The website adapts interface resolution settings automatically on different devices.

RSPN3
The website provides user to view the content effectively with user's browser.

RSPN5
The website provides access to the website through mobile devices.
Error Prevention

ERPR1
The website allows user to cancel or redo actions easily.

ERPR3
The website's error messages contain clear instructions on what to do next.

ERPR4
The website prompts user before correcting an input.

ERPR5
The website keeps user informed about unavoidable delays.

COLL1
The website provides user to share information with social media accounts.

COLL2
The website allows user to connect with other people.

COLL3
The website provides a convenient way to user for ratings and reviews.

COLL4
The website provides users to read the comments made by other people.

COLL5
The website provides users to give online feedback about their experience with the website.

Construct Code Item
Cultural Diversity and Multilingual Use Design

DVRS1
The website presents information in a familiar format (currency, temperature unit) for user.

DVRS2
The website adapts context according to geography.

DVRS3
The website supports other languages.

DVRS4
The website recognizes user's language preference automatically.

Table 1 .
Conceptualization of Website Usability.

Table 2 .
Matrix Example for Content Validity.

Table 8 .
Estimations of Structural Model.
* describes the items removed from survey instrument; ** describes the modified items with re-wording.