Development of an Approach to Evaluate Website Effectiveness

Abstract

The internet has been used by individuals, organizations, and governments for business, sports, health, banking, advertisement, education, and other services. Many websites have been developed and designed in the last several decades. However, most have not been developed and designed according to a shared set of design standards. Consequently, there is a need for an approach to evaluate the effectiveness of a website. A literature review was conducted to develop such an approach. Four experts were then consulted to inspect and evaluate the approach, and a questionnaire was completed by three categories: Internet users, website developers, and others to determine its final version. This research resulted in the development of an approach to evaluate website effectiveness, composed of three major criteria: design, content, and functionality, and 17 sub-criteria. The significance of this new approach is that it allows stakeholders to evaluate their websites and determine how to improve them in order to achieve their vision and mission.

1. Introduction

The usage of internet has increased sharply, and a website has become a popular communication tool for enterprises and organizations [1]. This enables commercial trade and allows individuals to access information that is provided on the internet [2]. A website can be designed based on the organization’s requirement, which can provide details about it [3]. Hence, a well-designed website and invaluable contents can consider as a competitive advantage for the organizations [4]. On the other hand, a poorly designed website can cause damage to the reputation of organizations [1]. A website is effective when both owners and users achieve their goals for the site. Effectiveness can be defined as “the power to produce the desired result”, and it is “often measured as the quality of the desired result” [5]. For users, some kind of sequence is usually involved. Most literature on website evaluation focuses on usability and accessibility [6].

Usability is one of the issues that have a stronger influence on the success or failure of software systems. Several studies have indicated that users face problems with software account for 80% of total maintenance expenses, rather than technical flaws. Moreover, usability issues account for 64% of these issues, which lead to users being unable to complete their intended actions. Websites are more sensitive to usability issues than other software since they are more information-oriented and entail more involvement. Consequently, website usability is crucial. Poor usability has been proven to harm the credibility of a website, causing the user to lose trust in the website [7].

Static websites have become outmoded as the internet has evolved into a more active and collaborative community. Users contribute by expressing their thoughts and experiences, as well as by adding, sharing, and publishing materials that have been gathered from other websites. As a result, a dynamic website that provides information exchange and communication with individuals who utilize these services is essential. This can assist organizations to attract new individual and understand their behaviors [4]. A considerable amount of research has been conducted to develop criteria for evaluating a website, as the number of websites published on the Internet has increased [8].

A study stated that electronic bank systems can be analyzed based on five criteria; which are quality, reliability and availability, visualization, functionality, and usability [9]. The quality of the website can be an essential factor that affects the satisfaction of individuals. Several criteria can influence the quality of a website, for example, security, interface design, loading time, content, and other factors. Two methods can be used to evaluate a website: by gathering the opinion of users or by running automated tools [10].

These criteria differ from study to study in their quality, impact, and measurement. Therefore, the contribution of this paper is to develop an approach that can effectively evaluate a website and impact website development in order to design a website that leads organizations to achieve their vision and mission.

The Technology Acceptance Model (TAM) introduced in 1986 is one of the most widely used models in the field of information systems (IS) [11]. Considering that acceptance is relative to effectiveness, TAM assumes that perceived usefulness (PU) and perceived ease of use (PEOU) are two factors impacting the acceptance of an IS. Perceived enjoyment is another factor introduced by [12]. This study extended TAM to explain individual acceptance and usage of websites. In addition to enjoyment, the study shows that perceived visual attractiveness, defined as “the degree to which a person believes that the website is aesthetically pleasing to the eye”, influences usefulness, enjoyment, and ease of use. Therefore, usefulness, enjoyment, ease of use, and visual attraction are factors that should be considered when evaluating a website.

As a result, the main objectives of this study are as follows:

• to investigate previous studies that evaluate websites in different domains.
• to identify the factors that are most frequently used to evaluate websites.
• to develop a new approach for the evaluation of website effectiveness.
• to validate the new approach for the evaluation of website effectiveness.

The rest of this paper is organized as follows: Section 2 reviews the literature, Section 3 describes the research methodology, and Section 4 analyzes and discusses the results. Finally, the conclusion is presented in Section 5.

2. Literature Review

Much research has been conducted to determine appropriate criteria for website evaluation. This section investigates the criteria that have been used to evaluate a website in different domains.

Using a variety of resources and empirical study, experience and knowledge were incorporated to propose a framework for a successful website. The framework can be used to compare website quality, define website enhancement, and assist developers and designers by providing a guideline. The framework was designed based on a review of the most recent approaches to evaluation criteria used in different e-business services. Four dimensions were used in this framework [13] as shown in Figure 1.

A study was conducted in 2012 to identify the factors affecting e-commerce website usability [14]. These factors can be categorized as six attributes:

• Accessibility refers to being useable by all people, including elderly and disable users.
• Interactivity refers to user ability to share, add, and edit information, as in the use of blogs.
• Personalization refers to user ability to adjust page settings according to individual preferences.
• Content refers to the quality of content provided.
• Navigation refers to straightforward utilization and provision of clear directions telling users what they will discover when they click.
• Design-structure refers to quality design and structure that is easily understandable.

The study indicated that successful website design should generate high usability. Usability can be defined as the measurement of how easy the interface is to use. The study included users, evaluators, and software-based methods [14].

A framework was developed to evaluate the issue of website usability, based on the results of user testing, heuristic evaluation, and Google Analytics software. The framework was then used to evaluate three e-commerce websites in Jordan. It consists of the following factors: accessibility, navigation, internal search, architecture, content/design, customer service, security and privacy, purchasing process, functionality, and consistency. It is stated that it may be used either individually or in combination [15]. In 2014, the author asserted that the most important criteria in evaluating educational websites are content and navigation, with design being the second most important element. The study showed that ease of use and communication, however, were of least importance among these categories [16].

Ref. [17] proposed key performance areas (KPAs) for an evaluation system known as the attraction website maturity model. This model used a questionnaire method to collect data. The total number of completed questionnaires was 120 responses from different groups. This model was used to evaluate 357 websites in mainland China. The model consists of six KPAs which are:

• Accessibility refers to the measurement of a user’s ease of access to a website.
• Usability refers to the measurement of how users feel about a website.
• Functionality refers to the measurement of effective information provided to users.
• Interaction refers to the measurement of communication with a website.
• Commerce refers to the measurement of commercial services delivered.
• Marketing refers to the measurement of marketing attractions.

Another study aimed to develop an inclusive tool to evaluate the overall appeal of e-tailing websites. It used a theoretical framework of 13 elements as a guideline to conduct a literature review. The study identified 108 general elements to evaluate website appeal, which were combined into five categories and 26 elements. These categories are look and feel, navigation, credentials, content, and customization. Each category can be evaluated using several elements [18] as presented in

Table 1. General elements for assessing website appeal, Source: [17].

Look and Feel	Credentials	Customization
Aesthetic Elements Amount of Content Spelling or Grammar Readability Compatibility Download Time	Management Statement Mission/Vision/History Contact Details Privacy Policy Terms and Conditions Copyrights	Accessibility for Disabled Customization Options Adaptive Website
Navigation	Content
Logical Organization of Pages Site Map/Index Search Engine Broken Links Intuitive Navigation Internal Links	Necessary Information Accurate Information Regularly Updated Information Quality of Graphics Valuable Animations

.

Ref. [19] investigated the attention, interest, desire, and action model (AIDA) in order to evaluate the quality of travel websites from Chinese customers’ perspectives. The AIDA model is a marketing model to improve product and service sales which is composed of eight attributes: visual appearance, information quality, ease of use, navigability, accessibility, personalization, interactivity, and flexible reservation. The study used the decision tree method as an evaluation tool. Decision tree learning is “a common approach in data mining, which is a ‘divide and conquer’ method that can portray the data in a pattern from a set of independent instances”. The data was collected by choosing 25 MBA students from a local Chinese university to contribute to the study’s evaluation of four Chinese travel agencies.

An exploratory case study evaluated Egypt’s government portal effectiveness. Different main pages were selected by the author to be tested using an embedded single-case design. The study used automatic evaluation tools that examine the source code of webpages in order to define the compatibility of webpages with identified rules. The authors used a model consisting of three factors, which are accessibility, usability, and performance, in order to define problems related to these factors [20].

Another study employed quantitative analysis to evaluate a bank website using criteria from the modified evaluation framework based on the 2QCV3Q-model. These criteria include identity, content, services, location, maintenance, usability, and feasibility [21]. This study combines counting and automated methods, and seven sections including 43 questions were developed using a 7-point Likert scale. The total number of participants was 610, and 77 responses were disregarded.

A cross-sectional analysis approach was used to evaluate hospital websites in Kuwait, with potential hospital websites selected based on the Ranking Web of World Hospitals and Google Search. Four dimensions were adopted to evaluate the websites in order to enhance consumer engagement and access to health information. These dimensions were accessibility, usability, presence, and content. The evaluation was conducted by two experts. Fifteen websites were involved in this study, and six websites were excluded [22].

This study used three instruments to characterize the population’s internet use, evaluate content related to Alzheimer’s disease, and assess the technical quality of the website. The Health-Related Web Site Evaluation Form (Emory) questionnaire was used, including accuracy, authors, updates, public, structure, content, navigation, and links. The total number of participants was 56 individuals, comprised of 16 elderly people, 12 caregivers, and 28 SLP therapists [23].

Another study evaluated five large museum websites in Europe, using the inspection evaluation method (IEM). This study employed three dimensions, which were usability, functionality, and mobile interaction, in order to evaluate the websites, the aim of which was to attract visitors and ensure they meet their goals [24].

A literature review was conducted, and a web evaluation guide called GoodWeb was developed using 69 studies. Three methods were used, which were a questionnaire, interviews, and web usage analysis, and both users and experts contributed to the evaluation. Many attributes were retrieved from the literature review, and these attributes were categorized into eight categories and 37 subcategories as demonstrated in

Table 2. Website attributes, Source: [25].

Attribute Category	Attribute Sub-Category
Usability	Intuitiveness/Learnability/Memorability/Information Architecture/Effectiveness/Efficiency
Content	Completeness/Accuracy/Relevancy/Timeliness/Understandability of the Information
Web design	Use of Media/Search Engines/Help Resources/Originality of the Website/Site Map/User Interface/Multilanguage/Maintainability
Functionality	Links/Website Speed/Security/Compatibility
Loyalty	First Impression
Appearance	Layout/Font/Colors/Page Length
Interactivity	Sense of Community/Personalization/Help Options/Background Music
Satisfaction	Usefulness/Entertainment/Look and Feel/Pleasure

[25].

Ref. [26] conducted a literature review to develop a tool to evaluate the official destination website of Maharashtra state in India from the customer perspective. The study compiled a variety of features which were then grouped into four factors and 24 attributes as illustrated in

Table 3. List of factors and their attributes, Source: [26].

Factors	Attributes
Quality of information	Accurate information/up-to-date information/detailed information on the product/detailed information on the ancillary services/sufficient audiovisual contents/online booking facilities/privacy/security.
Identity- and trust-building components	DMO name/brand logo or taglines/easy URL
Ease of use	Easy accessibility to the website on internet/easy accessibility to differently abled persons/easy and fast navigation/easily readable and understandable text/faultless performance/memorable and attractive website design.
Customization and interactivity	Multilingual capacity/customization/interactive maps and location identifier/interactive features/interactive communication services/links to mobile apps and social media.

. The authors then designed a questionnaire in order to evaluate 3 world heritage websites. The study involved choosing 100 participants who had visited each of the 3 websites from a total sample size of 300 participants. Analysis of the questionnaire revealed the necessity of adding the factor of online booking to the existing 3 factors. This factor was not suitable for our study; therefore, it has been disregarded.

An additional framework was designed to evaluate e-commerce websites in two main areas. These were an evaluation of the e-commerce website’s functionality, which included 10 criteria, and an evaluation of the e-commerce website as a marketing tool, which also included 10 criteria, presented in

Table 4. Areas and their criteria for e-commerce website evaluation, Source: [27].

Areas	Sub-Criteria
The functionality of the e-commerce website	Structure and design/Possibilities for sorting and filtering goods and services/Opportunities to search and compare goods and services/Registration system/Shopping cart and order process/Integration with payment systems and bank credit cards/Integration with delivery systems/Means of personalization/Data transfer security/Multilingualism of the website and compatibility with different currencies.
The e-commerce website as a marketing tool	Graphic design of the website/Site content/Organization and navigation of the site/Responsive design/Clients’ instruments–chat systems, comments, etc./Program code and meta data/Inbound and outbound connections/Site loading speed/Domain and hosting/Sharing on social media and on other websites.

. A questionnaire was designed for experts to evaluate the e-commerce website. The study proposed a 6-step scale to be used in the evaluation tool. A failing score was 0–49 points, satisfactory 50–69 points, good 70–79 points, very good 80–89 points, and excellent 90–100 points [27].

Another study proposed usability metrics that consisted of six main categories: navigation, organization, ease of use, design, communication, and content. A survey was developed to ask future students via email and Facebook for feedback on the metrics. A total of 265 students were surveyed. A total of 86 university websites were chosen from Times Higher Education (World University Ranking 2016–2017) in Europe, Canada, and the United States [28].

Ref. [29] proposed an instrument to evaluate the quality of e-learning websites. A literature review was conducted, and a hierarchical quality model was used to identify and prioritize factors impacting the quality of the websites. A questionnaire was then circulated to determine the most important criteria affecting quality. The study included 157 subjects in order to develop the instrument. Five dimensions were identified as quality dimensions to be used for the instrument, along with 22 sub-dimensions as shown in

Table 5. Extracted quality dimensions, Source: [29].

Content	Design	Organization	Usability
Timely Relevant Multilanguage Variety of Presentation Accuracy Reliability of Content	Appropriateness Color Multimedia Elements Text Browser Compatibility	Index Navigation Consistency Links Logo Domain User	Domain User-friendly Reliability Availability Interactive Features

[29].

3. Research Methodology

Research methodology can be described as a method to solve or answer research problems [30]. Research methods can be categorized into three types: quantitative, qualitative, and mixed methods [31].

Research approach:

This study used a mixed method as follows:

• A qualitative approach was used by conducting a literature review in order to identify existing criteria for evaluating websites. These criteria were then refined and merged to develop the first proposed instrument.
• A quantitative approach was used by carrying out a questionnaire to validate the developed instrument. It was given to applicants through an online method using office.com. It involved providing the applicants with an introduction to the study and details and guidance concerning ethical matters such as the ability to withdraw from the questionnaire. An ethical approval from Shaqra University was obtained. The Arabic and English languages were used. The questionnaire was published on Twitter, LinkedIn, and WhatsApp. A snowball method was used to carry out this questionnaire, which is a method in which applicants are asked to pass the questionnaire on to other potential applicants [32]. Quantitative data was analyzed using descriptive statistics in the form of percentages, using the computer-aided software Microsoft Excel.

Research question:

The study addressed the following research question:

RQ1: What are the criteria and sub-criteria that can be used to evaluate website effectiveness?

Research strategy

Studies were selected using the Saudi Digital Library, which contains many different databases such as Web of Science, Scopus, IEEE Xplore, Sage, AAAS Science, and others. Google Scholar was also used to find related studies. Key terms were determined by the research question. Therefore, the key terms that were used in this study are: “website evaluation”, “criteria+factors+elements+attributes for evaluating a website”, or “website effectiveness”. A total of 17 studies were selected according to the contents studied by the authors.

3.1. Process of Developing the Instrument

First, the literature review showed that there are 243 web elements/categories/factors to evaluate websites as proposed by different researchers. This number would increase if a further literature review was conducted. These elements were reviewed by the authors to avoid repetition, and the duplicated items were shortlisted. The inspection of the elements resulted in a list of 24 factors.

Second, the 24 elements were grouped into three categories and 21 sub-categories, which led to forming the approach and included such statements as follows:

3.1.1. Design

• Aesthetic: The website looks aesthetically pleasing.
• Structure: The information is well-structured.
• Color: The chosen colors are suitable.
• Text: The text is readable.
• Size: The font size is acceptable and readable.
• Multimedia: The chosen multimedia is related to the content.
• Logo: It reflects the purpose of the website.

3.1.2. Content

• Authority: It provides information about the author.
• Objective: The objectives are listed on the website.
• Accuracy: The information is accurate.
• Personalization: It is tailored to the audience.
• Consistency: The website provides consistent flow.
• Relevant: The content is relevant to the purpose.
• Reference: The sources of information are identified.
• Up to Date: The content is up to date.

3.1.3. Functionality

• Navigation: It is clear how to use each page.
• Interactivity: The website allows the user to interact with it.
• Accessibility: It is easy to access for elderly and disabled people.
• Responsiveness: It responds to inquiries.
• Search Engine: It is easy to find information.
• Links: The links work properly.

Third, four experts from Saudi Arabia were consulted to review and inspect the approach by email. After reviewing many experts’ profiles in the domain of Computer Science, four experts were selected based on their qualifications and experiences to examine and inspect the criteria and sub-criteria. Nielsen indicated that the number of experts may be between three and five, and they should be provided a list of tasks or questionnaire [33]. The experts examined the list of criteria and sub-criteria during the period time from 10 July to 18 July 2020 and offered valuable instructions and suggestions as follows:

• Expert-1:

• Change “Text and Font Size” to “Text Readability”.
• Add Compatibility with all browsers.
• Add Security if dealing with private data.
• Remove Currency, which is not needed, and include Links with Accuracy.

• Expert-2:

• Change Aesthetic to appealing.
• Text and Font Size I think can be combined into one criterion, call it “Text” or “Text Readability”.
• Change “terrible element” to “bad”.
• Move Multimedia to content.
• Change the “statement of navigation: to the “website provides easy means of navigation”.
• Change the “statement of Responsiveness” to the “website is responsive and fast”.
• Links and Navigation should be merged into one criterion.

• Expert-3:

• Add real-world feel: System provides information about the organization and/or actual people behind its content and services.
• Add authority: The system refers to people in the role of authority.
• Add trustworthiness: System provides information that is truthful, fair, and unbiased.

• Expert-4:

• I recommend that the website should be mobile-friendly, as responsive as possible, and can be browsed easily from a phone web-browser.
• It is also important to use text with large font for those who suffer from vision or hearing loss.
• Multicultural is a very important criterion.
• The content should be in Arabic.

Fourth, the authors studied these suggestions carefully and improved the approach according to the experts’ reviews. Several sub-criteria were changed to those recommended by the experts, and new sub-criteria were added. The final list of criteria remains as it was, and there are now 23 sub-criteria. The finalized criteria and their sub-criteria that will be used for website evaluation are illustrated in Figure 2.

3.2. Validation of New Approach

The first iteration of the approach development was developed based on a combination of previous approaches and studies. The second iteration of the approach development was developed based on the recommendations of the four experts. Therefore, a questionnaire was designed to produce the third iteration. The questionnaire aimed to validate the sub-criteria used to build the approach. The questionnaire was distributed to many participants via social media announcements such as Twitter, Linkedin, WhatsApp, and emails during the period of time from 3 August to 29 August 2020. A total of 436 responses were received and evaluated for each of the 3 categories (23 sub-criteria). The questionnaire was voluntary, and it was provided in both English and Arabic. The average time to complete it was approximately four minutes. The questionnaire also requested demographic information such as gender, age, type of user, and type of website usage.

The questionnaire was designed using Microsoft forms in order to validate the new approach, which was developed from a literature review and evaluated by experts. It measures website evaluation according to three categories (Design, Content, and Functionality); each category includes sub-criteria. The design category includes six questions, the content category includes nine questions, and the functionality category includes eight questions.

4. Results and Discussion

The purpose of the demographic information was to enable an assessment of the responses’ value while weighting the sub-criteria. Figure 3 shows that out of the 436 respondents, 352 were male (80%), and 84 were female (20%). In addition, respondent ages were categorized as either 18–30 or 31 and above. The results show that 376 of the respondents were over 30 years old (86%). Finally, Figure 3 shows that the majority of participants were average internet users (92%), with fewer being developers, or other individuals.

The summary below shows the value of applicant responses for all 23 sub-criteria.

Table 6. Applicants’ percentage for 23 sub-criteria.

Category	Q#	Sub Criteria	Important %	Neutral %	Unimportant %	Total
Design	QD1	Appealing refers to how the website looks aesthetic.	82.1	14.7	3.2	100
	QD2	Structure refers to how the information is structured well.	90.6	8.3	1.1	100
	QD3	Color refers to how the chosen colors are accepted.	73.9	22.7	3.4	100
	QD4	Text Readability refers to how the text and the font size is acceptable and readable.	89.4	9	1.6	100
	QD5	Logo reflects the meaning of the purpose of the website.	68.3	27.1	4.6	100
	QD6	Mobile-friendly refers to how the website is designed to be mobile-friendly.	86.7	11.2	2.1	100
Content	QC7	Authority refers to how the website provides information about author.	57.3	29.4	13.3	100
	QC8	Objective refers to how the objectives are listed in the website.	72.7	22.9	4.4	100
	QC9	Accuracy refers to how the information is accurate.	89.7	9.4	0.9	100
	QC10	Personalization refers to how the website is specified to the audience.	62.8	28.2	9	100
	QC11	Consistency refers to whether the website is consistent.	77.5	20.2	2.3	100
	QC12	Relevant refers to whether the website content is relevant to the purpose.	84.6	13.3	2.1	100
	QC13	Reference refers to how the sources of information are identified.	79.3	16.3	4.4	100
	QC14	Up-to-Date refers to whether the content is up to date.	84.9	12.4	2.7	100
	QC15	Multimedia refers to how the chosen multimedia is related to the content.	75	21.1	3.9	100
Functionality	QF16	Navigation refers to whether it is clear to use it in each page.	84.2	14	1.8	100
	QF17	Interactivity refers to how the website allows you to interact with it.	73.9	21.8	4.3	100
	QF18	Accessibility refers to how easy it is to access the website for elderly and disable people.	79.6	17.7	2.7	100
	QF19	Responsiveness refers to how the website responds to inquiries.	85.6	11.4	3	100
	QF20	Search Engine refers to how easy it is to find information.	89.9	8.9	1.2	100
	QF21	Compatibility refers to how the website is compatible with all browsers.	85.1	13.3	1.6	100
	QF22	Security refers to how the user’s data is secured.	90.8	7.8	1.4	100
	QF23	Support refers to how the website provides support to users.	79.3	18.6	2.1	100

illustrates the three different categories, which are Design, Content, and Functionality. Each category has sub-criteria, the meaning of which were explained in the questionnaire. The Design category has six sub-criteria which are Appealing, Structure, Color, Text Readability, Logo, and Mobile-friendly. The Content category has nine sub-criteria which are Authority, Objective, Accuracy, Personalization, Consistency, Relevant, Reference, Up to Date, and Multimedia. Finally, the Functionality category has eight sub-criteria which are Navigation, Interactivity, Accessibility, Responsiveness, Search Engine, Compatibility, Security, and Support.

4.1. Design Category

Appealing sub-criterion refers to whether the website looks aesthetic. A total of 82.1% of participants indicated that Appealing is important, while 14.7% indicated that it is a neutral sub-criterion, and only 3.2% deemed it unimportant. Structure sub-criterion refers to whether the information is well-structured. A total of 90.6% of participants indicated that Structure is important, while 8.3% considered it neutral, and only 1.1% deemed it unimportant. Color sub-criterion refers to the suitability of the chosen colors. A total of 73.9% of participants indicated that Color is important, while 22.7% indicated it is neutral, and only 3.4% considered it unimportant. Text Readability sub-criterion refers to whether the text and font size is acceptable and readable. A total of 89.4% of participants indicated that Text Readability is important, while 9% deemed it neutral, and only 1.6% indicated that it is unimportant. Logo sub-criterion reflects the purpose of the website, and 68.3% of participants indicated that Logo is important, while 27.1% considered it neutral, and 4.6% indicated that it is unimportant. Mobile-friendly sub-criterion refers to whether the website is designed to be mobile-friendly. A total of 86.7% of the participants indicated that Mobile-friendly is important, while 11.2% deemed it neutral, and only 2.1% indicated that it is unimportant. Figure 4 shows the participants’ responses for the Design category, which includes six questions identifying the participants’ responses.

4.2. Content Category

Authority sub-criterion refers to whether the website provides information about the author. A total of 57.3% of participants indicated that Authority is important, while 29.4% considered it neutral, and only 13.3% indicated that it is unimportant. Objective sub-criterion refers to whether the objectives are listed on the website. A total of 72.7% of participants deemed Objective important, 22.9% considered it neutral, and 4.4% believed it unimportant. Accuracy sub-criterion refers to how accurate the information is. A total of 89.7% of participants indicated that Accuracy is important, 9.4% considered it neutral, and 0.9% deemed it unimportant. Personalization sub-criterion refers to whether the website is tailored to its audience. A total of 62.8% of participants indicated that Personalization is important, 28.2% considered it neutral, and 9% considered it unimportant. Consistency sub-criterion refers to the website provides consistent flow. A total of 77.5% of participants believed Consistency is important, while 20.2% deemed it neutral, and 2.3% considered it unimportant. Relevant sub-criterion refers to whether the website content is relevant to the purpose. A total of 84.6% of participants indicated that Relevant is important, 13.3% indicated it is neutral, and only 2.1% indicated it is unimportant. Reference sub-criterion refers to whether the sources of information are identified; a total of 79.3% of participants indicated that Reference is important, 16.3% believed it neutral, and 4.4% indicated it is unimportant. Up-to-Date sub-criterion refers to whether the content is up to date. A total of 84.9% of participants indicated that Up-to-Date is important, while 12.4% believed it neutral, and 2.7% marked it as unimportant. Finally, Multimedia sub-criterion refers to whether the chosen multimedia is related to the content. A total of 75% of participants believed Multimedia is important, 21.1% considered it neutral, and only 3.9% indicated that it is unimportant. Figure 5 shows the participants’ responses to the Content category, including nine columns that identify the participants’ responses for each question.

4.3. Functionality Category

Navigation sub-criterion refers to how clear it is to use each page. A total 84.2% of participants indicated that Navigation is important, while 14% indicated it is neutral, and only 1.8% indicated that it is unimportant. Interactivity sub-criterion refers to whether the website allows the user to interact with it. A total of 73.9% of participants indicated that Interactivity is important, 21.8% deemed it neutral, and 4.3% considered it unimportant. Accessibility sub-criterion refers to how easy it is to access the website for elderly and disabled people. A total of 79.6% of participants indicated that Accessibility is important, 17.7% believed it neutral, and 2.7% believed it unimportant. Responsiveness sub-criterion refers to whether the website responds to inquiries. A total of 85.6% of participants indicated that Responsiveness is important, 11.4% considered it neutral, and 3% considered it unimportant. Search Engine sub-criterion refers to how easy it is to find information, A total of 89.9% of participants indicated that Search Engine is important, 8.9% deemed it neutral, and 1.2% considered it unimportant. Compatibility sub-criterion refers to whether the website is compatible with all browsers. A total of 85.1% of participants indicated that Compatibility is important, 13.3% deemed it neutral, and 1.6% considered it unimportant. Security sub-criterion refers to how secure user data is. A total of 90.8% of participants indicated that Security is important, 7.8% believed it neutral, and only 1.4% considered it unimportant. Support sub-criterion refers to whether the website provides support to users. A total of 79.3% of participants indicated that Support is important, while 18.6% considered it neutral, and 2.1% considered it unimportant. Figure 6 shows the participants’ responses to the Content category, including eight columns that identify the participants’ responses for each question.

4.4. The Final Version of the Approach

The development of the approach began with the literature review; after which it was sent to four experts to be inspected. A questionnaire was then developed for the approach to be evaluated by Internet users and website developers. The authors decided to disregard any sub-criteria that were considered important by less than 75% of applicants. The sub-criteria that received less than 75% importance in the responses are Color, Logo, Authority, Objective, Personalization, and Interactivity, as shown in

Table 7. Excluded sub-criteria.

Sub Criteria	Important %	Neutral %	Unimportant %	Total
Color refers to whether the chosen colors are suitable.	73.9	22.7	3.4	100
Logo reflects the purpose of the website.	68.3	27.1	4.6	100
Authority refers to whether the website provides information about the author.	57.3	29.4	13.3	100
Objective refers to whether the objectives are listed on the website.	72.7	22.9	4.4	100
Personalization refers to whether the website is tailored to the audience.	62.8	28.2	9	100
Interactivity refers to whether the website allows the user to interact with it.	73.9	21.8	4.3	100

.

The final version of the approach that can be used to evaluate website effectiveness is composed of three major criteria and 17 sub-criteria, as illustrated in Figure 7.

5. Conclusions

The primary aim of this study was to develop an approach with which individuals, organizations, and governments may evaluate the effectiveness of their websites to achieve their mission and satisfy users. A literature review was conducted in order to combine evaluation criteria from different research papers to build a single approach. A total of 243 items (evaluation criteria) were selected for the first iteration of the new approach and then refined and merged into 24 criteria and sub-criteria. Design, Content, and Functionality became the three main criteria with 21 sub-criteria. In the second iteration, the criteria and sub-criteria were sent to four experts to be evaluated, and the experts offered valuable insights that were carefully studied by the authors. The approach was then redeveloped and consisted of three major criteria, which are Design, Content, and Functionality, and 23 sub-criteria. In the third iteration, a questionnaire was developed in order to validate the redeveloped approach by expertise. To finalize this version of the approach, the authors decided to exclude any sub-criteria that received less than 75% importance in applicant responses. Therefore, the final version of the approach is composed of the three major criteria Design, Content, and Functionality and 17 sub-criteria. The new approach can assist stakeholders in evaluating their websites in order to improve them and enhance their ability to achieve their vision and mission.