Next Article in Journal
A Novel Hybrid Strategy for Detecting COD in Surface Water
Next Article in Special Issue
Where We Come from and Where We Are Going: A Systematic Review of Human Factors Research in Driving Automation
Previous Article in Journal
A Deep-Learning-Based Bearing Fault Diagnosis Using Defect Signature Wavelet Image Visualization
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

A Heuristic Method for Evaluating Accessibility in Web-Based Serious Games for Users with Low Vision

Department of Software and Computing Systems, University of Alicante, 03690 Alicante, Spain
Intelligent and Interactive Systems Laboratory, Universidad de Las Américas, Quito 170125, Ecuador
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(24), 8803;
Received: 31 October 2020 / Revised: 16 November 2020 / Accepted: 25 November 2020 / Published: 9 December 2020
(This article belongs to the Special Issue Human-Computer Interaction: Theory and Practice)


Nowadays, serious games have become a beneficial resource in the learning process; they are part of our culture and promote social inclusion. Designing accessible serious games is a complete challenge, even more for non-experts. Most existing serious games do not meet accessibility standards because of a lack of methods that include standards and help create more accessible serious games. For this reason, our research presents a heuristic method with three modifications to Giorgio Brajnik’s barrier walkthrough method and based on the Web Content Accessibility Guidelines 2.1 (WCAG 2.1). We defined 28 barriers for the users with low vision and the related impact and persistence variables by defining severity ranges to evaluate accessibility. This method allows measuring the accessibility of web-based serious games; the method proposed in this article can be a good help for non-experts. As a case study, this heuristic method was applied to 40 web-based serious games. The evaluators concluded that serious games should apply WCAG 2.1 to achieve an adequate and inclusive accessibility level. However, this study has limitations; the heuristic method depends on the evaluators’ experience. This work can contribute to studies related to accessibility heuristics in serious games; it can also help construct a software tool that applies WCAG 2.1 and helps experts and non-experts evaluate accessibility in serious games.

1. Introduction

At present, web-based serious games significantly contribute to our culture and can promote social inclusion. Additionally, students’ participation and motivation in the classroom are of great importance in the learning process, so the concept of gamification [1] has taken on great importance. Gamification allows (1) stimulation and support of creativity; (2) player motivation through modifications and new levels of play; (3) use as a teaching aid; (4) inclusion as a trendy means of recreation; and (5) mind training [2].
The International Game Developers Association (IGDA), according to its survey of 1116 developers in 2019, has revealed that 85% of developers considered diversity in the games industry to be essential [3]. On the other hand, the Game Accessibility Special Interest Group (SIG) aims to support the design of universally accessible games [4]. This group has worked since 2003 to aid the game industry to develop more accessible video games.
The accessibility requirements themselves are described in the comprehensive legislation, specifically section 14.21 of the Century Communications and Video Accessibility Act (CAVAA) [5]. Similarly, Microsoft [6,7,8], and Sony [9] are committed to ensuring that products and services are designed for everyone, including the one billion people with disabilities, and therefore apply sets of guidelines to design and evaluate the accessibility of software and hardware developed by companies.
According to the World Health Organization (WHO), more than one billion people have some disability; this corresponds to the fact that there is 15% disability worldwide [10]. Therefore, accessibility in serious games is essential to improving the interaction between users and serious games. The primary motivation for building serious accessible games is to offer easy access to many users, including people with disabilities.
Therefore, accessibility in the context of serious games [11] aims to make applications usable by the maximum number of people, regardless of their knowledge or personal abilities and technical characteristics of the equipment used to access the serious games. Accessibility implies the way users perceive, navigate, understand, and interact with serious games.
This study considered the four principles of accessibility of the Web Content Accessibility Guidelines (WCAG) 2.1 [12]; we used a heuristic method to assess accessibility in serious games. The proposed method is based on the barrier walkthrough (BW) suggested by Brajnik [13,14,15] and WCAG 2.1 [12]. Therefore, a barrier is an impediment that hinders the interaction between the serious game and the user.
The barriers found through the assessment can produce obstacles in interacting with serious games, especially for people with disabilities. In this study, the authors considered the impact and persistence of each barrier faced by users with disabilities. After establishing scales for impact and persistence, the authors determined the severity of each barrier.
This study was developed from previous works [15,16,17]; in the current work, the authors apply three modifications to the BW method. The first modification comprises widening the scale to examine the persistence and impact between zero (0) and three (3). The second modification is applied to persistence, which depends on the number of barriers presented in the assessment. The third applies the scales defined to evaluate effectiveness, productivity, satisfaction, security, and impact.
The heuristic method proposed has eight phases: (1) choose the web-based serious games; (2) describe the category of user; (3) determine user activities; (4) investigate the serious games; (5) make a list of the barriers; (6) evaluate with the heuristic method; (7) record the data; and (8) interpret and analyze the data.
As a case study, this method was applied to a total of 40 web-based serious games selected with simple random probability sampling from various websites for educational and learning purposes. During the evaluation of the serious games with the heuristic method, two evaluators participated, with accessibility experience in serious games from 2015. In this case study, the authors applied the 28 guidelines for users with low vision. This heuristic method can be replicated for other categories of disabilities by considering the appropriate barriers with the type of disability. However, this heuristic method is manual and requires experience and training in web accessibility by the evaluators. This investigation can provide a beginning step for future studies associated with accessibility heuristics in web-based serious games.
The rest of the document is structured as follows: Section 2 defines the background and work related to accessibility in serious games; Section 3 defines the heuristic method; Section 4 presents the case study; Section 5 explains the results; Section 6 presents the discussion of the results. Finally, Section 7 describes the perspectives and future work.

2. Background

In this section, we explain serious games, considered as a resource to support learning [18]. Some authors maintain that serious games improve the comprehension capacity of students when they learn [19]. Accessibility in serious games implies that users can perceive, understand, navigate, and interact [12] without the application’s problems. The evaluation methods of the accessibility in serious games, several previous studies about the subject, are presented and introduced to our study. The heuristic method addresses several studies and concepts applied in this research.

2.1. Serious Games

Serious games [20] or training games have a different primary purpose than entertainment; they must also link the real and virtual world. They promote [21] the construction of knowledge and skills through exposure to different situations, cases, or problems. They allow simulations of a real situation to interact in the environment in the same way they would in a real environment. They are aimed at different audiences and can be of any kind, uses different technologies, and are designed for different platforms, computers, consoles, virtual worlds, and social networks. According to its final objective, [22], three categories are identified:
Edutainment, or educational games so that the game takes part in the education, the game provides an ideal environment for training by transmitting information and testing the knowledge.
Advergaming, a game to promote a product, one of the most popular applications of serious games is advertising communication.
Simulation facilitates learning represents a simplified model of reality, either with group games of physical or intellectual skills that reproduce the study’s process.
The video gaming industry is vast; in 2015, there were almost two billion video players worldwide; it is estimated that this number will increase to more than three billion gamers by 2023 [23]. According to statistics [24], the educational serious games market’s income will grow from 3.5 billion USD in 2018 to 24 billion USD in 2024.
According to Google Trends [25] trend on “serious games” and “educational games” in the last five years worldwide, taken from 15 November 2015 to 15 November 2020, in Figure 1a, we observe that “educational games” tend to grow from 2019. Figure 1b shows that the peak corresponds to 22 March 2020, coinciding with the coronavirus pandemic’s confinement in many countries. Because teachers, parents, and students were looking for “educational games” to reinforce students’ learning processes and compensate for the fact that they could not use face-to-face classes.

2.2. Accessibility in Serious Games

As we have mentioned, accessibility [12] is the inclusive practice of ensuring that serious games and technologies are designed and developed so that people with disabilities can use them. In the context of serious games, accessibility includes aids such as high-contrast or large font size, screen magnifiers, screen readers and reviewers, voice recognition software, keyboards, and adapted devices according to user needs.
The study [26] argues that application designers face a significant challenge in applying accessibility guidelines because they are complex and ambiguous. For this reason, the authors propose following the WCAG with a user-centered approach. They offer designers an innovative and engaging solution for broader adoption of accessibility guidelines that can create accessible games.
The study [27] indicates that games offer a traditional channel where all people can be involved; however, this only happens when creating inclusive design environments to achieve this challenge. Therefore, they suggest developing an accessibility vocabulary and language for games. The authors recommend including the terminology used, showing the controls that enable the play’s tasks and skills.
The investigation [28] indicates that play accessibility is about removing barriers for users with disabilities. Besides which, the authors argue that the challenge is that web-based games can better access the WCAG 2.1 and achieve an adequate accessibility level in the construct of more accessible games.
Previous studies by the study [29] indicate that it is essential to evaluate the users’ learning styles, for which they applied the Honey-Alonso test [30] for the development of serious games. The Honey-Alonso test includes active, reflective, theoretical, and practical techniques. The authors proposed constructing an educational video game with a Honey-Alonso test script and applied the WCAG.
The research [31] argues that making serious games universal and accessible are a challenge for interface creators and game designers. The authors indicate that more effort is needed and propose research with recommendations for accessibility in serious games. Consequently, the authors recommend ten strategies for designing games for visually impaired users. The results indicated that only three out of ten games were accessible.
Concerning accessibility, in June 2018, the World Wide Web Consortium (W3C) [32] established the WCAG 2.1 recommendations for making web content more accessible; furthermore, this proposal can be applied to achieve accessibility in serious games. WCAG 2.1 covers a wide range of recommendations to make web content more accessible. Applying [12] these guidelines will make serious game content more accessible to people with disabilities, including adaptations for blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity, and combinations, and some adaptations for learning disabilities and cognitive limitations. WCAG 2.1 comprises four principles, 13 guidelines, 78 compliance criteria, combined with an undefined number of sufficient techniques and advisory techniques. The four principles are described below:
Perceivable, users must distinguish content in audio, video, and text.
Operable; users must be able to use and interact with the interface modules.
Understandable, users must be able to recognize self-controls of the interface.
Robust, users must be able to access comfortably and understand consistently in any case of the present and potential technologies.
Therefore, the compliance criteria include three levels. Level A offers a minimum of accessibility; not achieving implies that users cannot access the content. Level AA allows an intermediate level of accessibility; it satisfies the levels A and AA; it indicates that the users cannot access all the content. Furthermore, Level AAA is the highest level of accessibility. Achieving this level means that users can easily access the content.

2.3. Accessibility Evaluation Methods

Serious games and gamification approaches in the educational context [33] allows for greater student participation in skill development. Due to the growing multidisciplinary interest in learning, serious games are considered a useful pedagogical tool. Therefore, the need arises to build serious games accessible and inclusive to access these learning resources. This section describes several of the accessibility evaluation works, which contribute with tools that guide the design of web-based serious games.
The research [34] evaluates accessibility related to visual impairments in educational video games to suggest improvements in this type of application’s design. A second study by the authors indicates that the evaluation of mobile serious games’ accessibility for people with hearing disabilities argue that accessibility has not been considered in the development of serious games. As a case study, they evaluated a mobile game’s accessibility, where they proposed an analysis tool. In a third study, the investigation [35] argues that accessibility is a feature that is not considered in the design of serious games, so they are not easily accessible. The study shares a compilation of accessibility guidelines for developing video games; they propose categorizing the existing guidelines to evaluate a video game’s accessibility for people with motor disabilities.
A previous study by the authors [36] proposes a method to motivate software developers to improve accessibility. The authors present a technique for evaluating the accessibility of serious games applying WCAG 2.1, which incorporates barriers for users with cognitive disabilities. The proposed approach can be replicated for people with other disabilities and can be used throughout the game to create a cycle to guarantee more inclusive and accessible applications. A second study [37] proposes to apply a combined method to evaluate accessibility in serious games, taking into account the WCAG 2.1. As a case study, the accessibility of 82 serious games developed by Physical Education Technology Interactive Simulations at the University of Colorado was evaluated. The authors suggest replicating the combined approach for users with various disabilities, considering the accessibility barriers. As future work, they suggest generating a heuristic accessibility evaluation focused on serious games to develop applications more inclusive.

2.4. Heuristic Method

In the literature review for this research, we did not find any automatic review methods or heuristics applied to serious games, so we started from the existing information on evaluating the heuristics of accessibility for websites; we have the following studies. The study [38] argued that accessibility heuristics are created as a complement to accessibility guidelines. Additionally, heuristics allow evaluators to identify which sections of an application have accessibility issues effectively. The results revealed that using accessibility heuristics can locate a broader range of barriers to solving accessibility problems. However, heuristics do not provide evaluators with compliance information, so heuristics studies are not a substitute for user experience.
The research [39] argued that accessibility evaluation methods help user-centered accessibility. The author notes that accessibility evaluation methods can find accessibility issues, for instance, guideline infringements, modes of failure, defects, or user implementation rates. An evaluation method can prescribe how to rank and rate topics in conditions of severity and priority. It presents a discussion of several evaluation methods, one of which is the BW method, which has the following advantages: low complication, validates knowledge, more excellent correction than conformance review, and produces severity indices.
A study by the authors [17] argues that determining whether interfaces are accessible is essential, and it requires accessibility evaluation. One form of assessment uses automated tools, which are not sufficient; human testing with experts and users with disabilities is already necessary. The authors present a study to evaluate accessibility and barriers present on the application for Banco do Brazil employing the BW method, which records and defines a sequence of barriers that are susceptible to each classification of the user. This survey’s objective was to examine whether this approach can be treated as a supplement to automatic assessment when observations involving users cannot be made—the method allowed for rapid and timely evaluation, decreasing the assessment’s complication. With the suggested study, they detected usability problems associated with accessibility, for which they presented some suggestions.
The study [40] argued that accessibility assessment methodologies assume that all guidelines must be met to achieve universal accessibility. The authors noted that the BW approach addresses issues of applying the guidelines to diverse groups of users. The process allows evaluators to assess the severity of the barrier being inspected. They determined that barriers could be combined with the BW method to evaluate the impact of an application on users and effectively relate to the subject.
A previous study by the authors [16] indicated that there are qualitative and quantitative methods for checking whether an application is accessible. The authors presented a modification to the BW method offered by Giorgio Brajnik based on WCAG 2.1. The change involved incorporating persistence to work out the severity of an accessibility barrier. With the process used in the experiment, the evaluators found that the method allows for (1) measuring the accessibility of applications; (2) testing a heuristic process; (3) assisting in manual evaluations, and (4) contributing to studies associated with accessibility heuristics.
Therefore, the heuristic method based on the BW suggested in the studies [13,14,15] can assist in supplementing accessibility assessment in serious games. The heuristic method is a systematic method built on testing and error inspections in which an evaluator thinks of some potential accessibility barriers corresponding to the WCAG 2.1 accessibility principles. Barriers incorporate characteristics according to user type, purpose, the perspective of use, and serious gaming site to draw appropriate conclusions related to effectiveness, productivity, satisfaction, and security [13,14], with severity scores designated to each accessibility barrier.
In this study, the barriers are defined in the conditions of the next parameters: (1) Effectiveness indicates the degree of compliance to accomplish the user’s aim accurately; (2) Productivity is associated with the time, effort, and cognitive load prerequisite to achieving a specific level of effectiveness; (3) Satisfaction characterizes the contentment and acceptance by the user; it implies the capacity to acclimate, and (4) Security represents the established weakness in the serious game assessed. The BW method [13,15] is an accessibility inspection technique created from accessibility designed principles, taking into account the user group, the type of assistive technology, the influence of users, the types of serious games, and the effects caused.
The study [41] considers that inspection-based evaluation techniques are standard, as they involve less formal instruction. It applies a method that combines task-oriented heuristic assessments that are easy to understand and use. Sears recommends methods for checking the authenticity and consistency of the assessment techniques. The results of heuristic assessments allow for more significant problems to be identified than in cognitive reviews.
The severity of a barrier, according to the BW method [13,15], is related to the physical and cognitive characteristics of the user and the context in which the activities are performed; in this way, it is possible to reach conclusions with effectiveness, productivity, satisfaction, and security, to achieve the applicable parameters of severity. This method recommends utilizing two factors to assess the severity of a barrier: the impact of the barrier on satisfaction and productivity user and the persistence with which the barrier is present [15].
The BW method states that the evaluator can measure the severity of the barrier on a scale between one (1) and three (3), where the value of one (1) corresponds to a “minor problem,” implying that the user can overcome and avoid the barrier [13,42]. An identified barrier may affect productivity or user satisfaction but not effectiveness and security. A barrier that seriously affects a task’s execution is assigned the value of two (2) and corresponds to a “significant problem.” Finally, when it is difficult to avoid the barrier, security and effectiveness are reduced. The value of three (3) is assigned, implying a “critical problem,” in which context users often fail to achieve the objectives. Therefore, the barrier causes a negative effect that could affect (1) productivity and effectiveness; and (2) security and user satisfaction.
For this research, we start with the authors’ previous studies of authors [16] on websites’ heuristic methods. The authors apply the heuristic method based on the barrier path using ranges and severity scales to evaluate websites. In this research for the heuristic evaluation of accessibility in serious games, we apply the concept of heuristics suggested by the authors [43], which involves a method of inspection that supports an interactive evaluation of the system; three modifications to the BW heuristic method [13] are proposed by Giorgio Brajnik.

3. Method

The authors tested an initial part of this method in previous studies, the authors [16] related to evaluating accessibility heuristics in websites. Therefore, in this study, we applied a method that considers three variations used for the first time in the heuristic evaluation of accessibility in serious games. This method consists of prioritizing the impacts of the barriers corresponding to the applied perspective. With this heuristic method, we can identify a more significant number of accessibility barriers that prevent interaction between serious games and users.
Once these barriers are identified, they can be corrected by the developers of serious games. Using the detailed scale in Table 1, we applied three modifications to the BW method in this investigation; this contains the scale and the meaning of the severity with the first modification of the BW [14,15,16]; it consisted of expanding the scale to evaluate the efficiency, productivity, satisfaction, security, and impact.
The second modification affects persistence, which depends on the number of barriers presented in the evaluation; we apply the persistence scale detailed in Table 2. The third modification was to apply the severity according to the value of impact and persistence, shown in Table 3.
The heuristic method to evaluate web-based serious games is condensed into eight phases, corresponding to Figure 2; each phase of the method is detailed in the case study.

4. Case Study

This method was applied to 40 web-based serious games, selected with a simple random probability sampling, from several websites for educational and learning purposes. The websites that contain the web-based serious games are shown in Table 4.
In this research, we applied accessibility barriers for low vision users; the evaluators specified heuristics associated with impact and severity involving serious games. The evaluators previously raised the accessibility barriers based on the WCAG 2.1, applied according to the disability and user impact, characteristics, and effects of the serious games evaluated.
Phase 1: Select the serious games; in this phase, we used the formula for calculating the sample size when the population size is unknown. Before calculating the sample size, we need to determine: (1) the size of the population defined as a collection of objects or individuals that have similar characteristics; (2) the margin of error or confidence interval, which is a statistic that expresses the amount of random sampling error in the results; (3) the confidence level that refers to the random intervals that are used to constrain a value with a given high probability. For our case, we applied Equation (1).
n = Z a 2     p     q d 2
where: n = size of the desired sample, Z = confidence level = 0.631, p = probability of success = 50%, q = (1 − p) probability of failure = 50%, d = accuracy, accepted estimation error = 5%.
When applying the calculations, we obtained a value of 39.8; therefore, a total of 40 serious games listed in Table 5 should be evaluated, containing the ID that is the identifier assigned to the web-based serious game, the URL, and the subject of the serious game.
Phase 2: Characterize the type of users; in this experiment, the guide for low vision users [44] was applied, who are characterized as those who have a situation in which the user’s vision cannot be rectified with glasses, affecting everyday events, reading, and driving. Low vision is most frequent among the elderly but can occur in people of any age because of degenerative diseases.
The WHO defines low vision [45] as a visual abnormality that restricts the ability to perform everyday visual tasks. The leading causes of visual impairment are uncorrected refractive errors, cataracts, age-related macular degeneration, glaucoma, diabetic retinopathy, corneal opacity, and trachoma. The fundamental principle of accessibility for users with low vision is “perceivable” [12].
Phase 3: Define user activities; the evaluators define scenarios to analyze the serious game and how users interact with the game to achieve the objective. The task was to enter each serious game, review each game’s functionality, check the menus, images, messages, and help options to see any accessibility barriers for low vision users. Additionally, in this phase, we describe the level of accessibility (A, AA, AAA) that would be assessed; in this situation, we operate up to the level AA. It was also necessary to detect accessibility support with a listing of web browsers that must be friendly. In this investigation, the evaluators used Google Chrome version 84.0.4147.125, Mozilla Firefox version 77.0.1, and Opera version 70.0.3728.106.
In this case study, a barrier for a user with low vision means that he or she cannot interact effectively with the serious game, implying that it is difficult for him or her to move precisely over the content of the serious game due to the complexity of his or her visual perception. The two evaluators identified the scenarios made up of users with low vision, the aims, and the activities that users must complete in the testing to apply the approach suggested in this investigation. A list of accessibility barriers for low vision users based on WCAG 2.1 [12] is listed in Table 6. The evaluators recommended this method to accomplish the aim, considering effectiveness, productivity, security, and user satisfaction. It was essential to recognize the scale of severity and range of persistence of each barrier that characterizes the low vision user’s impediment to achieve the aim.
Phase 4: Explore the serious games; the expert explores each serious game with the instructions defined in phase 3. The tasks that the user had to perform were: (1) interact with the first interface of the serious game; (2) review the functionality of each round; (3) identify whether the games have a screen magnifier; (4) review whether the games have the option to adjust the game parameters; (5) determine whether the game has the opportunity to help the user; (6) identify whether the user can read each message of the serious game; (7) review whether the images have adequate clarity and are apparent to the user; and (8) check whether the serious game has the option to change the language.
At this stage, the evaluators navigate all the serious game features to detect potential accessibility barriers. Finally, in this phase, the evaluators review the behaviors agreeing to the device used, the browser, and the structure applied.
Phase 5: List the barriers; we selected 28 barriers based on WCAG 2.1 for low vision users to apply this BW-based heuristic method. Table 6 contains the guideline identifier, the barrier, the WCAG 2.1 principle, the success criteria, and the level of accessibility.
Phase 6: Evaluate with the heuristic method, the evaluators carried out the following process: (1) run the serious game; (2) carefully examine whether each barrier meets the parameters of effectiveness, productivity, security, and satisfaction; we applied Table 1. (3) Find the impact related to the parameters defined in Table 1; (4) record the persistence of each barrier according to the parameters defined in Table 2; (5) apply the persistence scale defined in Table 2; (6) finally, apply the severity scale according to the impact and resulting persistence defined in Table 3.
To analyze the severity of a barrier, we need two parameters: (1) the impact of the barrier as a result of the effectiveness, productivity, security, and satisfaction of the user performing an activity; (2) the value of persistence, denoted by the number of times the barrier is repeated, is essential when evaluating a serious game.
For example, when checking the barrier “Accessible keyboard” in the game SG01 “Acid-base solutions”, we apply the following process: First, for the efficiency parameters, we register the value zero (0), in productivity the value of one (1), in satisfaction zero (0) and security the value of one (1). Second, we analyze whether the barrier affects productivity, satisfaction, effectiveness, or safety. In that case, the impact was minimal; when analyzing these parameters, the impact’s value is two (2). Thirdly, we evaluate the number of times the barrier corresponding to the persistence parameter presented by the barrier “Accessible keyboard” is presented. Fourth, we apply the persistence scale defined in Table 2. Fifth, we determine the severity defined in Table 3 for impact and persistence; for the barrier “Accessible keyboard” we have in the impact the value of two (2) and for the persistence the value of two (2), therefore the severity for the barrier “Accessible keyboard” is significant.
Phase 7: Record the data; the data from each serious game evaluated was documented in a spreadsheet. Table 7 is an example since it collects the data of only one set; it contains the identifier assigned to the pattern, the barrier, the effectiveness, the productivity, the satisfaction, the safety, the impact, the range of persistence, the total persistence, the result of persistence and the severity of the serious set with the code SG01 that corresponds to “acid-based solutions” of the PhET project of the University of Colorado. For the G01 “Accessible keyboard” barrier, we applied Table 1 and obtained the following results: in effectiveness we recorded zero (0), in productivity one (1), in satisfaction zero (0), and in safety one (1). Then we analyzed the impact; if the barrier affects productivity or satisfaction marginally, but not effectiveness or safety, the impact is minimal; when analyzing these parameters, we found impact equal to two (2) according to the scale of Table 1 this corresponds to significant. To evaluate persistence, we apply Table 2; we find that the barrier is present for more than four times; therefore, persistence is equal to two (2). Finally, we applied Table 3 to find severity, with the impact value equal to two (2) and persistence equal to (2); therefore, this barrier presents significant severity.
The recorded data are available in the repository [46] so that the assessment can be reproduced. Data recording is essential; the data allow evaluators to manage and condense information by diverse categories.
Phase 8: Interpret and analyze the data; the evaluators offer the analysis of the severities taken when applying the heuristic method; First, we organize and group the information by categories. To manage each serious game’s severity information, we use the dendrogram shown in Figure 3; it contains the distances between each serious web-based game evaluated consecutively. In the results section and the discussion section, a more in-depth analysis of the results related to the severity of the barriers, the accessibility principles, the levels, and the success criteria of the WCAG 2.1 is carried out.

5. Results

Table 8 summarizes the 40 serious games evaluated; it contains the serious games’ code, the severity of null, minor, significant, and critical.
The results obtained in this case study show that the total null severity has a value of 473, representing 42.2% of the total. The minor severity corresponds to the amount of 70, representing 6.3%. The significant severity is 55, which corresponds to 4.9%; finally, the critical severity has a value of 522 and represents 46.6%. A summary of WCAG 2.1 principles and the severity of the serious games’ barriers are presented in Table 9.

6. Discussion

Of the 40 serious games evaluated, we found that:
The SG01 “acid-base solutions” set has a value of six in the critical severity evaluation.
The SG09 “severity-and-orbits” set has nine and corresponds to the group’s most accessible serious games with less critical barriers.
The two web-based serious games correspond to the University of Colorado’s PhET project.
The serious games with the highest number of critical barriers correspond to SG28 “freeriderhd” with a value of 18, followed by SG35 “physicsatschool” with 16. The results indicate: (1) The most significant number of serious games is in the critical and significant severity rating; (2) For serious games to be accessible at an acceptable level, application programmers must correct the most significant number of accessibility barriers through serious game programming software; and (3) the alternative is to use adaptive hardware by applying assistive technology or digital ramps that allow users to interact with serious games easily.
Furthermore, to find similarities between the serious games evaluated with the heuristic method, we apply the hierarchical grouping that allows the finding of similarities between the serious games accommodated in groups so that the groups are well separated and that the serious games within them are as close as possible. The similarity between the serious games is graphed using a tree called a dendrogram. In this process, we apply the following steps: (1) the number of clusters or groups is formed according to the number of serious games; (2) the serious games are grouped based on a measure of similarity between them; (3) the groups are grouped based on their distance, forming a tree called a dendrogram; (4) they continue to group until only one grouping remains.
Figure 3 shows the dendrogram of the 40 serious games’ evaluations with the heuristic method considering the null, minor, significant, and critical variables to know the groups that obtained similar characteristics among the serious games. To measure the similarity between the serious games, we applied the Euclidean distance (2).
d ( x , y ) = x y   i = 1 n ( x i y i ) 2
The Euclidean distance method allows us to see how similar the evaluation data are for each serious game. The dendrogram in Figure 3 shows the height of the branches that indicates the similarity between the clusters, and contains five horizontal lines in green indicating the number of groups into which they are divided; in this case, it was divided into six groups containing similar values, which are detailed below:
The serious game SG01 is in a single group because it records low values in null, minor, significant, and critical; this implies that SG01 is the most accessible serious game of the 40 evaluated.
The SG28 serious game is in the second group. It is still not as accessible because the critical severity is 18, indicating that it contains some accessibility barriers that must be corrected even though null, minor, and significant are low.
The two serious games, SG21 and SG29, are in the third group are not very accessible because they have a critical severity of 15.
The five serious games, SG10, SG22, SG09, SG14, and SG17, are in the fourth group and are not very accessible because they have a critical severity.
The 12 serious games SG05, SG23, SG24, SG25, SG26, SG34, SG35, SG36, SG37, SG38, SG39, SG40 have a high critical severity and are therefore not accessible.
The 19 serious games SG02, SG03, SG04, SG06, SG07, SG08, SG11, SG12, SG13, SG15, SG16, SG18, SG19, SG20, SG27, SG30, SG31, SG32, SG33 are among the least accessible.
Figure 4a shows that the examined critical games violate some principles of WCAG 2.1.
In the 40 evaluated serious games, the evaluators found: (1) 342 barriers with critical severity, where 65.5% characterize the perceptible principle. (2) 37 barriers with critical severity represent 7.1% of the operable. (3) 19.7% corresponding to the understandable, and (4) 7.7% corresponding to the robust. These statistics indicate that accessibility barriers affect web-based serious games and should be corrected.
Figure 4b shows a summary of the accessibility levels corresponding to critical severity; we observe that for level A, 210 barriers representing 40.2%, for level AA, 227 barriers corresponding to 43.5%, and level AAA has 85 barriers corresponding to 16.3% of the total.
Figure 5 presents the success criteria for critical severity; these criteria can be found in Table 7; we observe that:
The on-focus barriers correspond to criterion 3.2.1. The help barrier of the success criterion 3.3.5 is the most critical with a value of 40.
It is followed by the content hovering over the focus barrier of criterion 1.4.13, subtitled criterion 1.2.4, and the automatic transcripts barrier of criterion 1.
With the value of 38, the barriers text alternative is sign language.
Through 37, the criterion status messages.
By 36, interface rearrangement.
With 32, we have adjusted the display settings and pause, stop, hide.
With 30, the criterion accessible keyboard.
Together with 23, the criterion useful audio techniques.
The rest of the criteria have values lower than 12.
We suggest incorporating solutions using digital ramps or software and hardware assistance technologies to have serious games accessible. We propose to include support tools that help users with sensory, motor, and visual disabilities with configuration options to enable an environment controlled by the user of the serious game. We propose that the serious game contains (1) A tool to control the serious game’s video, the subtitles, and playback. (2) A tool to support motor, sensory, and cognitive disabilities.
The tool in Figure 6 suggested should contain (1) an option to control the serious game’s execution that allows the configuration of subtitles, descriptions, keyboard, and automatic transcription; (2) the subtitle settings should include options to manage the position, font, font size, text color, background, and opacity; (3) an option to configure the serious game’s speed, subtitles, and audio replay in the application. The user must have the control to manage and customize the speed of video, subtitles, and audio according to the user’s disability.
Figure 7 suggests including a tool to support users with disabilities that allows voice commands, and the option for them to use the VoiceOver application that narrates what happens on the screen of any device; voice assistants are a great help for users with visual or motor disabilities. Another option is to include a chatbot based on artificial intelligence, which feeds the database as the user learns.
A system can be included to transcribe the video into text or sign language and transform the text into sign language for deaf users.
People with motor disabilities usually read the information presented by computer output devices through the monitor or printer without difficulty, but often have difficulty handling input devices such as the keyboard and mouse. For people with motor disabilities, filters can be incorporated to facilitate the use of the keyboard and mouse by a trackball, joystick, buttons, and custom devices.
As additional add-ons, serious games can include changing the language to customize and choose the audio description and subtitles’ language. Although sign language is not similar globally, this option should be included to help deaf and dumb users and feed a database with each region’s sign language. Therefore, assistive technologies have become an essential resource to overcome the barriers of access to digital technologies that positively impact people’s quality of life with disabilities.

7. Conclusions

This heuristic method can be used to evaluate the accessibility of any serious web-based game. One of these heuristic methods’ contributions is to evaluate each barrier by the parameters of effectiveness, productivity, satisfaction, and security related to impact and persistence. We recommend reproducing this heuristic accessibility evaluation method for any serious game considering the accessibility barriers that correspond to the type of motor, cognitive, or sensory disability. The evaluators identified some critical severity barriers, including the following two. The first barrier with the highest critical severity is the focus, and the second barrier is the help, both of which are related to the understandable principle.
These barriers occur when no contextual help is available, and any user interface component receives the focus but does not initiate a context change. Consequently, the evaluators recommend: (1) Applying techniques and tools to reduce the barrier help’s critical severity can use clear labels that can act as contextual help. (2) To reduce the on-focus barrier’s severity, the user interface components can be implemented as a programmable element to perceive the parts as separate controls. To assess the accessibility of serious games, evaluators recommend fusing automatic assessment tools with heuristic methods. One must remember that no instrument can substitute manual assessment by a serious game accessibility expert. The proposed method can be tested with serious game developers throughout the software development cycle to identify new barriers and correct them as they are built to achieve more inclusive applications.
Future research may propose new methods of evaluating multimedia resources concerning videos and sound recordings; in both cases, a transcript of the dialogues, a description of the sounds, and control of the playback speed should be provided. However, inappropriate use of multimedia elements may create a barrier to user access. To achieve more accessible serious games, we suggest testing with users with sensory, motor, and cognitive disabilities to identify the barriers users face with disabilities.
Future work should continue to refine heuristic methods related to serious games and socialize best accessibility practices, designing software that helps experts evaluate serious games’ accessibility by considering this heuristic method and the WCAG 2.1. Furthermore, we propose to survey serious game developers to find out if they know the WCAG 2.1 and if they have tried to apply some of its guidelines to design inclusive products that make an essential contribution in the field of accessible serious games.

Author Contributions

Conceptualization, L.S.-U. and P.A.-V.; methodology, L.S.-U.; investigation, L.S.-U. and P.A.-V.; writing—original draft preparation, L.S.-U., and P.A.-V.; writing—review and editing L.S.-U., S.L.-M., M.G. and P.A.-V.; supervision, S.L.-M.; project administration, P.A.-V. and L.S.-U. All authors have read and agreed to the published version of the manuscript.


This research was funded by Universidad de Las Américas-Ecuador, as part of an internal research project FGE.PAV.19.11.

Conflicts of Interest

The authors declare no conflict of interest.


  1. García-Iruela, M.; Fonseca, M.J.; Hijón-Neira, R.; Chambel, T. Gamification and Computer Science Students’ Activity. IEEE Access 2020, 8, 96829–96836. [Google Scholar] [CrossRef]
  2. Bonnechère, B.; Bier, J.; Van Hove, O.; Sheldon, S.; Samadoulougou, S.; Kirakoya-Samadoulougou, F.; Klass, M. Age-Associated Capacity to Progress When Playing Cognitive Mobile Games: Ecological Retrospective Observational Study. J. Med. Internet Res. Serious Games 2020, 8, e17121. [Google Scholar] [CrossRef] [PubMed]
  3. International Game Developers Association (2019) Developer satisfaction survey 2019: Summary report. Available online: (accessed on 21 October 2020).
  4. Bierre, K.; Chetwynd, J.; Ellis, B.; Hinn, D.M.; Ludi, S.; Westin, T. Game not over: Accessibility issues in video games. In Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Las Vegas, NV, USA, 22–27 July 2005; pp. 22–27. [Google Scholar]
  5. Federal Communications Commission 21st Century Communications and Video Accessibility Act (CVAA). Available online: (accessed on 21 October 2020).
  6. Microsoft Xbox Accessibility Guidelines. 2019. Available online: (accessed on 21 October 2020).
  7. Micosoft Making Video Games Accessible: Business Justifications and Design Considerations. 2018. Available online: (accessed on 21 October 2020).
  8. Microsoft Accessible Gaming with the Xbox Adaptive Controller. 2018. Available online: (accessed on 21 October 2020).
  9. SONY Accessibility and Usability. 2020. Available online: (accessed on 21 October 2020).
  10. World Health Organization (WHO) 10 facts on disability. 2017. Available online: (accessed on 21 October 2020).
  11. Salvador-Ullauri, L.; Acosta-Vargas, P.; Luján-Mora, S. Web-Based Serious Games and Accessibility: A Systematic Literature Review. Appl. Sci. 2020, 10, 7859. [Google Scholar] [CrossRef]
  12. World Wide Web Consortium (W3C) Web Content Accessibility Guidelines (WCAG) 2.1. 2018. Available online: (accessed on 21 October 2020).
  13. Brajnik, G. Web Accessibility Testing: When the Method Is the Culprit. In Proceedings of the International Conference on Computers for Handicapped Persons, Linz, Austria, 11–13 July 2006; Springer: Berlin/Heidelberg, Germany, 2006; pp. 156–163. [Google Scholar]
  14. Brajnik, G. Measuring Web accessibility by estimating severity of barriers. Int. Conf. Web Inf. Syst. Eng. 2008, 5176, 112–121. [Google Scholar]
  15. Brajnik, G. Barrier Walkthrough. 2011. Available online: (accessed on 21 October 2020).
  16. Acosta-Vargas, P.; Salvador-Ullauri, L.; Luján-Mora, S. A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision. IEEE Access 2019, 7, 125634–125648. [Google Scholar] [CrossRef]
  17. Braga, H.; Pereira, L.S.; Ferreira, S.B.L.; Da Silveira, D.S. Applying the barrier walkthrough method: Going beyond the automatic evaluation of accessibility. Procedia Comput. Sci. 2014, 27, 471–480. [Google Scholar] [CrossRef][Green Version]
  18. Martin, F.; Betrus, A.K. Instructional Simulations and Games. In Digital Media for Learning; Springer: Cham, Switzerland, 2019; pp. 85–110. [Google Scholar]
  19. Cheng, M.; Chen, J.; Chu, S.; Chen, S. The use of serious games in science education: A review of selected empirical research from 2002 to 2013. J. Comput. Educ. 2015, 2, 353–375. [Google Scholar] [CrossRef][Green Version]
  20. Abt, C. Serious Games; The Viking Press Inc.: New York, NY, USA, 1970. [Google Scholar]
  21. Michael, D.R.; Chen, S.L. Serious Games: Games that Educate, Train, and Inform; Muska & Lipman/Premier-Trade: Cincinnati, OH, USA, 2005; Available online: (accessed on 28 October 2020).
  22. Alvarez, J.; Rampnoux, O.; Jessel, J.-P.; Methel, G. Serious Game: Just a question of posture. Artif. Ambient Intell. AISB 2007, 7, 420–423. [Google Scholar]
  23. Statista Number of Active Video Gamers Worldwide from 2015 to 2023. In Surv; Time Period 2014 to 2020. Release Date June 2020. Available online: (accessed on 15 November 2020).
  24. Statista Game-Based Learning Market Revenue Worldwide in 2018 and 2024. In Surv; Time Period 2018. Release Date August 2019. Available online: (accessed on 15 November 2020).
  25. Google Google Trends. Available online: 5-y&q=serious games,educational games (accessed on 15 November 2020).
  26. Spyridonis, F.; Daylamani-Zad, D. A serious game to improve engagement with web accessibility guidelines. Behav. Inf. Technol. 2020, 1–19. [Google Scholar] [CrossRef]
  27. Cairns, P.; Power, C.; Barlet, M.; Haynes, G. Future design of accessibility in games: A design vocabulary. Int. J. Hum. Comput. Stud. 2019, 131, 64–71. [Google Scholar] [CrossRef]
  28. Westin, T.; Ku, J.J.; Dupire, J.; Hamilton, I. Game accessibility guidelines and wcag 2.0–a gap analysis. In Proceedings of the International Conference on Computers Helping People with Special Needs; Miesenberger, K.K.G., Ed.; Springer: Cham, Switzerland, 2018; pp. 270–279. [Google Scholar]
  29. Salvador-Ullauri, L.; Acosta-Vargas, P.; Jadán-Guerrero, J.; Guevara, C.; Sanchez-Gordon, S.; Calle-Jimenez, T.; Lara-Alvarez, P. Development of an Accessible Video Game to Improve the Understanding of the Test of Honey-Alonso. In Proceedings of the International Conference on Applied Human Factors and Ergonomics, Washington, DC, USA, 24–28 July 2019; Springer: Cham, Switzerland, 2020; pp. 289–298. [Google Scholar]
  30. Alonso, C.; Gallego, D.; Honey, P. Cuestionarios/Estilos de Aprendizaje. Available online: (accessed on 21 October 2020).
  31. Araújo, M.C.C.; Façanha, A.R.; Darin, T.G.R.; Sánchez, J.; Andrade, R.M.C.; Viana, W. Mobile audio games accessibility evaluation for users who are blind. In Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Toronto, ON, Canada, 17–22 July 2016; Antona, M.S.C., Ed.; Springer: Cham, Switzerland, 2017; pp. 242–259. [Google Scholar]
  32. World Wide Web Consortium (W3C) W3C Issues Improved Accessibility Guidance for Websites and Applications. Available online: (accessed on 22 October 2020).
  33. Almeida, F.; Simoes, J. The role of serious games, gamification and Industry 4.0 tools in the Education 4.0 paradigm. Contemp. Educ. Technol. 2019, 10, 120–136. [Google Scholar] [CrossRef]
  34. Jaramillo-Alcázar, A.; Luján-Mora, S. An approach to mobile serious games accessibility assessment for people with hearing impairments. In Proceedings of the International Conference on Information Technology & Systems, Libertad City, Ecuador, 10–12 January 2018; Springer: Cham, Switzerland, 2018; pp. 552–562. [Google Scholar]
  35. Jaramillo-Alcázar, A.; Salvador-Ullauri, L.; Luján-Mora, S. A Mobile Serious Games Assessment Tool for People with Motor Impairments. In Proceedings of the International Conference on Education Technology and Computers, Barcelona, Spain, 20–22 December 2017; pp. 172–177. [Google Scholar]
  36. Salvador-Ullauri, L.; Acosta-Vargas, P.; Luján-Mora, S. Accessibility Evaluation of Video Games for Users with Cognitive Disabilities. In Proceedings of the International Conference on Intelligent Human Systems Integration, Modena, Italy, 19–21 February 2020; Springer: Cham, Switzerland, 2020; pp. 853–859. [Google Scholar]
  37. Salvador-Ullauri, L.; Acosta-Vargas, P.; Gonzalez, M.; Luján-Mora, S. Combined Method for Evaluating Accessibility in Serious Games. Appl. Sci. 2020, 10, 6324. [Google Scholar] [CrossRef]
  38. Paddison, C.; Englefield, P. Applying heuristics to accessibility inspections. Interact. Comput. 2004, 16, 507–521. [Google Scholar] [CrossRef]
  39. Brajnik, G. Beyond conformance: The role of accessibility evaluation methods. In Proceedings of the International Conference on Web Information Systems Engineering, Dubai, UAE, 12–15 November 2018; Volume 5176, pp. 63–80. [Google Scholar]
  40. Lunn, D.; Yesilada, Y.; Harper, S. Barriers Faced by Older Users On Static Web Pages Criteria Used In The Barrier Walkthrough Method. Available online: (accessed on 10 November 2020).
  41. Sears, A. Heuristic Walkthroughs: Finding the Problems Without the Noise. Int. J. Hum. Comput. Interact. 1997, 9, 213–234. [Google Scholar] [CrossRef]
  42. Brajnik, G.; Lomuscio, R. SAMBA: A Semi-Automatic Method for Measuring Barriers of Accessibility. In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility, Tempe, AZ, USA, 15–17 October 2007; pp. 43–50. [Google Scholar]
  43. Nielsen, J.; Molich, R. Heuristic evaluation of user interfaces. In Proceedings of the Conference on Human Factors in Computing Systems, Seattle, WA, USA, 1–5 April 1990; pp. 249–256. [Google Scholar]
  44. Moreno, L.; Valencia, X.; Pérez, J.E.; Arrue, M. Exploring the Web navigation strategies of people with low vision. In Proceedings of the International Conference on Human Computer Interaction, ACM, Las Vegas, NV, USA, 15–20 July 2018; p. 13. [Google Scholar]
  45. World Health Organization (WHO) Blindness and Vision Impairment. Available online: (accessed on 22 October 2020).
  46. Acosta-Vargas, P.; Salvador-Ullauri, L. (Dataset) A Heuristic Method for Serious Games. Available online: (accessed on 28 October 2020).
Figure 1. (a) Google Trends about serious games and educational games in the last five years. (b) Google Trends about serious games and educational games in the 12 months.
Figure 1. (a) Google Trends about serious games and educational games in the last five years. (b) Google Trends about serious games and educational games in the 12 months.
Applsci 10 08803 g001
Figure 2. A heuristic method for accessibility evaluation method.
Figure 2. A heuristic method for accessibility evaluation method.
Applsci 10 08803 g002
Figure 3. Dendrogram with six clusters of the evaluated serious games with the heuristic method.
Figure 3. Dendrogram with six clusters of the evaluated serious games with the heuristic method.
Applsci 10 08803 g003
Figure 4. (a) WCAG 2.1 vs. critical severity. (b) WCAG 2.1 accessibility levels vs. critical severity.
Figure 4. (a) WCAG 2.1 vs. critical severity. (b) WCAG 2.1 accessibility levels vs. critical severity.
Applsci 10 08803 g004
Figure 5. WCAG 2.1 success criteria vs. critical severity.
Figure 5. WCAG 2.1 success criteria vs. critical severity.
Applsci 10 08803 g005
Figure 6. Accessibility suggestion for serious game control.
Figure 6. Accessibility suggestion for serious game control.
Applsci 10 08803 g006
Figure 7. A tool to configure the options according to the disability of the users.
Figure 7. A tool to configure the options according to the disability of the users.
Applsci 10 08803 g007
Table 1. Enlarged Scale.
Table 1. Enlarged Scale.
Table 2. Barrier–Persistence.
Table 2. Barrier–Persistence.
Barriers NumberPersistence
0 and 10
2 and 31
4 and 52
Greater than 53
Table 3. Impact, Persistence, and Severity value.
Table 3. Impact, Persistence, and Severity value.
Table 4. Websites where serious games have been obtained.
Table 4. Websites where serious games have been obtained.
URLDescription site created by Physical Education Technology (PhET) Interactive Simulations at the University of Colorado [18], which contains several free serious games for educational use in different subjects simulations and animations of physics, electromagnetism, mechanics, gravitational field, molecular physics and thermodynamics, and particular relativity theory. random selection of various websites that contain serious learning games, including some of the most popular game lists.
Table 5. List of selected serious games.
Table 5. List of selected serious games.
SG11 science
SG14 science
Table 6. Barrier vs. Web Content Accessibility Guidelines 2.1 (WCAG 2.1.)
Table 6. Barrier vs. Web Content Accessibility Guidelines 2.1 (WCAG 2.1.)
IDBarrierWCAG 2.1Success CriteriaLevel
G01Accessible keyboardOperable2.1.1 A
G02No luminance flash failuresOperable2.3.1 A
G03Animation from InteractionsOperable2.3.3 AAA
G04Content hovering over focusPerceivable1.4.13AA
G05Easy to read fontPerceivable1.4.4AA
G06Text alternativesPerceivable1.1.1A
G08Automatic transcriptsPerceivable1.2.5AA
G09Sign languagePerceivable1.2.6 AAA
G10Information and relationshipsPerceivable1.3.1A
G11Adjust display settingsPerceivable1.3.4AA
G12Interface rearrangementPerceivable1.3.5 AA
G13Use of colorPerceivable1.4.1A
G14Contrast without textPerceivable1.4.11 AA
G15Well-spaced elementsPerceivable1.4.12A
G16Good audio techniquesPerceivable1.4.2A
G17Contrast Perceivable1.4.3AA
G18Images as sharp as possiblePerceivable1.4.5AA
G19Visual presentationPerceivable1.4.8 AAA
G20 Pause, stop, hidePerceivable2.2.2A
G21Sensory CharacteristicsPerceivable1.3.3A
G22Screen reader supportRobust4.1.2A
G23Status messagesRobust4.1.3 AA
G24LanguageUnderstandable3.1.1 A
G25Consistent navigationUnderstandable3.2.3 AA
G26Labels or instructionsUnderstandable3.3.2 A
G28On FocusUnderstandable3.2.1A
Table 7. Evaluation with the BW heuristic method for the serious game SG01 “acid-base-solutions”.
Table 7. Evaluation with the BW heuristic method for the serious game SG01 “acid-base-solutions”.
G01Accessible keyboard01012 4 42Significant
G02No luminance flash failures10121 2 21Minor
G03Animation from Interactions10121 3 31Minor
G04Content hovering over focus00003 663Critical
G05Easy to read font10121 2 21Minor
G06Text alternatives01012 5 52Significant
G07Subtitled01012 4 42Significant
G08Automatic transcripts01012 4 42Significant
G09Sign language01012 5 52Significant
G10Information and relationships10121 3 31Minor
G11Adjust display settings111300 00Null
G12Interface rearrangement111301 10Null
G13Use of color10121 2 21Minor
G14Contrast without text10121 2 21Minor
G15Well-spaced elements111300 00Null
G16Good audio techniques01012 5 52Significant
G17Contrast01121 2 21Minor
G18Images as sharp as possible111300 00Null
G19Visual presentation111300 00Null
G20 Pause, stop, hide01121 663Significant
G21Sensory characteristics10121 663Significant
G22Screen reader support10121 663Significant
G23Status messages00112 663Critical
G24Language00112 663Critical
G25Consistent navigation111300 00Null
G26Labels or instructions00003 663Critical
G27Help00003 663Critical
G28On focus00003 663Critical
Where: E = Effectiveness, P = Productivity, St = Satisfaction, Sc = Security, I = Impact, P1 = 0 and 1, P2 = 2 and 3, P3 = 4 and 5, P4 > 5, Pt = Persistence total, P = Persistence.
Table 8. Summary of the severities of the serious games evaluated.
Table 8. Summary of the severities of the serious games evaluated.
Table 9. Summary of WCAG 2.1 principles and severity of barriers.
Table 9. Summary of WCAG 2.1 principles and severity of barriers.
PrincipleNull% NullMinor% MinorSignificant% SignificantCritical% Critical
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Salvador-Ullauri, L.; Acosta-Vargas, P.; Gonzalez, M.; Luján-Mora, S. A Heuristic Method for Evaluating Accessibility in Web-Based Serious Games for Users with Low Vision. Appl. Sci. 2020, 10, 8803.

AMA Style

Salvador-Ullauri L, Acosta-Vargas P, Gonzalez M, Luján-Mora S. A Heuristic Method for Evaluating Accessibility in Web-Based Serious Games for Users with Low Vision. Applied Sciences. 2020; 10(24):8803.

Chicago/Turabian Style

Salvador-Ullauri, Luis, Patricia Acosta-Vargas, Mario Gonzalez, and Sergio Luján-Mora. 2020. "A Heuristic Method for Evaluating Accessibility in Web-Based Serious Games for Users with Low Vision" Applied Sciences 10, no. 24: 8803.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop