Accessibility in Native Mobile Applications for Users with Disabilities: A Scoping Review

Abstract

The objective of this scoping review is to characterize the current scenario of mobile applications considering accessibility issues for people with cognitive, motor, and sensory disabilities. Nowadays, mobile devices have grown exponentially, giving way to new ways of relating, managing, and working. In this context, mobile devices seek to democratize access to knowledge on different topics; however, the application of accessibility guidelines is neglected. The reviewers extracted the most relevant articles published between 2000 and 2020 from the ACM Digital Library, IEEE Xplore, ScienceDirect, Scopus, and Web of Science databases. In this scoping review, the PRISMA-ScR checklist was used to extract scientific articles; Cohen’s kappa coefficient = 0.4117 was applied, which implies moderate concordance of reviewers; 22 primary studies were extracted from a total of 211. The results obtained in this research suggest applying WCAG 2.1 in mobile applications to achieve an adequate level of accessibility. Future work suggests designing review tools that include machine learning based on artificial intelligence algorithms.

1. Introduction

Today, the web presence is no longer enough; digital media consumption is higher on mobile devices; in this digital age, mobility and the Internet allow unlimited possibilities to connect people with various activities. Mobile applications, known as apps, offer organizations valuable content, management, and interaction from any device.

According to Statista [1], as of 2020, the most popular classification in Apple’s App Store was games, with over 22%. The second most popular app classification was business, supported by education and lifestyle applications. Mobile game applications usually produce most of their proceeds during promotion and in-app purchases, with only just over a third of games being funded transfers. IOS devices download iPhone apps from Apple’s App Store, while Android app clients download them from the Google Play Store.

There are three categories [2] of mobile applications: (1) native applications that use the device’s functions and are downloaded from the apps stores; (2) web applications that do not need to be installed and are used from a browser; and (3) hybrids, which are a combination of the two previous ones.

Native mobile applications present several advantages, such as (1) increasing the visibility of a company, product, or service; (2) strengthening brand recall as customers browse the stores; (3) growing the number of application users; (4) improving interaction with users; (5) incorporating a quick response channel; (6) complementing the website; and (7) expanding revenue by implementing the ability to purchase.

Native mobile applications include excellent benefits, but not all of them are accessible; that is, they do not allow easy navigation for users, especially those with disabilities or whose visual faculties have been reduced due to advancements over the years.

Similarly, accessibility in mobile applications [3,4] aims to make applications accessible to as many people as possible, regardless of their knowledge or skills and the technical characteristics of the devices they use. There are millions of applications, but software creators and developers often do not bother to make them accessible. Agreeing with the Web Content Accessibility Guidelines (WCAG) 2.1 [4], a few guidelines manage the potential to authenticate applications by making them more inclusive for all people, including citizens with disabilities.

On the other hand, Google Trends allows analyzing the interest of keyword searches over time. Figure 1 shows searches with historical data for the last five years on Google Trends; [5] suggests using data since 2016, the year Google implemented an improvement in the data collection system. By analyzing these samples, we can measure the interest in the accessibility of mobile applications in all Google searches carried out on the web and discover trends, comparing the popularity of searches for these keywords related to mobile apps, WCAG, web accessibility, and mobile accessibility. In Figure 1, we can see that mobile applications was higher before 2018; however, then WCAG was highest for a few years, and then in the last year, it competed with mobile applications. Therefore, mobile applications have the highest interest in searches performed by web users, followed by WCAG, web accessibility, and finally, mobile accessibility.

Accessibility [6] is considered an essential factor in mobile application development. The Web Content Accessibility Guidelines 2.0 (WCAG 2.0) help application developers ensure that content is accessible to all users, especially users with disabilities [7].

There are several services through the web or mobile applications available to citizens; unfortunately, there is a lack of accessibility, leading to the exclusion of some people and violations of human rights. The study [8] suggests eliminating the gap so that elderly or disabled people can benefit from using e-government services provided by governments through websites and aim to make content accessible to people with disabilities to achieve universal accessibility.

This review contrasts the scientific articles correlated with accessibility in native mobile applications for users with disabilities and presents a scoping review (SR) [6]. We start from the following research question: what accessibility evaluation standards for mobile applications in users with disabilities are the most used? This research describes the search strings that allow finding the most relevant research for accessibility in mobile applications. To understand the specific findings on the articles’ topics, we applied four criteria known as [9] population, intervention, comparison, and outcome (PICO).

Consequently, accessibility is a feature that benefits all people; the absence of accessibility excludes some population groups, such as people with disabilities. This scoping review provides an overview of the current state of accessibility of native mobile applications for users with disabilities. Additionally, this study allows (1) to identify the type of disability that most tools consider in the evaluation of mobile applications, (2) classify the methods and techniques most used in the design of mobile applications, (3) determine the most used accessibility guidelines and categorize the most used types of research in the evaluation of mobile applications, and (4) classify the types of tools and identify the domain areas that use accessible mobile applications.

After a process of extraction of 211 scientific articles, a compilation of 22 primary research papers was carefully chosen, employing Preferred Reporting Elements for Systematic Reviews extension for scoping reviews (PRISMA-ScR) [10]. The findings of this scoping review open a new line of research for future work to build an intelligent automatic review tool that includes WCAG 2.1 based on artificial intelligence algorithms.

This document is structured as follows. Section 2 presents readers to the subject of accessibility and mobile applications. Section 3 explains the research method employed in the scoping review. Section 4 incorporates the results of the analysis. Section 5 includes the discussion of the results and the limitations of the research. Finally, Section 6 shows the assumptions and future research work.

2. Background and Motivation

The conventional narrative given by [11] indicates that a “mobile application” or “mobile app” is an application of software; depending on the technologies engaged, it knows how to be a native application, as well as a web or hybrid application. Mobile applications are designed to be employed on intelligent devices or tablets; they can be transferred from a device company’s distribution platform.

This scoping review has identified several publications related to mobile applications that do not consider accessibility for users with disabilities. Therefore, this study’s need is justified to (1) find information on relevant research on mobile applications and accessibility; (2) categorize whether any accessibility guidelines are applied; (3) detect the different approaches to mobile applications for cognitive, motor, and sensory disabilities; (4) categorize accessibility guidelines based on WCAG used in mobile applications; and (5) identify the tools and methods applied to evaluate accessibility, as well as the domain areas for which accessible mobile applications are built.

Mobile Applications and Accessibility

Regarding accessibility in mobile applications, we found previous studies [2,12,13] where most have been developed for Android. Additionally, in the evaluation of accessibility, the automatic method is applied with Accessibility Scanner. Unlike the studies [14,15,16,17,18], oriented to the application of manual methods, in some cases, the study [12] includes the heuristic method. The results of the studies indicated that the apps present several accessibility barriers that need to be improved.

This scoping review (SR) found previous researches [2,14,15] with performance evaluations for mobile applications targeting visually impaired users. In addition to studies [2,12,13,18] using accessibility guidelines for mobile applications centered on the four accessibility principles of WCAG 2.1 [4], other researches [14,15,16,17] apply guidelines related to text and content design.

The research in [15] indicates that accessibility implies giving functionality to smartphone applications so that all people can access content regardless of disability. On the other hand, the analysis in [16] indicates that mobile applications are growing exponentially. Worldwide, the use of mobile applications has increased significantly. These previous studies [14,15,16,17,18] used a methodology with a fully manual evaluation considering quantitative and qualitative requirements. Most studies evaluated the most popular apps from the Google Play Store.

3. Method Applied for Scoping Review

This scoping review (SR) [19,20] started by defining a review protocol, the investigation question, and the methods. This SR was employed to conduct this study using the checklist of PRISMA extension for scoping reviews (PRISMA-ScR) [10], containing 20 essential information elements and two optional elements to include when completing a scoping review. The PRISMA-ScR method is usually applied in health issues [21]; this method was modified to detect accessibility-related studies in native mobile applications. Appendix A contains a checklist proposed by PRISMA-ScR in which the number of pages in accordance or non-compliance with the 22 aspects described in seven sections is recorded: (1) Title, (2) Abstract, (3) Introduction, (4) Methods, (5) Results, (6) Discussion, and (7) Funding.

The review development includes five phases: (1) the definition of the investigation questions to determine the scope, (2) search strategy to extract all the papers, (3) screening of the papers to obtain the most appropriate ones, (4) coding through summaries consuming the classification structure, and (5) the data extraction and review procedure to find the outcomes.

3.1. Definition of the Research Questions to Determine the Scope

The investigation questions arose because mobile applications have been widely incorporated into different processes and services that have revolutionized mobile technology. Due to the increase in their use during the last five years, the need arises to ensure accessibility for people with disabilities in these applications fully.

This investigation’s first objective is to introduce information related to the most pertinent research on accessibility and mobile applications. This SR contains several papers from digital libraries, specifying the authors, the year of publication, and the SCImago Journal Rank (SJR) impact factor.

The second objective is to identify the distinct methods applied in mobile applications to evaluate accessibility.

The third objective is to determine accessibility guidelines based on WCAG to determine trends and gaps in mobile application development.

Our research examines the results of published primary studies related to accessibility and mobile applications to identify current trends and open issues in this domain; the research questions and the motivation have been defined as:

RQ1. What are the disabilities that are most considered for accessibility evaluation in mobile applications? Investigate the types of disabilities considered in the evaluation of mobile applications developed between 2000 and 2020.

RQ2. What methods and techniques are applied to evaluate accessibility in mobile applications? Classify the methods and techniques used in the design of mobile applications according to disability.

RQ3. What are the accessibility guidelines applied to mobile applications to achieve accessibility? Determine the accessibility guidelines that apply to mobile applications.

RQ4. What types of research and contributions have been found related to mobile applications and accessibility? Distinguish the categories of research and support used in mobile applications and accessibility with disability in mind.

RQ5. What tools are used to test accessibility in mobile applications? Classify the types of tools in mobile apps and accessibility with disability in mind.

RQ6. Which domains are the most evaluated? Determine the domain areas in which accessible mobile applications are used.

3.2. Search Strategy to Extract the Documents

The primary research was obtained through a chain of queries obtained from the research questions. To understand the articles’ findings and the quality of the articles’ topics, we applied four criteria: [9] population, intervention, comparison, and outcome (PICO). Population refers to published studies. The intervention is related to accessibility and mobile applications. The comparison refers to studies carefully selected by disability, based on accessibility standards and type of research. The result includes published studies on mobile application accessibility; based on PICO, we posed five new questions to ensure the quality of the extracted articles, as shown in

Table 1. Document quality evaluation checklist.

N°	Quality Assessment Questions	Answer
QA1	Is mobile application accessibility detailed in the paper?	(+1) Yes/(+0) No
QA2	Is the mobile application accessibility evaluation method specified in the paper?	(+1) Yes/(+0) No
QA3	Does the paper discuss any findings of mobile application accessibility evaluation?	(+1) Yes/(+0) No
QA4	Are common mobile application accessibility errors described in the results?	(+1) Yes/(+0) No
QA5	Is the journal or the conference where the paper was published indexed in SJR?	(+1) if it is ranked Q1, (+0.75) if it is ranked Q2, (+0.50) if it is ranked Q3, (+0.25) if it is ranked Q4, (+0.0) if it is not ranked.

.

The search was carried out as of 6 January 2021; we selected five academic research databases used in engineering to retrieve primary information: ACM Digital Library, IEEE Xplore, ScienceDirect, Scopus, and Web of Science (WOS). The query strings for each one source were characterized from the search words related by Boolean AND/OR operators. The wildcard asterisk (*) was applied to incorporate both singular and plural forms of each word.

Table 2. Query string applied.

Database	String Search	Studies Number
ACM Digital Library	[Publication Title: accessi*] AND [Publication Title: mobi*] AND [Publication Title: applica*] AND [Publication Date: (01/01/2000 TO 01/31/2021)]	35
IEEE Xplore	(“Document Title”: accessibility) AND (“Document Title”: mobile application)	8
Scopus	Title (accessi*) and (title (mobi*) and title (applica*)) and (limit-to (pubyear, 2021) or limit-to (pubyear, 2020) or limit-to (pubyear, 2019) or limit-to (pubyear, 2018) or limit-to (pubyear, 2017) or limit-to (pubyear, 2016) or limit-to (pubyear, 2015) or limit-to (pubyear, 2014) or limit-to (pubyear, 2013) or limit-to (pubyear, 2012)or limit-to (pubyear, 2011) or limit-to (pubyear, 2010) or limit-to (pubyear, 2009) or limit-to (pubyear, 2008) or limit-to (pubyear, 2007) or limit-to (pubyear, 2006) or limit-to (pubyear, 2005) or limit-to (pubyear, 2004) or limit-to (pubyear, 2003) or limit-to (pubyear, 2002 ) or limit-to (pubyear, 2001) or limit-to (pubyear, 2000))	84
ScienceDirect	“accessi*” and “mobi*” and “applica*”	73
Web of Science	TI = (accessi*) AND TI = (mobi*) AND TI = (applica*)	11
	Total number of studies	211

shows the selected databases, the query string, and the number of documents extracted; by including the Scopus and WOS indexed databases, the largest number of citation databases, peer-reviewed bibliography abstracts, scientific journals, books, and conference proceedings are included. The query string was applied to the publication title with the keywords “accessi*”, “mobi*”, and “applica*”. We used a similar search syntax in the five databases to maintain consistency; the search period included studies published between 2000 and 2020.

3.3. Screening of Documents

The application of the protocol that includes the inclusion and exclusion criteria is necessary to categorize the outcomes. The inclusion and exclusion criteria objective to find significant primary documents to answer the investigation questions posed; the agreement between the evaluators was resolved by applying [22] Cohen’s Kappa coefficient = 0.4117 with a percentage of agreement of 83.4%, this value implies a moderate agreement between the evaluators.

Inclusion criteria: the preliminary research should be associated with (1) publications in journals, conferences, books, and book chapters on mobile applications and accessibility published between 2000 and 2020; (2) peer-reviewed primary studies; and (3) written in the English language. Exclusion criteria: the preliminary study is related to (1) summarizes the main speech, an introduction to the workshop, or just a summary; (2) duplicate documents of the same research from distinct sources; and (3) secondary studies related to the literature review.

This review applying PRISMA-ScR [10] is considered as a tool to enhance quality and clarity in systematic reviews avoiding bias. The examination and selection process is laid out in a flow chart. The process phases include a guide for the literature reviewer; this procedure incorporates the databases accessed, indicating the number of papers found for each database. In addition, the number of duplicate papers is included—the number of papers rejected in the process and the reasons for the elimination. Finally, the number of papers for the study is extracted.

Figure 2 illustrates the PRISMA flow chart [23] with the four phases of the article choice method.

Phase 1: identification. The authors incorporated the recordings found from the database searches: ACM Digital Library with 35 documents, IEEE Xplore with eight articles, ScienceDirect with 73 articles, Scopus with 84 articles, and WOS with 11; a total of 211 articles were extracted.

Phase 2: screening. The authors applied the inclusion and exclusion criteria. Of the 211 articles, 32 papers were rejected because they were duplicated in different databases; 179 articles were incorporated. Then, in the next filter, 32 duplicate documents, and 141 articles, including literature reviews and other accessibility problems, were removed. Of the 38 articles examined, 16 related to abstracts and written in a language other than English were eliminated; finally, a total of 22 reviews were approved for the next phase.

Phase 3: eligibility. The authors performed an in-depth full-text review of the 22 articles that explicitly focused on primary studies of accessibility and mobile apps; we did not exclude any full-text articles. We assessed the research articles’ quality responding to accessibility in mobile apps detailed in Table 1. This quality assessment (QA) aims to weigh each of the select articles’ importance when arguing the outcomes and to guide the findings’ analysis.

Phase 4: included. The evaluators applied Cohen’s kappa coefficient, which represents the degree of precision and reliability in the classification of the selected articles; Cohen’s Kappa = 0.4117, which was a moderate agreement between the evaluators. We documented 22 full-text documents in the quantitative synthesis. Figure 2 illustrates that we did not add any supplementary scientific articles.

3.4. Data Extraction and Review Process

The primary articles’ extraction was iterative; it was separated into stages in which different events were conducted. To obtain the data from the ACM Digital Library, we transferred it to BibTeX (BIB) format. Moreover, IEEE Xplore, ScienceDirect, Scopus, and WOS were transferred in Research Information Systems (RIS) format. We then exchanged the information of the five rec-dos in the StartLapes tool version 2.3.4.2, eliminating duplicate research. Then, the steps described in the PRISMA flowchart [23] was used. Finally, we took the 22 selected articles to Microsoft Excel to continue the analysis.

In this phase, the quality of the selected articles was assessed using the five questions detailed in Table 1.

Table 3. Selected scientific articles and quality assessment outcomes.

ID	Scientific Articles	Quality Assessment
		QA1	QA2	QA3	QA4	QA5	Score	Normalization
MS20	Accessibility of mobile applications: Evaluation by users with visual impairment and by automated tools	1	1	1	1	0	4	0.8
ASe20	Toward Accessible Mobile Application Development for Users with Low Vision	1	1	1	1	0.5	4.5	0.9
AS20a	Accessibility Assessment of Mobile Meteorological Applications for Users with Low Vision	1	1	1	1	0.5	4.5	0.9
AS20b	Heuristic method of evaluating accessibility of mobile in selected applications for air quality monitoring	1	1	1	1	0.5	4.5	0.9
AS20c	Accessibility assessment in mobile applications for android	1	1	1	1	0.5	4.5	0.9
PH19	Development of Automatic Evaluation Tool for Mobile Accessibility for Android Application	1	1	1	1	0	4	0.8
CB19	Accessibility in Mobile Applications of Portuguese Public Administration	1	1	1	1	0.75	4.75	1.0
AZ19	Accessibility Evaluation of Mobile Applications for Monitoring Air Quality	1	1	1	1	0.5	4.5	0.9
BJ18	Study of Accessibility Guidelines of Mobile Applications	1	1	1	1	0	4	0.8
CD18	Accessibility and Usability Problems Encountered on Websites and Applications in Mobile Devices by Blind and Normal-Vision Users	1	1	1	1	0	4	0.8
ZR18	Robust Annotation of Mobile Application Interfaces in Methods for Accessibility Repair and Enhancement	1	1	1	1	0	4	0.8
KK18	UX design guideline for health mobile application to improve accessibility for the visually impaired: Focusing on disease retinitis pigmentosa	1	1	1	1	0	4	0.8
Ro18	An assistive mobile application i-AIM app with accessible UI implementation for visually-impaired and aging users	1	1	1	1	0	4	0.8
RZ17	Epidemiology as a Framework for Large-Scale Mobile Application Accessibility Assessment	1	1	1	1	0	4	0.8
ZZ17	Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility	1	1	1	1	0	4	0.8
CP16	Methods to improve mobile application accessibility through applying a tactile user interface in a smartphone	1	1	1	1	0	4	0.8
NH16	Deaf mobile application accessibility requirements	1	1	1	1	0	4	0.8
BK16	Accessibility analysis of e-governance oriented mobile applications	1	1	1	1	0	4	0.8
KK16	UX Design Guideline for Health Mobile Application to Improve Accessibility for the Visually Impaired	1	1	1	1	0	4	0.8
CF16	Accessibility evaluation of Rich Internet Applications interface components for mobile screen readers	1	1	1	1	0	4	0.8
SC15	Accessibility Evaluation of E-Government Mobile Applications in Brazil	1	1	1	1	0	4	0.8
BB10	A unified methodology for the evaluation of accessibility and usability of mobile applications	1	1	1	1	0.75	4.75	1.0

offers a list of the selected articles, with the outcomes of the quality control. It contains the ID, the publication’s name, the five quality questions, the Score applied, and the normalization. In the normalization column, a standard scale from 0 to 1 is used. For normalization [24], the values in this column are converted utilizing Equation (1):

$$\mathrm{Normalization}=\frac{\mathrm{Score}-\mathrm{minimum}\left(\mathrm{Score}\right)}{\left[\mathrm{maximum}\left(\mathrm{Score}\right)-\mathrm{minimum}\left(\mathrm{Score}\right)\right]}$$

(1)

The minimum (Score) is equal to 0, the maximum (Score) is equal to 5, and the score is the value considered in Table 3.

4. Results

This section answers the six research questions initially posed by (1) a bibliometric assessment to compile publication records by year on the increase in research over time from journals, conferences, books, and book chapters on accessibility and mobile apps and (2) a scoping review to map studies according to the perceptions of accessibility and mobile applications.

4.1. Bibliometric Analysis

This analysis aims to respond to RQ1; Figure 3 shows the evolution of scientific production, presenting the number of documents each year. The years with the highest scientific production about accessibility in mobile applications are 2016, 2018, and 2020, which corresponds to 22.7% for each year. In 2010 and 2015, there is one scientific article corresponding to 9%. For 2017, we found two articles corresponding to 9.1%, and for 2019, we found three articles representing 13.6%. Finally, from 2000 to 2009 and 2011 to 2014, no article related to accessibility in mobile applications was found. The annual growth rate of articles published between 2000 and 2020 supports the polynomial Equation (2). The polynomial equation of degree six (2) with R-squared or coefficient of determination = 0.7826 indicates that it is a good trend line because as R-squared tends to be 1, it can be predicted that the number of publications tends to grow positively; the polynomial equation applied is the best suited because scientific production includes seasonal data with many variations.

y = −1 × 10 − 6X⁶ + 2 × 10 − 5X⁵ + 0.0009X⁴ − 0.0252X³ + 0.2179X² − 0.6859X + 0.5874

(2)

Figure 4 shows 20 conference studies demonstrating 90.9% of the total and two journal articles representing 9.1%. In this review of the literature, the most significant number of investigations found are focused on conferences. The most significant number of scientific documents are indexed in Scopus.

4.2. Review of the Literature to Map the Studies

Table 4. List of scientific articles selected in this review.

No	ID	Title	Reference	Year
1	MS20	Accessibility of mobile applications: Evaluation by users with visual impairment and by automated tools	Mateus, D.A. [9]	2020
2	ASe20	Toward Accessible Mobile Application Development for Users with Low Vision	Acosta-Vargas, P. [2]	2020
3	AS20a	Accessibility Assessment of Mobile Meteorological Applications for Users with Low Vision	Acosta-Vargas, P. [13]	2020
4	AS20b	Heuristic method of evaluating accessibility of mobile in selected applications for air quality monitoring	Acosta-Vargas, P. [12],	2020
5	AS20c	Accessibility assessment in mobile applications for android	Acosta-Vargas, P. [25]	2020
6	PH19	Development of Automatic Evaluation Tool for Mobile Accessibility for Android Application	Park, E. [15]	2019
7	CB19	Accessibility in Mobile Applications of Portuguese Public Administration	Carneiro, M. [16]	2019
8	AZ19	Accessibility Evaluation of Mobile Applications for Monitoring Air Quality	Acosta-Vargas, P. [18]	2019
9	BJ18	Study of Accessibility Guidelines of Mobile Applications	Ballantyne, M. [17]	2018
10	CD18	Accessibility and Usability Problems Encountered on Websites and Applications in Mobile Devices by Blind and Normal-Vision Users	Carvalho, M. [26]	2018
11	ZR18	Robust Annotation of Mobile Application Interfaces in Methods for Accessibility Repair and Enhancement	Zhang, X. [27]	2018
12	KK18	UX design guideline for health mobile application to improve accessibility for the visually impaired: Focusing on disease retinitis pigmentosa	Kim, W.J. [28]	2018
13	Ro18	An assistive mobile application i-AIM app with accessible UI implementation for visually-impaired and aging users	Russ, K.C.W. [29]	2018
14	RZ17	Epidemiology as a Framework for Large-Scale Mobile Application Accessibility Assessment	Spencer, R. [30]	2017
15	ZZ17	Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility	Zhang, X. [31]	2017
16	CP16	Methods to improve mobile application accessibility through applying a tactile user interface in a smartphone	Choi, H. [32]	2016
17	NH16	Deaf mobile application accessibility requirements	Nathan, S.S. [33]	2016
18	BK16	Accessibility analysis of e-governance oriented mobile applications	Balaji, V. [34]	2016
19	KK16	UX Design Guideline for Health Mobile Application to Improve Accessibility for the Visually Impaired	Kim, W.J. [35]	2016
20	CF16	Accessibility evaluation of Rich Internet Applications interface components for mobile screen readers	Carvalho, L.P. [36]	2016
21	SC15	Accessibility Evaluation of E-Government Mobile Applications in Brazil	Serra, L.C. [36]	2015
22	BB10	A unified methodology for the evaluation of accessibility and usability of mobile applications	Billi, M. [37]	2010

shows the 22 selected primary studies, sorted by year of most recent publication. It includes the article number, the assigned indicator (it was generated with the first words of the surnames of the first two authors and the year of publication), the title of the scientific article, the first author with the reference, and the year of publication.

This scoping review presents a classification scheme using the keywords. At this stage, we reviewed the 22 primary studies in their entirety, thoroughly analyzed: the abstracts, keywords, introduction, methodology, discussion, and conclusions. In a spreadsheet, we recorded the characteristics of each primary study related to the research questions.

To record the characteristics of the primary studies, we considered the six research questions that include: (1) investigate the disability types of accessible mobile applications developed between 2000 and 2020; (2) classify the methods and techniques used in the design of accessible mobile devices; (3) identify accessibility guidelines that apply to mobile applications; (4) categorize the types of research and impact on accessible mobile applications; (5) reveal the most used tools for evaluating accessibility in mobile applications; and (6) categorize the domain fields in which accessible mobile applications are used.

In the following, we present and discuss the answers to the research questions of this study; the dataset and evaluation for replication are available for replication in the Mendeley repository [38].

4.2.1. RQ1. What Are the Disabilities That Are Most Considered for Accessibility Evaluation in Mobile Applications?

We have chosen investigations that employ accessibility in mobile applications. An application is considered accessible for mobile devices when all the elements that compose it are easily manageable by all people, without any barrier or limitation.

Table 5. Studies by disability.

Type of Disability	ID
Cognitive (two studies)	NH16, ZR18
Motor coordination (one study)	Ro18
Sensory: visually impaired, hearing (19 studies)	AS20a, AS20b, AS20c, ASe20, AZ19, BB10, BJ18, BK16, CB19, CD18, CF16, CP16, KK16, KK18, MS20, PH19, RZ17, SC15, ZZ17

shows the primary accessibility studies by type of disability identified. In reviewing the papers, we have applied the definitions:

(1)

Cognitive or intellectual disability is an issue considered by mental development that disturbs the learning method.

(2)

Motor coordination is a problem associated with a deficiency of one or more parts of the body’s change capabilities.

(3)

Sensory disability is related to (1) vision, including low vision and deafness; (2) hearing impairment, which provides deafness and hearing loss.

Table 5 illustrates the accessibility associated with the disability; we found 19 primary studies on sensory disability representing 86.4% of the total; then, two studies apply accessibility for cognitive disability with 9.1%. One study on motor coordination corresponds to 4.5 % of the total.

4.2.2. RQ2. What Methods and Techniques Are Applied to Evaluate Accessibility in Mobile Applications?

Table 6. Evaluation methods applied by disability.

Method	Cognitive	Motor Coordination	Sensory (Visually Impaired, Hearing)
Automatic (four studies)			AS20c, AZ19, MS20, RZ17.
Combined: (three studies)			AS20a, ASe20, PH19.
Heuristic (four studies)			AS20b, BB10, BJ18, SC15.
Manual (11 studies)	NH16, ZR18.	Ro18.	BK16, CB19, CD18, CF16, CP16, KK16, KK18, ZZ17.

presents the methods and techniques applied to evaluate accessibility in mobile applications; reviewing the documents, we have applied the following definitions:

(1)

Automatic: consists of using an automatic review tool for which it is required to install an app on the mobile device, which issues a report of the accessibility barriers encountered.

(2)

Combined: consists of using two methods—one automatic and one manual. In the first phase, an automatic review tool is used. A manual review method is applied in the second phase, a simple manual process, or a more sophisticated method such as a heuristic.

(3)

Heuristic: a manual method based on establishing a scale of values assigned to a set of accessibility barriers (heuristics) reviewed manually by an accessibility expert.

(4)

Manual: based on a manual review process performed by an accessibility expert, not necessarily involving scales and heuristics.

This review found: (1) four scientific articles that mention an automatic method for accomplishing accessibility in a mobile application, representing 18.2% of the total; (2) three studies that indicate a combined method, representing 13.6%; (3) four studies that focused on the heuristic, representing 18.2%; and (4) 11 studies focused on a manual method, representing 50.0% of the total. This scoping review revealed that 86.4% of mobile app accessibility evaluation methods are used in sensory disability, followed by 9.1% in cognitive disability and 4.5% of the total motor coordination disability.

4.2.3. RQ3. What Are the Accessibility Guidelines Applied to Mobile Applications to Achieve Accessibility?

Table 7. Guidelines and disability.

Guidelines	Cognitive	Motor Coordination	Sensory (Visually Impaired, Hearing)
WCAG 2.0 (seven studies)			BB10, BJ18, BK16, CD18, KK18, PH19, SC15.
WCAG 2.1 (seven studies)			AS20a, AS20b, AS20c, ASe20, AZ19, CB19, MS20.
MWAG (one study)			KK16.
Other guidelines (seven studies)	NH16, ZR18.	Ro18.	CF16, CP16, RZ17, ZZ17.

shows the plans to improve accessibility by disability; in the review of the scientific articles, we employed the definitions:

(1)

WCAG 2.0 incorporates the primary research utilizing the Web Content Accessibility Guidelines with version 2.0 [39] to improve mobile application accessibility.

(2)

WCAG 2.1 comprises the primary researches that employed the Web Content Accessibility Guidelines with version 2.1 [4] to improve mobile application accessibility.

(3)

MWAG (Mobile Web Accessibility Guidelines) refers to the guidelines used to evaluate accessibility in mobile applications specified by the W3C.

(4)

Other guidelines include the primary research that helps improve accessibility in mobile applications by employing guidelines without specifying the standard.

This study found: (1) seven studies that refer to WCAG 2.0 to achieve accessibility in mobile applications, representing 31.8% of the total; (2) seven studies that apply WCAG 2.1, representing 31.8%; (3) one study focused on MWAG, representing 4.5%; and (4) seven studies focused on other guidelines, representing 31.8% of the total. Among the other guidelines, the selected studies suggest including closed captioning on videos, metadata annotation tools, compliance analysis of components with Accessible Rich Internet Applications 1.0, tactile user interfaces, review of the interaction between different errors, interaction proxies for accessibility customization, and descriptive video service.

4.2.4. RQ4. What Type of Research and Contributions Has Been Found Related to Mobile Applications and Accessibility?

Table 8. The research was employed for the design of accessible mobile applications.

Research	Cognitive	Motor Coordination	Sensory (Visually Impaired, Hearing)
Mixed (11 studies)	ZR18.	Ro18.	AS20a, AS20b, ASe20, AZ19, BB10, BK16, CB19, CF16, KK16.
Qualitative (nine studies)	NH16.		BJ18, CD18, CP16, KK18, MS20, PH19, RZ17, ZZ17.
Quantitative (two studies)			AS20c, SC15.

provides a summary of the type of research employed by disability related to accessibility in mobile applications; in the review of scientific articles, we have used the definitions:

(1)

Mixed research comprises a process that collects, analyzes, and pours quantitative and qualitative data into the same research study.

(2)

Qualitative research is an inductive and adaptive model conducted through narrative inspection of applications. This research includes small-scale research, emphasizes the validity of the research through immediacy to empirical actuality, and often does not test notions or hypotheses; statistical analysis is not possible.

(3)

Quantitative research generates numerical data that allow data to be accumulated and examined. This research relies on objectivity to arrive at learning; it uses specific, self-controlled dimensions. This research contains combined methods and assessments for data collection.

Table 8 includes 11 mixed research papers, representing 50% of the total, nine papers using qualitative research, representing 40.9%, and two quantitative research papers, corresponding to 9.1%.

Figure 5 presents the mixed studies include (1) one study applying cognitive disability, representing 4.5%; (2) one study focusing on motor coordination, corresponding to 4.5%; and (3) nine studies focusing on sensory disability, representing 40.9%. The qualitative studies include (1) eight studies related to sensory disability, corresponding to 9.1% of the total; and (2) one study focused on motor disability, representing 4.5%. The quantitative studies include two studies related to sensory disability, corresponding to 9.1% of the total.

4.2.5. RQ5. What Tools Are Used to Test Accessibility in Mobile Applications?

To answer this question, we reviewed the papers by reviewing the most used tools according to disability in the full papers, as shown in

Table 9. Documents according to the tool used to evaluate accessibility in mobile applications.

Tool	Cognitive	Motor Coordination	Sensory (Visually Impaired, Hearing)
Accessibility Scanner (seven studies)			AS20a, AS20b, ASe20, AZ19, BK16, KK18, RZ17.
Not specified (15 studies)	NH16.	Ro18.	AS20c, BB10, BJ18, CD18, CP16, KK16, PH19, SC15, CF16, MS20, ZZ17, CB19.

. In reviewing the papers, we applied the following definitions:

(1)

Accessibility Scanner: this is an automatic review tool from Google Play; it evaluates accessibility in Android applications without advanced technical knowledge.

(2)

Not specified: it does not specify the use of any tool to assess the accessibility of mobile applications; they are based on manual methods with guidelines defined according to some standards or defined by the authors performing the evaluation.

This review found (1) seven studies that use the tool Accessibility Scanner to assess sensory disability, representing 31.8% of the total, and (2) fifteen studies that do not specify any type of tool used corresponding to 68.2%. One study focuses on cognitive disability, representing 4.5%; another study is related to motor disability, representing 4.5%; the most significant number of studies focuses on sensory disability, with eight studies representing 91% of the total.

4.2.6. RQ6. Which Domains Are the Most Evaluated?

The following sectors can be observed regarding the domains addressed, as shown in

Table 10. Domain studies.

Sector	Studies
Administration	CB19
Air quality	AS20b, AZ19
Banking	CD18, AS20c, BJ18
Comunication	PH19, AS20c, BJ18
Education	ASe20, ASe20, BB10, CF16, MS20, NH16, AS20c, BJ18
E-Government	BK16, SC15
Entertainment	AS20c, BJ18
Epidemiology	RZ17
Health	KK16, KK18
Lifestyle	AS20c, BJ18
Meteorology	AS20a
Productivity	AS20c, BJ18
Services	CP16, Ro18, ZZ17
Shopping	AS20c, BJ18
Social media	ZR18, AS20c, BJ18

. The most significant number of studies correspond to the education sector, representing 21.6% of the total, followed by banking, communication, services, social media—each sector with 8.1% (the four sectors represent 32.4%). Air quality, entertainment, e-government, health, lifestyle, productivity, shopping, administration, epidemiology, and meteorology have less than 6% of each sector; the 10 sectors total 45.9%.

Finally, Figure 6 summarizes the most frequently repeated keywords in the 22 extracted articles: Key1 = “accessibility” is found in all 22 studies, followed by Key2 = “application” found in 15 studies. Key3 = “assessment” is repeated in nine studies. Key4 = “guideline” is found in nine studies. Key5 = “disabilities” is repeated in eight studies. Key6 = “user” is found in five extracted studies, and finally, Key7 = “design” is repeated in five of the 22 extracted studies. The authors also summarize the answers to RQ1 manifested with 22 studies related to disabilities in evaluating mobile applications developed between 2000 and 2020. RQ2 is answered with 22 studies that allowed us to classify the methods and techniques used in mobile application design according to disability. RQ3 is answered with 15 studies that determine the accessibility guidelines used in mobile applications. RQ4 is answered with 22 studies related to research and support categories used in mobile applications. RQ5 is answered by 12 studies related to mobile applications and accessibility tools considering disability. Finally, RQ6 is answered with the 22 extracted studies identifying the domain areas in which accessible mobile applications are used.

5. Discussion

This review used the PRISMA-ScR method to improve its quality and clarity by avoiding bias. This process allows three-step filtering to guide the reviewers. Additionally, the PRISMA-ScR includes a checklist (see Appendix A) to improve the quality of this review. In this review, most authors argue that (1) a mobile application can be accessible when incorporating elements that allow easy and intuitive interaction with physical or logical controls; for this, it is essential to incorporate bidirectional voice communication. (2) If the application contains video communication, the playback, transmission, and recording quality must include subtitles for the hearing impaired. (3) Audio description on videos is essential for visually impaired users. (4) It is necessary to include some software access to support products and authoring tools. Finally, the authors suggest designing accessible and inclusive mobile applications; the WCAG 2.1 [4] should be applied, containing guidelines grouped into four principles: perceivable, operable, understandable, and robust. Each guideline includes conformance criteria, which can be tested according to the three levels A, AA, and AAA, with level AAA being the highest level of accessibility.

In this research, consideration was repaid to preferring the most used and adapted search strings according to each database’s query structure. To reduce this limitation, we apply the PICO criteria to our search strings [9]. The choice of the five ACM Digital Library databases, IEEE Xplore, ScienceDirect, Scopus, and WOS, is sufficient because 32 of 211 scientific articles were duplicated, which implies that the coverage of the databases is high, so much so that some of them could have been excluded. However, the primary research search terms, such as accessibility and mobile applications, are traditional, well-defined, and accepted terms, reducing neglected research. In addition, because the research is focused on finding primary research on accessibility and mobile apps, there is less concern about capturing preliminary domain-related research.

Likewise, when searching for some scoping review studies, we found that most of them focus on evaluating the usability of applications or how they are applied in various domains [40]. However, they are not concerned with evaluating the accessibility of native mobile applications considering the users’ disabilities. Regarding accessibility in mobile applications associated with disability, we found that the most significant number of primary studies refer to the sensory disability with 86.4%, including visual and hearing disability, followed by cognitive disability with 9.1%. We found very few studies, around 4.5%, referring to motor disability.

The most used method for assessing accessibility in mobile applications according to disability is manual, with 50% of the studies extracted. Automatic review and heuristics achieve 18.2% of the extracted studies, while the combined method is little used, representing 13.6%. According to several authors [41], it is the most recommended because there are accessibility barriers that automatic review tools cannot detect. In the future, the great challenge for accessibility experts is to design an automatic evaluation tool that incorporates [6] artificial intelligence algorithms.

The statistics of this study correlate with the proposals to improve accessibility according to the type of disability in mobile applications; the scoping review revealed that 63.6% apply WCAG 2.0 and 2.1 [4] because they provide a better level of accessibility and allow anticipating future regulatory changes.

The selected studies revealed that the education sector has the highest number of accessible mobile applications with 21.6%, followed by banking, communication, services, and social media.

The quality of the 22 studies extracted was validated by five supplementary questions (see Table 1) related to the impact factor and the quartile in which the article was located. Another strength of this review was to apply Cohen’s Kappa coefficient to achieve a level of precision and reliability in selecting articles with a moderate concordance of 83.4% between reviewers.

This scoping review has its restrictions; it is not infallible as in any other secondary research method. One limitation of this study is that only studies in English were included. Another limitation is the lack of comparability of the included studies, as they may differ according to country, culture, and the different methods used, as they are not guided using the same standard for assessing accessibility in mobile applications.

6. Conclusions

This scoping review evidenced current trends and issues related to mobile application accessibility and sensory, cognitive, and motor disabilities by analyzing primary scientific articles published between 2000 and 2020.

Due to the nature of the scoping methodology, this study did not conduct an in-depth analysis of the articles; future research will include a more systematic review of individual differences in cognitive, sensory, and motor disabilities considered in the selected studies. This research has succeeded in identifying accessibility barriers in mobile applications and domain areas that need further investigation. Much work remains to be completed so that researchers can design intelligent tools to interpret mobile app accessibility evaluation results meaningfully.

Similarly, this research provides scholars and practitioners with the status of mobile applications related to cognitive, motor, and sensory disabilities. We recommend (1) complementing automatic review evaluations with a heuristic method to ensure an adequate level of accessibility in mobile applications for future work, (2) using this study as a starting point to create a software tool that employs WCAG 2.1 based on artificial intelligence algorithms to help developers evaluate mobile applications, and (3) working on legislation in each country and solutions incorporating user feedback to evaluate accessibility with systematization and monitoring tools.