Quality Assurance in E-Learning: A Proposal from Accessibility to Sustainability

Abstract

Given the importance of developing and offering accessible education for all, indispensable aspects of education for sustainable development (ESD) are needed. This study addresses that need by proposing a quality self-assessment for virtual education from an accessibility perspective. This proposal is based on previous literature about quality assurance in e-learning that considered accessibility and its application in the field of higher education. The bibliographic review was conducted by following Multivocal Literature Review (MLR) guidelines. The initial search returned 999 items from 5 academic databases and 32,200 professional sources from Google. After reviewing the sources, 37 of them were included. Then, the accessibility criteria were identified and integrated into an evaluation model. Such a model is divided into four dimensions: (1) organization, (2) student body, (3) teaching, and (4) infrastructure. The model also includes a set of standards (16), requirements (48), and evidence (63) that apply to each dimension. Moreover, self-assessment guidelines for accessible virtual education were proposed. They included a conceptual and theoretical framework, a self-assessment model, and a methodology for applying the model. The methodology included five phases: planning, model tuning or refinement of the model, evaluation, results, and continuous improvement. As future work, the implementation and validation of the guidelines will be carried out.

1. Introduction

The era of knowledge and technology has invigorated people’s social, educational, and organizational environments, forcing institutions to re-formulate their strategies to provide accurate quality responses to the environmental demands. The field of education has not been the exception, making room for new and varied forms of non-face-to-face education such as virtual education [1] and face-to-face education through electronic learning (e-learning). Learning during the COVID-19 outbreak has gone from a complementary methodology to a mandatory model [2,3] in education, becoming a challenge in many institutions due to the short time to adapt to the pedagogical process, added to the technical, academic, and communication difficulties caused by the pandemic [4]. In this sense, educational and business institutions must improve and ensure the quality of their service to achieve a competitive advantage by providing value to students and society as a beneficiary.

Quality education is established as an integral element of Education for Sustainable Development (ESD) [5]. ESD promotes sustainable development [5] that seeks to eradicate poverty and to promote economic prosperity, social inclusion, environmental sustainability, peace, and good governance for all. In 2015, the United Nations adopted the 2030 Agenda for Sustainable Development [6]. This agenda established 17 Sustainable Development Goals (SDGs), whose aspiration for 2030 was to place the world and its societies on a path towards a better future [6]. The importance of education for sustainable development is recognized in SDG-4: Quality Education [7], which aims to “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”. Quality education contributes to better results in the development of people, as well as their communities [7].

E-learning and information and communication technologies (ICTs) contribute to the SDGs, specifically SDG-4, by promoting virtual or non-face-to-face education. Through ICTs, students can have access to learning resources anywhere and at any time. Teachers can deliver training and can facilitate interactive tutoring virtually. Thus, breaking down economic, social and cultural barriers [8].

From the point of view of e-learning, the term “quality assurance” does not have a commonly accepted definition [9,10,11]; however, Vagarinho [10] defines it from the compilation of 24 characteristics grouped into four areas (learning platform, improvement, skills, and sustainability) that must be met. On the other hand, Marciniak [9] defines it as the process of striving to achieve the dimensions that make up the quality of virtual education and indicators associated with them; Duque [7] refers to it as the fulfillment of requirements, the satisfaction of customer needs and as a competitive strategy. From these definitions and those referred to in each study mentioned above, quality itself can be defined as the characterization given to a product, in this case, virtual education, in line with the needs expected by the user. The client, whether they are a student, a teacher, society, or the government, is considered a fundamental pillar of the management of training institutions to achieve excellence. Therefore, quality assurance must necessarily be evaluated from two main aspects: (a) the quality properties that virtual education possesses and (b) the value judgment given to these properties; that is to say, in a virtual environment, not only the evidence of the aspects that it possesses must be demonstrated, but also the usefulness of these properties [11] and the teaching–learning process itself should be supported by evidence.

There are many standards, models, and regulations as quality characterization instruments. When reviewing the literature, it is evident that many studies highlight the importance of quality in training or virtual education [12]. From there, new models are proposed, or existing methodologies are applied to assess the quality of training at different levels. These could be at the level of institution, program or study, career, virtual education platform, or a particular course [9]. Authors such as Hilera and Hoya [13] compile and highlight in their research ten e-learning quality standards and models that they denote since 1999, showing an interest in quality for approximately two decades.

The primary regulations referenced, which several proposals for new models take as a baseline, are from the ESG (Standards and Guidelines for Quality Assurance in the European Higher Education Area) [14], from which essential standards stand out [15]: (a) quality assurance policies, (b) program design and approval, (c) student-centered learning, teaching, and assessment, (d) student admission, (e) teaching staff, (f) learning resources (includes accessibility of materials and the e-learning system), student support, (g) information management, (h) public information, (i) continuous monitoring and periodic review of programs, and (j) external quality assurance.

The terms evaluation and self-evaluation have a connotation of quality verification. Those responsible for the guarantee, as mentioned above, and the quality control are local governments, quality assurance agencies, institutional management bodies, and the universities themselves as institutions with social responsibility [16]. A self-assessment process, by its essence, is a participatory process of quality management, which is supported by a defined standard or pattern (self-assessment model) to establish the strengths, weaknesses, and improvement plans that pursue a continuous assurance of quality [17].

Addressing quality from an accessibility and inclusion perspective is also a challenge and, in turn, a necessity for virtual education institutions since, for a long time, the importance of accessibility or the limited approach with which virtual education is conceived has been minimized. There has been a misconception of accessibility as simply having an accessible website, or even a lack of knowledge of accessibility. Kumar and Owston [18] proposed e-learning accessibility based on the e-learning platform’s accessibility and evaluated it from two methods: compliance tests (such as compliance with the WCAG) and user tests or usability tests to identify accessibility barriers. Seale [19] proposed accessibility in e-learning from a holistic view, which started from a conceptualization and study of disabilities and, in addition to the accessibility of the platform, considered elements such as institutional policies, support, and assistance to students and teachers with disabilities, as well as assistive technologies in other aspects.

Other studies reveal the need to rethink the universal learning design to better adapt to students with disabilities since it is not enough to have an accessible LMS (learning management system). Still, curricular adaptations are required to help these students [20]. All of this confirms the importance of accessibility in e-learning, placing accessibility as a transversal component in the quality of virtual education [21].

To achieve the SDG-4 [22], education must be of quality and must be accessible, in the sense of accessibility described above. In this way, based on accessibility, it will be possible to move towards sustainability in online education that uses primarily web-based technology, because accessible education gives people with disabilities the opportunity to learn in equal conditions [23], but web accessibility is also good for the planet because some of the flagship good practices in accessibility can reduce the carbon footprint of a website [24].

This article presents a quality self-assessment proposal for virtual education from an accessibility and inclusion perspective towards sustainability with a focus on the United Nations sustainable development goal on quality inclusive education. Thus, accessibility is approached through a holistic perspective, which goes beyond content accessibility and overcomes the bias of being dedicated exclusively to disability. The proposal is based on a literature review about quality assurance in virtual education. In particular, we conducted a systematic review of the literature of the last six years on models and standards for evaluating the quality of e-learning. One hundred thirty-four dimensions were identified with their respective indicators and criteria (504). After a comparative analysis of the similarities of their approaches, measures, and indicators, they could be summarized/categorized into a set of 18 dimensions. Then, 53 criteria related to accessibility and inclusion were identified, which were grouped into 8 criteria. In this way, the basis of the evaluation model proposed in this study was established. Based on those results, an evaluation model and a methodology for its implementation were defined as part of a quality self-assessment process. Finally, a guide for the self-assessment of the quality of accessible virtual education was built.

The rest of this document is organized as follows. First, in Section 2, the methodology followed for the construction of the evaluation model and the design of the proposed self-assessment guide is shown. Then, Section 3 presents the self-assessment guide with its main components in general. Next, in Section 4, the article’s discussion is established, and, finally, in Section 5, the conclusions of this study are presented.

2. Materials and Methods

The method followed for constructing the model and the self-assessment guide is shown in Figure 1.

2.1. Systematic Literature Review

As an initial stage, a study of the quality assessment models and their implementation was carried out, which consisted of a systematic review of the literature with the MLR (multivocal literature review) methodology [18]. This methodology involves an examination of scientific and gray literature to know the current situation concerning the evaluation of quality in virtual education. It is from this objective that three research questions appeared:

RQ1: What regulations, policies, or standards guide virtual education evaluation or quality assurance in HEIs?

RQ2: What processes, criteria, and/or dimensions have the evaluation or quality assurance models covered, and what accessibility aspects or characteristics do they address?

RQ3: What mechanisms are contemplated to put into practice the evaluation and/or accreditation of quality in accessible virtual education in HEIs?

The descriptors detailed in

Table 1. Search terms.

Term	Synonyms
model	standard, guideline, normative, criteria
quality evaluation	quality assessment, QA
higher education	college, university, technological institute
e-learning	e-learning, virtual education

were used as criteria regarding the search terminology.

The resulting search string is the following (1); this string was used in six search engine databases: SCOPUS, IEEExplore, ERIC, WEB OF SCIENCE, and GOOGLE SCHOLAR for formal literature and GOOGLE for gray literature.

(standard | model | guideline | normative | criteria) AND (“quality evaluation” | “quality assessment” | QA) AND (“higher education” | college | university | “technological institute”) AND (e-learning | elearning | “virtual education”)

(1)

Regarding the inclusion/exclusion criteria, it was determined to include studies published from January 2015 to December 2020, written in English and Spanish, and was related to the evaluation or assurance of quality in virtual education. In addition, documents that were not supported by models, standards, regulations, or quality were excluded, as well as documents whose complete access was not possible, duplicate papers, or documents with broken links. As of December 2020, a total of 33,199 articles were obtained. After applying the selection criteria and the systematic review process, 37 studies were included, which corresponded to formal or academic literature and 9 to gray literature (see

Table 2. Summary of the systematic review process concerning the selection of studies.

	Identification	Screening	Elegibility	Included
Google	32,200	110	29	9
Google Scholar	762	109	29	9
Scopus	166	83	24	8
Web of Science	46	44	15	9
IEEExplore	22	7	2	1
Eric	3	3	1	1
Total	33,199	356	110	37

).

From the selected studies and to determine the validity of the sources, their quality was evaluated based on a checklist composed of five criteria, the first four corresponding to the formal literature: QA1 Is the author’s recognition clearly identified or associated with a recognized organization based on the experience of the subject? QA2 Is the research methodology identified? QA3 Is the support of the objective adequately described impartially? QA4 Is there an unprecedented and significant contribution to research? And a fifth question for the gray literature: QA5 What is the relevance level of the document?

From the quality analysis of the academic literature of the 28 articles, 17 of them had a clearly identified research methodology (QA2); 25 of the papers adequately described the support of the objective in an impartial manner (QA3); two pieces presented an unprecedented and significant contribution to research (QA4). Of the nine articles selected as gray literature, three are level 1 (medium-high recoverability/credibility) since they correspond to popular science books, magazines, or specialized foundations. In addition, six articles are considered 2nd level (moderate recoverability/credibility) for being HEI publications or publications/studies of civil organizations. As part of the analysis process, Krippendorff’s alpha (α) was estimated for nominal data [25]. Two observers participated to measure the agreement between them, who performed the quality assessment independently in a sample of 18 publications of the 37 studied. As a result, it is concluded that the evaluators interpreted the data similarly and acceptably with an alpha value of 0.714.

Data extraction and analysis were supported with a data collection matrix for primary data extraction [26] with the following fields: title, author(s), year of publication, type of publication, source or bibliographic database, and related research question.

To build a solid foundation for a self-assessment model proposal, the previously mentioned research questions were posed; from them, the critical components of the self-assessment models and their accessibility criteria were determined.

Concerning RQ1, 21 models were identified; these were analyzed from a classification matrix (see details in Appendix A

Table A1. Models for evaluating quality in virtual education proposed.

MODEL/Author(s)—Year	Characteristics: Level Type Focus	Description
Hadullo, Oboko & Omwenga—2017 [38]	Institutional Conceptual or theoretical Developing countries	Model for evaluating the quality of e-learning systems in developing countries, the result of adapting the Briggs Framework to e-learning based on five existing models
Dilan & Fernandez—2015 [37]	Institutional Conceptual or theoretical Developing countries	Conceptual framework of quality of a virtual institution aimed at quality improvement and quality assurance.
SQAMELS Farid—2018 [52]	Platform Conceptual or theoretical Developing countries	Sustainable quality assessment model for e-learning systems from a soft-ware perspective. Its focus is on the technological platform that supports the programs, excluding sections such as pedagogical, personal, institutional, cultural, and social
TeSLA Huertas—2017 [57]	e-assessment Conceptual or theoretical European Union	A conceptual framework for an internal quality assurance system for e-assessment, in the context of higher education and e-learning, developed considering the standards and guidelines for quality assurance of the European Higher Education Area.
Marciniak—2018 [9]	Program Conceptual or theoretical Spain	A comprehensive model for evaluating the quality of online education programs, whose focus is firstly on assessing the quality of the online program itself, and then on the continuous evaluation of the online education program, to improve through feedback and self-adjustment.
Mejia-Madrid & Molina Carmona [47]	Institutional Conceptual or theoretical International	Model for evaluating the quality of distance higher education based on information and communication technologies (ICT), whose purpose is to guarantee the proper use of ICT in an institution’s teaching and learning processes, academic processes, and administrative processes.
OSCQR—2019 [50]	Course Certification International	Scorecard OSCQR allows course design review, constituted as a tool to improve the quality and accessibility of a course design, part of the OLC quality framework to guarantee the excellence of online learning of higher education institutions.
OCL—2011 [58]	Program Certification International	Online Program Management Scorecard aimed to measure the effectiveness of an institution’s online learning programs. Part of OLC’s quality framework to ensure online learning excellence for higher education institutions.
FRAMEWORK OF INSTITUTIONAL CONDITIONS FOR ONLINE TEACHING Luna—2018 [48]	Institutional Conceptual or theoretical México	An analytical framework for evaluating the institutional conditions of online teaching in higher education.
Torres-Barzabal—2019 [27]	Teaching Conceptual or theoretical European Union	Qualitative evaluation model of quality of online teaching from a pedagogical point of view, for undergraduate and postgraduate programs. The evaluation has two approaches, one related to the content and the information provided in the courses referring to the teaching action and a second approach about the teaching process applied in each class.
eMM (e-Learning Maturity Model) Marshall—2010 [49]	Institutional Self-assessment International	A quality framework for e-learning improvement, designed to assess the maturity of an institution to identify the key processes and practices necessary to achieve robust and sustainable improvements in e-learning quality. It can be considered a version of CMM from an educational perspective.
E-LEARNING QUALITY Masoumi & Lindstrom—2012 [59]	Course/Program/Institutional Self-assessment Developing countries	A framework to promote and ensure quality in virtual institutions sensitive to specific cultural contexts.
CAPEODL Khan—2005 [60]	Program Self-assessment International	Online program evaluation of the model. The model is the integration of the Continuity Model in E-learning P3 (Person as Processes-Products) and the E-learning Framework of Khan (2004) from the seven stages of e-learning (planning, design, production, evaluation, marketing, instruction, and maintenance).
ACCESSIBILITY ACCREDITATION MODEL—2013 [28]	Course Certification Latin America and the Caribbean	Accreditation model for accessibility in virtual education of the ESVIAL Project and the Latin American and Caribbean Institute for Quality in Distance Higher Education (CALED), whose objective is to certify the quality and accessibility of virtual courses
CALED—2010 [29]	Program Certification Latin America and the Caribbean	Self-assessment model of distance undergraduate programs of the Latin American and Caribbean Institute for Quality in Distance Higher Education (CALED) whose purpose is to contribute to the improvement of quality in the teaching of Distance Higher Education
COLOMBIA ACCREDITATION MODEL—2013 [30]	Program Certification Colombia	Official accreditation model of undergraduate, professional technical, and technical training programs for face-to-face and distance learning in Colombia.
MEXICO ACCREDITATION MODEL—2018 [31]	Program Accreditation Mexico	A model of evaluation and accreditation of educational programs in higher education institutions with distance or online modalities (2017). CIEES is an accreditation body endorsed by COPAES (Council for the Accreditation of Higher Education in Mexico).
ACCREDITATION MODEL OF COSTA RICA—2011 [32]	Program Accreditation Costa Rica	Official model of career accreditation of Costa Rican universities that guarantees that a quality service is provided through self-assessment and external evaluation processes.
PDPP Zhang & Cheng—2012 [51]	Course Conceptual or theoretical International	The four-phase evaluation model for e-learning courses includes planning, development, process, and evaluation of products (PDPP).
UNIQUe EFQUEL—2011 [41]	Institutional Certification European Union	UNIQUE is a high-quality institutional certification for the exceptional use of ICT in learning and teaching, whose model is subdivided into three evaluation dimensions (learning and institutional context, learning resources, and learning processes).
ELQ—2008 [53]	Institutional Conceptual or theoretical European Union	E-Learning quality model for evaluating the quality of e-learning in higher education.

). In response to RQ2, 134 dimensions and 504 criteria were identified, of which 53 measures are related to accessibility (see Appendix A

Table A2. Characteristic dimensions of quality assessment models in virtual education and accessibility criteria.

Model	Dimensions	Accessibility Criteria
Hadullo, Oboko & Omwenga—2017 [38]	Course Development (D1) Student Support (D2) Evaluation design (D3) Institutional factors (D4) User characteristics (D5) Acting in general (D6)	Ease of use of the platform (A1). Training for students and teaching staff in using the platform and e-learning resources (A2). Internet access and ICT access should be made available to students and teachers (availability for loan) (A3).
Dilan & Fernandez—2015 [37]	Knowledge management (D7) Economics and financing (D8) Teacher and staff training (D9) Role of the teacher and the student (D10)	Not evidence.
Sqamels Farid—2018 [52]	System Quality (D13) Service quality (D14) Charisma (D15)	The ability of the student to access learning with minimal effort. (A4). Possibility of access to the platform from various locations (rural/urban) prioritizing students with disabilities. (TO 5).
TeSLA Huertas—2017 [57]	Policy and strategy for quality assurance in e-assessment (D16). Environment and infrastructure of e-assessment (D17). Course curriculum and assessment resources (D18). Student support (D19). Support for teachers (D20). Learning analysis (D21). Public information (D22).	The course curriculum should reflect the environment and infrastructure of the e-assessment with the possible exam scenarios planned. The evaluation resources must provide teachers and students with different variants to support any learning style, as well as students with special needs (physical, social, mental, etc.). (A6).
Marciniak—2018 [9]	Justification of the program (D23). Program objectives (D24). Student profile (D25). Thematic content of the e-learning program (D26). Learning activities (D27). Online teacher profile (D28). Learning manuals (D29). Educational strategies (D30). Tutoring (D31). Assessment of student learning (D32). Virtual platform (D33). Initial evaluation of the program (D34). Process evaluation of the program (D35). Final evaluation of the program (D36).	Appropriate, sufficient, up-to-date, motivating, and accessible learning materials for students. (A7).
Mejia-Madrid & Molina Carmona [47]	Technology for learning and knowledge, (D37). Teaching and learning processes enhanced with ICT. (D38). Strategic processes that support distance education. (D39).	Allow for diversity and accessibility of learning resources (A8). Allow flexible learning based on student needs (A9).
OSCQR—2019 [50]	Summary and course information (D40). Technology and tools (D41). Design and layout (D42). Content and activities (D43). Interaction (D44). Evaluation and feedback (D45).	Printable syllabus available in PDF and HTML (A11) The course includes links to relevant policies on plagiarism, use of computers, complaints, and disability accommodations (A12). All technological tools comply with accessibility standards (A13). A logical, coherent, and orderly design is established. In addition, the course is easy to navigate (consistent color scheme and icon layout, related content organized together, prominent titles) (A14). There is enough contrast between the text and the background to see the content easily (A15). Text is formatted with headings and other styles to improve readability and document structure (A16).
OCL—2011 [58]	Institutional support (D46). Technological support (D47). Development and instructional design of online courses (D48). Structure of online courses (D49). Teaching and learning (D50). Social and student participation (D51). Support for teachers (D52). Student support (D53). Assessment and assessment (D54)	Development and instructional design of online courses: Alternative publishing content (CDs) is available for students who do not have permanent access to the internet or low-speed connections (A17). Alternative assessment systems are available for students who do not have permanent access to the internet (A18). Usability tests are applied to incorporate the recommendations issued or results obtained (A19). Web content accessibility guidelines (WCAG) are used in content and on the platform (A20). Instructional materials are easily accessible to students and are easy to use (A21). The course provides an adequate response to the needs of students with disabilities through alternative instructional strategies and referral to special institutional resources (A22). The program demonstrates compliance with and review of accessibility standards (A23). Text content is available in an easily accessible format, preferably HTML (A24). All text content is readable by assistive technology, including a PDF or any text contained in an image (A25).
Framework of Institutional Conditions for Online Teaching. Luna—2018 [48]	Institutional policy (D55). Institutional organization (D56). Institutional regulations (D57). Institutional plans and programs (D58). Online educational model (D59). Teaching work conditions (D60). Infrastructure and equipment (D61)	The technological infrastructure used in educational spaces for teaching courses (videoconferences, satellite links, internet applications, and others) ensures continuous and uninterrupted access for students and teachers during the course duration (A26). There are special and appropriate facilities for carrying out collective activities mediated by ICT (A27). There are a digital library and library services accessible to all students and teachers, regardless of their geographical location and when they are consulted (A28). A text equivalent is provided for each non-text element (alt tags, subheadings, transcripts, etc.) (A29)
Torres-Barzabal—2019 [27]	Course content and information: Identification of the teaching action (D62). Delimitation of the teaching action (D63). Design of teaching action (D64) Learning process: Teaching participation (D65). Feedback (D66). Motivation (D67) Evaluation (D68).	The content must be presented homogeneously concerning color, text, font, distribution of information, logical sequence, etc. (A30). Graphics, images, and videos are displayed in an accessible format (A31). The same content is provided in different formats (HTML, PDF, Word, audio, video, etc.) (A32).
E-learning Quality Masoumi & Lindstrom—2012 [59]	Institutional factors (D69) Instructional design factors (D70). Evaluation factors (D71). Technological factors (D72). pedagogical factors. (D73) Student support factors. (D74). Support factors for teachers (D75).	Learning materials must be reasonable and appropriately accessible to students whenever they wish. (A33). Access to learning materials should be granted to students with disabilities (e.g., ’screen readers’ for those with limited vision, ‘text narration’ for those with little or no hearing, etc.) (A34). The e-learning platform must meet adequate bandwidth demands (e.g., materials are accessible without long delays). (A35). The online learning platform should provide students with a user-friendly, evident and predictable environment, considering: (a) developing a user-friendly e-learning environment, (b) cognitive load through proper use of color and design, (c) helping users visually through the appropriate use of text, images, audio, video, animation, graphics, etc., (d) standardized navigation in which users can find their way with a minimum of clicks (A36). In the design and use of e-learning environments, students’ needs, skills, and knowledge must be addressed and supported to meet their individual needs or preferences (A37). Various learning scenarios should be provided to support multiple learning styles and learning abilities (A38).
eMM (e-Learning Maturity Model) Marshall—2010 [49]	Learning (D76). Development (D77). Support (D78). Evaluation (D79). Organization (D80).	The courses are designed to help students with disabilities (A39). Courses are designed to support various learning styles and learning abilities (A40)
CAPEODL Khan—2005 [60]	Pedagogical (D81). Technological (D82). Interface design (D83). Evaluation (D84). Management (D85). Resource support (D86). Ethical (D87). Institutional (D88).	The platform interface design considers content design, navigation, accessibility, and usability criteria (A41).
Esvial Accessibility Accreditation Model—2013 [28]	Technology (D89). Training (D90). Instructional Design (D91). Services and support (D92).	Guarantee access to all recipients, considering (A42): Adapted media. Standard and open technology. Assistive technology. Correct labeling and marking. Compliance with legislation. Compliance with web accessibility standards Guarantee usability and navigability, considering (A43): Organization and homogeneous design. Intuitive environment. Navigation map/situational bar. Aids or support tools Application of usability and accessibility test results.
CALED—2010 [29]	Leadership and management style (D93). Policy and strategy (D94). People development (D95). Resources and alliances (D96). Recipients and educational processes (D97). Results of the recipients and educational processes (D98). People development results (D99). Company results (D100). Overall results (D101).	The student profile is studied, identified students with disabilities and the nature of the disability (auditory, visual, physical) (A44). Consider computer systems the interoperability, compatibility, usability, and objectives of the program (A45)
Colombian Accreditation Model—2013 [30]	Mission, vision, and institutional project of the program. (D102). Students. (D103). Teachers. (D104). Academic processes. (D105). Research and artistic and cultural creation. (D106). National and international visibility. (D107). Impact of graduates on the environment. (D108). Institutional welfare. (D109). Organization, administration, and management. (D110). Physical and financial resources. (D111).	No evidence
Mexico Accreditation Model—2018 [31]	Purpose of the program (D112). General conditions of the program (D113). Curriculum (D114). Comprehensive training activities (D115). Instructional Design and Course Management (D116). Entry to the program (D117). School career (D118). Exit from the program (D119). Student results (D120). Academic and support staff (D121). Infrastructure (D122). Support Services (D123)	Instructional Design and Course Management: The educational program must have current, accessible, sufficient, and suitable instructional materials for the educational model (A46). Usability: The platform used by the educational program must be friendly for all user profiles and must have a non-linear or hybrid navigation system, well-designed interaction areas, logos, and attractive content spaces (A47). The platform used by the educational program must offer permanent access alternatives to materials in formats that meet the needs of students (A48). The structure of access to the different tools and services in the courses must allow students to become familiar with the interface in a short time and to carry out their activities without difficulty (A49). Infrastructure to manage courses: The learning management system (LMS) must present the following essential characteristics: interactivity, flexibility, scalability, standardization, usability, functionality, and ubiquity. (A50)
Accreditation Model of Costa RICA—2021 [32]	Relationship with the context (D124). Resources (D125). Educational process (D126) Results (D127).	No evidence
PDPP Zhang & Cheng—2012 [51]	Planning (D128). Development (D129). Process (D130). Evaluation (D131).	No evidence
UNIQUe EFQUEL—2011 [33]	Learning and institutional context (D132). Learning Resources, (D133). Learning processes (D134).	Institutional accessibility policies (disability) also cover the ICT offers of the institution (A51). The available learning resources have been tested for use and corrected to overcome common technical problems (A52). The course creation and production tools can cover a variety of current formats and thoroughly take into account the principles of usability, accessibility, interoperability, and durability, aimed at facilitating the application in the course (A53).

). From the analysis of the data presented in Appendix A Table A2 and a deep study of each model and its dimensions, it was possible to group the 134 dimensions into 18 (see Appendix A

Table A3. Dimensions abstracted from the studied models.

Generalized Dimensions	Related Dimensions	Description
Assessment and continuous improvement	D6, D34, D35, D36, D54, D95, D70, D71, D79, D84, D98, D99, D100, D101, D107, D108, D127, D131	It is characterized by the constant evaluation process that can be maintained on the course or program, in which aspects are considered to determine the achievement of objectives, the effectiveness of learning and re-deeming students, evaluation of teaching performance, the sustainability of the course or program, the satisfaction not only of students and teachers but also of society as well as the impact, compliance with rules and regulations, seeking to identify the weaknesses that allow decision-making that permits constant improvement.
Technological infrastructure and equipment	D4, D13, D14, D17, D21, D33, D37, D41, D47, D61, D72, D82, D85, D89, D96, D111, D122, D133	It is characterized by aspects related to the technological infrastructure supporting the virtual education system, the LMS platform (learning manager system), and the equipment made available to users who need it. Infrastructure and equipment are conditioned to be accessible, robust, safe, and, continuous
Learning strategies	D10, D30, D38, D44, D50, D59, D65, D66, D73, D76, D81, D91, D105, D116, D126, D129, D132	It is characterized by the pedagogical aspects that revolve around the development of the course, seen from the teaching–learning methodology, the scenarios and resources used, the interactivity and use of tools/resources for interaction between student and teacher, dedication, and timely feedback by the teacher
Content and learning resources	D15, D26, D27, D29, D42, D43, D48, D64, D70, D77, D83, D86, D91, D116, D129	It is characterized by the adequate design of a course’s contents and learning resources, considering having clearly defined objectives and learning outcomes, learning activities and interaction with the student, complying with usability and accessibility standards
Student Support and Guidance	D19, D31, D58, D67, D74, D77, D78, D88, D103, D109, D118, D123, D133.	It is characterized by aspects related to student welfare concerning access to scholarships and financing, guidance and academic advice, tutorials, monitoring of students with attendance irregularities, and other student services.
Assistance and technical support	D5, D9, D20, D52, D53, D58, D75, D90, D92, D95, D130, D134	It is characterized by aspects that relate to training in e-learning skills for both students and teachers concerning the use of the LMS platform and emerging technologies, as well as factors related to assistance and technical support in using the platform and technological tools during the course development
Course information or academic program	D1, D22, D23, D24, D40, D49, D62, D102, D112, D114, D125, D128	It is characterized by the dissemination and publication of course information such as justification, objectives, study plan, entry, exit profile, and information on the teaching and technical staff.
Rules and regulations	D3, D12, D16, D22, D55, D57, D69, D80, D94, D113, D133	It is characterized by the existence of policies and regulations concerning virtual education, in aspects such as e-learning instructions and guidelines for both teachers and students, as well as management of student complaints, evaluation, qualification, and academic honesty
Institutional organization	D11, D39, D46, D56, D88, D93, D110, D126	Characterized by aspects that go beyond the course and focus on parts of a structure under which online training processes are managed
Teacher support	D52, D60, D75, D77, D78, D104	It is characterized by aspects related to administrative support in terms of working conditions, intellectual property, pedagogical support, and strategies that pursue the professional development of teachers
Economics and technological financing	D8, D69, D96, D111, D113, D128, D133	Characterized by aspects related to allocating economic resources for virtual education and their management.
e-assessment	D18, D32, D45, D68, D116	Characterized by the aspects that relate to the process of measuring the achievement of learning results of a course, considering that the evaluation is precise and consistent with the objectives of the course and with adequate scenarios, notified and provided feedback on time by the teacher, in correspondence to institutional qualification and evaluation policies and regulations
Admission	D2, D25, D117, D124	Characterized by the process carried out for the admission of a student to the program or course, considering aspects such as dissemination strategies, promotion, evaluation of knowledge and minimum necessary skills, registration, among others
Research and innovation	D69, D106, D126, D132	Characterized by aspects related to strategies and efforts in research and innovation in virtual education.
Teacher Profile	D28, D41, D63, D121	Characterized by assessing the skills that the person who teaches classes should have.
Link with society	D51, D118, D119, D132	Characterized by aspects related to student participation in the community.
Diversity	D87	Characterized by aspects related to social influence, cultural diversity, prejudices of diversity, accessibility to information, and legal aspects, which revolve around the actors of virtual education
Knowledge management	D7	Characterized by aspects that are oriented to the sharing/reuse of resources, as well as the existence of formats and processes to document lessons learned and actions concerning the evaluation of teaching performance.

), through an abstraction process that is based on the matches of each approach.

Concerning the accessibility criteria, the 53 measures identified have been grouped into eight characteristics, which categorize the initial criteria (see

Table 3. Accessibility criteria abstracted from the studied models.

Accessibility Features	Related Criteria	Description
Content accessibility	A8, A9, A13, A15, A16, A20, A23, A24, A25, A28, A29, A30, A31, A33, A35, A36, A39, A41, A42, A46, A52, A53	View from the accessibility of the contents, resources, course information, program, or LMS platform, in relation to compliance with web content accessibility standards
Training	A2	View as the training of students and teaching staff in the use of the platform and e-learning resources
Alternative content	A11, A17, A18, A32, A48	View from the institution’s availability to provide course information and other resources in alternative media for students who do not have permanent internet access.
Continuity of service and access to internet and ICTs	A3, A6, A26, A27, A50	Seen from the guarantee of continuous and uninterrupted access to the LMS platform and the possibility of making internet access or ICT resources available to students and teachers, such as loans from various locations, including outside the institution
Curriculum flexibility	A7, A10, A22, A37, A38, A40	It refers to aspects that enable a flexible, open, and inclusive curriculum, which allows flexible learning based on the needs and abilities of a student
Accessibility policies	A12, A44, A51	The institution, program, or course has defined accessibility policies
Assistive technologies	A34, A42	View from the availability of learning materials or assistive technology for students or teachers who require it
Usability	A1, A5, A14, A19, A21, A36, A41, A42, A43, A45, A47, A49, A50	View from aspects that are related to ease of use, adequate navigability, good design, among others that the virtual education platform presents

).

In response to RQ3, seven studies that address the mechanisms contemplated for the praxis of evaluation and accreditation of quality in accessible virtual education were identified. Evaluations are framed as a process that can be carried out as a set of phases, as detailed in

Table 4. Phases of the implementation process of quality assessment models in virtual training.

Model	Praxis (Steps Are Numbered Systematically)
Torres-Barzabal—2019 [27]	Definition of objectives and scope (C1). Explanation of the criteria and reference of the model (C2). Review of supporting documentation/evidence of compliance with the indicators (C3). Online and management analysis (C4). Drafting the audit report and non-conformities for each topic and presentation of the audit report (C5).
Esvial Accessibility Accreditation Model—2013 [28]. Two-phase model.	Self-assessment: Organization and composition of the self-assessment team (C6). Analysis of the self-assessment and organization model (C7) Execution of the self-assessment process (C8). External evaluation: Constitution of the external evaluation committee (C9). External evaluation (C10). Issuance of the course certification opinion (C11).
Caled—2010 [29]	Application for evaluation for certification purposes (C12). Training of the self-assessment team (C13). Self-assessment process (C14). Constitution of the external evaluation committee (C15). External evaluation process (C16) Issuance of opinion (C17). Development of the improvement plan (C18)
Colombia Accreditation Model—2013 [30]	Compliance with initial conditions (institutional legality) (C19). Self-assessment (review, reflection, and intervention in programs) (C20). External evaluation (C21), Final evaluation and issuance of opinion (C22) Public recognition of quality (C23).
Mexico Accreditation Model—2018 [31]	A formal request for a function or institution to be evaluated (C24). Preparation of the self-assessment (C25) On-site visit by a commission of external academic peers (C26). Preparation of the final report of the visit (C27). Opinion and granting of the institution’s accreditation or functions (C28). Delivery of the evaluation report of the accreditation to the institution or the functions to the person in charge of the IES together with considerations of improvements or opportunities to be attended in a determined time (C29).
Accreditation Model of Costa RICA—2011 [32]	Information, motivation, and internal awareness in the institution (C30). Self-assessment to identify the strengths and weaknesses that can be improved (C31). External evaluation, made up of validation by international external academic peers and an on-site and direct assessment (C32). Accreditation (C33) and continuous improvement with the qualification of granting or not the official accreditation, as well as the conditions in which it is granted (C34).
UNIQUe EFQUEL—2011 [33]	Request to start the certification process (C35). Eligibility qualification (C36). Self-assessment (C37). Peer review (C38). Certification (C39). Continuous improvement (C40).

.

From the detail shown in Table 4, 40 phases or sub-processes defined in the different models are identified, observing typical steps (some of them with different names and others that can be related). After analyzing the elements of the models, seven phases were identified for the implementation of a quality evaluation model in virtual education: (1) beginning of the evaluation, (2) training, (3) self-evaluation, (4) external evaluation, (5) opinion, (6) continuous improvement and (7) public recognition of certification or accreditation (see

Table 5. Possible mechanisms or phases in quality assessment processes in virtual education.

Phase or Sub-Process	Sub-Processes Included or Related	Description
Start of the evaluation	C1, C12, C19, C24, C35, C36	It constitutes the formalization phase of the need for evaluation by the institution before a certification or accreditation body or the same institution and its authorities
2. Model Training	C2, C7, C13, C30	It includes aspects related to the socialization of the model within the institution and training for self-assessment or self-diagnosis
3. Self-assessment or self-diagnosis	C3, C6, C8, C14, C20, C25, C31, C37	This corresponds to the preliminary self-assessment carried out, altogether with the preparation of a detailed report on compliance with the requirements of the quality criteria established in the model
4. External evaluation	C4, C9, C10, C15, C16, C21, C26, C32, C38	It constitutes the on-site visit to the institution evaluated by a group of peer evaluators (experts) to determine the extent of compliance with the quality standards of the model through observation and verification of the facts declared in the self-assessment report
5. Certification or accreditation opinion	C5, C11, C17, C22, C27, C28, C33, C39	It constitutes the declaration of Certification or Acreditation of the institution, program, or course evaluated, accompanied by a report of the findings and in cases the conditions or actions susceptible of improvement with which the certification is granted
6. Continuous improvement	C18, C29, C34, C40	It refers to the continuous improvement thread to which the institution is committed based on the specific recommendations under which the certification or accreditation was granted, including an improvement plan proposed by the institution
7. Public recognition	C23	It includes public recognition of the quality achieved by the institution

).

2.2. Model Design

The construction of the model was based on four steps. First, a draft evaluation model consisting of 18 dimensions was built. Second, this model was in a panel of experts in two online meetings; they established the initial structure of the model composed of: (a) Dimensions, (b) Standards, (c.1) Requirements and (c.2) Evidence (see Figure 2). They also grouped 18 into 4 dimensions: (a) Organization, (b) Students, (c) Teaching, and (d) Infrastructure; 16 standards, 48 requirements, and 63 pieces of evidence (see

Table 6. Summary structure of the proposed model.

Dimension	Standards	Requirements	Evidence
Organization	5	16	19
Student body	3	9	13
Teaching	5	14	19
Infrastructure	3	9	12
Total	16	48	63

). The first meeting lasted three hours, the construction of the model was carried out while the second meeting lasted two hours, the revision and adjustments were done. Third, the proposal was sent for a peer-reviewed process. Finally, the adjustments were made according to the observations presented.

The panel experts (n = 3) were selected based on their professional experience. The inclusion criteria were: (a) being a university professor, (b) having participated in self-assessment or accreditation processes, and (c) having expertise in virtual education or accessibility. Similar criteria applied for the peer reviewed process.

2.3. Design of the Guide

The construction of the self-assessment guide was based on two stages. The first covered the definition of the self-assessment methodology or application of the proposed model; the second corresponded to the preparation of the document.

The first stage was based on three steps. The seven phases identified in the literature review (Table 5) were analyzed in detail in the first step. The applicability or correspondence of the phases for a self-assessment process was determined, discarding phases 5, “Certification or accreditation opinion” and 7, “Public recognition”, for being specific to certification or accreditation processes. In a second step, a panel of experts restructured the phases and defined the self-assessment process as being composed of (1) planning, (2) refinement of the model, (3) evaluation, (4) results, and (5) continued improvement. In each phase, a set of actions that pursue and guide an organized execution of the process was considered; the group of experts was the same as the one that participated in the design of the model. Finally, as a third step, the weighting and qualification matrix of the model was defined.

In the second stage, the guideline was prepared. Finally, it was submitted to review by a peer reader, then adjustments were made according to the suggestions.

3. Results

The guide is presented as a multipart, consisting of three sections: (a) a conceptual and theoretical framework that justifies and supports the construction of the guide, (b) a self-assessment model under a scheme structured in dimensions, standards, requirements and evidence of quality assurance, and (c) a model application methodology within a self-assessment process for a virtual education institution.

3.1. Self-Assessment Model

The proposed self-assessment model consists of 4 dimensions that can be measured independently:

▪

Organization: evaluation of the institution’s organization and general strategic actions that support the training process and permanent quality assurance that it must pursue.

▪

Students: evaluation of the actions that the institution promotes and applies for the benefit of students as training recipients.

▪

Teaching: evaluation of the actual virtual education process resulting from the construction of knowledge, educational innovation, and skills and abilities in the chair itself.

▪

Infrastructure: evaluation of the technological and technical support structure that enables the teaching–learning process.

Each dimension, in turn, is constituted and evaluated based on compliance with standards, which correspond to the reference points to be measured or valued and highlight the expected achievements and objectives that the institution can achieve.

Each standard, in turn, is made up of requirements seen as disaggregated criteria that guide compliance with a certain standard. Likewise, each contains a set of guiding information sources to facilitate its evaluation, allowing us to understand and appreciate the degree of compliance with the evaluated standard. These sources of information are documentary evidence that formally supports the self-assessment and external assessment, if applicable.

The resulting model contemplates four dimensions or evaluation areas, 16 standards, and 48 requirements or compliance criteria, distributed and organized as shown in Figure 3.

A summary of the model is presented in

Table 7. Summary of the model.

Dimension/Standard
1. Organization
	1.1. Organization: The institution has an organizational structure and a set of rules, policies, and regulations that support virtual education processes, highlighting accessibility and inclusion, as well as quality assurance and continuous improvement. 1.2. Information on the academic course or program: The public institution disseminates relevant information on the academic course or program in a clear and up-to-date manner that enables students to understand it fully. 1.3. Economy and technological financing: The institution has regulations and executes actions that guide the improvement and updating of the computer platform 1.4. Knowledge management: The institution has and applies regulations and procedures that guide the management of the knowledge generated within the different virtual education processes. 1.5. Research and innovation: The institution has regulations and executes actions that guide the improvement and updating of the computer platform that supports virtual education and other related processes.
2. Student body
	2.1. Student support: The institution applies regulations and procedures that seek comprehensive education and student well-being in the academic, personal-social, and psychological spheres, according to the student’s profile and their particular needs. 2.2. Admission: Regarding the admission process to the program/career or course, the institution considers strategies to pursue an adequate income without discrimination according to the student’s profile and needs. 2.3. Diversity and inclusion: The institution applies regulations and procedures that enable educational inclusion without discrimination around the different actors of virtual education.
3. Teaching
	3.1. Professor profile: The institution has competent professors who can teach classes within a virtual education program or course. 3.2. Teacher support: The institution has regulations and procedures for the benefit of teachers and their day-to-day teaching practice. 3.3. Learning content and resources: The institution applies regulations and procedures that pursue a flexible curricular design and an adequate learning content and resources design. 3.4. Learning strategies: The institution implements learning strategies that revolve around the development of the course, seen from the teaching–learning methodology, the scenarios and resources used, and the interactivity and use of tools/resources for interaction between student and teacher. 3.5. E-assessment: The institution applies regulations and procedures that seek to measure the achievement of learning results, considering an accurate and consistent evaluation with the course’s objectives and with appropriate scenarios
4. Infrastructure
	4.1. Technological infrastructure and equipment: The institution has a technological infrastructure that supports the virtual education process, the LMS platform, and technical equipment accessible to users who need it. Both the infrastructure and the equipment must provide a continuous service (24/7) and are conditioned to be accessible, robust, and safe. 4.2. Learning management platform: The institution has an accessible computer platform to support the virtual education process and administrative management. 4.3. Assistance and technical support: The institution guarantees assistance and training in e-learning skills for both students and teachers regarding the use of the LMS platform as well as other emerging technologies that support the training process

with a description of the 16 standards.

3.2. Implementation Methodology

The proposed self-assessment process constitutes an interactive and continuous process, which begins with a planning phase of the self-assessment activities, followed by the refinement or redefinition of the self-assessment model to the institutional reality and scope of application in the institution (institutional, program, course), continuing with the evaluation itself and ending with the analysis of results and the determination of strengths and weaknesses that give way to improvement proposals and their execution within a continuous improvement (see Figure 4)

Each phase involves a set of actions that are pursued and guide an organized execution of the self-assessment process. The phases are:

• Planning: This phase begins with the formalization of the need for self-assessment by the institution, formalization that, in addition to giving support to the process and its main actors, seeks the involvement and commitment of the members of the institution, being necessary to raise awareness of the process and its purpose. In this phase, the self-assessment team is formed, whose profile would refer to a group of professionals with experience in self-assessment processes and process management. Finally, the work plan is defined concerning the process.
• Model tuning or Refinement of the model: Its objective is to redefine or adjust the model to the institutional reality. An update of the model is sought, with (a) a possible inclusion/exclusion of requirements according to the objectives of the institution and the scope of evaluation (institutional, program, course), (b) inclusion of evidence that supports compliance with the standard and its requirements, (c) review and adjustment of the rating scale (weighting matrix) according to the value that the institution estimates according to the scope of the self-assessment and (d) determination of the assessment techniques to be applied and design instruments (evidence collection sheets, interview guides, etc.).
• Evaluation: Constitutes the evaluation execution based on previously defined techniques and instruments.
• Results: It constitutes the analysis and systematization of the results of the “Assessment” sub-process, which begins with the identification of the strengths and weaknesses of the institution in terms of virtual education and culminates with the preparation of the self-assessment report.
• Continuous improvement: It constitutes the phase aimed at minimizing the gap between the established quality standards and the level of compliance in practice, as well as maintaining the achievements obtained and guaranteeing that there is no evidence of a setback in the standards, also including other actions that enable the growth of the institution.

4. Discussion

Quality and accessible education for everybody could expand the scale of students who access it and, thus, support education for sustainable development (ESD) [34]. In this way, it is expected to achieve the sustainable development goals (SDGs) since no goal is attained without the education sector [34,35]. Equitable and greater access to quality education contributes significantly to the fourth SDG, an objective that through its 10 goals focuses on guaranteeing equitable and quality access for all children to primary and secondary education, as well as guaranteeing equitable access for all women and men to quality technical, vocational, and higher education, including university education, without discrimination, seeking to increase the number of people who have the necessary skills (technical and professional), to access employment and decent work [22].

This research aimed to propose a self-assessment guide for the quality of e-learning from accessibility. Accessibility is conceived of as an opportunity within education services and the use of ICT [36], which we started from a study referring to quality assurance in e-learning to build a solid knowledge base around the subject. In particular, the proposed self-assessment guide contemplates a set of actions that can contribute to the SDGs, these being: (a) ensure access and participate fully in the university for vulnerable and disadvantaged people, including people with disabilities, indigenous peoples and people with economic difficulties, (b) providing facilities that promote and encourage inclusivity in learning [7].

The literature review carried out in this study shows a growing interest in quality assurance in virtual education. For example, there are proposals for new models such as those described in [37,38] and proposals based on existing models, which result from adaptations focused on specific needs [27,39,40,41,42,43,44,45,46]. Countries such as Colombia [30], Costa Rica [32], Mexico [31], Ecuador [47], among others, have quality accreditation institutions based on their evaluation models; this is also true of international organizations, such as the European Union [14]. In addition, comparative analyses and compilations of known quality assessment models and standards relevant to the authors of certain studies can be identified in the literature [9,12,48,49,50,51]. Other studies have also aimed to propose aspects or recommendations for adaptations to new models [52,53,54].

During the literature review, it was noted that the scope of application or evaluation area differed between models. For example, there are models of institutional evaluation [33,37,38,41,42,55], others for the assessment of the program [3,9,39,56,57], or of a course [28,58,59], evaluation of the platform (LMS) [39], evaluation of teaching [27], or e-assessment [40,45]. This revealed that the quality assessment around training in virtual education can range from general to specific such as e-assessment. In this sense, the guide proposed in this study has flexible applicability at different levels (institutional, training program, particular course).

Likewise, based on the application approaches identified in each model, it is evident that an evaluation model is not generalizable, since it often responds to sociocultural and particular needs of sectors, countries, or regions [30,31,32]; For example, four studies [27,33,44,60] presented proposals for models whose approach, according to the authors, is specific for application in universities or training institutions in the European Union. Likewise, four models [37,38,39,61] focused their criteria on the context of developing countries. In addition, only one study [29] considered the sociocultural context and particularities of the countries that make up the Latin American region and the Caribbean countries. Considering this fact, the guide includes a refinement phase of the model to the local context.

The diversity of approaches becomes explicit when the dimensions that make up a model are addressed. Of the 18 generalized dimensions (Appendix A Table A2) from the 134 dimensions identified (Appendix A Table A1), there is no common dimension in the 21 models studied. Dimensions such as “continuous improvement” and “technological infrastructure and equipment” are the most common with 18 coincidences in the analyzed models, followed by “learning strategies” with 17 coincidences, “content and learning resources” with 15, “student support and orientation” with 13, being able to consider them as the most important or relevant when evaluating the e-learning quality. On the other hand, dimensions such as “knowledge management” and “diversity” are unique, followed by “connection with society”, “professor profile”, “research and innovation,” and “admission” with four coincidences; this also limits the scope of the model. Although the model proposed in this study is structured in four evaluation areas, all 18 dimensions identified in the literature and their objectives have been included in the model, either as a standard or a requirement

Regarding the three-level structure proposed for the evaluation model (dimensions, standards and requirements, and evidence), it is considered easy to understand and is generally accepted in evaluation and accreditation models. For example, the SINAES (National Higher Education Accreditation System) of Costa Rica organizes its model in (a) Dimensions, (b) Components, (c.1) Criteria, and (c.2) Evidence [32]. In Ecuador, the CACES (Higher Education Quality Assurance Council) proposes a structure made up of (a) Evaluation axes, (b) Standards, (c.1) Fundamental elements, and (c.2) Sources of information. In addition, the Inter-institutional Committee for the Evaluation of Higher Education in Mexico (CIEES) organizes its model in (a) Evaluation Axis, (b) Category, and (c) Standards [31]. Likewise, the CNA (National Accreditation Council) of Colombia structures it in (a) Factors, (b) Characteristics, (c) Aspects to Evaluate [62]. The structures are similar between models since the difference is focused on the name and not on the definition.

Although accessibility is present in the analyzed models, its depth or scope is limited. Of the 21 selected models, 17 present accessibility criteria, with a lower correspondence than the generalized criteria (Table 3). For example, the standard “accessibility of content” is considered in 11 models, followed by “usability” with nine matches, “flexibility of the curriculum” in five models, “service continuity and access to the internet and ICTs,” and “alternative content” with four, “accessibility policies” with three, “assistive technologies” with two models and “training” with one model. This peculiarity does not occur in the model proposed in this study because it includes within its standards and requirements the seven criteria identified during the review.

Ensuring the quality and effectiveness of e-learning is the responsibility of training institutions and regular organizations (governments) [63]. In this sense, governments, through accreditation agencies, certify quality within a formal process; in addition, training institutions, also responsible for the quality of their service, promote self-assessment processes in adherence to assessment models that respond to a national or a particular reality. The proposed guide aims to solve the need for self-assessment from a specific perspective that refers to accessible virtual education.

Regarding limitations, the literature review could be biased by the language, the period of analysis, and the research keywords that could limit the searches. Moreover, the construction of the guide could also be biased by the background of the panel of experts and their experience in the evaluation processes of a single country such as Ecuador. To lessen this threat, an overview of the previous literature was done.

5. Conclusions

Because of the importance of education to all of the SDGs, it is essential to provide quality education that is accessible, affordable, and inclusive for all people.

The proposed self-assessment guide is not conceived for accreditation purposes, but rather its character is one of continuous improvement. In this way, each institution can have the freedom and autonomy to select and use the standards and requirements that it deems pertinent that contribute to the internal quality of its processes and the scope/scope of evaluation pursued according to its context.

In addition, it should be noted that self-assessment constitutes a participatory process from its character of continuous improvement. Therefore, it is necessary to involve the university or institutional community from the beginning, seeking to achieve a sense of belonging in each member.

Finally, when considering the diversity of quality models studied, it was noted that a diagnosis or evaluation differs according to the complexity or extension of the model/standard and the particularities of the institution, program, or course to be evaluated.

As future work, it is proposed to focus on studying the statistical properties of the instrument (reliability and validity) to provide empirical evidence of the use of the model and the guidelines proposed. So too, the implementation and validation of the guidelines will be carried out.