1. Introduction
Quality has become a concept of essential importance in modern societies [
1,
2,
3]. Any organized social activity can be optimized, and this optimization occurs through the creation of quality evaluation mechanisms. These mechanisms allow the control of the efficacy, efficiency, functionality and reliability of the evaluated process, and this, by definition, is the basis for preserving and improving quality in any activity [
4]. At present, quality control systems are applied, in general, in all relevant areas of society: government programs, business activity, health processes, urban development, transportation, agriculture or food, among others [
5]. In recent years, there is no area of social action that has remained outside of quality control systems, and, of course, the field of education is no exception.
There are many studies within the educational context that have addressed the issue of quality control in recent years [
6,
7,
8]. According to Mateo [
1], this growing interest in quality control in the educational field is determined by the development of a new management paradigm in which four principles are highlighted:
Principle of purpose: Educational actions pursue the attainment of previously defined objectives, both at the operational level and at the strategic level;
Principle of accountability: All elements or agents of the system must be audited to evaluate the degree of attainment achieved in the objectives preliminarily set;
Principle of subsidiarity: Although decisions should initially be made at the same level in which they will be applied, there is the possibility of transferring decision-making to a higher level with strategic competencies; and
Principle of self-organization and development: It is understood that the system is not static and, consequently, that agents have the obligation to manage themselves efficiently to face future changes.
Although research on quality has addressed this concept in different aspects of the educational system [
9,
10,
11], it is within the context of higher education that this reality has been explored most extensively [
12,
13,
14,
15]. Ruiz Carrascosa [
16] notes that the importance of the university as a key service in society coupled with the strong investment of funds that it requires intensifies the concern regarding the quality control of the service. In the same vein, Sierra Sánchez [
3] also notes that quality control has become one of the great challenges of university management in the 21st century.
However, measuring the quality of a service with an intangible nature, as in the case of education, is not an easy task. The concept of quality in higher education can have multiple connotations, approaches or meanings, and this is reflected in the literature. Among the most common approaches, three stand out: those that focus on the idea of service, those that explore quality from the perspective of the student body, and those that approach this concept from the perspective of the teaching staff (see
Table 1).
Gil Edo, Roca Puig and Camisón Zornoza [
17], in their study of the customer-oriented quality of service models in public universities, highlight seven determining traits: (1) the technical dimension of the faculty, (2) the functional dimension of the faculty, (3) the accessibility and academic structure, (4) the attention of the service personnel, (5) the tangible and visible aspects of the facilities, (6) the visible aspects of the staff, and (7) the existence of complementary services (restoration, reprography, et al.).
In the same line of the service-based approach, Veciana Vergés and Capelleras i Segura [
18] emphasize the importance of four relevant aspects when defining quality in the area of higher education: (1) the attitude and competence of teachers, (2) curriculum content, (3) equipment and facilities, and (4) the organizational aspect in the institution.
Resino Blázquez, Chamizo González, Cano Montero and Gutiérrez Broncano [
2], in their work on quality indicators that determine student satisfaction, highlight three dimensions: (1) facilities and resources (library services, transportation, etc.), (2) academic aspects (teaching, reputation of degrees, etc.), and (3) social aspects (sports activities, exchange programs, etc.).
Similarly, a study by Alvarado Lagunas et al. [
6] on the quality of a university from the student perspective reflects the existence of four critical aspects to consider: (1) physical infrastructure, (2) the teaching staff, (3) the teaching materials, and (4) the comprehensive development of the student.
Also adopting the approach of the student’s perspective, the research by González López [
13] on the factors that determine quality in higher education indicates the existence of up to 13 elements: (1) competencies training, (2) the development of skills to access the labor market, (3) the development of critical thinking, (4) mechanisms of institutional evaluation (teachers, resource management, etc.), (5) services available to students, (6) functioning of governing and representative bodies, (7) student involvement in institutional objectives, (8) optimal professional specialization, (9) satisfaction of students with their personal performance, (10) the existence of associative movements, (11) availability and access to academic information, (12) the provision of supplementary training, and (13) counseling on career opportunities.
Álvarez Rojo, García Jiménez and Gil Flores [
19], from discussion groups with teachers, indicate that quality in a university is defined by the interaction of four main variables: (1) the profession and teaching skills, (2) the art and vocation of teaching, (3) structural and social conditions (administrative processes, work opportunities, physical conditions or group size), and (4) the management of the dilemmas and paradoxes inherent to the university environment (research vs. teaching, innovation vs. inertia).
However, any of the previous works, regardless of the approach taken by researchers (a service-based approach, student perspective-based or teacher perspective-based), maintain a similarity: the indelible mark of the functional dimension of teachers as an essential element of quality in higher education. The control of teaching quality is not only a recurring element and cornerstone in all research of this scientific body, but it is also an aspect that sometimes overlaps with the concept of quality in university teaching in its broadest sense.
1.1. Quality Control in Teaching
Quality control in teaching has a dual purpose: on the one hand, formative; on the other hand, summative [
20,
21]. The formative purpose aims to obtain information on the weaknesses and strengths of the teacher with the ultimate aim of improving teaching [
22]. The summative purpose, for its part, is that the information collected serves as a support for decision-making regarding the professional accreditation of teachers [
23]. This dual purpose makes the quality control systems in teaching fundamental guarantors of quality in the field of higher education [
24].
Paradoxically, despite the important role played by quality control systems, they have suffered an alarming immobility over the years. The review of the literature by the authors reveals that, when the first quality control systems in teaching began to appear [
25], the same measurement pattern keeps repeating itself. This pattern is the implementation of student satisfaction surveys [
26].
These surveys gather the degree of agreement or disagreement of the student with a series of statements related to the teacher’s performance, generating feedback that is critical to satisfy the dual purpose of the evaluation. The degree of agreement was represented, generally, by responding to questions with Likert-type scales with between five and seven response levels [
27,
28]. In recent decades, this pattern of quality control has only experienced slight variations. One of these variations is one which concerns the delivery of the surveys, as, in 1990, delivery began its transition from paper surveys to systems with online questionnaires through the Internet [
29].
Although the pattern of teacher quality control through surveys of student satisfaction has proven to be reasonably efficient [
30,
31], the system is not free of limitations. This pattern addresses the psychometric challenges of satisfaction surveys: reliability, validity [
32,
33], leniency error [
34] or the halo effect [
35]. Additionally, these surveys are subject to the influence of different bias variables, such as teacher gender [
36], age [
37], size of the group [
38], or grades expected by the student [
39]. To all of the above must also be added the growing problem of low student participation.
1.2. The Problem of Response Rates for Determining Quality Control in Teaching
The low response rates of students have become one of the great threats to current quality control systems. The first consequence of these low participation rates is the overemphasis of several of the inconsistencies inherent in this quality control system [
40]. The reduced response rates can increase the leniency error and the halo effect and even increase the influence of biasing variables in the evaluation. Likewise, when participation rates are excessively low, the information collected is not very representative; this compromises the significance of the results, affects the psychometric measurement and, consequently, makes it difficult to infer real conclusions about the quality of the work of the teacher [
40].
This problem of participation has not only been corrected over time but has been aggravated by the implementation of questionnaires delivered online. There are many studies that show how response rates in teacher quality evaluation processes are lower when surveys are delivered online instead of on paper and in person (see
Table 2).
Among the aspects that cause low response rates in these surveys is the lack of knowledge that the student has about the objective and purpose of the evaluation as well as the indiscernible impact of this exercise on teaching in the eyes of the student, especially in the short term. However, in regard to online surveys, the main cause of these low participation rates is the lack of anonymity or confidentiality perceived by the student when filling out the questionnaire [
29,
54,
55]. This lack of confidence in the confidentiality of information is generated when, throughout the process, the student is forced to enter their credentials to access the online platform on which the survey is presented. Even when the identity of the student is irrelevant in the process and anonymity is guaranteed, this situation sows significant doubts in the student.
However, despite the misgivings that these low participation rates pose to universities, the advantages of online surveys, in terms of management, are so numerous that the transition from paper to an online questionnaire is an indisputable need. Online surveys eliminate the costs of printing, distribution, collection, scanning, data transcription and even physical storage of forms, significantly reducing the workload throughout the process [
46,
56].
Given that the implementation of online surveys in teacher quality control systems is such a consummate and true reality, as are their low participation rates, many institutions have adopted different types of strategies to increase response rates. Among these strategies, the following stand out: the use of reminders, the granting of extra credit and rewards in the form of coupons, or early access to grades [
41,
45,
57,
58].
However, the application of this type of measure while serving to improve response rates also entails significant drawbacks. The use of teacher quality control systems that use surveys related to obtaining incentives contributes to making the evaluation process a mechanical and obligatory task for the student to achieve the promised benefit. This causes the student to sometimes provide random answers and even answers without reading the statements [
59]. This situation compromises once again the significance of the information collected, thus distorting the objective of the process and making it impossible to use the results to satisfy the stated purposes, whether formative or summative.
1.3. Objectives
The panorama of quality control systems in teaching described above, with the exception of the slight variations referenced on the implementation of online surveys and the introduction of incentives for completion, has remained practically static for decades.
For instance, the use of online questionnaires actively delivered by email has not led to any variation in this static scenario. Studies such as those of Goodman, Anson and Belcheir [
58], Standish, Joines, Young and Gallagher [
60] or Boswell [
61] confirm that the use of email does not imply substantial improvements over other passive quality control systems that also use questionnaires delivered online.
This situation has generated a scenario of immobility that has not contributed to overcoming the limitations to which these quality control systems are subjected. These limitations are, therefore, a good example of the need for latent evolution in the field of quality control systems in the context of higher education. This research breaks away from the existing immobility by postulating a new strategy for the delivery of online questionnaires using mobile messaging services.
The present work, in addition to postulating a survey delivery system that improves the response rates of existing systems, is adapted to the social behaviors that young people currently exhibit. The authors, in view of other studies that corroborate the total integration of mobile devices among students in the university context [
62,
63,
64,
65], propose a delivery system which is sustainable and adapted to the evolution of youths’ social behavior at the current time.
Furthermore, according to the Sustainable Development Goals (SDG) plan set by the United Nations General Assembly in 2015 [
66], obtaining a quality education is the foundation of creating sustainable development, with the lack of adequate teachers being one of the reasons for the lack of quality education. Analyzing and improving the quality control systems in education through an efficient proposal which is not based on traditional paper questionnaires is in line with the SDG fourth goal on quality education, since one of the aims in this SDG plan is to increase the supply of qualified teachers.
The study developed by the authors, based on the implementation of the referenced strategy, raises two research questions:
- RQ1:
Are the response rates achieved in the delivery of quality control surveys for teaching efficiency using mobile messaging services greater than those obtained with traditional online delivery systems?
- RQ2:
Are there significant differences between the response rates achieved in the delivery of quality control surveys for teaching efficiency using mobile messaging services and those obtained with traditional online delivery systems?
4. Discussion
The participation rates achieved with the mobile messaging-supported delivery system considerably surpass the rates achieved in previous studies that also use online quality control systems for teacher evaluation [
45,
74]. It should also be noted that the aggregate response rate of 0.92 in group A is well above the average participation of 0.60 achieved in the studies by Chapman and Joines [
75] and Avery, Bryant, Mathios, Kang and Bell [
48] for online delivery systems.
The participation rates achieved through the delivery system via SMS also satisfy the response rate requirements per number of students recommended by Nulty [
71] based on the estimation formula proposed by Dillman [
76]. Although Dillman’s calculations are originally based on the premise that the probability that the student does not complete the survey is identical to the probability that he or she does (50:50), Nulty develops his estimates considering that the probability of not completing a survey is 70:30, supposing a stricter—but probably also more realistic—scenario. From there, Nulty’s work presents two scenarios of recommended participation rates: on the one hand, a scenario of “liberal” conditions and, on the other hand, a scenario of “stringent” conditions. For the first, the author assumes a sampling error of 10% and a confidence interval of 80%. For the second, a sampling error of 3% and a confidence interval of 95% are assumed. In the so-called “liberal” scenario, the expected response rates for groups, such as those considered in this study, between 27 and 61 students range from 0.25 to 0.58. In the strictest scenario, the response rates for these group sizes range from 0.90 to 0.97.
Considering the previously described scenarios,
Table 6 shows that the rates obtained for group A far exceed those required as a function of the size of the group under “liberal” conditions in the ten courses analyzed and that they moderately satisfy those estimated for “stringent” conditions. In the scenario of “stringent” conditions, the rates achieved with the messaging delivery system satisfy Nulty’s estimates in eight of the ten courses examined, leaving only the courses of programs 6 and 8 at 0.05 and 0.03 points, respectively, from the minimum recommended rate.
The delivery system described for group A, in line with Moss and Hendry [
77], eliminates password access to the questionnaire, making it unnecessary for the student to use their username and password in any step prior to completing the survey. The use of SMS to distribute the survey moves the response collection interface from the Student Services Portal to an open form on the student’s mobile device. As a consequence, as the information collection is not developed through the Student Services Portal and the institutional intranet, but in an open form, the student should not have to enter his/her credentials at any point in the process.
Limitations and Further Research
Even though the focus of the present research is not the lack of anonymity perceived during the survey but the improvement in the response rates, the authors, in line with previous studies [
29,
40,
54,
55], state that one of the reasons that could lead to this improvement in participation is the sense of privacy presumably experienced by the student in the mobile messaging-supported delivery system.
However, the present work does not explore this topic in depth, nor does it provide evidence in this regard. Therefore, the statement made by the authors regarding the influence of the perceived anonymity on the response rates should be considered only as an impression to be taken with caution. Although this issue has been explored more intensely by the studies previously referenced [
29,
40,
54,
55], further research is still needed to expand our understanding of this influence in the field of quality control systems in the university context.
5. Conclusions
The findings of this study respond positively and conclusively to the two research questions posed by the authors. The results show not only that the response rates obtained in both groups present significant differences, but also that the rates achieved using the SMS delivery of surveys are substantially improved compared with those obtained using the Student Services Portal delivery system.
Additionally, increases in response rates obtained with the SMS delivery system improve the representativity of the information collected, which increases the significance of the results and makes it possible for them to be used to satisfy the purposes—both formative and summative—of the quality control system with greater assurance. Considering the importance and significance of the feedback provided in the surveys of student satisfaction, using appropriate tools to increase response rates in these quality control systems is a critical issue.
The delivery of teacher quality surveys through mobile messaging services offers significant improvements in teacher quality control systems in terms of student participation. In light of the findings, the authors conclude that the delivery of SMS surveys represents an improved alternative to current teacher quality control systems, contributing to ending the immobility observed and opening new avenues of study.