Validation of the University Student Engagement Questionnaire in the Field of Education

Abstract

In this study, the Student Engagement Questionnaire (SEQ) developed by Kember and Leung (2009), was validated using a Spanish sample. This instrument is intended to provide a comprehensive evaluation of teaching and learning processes within university contexts, offering feedback to educators and institutions to support the improvement of these processes. To achieve this, the SEQ was administered to a sample of 561 university students in the field of education, drawn from two universities in Valencia: the Universidad de Valencia (32%), a public institution, and the Universidad Católica de Valencia (68%), a private one, during the start of the first quarter. The questionnaire is designed to assess both students’ competencies and teachers’ abilities to foster a conducive learning environment. Across its various versions, the SEQ has demonstrated considerable stability in its dimensions and the relationships between the variables it assesses. Confirmatory factor analysis has validated the proposed structure in the Spanish population. As a result, the SEQ serves as a suitable instrument for diagnosing and evaluating the development of university students’ competencies, as well as teachers’ capacity to structure teaching and assessment in a way that promotes a constructive and enriching learning environment. Furthermore, it assists in identifying areas for improvement in the methodological strategies employed, enabling educators to refine their pedagogical approaches to maximise the impact on student learning and development.

1. Introduction

Over the past few decades, society has been undergoing continuous and constant changes, driven in part by information and communication technologies (ICT) and the phenomenon of globalisation [1]. These changes have introduced new challenges and opportunities for progress and improvement across various social domains [2]. On the one hand, these transformations influence supply and demand within the labour market, requiring professionals equipped with the necessary skills to address both current and future challenges. On the other hand, they also impact societal transformation, prompting a need for socially competent and responsible citizens who can navigate this evolving environment [3,4,5,6].

In this uncertain and ever-changing society, it is essential to delve into a model that not only adapts to these shifts but is dynamic, focusing on the development of skills for academic and professional life, underpinned by values and respect for diversity [3]. According to [2], future citizens must be prepared for both existing jobs and those yet to emerge, for technologies not yet invented, and to solve problems that have yet to arise. Moreover, the impact of the pandemic has necessitated a re-evaluation of training methods, with an emphasis on fostering skills that can help individuals tackle new challenges [7].

For many years, debates regarding the alignment of university education with societal and labour demands have garnered significant interest within the academic community [8]. To meet these evolving socio-labour requirements, the educational system embarked on the creation of the European Higher Education Area (EHEA), leading to the development of the Tuning Project. This initiative emerged from the need to implement the processes initiated by the Bologna Declaration of 1999. Among other aspects, this declaration sought to align academic training more closely with the demands of the knowledge society, promoting a new teaching–learning model that adopts a comprehensive, student-centred approach aimed at skill development [9], that in a globalised and continuously changing society, came to complete the mastery of content [10].

The Tuning Project resulted in establishing that European universities should provide training in both specific and generic skills. Over time, specific skills training was incorporated into the various subjects of each degree. However, while generic skills were recognised as important for the holistic development of students, enabling lifelong learning and fostering higher employability and active, democratic citizenship [3,11], they were often developed indirectly through technical subjects, elective courses, or complementary training activities [6,12].

Given the increasing importance of these generic skills, it is now considered essential for students to be trained in both specific and generic skills, as without this dual focus, “there is a risk of training future professionals who are experts in their specific field but lack the ability to manage their personal and social circumstances” [5]. Furthermore, [13] it has been acknowledged that specific skills may become obsolete over time, whereas generic skills tend to be more enduring. Consequently, according to [14], training in generic skills is now regarded as a vital issue for the university educational community, as their “purpose is to respond to social, educational and labour market needs” [14]. The current context demands individuals who possess openness to cultural diversity, critical thinking, autonomous learning, empathy, flexibility, adaptability, and the ability to cooperate, communicate, and resolve conflicts, among other qualities [11,15]. Despite the significant attention, these skills have received from educational policymakers and stakeholders, and the efforts made by universities to include them as part of their educational objectives, expecting students to develop them throughout their academic journey, there remains a paucity of studies that can identify which aspects of the teaching–learning environment influence the development of these generic skills and the best methods for facilitating this process [12,15].

Research suggests that the success of skills development initiatives is partially dependent on the pedagogical methodologies and learning environments provided by educational institutions [16]. According to [12,17], fostering the development of generic skills requires the use of diverse teaching and learning methods, along with various pedagogical practices that allow for continuous and formative assessment of student learning. Furthermore, it is necessary to promote diverse forms of learning by offering students multiple opportunities to learn both inside and outside the classroom, as well as through extracurricular activities. Some studies indicate that generic skills are more effectively developed through specific pedagogical approaches, such as active learning methods and group activities [18,19,20]. In studies conducted by [17,21], it was found that the teaching–learning environment significantly influences students’ perceptions of the development of their generic skills.

For these reasons, high-quality instruments are needed to evaluate the teaching–learning process within the university context, in order to provide a solid diagnosis and propose measures for improvement to ensure educational success.

Although teaching and learning are closely interrelated, it is useful to describe these concepts separately, since teaching is designed to promote student learning [22].

The learning process itself can be conceptualised as an active, constructive, and meaningful process [23] through which individuals acquire or modify knowledge, abilities, behaviours, attitudes, and values through study, experience, instruction, reasoning, or observation. This process results in lasting changes in individuals, affecting their knowledge, abilities, and attitudes [22].

Although learning is an individual activity, it develops within a social and cultural context [24]. It is the outcome of cognitive processes that facilitate the assimilation and internalisation of new information—such as facts, concepts, procedures, and values—while enabling learners to construct meaningful and functional mental representations. These new representations enable learners to adopt new behaviours that can be applied in various contexts [25]. Learning extends beyond the memorisation of information; it also involves complex cognitive processes, including comprehension, application, analysis, synthesis, and evaluation [26].

The understanding of learning has evolved over time. This author [22] has identified three predominant paradigms that have shaped the learning process from the 20th century to the present day: learning as the acquisition or strengthening of responses (behaviourism, during the first half of the 20th century), learning as the acquisition of knowledge (cognitive psychology/information processing theory, in the 1950s and 1960s), and learning as the construction of knowledge (constructivism and socio-cognitive approaches, from the 1970s onwards). In contemporary approaches, the focus is placed on the learner, while the role of the teacher is redefined as that of a facilitator and mediator who supports the student in the autonomous construction of knowledge.

Adopting this perspective requires learners to develop and refine their learning strategies and self-regulatory mechanisms, enabling them to manage their own learning processes and thereby fostering greater autonomy and self-regulation [27]. This approach is particularly relevant in university contexts, where students, as adult learners, must cultivate the skills, capacities, and abilities necessary to adapt to a society that demands continuous learning.

Like the learning process, the teaching process is inherently multidimensional. It involves a series of actions undertaken by the educator, whose aim is to facilitate the acquisition of educational content—such as knowledge, habits, standards, abilities, techniques, and attitudes—by a student or group of students. These actions are carried out through the application of specific methods and resources, and tailored to defined objectives within a particular context so that students integrate the information both cognitively and behaviourally [22].

From a traditional perspective, this process may be viewed as a unidirectional transmission of knowledge, with the educator, as the holder of knowledge, imparting information to the student, who is perceived as a passive recipient [28]. In contrast, more contemporary approaches, such as constructivism and the socio-cognitive perspective, conceptualise teaching as a process of support, scaffolding, guidance, and facilitation. In this framework, the educator designs enriched learning environments and provides diverse opportunities, resources, and tools that enable students, with the support and mediation of the educator, to construct, organise, internalise, and integrate information meaningfully into their cognitive structures, resulting in the generation of knowledge. In this approach, students are considered active participants and constructors of knowledge [29]. This perspective requires the educator to possess a wide repertoire of skills and abilities in order to manage the teaching process effectively.

Moreover, the conception of the teaching process has evolved over time, influenced by advancements in learning theories. Since theories of teaching and instruction must be grounded in learning theories [30], the interpretation of teaching has shifted in line with theoretical developments and research findings in instructional processes. As a result, to be an effective educator, a diverse set of teaching abilities, capacities, and skills must be acquired, in addition to certain personal attributes.

Within the university context, a substantial body of scientific literature has contributed to defining the repertoire of abilities, capacities, and skills associated with effective teaching. These skills are generally grouped into several key areas: planning of teaching, management of teaching–learning activities through the application of pedagogical methods adapted to specific objectives and contexts, assessment abilities, communication, interpersonal relationships, tutoring, classroom climate management, innovation, teamwork, and the use of information and communication technologies (ICTs).

Given the complexity of these processes, as mentioned above, it is practically impossible to conduct a fully comprehensive evaluation that encompasses all dimensions and components. For this reason, the scientific literature has increasingly focused on the development of conceptual constructs that provide relevant information on the most significant aspects of these processes.

In relation to the learning process, explanatory constructs have been developed that address key elements such as the ability and techniques to study, learning strategies, approaches and styles, student abilities, skills, and academic performance. Similarly, in the field of teaching, various teaching styles, pedagogical models, and orientations have been outlined, in addition to teaching abilities, capabilities, and skills.

In this study, we propose the examination of the integrated evaluation of teaching–learning processes within the university context using a single instrument, with students serving as the primary source of information. The objective is to determine the extent to which the teaching–learning environment designed by educators influences the development of students’ generic skills.

To achieve this, a range of instruments has been developed, each addressing different aspects of the teaching and learning process. Among these, this study focuses on the Student Engagement Questionnaire (SEQ), whose purpose is to assess the effectiveness of teaching based on students’ perceptions and the development of the skills they achieve through their learning.

It should be noted that this instrument has already been validated by [31], however, this study aims to draw new conclusions by considering a different sample. Moreover, we aim to contrast our findings with those of other studies and continue providing insights regarding its validity within the Spanish context.

2. Student Engagement Questionnaire (SEQ)

The goal of Kember and his collaborators in developing the Student Engagement Questionnaire (SEQ) was to create a diagnostic tool rigorous enough to identify both strengths and weaknesses within the teaching–learning process. This tool also aimed to provide feedback to educators and institutions to enhance the quality of their educational processes. Moreover, it was crucial that the instrument aligned with existing research on the teaching–learning environment and effectively captured the key constructs necessary for evaluation.

The questionnaire was to cover two major topics:

-

The analysis of the skills that university students must acquire during the learning process.

-

The evaluation of the learning environment that educators establish in the classroom to facilitate the acquisition of these skills.

To design the range of skills, Kember collaborated with a panel of professors from various faculties, who developed lists of attributes required of graduates from each academic discipline. The results from this process were compared with previous studies in the scientific literature, such as those reviewed by [32], which revealed a high degree of consistency between both. Based on these comparisons, item banks were created to assess the selected skills, which were refined and reduced through successive validation iterations. This process constitutes the first part of the questionnaire.

The second section of the questionnaire was designed to evaluate the key elements of the teaching–learning environment that educators organise to support effective learning. Initially, the constructs included in this section were based on existing literature on the student experience within the educational process. In early versions of the instrument, a limited number of scales were included, as it was assumed that evaluations by teachers were already sufficiently comprehensive. However, in later versions, the number of scales in this second section was expanded.

Throughout the successive validation processes, different versions of the questionnaire have incorporated a variable number of dimensions, scales, variables, or aspects within the two key areas mentioned earlier (student capacities and the teaching–learning environment), all of which have remained consistent across versions.

Finally, the structure of the questionnaire comprises five dimensions, two relating to capacities—Intellectual Capacities and Teamwork—and three relating to the teaching–learning environment—Teaching, the Teacher–Student Relationship, and the Relationship Among Students. Additionally, the questionnaire includes 15 observed variables, grouped into scales that are integrated into the aforementioned dimensions, with seven corresponding to students and eight to the learning environment (see Table 1).

Table 1. Dimensions and variables of the SEQ questionnaire.

Student Capacity
Categories	Variables
Intellectual capacities	Critical thinking. Creative thinking. Self-managed learning.
Teamwork	Adaptability. Problem resolution. Interpersonal skills. Communications skills.
Teaching–learning environment variables
Teaching	Active learning. Teaching for understanding. Assessment. Curriculum coherence.
Teacher–student relationship	Teacher–student interaction. Feedback to help learning.
Relationship between students	Relationship with other students Cooperative learning.

In their model [19], the authors seek to examine the impact of the teaching–learning environment on the development of students’ capacities. According to the classification proposed by [19,33], the authors use nomological validity as evidence of the construct.

To achieve this, the eight scales describing the teaching–learning environment are considered latent indicators of the three previously mentioned dimensions: Teaching, the Teacher–Student Relationship, and the Relationship Among Students. These three constructs are correlated with one another, as indicated by [19,21,34].

Regarding the seven student competencies, which are grouped into two dimensions—Intellectual Capacities and Teamwork—the model establishes that Teamwork has an influence on Intellectual Capacities [19,21,34]. Furthermore, the proposed model demonstrates how the latent dimensions of the teaching–learning environment are linked to the constructs representing student capacities. Specifically, an influence of Teaching is observed on the two latent variables related to student capacities [19,21,34]. Likewise, there is a relationship of influence between the Relationship Among Students and Teamwork [19,21,34]. However, the relationship between the Relationship Among Students and Intellectual Capacities is indirect and mediated through Teamwork.

On the other hand, the authors did not identify a direct relationship between the Teacher–Student Relationship and the two latent variables representing student capacities [34]; instead, an indirect effect on these capacities is observed. The authors suggest that this relationship is interconnected with teaching–learning variables, serving as a prerequisite for effective teaching.

Finally, the model has been validated in subsequent research [20], presenting a consolidated theoretical structure regarding the dimensions and relationships between them, which have remained consistent over time.

The validation conducted by [31] within a Spanish population confirms the structure proposed by the original authors of the questionnaire, particularly in terms of the dimensions and relationships outlined in the model. For these reasons, we consider it appropriate to validate the questionnaire with university students in the field of education to further examine the stability of the proposed model.

3. Materials and Methods

A test validation design [35,36,37] was employed to corroborate the structure of the instrument.

3.1. Participants

The sample consisted of 561 university students from the field of education, attending two universities in the city of Valencia: one public, the Universidad de Valencia (32%), and one private, the Universidad Católica de Valencia (68%). Among the participants, 22.3% were male and 77.7% female, with an average age of 20.8 years. In terms of their academic degrees, 28.9% were pursuing a degree in Primary Education, 3.9% in Early Childhood Education, 21.4% a Dual Degree in Primary and Early Childhood Education, and 45.8% in Social Education. Regarding the year of study, 38.1% were in their first year, 26.6% in their second year, 26.2% in their third year, and 9.1% in their fourth year. The selection of participants was constructed through intentional non-probabilistic sampling, as the students were taught by lecturers who were part of the research team for this study. The criteria for selection included being enrolled in degree programmes related to education.

3.2. Instrument

The Student Engagement Questionnaire (SEQ), developed by [19] and translated into Spanish by [31], was used in this study. The questionnaire consists of 35 items and assesses 15 variables, 7 related to student competencies and 8 to the teaching–learning environment (see Table 1). It is a Likert-type scale with five response options, ranging from “strongly disagree” to “strongly agree”. The questionnaire is provided in Appendix A (see Table A1).

3.3. Procedure

The university students participating in the study completed the questionnaire at the beginning of the first quarter, via the virtual campus platform, after providing their informed consent.

3.4. Data Analysis

For the analysis, the statistical software packages SPSS and AMOS version 24.0 were used, allowing for various essential statistical calculations to determine the structure of the questionnaire.

For the exploratory analysis, the varimax rotation method was employed, alongside the Kaiser–Meyer–Olkin (KMO) and Bartlett tests, both of which were found to be suitable for conducting factor analysis using the Kaiser method.

In line with these recommendations, the Root Mean Square Error of Approximation (RMSEA) was selected as a key indicator, with values below 0.05 being considered acceptable. A 90% confidence interval was also considered. Additionally, the Comparative Fit Index (CFI) was calculated, yielding an acceptable value.

To assess the reliability of the instrument, Cronbach’s alpha was used [38], as proposed by [19], yielding a value of 0.94, which indicates excellent internal consistency.

4. Results

First, the descriptive statistics of the items are presented, followed by the analysis of the reliability and validation of the model.

4.1. Descriptive Statistics

The means for the various items ranged between 3.33 and 3.98 (see Table 2). The highest mean values were observed in items 5, 10, and 13, while the lowest were recorded for item 12. The standard deviation indicates that the students’ responses present a certain heterogeneity.

Table 2. Descriptive statistics.

Item	Mean	Dev.	N
1. In this subject I develop my capacity to judge alternative perspectives.	3.62	0.930	561
2. I have become more willing to consider alternative perspectives	3.71	0.871	561
3. I have been motivated to use my own initiative	3.76	0.887	561
4. I have been challenged to come up with new ideas	3.52	0.926	561
5. I feel I am able to take responsibility for my own learning	3.98	0.837	561
6. I have greater confidence in my capacity to continue learning	3.69	0.888	561
7. In this subject I have learned to be more adaptable	3.54	0.853	561
8. I have become more willing to change my perspective and accept new ideas.	3.62	0.898	561
9. I have improved my ability to use knowledge to solve problems in my field of study	3.55	0.885	561
10. I am able to offer information and different ideas to solve problems	3.81	0.842	561
11. I have developed my ability to communicate effectively with others	3.65	0.927	561
12. In this subject I have improved my ability to transmit ideas	3.33	0.933	561
13. I have learned to be an effective team member during group work	3.88	0.818	561
14. I feel confident dealing with a wide range of people	3.70	0.886	561
15. I feel confident using computer applications when necessary	3.68	1.032	561
16. I have learned more about using computers to present information	3.50	0.986	561
17. The teacher uses a variety of teaching methods	3.75	0.860	561
18. In this subject, students are given the opportunity to participate in classes	4.19	0.861	561
19. The teacher makes an effort to help understand the course material	4.19	0.725	561
20. The design of this subject helps students understand its contents	3.73	0.890	561
21. The explanations given by the teacher are useful when I have difficulties with the learning materials	3.88	0.823	561
22. There is enough feedback on activities and tasks to ensure that we learn from the work we do	3.78	0.844	561
23. The subject uses a variety of evaluation methods	3.62	0.804	561
24. To do well when being evaluated in this subject you need to have good analytical skills	3.65	0.834	561
25. The evaluation assesses our understanding of the key concepts in this subject	3.63	0.813	561
26. There is good communication between the teacher of this subject and their students	4.11	0.762	561
27. The teacher of this subject helps when asked	4.31	0.692	561
28. I am able to complete the programme requirements without feeling excessively stressed	3.36	0.952	561
29. The workload assigned in this subject is reasonable	3.73	0.770	561
30. I have a strong sense of belonging to my class group	3.31	1.013	561
31. I regularly work with peers in my classes	3.84	0.877	561
32. I have regularly discussed course ideas with other students outside of class	2.67	1.150	561
33. Discussing the course material with other students outside of class has helped me gain a better understanding of the subject	3.12	1.008	561
34. I can see how the subjects fit together to make a coherent study programme for my specialisation	3.36	0.888	561
35. The study programme for my specialisation is well-integrated	3.42	0.949	561

4.2. Internal Consistency Analysis

Through this analysis, the internal consistency of the instrument can be estimated; in this case, the Cronbach’s alpha obtained is 0.94, which indicates that its value is adequate (see Table 3).

Table 3. Cronbach’s alpha.

Cronbach’s Alpha	Number of Elements
0.941	35

4.3. Exploratory Factor Analysis

This statistical analysis allows us to explore the dimensions that make up the questionnaire. First, the varimax rotation method was applied, which aims to maximise factor loadings in order to determine the representativeness of the variables provided by the instrument and reduce them.

Following this, the KMO and Bartlett tests were applied to verify the correlations and adequacy between the variables. Both indicators are considered acceptable; the former is greater than 0.60, and Bartlett’s test of sphericity is significant (0.000), being less than 0.05 (see Table 4).

Table 4. KMO and Bartlett’s test.

Kaiser–Meyer–Olkin measure of sampling adequacy	0.942
Bartlett’s test of sphericity	Approx. Chi-squared 9088.286
gl	595
Sig.	0.000

These results allow for the application of factor analysis, based on the Kaiser method, enabling the use of the principal components method to synthesise the variables of the instrument. Following this, the total variance and the proportion of the variance explained by the common factors, referred to as “communality”, will be examined (see Table 5 and Table 6).

Table 5. Total variance.

	Initial Eigenvalues			Sum of Squared Loadings from the Extraction			Sum of Squared Loadings from the Rotation
Component	Total	% Variance	% Accumulated	Total	% Variance	% Accumulated	Total	% Variance	% Accumulated
1	12.144	34.698	34.698	12.144	34.698	34.698	3.458	9.879	9.879
2	2.221	6.345	41.043	2.221	6.345	41.043	3.240	9.257	19.135
3	1.941	4.687	45.730	1.941	4.687	45.730	1.970	5.628	24.764
4	1.893	4.266	49.997	1.893	4.266	49.997	1.725	4.929	29.693
5	1.841	3.545	53.542	1.841	3.545	53.542	1.683	4.809	34.503
6	1.760	3.315	56.857	1.760	3.315	56.857	1.679	4.796	39.299
7	1.662	2.750	59.607	1.662	2.750	59.607	1.535	4.385	43.684
8	1.629	2.655	62.262	1.629	2.655	62.262	1.521	4.347	48.030
9	1.555	2.442	64.705	1.555	2.442	64.705	1.521	4.345	52.375
10	1.502	2.292	66.997	1.502	2.292	66.997	1.504	4.296	56.672
11	1.466	2.189	69.186	1.466	2.189	69.186	1.476	4.216	60.888
12	1.435	2.099	71.285	1.435	2.099	71.285	1.467	4.191	65.079
13	1.317	2.050	73.335	1.317	2.050	73.335	1.438	4.109	69.188
14	1.250	1.858	75.192	1.250	1.858	75.192	1.246	3.561	72.749
15	1.126	1.788	76.980	1.126	1.788	76.980	1.228	3.508	76.257
16	1.007	1.679	78.659	1.007	1.679	78.659	0.841	2.402	78.659
17	0.970	1.628	80.287
18	0.828	1.509	81.797
19	0.709	1.453	83.249
20	0.693	1.409	84.658
21	0.687	1.392	86.051
22	0.556	1.302	87.352
23	0.553	1.293	88.645
24	0.426	1.218	89.863
25	0.402	1.148	91.011
26	0.384	1.097	92.107
27	0.374	1.068	93.176
28	0.354	1.011	94.186
29	0.349	0.999	95.185
30	0.313	0.896	96.080
31	0.309	0.882	96.963
32	0.292	0.834	97.797
33	0.283	0.809	98.606
34	0.248	0.709	99.315
35	0.240	0.685	100.000

Table 6. Communalities.

Component	Initial	Extraction
1	1.000	0.805
2	1.000	0.770
3	1.000	0.731
4	1.000	0.848
5	1.000	0.907
6	1.000	0.700
7	1.000	0.781
8	1.000	0.740
9	1.000	0.711
10	1.000	0.835
11	1.000	0.753
12	1.000	0.686
13	1.000	0.819
14	1.000	0.907
15	1.000	0.873
16	1.000	0.795
17	1.000	0.705
18	1.000	0.745
19	1.000	0.741
20	1.000	0.758
21	1.000	0.743
22	1.000	0.720
23	1.000	0.795
24	1.000	0.813
25	1.000	0.740
26	1.000	0.777
27	1.000	0.790
28	1.000	0.807
29	1.000	0.803
30	1.000	0.767
31	1.000	0.851
32	1.000	0.844
33	1.000	0.787
34	1.000	0.835
35	1.000	0.844

The explained variance reveals a total of 16 factors with Eigenvalues greater than one, representing the main dimensions of this questionnaire. For instance, the first component accounts for 9.879% of the variance of the original data, which makes it a principal component as it exceeds the expected 50% threshold in this type of analysis (see Table 5).

Regarding the proportion of variance explained by the common factors in a variable, it is observed that the model is able to account for 80.5% of the variability of component 1, and similarly for the rest of the components (see Table 6).

Next, the items that make up each of the extracted components or factors will be analysed. This can be seen in Table 7.

Table 7. Rotated component matrix.

Component	Items	Factor Loadings
Component 1	26	0.81
	27	0.84
Component 2	1	0.73
	2	0.75
Component 3	28	0.34
	29	0.42
	34	0.80
	35	0.82
Component 4	32	0.90
	33	0.80
Component 5	9	0.38
	10	0.80
Component 6	3	0.56
	4	0.82
Component 7	23	0.75
	24	0.80
	25	0.60
Component 8	30	0.69
	31	0.84
Component 9	13	0.76
	14	0.33
Component 10	21	0.58
	22	0.38
Component 11	11	0.43
	12	0.52
Component 12	15	0.85
	16	0.63
Component 13	7	0.51
	8	0.56
Component 14	5	0.88
	6	0.37
Component 15	19	0.60
	20	0.66
Component 16	17	0.27
	18	0.54

4.4. Confirmatory Factor Analysis

In this type of analysis, the focus lies on determining the structure of the data using empirical criteria. With this, it is possible to confirm the components of the data and determine the role that each element plays in its total set, based on three fundamental questions: total variance explained by the factors, explained variance of each factor and saturation of the items in the factors.

To do so, data adjustments must be made, which allow the establishment of the bases for the described procedure. Currently, statistical software offer several indices for this purpose. Two of them are of particular interest for this study, which will be discussed below:

-

Bentler’s CFI (Comparative Fit Index), which is based on a previous index, called BFI, corrects to avoid taking values that are beyond the range of between 0 and 1. To do this, it uses different procedures and it is understood that its measurement must be around 0.95 to consider that the model fits the data adequately. In this study, a value of 0.937 was obtained, indicating an adequate fit of the model to the data (see Table 8).

Table 8. CFI.

Model	NFI Delta1	RFI rho1	IFI Delta2	TLI rho2	CFI
Default model	0.895	0.850	0.938	0.907	0.937
Saturated model	1.000		1.000		1.000
Independence model	0.000	0.000	0.000	0.000	0.000

-

The RMSEA facilitates the determination of the amount of variance not explained by the model per degree of freedom. To consider that there is a good fit, its value must be <0.05. This is enhanced if the 90% confidence interval (C.I.) is between 0 and 0.05. The value obtained is 0.044, which is less than 0.5, indicating a good fit. Similarly, the confidence interval is set at 0.41, which further supports this conclusion (see Table 9 and Figure 1).

Table 9. RMSEA.

Model	RMSEA	LO 90	HI 90	PCLOSE
Default model	0.044	0.041	0.048	0.031
Independence model	0.162	0.159	0.164	0.000

Figure 1. Structural model.

The value is 0.044, less than 0.5, so it is considered that there is a good fit. At the same time, the confidence interval is 0.41, which reinforces the above.

These questions construct the following structural model:

5. Discussion

By applying various statistical methods, we were able to establish a set of approaches to the Student Engagement Questionnaire (SEQ) and its feasibility. Firstly, we determined that the mean of each of the items is similar, ranging between 2.67 as the minimum point and 4.31 as the maximum point.

One of the items that denoted a lower mean was number 32, associated with cooperative learning, suggesting that there is minimal interaction between the students when discussing content, both in and out of class. In addition, there are other numbers related to this same question, such as 30, which suggests a lack of connection or identification with the group. Among the numbers that have a higher mean, those that focus on the teacher’s work are particularly notable, specifically items 18, 19, 26, and 27, which indicate a committed teacher attentive to students’ needs.

Even though these questions allow us to approach the operation of the instrument, they are not the object of this study. These findings indicate that the instrument demonstrates a certain homogeneity, which concurs with the results achieved in other studies [19,31]. This consistency reflects the internal structure of the questionnaire, whose set of items collectively address the same issue and brings with it results that can be interpreted as a holistic measure of the functioning of the teaching–learning processes and the elements that influence it: intellectual capacities; teamwork; teaching; teacher–student relationship and relationships between students, as a set of interconnected elements.

Likewise, it is worth noting another relationship with the aforementioned study [31], which also focused on the Spanish population. The means obtained in this study closely resemble those reported in the Kember and Leung studies [19,21,34], with slight variations, so that there seems to be some consistency, per se, in the Student Engagement Questionnaire (SEQ) in terms of its structure and results.

Secondly, Cronbach’s alpha shows high reliability, reinforcing the notion that the instrument is dependable and well-designed. Therefore, the Student Engagement Questionnaire (SEQ) applied to a Spanish population shows excellent internal consistency, and its intended purpose is effectively fulfilled. Furthermore, it should be noted that the achieved measure is higher than that indicated by the authors [19,21,34] and similar to that achieved in Spain [31], where there appears to be a higher index.

Thirdly, the component analysis carried out allows us to determine that this questionnaire has 16 main factors, which indicates that this is the number of elements that can be measured from its structure (see Table 10). These are the following, all associated with the teaching and learning process:

Table 10. 16 factors.

Component	Factor
Component 1	Communication between the student and the teacher.
Component 2	Stimulation of critical thinking.
Component 3	Workload and time management according to the study programme.
Component 4	Cooperation between students.
Component 5	Acquisition of tools to solve problems.
Component 6	Strengthening student initiative.
Component 7	Teacher evaluation methods and assessments.
Component 8	Sense of belonging and teamwork.
Component 9	Integration in teams and appreciation of diversity.
Component 10	Teacher explanations and feedback on the activities.
Component 11	Development of assertive communication and transmission of information.
Component 12	Effective use of IT.
Component 13	Training of ability to adapt.
Component 14	Strengthening of self-confidence and accountability in learning.
Component 15	Teacher support and subject design.
Component 16	Use of teaching methods by the teacher and creation of participatory spaces.

It is worth mentioning here that the number of components identified and classified in this study is greater than the number proposed in the original study [19,21,31,34], which suggested a total of 15 factors. This study yielded one more additional factor, and also categorised the others more precisely, based on the observed factor loadings, shown in the previous table. This factor is referred to as the use of teaching methods by the teacher and the creation of spaces for participation. This aligns with other research [39,40] that explores the influence of teaching methodologies on the teaching–learning process.

In addition, it is important to consider the results obtained from the confirmatory factor analysis, which determine that, when applied, the responses obtained have sufficient reliability to be considered adequate, in such a way that they would be a sample of the 16 components already mentioned. They are analysed first in isolation, considering the items linked to each of them, and then holistically, as a sample of the willingness of students and teachers towards the teaching and learning process.

Therefore, these results allow us to establish that the Student Engagement Questionnaire (SEQ) is an instrument that can be used to effectively assess its intended objective within the Spanish context, which is the willingness of students and teachers to engage in the teaching–learning process, based on the dimensions it contemplates and for two purposes:

• Diagnosis of students’ capacities at the beginning of the course, at the end of a specific timeframe, before or after an intervention, etc.
• Teacher evaluation during an academic period, to establish new guidelines to follow, etc.

Thanks to the test’s reliability, a wide range of procedures can be performed. For example, undertaking intervention processes that improve students’ attitudes towards the teaching and learning process, either comprehensively or by focusing on certain dimensions that had a lower mean. It is also possible to build a general overview of students’ abilities at the end of specific timeframes and at key moments, such as halfway through the degree or when the academic workload has been completed.

Similarly, considering the reliability of the Student Engagement Questionnaire (SEQ), assessing the teaching staff’s capabilities is also possible, in turn establishing strategies to ensure their ongoing training, proper treatment of students, and regular updates of teaching materials, resources, techniques, methods, and so forth.

Regardless, it is a dependable instrument for Higher Education, especially when there is a need to understand its functioning and suggest new and improved approaches, centred around its core component—the teaching and learning processes.

6. Conclusions

This study has analysed the reliability of the Student Engagement Questionnaire (SEQ), based on data collected from a sample comprising 561 university students enrolled in educational degree courses from two universities in the city of Valencia, a public one, the Universidad de Valencia (32%) and a private one, the Universidad Católica de Valencia (68%).

After a thorough examination from three different angles, namely internal consistency analysis, component analysis, and factor analysis, it has been determined that the Student Engagement Questionnaire (SEQ) is a well-constructed instrument whose reliability in the Spanish context is considered satisfactory. Therefore, it is an optimal test to analyse the structure of teaching and learning processes in the university setting, as its inherent composition has been demonstrated to effectively address the question posed.

Furthermore, the study found that the statistics obtained, such as Cronbach’s alpha, are superior to those obtained by the creators of the instrument themselves [19,21,34] and from another validation conducted in Spain [31]. This suggests a significantly high level of adaptation within the country, leading to favourable results upon administration, such that it has the potential to become a crucial instrument in Spanish institutions and a valuable tool for researchers, given its verified structure and scope.

One limitation of this study is the use of intentional non-probabilistic sampling; however, while the sample cannot be considered representative of the Spanish population, the variability in qualifications helps to mitigate this potential limitation. In any case, in future research, the model should be tested with larger samples.

Finally, the model could be expanded to include other constructs, such as learning strategies or learning approaches, to gain a better understanding of university students’ learning processes.