Student Acceptance of E-Learning during the COVID-19 Outbreak at Engineering Universities in Spain

Abstract

This article analyses students’ intention to use a particular e-learning technology (MS Teams) at university during the COVID-19 outbreak in Spain using the Unified Theory of Acceptance and Use of Technology (UTAUT). The model was refined through a qualitative analysis based on six focus group discussions with students from different engineering faculties in Madrid, Spain. A survey involving 346 undergraduate students was subsequently fed into the model. Structural Equation Modelling (SEM) and SMART PLS software were applied for data analysis. The results shed light on theoretical and practical implications. The model was validated by the data and displayed a high predictive ability. Social influence was found to have the greatest influence over students’ acceptance, followed by the professor’s role in shaping the perception of improvement. Facilitating conditions were found to be the least relevant factor, probably due to the particular context in which this study was conducted. A significant difference was found between the public and private institutions in terms of the importance of the perceived usefulness for the professor (this factor was more important for students’ acceptance at the public university). In order to improve its acceptance and use under the current scenario, it is thus important for universities wishing to introduce e-learning to focus on creating a positive social environment around the e-learning platform, for example, by using social networks or relying on testimonies by professionals who could confirm the interest of such a platform in a future work environment. Understanding professors’ perspective on the implementation of the platform is also of paramount importance. More research is also needed regarding context-related differences when analysing students’ acceptance of e-learning.

1. Introduction

The COVID-19 crisis has transformed the world around us, and the university is no stranger to this transformation. It is clear that COVID-19 has resulted in a major disruption in the higher education system, whose consequences are still to be fully understood [1,2,3,4]. As a recent study from UNESCO points out, due to sudden and long-lasting school lockouts all over the world, the major impact of COVID-19 on teaching and learning in higher education is the increase in online education, and the hybrid mode of teaching has become the most popular form [4]. Indeed, universities worldwide have been pushed to experiment with e-learning due to the restrictions during the pandemic outbreak [5,6,7,8,9]. As the uncertainty regarding future global emergencies is high, distant education and e-learning could become even more strategic to avoid the discontinuation of a basic public service, such as education.

The issues associated with the successful implementation of e-learning in pre-pandemic times are well documented in academic literature [10,11,12]. The shift to e-learning implies a cultural transformation, and students, as well as teachers all over the world, have struggled throughout the e-learning adoption process [2,13,14,15,16,17]. Particularly, failures in e-learning adoption have been reported due to a lack of preparation of the institution and its constituents, specificities of the regional context, as well as difficulties in adapting to innovative teaching and learning approaches [18,19,20,21]. Indeed, the implementation of e-learning technologies involves a shift of focus from the lecturer to the educational process and the student experience [22], which is considered a revolution by some authors [23]. Beyond ensuring access to public service during difficult times, e-learning has the potential to improve communication, collaboration, knowledge transfer, and training to enhance the value provided to both individuals and organisations [24]. This implies contributing to the shift from a passive model of information transmission to an active model where the individual is monitored, tracked, and analysed in order to develop the best training process for each particular person [25].

An important success factor for the implementation of an e-learning system is the incumbent actors’ willingness to accept and actively engage in using this system [26,27,28,29,30,31]. Understanding these actors’ perspectives in this regard is of the utmost importance for higher education institutions. This is still under research, particularly under the new pandemic scenario [21]. Indeed, different theoretical models have captured the factors typically affecting technology acceptance by incumbent actors, particularly students and professors [32,33,34]. However, it is not yet clear the way the particular conditions created during the COVID-19 crisis have affected how these actors perceive and are willing to accept e-learning [35,36,37,38]. Characterising incumbent actors’ perceptions regarding e-learning during the pandemic is indeed the crux of the matter. This knowledge is essential to inform policymakers and higher institution managers on successful e-learning implementation in conditions similar to those created during the COVID-19 crisis. The objective of our study is to analyse the point of view of the students regarding e-learning adoption during the pandemic. In particular, this article presents a study on the acceptance and use of Microsoft Teams (hereinafter MS Teams) at universities in Madrid (Spain), using a sample of students from different engineering faculties belonging to two universities, one being a public establishment (Universidad Politécnica de Madrid, UPM) and the other a private one (CEU San Pablo). The Unified Theory of Acceptance and Use of Technology (UTAUT) model was used as a theoretical background for this research [32]. The particular research question addressed is formulated as follows: What are the main factors affecting students’ acceptance and use of an e-learning platform during the COVID-19 lockouts in engineering universities in Madrid?

Our study provides a fresh look at the acceptance of technology within higher education. Although a number of recent studies have explored student technology acceptance of e-learning during the COVID-19 outbreak [9,35,36,37,38,39,40,41,42], to the best of our knowledge, up to the current moment, no former study has been conducted in this particular regional context. Additionally, the number of works which have used the UTAUT model to study e-learning acceptance in this context is still limited [43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. Finally, no study to date has provided a comparison between public and private institutions. The minder of this article is structured as follows. Section 2 explains the theoretical background of this research, whereas the context of the study and the methodology used are explained in Section 3. Section 4 presents the results of the study. After that, the results are discussed in Section 5. Finally, Section 6 present some conclusions and limitations of this research.

2. Literature Review

2.1. The UTAUT Framework and E-Learning Acceptance during COVID-19

There exist several models designed to evaluate the factors affecting technology acceptance. Various studies have been based on the Technology Acceptance Model (TAM), such as [33,34]. Among several models derived from TAM, the UTAUT model was chosen (see Figure 1), which was created by unifying eight previously existing models that [32] tested and validated for different types of situations (including both optional and compulsory use by users, and these including both students and company employees). The versatility of this model was key in determining its suitability for this study. The model was chosen because of its explanatory power and completeness in the acceptance and use of IS studies [44]. Furthermore, the qualitative study conducted as part of this research (see Section 3.1) confirmed the significance of the factors considered in the UTAUT framework.

The UTAUT model is composed of six constructs, each defined as follows:

Performance expectancy (PE): the degree to which an individual believes that using the system will help him or her to attain a better job performance.

Effort expectancy (EE): the degree of ease associated with the use of the system.

Social influence (SI): the degree to which an individual perceives that influential people believe he or she should use the new system.

Facilitating conditions (FC): the degree to which an individual believes that an organisational and technical infrastructure exists to support the use of the system.

Behavioural intention (BI): the intention to use the system.

Use behaviour (UB): the actual user behaviour.

In the UTAUT model, four other variables are posited to moderate the impact of the four key constructs on usage intention (BI) and behaviour (UB): gender, age, experience, and voluntariness of use. For example, as depicted in Figure 1, the influence of performance expectancy on behavioural intention is moderated by gender and age; that of effort expectancy is moderated by gender, age, experience, etc.

Extant literature is still limited on the application of UTAUT to e-learning acceptance by students [44], particularly during the COVID-19 crisis. Experiences in the Middle East and East Asia [52,53,59,59,60,61], as well as in several African countries [50,51,55], are particularly represented in literature. Experiences in other parts of the world, such as Europe or America, are comparatively less represented [49,57]. Moreover, no previous studies have been conducted on the application of UTAUT to e-learning acceptance in the Spanish university context.

2.2. Hypothesis and Theoretical Framework

As it will be explained in the Methodology section, for the purposes of our study, this model was updated and fine-tuned with an additional construct (see Figure 2), which was introduced following the work of Escobar-Rodríguez et al. [34], and Hwang [62] This was deemed appropriate after conducting a preliminary qualitative analysis based on six focus-group discussions with students. The additional construct is defined as follows:

Perceived usefulness for the professor (PUP): defined as how the students believe the tool improves the usefulness to professors in terms of productivity, evaluation, and student follow-up.

The theoretical model was used as the basis upon which a number of hypotheses to be tested were drawn up. These hypotheses are:

Hypothesis 1.1 (H.1.1).

Perceived Usefulness for the Professor has a significant effect on the Performance Expectancy of MS Teams.

Hypothesis 1 (H.1).

Performance Expectancy has a significant effect on the Behavioural Intention to use MS Teams.

Hypothesis 2 (H.2).

Effort Expectancy has a significant effect on the Behavioural Intention to use MS Teams.

Hypothesis 3 (H.3).

Social Influence has a significant effect on the Behavioural Intention to use MS Teams.

Hypothesis 4 (H.4).

Facilitating Conditions have a significant effect on the Behavioural Intention to use MS Teams.

Hypothesis 5 (H.5).

Facilitating Conditions have a significant effect on the Use Behaviour of MS Teams.

Hypothesis 6 (H.6).

Behavioural Intention has a significant effect on the Use Behaviour of MS Teams.

These hypotheses were elaborated based on the conclusions drawn in the extant literature on the application of UTAUT to e-learning acceptance, particularly during the COVID pandemic [35,36,37,38,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. Indeed, most of the experiences represented in the literature validate the cause-effect relationships between the main constructs of the UTAUT model. Hypothesis H1.1 was drawn as a result of the qualitative study conducted as part of this research, which is described in Section 3.2.

3. Materials and Methods

3.1. Context

This study assessed the intention to use a collaborative e-learning IT tool in two engineering institutions in Madrid, one being a public university (Universidad Politécnica de Madrid-UPM) and the other a private one (CEU San Pablo). UPM is one of the oldest polytechnic universities in Spain. It was founded in its present organisational form in 1971, but its origins can be traced back to the beginning of the 19th century. It comprises 18 engineering faculties and offers degree programmes covering all areas of architecture and engineering. Meanwhile, CEU San Pablo was founded in 1933 and is one of the largest and most traditional private universities in Spain. CEU San Pablo offers a wide range of subjects and 6 faculties, including an engineering faculty.

In order to address this study, a particular IT platform was chosen based on the expectation that users’ responses would be more accurate when asked about a particular tool rather than generally about all tools in the sector. The tool chosen was MS Teams. MS Teams is a unified communication and collaboration platform that combines workplace chatrooms, video meetings, file storage (including collaborative editing), and application integration. Microsoft launched the service worldwide on 14 March 2017. By 19 November 2019, it reached 19 million users, and on 19 March 2020 (around the start of the COVID-19 pandemic), it reached 44 million users.

Within our sample, in some cases, MS Teams represented the only available option and was compulsory (at the private university), while in others, it was optional, with professors choosing which collaborative tool to use to deliver their online classes (among several available options, such as Moodle, BB Collaborate, etc.). In addition, although this article focuses on students, it is part of a broader investigation in which the acceptance and use by faculty (i.e., employees of a university) were also assessed, which will be discussed in subsequent works.

3.2. Qualitative Analysis

As explained in the previous section, the conceptual model employed in this study was a slightly modified UTAUT model, in which one construct was added to the six original constructs, as described in Section 2 above.

In order to fine-tune our model, a qualitative analysis was conducted. Six different student focus groups were organised considering the following criteria:

• Gender balance.;
• Inclusion of students belonging to different academic levels (undergraduate, graduate, and doctoral levels);
• Varied programme specialisations;
• Varied geographical locations within Madrid amongst the different university schools.

The focus group discussions were conducted between 18 February and 6 March 2020. A number of open questions were launched to the students regarding their experience with different e-learning tools available at UPM (Moodle, MS Teams, Virtual Labs, MOOCs, etc.), the role of the faculty members, and possible improvement opportunities. Several specific questions were posed regarding MS Teams.

Each focus group brought together around twenty students from different schools within UPM.

The focus groups involved students from different engineering schools belonging to UPM. Information coming from the focus group was transcribed and systematically analysed using content analysis techniques (Weber, 1990). Content analysis is a technique for analysing the content of a text; content might include words, symbols, pictures, or any other format that can be communicated. It has been extensively used in social sciences and particularly in education [63,64,65]. An important step in content analysis is codifying the text (or content) of a piece into various categories depending on certain criteria [66]. In this case, the codes were established a priori using the UTAUT categories. The codes were assigned to the text by three independent researchers; discrepancies were discussed until a consensus was reached.

Several cross-cutting themes were identified thanks to this analysis that were of use to fine-tune the conceptual model:

• The professor was acknowledged as a fundamental actor. The choice and efficacy of IT tools were deemed to be highly dependent on the kind of use made by the professor;
• The facilitating conditions and, particularly, having a help desk and a clear and available institutional repository of information were considered to constitute an important factor;
• The construct “Perceived usefulness for the professor” was deemed to be a key factor by the students. The role of professors in shaping the acceptance of the technology was thus incorporated as part of our model.

3.3. Data Collection

The survey involved a total of 346 undergraduate students from nine different engineering faculties belonging to the two analysed universities (50 students in the first year, 74 students in the second year, 137 students in the third year, 57 students in the fourth year and 28 in the fifth year). Indicators were designed to measure each variable (construct) in the model. These took the form of 34 questions that were posed to university students in an online survey. An online questionnaire was distributed by e-mail via the student associations present at the targeted university schools of engineering.

Responses were quantified using a 5-point Likert scale (where: 1 = totally disagree, 5 = totally agree), and in this manner, the indicators provided measurements for the variables in the model. The data collected were consolidated into tables containing all the responses. This data contained no personally identifiable information. Only non-identifying attributes were recorded: gender, the university and engineering school attended, and the academic level of the studies being pursued (undergraduate, graduate, or PhD). Our dataset was composed of the following groups:

• 249 responses from the public university (UPM) vs. 97 from the private university (CEU San Pablo);
• 165 responses from women vs. 181 from men;
• 160 responses from students of industrial engineering, 45 from architecture, 42 from industrial design, 31 from biomedical engineering, 26 from telecommunications engineering, 15 from mining engineering, 13 from aeronautic engineering, 12 from IT engineering, and 2 from road engineering.

3.4. Statistical Analysis

The model was quantitatively analysed using structural equation modelling (SEM), and SMART PLS software was applied for data analysis. SEM is a multivariate technique that enables evaluation and tests multivariate causal relationships. The technique has been increasingly used since the beginning of the last century in multiple scientific domains, including education [67].

The first step was to analyse the data sample’s appropriateness for the chosen model. This involved testing the sample size as well as its qualities (missing values and normality). The second step was to perform an analysis of the measurement system, which meant validating the indicators (the survey questions). The specific statistical tests employed in these steps are detailed in the Results section. The third and final step was an analysis of the structural system, which assessed the validity of the relationships between the latent variables (or constructs) by testing the hypotheses (H1.1, H1, H2, H3, H4, H5 and H6).

4. Results

This section is composed of three parts. To begin with, an assessment of the volume and quality of the data in order to ensure it was sufficient and appropriate for carrying out the study. Secondly, an evaluation of the quality of our measurement system, in which each of the constructs in the model was decomposed into indicators, corresponded to the questions posed in the survey. Finally, an analysis of the quality of our model (structural analysis), the objective of which was to ensure there were no redundant elements (collinearity analysis) and to determine the predictive ability of the model, together with the relative weight of each of the constructs in the model. The predictive relevance of the model was analysed, and the weight of each of the constructs was determined, independently of the data, with the use of blindfolding techniques.

4.1. Data Analysis

This subsection deals with the evaluation of different aspects of the dataset, which consisted of a total of 346 samples. To begin with, the sample size was assessed in relation to the chosen model employing three different methods. The rule of [68] suggested a minimum viable sample size of between 40 and 60 (respectively equivalent to the highest number of formative indicators of a construct and the highest number of structural relationships, each multiplied by one order of magnitude). An estimate based on statistical power, developed by [69], indicated a minimum sample size of 97. (This method uses four parameters: the effect size, the power, alpha, and the number of predictors). The last method involved using the G Power programme, as recommended by [70], which yielded a value of 98. The threshold values thus obtained were, in all cases, amply surpassed by our dataset of 346 samples.

The second part assessed the qualities of the dataset. Missing values were evaluated using SmartPLS, which yielded a total of 290 missing values among a total of 34 indicators and 346 samples, which represented 2.4% of the dataset (290/(346 × 34)). This value being below the 5% threshold, the amount of missing data was thus considered quite acceptable. With regard to the distribution of the data, PLS does not impose any assumptions concerning its normality.

The theoretical model was set up in SmartPLS, as shown in Figure 3.

4.2. Analysis of the Measurement System

The objective of this analysis was to determine the extent to which the indicators for each construct or latent variable meet the required reliability and validity.

For the reflective indicators (outward arrows from the constructs), reliability was analysed using Cronbach’s alpha, composite reliability [71], and consistent reliability [72]. The resulting values were above 0.7. Convergent validity, or the extent to which a set of indicators represents a single construct, was evaluated through load analysis, commonality analysis, and AVE. Finally, discriminant validity, or the extent to which an indicator is different from the rest, was assessed using cross-load analysis [51].

The analysis of these indicators is reflected in

Table 1. Reflective measurement system. Source: own elaboration.

Construct	Indicators	Reliability			Convergent Validity			Discriminant Validity
		Cronbach’s Alpha	Composite Reliability	Consistent Reliability	Loads	Indicator Commonality	AVE	HTMT	HTMT Interval Excluding 1
		0.6–0.9	0.6–0.9	0.6–0.9	>0.7	>0.5	>0.5	<0.9
BI	BI1	0.844	0.843	0.844	0.811	0.658	0.643	Yes	Yes
	BI2				0.770	0.593
	BI3				0.822	0.676
UB	BO1	0.514	0.843	1.369	0.283	0.080	0.788	Yes	Yes
	BO2				1.223	1.495
EE	EE1	0.838	0.839	0.846	0.801	0.642	0.637	Yes	Yes
	EE2				0.716	0.513
	EE3				0.869	0.755
PE	PE1	0.799	0.801	0.805	0.747	0.558	0.574	Yes	Yes
	PE2				0.821	0.674
	PE3				0.699	0.489

. Focusing on the results for UB’s indicator B02, Cronbach’s alpha was 0.514, below the lower limit of 0.6, but composite reliability (0.843) and consistent reliability (1.369) were both acceptable. The indicator BO1 was dropped for having a load of less than 0.7. The criteria of [73] were followed.

The convergent validity of the reflective indicators was assessed, to begin with, by analysing loads. BO1 had a value below 0.7. The commonality of each indicator represents how much of the variance of the construct is due to the given indicator (with load values of 0.7, yielding 50% of construct variance). AVE describes how variations in the indicators are reflected in the construct. AVE values were above 0.5.

The discriminant validity was tested using the classical methods of cross-load analysis and the Fornell–Larcker criterion. Cross-loads were calculated using the correlations between the construct scores and the standardised data [53]. To comply, no item should load more heavily on another construct than on the one it is meant to measure [46]. The Fornell–Larcker criterion states that the amount of variance a construct receives from its indicators (measured by AVE) should be greater than the amount of variance it generates for other indicators. The results are presented in

Table 2. (A). Measurement system for reflective indicators. Discriminant validity. Cross-load analysis. Source: own elaboration. (B) Measurement system for reflective indicators. Discriminant validity. Fornell–Larcker criterion. Source: own elaboration.

(A)	BI	EE	FC	PE	PUP	SI	UB
BI1	0.811	0.352	0.189	0.497	0.197	0.616	0.125
BI2	0.770	0.278	0.208	0.462	0.249	0.580	0.144
BI3	0.822	0.228	0.214	0.455	0.258	0.626	0.182
BO1	0.028	0.036	0.089	0.101	0.109	0.082	0.283
BO2	0.236	0.157	0.104	0.137	0.027	0.169	1.223
EE1	0.280	0.785	0.282	0.418	0.284	0.216	0.125
EE2	0.256	0.719	0.218	0.391	0.268	0.228	0.063
EE3	0.314	0.882	0.240	0.370	0.231	0.211	0.116
SI1	0.257	0.212	0.174	0.316	0.235	0.338	0.203
SI2	0.451	0.116	0.117	0.303	0.142	0.595	0.151
SI3	0.739	0.270	0.149	0.431	0.253	0.975	0.116
(B)	BI	EE	FC	PE	PUP	SI	UB
BI	0.801
EE	0.356	0.798
FC	0.254	0.308
PE	0.588	0.490	0.339	0.758
PUP	0.293	0.324	0.322	0.573
SI	0.758	0.272	0.165	0.460	0.265
UB	0.188	0.128	0.097	0.124	0.040	0.146	0.888

A,B. Cells in bold indicate links between the indicators and the constructs to which they belong (e.g., the intersections between BI and BI1, BI2, and BI3).

Ref. [74] demonstrated the lack of sensitivity of classical methods and devised the HTMT method. It represents the average correlation between HT (heterotrait-heteromethod) and MT (monotrait-heteromethod), and the values obtained should be below 0.9. The bootstrapping technique was then used to test whether the result was significantly different from 0.9. That is if the value 1 was included in the 90% confidence interval. The results for discriminant validity (HTMT) were as follows: EE effects in BI is 0.356; PE effects in BI is 0.588; PE effects in EE is 0.490; UB effects in BI is 0.188; UB effects in EE is 0.128; UB effects in PE is 0.124.

The following step was to evaluate the formative indicators (arrows from indicators towards constructs in Figure 3). It is important to note that the criteria applied to reflective constructors cannot be applied to constructive ones due to the inherent nature of how they are formed [75].

The formative constructs were tested through convergent validity analysis or redundancy analysis. This required a reflective indicator for every formative construct, which were included in the survey. Each formative construct was divided into two constructs. The first of these existed in the model, with the formative indicators, and it, in turn, communicate with a new global construct that had a single reflective indicator, which had already been considered in the survey. The path coefficient for this redundancy model will give us an idea of convergent validity. This is known as redundancy analysis [76]. A path coefficient value of around 0.7 or higher indicates convergent validity.

According to these criteria, the only formative indicators left were those for Social Influence (SI). The next step was to analyse the collinearity between them. VIF values < 3.3 indicate an absence of collinearity. Relevance and significance were then analysed. Given that the weights of the indicators decrease as they increase in number, the maximum possible weight value for each of the three SI indicators was 1/√3 = 0.58. The absolute importance of a formative indicator comes from its external load (loads come from simple regressions of the constructor with its indicators). Therefore, when the external weight of an indicator was not significant, the external load was analysed. If it was also below 0.5, its statistical significance was then analysed by applying the bootstrapping technique to more than 5000 samples. Doubts arose with respect to SI1, as it had the lowest external load, and its absolute contribution was 0.338 (below 0.5) and was not significant, but at the same time, it had no collinearity (VIF < 3.3) and was conceptually relevant [77]. See

Table 3. Reflective measurement system. Convergent validity. Collinearity (VIF). Relevance and significance. Source: own elaboration.

Constructor	Indicator	External Weight	External Load	VIF	t-Value	p-Value	95% Confidence Interval	Significance (p > 0.05)
SI	SI1	0.052	0.338	1.103	0.795	0.427	[−0.077, 0.181]	No
	SI2	0.228	0.595	1.230	3.532	0.000	[0.097, 0.350]	Yes
	SI3	0.968	0.975	1.241	18.303	0.000	[0.772, 0.955]	Yes

.

The remaining steps involved analysing the significance of the different groups of users that existed in the measurement system. These distinguished between:

• Public and private university;
• Men and women;
• Different university schools.

Two types of non-parametric statistic tests were employed. The Mann-Whitney U test was employed when comparing two groups (see

Table 4. Significance of the indicators for the group’s University and Gender. Mann–Whitney U test. Source: own elaboration.

	Tests Statistics University						Tests Statistics Gender
	Univ.	N	Mean Range	Range Sum	Mann-Whitney U	Signif.	Gender	N	Mean Range	Range Sum	Mann-Whitney U	Signif.
BO2	Public	249	168.44	41,941.00	10,816.000	0.022	Women	165	180.76	29,826.00	13,734.000	0.050
	Private	97	186.49	18,090.00			Men	181	166.88	30,205.00
PE1	Public	249	172.27	42,896.00	11,771.000	0.705	Women	165	178.20	29,402.50	14,157.500	0.388
	Private	97	176.65	17,135.00			Men	181	169.22	30,628.50
PE2	Public	249	172.20	42,878.50	11,753.500	0.685	Women	165	176.33	29,095.00	14,465.000	0.598
	Private	97	176.83	17,152.50			Men	181	170.92	30,936.00
PE3	Public	249	169.50	42,205.50	11,080.500	0.219	Women	165	164.16	27,087.00	13,392.000	0.087
	Private	97	183.77	17,825.50			Men	181	182.01	32,944.00
EE1	Public	249	174.24	43,386.00	11,892.000	0.810	Women	165	182.64	30,135.00	13,425.000	0.077
	Private	97	171.60	16,645.00			Men	181	165.17	29,896.00
EE2	Public	249	175.47	43,691.50	11,586.500	0.531	Women	165	170.55	28,141.50	14,446.500	0.576
	Private	97	168.45	16,339.50			Men	181	176.19	31,889.50
EE3	Public	249	177.57	44,214.50	11,063.500	0.166	Women	165	175.88	29,021.00	14,539.000	0.628
	Private	97	163.06	15,816.50			Men	181	171.33	31,010.00
PUPG	Public	249	173.39	43,173.00	12,048.000	0.972	Women	165	167.47	27,633.00	13,938.000	0.272
	Private	97	173.79	16,858.00			Men	181	178.99	32,398.00
FCG	Public	249	153.81	38,299.50	7174.500	0.000	Women	165	180.40	29,766.00	13,794.000	0.211
	Private	97	224.04	21,731.50			Men	181	167.21	30,265.00
SI1	Public	249	159.42	39,695.00	8570.000	0.000	Women	165	188.82	31,154.50	12,405.500	0.004
	Private	97	209.65	20,336.00			Men	181	159.54	28,876.50
SI2	Public	249	168.75	42,018.00	10,893.000	0.147	Women	165	187.97	31,015.00	12,545.000	0.009
	Private	97	185.70	18,013.00			Men	181	160.31	29,016.00
SI3	Public	249	172.79	43,025.50	11,900.500	0.827	Women	165	176.75	29,164.50	14,395.500	0.549
	Private	97	175.31	17,005.50			Men	181	170.53	30,866.50
BI1	Public	249	187.68	46,733.00	8545.000	0.000	Women	165	180.01	29,701.00	13,859.000	0.230
	Private	97	137.09	13,298.00			Men	181	167.57	30,330.00
BI2	Public	249	177.26	44,137.00	11,141.000	0.248	Women	165	180.44	29,772.00	13,788.000	0.204
	Private	97	163.86	15,894.00			Men	181	167.18	30,259.00
BI3	Public	249	167.57	1	10,599.000	0.070	Women	165	178.30	29,419.50	14,140.500	0.382
	Private	97	188.73	2			Men	181	169.12	30,611.50

).

The different significance by universities (private/public) is B02, FCG, SI1 and BI3. On the other hand, BO2, SI1 and Si2 show different significance by gender.

The differences between the nine different engineering schools in the dataset were evaluated using the Kursaal-Wallis test for multigroup data (see

Table 5. Kursaal-Wallis non-parametric test for the multigroup variable “School”.

Ind.	Question	Asymptotic Significance	Significance
BO2	Please tell us again how often you have used Teams during the COVID-19 pandemic.	0.004	Yes
PE1	Teams enables me to improve my time management.	0.536	No
PE2	Teams helps me achieve the objectives of the course.	0.119	No
PE3	By using Teams, I improve my chances of getting a good grade in this course.	0.214	No
EE1	Teams is easy to use.	0.753	No
EE2	Teams can be used by anyone, with no need for specific training.	0.829	No
EE3	I adapted quickly to using Teams.	0.445	No
PUPG	Overall, I consider Teams a useful tool for the professor.	0.006	Yes
FCG	The resources provided by the university have enabled me to easily adapt to online classes.	0.000	Yes
SI1	The professor advocates the use of Teams for class work.	0.003	Yes
SI2	My classmates have encouraged and helped me to use Teams.	0.228	No
SI3	Teams is a tool that professionals in my sector recommend.	0.323	No
BI1	I intend to use Teams on a daily basis.	0.001	Yes
BI2	I plan to use Teams regularly.	0.279	No
BI3	I find Teams’ functions useful and will continue to use them.	0.082	No

).

Significance variance by engineering school grouping, according to the Kursaal-Wallis non-parametric test for multigroup data, are BO2, PUPG, FCG, SI1 and BI1.

4.3. Analysis of the Structural System

As a means for testing the hypotheses, the capacity of the model to predict one or more constructs was evaluated through an analysis of the structural model [76].

The following steps were taken:

• Assessment of collinearity in the structural model;
• Assessment of the significance and relevance of the relationships within the structural model;
• Assessment of the level of R²;
• Assessment of the effect size (f²);
• Assessment of the predictive relevance (ϱ²);
• Assessment of the predictive significance (q²).

Collinearity was evaluated considering a variance inflation factor (VIF), for which the value obtained was below the threshold of 3, thus meeting the criterion of acceptability. See

Table 6. Structural measurement system. Collinearity analysis. VIF. Source: own elaboration.

	BI	EE	FC	PE	PUP	SI	UB
BI							1.069
EE	1.361
FC	1.165						1.069
PE	1.622
PUP				1.000
SI	1.273
UB

.

Path coefficient values oscillate between −1 and 1, indicating stronger and more important relationships as they approach 1. The results yielded positive values in all cases, therefore supporting the model. Statistical significance depends on the standard error that results from applying the bootstrapping technique to a data sample. In this analysis, the two-tailed and 5000-sample test was employed. The higher the empirical t-value is above the critical value, the greater the statistical significance. Thus, with an alpha of 5% = 0.05, the critical value would be 1.96, and the result will have significance. The p-value is often used because it is easier to remember, as it corresponds to the alpha value and reflects the probability of erroneously rejecting the null hypothesis when it is true. Thus, a p-value below the significance level (alpha) implies the significance of the path coefficient. However, if zero is found within the confidence interval, it indicates non-significance. If a path coefficient is statistically significant, it indicates the extent to which the exogenous construct is linked to the endogenous construct [77] (see

Table 7. Structural measurement system. Significance and relevance of construct paths. Source: own elaboration.

Hypothesis	Path	Original Sample (O)	Sample Mean (M)	Bias	p-Value	2.5%	97.5%	Sig. p < 0.05?
H1	PE -> BI	0.212	0.211	0	0	0.128	0.302	Yes
H1.1	PUP -> PE	0.518	0.518	0	0	0.426	0.597	Yes
H2	EE -> BI	0.071	0.074	0.003	0.078	–0.01	0.147	No
H3	SI -> BI	0.602	0.603	0.001	0	0.516	0.675	Yes
H4	FC -> BI	–0.11	–0.109	0.001	0.001	–0.178	−0.044	Yes
H5	FC -> UB	0.044	0.043	−0.001	0.377	–0.058	0.141	No
H6	BI -> UB	0.218	0.218	0	0	0.103	0.325	Yes

).

Therefore, our model is shown in Figure 4.

R-squared (R²) is useful as an assessment of the predictive ability of the model. It is calculated as the squared correlation between the actual and predicted value of a specific endogenous construct. This takes into account the combined effect of all exogenous constructs affecting the endogenous variable. Going further, an adjusted R-squared (R² adj) that attempts to correct for the bias is more appropriate for complex models. R² adj = 1 − (1 − R2) × (n − 1)/(n – k − 1), where n is the sample size and k is the number of exogenous variables affecting the endogenous construct being measured. The resulting values should be high enough to achieve a minimum explanatory power, which [78] state should be at least 0.1. [56] established that values of 0.67 and above were substantial, around 0.33 were moderate, and around 0.25 were weak in terms of explanatory power. [77] state that, in the field of marketing, values should be above 0.75, the explanatory power is substantial, around 0.5 and moderate, around 0.25. In the model, BI had a substantial predictive ability, PE moderate and UB weak. See

Table 8. Structural measurement system. Predictive ability. Source: own elaboration.

	R-Squared	Adjusted R-Squared
BI	0.660	0.657
PE	0.330	0.328
UB	0.055	0.053

.

Next, the impact of the non-significant constructs was measured, in particular, EE effects in BI and FC effects in UB. For the effect size, f² was used, which measures the impact of omitting the effect of an exogenous construct on an endogenous construct, in terms of predictive ability (R²). The contributions of FC and EE to the predictive ability of the model were very small.

The heuristic rule established by [69] states that:

• 0.02 ≤ f² < 0.15: small effect;
• 0.15 ≤ f² < 0.35: moderate effect;
• f² ≥ 0.35: large effect.

Therefore, there was a large effect on SI effects in BI and PUP effects in PE, a small effect on PE effects in BI, and a negligible effect on PUP effects in BI, FC effects in BI, FC effects in UB, PE effects in UB, SI effects in UB (marked in red). See

Table 9. Structural measurement system. Effect size (f2). Source: own elaboration.

	BI	FC	PE	PUP	SI	UB
BI						0.059
FC	0.04
PE	0.211
PUP			0.493
SI	0.917

.

The predictive relevance (ϱ²) provides an assessment of out-of-sample predictive ability. For this purpose, a blindfolding technique was used, which consisted of reusing the sampled data, eliminating the dth datum, and re-estimating the parameters with the remaining data. The predictive relevance was obtained by calculating the difference between the true data (which were produced when omitting the edth datum) and that which was predicted. The Handbook of Partial Least Squares Structural Equation Modelling (PLS-SEM) states that for “the relative measure of predictive relevance, values of 0.02, 0.15, and 0.35 indicate that, for a particular construct, an exogenous construct has small, medium, or large predictive relevance, respectively”. Thus, the predictive relevance for BI was considerably high (0.406); for PE, moderate (0.187); and for UB, small (0.035).

Next, an analysis of the effect size was performed based on the predictive relevance, similar to what was previously done based on the R2 values.

Therefore, considering the value of the path coefficients, the significance of the path coefficients themselves, and the effect size on both BI and UB, the latent variables EE on BI and FC on UB were dropped from the model.

The following step was to analyse the heterogeneity of the dataset. To this end, the following category groups were analysed:

• University (Public/Private);
• Gender (Men/Women);
• Schools.

To end, the extent to which the groups differed was analysed, so as to determine if the differences among them were significant or not. The technique proposed by [74] was used, which involves applying bootstrapping to the dataset with 5000 samples. Parameters for each group were thus estimated, and a comparison among these determined if the differences were significant or not. This analysis was carried out using MGA multigroup analysis with SmartPLS. The results in

Table 10. MGA multigroup analysis. Public/Private university. Men/Women. Bootstrapping. Significance. Source: own elaboration.

Public/Private University					Men/Women
Differences between. Opposed Categories	Path Coefficient	t-Value	p-Value	Significance p < 0.05?	Differences between. Opposed Categories	Path Coefficient	t-Value	p-Value	Significance p < 0.05?
BI -> UB	0.076	0.614	0.54	No	BI -> UB	0.076	0.614	0.54	No
FC -> BI	−0.085	1.116	0.265	No	FC -> BI	−0.085	1.116	0.265	No
PE -> BI	0.059	0.527	0.598	No	PE -> BI	0.059	0.527	0.598	No
PUP -> PE	−0.107	1.153	0.25	No	PUP -> PE	−0.107	1.153	0.25	No
SI -> BI	−0.035	0.368	0.713	No	SI -> BI	−0.035	0.368	0.713	No

show that only PUP was significant when comparing university types. Also, R² was greater in the public sector than in the private one (0.714 vs. 0.493). See Figure 5A–D and Figure 6, and Table 10.

Grams for different university engineering schools. The brackets indicate the number of respondents from each school. Source: own elaboration.

The school-based models did not have sufficient sample sizes to validate them, except in the case of Industrial Engineering (160 samples, above the minimum sample size of 97). Therefore, it was not possible to analyse the extent to which differences among schools were significant. In addition, the school-based models displayed rather atypical results due to the limited amount of data available (see Figure 6).

4.4. Summary of Results

Table 11. Summary of significant differences by group for each indicator. Source: own elaboration.

Construct	Indicator	Significant Difference
		Gender	Univ (pub/priv)	School
UB	BO2	Yes	Yes	Yes
FC	FCG	No	Yes	Yes
SI	SI1	Yes	Yes	Yes
SI	SI2	Yes	No	No
BI	BI1	No	No	Yes
PUP	PUPG	No	No	Yes

presents a summary of significant differences by group for each indicator.

• The indicator BO2 (use of MS Teams) displayed significant differences among all groupings. This is due to the tool being mandatory at the private university.
• The indicator SI1 (the professor’s social influence) displayed significant differences between genders and types of universities, and it reflected the important influence exerted by the professor on public students and women.
• The indicator SI2 (classmates’ social influence) displayed a significant difference between genders, being a more important factor for women than men.

The structural system shows a very good predictive ability (see

Table 12. Summary of predictive ability and path indicators by groups. Source: own elaboration.

Data	R2BI	R2UB	R2PE	Path
				PE -> BI	SI -> BI	BI -> UB	FC -> BI	PUP -> PE
Global	0.660	0.055	0.330	0.301	0.631	0.236	−0.117	0.574
Private	0.500	0.016	0.149	0.308	0.529	0.125	−0.093	0.386
Public	0.718	0.076	0.413	0.312	0.654	0.276	−0.076	0.642
Women	0.635	0.034	0.405	0.266	0.650	0.185	−0.072	0.636
Men	0.688	0.068	0.280	0.326	0.615	0.261	−0.157	0.529

). R2 values should be high enough for the model to achieve a minimum level of explanatory power [58] recommend values above 0.10, whereas [56] considers R2 values of 0.67, 0.33, and 0.19 as substantial, moderate, and weak, respectively. The model had a very high coefficient of determination (R² = 0.660) for the intention to use MS Teams (BI), it being higher for public universities (R² = 0.718) and men (R² = 0.688). The value of 0.384 for BI indicates a high coefficient of determination. The model is, therefore, valid for BI.

The elements that determine BI are especially SI (Social Influence) and PE (Perceived Expectancy) according to the values of the path coefficients.

The differences between path coefficients when comparing public and private universities suggest reasons for it being higher in the public sector, given that it is for PE that the non-parametric statistical data show a significant difference (see the numbers in blue in Table 12). It is, therefore, for PE that predictive ability shifts between public and private institutions.

5. Discussion

The most important factor determining the intention to use (BI) was Social Influence (SI). When analysing the SI indicators (SI1, SI2 and SI3), it is worth mentioning the enormous weight that opinions from social and professional networks had in determining the intention to use e-learning. Social networks and professional forums are, therefore, key tools for developing a positive attitude towards e-learning by students in the current situation. This should lead to a reflection on the permeability of professors and higher education institutions to tool choices coming from professional or social networks. This is, in general terms, consistent with results found in previous research dealing with the application of UTAUT to e-learning acceptance during COVID-19 in very different regional contexts [43,46,49,50,54,60]. However, it is possible to mention some exceptions, as a similar study focused on a developing country found that social influence did not affect students’ acceptance of e-learning [52]. This seems to imply that the impact of social influence on e-learning acceptance in the pandemic context could be context-dependent.

The second most important factor affecting BI was the Performance Expectancy (PE), which in the model was determined by the new construct introduced: perceived usefulness for the professor (PUP). In other words, the professor’s attitude towards the e-learning tool was the second most important element influencing the acceptance of TEAMS. Moreover, the results suggest that professors could have more influence over student acceptance of the e-learning platform at public universities than at private ones. Although more research is needed to confirm and explain this result, a possible explanation for this could be the differences in terms of governance between these two types of institutions. Indeed, governance at private universities in Spain usually follows a top-down approach, whereas public universities are less hierarchical [79]. Indeed, the use of TEAMS in the private institution was imposed on the professors, whereas it was optional at public universities. Moreover, private institutions are usually more student-centred (as the student is in part considered as a “customer”), while public institutions follow a more traditional, professor-focused approach.

It is surprising how little weight the facilitating conditions (FC) had on determining the use of the tool, as prior studies on the matter have mostly found that facilitating conditions have a direct impact on students’ acceptance of e-learning during the pandemic [43,49,50,51,52,57]. In this case, the means provided by the university did not determine the students’ attitudes towards e-learning. The same occurred with effort expectancy (EE). The effort involved in assimilating a new collaborative tool did not condition the learner’s intention to use it. This suggests that, in this case, if network “influencers” establish that one tool is better than another in terms of prestige, improvement, and usefulness, it will not be the means provided by the university (support, manuals, networks, computers, etc.) that condition the students’ use. It seems that a fast-learning curve was achieved through the internet and through students sharing experiences among themselves. This result could be linked to the particular regional and disciplinary context in which this study was conducted. Indeed, engineering students in a developed country such as Spain are likely to have significant prior e-learning experience as well as appropriate equipment at home, which would explain the reason why facilitating conditions provided by the university and effort expectancy were not as important in predicting e-learning acceptance as in other cases reported in the literature, coming mostly from developing countries.

6. Conclusions

The main conclusion of this research is that social Influence was the most important factor determining the acceptance of Ms Teams by the students, while the perceived usefulness of the professor occupied the second place. The facilitating conditions and effort expectancy did not affect acceptance, probably due to the particular pandemic context. A significant difference was found between the public and private institutions in terms of the importance of the perceived usefulness for the professor (this factor was more important for students’ acceptance of MS Teams at the public university).

To sum up, for students, managing the change to a new e-learning tool under special circumstances lived during the COVID-19 pandemic requires, first of all, that the software package is positively considered in personal and professional circles and on social networks. This implies that universities wishing to introduce Ms Teams under the current scenario should focus on creating a positive social environment around the platform, for example, by using social networks or relying on testimonies by professionals who could confirm the interest of such a platform in a future work environment. Universities should also be very attentive to proposing to students with e-learning solutions that are used and valued in professional environments.

The second element is performance expectancy (the degree to which the student believes that using the system will help him or her to attain a better performance), where professors play a key role. It is, therefore, particularly important to seek the involvement of professors when implementing e-learning platforms. In that sense, more research is needed to better understand the professors’ perspective and the factors that would facilitate their acceptance of e-learning technology.

An obvious limitation of this research is the fact that it has been conducted in two specific higher education institutions in a particular geographic setting. Furthermore, the findings of this study are limited to an exploration of MS Teams acceptance and may not be applicable to other e-learning platforms. Future research endeavours could include an exploration of the acceptance of e-learning in other geographic contexts using platforms other than MS Teams. More research is also needed regarding the context-dependency of the factors affecting e-learning acceptance during the COVID-19 outbreak.