Mixed Analysis of the Flipped Classroom in the Concrete and Steel Structures Subject in the Context of COVID-19 Crisis Outbreak. A Pilot Study

Abstract

A sudden lockdown was declared on 14 March 2020 due to COVID-19 crisis, leading to an immediate change from face-to-face to online learning in all universities within Spanish jurisdiction. At La Salle School of Architecture, the Concrete and Steel Structures subject started online classes immediately after the lockdown law was published, using a methodology based on the flipped classroom approach and adapting the monitoring of the student to the virtual environment. This article presents a pilot study to analyze the adaptation of the model to the online format using a mixed approach in which qualitative and quantitative surveys were conducted at the end of the course with 48 participants. Responses from both surveys were organized according to six categories (teachers, assessment, methods, class development, students and documents) and 14 subcategories, as developed in an undergoing research project involving the subject since the academic year 2017/2018. Thus, the open responses of the students have been analyzed alongside with the quantitative data. The results demonstrate a proper adaptation of the model, as well as the negative perception of the students of the online format due to the loss of face-to-face benefits of the flipped classroom.

1. Introduction

This article presents a pilot study on the sudden adaptation of the flipped classroom (FC) model from a face-to-face to an online format in the Concrete and Steel Structures subject of the undergraduate studies at La Salle School of Architecture in Barcelona [1], at Ramon Llull University. The abrupt change was caused by the lockdown due to the COVID-19 crisis that was declared on Saturday 14 March 2020 in all universities within Spanish jurisdiction [2]. The next Monday, 16 March, courses that were completely face-to-face just one week before re-started online.

The study presented in this article departs from a broader research started in 2016 that focuses on the continuous improvement of learning through learning analytics and the formative assessment of learning [3,4]. Data is obtained from quantitative and qualitative surveys conducted regularly, while information about the user experience is obtained permanently from the learning management systems used. In this context of learning analytics, collected data is analyzed and used to improve the course design acting on the weaknesses detected.

The aim of the present pilot study is to assess the adaptation of the flipped learning (FL) model used when the COVID-19 crisis suddenly emerged. With this purpose, the following two research questions have been stated, to be answered using the data obtained thanks to the ongoing research in the subject:

• Has the FL model used in the Concrete and Steel Structures subject adapted properly from a face-to-face format to the online format?
• Do the students of the Concrete and Steel Structures subject negatively perceive the change to the online format?

It is known that the FC works to enhance the quality of class time, reallocating lectures out of class time [5], usually with the help of learning material and videos uploaded in learning management systems [6]. Therefore, in the context of FL research, it is relevant to analyze this adaptation and the perception of the students when being suddenly forced to leave the face-to-face class and take the rest of the course in the online format.

Furthermore, this study aims to contribute to the lack of research on learning assessment in architecture undergraduate studies, especially when applying active learning in the science, technology, engineering and mathematics (STEM) subjects such as structures.

The ongoing research in the Concrete and Steel Structures subject, and specifically in the FL analysis, is linked to the previous research in the field of FL, as it is following the future lines recommended by Karabulut-Ilgu, Jaramillo and Jahren in “A systematic review of research on the flipped learning method in engineering education” [7]: (a) reforming engineering education through theoretically sound frameworks, (b) using of qualitative and longitudinal data to provide deeper understanding, (c) investigating systematic adoption of FL in engineering education, and (d) shifting the focus from academic skills to professional skills.

2. Theoretical Framework

2.1. Architecture; Art and Science

Architecture is a synthesis of art and science [8,9,10,11], but the balance between these two terms has not historically been the same [12]. De Architectura libri decem (Ten Books of Architecture [13]) written by Marcus Vitruvius in the 1st century BC is not only the oldest known architecture treaty but also the reference book for establishing the meaning of architecture [14] and supports that architecture is art and science. The Vitruvian theory also asserts that there are three architectural virtues, three interrelated qualities located on an equilateral triangle vertex: firmitas, utilitas and venustas (i.e., structural stability, appropriate spatial accommodation, and attractive appearance). Through the years, the Vitruvian Triad was widely discussed with these three interrelated terms stated together.

However, in 18th century, the equilibrium of this triad was altered by giving primacy to venustas. Thus, the Architecture (with A) essential prerequisite should be beauty, and the construction (i.e., firmitas and utilitas) were subordinated in the theory and development of buildings [15,16]. On the other hand, after 1800 when engineers began creating structures, the concept of venustas had no value in front of the laws of gravity. What is more, firmitas, the structural dimension of the triad, become more and more complex with the emergence of new materials and construction systems, new technologies, new structural calculation methods and new typologies of building [17,18].

Today, even as architecture theoretically embraces the technical dimensions worldwide, a deep knowledge of structures is not considered to be crucial in most countries. Thus, architects have different professional attributions in each country, and that has led to different teaching approaches and school curricula, especially on the competences required on technical subjects [19]. Most European countries do not require deep structural knowledge for architecture students, for they do not have professional attributions, while in other countries, like Spain, architects are considered to be the last responsible of building structures [20]. Even though these differences exist, European Directive 2005/36/CE [21] establishes automatic recognition of the Architect title in all EU countries, with the only condition of the possession of an Architecture title obtained in any state member of the EU, with no mention of the difference in professional attributions for each state. Therefore, the architectural curriculum in the Spanish system [22] must keep the proper balance between art and science, design and technology and building [23], and architecture study programs include a complete training in structures [24]: design and calculation, in coherence with architectural concept, and execution surveying. With some differences between the different architecture schools, all curricula include subjects on mathematics and physics, on resistance of materials and on application of current structural codes for concrete and steel structures.

2.2. Active Learning and Flipped Learning

Active learning is anything that “involves students in doing things and thinking about the things they are doing” [25] and can be defined as classroom based activities designed to engage students in their learning through answering questions, solving problems, discussing content or teaching others, individually or in groups [26]. Despite the effectiveness of active learning that has been established by several studies [27], and specifically in the science, technology, engineering and mathematics (STEM) subjects [28,29], traditional teaching methods are still dominant in undergraduate courses [30,31].

Notwithstanding, active learning has constantly been used in Architecture undergraduate studies, mainly in the projects subjects, which have traditionally applied project-based learning (PBL) [32], at least since the 19th century (e.g., at the Ecole des Beaux-Arts in Paris [33]), as well as in design studios [34] where PBL and problem-based learning [35] are used for knowledge integration from several disciplines.

In this context, FL is an active learning strategy that appears to be particularly well suited to engineering education [7] because of its potential to “combine learning theories once thought to be incompatible—active, problem-based learning activities founded upon a constructivist ideology and instructional lectures derived from direct instruction methods founded upon behaviourist principles” [36]. FL uses face-to-face class time to solve complex exercises where students can interact with the teacher and other students to settle the theoretical knowledge previously acquired outside the classroom through video watching or readings. It can be said that FL is a blended learning model that combines face-to-face and e-learning.

FL implementation is flourishing in the academic world at all educational levels with different learning designs [26,37]. Research in FL show multiple benefits such as flexibility for the students [38], an increase in motivation [39,40], increased participation [41], improvement in interaction between students [42,43] and between students and teachers [44,45], in student engagement [46], in professional skills and in academic results [47], among others.

On the other hand, research also shows FC challenges and limitations, such as increased workload for the faculty, student resistance [48], decreased interest, neglected material and problems for students with low capacity for abstraction, poor problem solving or with difficulties in using learning platforms [49].

2.3. Learning Assessment

The assessment of learning is generally conveyed through two approaches, the formative and the summative [50]. The main objective of the summative assessment is to evaluate the student’s learning at the end of the didactic unit, project, or assessment activity corresponding to a specific syllabus. Summative assessment provides teachers with information on the level of achievement of learning content or competencies. In contrast, the main goal of the formative assessment is to feed the instructional model to meet the needs of students. Formative assessment is used to check students’ level of understanding and plan the most appropriate learning instructional design to guide teachers throughout the learning process.

Formative assessment helps to (a) identify strengths and weaknesses, (b) report objectively with data, (c) carry out improvement actions and (d) detect and advance actions to resolve problematic situations in time and provides a database approach that can help to analyze the information. The Big Data movement integrated into education has transformed education from a dual technological and methodological perspective [51]. On the one hand, the adoption of educational technology has a strong analytical and interaction-gathering aspect. On the other hand, this integration is accompanied by a change in the way we do and make decisions based on data. The adoption of analytical methodologies goes beyond Educational Data Mining [52]. At the academic level, the Academic Analytics approach [53] is applied as a process to improve management. At the level of the teaching-learning process, the Learning Analytics (LA) approach [54,55] is applied with the ultimate goal of improving the educational context through the analysis of data in a five-phase cycle [56] (definition of improvement objectives, data collection, analysis and visualization of results, improvement actions, and iterative evaluation) that allows obtaining relevant information from students’ interactions in digital educational technology.

The relationship between FC, formative assessment and LA [57,58], as FC requires a continuous flow of information for content generation, class preparation, and student support, and formative assessment allows to extract this information continuously. This approach offers a firm data frame for the decision-making in the application of the FC methodology [40] and also for the own evaluation of its implantation.

Evaluation methods in learning research are typically quantitative, qualitative or mixed [59]. The method to be used depends on the type of data to be collected and the characteristics of the sample [60]. Both methods are scientifically valid, the results obtained are comparable and both methods are similar in terms of citation [61].

Quantitative studies are based on numeric data analysis intended to validate the phenomenon being studied. These methods usually work with a small number of variables and statistical analysis is usually required to check the reliability of the results. Data is commonly obtained from surveys with questions that have to be answered by the students in a Likert scale [62]. Quantitative methods are limited because the numerical descriptions provided give no details about the students’ perceptions or motivations.

On the other hand, the main purpose of qualitative methods is to get a better understanding of the human perspective of the sample, and detailed descriptions of specific items of the experiment are obtained.

While quantitative studies are usually focused to find the cause [63], qualitative studies are mainly explanatory [64,65]. The relationship between cause and effect using a control group is usual in quantitative research [66,67], while qualitative research could be phenomenological [68], ethnographic [69], theory-generation focused [70], etc. and it does not have to be necessary to have a control group. There are prospective [71], observational [72] or pilot [73] cohort studies in which the study does not have experimental manipulation but follows participants in the learning context by subjecting them all to the same “experimental protocol”. Conclusions are drawn, and the research is continued as experimental, quasi-experimental or non-experimental. Cause can be extracted when the data is quantitative [74] and explanation [65] and correlation [75] if the data is qualitative.

Cause requires to define the factors that actually cause it with specific quantitative techniques to explain it (such as chi-square [76] in the cause of an independent variable); correlation defines a part of the cause and it does not mean that it is the cause itself, but that the independent variable studied is one of the causes [77]; and explanation implies to take qualitative data and expose it [78].

In our research, we are using a qualitative approach using parametrization with the support of quantitative data to correlate flipped classroom effects in the context of the change from face-to-face to online education without defining a final cause. Mixed assessment use both quantitative and qualitative methods to minimize the weaknesses of both methods and to enhance their strengths, obtaining very robust results when are properly combined.

3. Methodology

3.1. Goals, Research Design and Methods of the Study

As stated, the purpose of the study is to assess the adaptation of the FC model implemented in the course Concrete and Steel Structures, from face-to-face to online.

With that purpose, two research questions (RQ) have been raised. Data obtained from qualitative and quantitative surveys conducted at the end of the course will be analyzed according to these RQ. These surveys are conducted regularly in the subject since 2017, as part of a broader research [3,4] that pursues the continuous improvement of the students’ learning on structures. In this case, students’ answers were delivered online, due to the lockdown.

3.2. Participants

Students were divided in two groups due to the language in which classes were conducted, even the course development was identical for both. The international group took the subject in English with one professor, and the local group took the subject in Spanish and Catalan with another professor. The international group was formed by 18 students, while the local group had 50 students. 16 students of the international group answered the surveys (88.88%) while for the local group 32 students answered them (64%). The total number of students who answered the surveys was 48 and represent 70.59% of the number of total students that took the subject. For the purpose of this study, there will be no comparison between groups, and the results show the data obtained from the 48 participants.

3.3. Course Design

The course of Concrete and Steel Structures is a 9 ECTS (European Credit Transfer and Accumulation System [79,80] credit subject of 3rd course of Architecture undergraduate studies at La Salle Ramon Llull University in Barcelona [1]. There is an introductory subject on resistance of materials with 9 ECTS credits that is given in the 2nd course, called Introduction to Structures. There is also another subject completing the program of the studies in 4th course called Geotechnics and Foundations, with 3 ECTS credits.

The Concrete and Steel Structures course affected by the COVID-19 crisis started on 16 September 2019, with a face-to-face model of 50 sessions of 2 h, from 8 am to 10 am, on Mondays and Tuesdays, finishing on 18 May 2020. The course was divided in 4 parts: (a) Actions, 10 sessions, (b) Concrete structures part 1, 12 sessions, (c) Concrete structures part 2, 12 sessions and (d) Steel structures, 16 sessions.

A flipped classroom model was implemented for the whole course, following this cycle of 2 classes that was completed each week:

• A brief introduction to the topic is explained at class;
• Learning materials (pdf documents) are uploaded in the Moodle platform of the university for parts 1, 2 and 3. In the case of part 4, a video on the topic is uploaded;
• Students read and study the notes at home and watch the video when available;
• Next class starts solving the doubts of the students. That could take up to 50% of class time, depending on the students’ demand. After the doubt solving time, exercises on the topic were solved at class;
• Next class starts also with doubt solving, with a limited time of 15 min. Then, a short exam of maximum 30 min is held. A brief introduction of the next topic is made the last hour of class, and the cycle is repeated from the point 2.

When lockdown was suddenly imposed on 14 March 2020, the subject suffered several changes to adapt to the online platform. The last face-to-face class was the 2nd session of the 4th part and was held on Tuesday 10 March 2020. The 1st online session was held on Monday 16 March 2020. The 1st short exam on part 4 was held in the class of 10 March, and a brief introduction to the 2nd topic on steel was done. At that time, La Salle University introduced the Blackboard Collaborate plug-in in the Moodle platform for online classes.

The main changes introduced in the subject were produced by the decision of not holding tests and exams online until the university had implemented a system for the exams, what happened to be ready after Easter. Therefore, only two short tests were made during lockdown time, joining the seven tests that were left. The rest of the course continued following the same cycle, with some little variations:

• Videos were uploaded in the Moodle platform of the university, as well as the course notes and exercises on the topic;
• Students watched the video and read and study the notes at home;
• Next class started solving the doubts of the students. After the doubt solving time, exercises on the topic were solved at class;
• Next class started also with doubt solving, with a limited time of 15 min. More exercises on the topic were solved at class. A brief introduction of the next topic was made the last half hour of class, and the cycle is repeated from the point 1.

Two classes were completely dedicated to do the joined short exams, following the general system of surveillance for the ordinary and extraordinary exams held in all the subjects of the university. The first one joined 5 of the previewed short exams in one, and the last one included two short exams. Thus, the 8 short exams previewed for part 4 were completed.

3.4. Instruments

A mixed approach is used to perform the study using both quantitative and qualitative surveys. Quantitative surveys conducted at the end of the course had several questions to be rated in a Likert scale from 1 to 5 related to the motivation and satisfaction of the students. The validity of the questionnaire was checked by five university doctors and experts in user-experience assessment belonging to the UserLab [81] at La Salle University, the first academic UserLab in the South of Europe.

Qualitative approach was also used for assessing the group satisfaction on the Flipped Classroom methodology, using the Bipolar Laddering Assessment, BLA [82]. BLA methodology is based in a Socratic Questionnaire with open questions to be answered in three steps and has demonstrated its validity in several studies with both qualitative approach and with mixed approach [83,84,85,86].

• Firstly, students are requested to identify strengths and weaknesses, five of each type at most;
• Secondly, they are asked to value these strengths and weaknesses between 1 and 10;
• Finally, they are invited to propose how to improve what they have identified as strengths and weaknesses.

Students freely decide and identify which points they consider positive and negative. No suggestions or direct questions are stated by the survey conductor to prevent predetermining the answers. This kind of survey provides quality information about which aspects are relevant for the student with no interferences.

Socratic questionnaires [87,88] are used in qualitative studies. It is a method in which the user is invited to reflect about the experience through a dialogue with the guide that helps him in the identification of strengths and weaknesses, and that solves the possible doubts. In this method, the paper of the guide is very different than a classic interview, because it is not strategic and there is no direct intervention in influencing the responses.

Figure 1 shows the format of the BLA interview. It is a real sample with responses and marks from one student. As the questionnaire was sent by email due to the lockdown, instructions to complete the interview were given in the message.

The BLA interview was performed in May 2020 by videoconference due to the lockdown. The form was sent by email and uploaded in the Learning Management System minutes before being filled, and the following instructions were given to the students by the teacher using the videoconference platform:

The BLA form should be filled in four steps:

• Write in the first 5 boxes the 5 aspects you most liked of this structures course.
• Write in the next 5 boxes the 5 aspects you most disliked of this structures course.
• Put a grade from 1 to 10 for each one of the aspects listed, in the box at the right of each one. For both strengths and weaknesses 1 means you completely dislike the aspect and 10 means you absolutely liked it.
• In the next 10 boxes, write what you think it could be changed to improve each aspect listed. There are 10 boxes, one for each aspect previously stated in both positive and negative aspects.

These instructions were given step by step, so that the instructions for step 2 are given after students have completed step 1, and so forth until all steps are completed.

BLA interviews were filled by the students and returned to the teacher by email with no interference of the teacher that has the only role of guiding students to correctly fill the form, giving them full freedom to answer whatever they perceive as most relevant.

3.5. Mixed Data Analysis

By its nature, the data obtained is mainly analyzed in a qualitative approach, and the quantitative data is mainly used to support the validity of the conclusions obtained. The frequency of one answer in open questionnaires has been proved to be valid to answer research questions even with small samples [89,90]. The scientific validity of qualitative research is widely proved [91,92].

To analyze the answers of the qualitative assessment, 6 categories and 14 subcategories have been described. Thus, the answers of the students are classified in these subcategories, allowing comparison between categories and subcategories, and making possible to analyze the evolution of each category after each change. This categorization is also appropriate to classify the changes in the subject, and this way it is possible to relate the changes with the answers in the surveys. Each category has a code of one letter to refer to it, while for each subcategory the codes have two letters: the first one defines the category and the second one is given in alphabetic order.

The 6 categories are: Teachers (T), Documentation (D), Course approach (C), Class activities, dynamics and planning (P), Assessment (A) and Students (S).

The 14 subcategories are: Teachers (TA), Documents (DA), Videos (DB), Content of the course (CA), Course approach (CB), Time/content rate (CC), Class development (PA), System and methodology (PB), Schedule (PC), Technology use (PD), Assessment method (AA), Assessment development/organization (AB), Students’ motivation (SA) and Student’s profit (SB).

This classification is the one used in the aforementioned research [3,4] and it has been defined according to the different changes and aspects that wanted to be measured. For example, implementation of flipped classroom is categorized as PB, an improvement of the course notes is classified as DA and the introduction of tests of continuous assessment belongs to the AA category.

When categorizing the answers of the students, the code of the answers has three letters, adding a previous character to the category code: for strengths, the code starts with a S, for weaknesses with a W and for proposals, with a P. Therefore, if a student writes that the exams are short of time in the weaknesses section, this answer is classified as WAB, or if one student find that the videos are very helpful, that is categorized as SDB.

The categorization system used in this study for the qualitative assessment can easily be adapted to any other subject once the appropriate categories for each case are identified. According to the results, these categories can suffer little variations or can be divided in more subcategories to get most relevant and detailed information.

Students can also assess each strength and each weakness that they have written in the BLA survey with a grade from 1 to 10. As stated by Pifarré and Tomico [68], extreme marks have a strong emotional component. Positive extreme marks (9 or 10) mean that the student is thankful for that element, and negative extreme marks (1 or 2) mean that the student especially dislike that element.

Data obtained from quantitative surveys is treated statistically to allow further analysis and better understanding. For each question in the survey, the number of responses for each value on the Likert scale from 1 to 5 (n) is stated, as well as the mean (M), standard deviation (SD), skewness (S_kw) and kurtosis (K_me). When required, the 95% confidence interval is also calculated (μ), as well as the upper (X1) and lower (X2) endpoints.

One implication of this research to be developed in future lines is the numeric management of qualitative data. Finding and checking proper ways to assign numeric values to quality data obtained in open questionnaires can lead to statistical analysis that can be scientifically treated to get valid conclusions. Even more when these data are transformed in a way that can be mixed with data obtained from quantitative methods.

4. Results

In this section, the results obtained from the qualitative and the quantitative survey conducted at the end of the course are shown, making special mention to the parameters related to the RQ stated.

Regarding the RQ1 (Has the FL model used in the Concrete and Steel Structures subject adapted properly from a face-to-face format to the online format?) the following parameters will provide relevant information:

• Index of mention of the proper adaptation of the course to the online format in the strengths section;
• Proportion of responses related with the FL in the strengths section;
• Proportion of responses related with the FL in the weaknesses section;
• Positive and negative extreme marks of the responses related to FL;
• Lower endpoint value for 95% of confidence of the mark in the BLA for the categories related to FL;
• Lower endpoint value for 95% of confidence of the responses in the quantitative survey for the categories related to FL.

Additionally, regarding the RQ2 (Do the students of the Concrete and Steel Structures subject negatively perceive the change to the online format?) parameters that provide relevant information will be:

• Index of mention of the online classes in the weaknesses section;
• Motivation in class activities and home activities expressed in qualitative surveys;
• Motivation in online classes expressed in qualitative surveys;
• Relationship between positive extreme marks and negative extreme marks of the responses related to classes in online format.

4.1. Qualitative Assessment

Table 1. Common elements in the strengths section mentioned by the students in the BLA survey (¹ Elements related to FL).

Element	Category	Mentions
Videos 1	SDB	39
Continuous assessment	SAA	34
Online classes development and adaptation 1	SPA	20
Guidance/proximity/accessibility/care of the teacher 1	STA	17
Documentation uploaded in Moodle 1	SDA	12
Division of topics 1	SPB	9
Explanations of the teacher	STA	7
Exercises solved in class 1	SPA	5
Subject organization 1	SPB	5
No coursework	SPB	3
Class activities/dynamics 1	SPA	3
Formulae given in exams	SAB	2
Students’ participation in online classes 1	SSA	2
Adequate workload 1	SPB	2
Combination of theory and practice 1	SPB	2
Real cases used as example	SPB	2
Good schedule (requires to be active)	SPC	2
Applicability of the subject to real cases	SCA	2
Flipped class 1	SPB	2
Quality of teachers	STA	2

and

Table 2. Common elements in the weaknesses section mentioned by the students in the BLA survey (¹ Elements related to FL).

Element	Category	Mentions
Too much content to be assimilated in such a short time	WCC	14
Exams are too long/short of time/stressful	WAB	12
Online classes	WPA	8
Classes schedule at 8:00 am	WPC	7
Lack of time for doubts in class 1	WPA	6
Too much theory/too few practice 1	WPB	5
Exams too close in time	WAB	5
More topics should be included in the syllabus	WCA	5
Rhythm is too high 1	WPA	3
Lack of coordination between groups	WPA	2
Online classes software	WPD	2
Lack of continuous assessment/Grade depends too much on the final exam	WAA	2
Some material is only posted in English and Catalan 1	WDA	2
No questions allowed in exams	WAB	2
Student feels difficult to ask at class 1	WTA	2
Incomplete notes 1	WDA	2
Organization and continuity of the topics 1	WPB	2

show the common elements (those that are mentioned by two or more participants) of the strengths and weaknesses sections of the BLA survey.

Table 3. Number of positive extreme marks (9 or 10).

Element	Category	Marks ≥ 9
Videos 1	SDB	11
Continuous assessment	SAA	6
Guidance/proximity/accessibility/care of the teacher 1	STA	6
Explanations of the teacher	STA	4
Class activities/dynamics 1	SPA	2
Online classes development and adaptation 1,2	SPA	2
Real cases used as example	SPB	2
Attendance counts	SAA	1
Formulae given in exams	SAB	1
Applicability of the subject to real cases	SCA	1
Documentation uploaded in Moodle 1	SDA	1
Good schedule (requires to be active)	SPC	1
Division of topics 1	SPB	1
Exercises solved in class 1	SPA	1
Subject organization 1	SPB	1
Use of 3 languages	SPA	1
Teacher concern to adapt to COVID crisis	STA	1

¹ Elements related to FL ² Elements related to the online format of classes.

and

Table 4. Number of negative extreme marks (1 or 2).

Element	Category	Marks ≤ 2
Classes schedule at 8:00 am	WPC	2
Exams	WAA	1
Lack of continuous assessment/Grade depends too much on the final exam	WAA	1
Exams are too long/short of time/stressful	WAB	1
No questions allowed in exams	WAB	1
Too much theory/too few practice 1	WPB	1
Organization and continuity of the topics 1	WPB	1
Online classes 2	WPA	1

¹ Elements related to FL; ² Elements related to online format of classes.

show the elements with extreme positive and negative marks written in the BLA survey by the students. Elements related to FL and to the online format of classes are indicated as they are used to set the value of the parameters defined at the beginning of Section 4 related to extreme marks. The element “Online classes development and adaptation” is related to both FL and online format, and the 2 extreme marks have been counted for both parameters.

In

Table 5. Strengths by subcategory mentioned by the students in the BLA survey.

Subcategory	Description	Mentions	Index of Mention	Category Weight
SAA	Assessment method	35	72.92%	19.35%
SAB	Assessment development/organization	3	6.25%	1.61%
SCA	Content of the course	4	8.33%	2.15%
SCB	Course approach	0	0.00%	0.00%
SCC	Time/content rate	0	0.00%	0.00%
SDA	Documents	13	27.08%	6.99%
SDB	Videos	39	81.25%	20.97%
SPA	Class development	27	56.25%	17.20%
SPB	System and methodology	21	43.75%	13.44%
SPC	Schedule	2	4.17%	1.08%
SPD	Technology use	1	2.08%	0.54%
SSA	Students’ motivation	2	4.17%	1.08%
SSB	Students’ profit	1	2.08%	0.54%
STA	Teachers	23	47.92%	15.05%

and

Table 6. Weaknesses by subcategory mentioned by the students in the BLA survey.

Subcategory	Description	Mentions	Index of Mention	Category Weight
WAA	Assessment method	3	4.17%	3.03%
WAB	Assessment development/organization	21	37.50%	21.21%
WCA	Content of the course	6	12.50%	6.06%
WCB	Course approach	0	0.00%	0.00%
WCC	Time/content rate	14	29.17%	14.14%
WDA	Documents	5	10.42%	5.05%
WDB	Videos	2	4.17%	2.02%
WPA	Class development	23	41.67%	23.23%
WPB	System and methodology	10	20.83%	10.10%
WPC	Schedule	8	16.67%	8.08%
WPD	Technology use	3	6.25%	3.03%
WSA	Students’ motivation	1	2.08%	1.01%
WSB	Students’ profit	0	0.00%	0.00%
WTA	Teachers	3	4.17%	3.03%

all responses at the strengths and weaknesses sections are classified by subcategories. These tables are sorted according to the alphabetic order of the subcategory and given a color code for better comparison between the data in the tables.

The index of mention is the rate between the students that mention one subcategory and the total number of students (48), while the category weight is the rate between the number of mentions of the category and the total number of mentions.

For a better interpretation of results, BLA survey asks the students to value with a grade from 1 to 10 each one of the strengths and weaknesses, even though some students did not write any grade. These grades can be found in

Table 7. Average rating and SD for both strengths and weaknesses grouped by subcategory.

		Strengths		Weaknesses
Subcategory	Description	M	SD	M	SD
AA	Assessment method	8.875	1.102	2.500	0.500
AB	Assessment development/organization	8.500	0.500	3.917	2.050
CA	Content of the course	9.000	0.000	5.500	0.500
CB	Course approach	-	-	-	-
CC	Time/content rate	-	-	4.500	0.957
DA	Documents	8.200	0.980	3.000	0.000
DB	Videos	9.267	1.123	-	-
PA	Class development	9.125	0.781	4.250	1.920
PB	System and methodology	9.333	0.943	4.000	2.098
PC	Schedule	9.000	0.000	2.667	1.700
PD	Technology use	-	-	4.000	1.000
SA	Students’ motivation	6.000	0.000	-	-
SB	Students’ profit	-	-	-	-
TA	Teachers	9.556	0.643	5.000	0.000

grouped by subcategory. Blank values mean that no student valued with a grade any response of that subcategory.

Table 8. Average rating, standard deviation and 95% confidence interval when combining strengths and weaknesses rates of the qualitative survey grouped by subcategory.

Subcategory	Description	n	M	SD	μ	X1	X2
AA	Assessment method	14	7.964	2.460	1.289	6.68	9.25
AB	Assessment development/organization	8	5.063	2.674	1.853	3.21	6.92
CA	Content of the course	3	6.667	1.700	1.923	4.74	8.59
CB	Course approach	-	-	-	-	-	-
CC	Time/content rate	6	4.500	0.957	0.766	3.73	5.27
DA	Documents	6	7.333	2.134	1.708	5.63	9.04
DB	Videos	15	9.267	1.123	0.569	8.70	9.84
PA	Class development	12	7.500	2.630	1.488	6.01	8.99
PB	System and methodology	11	6.909	3.088	1.825	5.08	8.73
PC	Schedule	4	4.250	3.112	3.050	1.20	7.30
PD	Technology use	2	4.000	1.000	1.386	2.61	5.39
SA	Students’ motivation	1	6.000	0.000	0.000	6.00	6.00
SB	Students’ profit	-	-	-	-	-	-
TA	Teachers	10	9.100	1.497	0.928	8.17	10.03

shows the average of grades for each subcategory considering in the same row all answers with grade for each one making no difference between strengths and weaknesses. Standard deviation is calculated to get the value of the 95% confidence interval μ, to get the lower and upper endpoint values for this degree of confidence.

4.2. Quantitative Assessment

Quantitative surveys to measure students’ motivation and satisfaction were also conducted after the course. For the purpose of this article,

Table 9. Answers regarding satisfaction respect several aspects of the course (n = 47).

Aspect and Subcategory Related		Likert Scale 1 n (%)					Parameters
		1	2	3	4	5	M	SD	Skw	Kme
Continuous assessment	AA	0 (0.0)	2 (4.3)	4 (8.5)	16 (34.0)	25 (53.2)	4.362	0.810	−1.228	1.221
Exams	AB	2 (4.3)	6 (12.8)	9 (19.1)	24 (51.1)	6 (12.8)	3.553	1.007	−0.772	0.215
Course approach	CB	0 (0.0)	1 (2.1)	8 (17.0)	28 (59.6)	10 (21.3)	4.000	0.684	−0.399	0.476
Time/content rate	CC	4 (8.5)	11 (23.4)	17 (36.2)	11 (23.4)	4 (8.5)	3.000	1.072	0.000	−0.510
Course documents	DA	0 (0.0)	0 (0.0)	5 (10.6)	20 (42.6)	22 (46.8)	4.362	0.666	−0.565	−0.652
Content of the videos	DB	1 (2.1)	0 (0.0)	8 (17.0)	18 (38.3)	20 (42.6)	4.191	0.866	−1.165	2.203
Organization of the videos	DB	1 (2.2)	1 (2.2)	6 (13.0)	18 (39.1)	20 (43.5)	4.196	0.900	−1.289	2.284
Quality of the videos	DB	1 (2.1)	3 (6.4)	4 (8.5)	15 (31.9)	24 (51.1)	4.234	0.994	−1.391	1.699
Usefulness of the videos	DB	1 (2.1)	1 (2.1)	8 (17.0)	10 (21.3)	27 (57.4)	4.298	0.966	−1.333	1.582
Online classes	PA	4 (8.9)	3 (6.7)	9 (20.0)	14 (31.1)	15 (33.3)	3.733	1.236	−0.824	−0.109
Exercises done in class	PA	0 (0.0)	3 (6.4)	15 (31.9)	14 (29.8)	15 (31.9)	3.872	0.937	−0.210	−1.054
Doubt solving	PA	0 (0.0)	3 (6.4)	4 (8.5)	24 (51.1)	16 (34.0)	4.128	0.815	−0.945	0.943
Theory and practice integration	PB/CA	0 (0.0)	3 (6.4)	14 (29.8)	18 (38.3)	12 (25.5)	3.830	0.883	−0.218	−0.755
Flipped class	PB	1 (2.4)	1 (2.4)	18 (42.9)	14 (33.3)	8 (19.0)	3.643	0.895	−0.234	0.395
Teachers’ explanations	TA	1 (2.1)	0 (0.0)	6 (12.8)	20 (42.6)	20 (42.6)	4.234	0.831	−1.350	3.287
Teacher-student interaction	TA	0 (0.0)	0 (0.0)	7 (14.9)	14 (29.8)	26 (55.3)	4.404	0.734	−0.797	−0.682

¹ Likert Scale from 1 = “Not at all satisfied” to 5 = “Extremely satisfied”.

,

Table 10. Answers regarding students’ motivation for different activities (n = 47). Questions are made in the format “The activity motivates me”.

Activity and Subcategory Related		Likert Scale 1 n (%)					Parameters
		1	2	3	4	5	M	SD	Skw	Kme
Attend to face-to-face classes	SA	3 (6.52)	4 (8.70)	10 (21.74)	9 (19.57)	20 (43.48)	3.848	1.251	−0.778	−0.385
Listen to teachers’ explanations	SA	3 (6.52)	0 (0.00)	10 (21.74)	14 (30.43)	19 (41.30)	4.000	1.103	−1.165	1.249
Do exercises at class	SA	1 (2.22)	1 (2.22)	12 (26.67)	17 (37.78)	14 (31.11)	3.933	0.929	−0.700	0.644
Solve doubts at class	SA	2 (4.26)	1 (2.13)	14 (29.79)	12 (25.53)	18 (38.30)	3.915	1.069	−0.772	0.300
Study notes at home	SA	4 (8.70)	2 (4.35)	18 (39.13)	12 (26.09)	10 (21.74)	3.478	1.137	−0.479	−0.073
Watch to videos at home	SA	4 (9.09)	3 (6.82)	12 (27.27)	12 (27.27)	13 (29.55)	3.614	1.229	−0.630	−0.342
Continuous assessment	SA	2 (4.35)	0 (0.00)	12 (26.09)	12 (26.09)	20 (43.48)	4.043	1.042	−1.010	0.970
Attend to online classes	SA	9 (19.57)	9 (19.57)	10 (21.74)	12 (26.09)	6 (13.04)	2.935	1.325	−0.048	−1.182

¹ Likert Scale from 1 = “Strongly disagree” to 5 = “Strongly agree”.

and

Table 11. Other questions in the survey related to subcategories (n = 47).

Question and Subcategory Related		Likert Scale 1 n (%)					Parameters
		1	2	3	4	5	M	SD	Skw	Kme
Structural knowledge is useful for architects	CA	0 (0.0)	1 (2.2)	4 (8.7)	20 (43.5)	21 (45.7)	4.326	0.724	−0.927	0.877
I will use technical knowledge in my professional life	CA/SA	4 (8.5)	5 (10.6)	11 (23.4)	14 (29.8)	13 (27.7)	3.574	1.233	−0.580	−0.521
The course has met my expectations	SB	0 (0.0)	2 (4.3)	9 (19.1)	23 (48.9)	13 (27.7)	4.000	0.799	−0.501	−0.061
The course has met my needs	SB	1 (2.2)	4 (8.7)	13 (28.3)	16 (34.8)	12 (26.1)	3.739	1.009	−0.476	−0.234
I learned a lot with this structural course	SB	2 (4.3)	0 (0.0)	8 (17.0)	20 (42.6)	17 (36.2)	4.064	0.954	−1.302	2.501

¹ Likert Scale from 1 = “Strongly disagree” to 5 = “Strongly agree”.

show the most significative results. The sample of the quantitative surveys is 47 students. In these tables the number of answers for each value in the Likert scale is shown (n is the number of answers and the value in brackets is the percentage of responses obtained for each value), as well as the mean (M), the standard deviation (SD), the skewness (S_kw) and the Kurtosis (K_me). Each question is related to one of the subcategories, in a way that will let combining results and getting global values grouped by subcategories that are shown in

Table 12. Statistical analysis per subcategory obtained from quantitative assessment (n = 47).

Subcategory and Description		Parameters				95% Confidence Interval
		M	SD	Skw	Kme	μ	X1	X2
AA	Assessment method	4.362	0.810	−1.228	1.221	0.232	4.130	4.593
AB	Assessment development/organization	3.553	1.007	−0.772	0.215	0.288	3.265	3.841
CA	Content of the course	3.904	0.698	−0.371	−0.432	0.200	3.705	4.104
CB	Course approach	4.000	0.684	−0.399	0.476	0.196	3.804	4.196
CC	Time/content rate	3.000	1.072	0.000	−0.510	0.306	2.694	3.306
DA	Documents	4.362	0.666	−0.565	−0.652	0.190	4.171	4.552
DB	Videos	4.230	0.872	−1.424	2.709	0.249	3.981	4.480
PA	Class development	3.915	0.745	−0.467	−0.149	0.213	3.702	4.128
PB	System and methodology	3.755	0.798	−0.145	−0.538	0.228	3.527	3.983
PC	Schedule	-	-	-	-	-	-	-
PD	Technology use	-	-	-	-	-	-	-
SA	Students’ motivation	3.712	0.793	−1.342	2.572	0.227	3.485	3.938
SB	Students’ profit	3.940	0.823	−0.654	0.204	0.235	3.704	4.175
TA	Teachers	4.319	0.631	−0.630	−0.556	0.180	4.139	4.500

.

Answers about satisfaction are gathered in Table 9, while the ones regarding motivation are shown in Table 10. Other closed questions included in the qualitative surveys related to some of the subcategories are exposed in Table 11. All questions should be answered in a Likert scale from 1 to 5, making possible to manage and group the data obtained from the different tables.

Table 12 shows the results shown in Table 9, Table 10 and Table 11 combined according to the related subcategory. Values are obtained from the average. The statistical parameters shown are calculated for n = 47 (the number of students that answered the survey) using the average grade of the answers of each student related to each category. Standard deviation is calculated to get the value of the 95% confidence interval μ to get the lower and upper endpoint values for this degree of confidence.

5. Discussion

Flipped learning (FL) is a blended learning methodology that combines face-to-face time in class with activities out of class, that in most cases are related to technology, like the Learning Management Systems (LMS) and the support of ad hoc videos, previous recordings or other documents that can be found online. FL use is blooming and there is recent extensive research on it, that underline benefits such as flexibility for the students, increased motivation, increased participation, improvement in interaction between students and between students and teachers, in student engagement, in professional skills and in academic results, among others.

Some of these benefits seem to be strongly related to face-to-face learning (e.g., interaction between students [93,94], interaction between student and teacher [95,96], and others (motivation, participation, engagement) could be linked to class activities with the support of the teacher (complex exercise solving, doubt solving, etc.). Improving the quality of face-to face sessions is a fundamental part of the FL system and the main reason to move the lecture out of the class time [5]. The COVID-19 crisis outbreak implied the removal of the face-to-face sessions of the course and led to a sudden adaptation of the model to completely online.

The aim of this study is to analyze this conversion between class formats to contribute to check the validity of FL in blended education, as well to confirm the benefits of FL related to face-to-face classes. In this context, two research questions have been stated:

RQ1

Has the FL model used in the Concrete and Steel Structures subject adapted properly from a face-to-face format to the online format?

RQ2

Do the students of the Concrete and Steel Structures subject negatively perceive the change to the online format?

To answer RQ1, and according to the results shown in the previous section, obtained from the data analysis, the value of the six relevant parameters listed in Section 4 is:

• Index of mention (IM) of the proper adaptation of the course to the online format in the strengths section: 41.66%. It is the third most cited element, after videos (IM = 81.25%) and continuous assessment (IM = 70.83%);
• Proportion of responses related with the FL in the strengths section: 65.01% (121 responses out of 186 total responses);
• Proportion of responses related with the FL in the weaknesses section: 27.27% (27 out of 99 responses);
• Positive and negative extreme marks of the responses related to FL: 25 extreme positive marks (58.14% of the 43 total positive extreme marks) and 2 extreme negative marks (22.22% of the 11 total negative extreme marks);
• Lower endpoint value for 95% of confidence of the mark in the BLA for the categories related to FL. In a scale from 1 to 10, this value is 8.70 for category DB (videos), 6.01 for category PA (class development) and 5.08 for category PB (system and methodology);
• Lower endpoint value for 95% of confidence of the responses in the quantitative survey for the categories related to FL. In a scale from 1 to 5, this value is 3.98 for category DB (videos), 3.70 for category PA (class development) and 3.53 for category PB (system and methodology).

In addition, an external relevant parameter regarding RQ1 can be added to this analysis. In the ordinary survey conducted at the end of the academic year in La Salle for 46 subjects of the Architecture undergraduate studies, the Concrete and Steel Structures subject is highly rated by the students when asked about the adaptation of the subject to the online learning format (4.55 over 5), reaching the fifth position, and the second place when considering only the subjects that have more than five responses.

Similarly, the value of each one of the four relevant parameters listed in Section 4 to answer RQ2 is shown below:

• Index of mention of the online classes in the weaknesses section: 16.67%. It is the third most cited element, after the ratio content/time (IM = 29.16%) and time given for exams (IM = 25.00%);
• Motivation in class activities and home activities expressed in qualitative surveys: in a scale from 1 to 5, average rating of motivation for class activities is 4.00 (listen to teachers’ explanations), 3.93 (do exercises at class) and 3.92 (solve doubts at class). The rating for home activities is 3.478 (study notes at home) and 3.614 (watch to videos at home);
• Motivation in face-to-face and online classes expressed in qualitative surveys: in a scale from 1 to 5, average rating of motivation for attending face-to face classes is 3.85 while the value for attending online classes is 2.935;
• Positive and negative extreme marks of the responses related to classes in online format: two extreme positive marks (4.65% of the 43 total positive extreme marks) and one extreme negative mark (11.11% of the 11 total negative extreme marks).

Although it is not related to the research questions stated, we think that it is important to underline that even though students do not like the exams (3.55 rating in satisfaction quantitative survey and 4.31 in qualitative survey), they answer that the most motivating activity of the course for them is being assessed continuously (4.04 over 5 in quantitative surveys and 7.96 over 10 in the average for the subcategory in qualitative surveys). Continuous assessment has been proven to be crucial for the involvement of the students in the FL methodology during the whole course [97] and can be determinant in the success or failure of FL implementation.

6. Conclusions

According to the results obtained, and regarding RQ1, it can be deducted from the six relevant parameters analyzed that the FL model used in the Concrete and Steel subject has properly adapted from a face-to-face format to the online format.

Therefore, this study reveals that, even the students rate lowly their motivation on the flipped classroom, the FL model designed for the Concrete and Steel Structures subject has been proven to work properly for both face-to-face and online classes and can be used for hybrid situations where the course needs to be developed mixing face-to-face and online classes, as it is currently happening due to the pandemic crisis.

Concerning RQ2, and according to the values of the four parameters used, it can be inferred that the students of the Concrete and Steel Structures subject perceive the change to the online format as negative.

Therefore, the research supports previous findings in FL research [7], related to the benefits linked to face-on-face classes [94], like increased motivation, participation and engagement [95]. One of the fundamentals of FL is to enhance the quality of the class time, where students can interact with other students and with the teacher, thanks to the reallocation of the lecture out of the class [5]. Online classes restrain these benefits, and even FL efficiency in online formats has been proven [98], results reveal that students who are getting the advantages of face-to-face classes in an FL model and are forced to move online show strong opposition to the online format.

This study has several known limitations. The experiment was only developed in a specific geographic and academic context, and only one course was considered. There is no control group to check statistically the validity of the findings exposed. The size of the sample, despite that it can be justified and supported by previous studies, can lead to unbalanced results. Therefore, the evidence must be treated with caution as it cannot be generalized to the world population as a whole.

Similar studies in other regions and academic contexts can help to amend these limitations and compare the results to find indicators to be used to verify the improvement of learning, as replication of the conditions of experiment is not possible because of its nature related to the outbreak of the COVID-19 crisis.