The Effects of Blended Learning on the Performance of Engineering Students in Mathematical Modeling

Abstract

This paper presents the implementation of an active learning methodology known as blended learning in an ordinary differential equations (ODE) course for engineering students. Our purpose was to evaluate the effect of b-learning on students’ mathematical modeling performance. To this end, synchronous and asynchronous activities were made available to the students as face-to-face and remote learning sessions, in which the experience acquired by students during the sanitary isolation due to COVID-19 was crucial. Benjamin Bloom’s cognitive domain taxonomy was used to design the present didactic proposal. Results show that the students moved upward from the lower knowledge and understanding taxonomical levels, to the upper analysis and application levels, as they learned systems modeling using ODEs.

1. Introduction

Training university students is the main responsibility of higher education institutions in any country and it is, at the same time, a valuable opportunity to contribute to the development of better human beings. Our current 21st-century society needs young students in science, technology, engineering, and mathematics (STEM) to acquire specific knowledge and skills or abilities that commit them to social and economic progress. To achieve a defined university profile, didactic research in the engineering field uses several active learning approaches supported by existing digital technologies to study the fundamentals of a discipline from different perspectives and strengthen the students’ theoretical and practical understanding of the field.

Collaborative, problem-based, project-oriented, and competency-based learning, as well as the flipped classroom and gamification techniques, among others, are some of the most commonly used active learning approaches in mathematics education and engineering training [1,2,3,4,5,6,7,8]. Each approach or methodology has particular characteristics, but all share the notion that the student is at the center of the learning process. For example, the problem-based and collaborative approaches encourage teamwork among members with different skills and share responsibility for achieving a goal [9].

The interactions among these different personal perspectives can be enabled using a combination of electronic media (online tools operating synchronously or asynchronously) and face-to-face interaction, which gives rise to an educational practice known as blended learning or b-learning. B-learning requires the student to use learning styles other than their preferred one and to adapt to the different activities and materials available for the subject under study [10].

Currently, most STEM programs are designed to work best in the classroom, where discussing analytical methods and laboratory practices are some of the learning activities. During the partial lockdown due to the COVID-19 pandemic, the performance of these activities was limited; however, digital technologies, such as online software platforms and virtual laboratories, played a very important role as mediators for students to adapt more quickly to these circumstances [11,12,13,14,15,16,17,18].

1.1. The B-Learning Approach

B-learning is a pedagogical approach to teaching and learning that can potentially harmonize the best practices normally conducted in the classroom with different technological applications [19,20]. This procedure requires an adequate interaction environment involving direct or remote communication for students to use different learning styles [21]. In b-learning, different virtual media are used to present interactive content that favors interaction and enhances the STEM student’s learning experience, for example, images or diagrams (infographics or graphic organizers), animations, videos, software, online platforms to promote collaborative learning [9], and virtual laboratories [22].

This approach can have multiple benefits for student performance, even more than a completely face-to-face learning environment because the course contents and activities are not restricted to the classroom, and students can study these contents and carry out the practical tasks in their schedule. On the other hand, as [23] has been pointed out, diagnostic conversations or personal feedback in face-to-face environments are impractical and comments received within an online environment are found to be more pleasant for students; therefore, b-learning increases the interest of students in participating in learning activities.

The relevance and pertinence of adopting a b-learning approach became evident during the COVID-19 lockdown, and most college students are already using their cell phones, tablets, or computers to carry out almost all of their academic activities. Unfortunately, the adjustments made to bring teaching to the virtual environment had to be made in haste to mitigate the negative effects of the lockdown on the educational process. Now that the health emergency has subdued, the empirical learning of the experience must be used to implement b-learning systematically with planned and significant activities to provide the student with the experience of true active learning. Consequently, the teaching and learning processes currently used in mathematics training within university-level engineering programs must be rethought.

According to [24], the promise of better learning methods is possible if the teachers’ resistance to the use of digital technology is overcome, as well as their inertia to value face-to-face interaction over the potential benefits of online learning or a combination of both. Therefore, teachers should enthusiastically assume the responsibility and commitment to redesign courses to incorporate content and activities with a b-learning approach and provide students with an increased and more varied learning experience than that available only online or only in the classroom.

1.2. Taxonomy of the Cognitive Domain

Engineers use their skills, competencies, knowledge, and techniques in their professional practice to solve problems specific to their discipline. One way to measure and assess how these skills, competencies, knowledge, and techniques are acquired, developed, or strengthened through the teaching and learning process carried out in college uses the taxonomy proposed by Bloom [25], which classifies the achievement of educational objectives at six different levels: knowledge, understanding, application, analysis, synthesis, and evaluation (see Figure 1).

According to Bloom, knowledge is the most basic achievement level, which includes behaviors and situations where ideas, content, or phenomena are recognized or recalled. At this level, a student’s response to a test situation should be similar to what it was during the original learning situation. At the understanding level, if students are confronted with oral, written, verbal, or symbolic communication, they are expected to know what is being communicated and to use the concepts or ideas contained in the information. At the application level, in a situation where no solution is specified, students are expected to correctly use the appropriate abstraction to solve a given problem.

At the analysis level, the student separates the information into its constituent parts, identifying the relationships, interactions, or connections between them and recognizing their structure. At the synthesis level, the student activates their most creative part, refers to many different sources, and builds a scheme or pattern that was not clear before to solve a problem within some theoretical or methodological framework. Finally, at the highest cognitive domain level, evaluation, the student is expected to make quantitative or qualitative value judgments using specific criteria and parameters to measure the accuracy or effectiveness of information, methods, or solutions.

Bloom’s taxonomy is idoneous for the design of mathematics learning activities aimed at engineering students. For example, to analyze a system’s dynamics or phenomenon using first-order ordinary differential equations (ODEs), the student requires knowledge and understanding of the characteristics and other important aspects of ODEs. For this purpose, a b-learning environment is favorable for combining both remote and classroom-based activities. The challenge is to identify the most adequate and compatible strategies and activities to meet the goals of each educational situation.

1.3. Application of ODEs to Mathematical Modeling

In a mathematical modeling process, students move higher through the different cognitive domain levels. In the case of ODEs, this means knowing the relevant parameters and variables and their dimensional analysis [26], understanding the meaning of each term in the ODE [27], applying methods to obtain the analytical solution and its corresponding error analysis [28], and interpreting the solution [29]. Implementing approaches, such as b-learning, can support students as they experience these complex cognitive processes [30] and improve the comprehension and use of mathematical concepts [31].

When engineering students take on the challenge of modeling actual systems based on ordinary life or related to their professional field, they become immersed in a truly significant learning process [32] in which mediators are essential for generating an environment conducive to the formation and discussion of new concepts [33,34]. In such an environment, students regulate their learning themselves while they also spend time on teacher–student interaction during face-to-face activities, for example, counseling using the board and support for carrying out projects, both of which are indispensable in engineering contexts [23].

1.4. Theoretical Basis

The design and implementation of this study were based on Vygotsky’s sociocultural theory of cognitive development, whose fundamental hypothesis states that higher mental functions, such as application and analysis, are socially constructed and culturally transmitted. In an educational context, changing the thinking tools available to the student promotes the emergence of a radically different mental structure [35,36].

The concept of the zone of proximal development (ZPD) is one of the most important contributions of Vygotsky’s theory to education. This concept stems from the discussion about the relationship between development and learning. Vygotsky recognizes the existence of a relationship between the maturity of an individual’s organism and their ability to learn certain subjects. However, he adds that two levels of development must be considered in said relationship between maturity and learning skills: current development and potential development. Current development is determined by the individual’s ability to solve problems independently. Potential development is determined by their ability to solve problems in collaboration with a more capable partner or under the guidance of an adult.

Vygotsky defines ZPD as the distance between an individual’s current developmental level, as determined by the ability to independently solve a problem, and the potential developmental level, as determined by their ability to solve problems under the guidance of an adult or in collaboration with other, more capable companions. The author also considers that an essential aspect of learning is that it creates the ZPD; that is, learning awakens a variety of developmental processes capable of operating only when the individual interacts with other people in their environment and collaborates with their peers. This notion highlights, on the one hand, the importance of studying the individual when immersed in their social environment and interacting with it and, on the other hand, the interdependence between the individual’s development process and the resources that the social environment provides for such development.

Although Vygotsky’s ZPD theory emphasizes the support of a more capable person, there are more and more researchers who consider that there may also be other factors as efficient as ZPD that facilitate the appropriation of knowledge by the student. For example, a structured environment can guide the student toward the use of elements that are new to them but accessible in their ZPD. Similarly, it has been stated that, within the ZPD, impersonal feedback from the material with which the individual interacts can be as effective in promoting development as interpersonal support.

This emphasis on the role of the ZPD environment is particularly promising when considering school instruction where, due to the large number of students that often make up classes, it is not always easy for the teacher to establish an interpersonal relationship with each person and provide each one with the necessary feedback. Therefore, the idea of investigating the possibility of structuring the school environment in a way that helps the student to increase their understanding potential within their ZPD for specific topics, particularly mathematical ideas, is very attractive. Recent studies have investigated this possibility, obtaining encouraging results. However, it was observed that, although structuring the environment and the activities was crucial to optimize the ZPD’s potential, working in dyads or in groups, as well as teacher interventions to provide support and guidance, continued to be essential elements of the learning process.

2. Methodology

This study aimed to evaluate the effect of b-learning on student performance in constructing and resolving a mathematical model using first-order differential equations. For this purpose, we recruited a small group of students enrolled in an ODE course in a university-level engineering program (n = 19). The teacher was not involved in the sample selection; therefore, since the desired randomness failed to be achieved, a pre-experimental design including pre-test and post-test was implemented. The sample was considered time-dependent in the experimental design, and its members could easily work in a digital technology environment.

This research was carried out at the beginning of 2022, during the transition from virtual to face-to-face activities at an institution with different venues and 62 ODE courses. For practical reasons, the study focused on a single campus with nine ODE groups and 191 registered students (the population under study). In addition, the group of students (sample) was assigned to the teacher, and its size agreed with the literature, which indicated that the b-learning approach worked better with small groups [23,31]. The results of this pilot study will allow for obtaining information to implement it with more students from the study’s population.

2.1. Research Question

Does the use of b-learning improve student performance in mathematical modeling?

2.2. Hypothesis

To answer the research question, this paper proposes the following hypotheses.

H₀. 

The use of b-learning does not affect student performance in mathematical modeling.

H_A. 

The use of b-learning affects student performance in mathematical modeling.

2.3. Goal

The purpose of the present study was to evaluate the effect of b-learning on the performance of engineering students in constructing and resolving a mathematical model using first-order differential equations. To this end, we designed remote and face-to-face learning activities to analyze a system or phenomenon. Bloom’s taxonomy of cognitive domain levels was used in the design of these activities.

2.4. Implementation of the Didactic Proposal

According to the b-learning approach, the learning activities were designed to be carried out in two ways: online (synchronous or asynchronous) and face-to-face. The asynchronous activities were carried out via Google Classroom, where educational content such as reading materials, exercises, videos, and tutorials was available. The synchronous activities consisted of Google Meet videoconferences supported by a virtual whiteboard where the procedures of different methods to obtain analytical solutions to ODEs were presented; in addition, tutorials on Matlab’s online platform taught students how to obtain the symbolic and numerical solutions of an ODE. On the other hand, the face-to-face activities focused on highlighting important aspects of the subject, such as the qualitative aspects of mathematical modeling, and clarifying doubts related to the analytical solution or software use.

The content of all the activities was designed to favor interaction among the students and with the teacher. At the same time, they moved upward through the cognitive domain levels [25]; that is, from the most basic levels, where the student must know and understand the definitions, classification, terminology, and symbology of first-order ODEs, to activities focused on the modeling itself, where the analysis, argumentation, relationship, debate, and discussion of the subject were carried out in a collaborative environment [37]. Finally, we considered different situations where a simple mathematical model using first-order ODEs can be built for the central activity, such as exponential growth, the mixing of substances, temperature changes, and electrical circuits.

The set of didactic activities, see

Table 1. Organization of learning activities in different remote and face-to-face sessions.

Session	Type	Learning Activities
1	Synchronous remote	Presentation of subject, purpose, and support materials. Diagnostic evaluation. Formation of work teams.
2	Asynchronous remote	Readings, videos, and activities focused on the principles of ODEs. Matlab R2022a tutorials (algebra, graphs, calculus, and ODEs). Readings, videos, and exercises to solve ODEs analytically.
3	Synchronous remote	Advice regarding analytical method procedures to solve ODEs.
4	Asynchronous remote	Administration of Questionnaire 1. Readings and videos of exponential growth systems modeling.
5	Synchronous remote	Development of a mathematical model for a system and its analytical solution. Numerical solution of a system’s mathematical model on the online software platform.
6	Face-to-face	Analytical and numerical comparison of solutions to the system’s dynamics. Random assignment of exercises from different situations to build a mathematical model using first-order ODEs.
7	Asynchronous remote	Construction of a mathematical model and an analytical solution. Obtention of the numerical solution.
8	Face-to-face	Drafting of a written report. Presentation and discussion of results. Administration of the rubric.
9	Asynchronous remote	Administration of Questionnaire 2. Administration of the survey.

, carried out remotely and in person, were organized in different sessions as follows:

During the asynchronous remote sessions, the students worked independently and in teams, and the instructor guided the synchronous remote sessions through videoconferences. Following the ZPD approach, face-to-face sessions were also held [35,36]. It is worth mentioning that, to motivate students’ interest in modeling, activities 5 and 6 considered the current context and focused on the evolution of the number of people infected by SARS-CoV-2 in the city where their university is located. Therefore, the task assigned to the teams consisted of systems of interest to engineering students, such as growth and decay, carbon dating, Newton’s law (cooling/warming), and series circuits.

Questionnaires were included as part of the observation and measurement instruments to assess the effect of the learning activities; these instruments are essential in asynchronous online activities because they can make a difference in student learning. It has been reported that online questionnaires, together with support material, can homogenize or level the knowledge acquired by students during their first engineering cycles [23]. Other data necessary for the present study were obtained using a rubric and a survey (see

Table 2. Observation and measurement instruments designed for the study.

Instrument	Contents	Purpose
Diagnostic Evaluation	Items/questions regarding the definition, domain, differentiation, and integration of one-variable functions.	Verify the homogeneity of prior knowledge possessed by the sample using converging questions (multiple choice).
Questionnaire 1 (Pre-test)	Items related to ODE definitions, symbols, classification, use, structure, and solution methods.	Measuring the extent of ODE knowledge and understanding (cognitive domain levels 1 and 2).
Questionnaire 2 (Post-test)	Items related to the meaning of ODE terms, rules, and procedures for a mathematical model.	Measuring how effectively ODEs are applied and used to analyze a system (cognitive domain levels 3 and 4).
Rubric	Five criteria with four levels.	Qualitative evaluation of hard and soft skills development.
Survey	Items with responses on a six-point Likert scale.	Qualitative evaluation of student satisfaction with the b-learning approach adopted in the activities.

). The rubric has five criteria: (1) the mathematical model, (2) the analytical solution, (3) the numerical solution, (4) the written report of the work completed, and (5) the verbal presentation of the work. The answers have four levels on the Likert scale (null, regular, well, and excellent). The first three criteria evaluate the development of hard skills, while the last two evaluate the development of soft skills.

Regarding Questionnaire 1, five questions and five exercises were integrated into the pre-test. (1) What is a differential equation (DE)? (2) What types of differential equations exist? (3) What does the order of a DE mean? (4) What is a condition of a non-linear DE? (5) What type of solution can a DE have? In addition, three exercises were given to check the solutions of DE and two to obtain the general or particular solution of DE. The pre-test content was designed to measure the student’s knowledge and understanding of the basics of a DE. The five questions measure knowledge of a DE’s concept, classification, and type of solution. In contrast, the five exercises measure understanding the structure and the method to solve a DE. The pre-test is located at levels 1 and 2 of Bloom’s taxonomy.

Regarding Questionnaire 2, six questions were integrated into the post-test. (1) How can a DE be applied? (2) What methods do you know to solve a DE? (3) Did you understand the meaning of each term that appears in the DE? (4) What do you understand by mathematical model? (5) How useful is a mathematical model? (6) What tools do you know to analyze a mathematical model? The post-test content was designed to measure the student’s ability to apply the DE in a basic mathematical model and to apply solution techniques for a DE; likewise, for the analysis of the meaning of the terms of a DE and the response or behavior of the modeled system. The post-test is located at levels 3 and 4 of Bloom’s taxonomy.

The data and information obtained by the research instruments were processed and analyzed by quantitative and qualitative methods [38]. The quantitative analysis was performed during the diagnostic evaluation using a normality test, and during the pre-test and post-test using descriptive statistics. A hypothesis test compared the means of paired samples using Student’s t-test in OriginPro 2022 software (The University of Guadalajara, Guadalajara, Mexico). The qualitative analysis consisted of a rubric evaluating student performance in the application of ODEs to the modeling and analysis of a system’s response. In terms of soft skills, the achievement was assessed based on student reports, teamwork, communication, and a survey to evaluate the students’ activities.

3. Results

The purpose of the diagnostic evaluation was to measure the homogeneity of the sample in the recall (cognitive level 1) of previous topics, such as the definition, domain, differentiation, and integration of one-variable functions. For this purpose, the Shapiro–Wilk normality test was applied to the diagnostic evaluation results, and a p-value > 0.05 (see

Table 3. Shapiro–Wilk normality test on the results of a diagnostic evaluation.

Shapiro–Wilk Test for a Sample
DF	Statistic	p-value	Decision at level (5%)
19	0.91865	0.10684	Normality cannot be rejected

) was obtained, indicating that the sample comes from a normally-distributed population. This means that, at the beginning of the study, the students had a homogeneous degree of prior knowledge.

The student sample carried out the learning activities in three synchronous and asynchronous remote sessions; after that, Questionnaire 1 (pre-test) was used to measure knowledge and understanding (cognitive levels 1 and 2, respectively) of ODE fundamentals, such as definitions, symbols, classification, use, structure, and solution methods. Quantitative analysis with descriptive statistics of the pre-test results (see

Table 4. Statistical results of the pre-test and post-test.

Descriptive Statistics
	N	Mean	Standard Deviation	Variance	Minimum	Median	Maximum
Pre-test	19	6.15789	2.33959	5.47368	2	7	10
Post-test	19	8.84211	1.70825	2.91813	4	9	10

) showed a mean of 6.15 with a variability of 2.33, which indicates that, at the end of the three sessions, the group of students knew or understood basic ODE concepts. This can be explained because the topics were addressed abstractly, without the reference context in which the ODEs are applied; therefore, the students’ interest in understanding these topics was not yet generated.

Team-based activities were carried out during five synchronous and asynchronous face-to-face or remote sessions. Questionnaire 2 (post-test) was then applied to assess how ODEs were applied to the construction of a mathematical model and how the response of the modeled system was analyzed (cognitive levels 3 and 4, respectively). Quantitative analysis with descriptive statistics of the post-test results (see Table 4) showed an increase in the mean and a lower dispersion than the pre-test results. This indicates that the learning activities had a positive effect on student learning.

However, for greater formality, a paired sample Student’s t-test was applied to pre-test and post-test results to test the hypothesis (see

Table 5. Hypothesis test of the pre-test and post-test.

Paired Sample Student’s t-Test
t0 statistic	DF	t0.05,18	p-value
−3.69885	18	−1.73406	8.21435 × 10−4

). The null hypothesis H₀ was established as H₀: μ₁ − µ₂ = 0 and an alternative hypothesis as H_A: µ₁ − µ₂ < 0, taking α = 0.05 as a significance level. The result shows that H₀ should be rejected because, on the one hand, in the test statistic method, t₀ = −3.69885 is lower than the critical value of the test, in which 18 degrees of freedom corresponds to t_0.05,18 = −1.73406 and, on the other hand, with the p-value method, it was observed that the result was 8.21435 × 10⁻⁴ < α, confirming the rejection of H₀. Consequently, the H_A was accepted, which means that there is a statistically significant difference between the two means in favor of µ₂ and, with a confidence level of 95%, it was concluded that the delivery of activities based on the b-learning approach (treatment) had a significant effect on student learning concerning the application of ODEs and the analysis of the modeled system’s response.

Concerning the qualitative evaluation, the results of the rubric showed that the design of the activities favored interaction among the students and allowed them to apply the ODEs to model the system assigned to each team. Similarly, the analysis of the system’s response; that is, obtaining the analytical and numerical solutions, was adequately achieved. As for soft skills, we observed remarkable progress in areas such as teamwork, report drafting, organization skills, and verbal communication used to convey information to the group.

Regarding the feedback for this work, the survey responses revealed that the group of students felt comfortable with the activities and work dynamics proposed.

Table 6. Survey responses.

Item	Sample Responses
Describe the interaction or collaboration in your team when carrying out these activities.	Collaboration was very good, each person did what they had to do, and there was a lot of chemistry since my partner and I have known each other for a long time; that is why I feel that everything went just fine. There was adequate cooperation and collaboration when we carried out the tasks; we agreed on ideas and made everybody’s work easier. The communication with my partner was very good; we both contributed equally to the task.
What do you think about the material (text, software, etc.,) you used to solve your task?	Very useful to solve the problem and acquire the right solution with the right tools to better understand the subject. After researching different sources, we could completely satisfy our doubts and clear up any issues that came our way. As far as the problem was concerned, it was very clear, we also relied on the book, and we had all of Matlab’s support materials.
Describe the difficulties you had during this activity.	In my opinion, the most difficult thing was that I could not fully understand the difference between the numerical and the analytical solution. Better understand the relationship between the variables. Implementing the script in Matlab for it to provide a numerical solution.

shows some answers provided by the students, which are considered as feedback to improve the design of this didactic proposal for future applications.

Finally, this research work has the following limitations. (i) The students’ sample was only from an educational program; therefore, a broader sample is necessary to include students from other educational programs and generalize the conclusions. (ii) It is necessary to validate the observation and measurement instruments in order to strengthen their internal consistency and the evaluation of the results. In future implementations, it is essential to consider the participation of other teachers or other institutions and the use of the statistical technique of analysis of factors to understand better the information provided by the measuring instruments.

4. Conclusions

In the present study, we implemented a didactic proposal for mathematical learning based on the b-learning approach, which led to the following observations.

The results from Questionnaire 1 indicate that it is necessary to diversify the activities, enriching the learning environment from the beginning with possible applications of ODEs; this can generate greater interest in students in understanding the basic concepts of ODEs. The descriptive statistics and the hypothesis test of Questionnaire 2 results allow us to conclude that the delivery of activities based on the b-learning approach (treatment) had a significant effect on student learning concerning the application of ODEs and the analysis of the modeled system’s response. However, during the qualitative evaluation, it was possible to detect the need to add new dimensions to the rubric and questions in the survey, and to have more elements to improve the evaluation of the development of soft skills.

The use of b-learning had a positive effect on student performance in a task that consisted of using ODEs for mathematical modeling and systems analysis.

Active methodologies, such as blended learning and ICTs, are powerful resources to advance the research in mathematics education, contribute to improving educational processes, and significantly enhance knowledge acquisition by engineering students. To study this process, it is necessary to enable an adequate experimental design to formalize the analysis and observation of interactions in the classroom and in virtual environments that facilitate learning.

The use of Bloom’s cognitive domain levels in the design of the learning activities was found to be adequate to organize the content and degrees of difficulty in the learning activities.

Students’ and teachers’ experiences during the lockdown due to the COVID-19 pandemic were a facilitating antecedent for the proper development of synchronous and asynchronous remote activities. As educational institutions return to regular activity within their facilities, such experience should be taken into account. Teachers can enrich their courses with the advantages of b-learning; their challenge is to design adequate activities and materials and to find a balance between online and face-to-face interaction. Students can activate different learning styles by taking advantage of the potential offered by the different multimedia tools and digital technologies available in blended courses.

Educational institutions should continue analyzing and appraising the extent to which integrating technology into educational processes enhances active learning.