The Concept of Active Learning and the Measurement of Learning Outcomes: A Review of Research in Engineering Higher Education

Abstract

Active learning has gained growing political, instructional, and research interest. However, the definitions of active learning are wide. The learning outcomes related to it have been mostly positive but the measurement methods are not without problems. This review provides an overview of active learning, especially in the context of engineering higher education, by answering two research questions: (1) How is the concept of active learning defined and justified in engineering higher education research? (2) What are the learning outcomes connected to active learning and how is learning measured in engineering higher education research? Sixty-six empirical articles were analyzed inductively with qualitative content analysis. The analysis showed that active learning was defined in various ways, and in some articles, it was not defined at all. In addition, justification (theoretical or empirical) for the use of active learning was seldomly reported. Finally, the indicators used to measure the impact of active learning on students’ learning outcomes were mostly based on students’ self-report data and focused on course specific development in subject-related knowledge. More thorough descriptions and theoretical justifications, as well as the consideration of learning outcomes with appropriate research methods, could reinforce the transparency of empirical interventions and the application of active learning.

1. Introduction

The higher-education field has been facing a challenge to assess the traditional instruction practices and modify them to a more student-centered direction. The pressure for this has risen from the needs of students and working life, as well as from the broader economic and political changes [1]. More student-centered and activating instructional procedures have been connected to greater achievements in student learning and, especially lately, in generic working life skill or competence development [2,3]. Generic working life skills have been viewed as important learning goals, because of their significant role when an individual transfers from higher education to the labor market [4].

Consequently, the concept of active learning has received increased interest. Active learning is a wide concept, most often referring to student-centered and activating instructional methods and instructor-led activities [5,6,7,8]. Therefore, it is generally not a concept of learning but a concept of instruction. Previous research on active learning from the viewpoint of student learning outcomes has been mostly positive [5,9,10,11,12]. Thus, research has supported the role of active learning as a superior approach compared to traditional, more content-centered approaches, such as lecturing.

The effectiveness of active learning and the robustness of the evidence have also raised some concerns. They relate to varying definitions of active learning and to valid indicators to measure related (positive or negative) learning outcomes [11,13]. The idea that active learning enhances learning has encouraged multiple political organizations, such as Unesco, as well as professional associations and accreditation organizations to recommend using activating instructional methods in education [14]. In these circumstances, the application of active learning and the empirical research on its effects on learning outcomes require systematic attention.

This review study seeks to provide an overview of the current state of active learning research by investigating the use and justification of the concept of active learning as well as the measurement methods of learning outcomes in active learning research. The review is contextualized in engineering higher education because teaching in sciences and applied sciences has traditionally been more content-oriented and instructor-focused than student-centered [15]. On the other hand, engineering higher education has experienced a rapidly growing interest towards active learning, as well as produced encouraging results on the effectiveness of active learning in previous studies [14,16]. Therefore, taking a look at the phenomenon of active learning in this particular context is, besides fruitful, also of high value to the entire higher education field.

The range of research reviewed in this study aims to deepen researchers’ understanding of the current state of knowledge about the conceptual use and the methodological investigation of active learning and to provide theoretical and methodological directions for future research. In addition, the study aims to provide guidelines for instructors to enhance their critical and theoretical understanding of active learning in higher education.

2. Active Learning as an Instructional Approach

Conceptually, active learning is not an easy target. Its theoretical roots are in constructivist learning theories. Constructivism has become a leading learning paradigm, and it views learning as a construction process of new knowledge in relation to previous knowledge [17,18]. Constructivism criticizes the idea that learners receive knowledge from external sources and highlights understanding instead of memorizing [19]. Aiming to understand, rather than memorize, is also characteristic of a ‘deep approach’ to learning [20]. Depending on the constructivist theory, learning can be considered as individual cognitive processes (cognitive constructivism), social co-construction of knowledge (social constructivism), or as a hybrid of these two [21].

Nevertheless, it is difficult to define aspects of effective constructivist teaching because constructivism is a theory of learning and not a theory of teaching [22]. Constructivism can be and has been used as a guide for forming instructional strategies that aim to enhance deep understanding [21,22]. For example, active learning as an instructional approach aims to enable constructivist learning by emphasizing students’ self-construction of knowledge [10,11], and students’ responsibility for their own learning [11]. However, instruction that aims to be constructivist does not always succeed in its intentions. According to [17], any constructivist instructional methodology should, but often fails to, elicit prior knowledge, create cognitive dissonance concerning the prior and the new knowledge, and include application of the knowledge with feedback as well as reflection on learning.

Thus, it is not mandatory to use a specific activity to be recognized as constructivist, and, similarly, adopting a specific activity does not ensure that the activity is constructivist [17]. This applies to active learning activities, as well. Even though the concept of active learning relies on constructivism, or is at least inspired by it, the concept captures a wide range of views [23]. The authors of [24] present that there is often a lack of clarity and consensus on the meaning of active learning, and definitions of it frequently lack robustness. Usually, the definitions include the broad idea of being instructor-driven activities that are used to activate students [5,6,7,8]. Active learning has been tied to formal instructional events in a classroom [5,9], or at least to being introduced in a classroom [8]. It is often contrasted to passive learning, e.g., lectures, where knowledge is more passively obtained [25]. This can be problematic from the viewpoint of constructivist learning theories that consider that all knowledge, regardless of its source, is constructed [24]. Overall, active learning is not just something that students do on their own but is somehow organized and monitored by an instructor, therefore, being an instructional approach that guides learning. However, the authors of [24] note that, in some instances, active learning has been also understood as an approach to learning, therefore, focusing on a learning process instead of an instructional process.

As a consequence of the wide range of definitions, active learning as an instructional approach can include different forms of activation, such as increased physical activity, interaction, social collaboration, deeper processing, elaboration, exploration of the material, or metacognitive monitoring [23]. In addition, the various activities under the concept of active learning may involve different forms of instruction [11] and be related to different cognitive processes [13]. The lack of clear definitions of active learning and shared terminology may cause some discrepancies when active learning activities are considered [13]. For example, researchers and instructors in a field such as engineering might have little experience on educational theories [26,27], which they could reflect in their experiments. Consequently, applying active learning without an insufficient experience or knowledge on it may not yield the expected learning results, and may even have the opposite impact of demotivating and discouraging students [28]. In addition, the varying terminology of active learning is a challenge for research and raises the question of whether it is possible to generalize results of the effectiveness of active learning if the activities can vary to a great extent. Thorough descriptions of the fine features of active learning strategies as well as their appropriateness in reaching certain learning goals could help in comparing different strategies [16].

As interest in active learning has increased, many instructors have been eager to try out new procedures and also report them to their fellow instructors. For example, the authors of [27] suggest that action research, with spiraling steps of planning, action, fact finding and reflection or evaluation, is an appropriate way for engineering educators to study active learning in their instructional practices. However, the research on active learning is not without problems. For example, measurement methods of learning and their validity are seldom reported in the active learning literature [13]. Further, although instructors agree with researchers about the direction of educational development of Finland, a review study [29] showed that instructors actively piloted new teaching methods without conducting any research on them. Ishiyama [30] reviewed studies that had applied simulation studies in political science education and discovered that most of the articles were mere descriptions of procedures without providing any systematic evidence on their effects on learning, and that some of those that provided evidence were only anecdotal cases. In addition, the authors of [31] reviewed studies on experiential learning techniques and argued that even though it can be noted that learning had occurred in the original articles, the methodological investigation revealed that the results did not meet the highest of research design and measurement standards and, therefore, can lead to only tentative conclusions about the true effectiveness of the approaches. Prince [5] notes that comprehensive assessment of active learning is often difficult because studies do not usually consider a broad range of learning outcomes and when they do, the results are often mixed or a matter of interpretation. Overall, more discussion is still needed on the measurement methods of learning in active learning research.

3. Materials and Methods

3.1. Study Design

This study is a critical review of active learning. A critical review aims at a critical assessment of extant literature and addresses questions with a broad scope [32]. The objective of a critical review is not to compare the included articles to one another; rather, the aim is to strengthen knowledge development in the field by viewing the studies against a criterion and highlighting variation and issues to focus further studies and experiments see, e.g., [32]. In this study, we wanted to take a broad look at the research on active learning in engineering higher education and, instead of comparing the studies, to recognize their conceptual and methodological features and view them against the literature. Therefore, the analysis goes beyond description of the data and aims to identify conceptual contribution [33] as well as to provide suggestions for improvement in future studies [32]. The following research questions are addressed:

• How is the concept of active learning defined and justified in engineering higher education research?
• What are the learning outcomes connected to active learning and how is learning measured in engineering higher education research?

Critical reviews can include either a selective or representative search strategy [32]. This review included a systematic search strategy to cover a wide range of original empirical studies. Recent research and selected studies were identified for inclusion with a database search in the Andor search portal maintained by the Technical University of Tampere library. The portal includes hundreds of databases, such as Scopus, ERIC, Web of Science, and Academic Search Premier. After preliminary searches, the final search was implemented in early May 2018.

3.2. Materials

Based on the research questions, two broad keywords, active learning and engineering, were used to search for abstracts of scholarly and peer-reviewed journal articles that were published from 2000 to 2018. These keywords were chosen to include as many relevant articles as possible in the search. The inclusion process of the final articles is described in Figure 1.

The search produced a total number of 347 articles. The abstracts of these articles were judged against the following inclusion criteria: (1) Written in English; (2) based on empirical research; (3) implemented in the context of engineering higher education; and (4) reported research on the effects of an active learning activity on student learning outcomes. With these criteria, we wanted to address only those journal articles that studied empirically the learning outcomes of instructional strategies. In this process, 268 articles were excluded, mostly because they did not examine the activities’ effects on student learning outcomes (n = 113). Other main exclusion criteria included a lack of active learning activities based on instructor-chosen activities (n = 48) and a lack of an engineering higher education context (n = 39).

To extract the essential information from the remaining 79 articles, a data matrix was composed to show the following information for each study: The author(s), year of publication, setting and sample, design, active learning activities, definition of active learning, argument for the use of active learning, and key findings related to learning outcomes. While collecting data from these studies, 13 more were excluded at this point in the process. Two main reasons for exclusion were that, despite the promising abstracts, the studies did not examine the effects of the activities on student learning outcomes (n = 6), or the activities did not meet our adopted definition of active learning (n = 5) tied to formal instructional events.

In the end, 66 studies were included in the review. Asterisk (*) symbol in the reference list indicates an article included in the review (N = 66). In these studies, the sample sizes varied from small samples of about 10 students to samples of more than 3500 students. However, some studies failed to report several important aspects of the study setting, such as the overall number of participants or the description of a control group. Only 49 studies successfully reported the number of participating students (together, 10,557 students). Four studies failed to report the number of participants, and 13 studies only partially mentioned the number of participants (e.g., only in an experimental group) or only provided some kind of estimation of the number. The approximated overall number of students in these 13 studies is, therefore, at least 2714. Most studies were conducted in the United States and Europe; the other locations were Asia, South America, Asia, Australia, and Canada.

3.3. Analysis

Critical reviews can be conducted with a variety of data synthesis methods [32]. In the current review, the analysis followed the principles of a qualitative content analysis [34,35], which was applied inductively by creating categories from the data [34]. Inductive approach was chosen because it was coherent with our intentions to find out how the studies themselves had approached active learning and because we did not want to force those approaches to any existing theory. The content analysis included steps of open coding, creating categories, and abstraction [34]. The unit of analysis distinguished for coding was a meaningful piece of text [35,36].

The first aim of the analysis was to discover how each included study (N = 66) approached active learning conceptually (RQ1). In the first step of categorization, the articles were coded into two main categories: Those that included a definition for active learning and those that did not. The latter category was further divided into two separate categories. In the abstraction phase, the three mutually exclusive categories (each article was coded only once under one main category) were conceptualized with content-characteristic words as ‘active learning defined as an instructional approach’, ‘active learning not defined but viewed as an instructional approach’, and ‘active learning not defined but viewed as a learning approach’. These main categories were created to cover one aspect of the material to meet the requirement of unidimensionality [35].

In the second step, sub-categories were formed based on the qualitatively different descriptions of active learning in the studies. The unit of analysis was a textual description and one study could, therefore, be coded into various sub-categories within one main category. Some complexity of hierarchical levels of coding frame was apparent in multi-dimensional sub-categories and their relation to main categories; see [35]. The coded subcategories could have given reason for creating levels of hierarchy and for further investigation of relations between the subcategories. In this study, however, the decision was made to study the variation of descriptions of the concept of active learning in the original studies instead of these descriptions’ relations to one another.

In the third step, the descriptions of active learning inspired us to get deeper into arguments the authors described for adopting the concept of active learning in their studies. Because of our interest in instructional designs, this step of the analysis focused on main categories 1 and 2 and studies that viewed active learning as an instructional approach (n = 48). Arguments for using active learning were identified and labelled accordingly in a data-driven procedure. Consequently, they formed the final seven sub-categories that were conceptualized (e.g., ‘justified superior learning’ or ‘working life expectations’). Each original study was included only within one main category but could include multiple arguments (subcategories) for using active learning as an instructional approach. Thus, the unit of analysis was an argument, and to capture the true variation of arguments across the articles, an article could be categorized in various sub-categories.

The second aim of the analysis was to discover the learning outcomes that were found regarding active learning and how learning was measured in the included studies (RQ2). Analysis of the second research question concentrated only on the studies that viewed active learning as an instructional approach (main categories 1 and 2, n = 48) because of our interest in instructional designs. The analysis consisted of two parts. The first part focused on analyzing the learning outcomes. First, the descriptions of learning outcomes were recognized and labelled based on what outcome were studied and whether the effect on the outcome was positive, negative, or mixed (both positive and negative) (e.g., ‘negative subject-related knowledge’, ‘positive group work skills’). These outcomes were then grouped to form the final five learning outcome sub-categories which were then conceptualized (e.g., ‘social skills’). In the second part, the measurement methods of learning outcomes were analyzed. They were first recognized and labeled (e.g., ‘survey’ or ‘final exam’). These categories were then grouped into the final five sub-categories and labeled accordingly. As the units of analysis were learning outcomes and measurement methods, each study could be included in more than one sub-category.

Evaluating the quality of data analyses and the coding frame involves examining the coding process in terms of consistency and validity [35,37]. In this study, the first author conducted the trial coding and modified the initial coding frame through the systematic classification process of coding and identifying the themes and patterns from the selected articles; see [38]. The first and second authors conducted the modification of the final categories based on the two criteria for evaluating coding frame [35]; (1) the examination of inconsistency in main categories and subcategories to make revisions where needed; (2) and the evaluation of the extent to which the categories adequately describe the material and the concepts that are part of the research question. The first and third authors were responsible of validating the quantitative material. All four authors conducted the final conclusions of the coded data by discussing the patterns and relations of the categories and the theoretical interpretations.

4. Results

4.1. The Concept and Justification of Active Learning

This review clarifies the current use the concept of active learning in engineering higher education by focusing on the definitions of the concept itself and on the arguments for applying it. According to the results, active learning was defined in 66 articles in three major ways: (1) Active learning defined as an instructional approach, (2) active learning not defined but viewed as an instructional approach, and (3) active learning not defined but viewed as a learning approach. Our analyses showed that each of these three main categories consisted of several sub-categories.

Table 1. The definitions of active learning (N = 66).

The concept of Active Learning	Sub-Category	Example	References	Total n 1
Main category 1: Active learning defined and viewed as an instructional approach (n = 25)	Student-centered	‘Given students’ proclivity towards misconceptions and surface learning approaches, it is proposed that an active learning approach, which calls for more interactive engagement and learner-centred approaches, can facilitate the construction of knowledge through deep meaningful learning experiences’ [39] (p. 69).	[13,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54]	17
	Opposite of lectures	‘Active learning, on the other hand, as implied by its very title, is something “other than” the traditional lecture format. The concept of active learning in this study is simple, rather than the instructor presenting facts to the students, the students played an active role in learning by exploring issues and ideas under the guidance of the instructor (scaffolding)’ [55] (p. 8).	[13,40,47,48,49,53,54,55,56,57,58]	11
	Reflection and thinking	‘It requires students to engage in meaningful learning activities and reflect on the knowledge that teachers are making available’ [40] (p. 56).	[40,43,47,50,51,57,58,59,60]	9
	Student action	‘In essence, students should do more than just listen. They should read, write, discuss, or be engaged in solving problems’ [50] (p. 509).	[44,47,50,51,55,57,58]	7
	Construction of knowledge	‘Indeed, active learning is consistent with the constructivist theory that maintains that knowledge cannot simply be transmitted from teachers to learners; rather, learners must be engaged in constructing their own knowledge’ [41] (p. 326).	[13,39,41,43,46,56,61]	7
	Collaboration	‘The proposed method is based on active-learning. This approach provides for supporting knowledge conceptualization, development in uncertain contexts and collaborative work by the students’ [60] (p. 885).	[13,39,40,44,46,54,60]	7
	Activating activities	‘As stated by Prince (2004), such active learning is designed to get the students to think about what they are doing through meaningful learning activities’ [59] (p. 692).	[13,58,59,60,62]	5
Main category 2: Active learning not defined but viewed as an instructional approach (n = 23)	Concrete action in class	‘Active learning methods can include projects with class participation where the lecture material is illustrated through hands-on experiments or demonstrations in which the students participate directly’ [63] (p. 136).	[63,64,65,66,67,68,69,70,71,72]	10
	A single activity	‘In this paper we report a pilot-test of the use of one method of active learning, namely, student peer teaching of core content, which has not been widely used in science and engineering education’ [73] (p. 166).	[73,74,75,76,77,78,79,80]	8
	Built on previous studies	‘Features of active learning pedagogies have been found to have positive effects on student outcomes … [S]tudies have connected active and collaborative learning environments to improvements in metacognitive self-regulation … and self-efficacy’ [81] (p. 276).	[81,82,83,84,85]	5
Main category 3: Active learning not defined but viewed as a learning approach (n = 18)	Enabled learning	‘This platform is a low-cost and safer alternative to a conventional PIV system that provides unique active learning experience in biofluid mechanical principles more effectively than traditional didactic programs’ [86] (p. 39).	[86,87,88,89,90,91,92,93,94]	9
	Student behavior	‘This is likely to engage students and shifting them from a “passive” (what they are used to) to an “active” (what they are empowered to) learning process’ [95] (p. 76).	[95,96,97,98]	4
	Active learning environment	‘Peer instruction (PI) is an in-class instructional technique implemented to promote an active learning environment’ [99] (p. 1).	[99,100,101]	3
	Learning outcome	‘Traditional engineering teaching methods, such as lectures, exercises and lab work have been criticized because they do not prepare engineering students to collaborate and to learn active learning behaviour, which emphasize interaction between students’ [102] (p. 5).	[102,103]	2

¹ Total n may include multiple entries of a single study (e.g., definition in [40] covers four sub-categories: Student-centered, opposite of lectures, reflection and thinking, collaboration).

shows the three categories for these definitions with examples.

The definitions of those studies that defined active learning as an instructional approach (main category 1, n = 25) were grouped into seven sub-categories: (1) Student-centered; (2) opposite of lectures; (3) reflection and thinking; (4) student action; (5) construction of knowledge; (6) collaboration; and (7) activating activities. One study could define active learning with several of these aspects. As the categories show, the role of the student in the center of the activity was emphasized in many studies. This includes involving students in the learning process, engaging students in learning, learning being student- or learner-centered, and highlighting students’ responsibility for their own learning. ‘Reflection and thinking’ refers to students’ reflection during activities, their involvement in higher-order thinking, or critical and active thinking processes. ‘Student action’ emphasize that students participate in learning related activities. Finally, some studies saw students’ self-construction of knowledge as a vital part of active learning. Interactive work with others was connected to active learning through collaboration, interaction, and dialogue.

Several studies (n = 23) in this review did not give a definition for active learning, but nevertheless viewed it as an instructional approach (main category 2). These studies fell exclusively into one of the three sub-categories: (1) Concrete action in class, (2) a single activity, and (3) built on previous studies. The first sub-category consisted of studies that viewed active learning as a concrete action that occurs in the class. In the second sub-category, studies simply named the method used as an example of active learning, thus focusing on a single activity, without considering the concept of active learning further. The third sub-category consisted of studies that labelled their pedagogical approach as active learning based on previous studies and on its impact on activating students to learn.

The data also included studies that did not view or define active learning from an instructional point of view but rather viewed it from the perspective of student learning (main category 3, n = 18). These studies fell exclusively into one of the four sub-categories: (1) Enabled learning, (2) student behavior, (3) active learning environment, and (4) learning outcome. None of these studies clearly defined the concept of active learning, but active learning was viewed as a target, that is, it is the kind of learning that is promoted in the study. In some of these studies, active learning as a learning approach was seen through student behavior or as a learning outcome. In some cases, active learning from a students’ perspective was simply seen as a part of an environment where students learn actively.

In order to investigate the justifications that the studies used to support the use of active learning as an instructional approach, we took a closer look at the 48 studies in the first two previously presented main categories ((1) active learning defined as an instructional approach and (2) active learning not defined but viewed as an instructional approach). Analysis of the arguments, presented in

Table 2. The arguments for the adaptation of active learning (N = 48).

The Argument for Use	Example	References
Empirical research-based justification for superior learning (n = 17) 1	“Numerous studies have demonstrated that students learn better when they are actively engaged in the learning processes (references).” [47] (p. 240)	[13,39,41,47,48,51,52,54,55,56,60,62,63,64,75,81,85]
Taken-for-granted justification for superior learning (n = 14) 1	“Traditional lecturing is equated to passivity, which leads to learning ineffectiveness. Modifications to the traditional lecture format, however, can lead to more effective learning. These modifications often fall within the context of active learning ….” [49] (p. 522)	[40,42,45,46,49,50,56,61,69,70,73,79,83,84]
Policy level recommendations (n = 9) 1	“To this end, since more than a decade ago, a number of national reports have considered the prevailing education practices inadequate to prepare future leaders and illustrated the need for more dynamic learning techniques in both the undergraduate and graduate levels ….” [46] (p. 1).	[42,46,50,56,59,61,74,77,84]
Working life expectations (n = 9) 1	“Whilst modern engineering requires a good level of technical competence, other skills are also important in the profession. Some studies conducted to determine the abilities that the industry requires of engineers show that engineering graduates need to have strong communication and teamwork skills, and a broader perspective of other issues related to their profession … The lack of effectiveness of traditional pedagogy to develop some skills led to the use of new didactic approaches.” [82] (p. 408)	[42,45,46,52,54,58,59,82,84]
Own perceptions (n = 9) 1	“The authors’ experience in teaching is that passive learning does not work well. The problems seem easy when the instructor works them at the board; it is not until problems are assigned and students get stuck part way through or fail to complete them that real learning occurs.” [48] (p. 188)	[40,41,43,47,48,68,82,84,85]
Learning theory (n = 8) 1	“The roots of the active learning methodology reach back to Confucius (450 B.C.) who coined the phrase: ‘Tell me, and I will forget, Show me, and I may remember, Involve me, and I will understand’. Pestalozzi in 1801 envisioned schools that were homelike institutions where teachers actively engaged students in learning by sensory experiences through engagement in activities. Students were to learn useful vocations that complemented their other studies (Education Encyclopedia 2011). Dewey (p. 7, 1938) argued that “there is an intimate and necessary relation between the processes of actual experience and education”.” [53] (p. 116)	[39,42,43,47,53,59,61,77]
Overall trend (n = 6) 1	“More recently, engineering educators have recognized the need to transform students from passive listeners in the classroom to active learners. The paradigm shift is from a teacher-centered delivery approach to that of a learner- centered environment.” [78] (p. 31)	[45,52,55,56,78,79]

¹ Frequencies may include multiple entries of a single study (e.g., the article of [42] uses argumentation based on superior effects on learning, policy level recommendations, working life expectations, and learning theories).

, resulted in seven categories: (1) Empirical research-based justification for superior learning, (2) taken-for-granted justification for superior learning, (3) policy-level recommendations, (4) working life expectations, (5) own perceptions, (6) learning theory, and (7) overall trend. The most common argument for the use of active learning was that previous research had found encouraging results for its effectiveness in promoting learning (n = 17). The second most popular argument for using active learning was also based on the belief of its benefits for learning, but without references to research evidence (n = 14). Several studies justified the need for active learning by referring to policy-level recommendations (n = 9) or the demands of today’s working life (n = 9), usually referring to general skills that graduates need and that students can develop through these methods. Some arguments were based on authors’ (or their affiliations’) perceptions of the need to change current instructional methods (n = 9). Some studies brought up learning theories as the foundation for active learning or presented active learning as an overall trend that is taking space from traditional instructional methods.

4.2. Learning Outcomes and the Measurement of Learning

Most of the studies that viewed active learning as an instructional approach (N = 48) found that the approach was positively related to various aspects of student learning. Seven studies found that active learning had a mixed effect (both positive and negative) on learning, usually related to the development of subject-related knowledge.

Table 3. Active learning related student learning outcomes and their measurement (N = 48).

Learning Outcomes Measured by
Learning Outcomes		Students’ Self-Reports (n = 27)	Course Grades (n = 11)	Exam Grades (n = 16)	Pre- and Post-Tests (n = 5)	Assessment of Students’ Work (n = 5)	Total n 1
Positive effect on the development of	Subject-related knowledge (n = 37)	3 (C) [40,61,79] + 1 (PC) [56] + 12 (NC) [46,51,52,58,60,62,64,68,70,73,74,84]	10 (C) [40,41,48,53,55,58,65,69,77,79] + 1 (NC) [50]	9 (C) [39,48,54,57,61,63,66,79,80] + 1 (PC) [56] + 4 (NC) [13,46,67,83]	1 (PC) [81] + 2 (NC) [45,71]	3 (NC) [43,60,73]	47
	Professional skills (n = 16)	3 (C) [54,69,79] + 10 (NC) [44,45,46,58,59,61,62,76,78,82]		1 (C) [80]	1 (NC) [47]	1 (C) [85] + 1 (NC) [59]	17
	Social skills (n = 5)	5 (NC) [45,62,68,69,76]					5
	Communication skills (n = 4)	4 (NC) [62,68,73,84]					4
	Meta-competences (n = 6)	2 (C) [54,79] + 3 (NC) [42,61,68]		1 (C) [80]			6
Mixed effect on learning outcomes (n = 7)		1 (C) [72] + 2 (NC) [44,49]	1 (C) [77]	2 (C) [72,75]	1 (NC) [74]	1 (C) [85]	8

¹ Total n includes multiple entries of a single study (e.g., the article of [40] includes both self-reports and course grades). Note. C = control group (20 studies); PC = presumed control group (control group not clearly described in the article, 2 studies); NC = no control group (correlational design, 29 studies).

shows that the development of subject-related knowledge was the most studied aspect of learning (n = 37). The next most frequently studied aspect (n = 16) was the development of professional skills, such as design, analytical, or problem-solving skills, that are important for engineers. In addition, a few studies examined the development of social skills and communication skills (n = 9) or of meta-competences (n = 6), such as critical thinking or higher order thinking skills. Therefore, the more generic skills, those not bound to any specific profession, were given the least attention.

Self-report methods (mostly surveys, as well as interviews in some studies) and correlational cross-sectional designs were the most popular way to measure various aspects of student learning and the development of their generic skills. Some studies used more objective methods, such as course (n = 11) and exam grades (n = 16), and control group designs to measure the development of subject-related learning. Only five studies used pre- and post-tests to study the effects of active learning on learning, and only one of these presumably used a control group. In five studies, an assessment of students’ work, usually by an instructor, was used to measure learning.

Most of the studies included many different active learning activities. However, only a few studies considered these varied activities in the measurement of learning outcomes. This was done by asking students to evaluate what they had learned through various activities via a survey or by testing students’ success in each activity separately. However, in most studies, the methods for measuring student learning targeted overall achievements in the course rather than examining the effects of various activities individually.

5. Discussion

This review focuses on examining previous empirical studies of active learning in engineering higher education. The review found results related to both the use of the concept of active learning as well as the measurement methods of learning. Overall, the results highlight that future empirical studies of active learning should pay more attention to describing how they view active learning and considering the quality of measuring learning outcomes. The conclusions of the study are presented as a framework for formulating active learning research in Figure 2.

Due to the inclusion criteria, all 66 original studies included instructional solutions that targeted student activation. Nevertheless, only less than half of the studies actually defined active learning as an instructional approach. The rest of the studies did not define the concept but viewed it either as an instructional or as a learning approach. The results support [24], who state that the concept of active learning often lacks clarity and consensus and can also refer to a learning approach.

When active learning referred to an instructional approach, it was defined and viewed mostly through student activation, which is in line with the wide definitions of active learning in previous literature [5,6,7,8]. Some studies also contrasted active learning with lectures, assumedly a more passive teaching method, which is highly common in active learning literature [25]. The theories of constructivism were present in some definitions, when students’ construction of knowledge and reflection and thinking were mentioned as key aspects of active learning [10,11]. Based on the results, however, it seems that active learning as an instructional approach is somewhat larger than what constructivist theories would expect from an instructional methodology, e.g., [17].

The absence of learning theories justifying the adoption of active learning was also prevalent. Previous research was, to some extent, used as an argument for active learning but much of the superiority of active learning was purely accepted as a known, taken-for-granted issue or overall trend. In addition, some studies saw active learning as an answer to policy-level recommendations or working life expectations, which are aspects often connected to active learning [1,14,24].

As writing a research article always includes choices, it is naturally possible that the authors shared some idea of the concept and of the theoretical underpinnings of active learning that they, for some reason, did not include in the report. However, the definitions of active learning and the arguments for its use in intervention studies are vitally important as they tie the often multi-faceted, wide concept to the assumptions that the authors view as important for the instructional intervention in question and, thus, reinforce the transparency of active learning interventions. Unclear definitions may limit other instructors’ understanding and consideration of the theoretical underpinnings of active learning and support the view that active learning is ‘anything that activates students’. Therefore, applying active learning without sufficient experience or knowledge may be challenging and lead to unexpected results [28]. For researchers, it is difficult to compare various activities and methods if it is not truly clear what are the fine features of activities in question [16].

The recognition of conceptual underpinnings is even more important today when active learning is heavily underlined by different parties, such as accreditation agencies, policy makers, and higher education institutions. Research on active learning, which there is plenty of, is one important factor that facilitates the mission to modify instruction to a more active direction. Methodological investigation of the 48 studies (out of 66 included in the review) that had defined or viewed active learning as an instructional approach showed the active learning interventions had led to mostly positive learning outcomes. Nevertheless, certain challenges in measuring learning outcomes were observed in this research.

First, clearly most of the studies examined effects on students’ subject-related knowledge. Other studied learning outcomes were professional (engineering) skills, social and communication skills, and meta-competences. These aspects are considered also in previous research. For example, in a holistic model of competence, cognitive competence covers knowledge and understanding, functional competence includes skills (i.e., practical know-how), social competence includes the competencies related to behavior and attitudes, and meta-competence is related to the acquisition of other competences (‘learning how to learn’) [104,105]. Overall, the more generic skills, those not bound to any specific profession, were given the least attention in the studies of this review, even though they are considered important in the transition process from higher education to working life [4], and are viewed as important products of active learning [14,24]. The result also confirms that the content-oriented nature of education in the field of engineering is still clearly visible [15]. In the future, there is more need for research on various skill development. After all, to determine whether an active learning activity is effective, various learning outcomes should be considered in measurements [5].

Second, most of the measurement methods relied on students’ own perceptions of their learning through self-reports. This can be problematic as students’ opinions of the course or any aspect of it can impact the estimates that they make on their learning. Especially the development of generic skills was investigated by using self-perceptions. This might be because more generic skills are often difficult to measure [5]. In addition, the results show that only 20 of the 48 pedagogical interventions applied control group designs. However, when no control groups are used, researchers may reach varying conclusions regarding whether an improvement in an aspect is significant or not. Pre- and post-test could also give interesting insight into student learning but according to the findings of this review, they are still sparingly used (5 out of 48 studies). In future studies, learning outcomes could be measured more deeply if versatile methods of measurement were used. For example, the authors of [27] suggest triangulating the evidence using both qualitative and quantitative data.

Third, most of these studies measured student learning in a course in general, rather than focusing on the effects of various activities on learning. This is especially problematic for studies that combine two or more active learning activities, because it is difficult to assess the effectiveness of the activities if the different components are not studied separately [5]. For example, using only students’ course grades as a measurement of active learning may cause uncertainties, as improved grades can be the result of a specific instructor’s expertise, the quality of the students, or the measurement method itself instead of the pedagogical practices [27].

The methodological consideration of active learning intervention studies proposes that in the future, researchers should pay attention to the descriptions of study settings (e.g., the number of participants) as well as to how they approach learning outcomes. Quite many of the studies reviewed here focused on a single active learning activity and only one aspect of student learning (e.g., increase in subject-related knowledge). Further, studies preferred to use cross-sectional correlational designs and self-report measurements. The research perspective on active learning could be deepened by focusing on (1) various active learning activities, (2) content-related and affective learning outcomes, (3) objective and self-reported measurements, and (4) longitudinal designs with pre-post tests and a control group. This would also help instructors to better consider various aspects of learning activities from the viewpoint of the learning goals of their courses. After all, the development of higher education requires a systematic approach, that is research: In addition to the fact that evidence-based development offers a critical and analytical approach to explore and enhance quality in higher education, it often reveals complicated relationships or even counter-intuitive aspects that are important to take into account when enhancing instruction and learning [106].

Overall, to continue the conceptual development of active learning models, to deepen the consensus on active learning, and to enable greater possibilities for comparisons of empirical data, more systematic and justified empirical and theoretical work is needed in the future. More discussion is also needed on the complex nature of internal and external activity of an individual, as individuals can be active as a result of their own or an instructors’ action. In addition, more support for the teaching staff in understanding active learning and the variation of active learning activities is needed.

Limitations

This study has limitations that need to be acknowledged. Most importantly, the search aimed to identify relevant studies related to the research questions, but some relevant studies (e.g., written in other language than English) may have been left out. The choices of databases and broad keywords are in accordance with the principles of a critical review, as the data provided us a wide scope of earlier research on the topic. However, we recommend future studies to focus more specifically on various active learning approaches and to use data from more diverse sources to increase the scope and depth of analyses.

The choices made in the analytical phase of the research also created limitations, insofar as other choices could have been made. However, we sought to discover and report all the aspects of the included studies that related to our research questions and have attempted to ensure that readers can identify all the significant procedures in the methodology section. Finally, because the data consist entirely of previously published studies, bias may exist, for instance, because the original studies that have found statistically significant differences are more likely to get published, or because primary authors only wanted to report certain outcomes [107].