Geology and Environment: A Problem-Based Learning Study in Higher Education

Abstract

We aimed to contribute to a shift in higher education teaching and learning methods by considering problem-based learning (PBL) as an approach capable of positively affecting students from a geology and environment (GE) curricular unit. In a convenience sample from a Portuguese public university, two groups of students were defined: (1) an experimental group (n = 16), to which an intervention program (IP) based on PBL was applied, and (2) a comparison group (n = 17), subjected to the traditional teaching approach. For nine weeks, students subject to the IP faced four problem scenarios about different themes. A triangulation of methods was chosen. The study involved two phases: (1) qualitative (sustained on content analysis of driving questions raised by students, registered in a monitoring sheet) and (2) quantitative (quasi-experimental study, based on data from a prior and post-test knowledge assessment). The qualitative results point to the development of more complex cognitive-level questioning skills after increasing familiarity with PBL. The data obtained in the quantitative study, which included both a “within-subjects” and a “between-subjects” design, show higher benefits in the experimental group, documenting gains in terms of scientific knowledge when using the PBL methodology.

1. Introduction

Fostering 21st century skills necessitates the use of methods promoting students’ involvement in the construction of their own knowledge, assigning them a role of critical and argumentative questioners in relation to everyday problems in a collaborative context. Thus, traditional educational practices in higher education are not compatible with contemporary society [1], which requires skills at several levels often referred to in international documents (e.g., [2]). In fact, the Bologna Process, started in the late nineties of last century, challenged the higher education system with an unprecedented rethinking of the teaching paradigm, which implied changing the focus from the teacher to the student [3]. Moreover, in the document “Furthering the Bologna Process”, it is expressed that “problem solving skills (…) are an essential part of the education process” [4] (p. 12). However, recent international studies indicate lecture-based learning is still the primary applied method in the classroom in higher education [5].

In Portugal, the reorganizations introduced by the Bologna Process in higher education also call for an actual paradigm shift, as it is considered merely a rudimentary shift [6]. Problem-based learning (PBL), having socio-constructivist roots [7], has emerged as a teaching methodology capable of responding to the challenge, as this changes the dynamics of the teaching–learning process, wherein the student plays a major active role in this process, while stimulating collaborative learning [8,9]. PBL distances itself from other methods as it is a result of a combination of unique aspects, where one of its distinguishing features is that meshing with the problem starts before formal study, unlike what occurs with a traditional curriculum, which measures the ability to apply knowledge after a given program content is given for such purpose [8]. According to Marra, among others [10], the PBL learning environment has five distinctive features: it is problem-focused, student-centered, self-directed, self-reflective, and facilitative.

PBL presents students with authentic, but ill-structured, scenarios that enhance students’ soft skills because they work collaboratively in small teams [11,12]. During the PBL learning cycle, when analyzing problem scenarios, students plan, collect, and synthesize information from multiple sources, generate questions and hypotheses, and communicate their ideas. It is expected that they understand and use available information, namely the application of scientific concepts to build and assess possible actions [11]. In this sense, PBL is based on the epistemological assumption that students are not a blank page [7], but rather are able to build new knowledge based on existing knowledge. In a lecture context, students do not usually ask questions spontaneously [13], but the PBL cycle fills this gap, as formulating questions is a compulsory step in the teaching and learning process. The teacher’s role ceases to be the “sage on the stage” and becomes the “guide on the side” [14], permitting students to tackle their own ideas and misconceptions [11]. Despite PBL’s worldwide application for some decades, especially in European, North American, and Australian universities [15], and in a wide range of areas (e.g., language learning [16], history [17], and psychology [18]), in terms of research, most studies conducted within the PBL scope focus on the area of medicine [19]. This is probably due to the teaching and learning methodology’s historical origin and field of development, which started in MacMaster University in the late 1960s [7], where medical students were able to memorize information but lacked the development of clinical skills to diagnose a patient with an authentic problem [11]. So, PBL as an active learning method emerged to develop transferable knowledge and skills in medical school [20]. Nevertheless, although this learning method can be successfully applied to science, technology, engineering and mathematics (STEM) areas [21], and it is also viewed as a pedagogy that promotes science literacy among students [12], there is less focus in the areas related to earth or environmental sciences [22]. In the Portuguese context, PBL is still a relatively recent method, with pioneer research in the field of education dating from the beginning of the 21st century (e.g., [23]). The existing works mainly focus on the third cycle of basic education (middle school) and secondary education (high school), with few PBL studies on the geology and environment (GE) area in higher education, which is the object of this study.

In general, geology courses in Portugal have followed a sustained path within the framework of the Bologna Process, meeting contemporary societal problems and reflecting concerns in new training possibilities that have more recently emerged, such as those related to the environment [24]. The GE curricular contents explored in this study also integrate the study plans of several curricular units from a broad spectrum of geosciences degree courses in the Portuguese public university setting. Common threads exist between these curricular units in terms of programmatic content, such as themes related to risks and hazards associated with seismic activity, volcanic activity, landslides, floods, and coastal erosion or references to the prevention and mitigation measures of geological risks, as well as the impacts of using natural resources and climate change [25]. Because GE often deals with problems for which there is no exact answer, or there are several possible ones, analyzing the situations described requires an effective integrated vision by those who study them and may be confronted by them in their professional future, requiring a greater degree of interrelationship knowledge from different areas. Moreover, it is important to promote an environmental geoculture that also incorporates consciousness about natural hazards and risks [26]. Thus, if higher education intends to train individuals in critical and reflective thinking [1], working with real problems using methods such as the PBL may eventually lead to improved professional training [27].

As an educational method, PBL has been gaining ground and many studies point out several advantages: student satisfaction [28], ability to integrate new information into previous knowledge [29], a greater ability to transfer concepts to new problems [30], better results [31], development of various general skills [32], increasing student attitudes and interest in a STEM career [33], providing more opportunities for students’ collaboration, and, thus, promoting higher-order thinking skills [5] and enhancing student achievement [34], among others. In broader studies, literature reviews and meta-analyses reported positive reactions to the methodology [35], gains in attitudes [36], and retention of knowledge for longer periods of time compared to the traditional methodology [37], increasing articulation compared to other active learning strategies [20], but not always better academic results [38]. In a recent systematic bibliometric review, Hallinger [20] analyzed about 12,000 Scopus-index documents published for 45 years (1974–2019), and concluded that there was a clear growth in PBL-related documents (57%) in the last decade (2010–2019), showing that there is greater academic interest in this subject. According to the same author, if compared with other methods of active learning (e.g., case-based learning or flipped classroom), PBL is also the most documented approach of active learning, although most of the studies conducted focus on medical education. It should be noted that, in Portugal, the number of studies on PBL is still scarce, if we take into consideration that our country does not appear on the 15 top-placed countries with the highest number of publications from 1992 to 2013, lead by the USA [39]. Thus, we sought to research whether the PBL methodology improves students’ learning in a GE curricular unit in higher education in terms of (i) development of questioning skills and (ii) learning scientific concepts related to different thematic areas within the GE scope. In this regard, we implemented an intervention program (IP) based on the exploration of four problematic scenarios according to the following themes related to the theoretical and practical GE curriculum: (1) volcanology and volcanic risk prevention; (2) seismic evaluation, prediction, and prevention; (3) degraded mining area and water contamination; and (4) slope stability. In this context, the following specific goals were outlined: (i) to examine the type of student questioning given the problem scenarios and (ii) to examine the construction of substantive knowledge by implementing a new teaching methodology (PBL), compared to the traditional methodology of teaching the GE curricular unit.

2. Materials and Methods

A triangulation of methods was chosen, and a two-stage qualitative and quantitative (QUAL–QUANT) research plan was designed, with nonrandom participants selected. The students that participated in the current study belonged to classes already defined by the faculty administration in the beginning of the semester. The geology and environment (GE) curricular unit is optional and can be chosen by students from different backgrounds who are attending the second and third years of their academic degrees (biology, geology and science and environmental technology—SET), which last three years. Two groups of students were used and assigned to a type of teaching and learning methodology. A PBL methodology-based IP was applied to the experimental group (n = 16), and the comparison group (n = 17) was subjected to the traditional approach (expository lectures and performance in some practical pen and paper activities) for the GE unit.

Two main specific research questions guided our work. Question 1 (Q1) is “how can the different types of scenarios used in the PBL method contribute toward developing questioning skills?”, and Question 2 (Q2) is “does PBL methodology favor students’ learning scientific concepts in the GE area?”. To answer Q1 and considering the corresponding specific goal (“to examine the type of student questioning given the problem scenarios”), a qualitative study was selected for the experimental group of students. The study’s second phase, characterized as quantitative (quasi-experimental), used a “between-subjects” and “within subjects” research design, intended to answer the research question (Q2) for both established groups of students (experimental and comparison). In this case, the outlined null research hypothesis (H₀) is: “there is no difference in substantive knowledge development between the students of the experimental group and those of the comparison group”. The alternative hypothesis (H₁) is: “students of the experimental group subjected to the PBL methodology differ from those of the comparison group in terms of substantive knowledge development”.

The GE curricular unit is composed of two types of classes: theoretical (2 h per week) and theoretical–practical classes (2 h per week). Both groups (experimental and comparison) have the same programmatic contents addressed during theoretical classes. Only in the experimental group was the PBL method applied in the theoretical–practical classes.

2.1. Intervention Program

The IP, involving the PBL method applied to the experimental group, occurred weekly in 2 h theoretical and practical classes of the GE curricular unit, covering a total of 18 h for 9 weeks during almost one entire academic semester (which lasts 14 weeks). The program involvement started during the first week with the presentation of the research under development, collection of informed consent, application of the knowledge pretest and simulation of a problematic situation using the PBL methodology (because many students were not yet familiar with this method). In the last session, the knowledge post-test was administered, as referred to in

Table 1. Summary of the activities planned during the IP.

Time Line (2 h Lesson/Week)	Activity
Week 1	Presentation of the investigation to be carried out and its objectives. Collection of informed consent. Application of the knowledge pretest. Simulation of a problematic situation using the PBL methodology.
Weeks 2 and 3	Launching the problematic scenario about volcanology and volcanic risk prevention (Corvo Island, Azores, Portugal). Exploration of corresponding monitoring sheet. Sharing and resolving the problem.
Weeks 4 and 5	Launching the problematic scenario about seismic evaluation, prediction and prevention (the Loma Prieta Earthquake, 1989, San Francisco, CA, USA). Exploration of corresponding monitoring sheet. Sharing and resolving the problem.
Week 6	Launching the problematic scenario about degraded mining area and water contamination (Terramonte’s mining legacy, Portugal). Exploration of corresponding monitoring sheet. Sharing and resolving the problem.
Weeks 7 and 8	Launching the problematic scenario about slope stability (the landslide in Maierato, 2010, Italy). Exploration of corresponding monitoring sheet. Sharing and resolving the problem.
Week 9	Application of the knowledge post-test

.

Between the weekly sessions previously mentioned, students subjected to the IP were divided into working groups with about five elements each and a respective spokesperson was elected. In the remaining sessions, the scenarios explored were based on diverse thematic contexts and were focused on the following particular cases: (1) volcanology and volcanic risk prevention—Corvo Island (Azores, Portugal); (2) seismic evaluation, prediction, and prevention—Loma Prieta, 1989 (San Francisco, CA, USA); (3) degraded mining area and water contamination—Terramonte’s mining legacy (Castelo de Paiva, Portugal); and (4) slope stability—the landslide in Maierato (Italy), 2010. Thus, two national context scenarios were developed, interspersed with an equal number of international scenarios. This was intended to create challenging opportunities for students when facing different contexts and allowing for differentiated bibliographic research. In each scenario, a monitoring sheet was provided to guide students through the whole process of exploring the PBL approach. The monitoring sheet for each scenario presented the following items: (1) expected timeframe, (2) objectives, (3) keywords and concepts, (4) description of the problem scenario, (5) a list of facts about the problem scenario presented, (6) driving questions or problem-questions or “what the group needs to know”, (7) research planning, (8) a proposed solution or final product, (9) data sources, and (10) a new application situation [7]. The teacher played the role of tutor, mediating learning, clarifying small doubts and checking whether students had access to all the relevant documentation. Notably, a collection of relevant educational resources to investigate the driving questions was gathered in advance and provided to all groups, as advised in the literature [11,40], so that the research carried out by students was not excessively time-consuming [7]. Students were also encouraged to access other sources they eventually found useful and perform their own investigations outside the classroom.

2.2. Contexts Explored in the Scenarios

PBL can be considered a method based on the principle of using real-life scenarios as a starting point for the construction of knowledge [40,41]. Thus, the contexts explored in each of the scenarios were diverse, but they always referred to problems based on facts. Regarding scenario 1, students were assigned the role of volcanologists and were confronted with a problem concerning the determination of the most suitable location for the construction of a health unit on Corvo Island in the Azores. In scenario 2, a short documentary on the Loma Prieta earthquake (San Francisco, CA, USA) was used to engage students in the subject and as an introduction to a fictitious dialogue between two friends who discussed the earthquake’s characteristics. Given the situation, each student was assigned the role of one character who must assess whether it would be possible to accurately predict the next earthquake and if it is advisable to warn the population of this occurrence, after analyzing the data collected. The third scenario, the abandoned Terramonte mine (Castelo de Paiva, Portugal), was based on a story built around the exploration of a published article from a daily newspaper about Terramonte’s mining legacy and the concentration amounts of chemicals that were polluting the Castanheira Stream, obtained from scientific studies conducted in that area. The students were assigned the role of the main character who read the newspaper article and decided to prepare a final report on the subject to be delivered to the president of the civil parish. The exploration of scenario 4, “the case of the mass movement in Maierato (Italy)”, began with the viewing of a news story on a landslide that occurred in that Italian region in 2010, so that students were made aware of the magnitude of this event while awakening their curiosity about the theme of slope stability. When reading the fictitious dialogue between two friends on the subject, students were challenged to plan experimental activities that could test the influence of various factors on mass movements.

2.3. Sample

The sample consisted of 33 students that enrolled in the GE curriculum at a public Portuguese university, 16 of which belonged to the experimental group and 17 to the comparison group, with a slight predominance of male students (54.5%) over female students (45.5%). Students in the comparison group were younger (average of 19.94 years old) than those in the experimental group (average of 21.56 years old). The traditional teaching methodology was mostly applied to students in the science and environmental technology (SET) degree course; the group submitted to the IP included students of SET, geology and biology in an approximately equivalent number (

Table 2. Sociodemographic characterization of participants in the study.

Variable	-	Experimental Group (n = 16)		Comparison Group (n = 17)		Total (n = 33)
-	-	f	%	f	%	f	%
Sex	Female	6	37.5	9	52.9	15	45.5
	Male	10	62.5	8	47.1	18	54.5
Age	M (SD) Min–Max	21.56 (2.943) 19–29		19.94 (1.029) 19–22		20.72 (2.295) 19–29
Degree course	SET	6	37.5	13	76.5	19	57.6
	Geology	6	37.5	4	23.5	10	30.3
	Biology	4	25.0	0	0	4	12.1
Curricular year	2nd	7	43.8	9	52.9	16	48.5
	3rd	9	56.3	8	47.1	17	51.5

Note: SET, Science and Environmental Technology; f, frequency; M, mean; SD, standard deviation; Min, minimum; Max, maximum.

).

Both groups had students from the second and third years of the courses mentioned in a similar number.

2.4. Data Collection, Instruments and Procedure

There were two types of data collection instruments: (i) monitoring sheets, structured as published in the literature [7], to record questions raised by students subjected to the IP; (ii) a knowledge test (pre- and post-tests), applied to both groups of students (experimental and comparison).

2.4.1. Qualitative Study

During the cyclical process inherent to the PBL method, the groups of students were asked to formulate all the “driving or problem questions” or “what the group needs to know” and register them in the corresponding space of the monitoring sheet after discussion within the student work group. Afterward, each spokesperson read the questions to the class. At this stage, there was also a distinction made regarding questions that should be more thoroughly debated and others considered marginal or with less importance, which were answered immediately with the assistance of the tutor teacher [7]. Research suggested that the formulation of questions (both at the individual and group levels) should always involve discussion within the class group under the teacher’s guidance so that a decision can be made together regarding it [42]. Therefore, a summary of all the questions raised by students was prepared, which was then shown in real-time to the class by the tutor teacher using a computer and a word processing program. Repeated questions on the same subject were eliminated as the intended study was of a qualitative nature. The above-described process was repeated for each of the four scenarios during the IP. After this data collection period, content analysis was used, following a closed classification procedure in accordance with the systematization prepared by the authors in the field [43].

2.4.2. Quantitative Study

Regarding the quasi-experimental study, the GE knowledge test (pre- and post-test) was used, totaling a maximum of 200 points. This data collection instrument consisted of a total of 30 questions distributed over five groups, covering the following themes: group of questions I (QI)—volcanology and prevention of volcanic risk, group of questions II (QII)—seismic evaluation, prediction and prevention, group of questions III (QIII)—slope stability, group of questions IV (QIV)—degraded mining areas and water contamination, and group of questions V (QV)—coastal erosion. Sixty minutes was the maximum timeframe for completion of the knowledge test. Students were asked not to leave items deliberately blank, especially those that required a more elaborate answer due to the question’s complexity.

All themes were addressed during the theoretical and practical components of the curriculum. However, in the experimental group, only the themes of the first four groups of questions were integrated in the IP; the last theme (group V—coastal erosion) was addressed in class using the traditional methodology. This set of higher cognitive level questions (QV) called for a more procedural reasoning toward a situation based on real facts and for which students would have to recommend solutions.

The pre- and post-tests were previously validated from the scientific and pedagogical point of view by two specialists with PBL and GE training, who also assessed the corresponding proposed criteria for correction. In addition, both pre- and post-tests were marked by the same researcher to avoid assessment biases.

The analysis of the collected data was made using SPSS Statistics software, version 25 (IBM, NY, USA). The statistical tests to be applied were selected considering the study goals and the nature of the variables to be analyzed. Thus, the development of substantive knowledge in GE students was measured: (1) by testing differences in a “between-subjects” design (i.e., comparison between independent groups: experimental vs. comparison); and (ii) by testing differences in a “within-subject” design (i.e., checking the differences wherein each group of individuals is compared with themselves between the pre- and post-tests in paired samples).

For the inferential statistical analysis, different nonparametric statistical tests were used to analyze the distributions under study: the Mann–Whitney test, to compare two independent groups, and the Wilcoxon test to compare paired samples (the changes from pre-test to post-test). The mean and standard deviation were also considered. In all statistical procedures, the significance value set was 0.05.

3. Results

3.1. Qualitative Study

We found that students raised elaborate questions regarding all categories (low and high cognitive levels), but not all subcategories were considered as the scenarios were explored. Thus, scenario 1 did not trigger solution-oriented questions and scenario 4 did not register any meaning-oriented questions. In general terms, we found a positive evolution in the questions raised from the first to the last scenario with a predominance of the category of productive questions or high cognitive-level questions compared to reproductive or low cognitive-level questions. In terms of subcategories, the more representative were relational questions, followed by encyclopedic questions, value-oriented questions, application questions, meaning-oriented questions and solution-oriented questions.

A more specific analysis showed that in the case of scenario 1, students emphasized low cognitive-level questions (encyclopedic questions). Concerning productive questions, value-oriented questions stood out. Notably, students mainly focused on the volcano’s geologic history and geotectonic framework, as well as on methods to minimize a region’s volcanic risk, namely regarding the parameters related to the construction of a volcanic risk map. In the following scenario (scenario 2), students already mastered high cognitive-level questions, and we noticed a diversification in all their subcategories (except for value-oriented questions and more emphasis on meaning-oriented and relational questions); a lower number of low cognitive-level questions were raised. Regarding the subjects, students assigned more importance to questions related to fundamental concepts of seismology, factors that may influence the occurrence of an earthquake and how to prevent and predict this phenomenon in terrestrial geodynamics. In scenario 3, high cognitive-level questions remained more prominent; the less formulated questions belonged to the subcategories of meaning-oriented, value-oriented and application. Specifically, the questions focused on the types of pollutants in the spoil piles, the impact on the ecosystem and on the surrounding population’s health, as well as solutions that could be implemented to mitigate or minimize the pollution effects caused by mine abandonment. An analysis of the students’ questions in relation to scenario 4 showed that high cognitive-level questions were prominent, with a predominance of the subcategory of relational questions and an absence of meaning-oriented questions. The focus of students’ interest in their research was on the influence of different factors on mass movements and their consequences, how populations can protect themselves from these events, and the role that science and technology could play in forecasting and alerting populations.

3.2. Quantitative Study

Before the IP’s implementation, the initial situation of students from experimental and comparison groups, with regards to the domain of conceptual contents, was measured via the application of a knowledge test (pretest) according to an intergroup research design. As shown in

Table 3. Statistical values of geology and environment students’ pretest, in global terms and for each group of questions (QI, QII, QIII, QIV, and QV), considering an intergroup design.

Pre-Test Questions	Group	n	M (SD)	Min–Max (Observed)	Min–Max (Theoretical)	U	p
QI	Experimental	16	7.13 (3.931)	3–14	0–20	109.5	0.345
	Comparison	17	8.06 (3.631)	3–14
QII	Experimental	16	15.50 (4.531)	7–22	0–48	130.0	0.845
	Comparison	17	16.47 (6.104)	6–29
QIII	Experimental	16	22.13 (8.966)	3–36	0–44	96.5	0.157
	Comparison	17	26.35 (5.820)	16–36
QIV	Experimental	16	6.25 (5.222)	0–13	0–26	135.0	0.986
	Comparison	17	6.00 (5.523)	0–18
QV	Experimental	16	11.88 (9.715)	0–29	0–62	112.0	0.402
	Comparison	17	15.59 (12.078)	0–43
Global Test	Experimental	16	62.88 (16.157)	38–86	0–200	100.0	0.204
	Comparison	17	72.47 (17.969)	52–120

Note: QI, QII, QIII, QIV, and QV—group of questions I, II, III, IV, and V, respectively; M—mean; SD —standard deviation; Min–Max—minimum and maximum value obtained corresponding to each group of questions; U—Mann–Whitney test.

, the experimental group had a mean of 62.88 points, slightly lower than the comparison group (72.47 points). In both groups (experimental and comparative), the knowledge pretest mean was below the score of 100 points, which is regarded as the conventional transition mark for a satisfactory level for the curricular content. In other words, the knowledge test required specific knowledge in the GE field, because the experimental and comparison groups’ means were below the pass value (100 points) in this first approach. In a more detailed analysis, for each group of questions included in the test, the participants produced similar results in all groups of questions (QI to QV), with no statistically significant differences between the experimental group and the comparison group, both in the pretest overall score (U = 100; p = 0.204) and in each of the groups of questions under analysis. Students submitted to the IP and to the traditional teaching methodology were equivalent in terms of knowledge in the assessed GE fields.

Regarding the results obtained in the overall post-test (

Table 4. Statistical values of geology and environment students’ post-test, in global terms and for each of the groups of questions (QI, QII, QIII, QIV, and QV), considering an intergroup design.

Post-Test Questions	Group	n	M (SD)	Min–Max (Observed)	Min–Max (Theoretical)	U	p
QI	Experimental	16	13.06 (4.328)	6–20	0–20	114.0	0.444
	Comparison	17	12.18 (3.957)	6–20
QII	Experimental	16	23.63 (8.326)	12–36	0–48	110.0	0.363
	Comparison	17	21.00 (5.788)	12–33
QIII	Experimental	16	24.13 (4.395)	17–32	0–44	76.0	0.031
	Comparison	17	28.06 (5.847)	17–38
QIV	Experimental	16	13.44 (6.022)	3–26	0–26	126.0	0.736
	Comparison	17	12.65 (7.297)	0–23
QV	Experimental	16	32.75 (9.335)	16–47	0–62	72.0	0.021
	Comparison	17	24.88 (8.964)	12–42
Global Test	Experimental	16	107.00 (16.629)	75–130	0–200	88.5	0.087
	Comparison	17	98.76 (17.559)	60–133

), which were marked by the same researcher to avoid assessment biases, the experimental group reached a mean of 107.00 points, a slightly higher result than the comparison group with a mean of 98.76 points. In both groups of participants, students’ knowledge post-test improved compared to the pretest and exceeded (in the case of the experimental group with 107.0 points) or approached (the comparison group, with a result of 98.76) the 100 points borderline reference, which is the mark that distinguishes satisfactory results.

Despite some differences in the results obtained in the two groups, no statistically significant differences (U = 88.5; p = 0.087) were observed in the overall test result. Therefore, the null hypothesis was supported, and the result obtained will be due to chance, given the lack of differences between the experimental and comparison groups regarding the “teaching methodology” variable (PBL vs. traditional methodology). However, as that p < 0.10 (p = 0.087), which is close to the 0.05 cut-off, this can be considered marginally significant [44,45]. By performing a more refined statistical analysis of the knowledge test, post-test results showed that in QIII and QV, there were statistical differences between participants. Regarding QIII, students in the comparison group obtained a higher mean (M = 28.06) than those in the experimental group (M = 24.13; SD = 5.847); in QV, the experimental group (M = 32.75; SD = 9.335) obtained a higher mean than the comparison group (M = 24.88; SD = 8.964). So the experimental group has a better performance in the higher cognitive-level questions that integrated QV, built about a theme not explored during the IP.

The analysis of the intragroup differences (paired samples) in terms of time, i.e., the pre- and post-test learning change, showed that both groups of students (experimental and comparison) evolved in different fields of knowledge in GE from one phase to the other (pretest and post-test). This difference was only insignificant in QIII (

Table 5. Statistical values of GE students’ change in intragroup learning (paired samples) in global terms and for each group of questions (QI, QII, QIII, QIV and QV).

Questions	Group	n	Post − Pre = Dif.	SD	W	p
QI	Experimental	16	5.93	3.53	134.5	0.001
	Comparison	17	4.12	4.68	132.0	0.008
QII	Experimental	16	8.13	7.80	124.5	0.003
	Comparison	17	4.53	7.62	122.5	0.029
QIII	Experimental	16	2.00	8.16	80.5	0.517
	Comparison	17	1.71	6.73	79.5	0.267
QIV	Experimental	16	7.19	7.47	123.5	0.004
	Comparison	17	6.65	7.16	125.5	0.003
QV	Experimental	16	20.88	13.53	132.5	0.001
	Comparison	17	9.29	8.89	130.0	0.001
Global Test	Experimental	16	44.13	20.27	136.0	0.000
	Comparison	17	26.29	17.84	153.0	0.000

Note: Post − Pre = Dif is the final point average (post-test) minus initial point average (pretest); W—Wilcoxon test.

). However, this group of questions (QIII) involved the theme of slope stability, which implied experimental activities classes performed by all students in this research study. These laboratory activities were planned in the GE curriculum. Thus, both groups performed laboratory work and, probably because they were submitted to equivalent strategies, they also responded similarly. This could be viewed as an external validation for IP. Nevertheless, the difference in the mean results between post- and pretest (post-test − pretest = difference) was greater in the experimental group (2.00 points) than in the comparison group (1.71 points).

The data in Table 4 show that, in all groups of questions, learning in the experimental group had a higher score difference between the post- and pretests (Post-Pre = Dif), indicating a more noticeable evolution in terms of learning. Nevertheless, this difference is significant in both groups (with p < 0.05 in the Wilcoxon test) except in QIII. It should be also stressed that this difference is greater in the QV group of questions (20.88 points in the experimental group vs. 9.29 points in the comparison group), which reinforces the fact that the experimental group seemed to accomplish greater achievements in terms of higher-order thinking skills. Given the results, we found statistical evidence that the results were different in the two groups in terms of the students’ development of substantive knowledge before and after implementation of the IP.

4. Discussion

4.1. Qualitative Study

The content analysis regarding the types of questions raised by the students showed that encyclopedic questions are more noticeable in scenario 1; in the remaining scenarios, the low cognitive-level questions were residual. As familiarity with the PBL methodology grew, the type of questions raised by students tended to be increasingly complex. It is considered that relevant issues in the PBL are those of an investigative nature, which should require, at least, meaning-oriented [42], value-oriented or solution-oriented [46] questions, which aligns with the qualitative results obtained in this study. The results obtained suggest that soft skills (e.g., communication skills) can be improved through the ability to pose more complex questions, which could be fundamental in a scientific context.

Our findings are in line with those reported by other studies [47], which stated the formulation of varied questions with a clear preponderance for higher level or productive questions (relational, value-oriented and solution-oriented). However, prior research conducted with students attending the third cycle of basic education (middle school) and secondary education (high school) showed a majority of encyclopedic questions’ elaboration to the detriment of high cognitive-level ones [48], or included, in addition to encyclopedic questions, a preferential distribution between the subcategories of meaning-oriented and relational [43].

4.2. Quantitative Study

Studies on PBL effectiveness in the development of knowledge in specific areas of learning show some ambiguity regarding the PBL methodology’s unequivocal proof of superiority over other teaching and learning methods. Our results seem to indicate that there is a measurable benefit in the area of GE, although it is not absolute in statistical terms, in comparison to traditional methodology. The results are relatively consistent with national and international literature in the field, from different perspectives, as will be discussed below.

Other studies reported statistically significant differences in intergroup design (PBL vs. traditional methodology) in terms of overall performance, conceptual and procedural knowledge, among other assessment aspects [49], or regarding scores from the attended training degree’s final exam, that is, in terms of academic success [50]. Given the scarcity of higher education studies in the area of GE in the Portuguese context, for example, in the subject of geography, with a ninth grade class, the PBL had a statistically significant impact on pre- and post-testing among students submitted to that methodology in relation to the comparison group [51]. Contrary to previous studies, we did not find a statistically significant (but only a marginally significant) intergroup difference, which may be due to the combination of several factors: the difficulty in controlling all the variables that influence students’ substantive knowledge evolution, the PBL methodology’s innovative nature in national higher education and the appropriation of its process being more demanding and requiring a longer IP time duration. However, in a meta-analysis [38], the assessed students’ scientific knowledge was similar when they were submitted to other types of methods. Similarly, a review of the literature [52] showed equivalent gains among teaching methods (PBL and others), noting, however, evidence that students work better in professional context activities and generally have a favorable impression of teaching using a PBL format.

In GE students’ intragroup learning design, the statistically significant improvement between the pre- and post-tests of students submitted to the IP is supported by a previous study [53], in which a comparative study was conducted between lecture teaching and PBL. The authors concluded that there were statistically significant differences between the pre- and post-test within each group of participants, as was verified in this work.

From our results, PBL enhances students’ development of substantive knowledge, and therefore, their learning of hard skills should benefit with this learning method, comparing to the traditional one.

Thus, PBL is a methodology that may lead to a positive evolution in the learning of curricular contents between different time periods.

5. Conclusions

In this study, we used two phases of distinct but complementary methods. The qualitative study focused on the driving questions about the different problem scenarios raised by students, with the corresponding classification from the reference literature on the subject. From a global perspective, the results point to an evolution toward an increased complexity in question elaboration, i.e., to the predominance of high cognitive-level questions in relation to low cognitive-level questions, as familiarity with the cognitive procedures inherent to the PBL methodology increased. The quantitative study pointed to apparently convergent meanings: (i) the difference between the experimental and comparison groups in the knowledge post-test is marginally significant, favoring the group undergoing the IP; (ii) the difference in learning changes within the same group of students is statistically significant for both types of methods but is more favorable in the experimental group. These students also achieved higher differences in all groups of questions regarding pre- and post-test evaluation (especially in QV, which was composed of high cognitive-level questions). The GE curriculum integrates knowledge areas aimed at solving problems for which exact responses do not exist or for which there are several solutions that reflect different theories. The PBL method’s educational potential was applied to resolve this type of problem, which requires a cyclical investigative path of a collaborative nature. We found that PBL may contribute to the improvement of multiple and diverse skills in higher education students, both soft and hard skills, sustaining more consistent academic training in line with 21st century challenges to which the Bologna Declaration attempts to successfully respond. Finally, we stress that PBL, as an active learning method, does not impair academic results compared with traditional classes. Moreover, PBL has characteristics that enhance science teaching, improves students’ communication and problem-solving skills and promotes an inquiry mindset, which can encourage teachers to be more proficient in this method, making it worthwhile to investigate PBL effectiveness, among other active learning methods.