Innovative Approach to Assist Architecture Teachers in Choosing Practical Sessions

Abstract

This article presents the first results of the project Architecture 360, which focuses on learning alternatives for developing working skills in higher education courses, and specifically construction competences for architecture students. The project aims to help teachers to choose the best learning solutions for their classes from numerous alternatives of strategies, dynamics and activities. The assistance is based on developing a new approach that combines several methods (strengths, weaknesses, opportunities and threats (SWOT); multi-criteria decision-making; Delphi; and the Knapsack problem) and draws from teachers’ experience, a panel of experts’ expertise, the revised Bloom Taxonomy and neuroscience for education. The new approach to assisting university teachers in choosing the best practical learning alternatives was successfully developed and validated for the case study of a course at Barcelona Architecture School. In general, the approach defined the main strengths, weaknesses, opportunities and threats of 26 learning alternatives. In the case study, the following optimized set of alternatives were identified: blended learning, challenge-based learning, reflective learning, videos of real cases, case studies, site visits, interactive simulation and gamification. Moreover, 23 activities were analysed. It was concluded, for instance, that active alternatives would improve implementation, including teachers’ available teaching materials and dedication outside class.

1. Introduction

Universities have many important roles in our society [1], such as providing a proper higher education learning environment for millions of future professionals each year [2]. However, some studies indicate that higher education studies cannot always provide graduates with skills and knowledge that meet employers’ expectations [3]. To achieve these expectations, universities and faculties follow diverse strategies [4]. For example, many institutions incorporate work-integrated learning (WIL; Appendix A presents a complete list of abbreviations) activities [5]. WIL alternatives include work placements and internship programmes [6].

Apart from these strategies, regular courses during university degrees aim to cover certain practical or professional competences of their students to promote readiness to practice [7]. These courses work on a specific part of the practical discipline. This part is learnt in depth, including theoretical and practical aspects, so that students gain related working skills. Numerous experiences are carried out, from site visits [8] to virtual reality applications [9]. These alternatives are not used excessively and the same course can combine several of them [10]. Nevertheless, COVID-19 pandemic prevention and lockdown measures increased the need for blended learning [11] and e-learning [12]. All these experiences and alternatives have singular, specific characteristics. For instance, in terms of experience, some are well known and applied, while others are new and under implementation, like face-to-face and virtual laboratories [13]. They also vary in the dedication required of students and teachers, and in students’ engagement.

Choosing the most suitable practical alternative or set of experiences for a specific course is a crucial multi-criteria decision-making process that should consider the characteristics of the alternatives and many other factors. This article divides the factors into stakeholders and contextual aspects. Numerous stakeholders are involved in higher education [14]. However, students and teachers are the main stakeholders in the learning process, which is the focus of this research paper. In addition, various contextual factors [15] are considered crucial in this project. These are the definition of courses (the objectives, competences, contents and assessment) and aspects of the institutions (budget, spaces, resources, programmes and industry). For example, the level of collaborative complicity between professionals and the university is fundamental, as are the laboratories available for each degree. Among other factors, students’ learning processes and cognition level [16] and aspects of educational neuroscience [17] should also be considered.

This article presents a new approach that aims to assist teachers in choosing the best set of activities for the practical sessions of a specific course. The approach considers the characteristics of the learning alternatives, stakeholders and contextual factors. It draws on a previous review of related technical literature [18], based on which the authors define this new approach and its six main steps following Delphi, expert seminars and focus groups. This article presents the first version of the approach, which focuses on practical learning in architecture schools. The steps include a multi-criteria decision-making methodology called an integrated value model for sustainability assessment (MIVES) and a Knapsack algorithm, which is based on a similar approach by the authors that successfully helped teachers to choose the most suitable active learning activities in lectures for large groups [19]. Thus, the main difference in the present approach is its focus on practical sessions, and there is a general improvement based on the results of the previous approach’s implementation. To validate this new approach, the authors applied it for the first time to a specific course at the Universitat Politècnica de Catalunya (UPC). The next section describes potential learning alternatives. Then, Section 3 presents the new approach, Section 4 identifies the problem, and Section 5, Section 6 and Section 7 are the results, discussion and conclusions of applying this approach for the first time.

2. Alternatives Analysis

To identify the main experiences used for university students to learn specific work-related competences, a literature review was carried out in July 2021 [18]. This review analysed the number of related publications in the Web of Science (WoS) Core Collection database [20]. It considered 64 results from a search on university active learning activities for working and practical learning, and 86 results from another search focused on WIL activities. Studies that were found during other steps of this research project were also considered. Other experiences that have not been reviewed or published yet are expected to be considered in future research phases. The review followed a rigorous methodology [21], to include the maximum number of experiences. Publications were analysed from the perspective of general to detailed issues, and factors from the publication date to learning alternatives were considered. Depending on the approach to the learning scale, the review considered three types of alternatives: (t1) online strategies, (t2) learning dynamics and (t3) activities during class or outside the university. At the same time, considering the learning methodologies, the review classified three main interrelated groups of alternatives: (g1) recent digital technologies, (g2) active learning and practical activities, and (g3) real experiences. Table A2 in Appendix B presents these alternatives.

The first type, t1, includes blended and e-learning alternatives, which represent two different intensities in the use of online resources within g1. The first combines online and face-to-face learning, while the second exclusively uses online resources and communication. The second type, t2, mainly encompasses active learning and practical experiences. Active learning includes challenge-based learning (CBL) and team-based learning (TBL), like modified case-based learning exercises called active learning groups [22]. Practical experiences include degree apprenticeships [23], placements and dual vocational education and training (VET) [24]. The third type, t3, has activities within all the groups of alternatives. The first are activities based on recent digital technologies. They may involve experiences that are part of blended and e-learning (e.g., practical activities on the web, such as practical active learning stations) [25] or virtual learning activities (such as virtual laboratories and virtual learning explorative activities) [13]. Other active learning activities include case studies and storytelling experiences. Finally, real experiences are carried out, such as active practices with real material [26], role play or work activity simulation [27], and onsite visits to observe professionals in the workplace [28].

The outcomes of the WIL experiences that were highlighted by the studies were generally the contribution of these experiences to personal and professional growth. During WIL, the co-presence of industry members and teachers is essential. Other valued points were, from major to minor importance: the work–study–life balance, industry involvement and support for WIL activities, cases of parallel rather than integrated learning in which university and industry synergies are insufficient, equity among students’ opportunities, cultural dissonance, for instance between students and the placement environment, competences, technology integration and employability.

3. Methodology

To find the best methodology to help teachers to choose and organize their practical sessions, the authors relied on a review of the assessment of learning activities (Lr2), in which up to 201 publications were eligible and were analysed. Lr2 identified similar previous studies. However, they were limited to fewer indicators and alternatives and used methodologies that were more appropriate to these limits. The methods used in these studies were diverse: surveying and interviewing (30 publications), quasi-experimental design (25), statistical analysis (22), qualitative analysis (22), quantitative analysis (12), case studies (9) and other methods in the rest of the publications. The closest methods to this article’s new methodological approach were the development of frameworks in some studies [29,30,31]. This research project follows the six-stage methodology presented in Figure 1, which relies on a similar former project by the authors on the subject of lectures [19].

Figure 1. The six-phase methodological approach developed in this project, Architecture 360. Legend: SWOT—strengths, weaknesses, opportunities and threats.

This approach was chosen because, based on its previous application, it can give useful advice to teachers about which alternatives or sets of alternatives are most suitable for their practical sessions. The approach can include the main indicators, adapted to the characteristics of any case study, assess any type and number of alternatives, and provide integrated quantitative results.

The following subsections explain these six phases: in (P1) and (P2), the teaching team classifies the professional contents and practical alternatives respectively; in (P3), the MIVES–Delphi tool is used to assess these alternatives; in (P4), the teaching team estimates the available and required sessions; in (P5), the Knapsack algorithm suggests sets of alternatives for the practical sessions; and in (P6), the teaching team analyses these sets and reaches a final decision.

3.1. Phases 1 and 2

During these first phases, the teaching team for the course follows Bloom’s Taxonomy, revised by Anderson [32], to classify the practical course contents and feasible alternatives. The alternatives are also studied according to the strengths, weaknesses, opportunities and threats (SWOT) technique [33]. The resulting matrix depicts the main characteristics of the alternatives. SWOT results are based on an extensive literature review [18] and contain strengths and weaknesses that are more commonly related to internal issues, and opportunities and threats that are focused more on external factors [34]. This taxonomy considers three thinking levels: (a) lower-order thinking level (LOTL), such as remembering and recounting concepts; (b) middle-order thinking level (MOTL), such as applying and understanding ideas; and (c) higher-order thinking level (HOTL), such as analysing, evaluating and creating your own proposals. These practical contents are the course-related working skills that students are expected to acquire, as explained in the introduction to this article. The feasible alternatives are practical alternatives from the review in the previous section and other sources that the teachers consider applicable to their course context.

3.2. Phase 3

This third phase uses the integrated value model for sustainability assessment (MIVES) because a multi-criteria decision-making method was needed to assist teachers in taking multi-criteria decisions that have multiple indicators with different values and tendencies. Among the available multi-criteria decision-making methods [35], MIVES was chosen because it is a consolidated method that has been successfully applied since the 2010s [36]. It incorporates value functions that allow integrated assessments of various indicators with different units and tendencies. It provides a global sustainability index and partial sustainability indexes. It enables agile sustainability assessments for specific case studies and boundaries. Finally, it can be combined with robust weighting methods such as Delphi. Thus, Phase 3 defines the assessment limits, considering contextual factors, among others. Then, it builds a requirements tree (RT) based on related technical literature. It weights requirements, criteria and indicators following Delphi. Next, it defines the indicators’ value functions and finally it assesses the alternatives.

This assessment system’s limits are higher education alternatives that cover certain professional competences to promote students’ readiness to practice, as explained in the previous sections.

Table 1 presents the requirements tree with its indicators, criteria and requirements based on: (1) the aforementioned MIVES for large groups [37], (2) the Lr2 state of the art, and (3) the characteristics of practical sessions, drawn from the authors’ experience [10] and the literature review [19]. Requirements tree components were defined to obtain a limited number of discriminative indicators, as required by MIVES [38]. The definition process involved including (a) three indicators from Lr2 and (b) three indicators that are crucial for practical alternatives, and (c) adapting and complementing two former indicators, so that they do not overlap with the former indicators. These are presented and explained in Table A3 in Appendix B. This process also discarded three indicators that were used in the predecessor. One of these indicators was excluded because it focuses on innovation in lectures. The other two, which are related to stakeholders’ satisfaction, were not considered in this study as sole indicators because the new indicators b2, b3, c1 and c2 already include them. The definition process included three new criteria (C3, C6 and C7), following the reasons explained previously for the indicators. The requirements are the three main sustainability pillars (economic, environmental and social), plus the applicability requirement from the expert seminars.

Table 1. Decision-making requirement tree with weights as a percentage.

Requirements	Criteria	Indicators
R1. Applicability (26%)	C1. Application (58%)	I01. Ease of application (53%)
		I02. Flexibility for adaptation (47%)
	C2. Transferability (42%)	I03. To other teachers (57%)
		I04. To other disciplines (43%)
R2. Economic (21%)	C3. Cost (38%)	I05. Direct costs (45%)
		I06. Logistic and scheduling issues (55%)
	C4. Time (62%)	I07. Dedication in class (39%)
		I08. Teachers’ dedication outside (28%)
		I09. Students’ dedication outside (33%)
R3. Environmental (11%)	C5. Impact (100%)	I10. Extra-environmental impact (100%)
R4. Social (42%)	C6. Learning process (Chickering and Gamson principles, among others) (35%)	I11. Roles, talents and ways of learning (23%)
		I12. Encouraging cooperative work (28%)
		I13. Autonomous work (30%)
		I14. Students’ cognitive load (19%)
	C7. Interaction (Chickering and Gamson principles, among others) (40%)	I15. Students’ interest and participation (33%)
		I16. Students and faculty contact (22%)
		I17. Feedback to students’ time (19%)
		I18. Learning outcomes (cognition and affect) (26%)
	C8. Innovation (25%)	I19. University learning (53%)
		I20. Teachers’ new functions (47%)

This project followed the Delphi-based approach explained by Casanovas-Rubio and Armengou [39] to define requirements tree weights, choose the experts who participated in assigning the weights and manage the related surveying process. First, the authors used Delphi to select and ask the participation of 22 qualified experts, the expertise and main data of which is summarized in Table A4 in Appendix C. Twenty of them completed the surveying procedure and proposed weights by the direct assignment method. The number of experts was higher than the recommended minimum number of panellists for this approach [39]. Consensus of the experts was reached when the median absolute deviation was below one-tenth of the possible values range, which was 10% because the range of weights was from 0% to 100%. The project needed two rounds of surveys to reach this consensus. The first round invited experts to propose weights for the indicators, criteria and requirements of this project requirements tree, according to their judgement. The only condition was that the total weight of the sum of requirements and each group of criteria and indicators must equal 100%. The second round invited experts to consider adjusting their own first-round weights, taking into account the mean values of the first round and keeping their weights within their preferences. Experts were asked to justify any weights that deviated more than 10% from the mean of the weights in the first-round survey. To reduce judgement-based bias, the two rounds of the surveys had randomized question order, iteration and anonymity. Table A5 and Table A6 in Appendix C show the results of these two rounds.

Then, the project defined the assessment of each indicator considering the previous MIVES for lectures and related technical literature. The result was value functions for each indicator. The values ranged from 0 to 1, which represented maximum and minimum satisfaction, respectively, for the 20 indicators’ values. The addition of each set of indicators’ adimensional values (V_i,k) resulted in eight criteria satisfaction values (V_CRi,k), each set of which could be added to obtain four requirement satisfaction values (S_Ri,k). Finally, the addition of the requirement satisfaction values resulted in the global sustainability index (GSI_k). The calculations follow Equations (1)–(3).

$$V_{CRi,k}=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{i,k}·V_{i,k}(x_{ind})$$

(1)

$$S_{Ri,k}=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{CRi,k}·V_{CRi,k}$$

(2)

$$GS{I}_k=\underset{i=1}{\overset{j}{\Sigma }}{\lambda }_{Ri,k}·S{I}_{Ri,k}$$

(3)

The definition of the value functions depends on the shape that each indicator assessment requires. Most functions are defined by five parameters, as shown in Equation (4). These parameters determine the shape of the function and how each indicator variation is transformed to the 0 to 1 scale.

$$V_{ind}=B\cdot \left[1-e^{-ki\cdot {\left(\frac{\left|Xalt-Xmax\right|}{Ci}\right)}^{Pi}}\right]$$

(4)

These five parameters are as follows: (1) Xalt is the abscissa for each assessed alternative indicator that generates a V_ind value; (2) Pi is a shape factor that determines the curve shape, such as concave, convex, lineal and “S” shaped; (3) ki defines the value for the ordinate point Ci (4); and (5) B is the factor that is capable of maintaining the function within the adimensional range, which Equation (5) depicts.

$$B={\left[1-e^{-ki\cdot {\left(\frac{\left|X\max -X\min \right|}{Ci}\right)}^{Pi}}\right]}^{-1}$$

(5)

The functions for indicators I13 and I20 are different because they have an increasing tendency of satisfaction up to a maximum value, from which they decrease back to zero. In consequence, these functions follow the quadratic Equation (6).

$$Y=a{X}^2+bX+c$$

(6)

Table 2 presents the definition of the indicators’ assessment and Table 3 the units and function parameters of this study of new indicators. Most indicators maintain the units and function shapes assigned in the lecture-based MIVES. Some indicators have changed their code: indicators I07, I08, I09, I10, I11, I12, I16, I17 and I19 correspond to the previous study’s [19] indicators I05, I06, I07, I08, I12, I10, I11, I09 and I14, respectively, and their respective value functions. The six new indicators I05, I06, I13, I14, I15 and I18 have linear functions for this first application of the new approach, while other shapes that are more sensitive to each indicator’s tendency will be considered in future applications. The function for indicator I20 has a quadratic equation that: (a) crosses the origin at X = Y = 0; (b) increases up to the vertex at Y = 1; and (c) decreases symmetrically until it crosses the X axis at X = 1.

Table 2. Definition of the indicators’ assessment.

	Assessment Parameters
I01	Work required before, during and/or after class, i.e., prepare and handle material, correct, give feedback
I02	Adaptability to different space, time and resource characteristics, i.e., spaces, types or number of sessions
I03	Available technical literature on each alternative, its relation to the case study and ease of use
I04	Number and interdisciplinarity of the related 6-digit UNESCO nomenclature on areas of expertise
I05	Costs per student that must be covered to complete each activity and comply with legal and ethical issues
I06	Number and complexity of logistic and scheduling arrangements required for each alternative
I07	Average time dedicated per session during class, by teachers outside of class, or by students after class, respectively
I08
I09
I10	The energy consumption and waste generation that each alternative involves
I11	The number and diversity of roles, styles, approaches and methods that each alternative allows
I12	The extent to which each alternative incorporates, allows and promotes teamwork and dynamics
I13	The extent to which each alternative enables and promotes students’ autonomous learning processes
I14	Cognitive load theory (CLT), according to the information processing model and methods, to manage the cognitive load
I15	Students’ satisfaction with engagement, interaction, participation, attendance, diversion and expectations
I16	Promotion of contact between students and faculty and a sense of belonging to the institution
I17	Average feedback time from teachers on students’ tasks and activities
I18	Average quantitative and qualitative results from grades and commentaries
I19	Previous university research projects and technical literature about each alternative
I20	Number and complexity of new concepts and skills required by teachers for each activity

Table 3. Indicator units and function shapes.

	Unit	Equations	Shape	Xmin	Xmax	C	K	P	a	b	c
I05	Points	(4) and (5)	DL	0	100	50	0.05	1	-	-	-
I06	Points	(4) and (5)	DL	0	100	50	0.05	1	-	-	-
I13	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I14	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I15	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I18	Points	(4) and (5)	IL	0	100	50	0.05	1	-	-	-
I20	Points	6	Pb	-	-	-	-	-	−0.04	0.4	0

3.3. Phase 4

The teaching team estimates the available practical sessions for the studied course and the minimum number of sessions for each learning alternative, relying on their experience and the previous literature reviews.

3.4. Phase 5

The Knapsack algorithm generates proposals of sets of activities based on the estimation from previous sessions. This algorithm is used because it provided similar sets in the previous methodology for large lectures [37] and it was successfully combined with MIVES in several former research projects [40]. Knapsack maximizes some values according to each study measure to obtain one or more sets that have values equal or less than the established measures and have the maximum satisfaction for the main defined value. In this study, the best set of alternatives was chosen for the practical sessions in a full semester. Thus, the two parameters of the Knapsack problem are that the value is the GSI and the weight is the number of sessions these activities need to be implemented.

3.5. Phase 6

In this last phase, the teaching team chooses the best set of alternatives, taking into consideration the Knapsack results and their experience, expertise, the technical literature, Bloom’s Taxonomy revised by Anderson, and neuroscience in education [41]. The Knapsack results include the MIVES–-Delphi assessment (Section 3.2, Phase 3), which is based on feasible alternatives from the SWOT analysis. The teachers’ final decisions consider the specific needs and context of each course, group and students. They check the automatic results and, for instance, make sure that the thinking level of the course contents is compatible with the activities (Phase 1).

4. Identification of the Problem

This new methodology was applied to the Construction II degree course at Barcelona Architecture School (ETSAB). Since 2019, the teaching hours of this course have been divided into 40% lectures and 60% practical sessions, to develop in depth the course contents from theoretical and practical perspectives, so that students gain the related professional skills and competences [42]. This course is a mandatory undergraduate third-year course that has four sessions per year, two each semester, one morning shift and one afternoon shift, with around 80 and 60 students, respectively. Since the pandemic, this course has combined blended and e-learning, depending on the restrictions.

Construction II invites students to ask themselves about how architects construct buildings’ structures, and what architects should consider when they design, represent and supervise the construction works of their architectural projects to build the best performing architectural structures [43]. Furthermore, teachers encourage meta-cognitive development and deep learning through extra activities to discuss the course curricula that, along with the objectives, are aligned with these main questions on the course. This course covers transversal competences, such as teamwork and autonomous learning, general competences, such as understanding structural design, construction and engineering architectural problems, and specific competences, such as knowledge of offsite construction systems [42]. It includes formative and summative assessments, with partial accumulative continuous assessments, submission of a practical assignment, two theoretical exams during the course, and an optional final exam including theoretical and practical parts. Since the last decade [19], teachers of this course have tried to improve students’ learning process by understanding their background and improving their motivation and learning autonomy. To achieve this, teachers continuously collect information from questionnaires, surveys and informal encounters at the beginning, middle and end of the course, as presented in previous articles [18,37,44]. These assessment results justify the present research project and its characteristics, including its participative approach (Figure 1) and its new indicators (Table A3 in Appendix B).

The practical sessions in this course cover three main topics: site soil, foundations, and structures of buildings. Classes on soil focus on which data are required from the soil to design buildings’ foundations and structure, and how architects can obtain this soil information. Foundation and structure activities cover ways that architects can design these elements to obtain the best results, construction processes for these elements, and how architects can provide specifications and draw details of their designs to optimize the construction processes and outcomes of architectural structures. Until now, this has been achieved mainly following team-based learning, case studies, problem-solving and hands-on activities organized around project-based learning (PBL) on an architecture project.

5. Results

This section presents the results of applying the new approach presented in Figure 1 to the Construction II course. The outcomes of each phase are detailed.

5.1. Phase 1

The learning activities involved in the course’s practical sessions and their respective thinking levels are: (1) read and understand soil data sources, MOTL; (2) propose a justified hypothesis of soil for a specific building site, HOTL; (3) extract data from this soil that students will require for their next steps, MOTL; (4) propose specific foundations and structure for a particular soil site and building design, HOTL; (5) design these specific foundations and structure following given methods, MOTL; (6) give specifications about these structural elements following given instructions, MOTL.

5.2. Phase 2

The aforementioned literature reviews [18] classified these alternatives into three types and three groups. Appendix B, Table A2 classifies these learning alternatives and gives references to understand them in detail. These reviews also classified them depending on thinking levels from Bloom’s Taxonomy, revised by Anderson, on which these alternatives could work. The classifications and the SWOT analysis (Table A7, Table A8, Table A9 and Table A10 in Appendix D) confirmed that most of the activities were feasible alternatives for the case study. However, there were three alternatives that could not be introduced due to their incompatibility with the undergraduate course organisation and students’ schedule. These were internships, placements and dual vocational education and training (VET). Currently, some Construction II students carry out these activities outside of the framework of the course. Thus, feasible alternatives are: (A1) blended learning, (A2) e-learning, (A3) technology-enabled active learning, (A4) challenge-based learning (CBL), (A5) team-based learning (TBL), (A6) flipped classrooms, (A7) project-based learning (PBL), (A8) reflective learning, (A9) industry–community projects, (A10) interactive simulations, (A11) social media activities, (A12) videos of real cases, (A13) virtual learning activities, (A14) case studies, (A15) discussions, (A16) gamification activities, (A17) interdisciplinary activities, (A18) problem-solving activities, (A19) storytelling, (A20) real material practices, (A21) hands-on activities, (A22) role play and (A23) site visits.

5.3. Phase 3

The global sustainability index (GSI) and the requirement satisfaction values were the main results of this phase. Table 4 presents the requirement satisfaction values and GSI, while Table A11 in Appendix E presents the complete results with all the indicators and criteria satisfaction values. These tables show that challenge-based learning (CBL), reflective learning and case studies achieved the highest GSI of 0.71, while industrial and community projects achieved the lowest GSI of 0.56. Thus, the range of GSI was only 0.15 points. The interval of values for the satisfaction of all requirements was from 0.38 to 1.00. In the indicators of requirement satisfaction, social media and hand-on activities had values of 0.84 and 0.47 for applicability satisfaction (R1), respectively. For the economic requirement (R2), discussions and storytelling were rated 0.94 and 0.95, respectively, while industry–community and interdisciplinary projects had a 0.41 satisfaction value. For the environmental requirement (R3), videos of real cases, storytelling, real material and hands-on activities achieved complete satisfaction, while many activities achieved 0.56 (A4, A6, A7, A10, A11, A13 and A15). Interdisciplinary activities achieved 0.68, while e-learning had a value of 0.38 for social requirement satisfaction (R4). Considering all the alternatives, the highest average satisfaction was in the category of R2 economic, while the lowest was in R4 social.

Table 4. Global sustainability index (GSI) and requirement satisfaction.

Code	Alternative	R1	R2	R3	R4	GSI
A1	Blended learning	0.66	0.79	0.69	0.60	0.67
A2	e-learning	0.78	0.77	0.57	0.38	0.59
A3	TEAL	0.64	0.68	0.57	0.56	0.61
A4	CBL	0.69	0.88	0.56	0.68	0.71
A5	Coop. TBL	0.66	0.80	0.73	0.55	0.65
A6	Flipped learning	0.59	0.66	0.56	0.57	0.59
A7	PBL	0.71	0.77	0.56	0.55	0.64
A8	Reflective learning	0.72	0.89	0.91	0.57	0.71
A9	Ind. com. proj.	0.63	0.41	0.69	0.57	0.56
A10	Interact. simulation	0.64	0.67	0.56	0.57	0.61
A11	Social media	0.84	0.90	0.56	0.51	0.68
A12	Videos of a real case	0.78	0.84	1.00	0.51	0.70
A13	Virtual learning	0.75	0.67	0.56	0.44	0.58
A14	Case studies	0.77	0.86	0.81	0.58	0.71
A15	Discussions	0.51	0.94	0.56	0.67	0.68
A16	Gamification	0.62	0.74	0.81	0.68	0.69
A17	Interdisciplinary	0.57	0.41	0.69	0.69	0.60
A18	Problem-solving	0.75	0.91	0.81	0.42	0.65
A19	Storytelling	0.56	0.95	1.00	0.49	0.66
A20	Real material	0.68	0.86	1.00	0.43	0.65
A21	Hands-on activities	0.47	0.79	1.00	0.63	0.66
A22	Role play	0.71	0.81	0.91	0.50	0.66
A23	Site visits	0.60	0.60	0.91	0.51	0.60

Legend: technology-enabled active learning (TEAL), challenge-based learning (CBL), project-based learning (PBL), team-based learning (TBL).

5.4. Phase 4

The teaching team defined two scenarios of a maximum number of practical sessions per semester (12 and 14). This number can change because of external factors, such as the university calendar and local restrictions, including compulsory or unforeseen days off. The team also prepared the classification presented in Table 5, which organizes alternatives according to their exclusivity and the number of minimum required sessions. The first three alternatives can be combined with any of the other 20 following learning options, because the first two are general course strategies and the second is a combinable learning dynamic (see Phase 2). The other 20 alternatives require exclusive sessions. Thus, to be used in the course that was studied, these alternatives require a minimum of 1, 3 or 8 specific sessions. These numbers could vary in other contexts.

Table 5. Minimum number of sessions the activities require to be implemented.

Exclusivity	Type and Group	Minimum Required Sessions	Alternatives
Can be combined with any activity	t1 or t2, g1	1	(A1) Blended learning, (A2) e-learning, (A3) TEAL
Requires exclusive session/s	t2 or t3, g2 or g3	1	(A5) TBL, (A6) flipped classroom, (A8) reflective learning, (A10) interact. simulation, (A11) social media, (A12) videos of a real case, (A14) case studies, (A15) discussions, (A16) gamification, (A18) problem-solving, (A19) storytelling, (A20) real material, (A22) role play, (A23) site visits
		3	(A13) Virtual learning, (A21) hands-on activities
		8	(A4) CBL, (A7) PBL, (A9) ind.-com. proj., (A17) interdisciplinary

Legend: technology-enabled active learning (TEAL), challenge-based learning (CBL), project-based learning (PBL), team-based learning (TBL).

5.5. Phase 5

In the first scenario of a 12-session course, the best Knapsack results were (a1) among the first three alternatives and, giving a total GSI of 0.66, blended learning (A1) was selected for all sessions; and (b1) among the other 20 learning options, with a total GSI of 0.72, CBL (A4) was selected for 8 sessions, and videos of real cases (A12) for 4 sessions. In the second scenario of a 14-session course, the best sustainability results were (a2) among the first three learning alternatives, giving a total GSI of 0.65, A1 was selected for all sessions; and (b2) between the other alternatives, with a total GSI of 0.72, A4 was selected for 8 sessions and A12 for 6 sessions.

5.6. Phase 6

The course teachers’ final decision ratifies Knapsack proposals for the first three alternatives (a1 and a2), although A1 could be replaced in the case of external conditions. For example, in the case of pandemic lockdowns, e-learning (A2) would be extended to all sessions, to achieve a lower GSI of 0.59. Among the other alternatives, teachers proposed: (b1) for the 12-session course scenario, with a total GSI of 0.71, CBL (A4) for 8 sessions and reflective learning (A8), videos of real cases (A12), case studies (A14) and site visits (A23) for one session each; and (b2) for the 14-session course scenario, with a total GSI of 0.70, CBL (A4) for 8 sessions and, A8, Interactive simulation (A10), A12, A14, Gamification (A16) and A23 for one session each. To sum up, the following changes were applied: (a) multiple activities that required at least one session were applied instead of repeating one activity 4 or 6 times; (b) activities A8, A10, A14, A16 and A23 were added. These changes were applied, although the GSI was slightly lower because the range of activities could improve students’ learning process by introducing variety and surprise factors that could further engage and awaken students’ brains with joy and wonder [45]; A8 allows work on the important aspect of students’ meta-cognition; A10 is an online activity that the teaching team is developing as an alternative to A23 in the case of threats (Table A10 in Appendix D); A14 works with real cases as examples that help students’ understanding [46]; A16 is a learning-by-playing alternative that also increases students’ engagement and readiness to learn [41]; and A23 works with a real environment so that students learn from a different, unique perspective.

6. Discussion

The first phase of this study proves that most alternatives are compatible with the medium- and higher-order thinking level of this course’s practical sessions [18], although CBL, PBL and discussions are optimized for HOTL contents. Therefore, the four MOTL activities that were presented in phase one’s results cannot be resolved with these alternatives. These results explain why, during previous courses, some MOTL activities such as extracting, reading and understanding data were difficult to perform using PBL. The SWOT confirms the potential and threats of the 26 alternatives that were studied. The complexity of the alternatives justifies the need for a methodology such as that developed in this project to address implementation in courses. In terms of the types and groups of practical session alternatives (Table A2, Appendix B), the three activities with the highest GSI are active learning activities (g2). Each of the 23 alternatives has its own performance regarding each requirement’s average satisfaction: recent digital technologies (g1) have the highest satisfaction value for applicability because of their notable performance; active learning alternatives (g2) have the highest satisfaction value for economic issues due to their low costs; and real experiences (g3) have the highest environmental value because of the low added environmental impact. The results for these alternatives are in relative terms and are not applicable to the environmental impact of higher education activities and facilities that require specific studies [47] beyond the boundaries of this research project.

The resulting requirements tree (Table 1) includes the 20 main discriminative indicators classified into 4 requirements: applicability, economic, environmental and social. This classification allows the study of satisfaction in each sustainability branch. The resulting GSI and requirements for the sustainability indexes can be applied to the context of the specific case study, which focuses on the learning process (Table 2). Satisfaction with the applicability requirement ranges from 0.47 to 0.84, with an average satisfaction of 0.67/1.00. This is due to the ease of application of most alternatives (I01) and their flexibility (I02). However, in general, the alternatives are difficult to transfer among teachers (I03). Thus, to improve their applicability, more and better material should be available to the teaching community, especially regarding new digital alternatives (A10, A13 and A16). Satisfaction with the economic requirement is even more varied as it ranges from 0.41 to 0.95, with an average satisfaction of 0.77/1.00. This is due to the general notable satisfaction with logistic issues (I06) and students’ dedication after classes (I09), good direct costs (I05) and dedication in class (I07), and fair satisfaction with teachers’ dedication after classes. Satisfaction with the environmental requirement is high and medium for all alternatives. It ranges from 0.56 to 1.00, with an average satisfaction of 0.74/1.00. This confirms that all the assessed learning activities have a similarly low extra-environmental impact. Satisfaction of the social requirement is lowest. It ranges from 0.38 to 0.69, with an average satisfaction of 0.55/1.00. One reason is the general complementary behaviour between the capacity to encourage cooperative (I12) or autonomous work (I12), in which most alternatives perform outstandingly in one, but poorly in the other, except for some alternatives of A4, A7, A8 and A16. Indicators I19 and I20 detect diversity within the activities’ innovation and teacher training to achieve new skills to apply the alternatives. No relation was found between these GSI or requirement satisfaction (Table 4) and the minimum sessions required for a learning alternative to be applied (Table 5).

Using the scenarios in Table 6 and data, a sensitivity analysis further analyses these course context implications and the robustness of the new approach. The five scenarios are: (Ws1) this project’s reference weights (Table 1); (Ws2) a neutral scenario with the same weight for each requirement; (Ws3) prioritizes the applicability of the learning alternatives with the highest weight; (Ws4) gives more weight to the economic requirement because the cost issues are considered crucial; and (Ws5) focuses on social issues and gives the highest weight to this requirement. This research did not consider an environmental requirement-driven scenario due to its limited importance in this project in terms of weight and number of indicators.

Table 6. Description of the weighting scenarios considered in the sensitivity analysis.

	Weighting Scenario Description	R1	R2	R3	R4
Ws1	This research project weighting, based on Delphi	26	21	11	42
Ws2	Equal weights for all indicators	25	25	25	25
Ws3	Applicability requirement-driven scenario	55	15	15	15
Ws4	Economic requirement-driven scenario	15	55	15	15
Ws5	Social requirement-driven scenario	15	15	15	55

The sensitivity analysis presented in Figure 2 and Figure 3 confirms the robustness of this approach, which presents a similar tendency for most alternatives. Exceptions are, for example, (A15) discussions that have higher satisfaction in a cost-driven scenario, and (A21) hands-on activities with lower satisfaction in an applicability-driven scenario. On the other hand, some have more similar GSI in all scenarios (A3, A6, A10 and A23) than others (A2, A9, A11 and A15). Moreover, alternatives A2, A11, A12, A15 and from A18 to A23 have a broader difference between scenarios.

Figure 2. Sensitivity analysis for the five different weighting scenarios.

Figure 3. Sensitivity analysis for the 23 assessed learning alternatives.

Phase 5 is a crucial step that gives a set proposal to teachers based on the new methodology (Figure 1) that mitigates bias possibility in the outcome. This new methodology calculates the resulting GSI of these two sets and other alternative sets and can relate them to current course scenarios. Combining MIVES and Knapsack allows a sensitivity analysis to be carried out to obtain the best set of learning alternatives for each of the five scenarios in Table 6 and their GSI. Table 7 presents the combined analysis results. It gives the expected outcomes because the best performing learning alternative for each requirement with the best GSI is the chosen alternative for each requirement-driven scenario. By mixing results from different scenarios, the teachers’ proposal can be reached (phase 6). This result should be further investigated, if it can be used for future automated versions of this research’s new methodology.

Table 7. Sensitivity analysis combining MIVES and Knapsack.

Weighting Scenarios	12 Sessions		14 Sessions
	Proposed Set	GSI	Proposed Set	GSI
Ws1	A4 (8); A12 (4)	0.72	A4 (8); A12 (6)	0.72
Ws2	A12 (12)	0.78	A12 (14)	0.78
Ws3	A12 (12)	0.78	A12 (14)	0.78
Ws4	A19 (12)	0.83	A19 (14)	0.83
Ws5	A8 (2); A16 (10)	0.70	A8 (4); A16 (10)	0.70

However, in this present version, Phase 6 is essential to adapt better the set solution from Phase 5 to the specific students and context, relying entirely on the teaching team’s thinking process. In the case study, the Phase 6 solution has a GSI of 0.71. This is a 9% improvement on the current practical sessions in Construction II that have a GSI of 0.65: PBL (A7) in 8 sessions, plus cooperative learning (A5), case studies (A14), problem-solving activities (A18), hands-on (A21) activities and site visits (A23) in one session each. The next step is to adjust this course to the new SET progressively and monitor it.

7. Conclusions

This article presents the successful development of a new approach to assist teachers in choosing the best set of strategies, dynamics and activities for the practical sessions of architecture courses. The development of this approach incorporated several methods, such as a SWOT analysis to define the main strengths, weaknesses, opportunities and threats of 26 alternatives for learning specific work-related competences at university. The strengths of this new approach include its robust methods and the incorporation of the teacher’s team experience into its result.

The first application of this method successfully helped teachers in the case study to improve their practical sessions, with a new set of alternatives that has a sustainability index improved by 9%. Moreover, in the case study, this assessment proved that active alternatives should improve implementation-related issues in the teachers’ teaching materials and dedication outside class. Nevertheless, this approach has room for improvement with future steps such as: (a) implementing, monitoring and assessing the outcomes of the case study; (b) improving the approach considering other case studies and its automation.