Next Article in Journal
Does Reduction of Contribution Rate Affect the Sustainability of China’s Basic Endowment Insurance Fund?—Based on the Background of National Pooling and Collection Responsibility Transformation
Next Article in Special Issue
What Do Freshmen Know about Sustainability? Analysing the Skill Gap among University Business Administration Students
Previous Article in Journal
The Impact of Family Ownership on Quality and Disclosure of Internal Control in Pakistan
Previous Article in Special Issue
Students’ Perception about Sustainability in the Engineering School of Bilbao (University of the Basque Country): Insertion Level and Importance
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Collaborative Learning Activity to Analyze the Sustainability of an Innovation Using PESTLE

1
School of Informatics, Universitat Politècnica de Catalunya, 08034 Barcelona, Spain
2
Institute of Sciences of Education (ICE), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(16), 8756; https://doi.org/10.3390/su13168756
Submission received: 28 June 2021 / Revised: 29 July 2021 / Accepted: 31 July 2021 / Published: 5 August 2021
(This article belongs to the Collection Education for Sustainable Development in Higher Education)

Abstract

:
This paper presents a collaborative learning activity for courses in engineering degrees or master’s programs that cover the subject and skills of sustainability, social compromise, and ethics for engineering. The activity consists of performing a multidisciplinary analysis of a case of study, which presents a technology or innovation, to detect and debate possible issues and solutions. This learning activity is based on the PESTLE methodology that is used to analyze the political, economic, social, technological, legal, and environmental aspects of a technology or innovation. The PESTLE analysis is implemented using the jigsaw collaborative learning technique. This learning activity can be completed in two sessions of two hours. To validate its usefulness, the learning activity was applied for two semesters in a study on two courses: one in Informatics Engineering Degree and the other one in the Master Degree of Sustainability at the Universitat Politècnica de Catalunya, with very good results.

1. Introduction

The United Nations defined at the Millennium Summit in 2000 eight development goals to work towards sustainable development. These goals were named the Millennium Development Goals (MDG), an initiative to work towards sustainability in specific areas until 2015. In 2015, the United Nations extended the previous goals to seventeen in what was called the 2030 Agenda for Sustainable Development, and the seventeen goals were called Sustainable Development Goals (SDG), a blueprint to achieve a more sustainable world.
Nowadays, working towards sustainability is a priority. Before the SDG, sustainable development has been described in the literature in terms of three spheres, dimensions, domains, or pillars, i.e., the environment, the economy, and the society. Different authors proposed the three-pillar framework. Although there seems to be different sources [1], Barbier presented in 1987 a diagram with explicit economic, social, and biological spheres [2]. The three pillars have been expanded by some authors to include other pillars, such as institutions and governance.
Another important framework towards sustainability is The Triple Bottom Line (TBL), which makes corporations focus not only on the economic added value but also on the environmental and social balance. This framework expands the traditional reporting framework of the sustainable development definition to take into account social and environmental performances in addition to financial performances of companies [3,4].
The ESG (Environment, Social and Governance) paradigm proposes to use environmental, social, and governance factors to evaluate companies on how far advanced they are with sustainability. In recent years, stock market investors have become concerned regarding ESG issues of the companies involved, because ESG combines sustainable returns and risk reduction, with accountability towards the environment and society [5].
Sustainability science is an applied science defined by the practical problems it addresses—specifically, the problem of sustainable development [6]. The book written by De Vries provides an introduction to the patterns of past and present (un)sustainable development and to the emergence of the notion of sustainable development [7]. The work of Clark and Harley tried to synthesize the work done in different fields towards a more sustainable development. The authors also identified six capacities that have to be promoted to achieve a more sustainable world [8].
Technology has a clear influence on the way we live, our culture, the economy, the inner workings of society, and our relationship with the environment. In today’s world, it is very important to study the impact of new products, technologies, and services in society, the environment, and the economy.
In this context, more and more universities acknowledge the importance of introducing sustainability into their curriculum. Specifically, in engineering education, where students learn to develop new products and services that will have implications for the three pillars of sustainability, which may lead to ethical dilemmas. Therefore, engineers must be aware of the impact of the practice of engineering, and engineering education needs to provide the necessary knowledge and skills about sustainability and ethics.
“How to introduce sustainability in higher education?” has been the topic of many experiences and studies over the last 10–20 years. The main approaches to this problem are the following:
(a) To consider the issue of sustainability as a set of contents and competencies to be taught on a specific course.
(b) Sustainability as a set of competencies to be developed by the students in different courses not directly related to sustainability but that have points of contact with the issues of sustainability, adding specific modules or learning activities related to sustainability throughout the curriculum.
Both approaches are not exclusive and can and do coexist in some cases. For example, some authors have proposed the integration of sustainability in degrees, creating specialized courses applying computer science disciplines to sustainability (like machine learning for sustainability) to be integrated into the Tracks, Minors, and Majors of Informatics Sustainability [9].
The study presented in reference [10] showed that 55% of the works reviewed in the literature employed the first technique (adding modules in different subjects through the curriculum). This approach is used especially in universities with different engineering programs (Industrial, Telecommunications, Chemical, etc.) and is usually used in common engineering basic courses to provide an early perception and a generic idea of sustainable development. The other 45% use standalone courses about sustainability traditionally developed in specific programs. However, these courses are usually optional courses in the curriculum.
The classification method used in reference [10] was based on cases studies in the literature that reported experiences or contributions to the field, as well as to encourage future research. The findings included a description of the geographic distribution of the publications in sustainability in engineering education. Europe (UK, Denmark, Spain, Italy, The Netherlands, and Sweden, among others) with 49% and North America (USA) with 39% are the regions leading the publication of initiatives on sustainability in engineering education, followed by Australia with a 9% participation and Asia with 3%.
An interesting study that promotes the introduction of sustainability through the curriculum presented a plan for integrating sustainability education into the practices of nearly any college or university was conducted by Stewart [11]. The study emphasized the importance of introducing the concept of sustainability early in the program to remind students that their professional future work will have to consider sustainability and make them live in a campus that is committed to sustainability so they can live the experience. Greening campus operations are a critical piece of the campus sustainability movement. Stewart proposed the creation of a learning environment where all students receive an orientation to the campus culture of sustainability, an introduction to sustainability concepts as part of the first-year program degree, opportunities to explore sustainability issues through a graduation requirement, and holistic out-of-the-classroom learning experiences by living on and participating in a model sustainable community.
Another study that proposed to work the sustainability competence throughout the curriculum was presented in reference [12]. In this study the authors proposed the “Sustainability Competency Map”, a tool to define the learning outcomes in sustainability in engineering education degrees.
An initiative of standalone courses that presents 20 years of experience where sustainability and ethics are taught as an optional course in the Barcelona School of Informatics can be found in reference [13]. In the paper, the teaching strategies and experiences were presented and discussed.
Other related works presented workshops or courses to share resources and experiences related to sustainability. In reference [14], the authors proposed targeted training, like workshops, to create a shared vision of and materials on sustainability across disciplines. They promoted and shared educational materials about sustainable engineering among universities. This initiative took place in several universities in the USA.
Since sustainability is important, the study presented in reference [15] examined the impact of a procedure implemented at one Swedish university to promote the integration of the concept of sustainability into the courses. The idea is to give educators and researchers incentives to introduce sustainability into the curriculum. There was a study at the University of Gävle in Sweden where faculty members were asked to classify their courses and research funding applications regarding the contributions thereof to sustainable development. The results of the study indicated that this procedure can indeed stimulate faculty members to integrate sustainable development in their courses.
Other studies analyzed the degree in which sustainability is present in higher education. For example, studies performed on several degree programs in Spanish universities [16] showed that sustainability is present in most degrees but not uniformly so. The authors found out a great disparity among the courses that developed sustainability-related competencies. The competency that is most present is the “The application of ethical principles related to the values of sustainability in personal and professional behaviors”, while the “Sustainable use of resources and prevention of negative impacts on the natural and social environment” is the one that was less present. The results of this study also suggested that sustainability is not developed uniformly in the different universities.
In the work presented in reference [17], the authors analyzed the progress made in learning sustainability competencies by students of several bachelor education degrees. Specifically, the authors analyzed the student’s perceptions of their own learning related to sustainability. The results showed that only in one of the degrees did students perceive that their sustainability competence improved significantly after the courses.
In Engineering education, the scope of sustainability is predominantly about ecology (eco-design) and energy efficiency [18], and the teaching approaches are often limited to a single pillar rather than the integration of the three pillars (economic, environmental, and social) [19].
There is also a growing trend in the adoption of transdisciplinarity (Td) for sustainability, which connotes a research strategy that crosses many disciplinary boundaries to create a holistic approach [20,21,22].
Another aspect that is important to the study of sustainability is the study of ethics and its implementations in higher education. With the understanding of ethical foundations, students and future professionals could promote and achieve sustainable development. A comprehensive introduction to the ethics of sustainability was presented in reference [23].
Introducing sustainability in Engineering education is a challenge. However, since achieving more sustainable development is a goal proposed by the United Nations, it is also the study of how new technologies, products, and services can impact the economy, society, and the environment. We think it is important to provide students not only with theoretical concepts of sustainable development but also with methodologies and tools they can apply when analyzing the sustainability of a solution.
In order to provide some applicability and methods, this paper proposes to apply the PESTLE methodology as a collaborative learning activity in one standalone course dedicated to the study of sustainability and ethics in an Informatics Engineering degree and in a mandatory course of the sustainability master’s degree in Civil Engineering. The PESTLE methodology will provide a multidimensional perspective to gain insight into the three main pillars of sustainability when studying a new technology, product, business model, or service. The PESTLE analysis was applied with the jigsaw collaborative learning technique.
The rest of the paper is organized as follows: In Section 2 we present the context of the courses selected for the study. We also introduce a collaborative learning activity where the students do a PESTLE analysis of a given case study. The PESTLE analysis is a tool to analyze the impact of a given technology, product, business model, or service from different perspectives, which include the dimensions of sustainability and a social compromise. In Section 3, we present the results of the application of the method where the students have gone through the learning activity in different situations (presence, online, synchronous, and asynchronous) working in two study cases. We finish with a presentation of the discussion of the lessons learned and conclusions.

2. Materials and Methods

2.1. The Courses Used to Apply a Collaborative Learning Activity

The main purpose of this paper is to present a collaborative learning activity—which we will call PESTLE Collaborative Analysis—for courses in engineering degrees or master’s programs that cover the subject and skills of sustainability, social compromise, and ethics for engineering students. The activity consists of performing a multidisciplinary analysis of a case of study that presents a technology or innovation to detect and debate possible sustainability issues and solutions.
Our objective is to develop a didactic practice to teach the skills of sustainability and social responsibility to engineering students while they also learn a skill that can be applied in the workplace (the PESTLE Analysis). Our hypotheses are: (1) the PESTLE analysis of a study case can be an activity to help to learn sustainability and social responsibility skills, and (2) the activity must be a collaborative one. These two hypotheses guided the learning design of the PESTLE Collaborative Analysis as a learning activity. Our research question is: Is the PESTLE Collaborative Analysis a useful learning activity for engineering students to learn and practice the skills of sustainability and social responsibility?
This learning activity has been put into practice in two editions of two courses.
The first course is an optional one in the Informatics Engineering Degree in the School of Informatics of the Universitat Politècnica de Catalunya (UPC). The course, taught in the Catalan language, is usually taken in the last year of the degree. Its name is “Social and environmental aspects of information technologies” (ASMI from now on). ASMI has the following main topics: (1) Society and technological change, (2) Environmental Aspects of information technologies, (3) Social aspects of information technologies, (4) Ethics and professional responsibility, (5) Legal issues of information technologies, and (6) the history of computer science. The background of this course was described in the study published by the authors in reference [13]. The students in this course are usually in their 3rd and 4th years of their engineering degrees; at least 80% are male, since this has been the usual distribution of student genders at the Barcelona School of Informatics for the last two decades. The course is taught in the Catalan language, and most of the students are local.
The second course is a mandatory course of the master’s degree in Sustainability offered by the Sustainability Institute at UPC. The course is called “Fundamentals of business ethics and innovation” (FEEI from now on). FEEI has three main topics: (1) ethics applied in the field of engineering and legal frameworks for the development of professional activity in engineering, (2) corporate social responsibility and ethics in companies and organizations, and (3) innovation. The course is taught in the English language, and the students are local, national, and international; they are aged between 25 and 40 years old and split evenly among genders.
The collaborative learning activity is useful in ASMI to do some applied work in the five first topics. In FEEI, it is useful to work the innovation topic.
There was no control group, since this was the first time that we put into practice a PESTLE analysis; in previous years, it only was introduced as a theoretical resource, plus an example.

2.2. The PESTLE Analysis

This section is about PESTLE methodology: how it works and why it is useful for sustainability assessment purposes.
The PESTLE method is a framework used usually in strategic planning and marketing. Many companies use it to analyze the viability and impact in the company and the environment in which they plan to launch a new product, service, or innovation. PESTLE is an acronym that stands for Political, Economic, Social, Technologic, Legal and Environment. It introduces the need for a multidisciplinary team with a specific set of knowledge, backgrounds, and skills to cover each dimension of an analysis. We can find applications of this methodology in analyses and surveys related to sustainability in fields like spatial planning [24], water intake infrastructures [25], biofuel industry [26], energy port management [27], and the outboards and boats industry [28].
The PESTLE methodology arises from the criticisms of the SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis usually used in strategic business planning and universally taught in business courses. According to Hill and Westbrook [29], SWOT has proven to be ineffective as a means of analysis or as part of a corporate strategy review. For Hill and Westbrook, SWOT activity and its outputs do not constitute a valid analysis because of its lack of depth. Vorthman [30] argued that SWOT is useful to recognize the positive and negative internal and external factors of a system but that it is necessary to use PESTLE for any further analyses of these factors. Hence, a combined SWOT/PESTLE analysis is deeper and more detailed.
In Figure 1, we see how the PESTLE analysis plus the distinction of the aspects internal to the organization and its external environment allows for a multifaceted detection.
The matrix can hold all the relevant observations gathered during the multidisciplinary discussions and research in the PESTLE analysis. These observations can also benefit from prioritization, considering the magnitude, urgency, and possible impacts. Last but not least, this matrix of prioritized observations in PESTLE/SWOT can be used to redefine the proposition in an iteration or give recommendations for the implementation phase, which, in the domain of engineering, is of great importance, because it determines the requirements of the project.
It is important to note that the PESTLE analysis provides a way to include sustainability in strategic analyses, because it includes, by definition of the methodology, the main dimensions or pillars of sustainable development (environment, society, and economy).
Since a PESTLE can be used to derive recommendations for the requirements of a given project from a sustainability perspective, we find in the PESTLE analysis a way to introduce sustainability in the engineering process and in the education and training of engineers.

2.3. Design of the PESTLE Collaborative Analysis Learning Activity

When teaching environmental and social aspects of information technologies in the ASMI and FEEI courses, the authors combined traditional learning with active methodologies. Active methodologies refer to a range of more learner-centered instructional methodologies such as collaborative and project-based learning [31].
There are many studies that argue that active methodologies are especially effective in engineering education. For example, the work presented in reference [31] reviews several of the most commonly used active or inductive teaching methods, as well as the effectiveness of the methods. Broadly inductive methods are consistently found to be at least equal to, and, in general, more effective than, traditional deductive methods for achieving a broad range of learning outcomes. An example of a specific study of the gains of problem-based learning on undergraduate electrical engineering students can be found in reference [32]. In the same line, reference [33] proposed a wide variety of teaching techniques that have been repeatedly shown to be effective in the context of engineering education. An example of how teaching methodologies changed in introductory database courses can be found in reference [34]. Finally, in reference [35], the authors examined the extent to which undergraduate engineering courses taught using active and collaborative learning methods differed from traditional lectures. They have the ability to promote the development of students’ engineering design, problem-solving, communication, and group participation skills. The results indicated that active or collaborative methods produce both statistically significant and substantially greater gains in student learning than those associated with more traditional instructional methods.
In particular, in computer engineering education, there are examples of the use of active methodologies in different courses. Cooperative learning is a well-known active methodology. There are many different cooperative learning techniques that can be used to have a learner centered instruction [36].
Among the potential benefits of cooperative learning, we can mention that it has proven to improve student’s motivation and student’s academic results. In the studies, students’ perceptions of the quality of teaching and their academic results were significantly enhanced when compared with those students that were exposed to only one active methodology or none at all [37].
In particular, the authors selected a cooperative learning technique, a variant of the Jigsaw technique, to combine it with the PESTLE methodology. When we evaluate new technologies or innovation using PESTLE, several aspects (i.e., environmental, economic, or legal issues) have to be considered. The Jigsaw technique allows us to work in depth each particular aspect (i.e., environmental or social) in teams.
The Jigsaw technique has been applied in the learning activity by following four steps.
  • Step 1. Introduce the case and the methodology.
First, the professors present to the classroom a written document introducing a case of a new technology, business idea, or business model. The professors also present the PESTLE analysis with examples and explain how the learning activity is going to be developed, the group activities the students are going to perform, and the deliverables they have to present. This step usually takes 20 min.
  • Step 2. Work in groups of experts.
Among the students of the class, we will create 6 groups of students that will be known as “groups of experts” (see Figure 2). Each group will analyze the case from the point of view of a particular aspect of the PESTLE method. One group will analyze the case looking for political issues and implications, another group for economic ones, the third group social, and so on for the six aspects of the PESTLE acronym.
Each expert group has to conduct research, analyze, discuss, and write a consensual report. By participating in the activities of the expert group, the student gains the perspective of the given domain, becoming an “expert”.
  • Step 3. Multidisciplinary groups.
Then, in the third step, the students are reshuffled in new groups of students (see Figure 3). These new groups are created from the previous groups of experts, so the new groups at least include one expert member for each of the PESTLE letters. This means each group has one or more students roleplaying the expert in each dimension: one expert in political issues, one expert in economic issues, one expert in social issues, one expert in technological issues, one expert in legal issues, and finally, one expert in environmental issues.
So, in the previous step, the students do the research and work to be knowledgeable in a specific dimension of the case, and in the third step, they play the role of expert in their assigned PESTLE dimension.
This step of the activity is where the actual PESTLE analysis takes place. The members of each group discuss the case each one presenting the different issues they have encountered from their expertise background. Finally, each group write a final report with the PESTLE analysis of the case.
  • Step 4. Whole group discussion.
The last step consists of sharing the ideas with the class. Each group has a spokesman that sums up the assessment of his group. Each group does the same. At the end, we do a final discussion moderated by the professor.
As guidelines for the students on how to apply the PESTLE methodology, they are provided with a short question list for each dimension of the analysis. The list is presented in the table below (see Table 1):

2.4. The PESTLE Activity with and without Online Tools

The first application of the PESTLE activity was performed in early March 2019 in ASMI, just a couple of weeks before we entered a home confinement due to the COVID-19 pandemic. The grouping of the students was done in an organic way, first asking the students to move the tables of the classroom and form 6 groups. After a 20-min introduction of the activity and the case, the students would work for 45 min on the “Expert meetings” while the teacher answered questions moving from table to table. The students had smartphones and laptops to do web research when needed. Then, they are reshuffled and work for another 45 min. At the end, the whole class has a short debate sharing results. The final written report is submitted at the end of the day.
The FEEI group could not do the activity in the classroom due to the pandemic. We decided to turn the PESTLE activity into an asynchronous online activity on the university’s Moodle server. Using Moodle’s grouping utility, we created 6 groups of “experts” and sat up a forum activity on separate groups mode. The students used the forum to debate and prepare the report for a week period, which took approximately 2 h of homework. Then, new Moodle groups where created. The Moodle grouping utility is quite useful for the jigsaw technique, a new activity forum was created, and the students worked on it for another week. Finally, on a videoconference, the results were debated with the professor for an hour.
For the next edition of the courses, we decided—for reasons exposed in the conclusions—to repeat the format of meetings but using two sessions of two hours. The first session would consist of the introduction of the PESTLE analysis and the meeting of experts. The second session would be for the multidisciplinary meeting and the final debate of the whole class.
However, the 2021 ASMI and FEEI courses kept on being fully online. We decided to implement the activity using the grouping features of the videoconferencing platform and do synchronous meetings. The professor divided first the students into 6 separate videoconference rooms where the students could do the “expert” analysis for 45 min. The videoconferencing allowed the students to “call” the professor to solve doubts and clarify directions. The second grouping had to be set up by manually adding the appropriate students to the new meeting rooms.

3. Results

This activity was applied for two academic years in the previously mentioned courses; that is, in the ASMI engineering degree course and in the FEEI Masters course. In this section, we are going to explain the cases we worked with students during these two years, the main findings, the validation of the experience, and the proposed method to evaluate it.

3.1. First Edition: The Electric Scooter Renting Case

The first time we did this collaborative activity, we studied an emerging technology. The goal of the case was to do the multidimensional analysis using PESTLE and see how sustainable this technology was. This case is a new business model that wants to scatter electric scooters at various recharging points in a city and offer them for rent on demand through a mobile app. Some examples of the findings and impressions of the students are:
  • Political: there are different regulations and requirements such as the need of an insurance or registration to introduce this business. The regulations vary depending on the municipality.
  • Economic: The cost of the scooter is quite low the main reason for the success of the business. Nevertheless, costs derived from charging the batteries or moving the scooters to different locations have to be considered. Scooters decreases in short commutes the use of public transportation.
  • Social: This mean of transportation is used mainly for short commutes. It is highly sensitive on current fashion. It creates inconveniences to pedestrians. There is approximately one accident a day.
  • Technological: a GPS, a smartphone, and connection to the internet is required. The batteries life cycle is short (2 years).
  • Legal: there are not specific regulation for the business. There are penalties applied in certain cases such as people driving without a helmet o driving without lights at night.
  • Environmental: The total load of a scooter consumes the same as a dryer for 5 min. The main drawback is that recycling process that is expensive and usually not considered. The maintenance and repairs are also issues. Many times, it is cheaper to buy new scooters rather than repair them.
During the task, students defined important questions that needed to be answered. For example: what is the average lifetime of a rental scooter? What is the carbon footprint of the production, distribution, and disposal of a scooter? What are the relevant regulations in our city regarding the case? In other cities? Have there been changes during the last years?

3.2. Second Edition: The Trunk Sharing Case

The case introduces a fictitious e-commerce company, Niagara.com, whose business was affected by the COVID-19 pandemic. After the COVID-19 pandemic, e-commerce disrupted heavily the retail sector. More than one-third of the B2C (Business to Customer) purchases were done online. Niagara.com was experiencing a huge, unprecedented increment in sales, courier’s activity, and deliveries of packaged goods from Niagara.com’s huge, automated warehouses to customers’ homes and workplaces.
The increase of activity showed that the system was far from perfect. The couriers often did not find the customer at home, and deliveries had to be rescheduled. Every day, lots of miles were driven in vans or cars just to deliver packages, with the economic costs for Niagara.com plus carbon emissions to the environment.
There was an opportunity because of data gathered from the Niagara.com app installed in the customer’s devices. The company knew that a considerable number of customers had their workplace in a relatively short distance from Niagara.com’s warehouses. Some of these customers drove regularly from their homes, often tens of miles away, with the trunks of their cars unused.
The proposed solution was to create a program of trunk-sharing with the customers of Niagara.com. Customers could earn money or discount vouchers for their purchases in Niagara.com online store by sharing their car’s trunk for packages transportation to their neighborhood.
Niagara’s couriers could place packages in the customer’s car trunk (or “boot” in the UK) while he or she was at the workplace close to a Niagara.com distribution warehouse. He would drive home carrying the package(s). There, he (or Niagara.com’s system via the App) could set up a schedule for their neighbors to pick up their packages. All this activity would be coordinated via Niagara.com’s mobile app in the customer’s phones, using GPS, Bluetooth, and NFC to interact with the cars; QR Codes; and using data science and machine learning algorithms in Niagara.com’s cloud to coordinate everything.
The goal of the activity was the same as before: see how viable and sustainable this business idea, technology, or innovation was by performing a PESTLE analysis. The students performed interesting analysis of the trunk-sharing cases. Some examples of the findings and impressions of the students are:
-
Concerns about the legality of the trunk-sharing model, since it is not clear what is the status of the customer who shares his trunk (the trunk-sharer). Is she working when returning home? Does she need to be registered as an independent worker? If she has a car accident when she is carrying a package, is Niagara.com liable? Some students pointed that there an opportunity for an insurance business, since insurance depends on statistical knowledge and Niagara.com has plenty.
-
The questionable social value of the project, since it allows high income citizens—those who have a job and use a car instead of public transportation to commute—to earn more money or benefits, instead of offering opportunities to unemployed or less well-off citizens. Which increases the inequality of the economic system. The students are very aware of the problem of inequality.
-
On the ecological front the students had mixed views; for one part, they liked the idea of less delivery vans going around. They were not concerned for the courier’s jobs, since the trunk-sharing system was aimed to cover for part of the increase in deliveries. However, they feared that the project would incentivize commuting by car instead of public transportation, walking, bicycles, or working from home. The students proposed running a pilot experience to measure the behavior changes of the trunk-sharers and then perform a utilitarian calculation. They learned about utilitarianism as contents of the course.
-
Interestingly, the students where very concerned about the privacy of Niagara.com’s customers with respect to their neighbors and the introduction of forced social interactions to deliver packages with neighbors who dislike each other or have some enmity, especially if previous romantic partners (ex). The more technologically oriented proposed features of blacklisting and blocking of other users. The professors here were the ones who had to point out the issues of privacy with respect to the data gathered by Niagara.com.
-
On the technological front, the students only were puzzled by the matter of the courier opening the cars trunks to place packages there, since the case did not mention a specific technology to solve the problem. Several solutions were proposed, including the business opportunity of creating a fleet of electric cars prepared for trunk-sharing on the “frunk” (trunk in the front of electric cars in the space freed by the absence of the internal combustion engine), rent these cars to the customers, and allow the customers to earn discounts in the renting by doing the trunk-sharing thing.

3.3. Validation

At the end of the activity we did in the second year, we tried to determine how students perceived the case study in terms of the perceived values. So, they were asked to fill in a questionnaire. It was a five-question questionnaire and nonmandatory. The questions consisted of a sentence following a 5-point liker scale: strongly disagree (1), disagree (2), indifferent (3), agree (4), and strongly agree (5). The questions presented to students (see Table 2) are the following:
The design of the questionnaire follows the following criteria. The first question tries to determine if the students have understood the method to do a sustainability analysis of an innovation or proposed technology. The second question determines if students understand that there is often an interrelation between the environment, the socio-economic organization, and the technology used in a society. If this interrelation is not taken into account, many times, the society falls apart [38]. This is one of the first course topics we cover.
Questions three and four are designed to determine whether students can find moral dilemmas from the case analysis and, if so, if they can do some ethical argumentation in a role play proposed by teachers to try to solve these dilemmas. These questions are oriented to link the PESTLE analysis with the ethic topic that we also cover in the course.
Finally, question five tries to analyze if students would be able to find alternatives solutions to the problems encountered in the case study in order to find viability to the project.
The answers provided by students of both courses are similar. In ASMI, students’ answers were above 4 in all questions, being the highest scores the answers to questions 1 and 3. Twenty-two students out of forty-nine answered the survey (see Figure 4).
In FEEI, we received answers from nine students out of twenty. In this course, the students’ answers were above 4 for questions 1,2, 4, and 5. However, question number 3 had an average below four (see Figure 4).
The ASMI students averaged 22–23 years old, all from Catalonia or, at least, were Catalan speakers. FEEI students had a wider range of ages, from 25 to 40. The FEEI course was taught in English, and half of the FEEI students where actually in their origin countries (USA, Italy, Peru, Bolivia, Argentina, and Mexico). The results from the age and location homogeneous group (ASMI) were quite similar in most of the questions about the results of the group, with more diversity in ages and locations (FEEI).
In addition to the survey, at the end of the semester when asked for the overall feedback about the course, some students provided some feedback about the PESTLE activity. Some examples are the following quotes:
“Regarding the PESTLE activities, I think it was an interesting tool to get us to discuss and share opinions. This activity is more relevant when it is developed in home confinement. I deduce that these activities make more sense when you meet face to face with the class (extrapolated to the rest of the semester), however, the tool fulfilled its purpose to reason, discuss, reach consensus and shape. So, I think it is a valid tool to reinforce the topics discussed in class”.
“The first would be the PESTLE assignment; gaining that perspective of how to analyze a project from different viewpoints I found to be a very valuable skill that I will now take with me to use for future assignments and real-life projects”.
“The PESTLE exercise was very good for small groups to discuss impacts and reflect on future scenarios. Focusing on a single aspect of the project allowed us to go deeper and draw more results and conclusions than if we had wanted to reflect on all the aspects at once that influenced the project. The subsequent debriefing with a participant from each group allowed us to round out the ideas that we had reflected on with the initial group and to form a much more robust thinking”.

3.4. Assesment

To perform the assessment of the work of the students, we developed a rubric, shown in Table 3, where different aspects of the student’s reports were evaluated. The rubric was applied to both deliveries of the activity: the experts report and the integrated multidisciplinary report. The second part had an additional row in the rubric.

4. Discussion

The first time we conducted the activity in the classroom in a 2-h session proved to be a successful experience. However, the students observed that, with more time for each meeting, they could do a deeper and better analysis. Hence, we decided to use two sessions of two hours for the next edition.
The results of the experience with the Moodle forums were fine, especially because of the grading features of the Moodle forum, which allowed for an easy assessment of the participation of every student. If the aim of the activity was to develop discursive writing and debate, it would be our choice for future editions of the course. However, we wanted the students to concentrate on the analysis and not so much on impressing the professor with their rhetoric written skills. So, for the 2021 edition, we decided to discontinue the asynchronous mode.
Our impression is that the synchronous meetings in the classroom and the videoconferencing rooms were more dynamical and more to the point during the debates on the forum, and we intend to keep doing the sessions live either in the classroom on the campus or in the virtual classroom. The new duration of two sessions of two hours proved to improve the experience, because the quality of the analyses and reports improved.
The experience with the PESTLE learning activity suggests that the students already have a concern for the problem of sustainability and have a strong set of values of social compromise. As Shepard suggested, one key element of education for sustainability is the quest for affective learning outcomes of values, attitudes, and behaviors [39]. To give students lectures about the topic would be like, as Nicholas Taleb said [40], to teach birds how to fly. However, the students benefitted from a given time, space, and problem to analyze and debated their points of view with their peers and the professor.

5. Conclusions

The authors presented the PESTLE collaborative activity as a learning tool to work on the skills of sustainability, social compromise, and ethics for engineering students. The activity can be applied in presence, hybrid, and online settings. This collaborative learning activity was used in two courses at the Universitat Politècnica de Catalunya.
In ASMI, this new activity is useful to do some applied work about the environmental, social, and economic implications of information technologies. The students who choose the ASMI course are already aware of the importance of sustainable development and the problems that the materialistic progress have caused to the environment. We work the basics of sustainable development into the theory lectures. However, students lack some methodologies to determine if a new technology that may be developed in the very near future is going to be sustainable or not. So, the aim of this activity is to provide this method using case studies. During the case, it is usual that students find ethical dilemmas that we cover afterwards when we study the ethics topic.
In the FEEI course, students have a solid awareness about sustainable development, but they lack tools for the assessment and analysis to improve the sustainability of a given case. Hence, the PESTLE activity is really useful for them.
This activity can also be applied in other courses different from the ones described in this paper, and we hope the readers can do so.
The authors consider that the presented collaborative learning activity has the following strong points that make it a very useful learning tool:
  • The students learn by doing. The students learn the relevant concepts and theory of the PESTLE analysis—which is an actual useful skill for engineers to evaluate the sustainability of their work—by going through a complete case.
  • The students learn together and share their results with each other. Social constructionism tells us that we acquire new knowledge when we create artifacts in group (like a PESTLE analysis) to be used by others. So, the activity design is consistent with at least one major learning theory and pedagogical framework [41].
  • The students have to roleplay and take a holistic approach. Without being actual experts in the PESTLE specialties, the students have to first analyze the case from a given specialist point of view, which gives them parameters for their research and points they can raise. On the second iteration, the students have to roleplay the expert and then empathize with the approaches developed by the other students, considering other points of view.
  • The activity takes a multidisciplinary and holistic approach to the learning of sustainability.
The activity requires the professors to know the study case well and to have done a preliminary analysis on their own to better guide the students, answer questions, and still be surprised by the student’s questions and issues raised. The activity success depends heavily in the ability of the students to play the expert “role”, so the guidance of the teachers is paramount.
We consider that we achieved the objective to develop a didactic practice to teach the skills of sustainability and social responsibility to engineering students while they also learn a skill that can be applied in the workplace (the PESTLE analysis). Our impression as teachers, the data in the survey, and the qualitative answers provided by students suggest the confirmation of the hypothesis, and the answer to the research question “Is the PESTLE Collaborative Analysis a useful learning activity for engineering students to learn and practice the skills of sustainability and social responsibility?” is affirmative. Due to that, we plan to continue to work in developing new study cases for our students.
In further works, we plan to develop quality study cases and open source them so other professors can apply them in their teaching and improve on the method. We also plan to extend the activity to include the study of the ethical and deontological aspects. We want the students to identify the ethical dilemmas that the case could lead to and then set up different groups to work on such dilemmas from the point of view of different ethical theories, sets of values, and deontological codes. Afterwards, we can compare the conclusions of every group and debate about it.

Author Contributions

Conceptualization, M.J.C. and M.A.; methodology M.J.C. and M.A.; validation, M.J.C., A.L. and M.A.; writing—original draft preparation, M.J.C. and M.A.; writing—review and editing, M.J.C., A.L. and M.A.; project administration, A.L.; and funding acquisition, A.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by EU within the framework of the Erasmus+ Programme, Project Tech4Edu4, Grant Agreement no. 2020-1-HR01-KA203-07777.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institut de Ciències de l’Educació (UPC, February 2021).

Informed Consent Statement

All students participating in the survey where informed and participation was optional.

Data Availability Statement

Not applicable.

Acknowledgments

We acknowledge the support of the Sciences Institute of Education of UPC https://www.ice.upc.edu (accessed on 4 August 2021).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of the data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Barbier, E.B. The Concept of Sustainable Economic Development; Routledge: London, UK, 2017. [Google Scholar]
  2. Purvis, B.; Mao, Y.; Robinson, D. Three pillars of sustainability: In search of conceptual origins. Sustain. Sci. 2019, 14, 681–695. [Google Scholar] [CrossRef] [Green Version]
  3. Elkington, J. Cannibals with Forks. The Triple Bottom Line of 21st Century; New Society Publishers: Gabriola Island, BC, Canada, 1998. [Google Scholar]
  4. Elkington, J. Enter the triple bottom line. In The Triple Bottom Line; Routledge: London, UK, 2013; pp. 23–38. [Google Scholar]
  5. Przychodzen, J.; Gómez-Bezares, F.; Przychodzen, W.; Larreina, M. ESG Issues among fund managers—Factors and motives. Sustainability 2016, 8, 1078. [Google Scholar] [CrossRef] [Green Version]
  6. Kates, R.W.; Clark, W.C.; Corell, R.; Hall, J.M.; Jaeger, C.C.; Lowe, I.; McCarthy, J.J.; Schellnhuber, H.J.; Bolin, B.; Dickson, N.M. Sustainability science. Science 2001, 292, 641–642. [Google Scholar] [CrossRef]
  7. De Vries, B.J.M. Sustainability Science; Cambridge University Press: Cambridge, UK, 2012. [Google Scholar]
  8. Clark, W.C.; Harley, A.G. Sustainability science: Toward a synthesis. Annu. Rev. Environ. Resour. 2020, 45, 331–386. [Google Scholar] [CrossRef]
  9. Fisher, D.H.; Bian, Z.; Chen, S. Incorporating sustainability into computing education. IEEE Intelligent Systems 2016, 31, 93–96. [Google Scholar] [CrossRef]
  10. Mesa, J.A.; Esparragoza, I.E.; Maury, H.E. Sustainability in engineering education: A literature review of case studies and projects. In Proceedings of the 15th LACCEI International Multi-Conference for Engineering, Education Caribbean Conference for Engineering and Technology, Boca Raton, FL, USA, 19–21 July 2017; Latin American and Caribbean Consortium of Engineering Institutions: Boca Raton, FL, USA, 2017. [Google Scholar]
  11. Stewart, M. Transforming higher education: A practical plan for integrating sustainability education into the student experience. J. Sustain. Educ. 2010, 1, 195–203. [Google Scholar]
  12. Segalaàs, J.; Saánchez-Carracedo, F.; Hernaández, A.; Busquets, P.; Tejedor, G.; Horta, R. The EDINSOST Project Training Sustainability Change Agents in Spanish and Catalan Engineering Education. In Proceedings of the 9th International Conference on Engineering Education for Sustainable Development, Glassboror, NJ, USA, 3–6 June 2018; pp. 109–116. [Google Scholar]
  13. Casañ, M.J.; Alier, M.; Llorens, A. Teaching Ethics and Sustainability to Informatics Engineering Students, An Almost 30 Years’ Experience. Sustainability 2020, 12, 5499. [Google Scholar] [CrossRef]
  14. Davidson, C.I.; Heller, M. Introducing sustainability into the engineering curriculum. In Proceedings of the International Conference on Sustainable Infraestructure: Creating Infrastructure for a Sustainable World, Long Beach, CA, USA, 6–8 November 2014; pp. 1029–1038. [Google Scholar]
  15. Sammalisto, K.; Lindhqvist, T. Integration of Sustainability in Higher Education: A Study with International Perspectives. Innov. High. Educ. 2008, 32, 221–233. [Google Scholar] [CrossRef] [Green Version]
  16. Sánchez-Carracedo, F.; Ruiz-Morales, J.; Valderrama-Hernández, R.; Muñoz-Rodríguez, J.M.; Gomera, A. Analysis of the presence of sustainability in Higher Education Degrees of the Spanish university system. Stud. High. Educ. 2021, 46, 300–317. [Google Scholar] [CrossRef]
  17. Muñoz-Rodríguez, J.M.; Sánchez-Carracedo, F.; Barrón-Ruiz, Á.; Serrate-González, S. Are We Training in Sustainability in Higher Education? Case Study: Education Degrees at the University of Salamanca. Sustainability 2020, 12, 4421. [Google Scholar] [CrossRef]
  18. Allenby, B. The Theory and Practice of Sustainable Engineering, 1st ed.; Pearson: London, UK, 2011. [Google Scholar]
  19. Arsat, M.; Holgaard, J.; de Graaff, E. Three dimensions of characterizing courses for sustainability in engineering education: Models, approaches and orientations. In Proceedings of the 3rd International Congress on Engineering Education (ICEED), Kuala Lumpur, Malaysia, 7–8 December 2011. [Google Scholar]
  20. Tejedor, G.; Segalas, J. Action research workshop for transdisciplinary sustainability science. Sustain. Sci. 2018, 13, 493–502. [Google Scholar] [CrossRef]
  21. Takeuchi, M.A.; Sengupta, P.; Shanahan, M.C.; Adams, J.D.; Hachem, M. Transdisciplinarity in STEM education: A critical review. Stud. Sci. Educ. 2020, 56, 213–253. [Google Scholar] [CrossRef]
  22. Scholz, R.W. The normative dimension in transdisciplinarity, transition management, and transformation sciences: New roles of science and universities in sustainable transitioning. Sustainability 2017, 9, 991. [Google Scholar] [CrossRef] [Green Version]
  23. Kibert, C.J.; Monroe, M.C.; Peterson, A.L.; Plate, R.R.; Thiele, L.P. Working toward Sustainability: Ethical Decision-Making in a Technological World; John Wiley & Sons: Hoboken, NJ, USA, 2011; Volume 35. [Google Scholar]
  24. Mauree, P.P.; Geneletti, D. Assessing barriers to effective spatial planning in Mauritius. A combination of SWOT and gap surveys. J. Environ. Plan. Manag. 2017, 60, 1324–1346. [Google Scholar] [CrossRef]
  25. Srdjevic, Z.; Bajcetic, R.; Srdjevic, B. Identifying the criteria set for multicriteria decision making based on SWOT/PESTLE analysis: A case study of reconstructing a water intake structure. Water Resour. Manag. 2012, 26, 3379–3393. [Google Scholar] [CrossRef]
  26. Achinas, S.; Horjus, J.; Achinas, V.; Euverink, G.J.W. A PESTLE Analysis of Biofuels Energy Industry in Europe. Sustainability 2019, 11, 5981. [Google Scholar] [CrossRef] [Green Version]
  27. Christodoulou, A.; Cullinane, K. Identifying the Main Opportunities and Challenges from the Implementation of a Port Energy Management System: A SWOT/PESTLE Analysis. Sustainability 2019, 11, 6046. [Google Scholar] [CrossRef] [Green Version]
  28. Patuk, I.; Borowski, P.F. Business plan of the company of repair and maintenance outboards and boats—“Technoservice”. World Sci. News 2017, 86, 193–204. [Google Scholar]
  29. Hill, T.; Westbrook, R. SWOT analysis: It’s time for a product recall. Long Range Plan. 1997, 30, 46–52. [Google Scholar] [CrossRef]
  30. Vorthman, R.G.J. Toward Best-Practices Program Management. Sigma 8; Noblis: Falls Church, VA, USA, 2008. [Google Scholar]
  31. Prince, M.J.; Felder, R.M. Inductive teaching and learning methods: Definitions, comparisons and research bases. J. Eng. Educ. 2006, 95, 123–138. [Google Scholar] [CrossRef]
  32. Yadav, A.; Subedi, D.; Lundeberg, M.A.; Bunting, C.F. Problem-based learning: Influence on students’ learning in an electrical engineering course. J. Eng. Educ. 2011, 100, 253–280. [Google Scholar] [CrossRef]
  33. Felder, R.M.; Woods, D.R.; Stice, J.E.; Rugarcia, A. The future of engineering education II. Teaching methods that work. Chem. Eng. Educ. 2000, 34, 26–39. [Google Scholar]
  34. Robbert, M.A.; Wang, M.; Ricardo, C.M. Trends in the evolution of the database curriculum. ACM SIGCSE Bull. 2003, 35, 139–143. [Google Scholar] [CrossRef]
  35. Terenzini, P.T.; Cabrera, A.F.; Colbeck, C.L.; Parente, J.M.; Bjorklund, S.A. Collaborative learning vs. lecture/discussion: Students’ reported learning gains. J. Eng. Educ. 2001, 90, 123–130. [Google Scholar] [CrossRef]
  36. Barkley, E.F.; Cross, K.P.; Major, C.H. Collaborative Learning Techniques; Jossey-Bass, Wiley Print: San Francisco, CA, USA, 2005. [Google Scholar]
  37. López, D.; Sánchez, F.; Garcia, J.; Alier, M.; Piguillem, J.; Velasco, M. Introducing “Sustainability and Social Commitment” skills in an engineering degree. In Proceedings of the Frontiers in Education Conference (FIE), Rapid City, SD, USA, 12–15 October 2011. [Google Scholar]
  38. Ponting, C. A Green History of the World; Sinclair-Stevenson: London, UK, 1991. [Google Scholar]
  39. Shephard, K. Higher education for sustainability: Seeking affective learning outcomes. Int. J. Sustain. High. Educ. 2008, 9, 87–98. [Google Scholar] [CrossRef] [Green Version]
  40. Taleb, N.N. Antifragile: Things that Gain from Disorder, 1st ed.; Penguin: City of Westminster, UK, 2013. [Google Scholar]
  41. Alier, A. A social constructionist approach to learning communities: Moodle. In Open Source for Knowledge and Learning Management: Strategies beyond Tools; IGI Global: Hershey, PA, USA, 2007; pp. 369–381. [Google Scholar]
Figure 1. PESTLE analysis schema plus the internal and external aspects of the organization, which connects the PESTLE analysis with the SWOT analysis.
Figure 1. PESTLE analysis schema plus the internal and external aspects of the organization, which connects the PESTLE analysis with the SWOT analysis.
Sustainability 13 08756 g001
Figure 2. Second part of the method: groups of experts for each letter of the PESTLE method.
Figure 2. Second part of the method: groups of experts for each letter of the PESTLE method.
Sustainability 13 08756 g002
Figure 3. Third step of the method students reshuffled to create mixed groups.
Figure 3. Third step of the method students reshuffled to create mixed groups.
Sustainability 13 08756 g003
Figure 4. Summary of the survey results.
Figure 4. Summary of the survey results.
Sustainability 13 08756 g004
Table 1. Questions to apply the PESTLE method.
Table 1. Questions to apply the PESTLE method.
PESTLE DimensionQuestions
Political1. What interactions may occur between the development and deployment of the new technology and the (a) Geopolitics: the relations of power between different states and international treaties that may affect or are affected by the new technology. (b) Internal Politics: how the technology can affect or be affected by internal political struggles, pressure groups, and lobbies.
2. Is there a political opposition to the new technology and its possible impact?
EconomicInvestment
1. Does the new technology need large investments?
2. What are the estimated returns?
3. What is the amortization timeframe?
4. Related to the estimated exploitation timespan is there any legal protection (patents, licenses, the time before new tech may be rendered obsolete)?
5. Will the new technology disrupt our business model? (see Innovators dilemma)
Impact on the economy
6. Is there any disruption in the economy of existing business models and services (Schumpeter’s Creative Destruction)?
7. Are there new efficiencies and opportunities created by technology?
Social1. What are the socio-cultural factors that may affect the spreading or rejections of the new technology?
2. How will the new technology affect the current social and cultural structure (i.e., demography, cultural trends, emotional needs, economic background, social organization, methods of communication)?
3. Is the society ready/willing to learn, adopt, and integrate with the new technology?
Technological1. What technologies does the new technology replace? Is there a multiplicative factor on cost, efficiency, ease of use, outcomes, etc.? Who owns and benefits from them?
2. What possible technologies can we foresee that could replace or make obsolete this new technology? When are these technologies due?
3. Are there other new technologies that may converge this new technology to allow for yet unforeseen new opportunities or threats?
4. Are there technological barriers for the deployment of the New Technology in the considered country? Availability and quality of electricity, water, communications, and other services?
Legal1. Are there any current legislations that regulate the sector of the technology/invention? Or can there be a change in current legislation?
2. How these legislations affect the deployment of the New Technology.
3. Is the foreseeable impact of the New Technology considered by existing regulations?
4. If not, are there relevant legislations that have been recently developed in other countries.
Environmental1. What environmental impact may have the new technology? (positive or negative, concerning usage of natural resources, energy, possible externalities, waste, etc.)
Table 2. Questions from the survey that was passed to the students.
Table 2. Questions from the survey that was passed to the students.
Question
1. I can analyze the possible social and environmental impact of a technological and/or business innovation project.
2. I understand the interrelationships between the economic, social, and environmental dimensions of sustainability.
3. I can identify possible moral and ethical dilemmas that may arise from technological or business innovation.
4. I am able to make arguments for and against different positions in the face of these dilemmas.
5. I am able to propose possible modifications to a technological and/or business innovation project based on the complexities identified in the PESTLE analysis.
Table 3. Assessment rubric for the PESTLE activity.
Table 3. Assessment rubric for the PESTLE activity.
Aspect to ConsiderUnsatisfactorySatisfactoryExceeds Expectations
Pros and consNot enough points for or against are detected.Points for and against that affect the case are detected.An extensive and detailed detection of aspects that positively or negatively affect the case is made.
Internal and external aspectsInternal or external aspects are ignored.An analysis of the internal and external aspects is made.An extensive and detailed analysis is made of the aspects that affect the case internally or externally.
Questions about specific aspects of the domainIt does not give a satisfactory answer to the initial questions provided in the guide.The questions in the guide are answered.Additional questions are asked that lead to specific search for information.
Analysis and evaluation criteriaNo clear analysis and evaluation criteria are proposed.Some criteria for analysis and evaluation of the different aspects of the case are provided.Criteria for analysis, evaluation models and calculation of the different aspects that affect the case are defined.
Interactionsbetween dimensionsNo interactions between factors of different dimensions are identified.Interactions between factors of different dimensions are detected.Interactions between factors of different dimensions are detected and creative ideas are provided for possible solutions and opportunities.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Casañ, M.J.; Alier, M.; Llorens, A. A Collaborative Learning Activity to Analyze the Sustainability of an Innovation Using PESTLE. Sustainability 2021, 13, 8756. https://doi.org/10.3390/su13168756

AMA Style

Casañ MJ, Alier M, Llorens A. A Collaborative Learning Activity to Analyze the Sustainability of an Innovation Using PESTLE. Sustainability. 2021; 13(16):8756. https://doi.org/10.3390/su13168756

Chicago/Turabian Style

Casañ, Maria José, Marc Alier, and Ariadna Llorens. 2021. "A Collaborative Learning Activity to Analyze the Sustainability of an Innovation Using PESTLE" Sustainability 13, no. 16: 8756. https://doi.org/10.3390/su13168756

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop