Next Article in Journal
Secondary School Students and Internet Forums—A Survey of Student Views Contrasted with an Analysis of Internet Forum Posts
Next Article in Special Issue
Eco-design and Eco-efficiency Competencies Development in Engineering and Design Students
Previous Article in Journal
How Do Physics Teacher Candidates Substantiate Their Knowledge? An Analytical Framework for Examining the Epistemic Dimensions of Content Knowledge in Higher Education
Previous Article in Special Issue
Lessons Learned from the Development of Open Educational Resources at Post-Secondary Level in the Field of Environmental Modelling: An Exploratory Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Fostering Professional Competencies in Engineering Undergraduates with EPS@ISEP

1
Department of Electrical Engineering, Polytechnic Institute of Porto, 4249-015 Porto, Portugal
2
Mathematics Department, Polytechnic Institute of Porto, 4249-015 Porto, Portugal
3
Department of Informatics, Polytechnic Institute of Porto, 4249-015 Porto, Portugalll
*
Authors to whom correspondence should be addressed.
Educ. Sci. 2019, 9(2), 119; https://doi.org/10.3390/educsci9020119
Submission received: 30 April 2019 / Revised: 17 May 2019 / Accepted: 21 May 2019 / Published: 29 May 2019
(This article belongs to the Special Issue Engineering Education and Technological / Professional Learning)

Abstract

:
Engineering education addresses the development of professional competencies in undergraduates. In this context, the core set of professional competencies includes critical thinking and problem solving, effective communication, collaboration and team building, and creativity and innovation—also known as the four Cs—as well as socio-professional ethics and sustainable development—referred in this paper as the two Ss. While the four Cs were identified by the associates of the American Management Association based on the needs of the society, professional associations, and businesses; this paper proposes the two S extension to ensure that future engineers contribute to the well-being of individuals and the preservation of life on Earth. It proposes a tangible framework—the 4C2S—and an application method to analyse the contributions made by engineering capstone programmes to the development of these core competencies in future engineering professionals. The method is applied to an engineering capstone programme—the European Project Semester (EPS) offered by the Instituto Superior de Engenharia do Porto (ISEP)—and a specific project case—EPS@ISEP Pet Tracker project developed in 2013, constituting, in addition, a road map for the application of the 4C2S framework to engineering capstone programmes. The results show that EPS@ISEP complies with the 4C2S framework.

1. Introduction

Engineering education aims to prepare professionals to address the challenges of the future. This is a highly demanding goal since it not only implies a forecast of future needs but also anticipates scientific and technological advancement trends. Society (while beneficiaries), academia (while educators), and businesses (while employers) must work together to define the set of core competencies of future engineers.
The American Management Association (AMA) conducted in 2012 a critical skills survey among its corporate associates regarding the core professional competencies of the 21st century workforce. From the entrepreneurs’ perspective, the 21st century business requires, beyond the basics of reading, writing, and arithmetics (the three R), skills such as critical thinking and problem solving, effective communication, collaboration and team building, and creativity and innovation (the four Cs) [1]. Specifically, the AMA defines critical thinking and problem solving (CTPS) as the ability to make decisions, to solve problems, and to take action as appropriate; effective communication (EC) as the ability to synthesise and transmit ideas both in written and oral formats; collaboration and team building (CTB) as the ability to work effectively with others, including those from diverse groups and with opposing points of view; and creativity and innovation (CI) as the ability to see what is not there and to make something happen.
However, according to Cohen and Grace [2], social responsibility should be seen as integral to the performance of engineers as individuals and of engineering as a profession. It involves thinking positively of social responsibility, avoiding harm, and consciously choosing to do good [2]. Since the engineering practice deals with the environment, professional ethics and behaviours, matters of health and safety, and discipline [3], additionally, this paper advocates the need to foster sustainable development (SD) and socio-professional ethics (SPE) (the two Ss) in the engineering practice. In this context, sustainable development corresponds to “development that meets the needs of the present without compromising the ability of future generations to meet their own needs” [4] and socio-professional ethics translate to the set of values governing the conduct of engineers in their role as professionals as well as individuals.
In the Bruntland report, the definition of sustainable development encompassed two key concepts: (i) the concept of needs, in particular the essential needs of the world’s poor to which an overriding priority should be given, and (ii) the idea of limitations imposed by the state of technology and social organisation on the environment’s ability to meet present and future needs [4]. Later, in 2004, the Barcelona Declaration reinforced the need to educate engineers for sustainable development. Specifically, it called for the multidisciplinary, systems-oriented, critical thinking, participative and holistic education of engineers [5]. More recently, in 2016, the United Nations 2030 Agenda for Sustainable Development, which defines 17 Sustainable Development Goals (SDG), officially came into force. While the SDG are not legally binding, governments are expected to take ownership and to establish a national framework for their achievement [6].
Ethics, as a whole, deals with the moral choices that are made by each person in his or her relationship with others, including those made while practising engineering. Engineering ethics encompasses the more general definition of ethics but applies it more specifically to situations involving engineers in their professional lives. Thus, engineering ethics is a body of philosophy indicating the ways that engineers should conduct themselves in their professional capacity [7]. Whether working on multinational project teams, navigating geographically dispersed supply chains, or engaging customers and clients abroad, engineering graduates encounter worlds of professional practice that are increasingly global in character [8]. In this context, the global engineer needs to be aware of effective ways to navigate these cultural differences, which is crucial for achieving their common goals [9]. This makes a strong case in favour of using multinational teamwork as a setup to learn socio-professional ethics. When students communicate, discuss, or organise their project work, they learn to respect the values of others and to develop global engineering competency as defined by Jesiek et al. [8].
The 4C2S framework proposes an enlarged set of competencies to analyse engineering education programmes. The framework allows the adoption of an evidence-based approach to identify how a programme contributes to the development of these competencies in future engineering professionals. The application of the approach is illustrated using an engineering capstone programme—the European Project Semester (EPS) offered by Instituto Superior de Engenharia do Porto (ISEP)—and a specific project case—the EPS@ISEP Pet Tracker project. The adopted method analyses the context and the team’s learning journey using the framework, searching for evidences of the development of professional competencies within the process and timeline of the pet tracker project (activities, milestones, and deliverables).
The main contributions of this work are the proposed framework and method, which together attempt to identify and quantify evidences of the development of professional competencies within an engineering capstone programme.
In terms of organisation, this paper includes the following sections: Section 1 introduces the problem of the development of engineering professional competencies in undergraduates, the 4C2S analysis framework, and the structure of this document; Section 2 provides the perspectives of professional engineers associations from Europe, the United Sates of America (USA), and Australia regarding the most sought professional competences of future engineers; Section 3 presents similar capstone programmes; Section 4 details the approach followed to identify how a programme contributes to the development of these competencies in future engineering professionals; Section 5 describes, in general, the EPS and, in particular, the experiential active learning process of the EPS@ISEP from the perspective of the 4C2S framework; Section 6 analyses the case study of the development of the Pet Tracker project at the light of the proposed 4C2S framework; Section 7 discusses the outcomes; and Section 8 draws the conclusions.

2. Professional Competencies

Several national and international engineering accreditation organisations and engineering professional bodies around the world have disclosed criteria that should be followed by high education institutions graduating engineers:
European Network for Accreditation of Engineering Education 
(ENAEE) authorises the associated accreditation and quality assurance agencies in Europe to award to accredited engineering degree programmes the EURopean-ACcredited Engineer (EUR-ACE®) label, which is one of the European quality labels in higher education sponsored by the European Commission. The EUR-ACE® Standards and Guidelines for Accreditation of Engineering Programmes (EAFSG) are described in terms of student workload requirements, programme outcomes, and programme management [10]. ENAEE screens bachelor and master engineering programmes with reference to knowledge and understanding; (ii) engineering analysis; (iii) engineering design; (iv) research; (v) engineering practice; (vi) judgement making; (vii) communication and team-working; and (viii) lifelong learning. In particular, EASFG claims that exposing undergraduates to
  • engineering practice enables the acquisition of “knowledge and understanding of the nontechnical—societal, health and safety, environmental, economic, and industrial—implications as well as critical awareness of economic, organisational, and managerial issues”;
  • judgement making enables the “ability to gather and interpret relevant data and handle complexity within their field of study; to inform judgements that include reflection on relevant social and ethical issues; to identify, formulate, and solve engineering problems in their field of study; as well as to manage complex technical or professional activities or projects in their field of study, taking responsibility for decision making”;
  • communication and team-working enables the ability “to communicate effectively information, ideas, problems, and solutions with the engineering community and society at large as well as to function effectively in a national and international context, as an individual and as a member of a team and to cooperate effectively with engineers and non-engineers”;
  • lifelong learning enables the ability “to recognise the need for and to engage in independent life-long learning and to follow developments in science and technology” at the bachelor level and “to engage in independent life-long learning and to undertake further study autonomously” at the master level.
Moreover, ENAEE details that master graduates need to demonstrate the ability to analyse, conceptualise, and solve unfamiliar problems, including the design of innovative analyses and problem-solving methods.
Engineering Council 
(UKEC) sets in the United Kingdom the overall requirements for the Accreditation of Higher Education Programmes in engineering in line with the UK Standard for Professional Engineering Competence. In order for an engineering degree to be accredited in UK, six broad areas of learning are analysed: (i) science and mathematics; (ii) engineering analysis; (iii) design; (iv) economic, legal, social, ethical, and environmental context; (v) engineering practice; and (vi) general skills [11]. According to the Engineering Council [11], Bachelor’s degrees with honours are awarded to students who have demonstrated the following:
  • systematic understanding, including the acquisition of coherent and detailed knowledge, and conceptual understanding to critically evaluate, make judgements, and frame appropriate questions to achieve a solution—or identify a range of solutions—to a problem;
  • awareness of the uncertainty, ambiguity, and limits of knowledge;
  • ability to accurately apply methods and techniques of analysis and enquiry to review, consolidate, and extend their knowledge and understanding and to initiate and carry out projects;
  • ability to communicate information, ideas, problems, and solutions to both specialist and nonspecialist audiences.
  • ability to manage their own learning and to make use of scholarly reviews and primary sources.
In terms of professional competencies, engineering graduates in the UK are expected to exhibit the following professional competencies: (i) exercise of initiative and personal responsibility; (ii) decision-making in complex and unpredictable contexts; and (iii) the learning ability needed to undertake appropriate further training of a professional or equivalent nature.
Accreditation Board of Engineering and Technology 
(ABET) in the United States defines a set of standards, called the Engineering Criteria 2000 (EC2000) [12], for engineering degrees. EC2000 shifted the basis for accreditation from inputs—what is taught—to outputs—what is learned—with the introduction of programme outcomes criteria [13]. The aim of these criteria is to ensure that students attain an understanding of professional and ethical responsibility as well as the broad education necessary to understand the impact of technical solutions in a global, economic, environmental, and societal context. Specifically, ABET specifies under Criterion 3 the so-called a–k list of student outcomes [14]. Moreover, according to a survey distributed to USA employers by the National Association of Colleges and Employers (NACE) [15], the three most important skills of an engineer today are (i) the ability to communicate and work in teams; (ii) the ability to solve or troubleshoot problems in new or unfamiliar situations; and (iii) knowledge of a specific engineering discipline.
Engineers Australia 
(EA) which performs in Australia the professional accreditation of engineering programmes, defines that engineering graduates must demonstrate at the point of entry to practice the following set of competencies [16]:
  • knowledge-oriented—comprehensive and conceptual understanding; knowledge development and research; awareness of contextual factors impacting the engineering discipline; and understanding of the scope, principles, norms, accountabilities, and bounds of contemporary engineering practice;
  • application-oriented—application of engineering methods, techniques, tools and resources and systematic engineering synthesis, design processes, and approaches to run and manage engineering projects;
  • profession-oriented—ethical conduct and professional accountability, effective communication, creativeness, innovation and pro-activity, professional management and conduct, effective team membership, and team leadership.
The Engineers Australia summarises in Reference [17] its accreditation criteria, including the complete set of desired student educational outcomes.
Table 1 maps the desired engineering student skills identified by ABET, EA, UKEC, and ENAEE to the 4C2S framework. These employability skills correspond to the “ability to perform engineering related knowledge, skills, and personal attributes to gain employment, maintain employment, and succeed in the engineering field” [18].

3. Engineering Capstone Programmes

The importance of grounding engineering education in real world experiences is highlighted by the National Academy of Engineering (NAE) in several publications [19,20]. In 2012, NAE reported on a selection of 29 capstone programmes offered by public and private universities and colleges in the USA, which successfully infused real world experiences into engineering undergraduate education [21]. The report features a diverse range of programmes (institution, category, scope, location, and longevity), potential implementation barriers, and strategies to overcome these barriers.
According to Hackman et al. [22], there are four main trends in engineering capstone projects: (i) the key role played by technology in the capstone experience (integration of technology into course administration, instructional methods, industry sponsorship and integration, and course evaluations); (ii) service-learning and community-based projects, in an attempt to provide real-world experiences while simultaneously providing a benefit to society; (iii) multidisciplinary projects, where student design teams are assembled from different majors or from different emphasis areas within a major; and (iv) the incorporation of sustainability principles into the capstone project, often in conjunction with community-based and service-learning projects. EPS@ISEP follows these trends (see Section 5).
The following engineering capstone programmes share similarities with the EPS@ISEP:
  • Oladiran et al. [23] introduced the Global Engineering Teams programme, which adopts a multinational, intercultural, and geographically dispersed team-based approach. It tackled practical engineering problems, and each edition lasted for about six months between April and October. The groups in the programme were virtual teams consisting of students located in different countries and usually across multiple time zones, working in collaboration with industry partners. This programme, like EPS, implemented multinational, multidisciplinary teamwork and favoured real-world problems.
  • Sheppard et al. [24] presented a two-semester pilot project at Stevens Institute of Technology to develop a systems engineering framework for multidisciplinary capstone design. It provided a series of workshops through the course of the capstone project to teach relevant systems engineering concepts in what approximates to a just-in-time mode. Interdisciplinary projects of significant scope were performed by teams of students from engineering and product architecture fields, working with external stakeholders and mentors. It was part of an initiative involving 14 institutions (including all the military academies), sponsored by the Department of Defense. The goal was to inculcate aspects of systems engineering into the education of students in all engineering disciplines through their major capstone project. The similarities with EPS included short intensive project supportive workshops, interdisciplinary projects, and multidisciplinary teams.
  • Hackman et al. [22] described the new approach adopted by the School of Industrial and Systems Engineering at Georgia Tech to the capstone senior design course. The course structure creatively integrated internal and external resources for teaching, like EPS@ISEP, to promote business skills, soft skills, professionalism, and legal issues in an interdisciplinary, on-demand team-teaching format. Students formed teams and identified, scoped, and executed projects for real-world clients. The results showed that project quality and student nontechnical skills improved.
  • Stanford et al. [25] reported on a capstone programme for civil engineering undergraduates employing a class-wide jigsaw approach and addressing community-based, sustainability-related problems. Results revealed that real-world projects with a focus on sustainability have a positive impact on students’ critical thinking skills, leading to an increased knowledge of sustainability, and that open-ended problems with real project constraints could yield a uniquely beneficial learning experience without sacrificing the quality of student design or project deliverables. The pervasive concern with sustainability and the selection of real open-ended problems are common to EPS@ISEP.
  • Palacin-Silva et al. [26] described a team-oriented capstone for software engineering undergraduates directed to the development of software services for sustainability. The course followed a collaborative learning approach, where students worked together to engineer a software project with the lecturer as a facilitator. The projects’ challenge was to link Information and Communication Technologies to greening solutions, incorporating social, economic, and environmental concerns by involving computing, environmental sustainability, and citizen engagement. This approach shares with EPS@ISEP the focus on sustainability-oriented design and sustainability problems.

4. Method

The 4C2S framework allows the adoption of an evidence-based approach to identify how a capstone programme contributes to the development of the identified competencies in future engineering professionals. To illustrate this empirical approach, the EPS@ISEP programme and the Pet Tracker project case are analysed from the 4C2S perspective.
The proposed method analyses the programme and the learning process of the students throughout the semester, searching for evidences of the development of critical professional competences, including the deliverables and the activities performed by the team. Specifically, the method looks for signs of professional behaviour and team work, e.g., the ability to meet deadlines, to define agendas, to lead meetings, to solve conflicts, to report and discuss findings, and to together reach a solution to an open-ended problem.
First, it identifies the skills related to the six core competencies of the framework (Table 1). Then, at the EPS@ISEP programme level, it matches these competencies against the aims (Table 4), learning process (Figure 1), and mandatory team deliverables (Table 3). Finally, at the project level, it analyses in detail the timeline of the Pet Tracker project to quantify the evidences of the development of the 4C2S competencies by the team. This timeline includes the scheduled activities—the project weekly meetings, the supportive module seminars, invited talks, presentation, and assessment events—and multiple milestones, involving the handing-in of different deliverables—Gantt chart, cardboard model, structural and control drawings, list of materials, components and providers, leaflet, brochure, report, poster, video, wiki, and prototype.

5. European Project Semester at ISEP

The European Project Semester was started in 1995 by Arvid Andersen [27]. It is a one semester student-centred international capstone programme currently offered by a group of European high education institutions called the EPS Providers as part of their student exchange programme portfolio more information available at http://www.europeanprojectsemester.eu/. While EPS Providers have the freedom to implement the programme with distinct flavours, they must comply with the EPS 10 Golden Rules listed at http://www.europeanprojectsemester.eu/. EPS provides an integrated framework for undertaking engineering capstone projects, which adopts a project-based learning methodology; focuses on teamwork; and exposes students to cultural, scientific, and technical diversity. In fact, it was designed for the capstone semester of engineering, product design, and business degrees. EPS aims to prepare future engineers to think and act globally [28].
The EPS is a package organised around one central module—the EPS project—and a set of complementary supportive modules. The project proposals refer to multidisciplinary real-world problems, i.e., draw on knowledge from diverse fields, and are open-ended, i.e., specify exclusively top-level requirements such as the compliance with the applicable directives/norms and the budget. According to the EPS 10 Golden Rules, the teams, which are composed of four to six students from different scientific backgrounds and nationalities, are fully responsible for the development of their projects [29]. Multicultural and multidisciplinary educational backgrounds contribute to product development and innovation, develop communication skills, and catalyse collaborative learning among team elements.
The EPS@ISEP programme—the EPS provided by the School of Engineering Instituto Superior de Engenharia do Porto of the Porto Polytechnic—welcomes engineering, business, and product design students since 2011. This 30 European Credit Transfer System Units (ECTU) programme is composed of six modules: Project, Project Management and Team Work, Marketing and Communication, Foreign Language, Energy and Sustainable Development, and Ethics and Deontology. Table 2 presents the programme syllabus. The 2 ECTU modules are project supportive seminars oriented towards the specificities of each team project.
As far as project supervision is concerned, EPS@ISEP adopts a unique model where a panel of multidisciplinary expert advisers acts as a coaching and consulting committee. Concerning communication, the panel is aware that it interacts with students from diverse scientific and cultural backgrounds. Furthermore, in the weekly supervision meeting, only the topics previously specified by the team in the wiki agenda are discussed. Another very important aspect of the coaching methodology is the prompt feedback given to the students. Students meet with the panel once a week to discuss the topics the team previously posted in the wiki agenda.
During the semester, the teams maintain the project wiki and produce several deliverables, including the report, video, paper, manual, brochure, and proof of concept prototype. The structure and presentation of the deliverables are addressed in the communication seminar. The report structure (provided beforehand) includes as mandatory sections the introduction, state of the art, project management, marketing plan, sustainability, ethical concerns, conclusions, and bibliography. The marketing, ethical and deontological concerns as well as eco-efficiency and sustainability measures chapters are produced and refined within the corresponding complementary modules. Table 3 identifies the professional competencies developed with the production of these deliverables. In particular, the state of the art, project management, marketing plan, eco-efficiency measures for sustainability, and ethical and deontological concerns chapters not only report the team’s corresponding studies but also specify the set of product requirements which were directly derived. The wiki is a key tool of the process. It is a collaborative work platform for team members and advisers, as well as the project show case, integrating the project plan, the weekly logbook, the report, and the deliverables areas.
Figure 1 illustrates this process.
The full process takes 15 weeks and includes the set of scheduled activities—the project weekly meetings, the supportive module seminars, invited talks, and global events—and multiple milestones. The global events, which involve lecturers, project coaches, students, and invitees, are the opening session in February (Week 1), the interim presentation in April (week 8), the final presentation in June (week 15), and the closing session, including certificate awarding and prototype hand-in. The milestones correspond to mandatory team and staff inputs. The different modules help drive the process:
Team Building 
| Week 1—Selection of the proposal and establishment of the Team Agreement, which defines the team’s preferred conflict resolution method.
Project Management 
| Week 2–15—Definition of the activities and tasks, task allocation, and Gantt chart of the project (Week 2), followed by a continuous iterative refinement and adjustment cycle.
Communication 
| Week 2–15—Wiki maintenance, uploading of the interim deliverables (Week 7), presentation of interim outcomes (Week 8), refinement of interim deliverables (Week 10), uploading of final deliverables (Week 14), presentation of final outcomes (Week 15), and refinement (Week 16).
Sustainable Development 
| Week 1–7—Survey, application, and reporting of the relevant sustainable development practices and derivation of corresponding product requirements in the chapter “Eco-efficiency Measures for Sustainability”.
Ethics and Deontology 
| Week 2–7—Study, selection, application, and reporting of applicable codes of ethics in order to derive product requirements in the chapter “Ethical and Deontological Concerns”.
Marketing 
| Week 2–7—Research, definition, and reporting of the marketing plan of the proposed product and identification of resulting product requirements in the chapter “Marketing Plan”.
Project Design 
| Week 2–7—Specification of the black box system diagrams and structural drafts (Week 3); analysis of the state of the art (reported in chapter “State of Art”); detailed specification of product requirements (Week 4), detailed system schematics, and structural drawings together with the card board scale model of the proposed solution (Week 5); and definition of the complete list of materials and components (Week 6).
Prototype Implementation and Operation 
(Week 9–15)—Procurement of components and materials, assembly, development, tests, and debugging.
Feedback and Assessment 
| Week 8 and Week 15—Interim Self and Peer (S&P) assessment (Week 7); discussion of the interim outcomes as a team (Week 8); feedback from peers (based on the S&P assessment) and staff, including improvement suggestions from staff; S&P assessment (Week 14); and discussion of the final outcomes as a team and individually (Week 15).
EPS@ISEP contributes to the development of (i) critical thinking and problem solving by specifying open-ended project proposals and by promoting inner team brainstorms and weekly meetings with a group of multidisciplinary coaches; (ii) effective communication by fostering intercultural and professional communication skills—the preparation of multiple textual and media deliverables during the semester and the definition of the agenda, leading, reporting the achievements, and writing the minutes of the weekly project meeting with the panel of advisers; (iii) collaboration and team building by working together in multicultural and multidisciplinary teams (defined according to the “EPS 10 Golden Rules” and the Belbin profiles); (iv) creativity and innovation by being expose to ill-defined open-ended problems, i.e., with very general requirements such as the budget, the applicable European Union (EU) directives, and the adoption of the International System of Units; (v) socio-professional ethics by tackling the related aspects of the project within the ETHDO module, with the writing of a dedicated report chapter, and the PROJE module, with the establishment of associated project requirements; and (vi) sustainable development by addressing all aspects related within the ESUSD and PROJE modules, resulting in the writing of a dedicated report chapter and the identification of related project requirements, respectively. Table 4 maps the EPS aims with the 4C2S framework.

6. EPS@ISEP Pet Tracker Case Study

In the spring of 2013, a team of four students from Finland, Poland, Portugal, and Spain, with background education in Industrial Engineering and Management, Biotechnology, Electrical and Computer Engineering, and Computer Engineering chose to develop a pet tracker [30].
A pet tracker is a device used to monitor and track a pet, which can be used for different purposes, e.g., to follow hunting dogs or to verify the whereabouts of a domestic pet. While in terms of the determination of the location of the animal, these devices can use Global Navigation Satellite Systems (GNSS) or the Global System for Mobile Communications (GSM), in terms of the range of operation, i.e., communication with the owner, they rely either on the GSM/General Packet Radio Service (GPRS) network or on Radio Frequency (RF) transceiver pairs. The positioning accuracy corresponds typically, in the case of the usage of the GSM, to the cell size, i.e., from hundreds to thousands of meters while, in the case of the GNSS, to less than 5 m . While the main objective was to design and develop a pet tracking system, after analysing the state of the art, the team decided to add an activity monitoring feature for product differentiation. This feature allowed the owner to keep track of the pet’s activity and to schedule the exercise it needed. According to the team, the product should “create a unique environment for the pet owner, where functionalities meet the client needs” [31]. Therefore, the final goal was to design, develop, and build a system allowing the pet owner not only to check the location of the pet but also to monitor and share on social media its daily level of activity.
The team, based on the market research, concluded that the Pet Tracker would be the first European pet activity tracking solution. This distinctive feature would make Pet Tracker unique in the European market. Regarding the particular market to address, the team decided first to aim for the Finnish market. The Finnish monthly median gross income was at the time 2776 € [32] with approximately 600,000 registered dogs, including approximately 450,000 pure-bred [33]. Finally, the students determined a final price for the Pet Tracker of 230 €, including fixed costs and a three month service. The monthly service fee packages contemplated by the team were (i) the 3-month package (19.99 €); (ii) the half-year package (34.99 €); and (iii) the full-year package (64.99 €).
The team considered the environmental, social, and economic components of sustainability. Concerning the environmental aspects, the students focused on the control and reduction of the materials and resources used, as well as on the waste produced. The goal was to select materials as durable and recyclable as possible, since using durable and recyclable materials reduces the ecological footprint and creates a better image for the company. They also planned to adhere to electronic device disposal services, promoting the reuse of components and the dispatch of leftovers to the appropriate disposal centres. Regarding the economic aspects of sustainability, among others, the team declared the need to have a continuous improvement/development process, including performance measurement, target setting, action taking, and result review. Such a continuous improvement process implies the development and adoption of quality, environmental, and risk management systems. The social sustainability perspective, i.e., the health and security of the customer and of the pet, was one of the most important concerns of the team. Consequently, the team became committed to designing a harmless and comfortable product both to humans and animals, i.e., the materials, components, and shape of the device must be safe for both the pet and the owner. Finally, the team analysed different codes of ethics and embraced the Fundamental Canons of the Code of Ethics for Engineers of the National Society of Professional Engineers [34]. The code was applied at different stages of the project. The team detailed the main concerns regarding the Pet Tracker from the marketing, environmental, safety and health, manufacturing, intellectual property, and liability viewpoints.
After taking into consideration these different project design dimensions, the team decided to create a product with the following requirements: (i) provide a Web interface for pet owners; (ii) display tracks using Google Maps; (iii) adopt open source technologies; and (vi) offer a light, small, wearable device for pets with on-board data storage, data download interface, and power autonomy of at least 48 h .
The proposed analysis framework was first applied to the team’s wiki, which is available at http://www.eps2013-wiki2.dee.isep.ipp.pt/, specifically, to the project logbook. It found a total of 54 supporting evidences of the six core competencies. Figure 2 displays the results. As expected, the logbook provides multiple evidences of the different framework dimensions. Effective communication, critical thinking and problem solving, collaboration and team building, and socio-professional ethics are predominant. While the logbook provides few creativity and innovation and sustainable development evidences, multiple evidences of this competency can be found in the paper, report, and video of the team.
Figure 3 shows the EPS@ISEP timeline, providing an overall view of the global events, the mandatory teacher feedback on the left size and the team inputs on the right side. Furthermore, the corresponding Pet Tracker team inputs (deliverables) were analysed using the 4C2S framework. The central strip identifies the professional competencies developed by the team based on the evidences found in the deliverables, using a matrix format. The columns correspond, from left to right, to critical thinking and problem solving, effective communication, collaboration and team building, creativeness and innovation, sustainable development, and socio-professional ethics, whereas the lines identify milestones or deadlines. Whenever there is a clear evidence that the team activity induced the development of the corresponding professional competency, the cell is highlighted with the matching colour and a tick.

7. Discussion

Senior capstone programmes provide a comprehensive experience that helps students make the transition from academic life to professional life [22]. Not only do team-based projects encourage students to develop and improve employability skills, such as team work, problem solving, and intra-team communication [35] but also students undertaking real community-oriented projects report a significantly stronger development of work-ready skills [36]. EPS@ISEP not only develops these skills but also follows the trends identified by Hackman et al. [22]: relies on technology, selects service-learning and community-based multidisciplinary projects, and incorporates ethical and sustainability principles into the capstone project.
The 4C2S analysis framework is intended to identify evidences supporting the development by capstone programmes of the core professional competencies in future engineering graduates. Although it can be further detailed, it covers the critical competencies pinpointed by ABET, EA, UKEC, and ENAEE, namely, the scientific, personal, societal, ethical, and sustainability dimensions. These competency dimensions have been mapped into the set of desired engineering employability skills. According to the Royal Academy of Engineering [37], they comprise knowledge, wider employability skills, such as communication, team working, and organisational skills, and important professional behaviours such as ethics. Robinson [38] identified a growing demand for sustainability literacy as organisations across the public, private, and voluntary sectors sought graduates who could help them adapt their policies and practices to meet sustainability objectives. Moreover, the proposed extension of the 4Cs with 2Ss competencies is aligned with the employability agenda, which is about getting graduates to adapt to the new flexible workplace. On the one hand, Conlon [39] stated that “a narrow focus on the skills and values of individual students related to employability is not adequate to prepare them for the challenge of delivering sustainable and just engineering solutions” and that “students need to develop the capacity to situate their individual practice as engineers in its wider social context”. On the other hand, for Robinson [40], “employability is about what makes for successful employment and about maximising good consequences for the individual and society; it is also intrinsically a good, and therefore not value-free”.
Existing skills frameworks typically fall into the 4Cs competencies but fail to include the 2Ss. It is the case of the engineering employability skills framework of Yuzainee et al. [18] for Malaysian engineering graduates. It comprises three main groups—personal knowledge, personal attributes, and personal skills—and ten types of skills—communication, team work, lifelong learning, professionalism, problem-solving and decision-making, competency in application and practice, knowledge of science and engineering principles, knowledge of contemporary issues, engineering system approach, and competency in specific engineering disciplines.
The proposed 4C2S framework was applied to the learning process of a team of students throughout the capstone semester, searching for evidences of the development of the six critical professional competences. Specifically, it inspected the process timeline, the produced deliverables, and the attitude of the teams, identifying signs of professional behaviour and team work, e.g., the ability to meet deadlines, to define agendas, to lead meetings, to solve conflicts, to report and discuss findings, and to together reach a solution to an open-ended problem. It has been shown that the EPS@ISEP programme complies with the 4C2S framework, as depicted in Figure 2 and Figure 3. On the one hand, the experiential learning techniques, such as those adopted by EPS@ISEP, provide various learning experiences such as critical thinking, collaborative learning, and peer evaluation [41]. On the other hand, the pervasive focus on sustainability and ethics imprints these concerns in the future engineers. The goal of the EPS programme is more ambitious than just expecting the students to implement prototypes (in this case the “Pet Tracker”)—it also makes them contribute with their distinct visions of the problem to a common consensual solution. This process is not always easy, since at this educational level the students are not used to collaborating with peers from different nationalities (implying distinct cultural backgrounds) and from different study backgrounds (engineering students tend to think differently from business and product design students).

8. Conclusions

Engineering degrees have different focuses when it comes to the development of professional competencies in undergraduates. While the academic-oriented concentrate on the development of traditional hard competencies and the market-driven consider the 4Cs, this work proposes a holistic perspective, extending the 4Cs with socio-professional ethics and sustainability competencies (2Ss). This extension aims to reposition the engineering profession at the heart of the society by introducing well-being and well-doing as central goals of the engineering practice. In terms of contributions, this work proposes the 4C2S analysis framework, which derives from the needs identified by the society, professional associations, and businesses for Engineering education, as well as an application method for engineering capstone programmes. The adopted method searches for evidences of the development of the desired professional competencies in engineering undergraduates.
To illustrate the application of the framework, the EPS@ISEP engineering capstone implementation and the Pet Tracker project were described and analysed in the light of 4C2S. It involved the mapping of the aims, learning processes, and mandatory deliverables of the EPS@ISEP to the development of the six core competencies and the collection of related evidences from the deliverables and learning journey of the Pet Tracker team. The result shows that EPS@ISEP contributes to fostering the desired 4C2S competencies in engineering undergraduates, comprising the much sought combination of soft, hard, and character employability skills.

Author Contributions

Conceptualization, B.M.; Formal analysis, P.G.; Methodology, B.M.; Project administration, B.M.; Writing—original draft, B.M., Pedro Guedes, M.F.S. and P.F.

Funding

This work was partially financed by National Funds through the Portuguese funding agency, FCT—Fundação para a Ciência e a Tecnologia, within project UID/EEA/50014/2019.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
4C2SCTPS, EC, CTB, CI, SI, and SPE
AMAAmerican Management Association
ABETAccreditation Board of Engineering Technology
CICreativity and innovation
CTBCollaboration and team building
CTPSCritical thinking and problem solving
EAEngineers Australia
ECEffective communication
ECTUEuropean Credit Transfer System Unit
ENAEEEuropean Network for Accreditation of Engineering Education
EPSEuropean Project Semester
ESUSDEnergy and Sustainable Development
ETHDOEthics and Deontology
GETGlobal Engineering Teams
GPRSGlobal Packet Radio Service
GNSSGlobal Navigation Satellite System
GSMGlobal System for Mobile Communications
ISEPInstituto Superior de Engenharia do Porto
MACOMMarketing and Communication
NAENational Academy of Engineering
NSPENational Society of Professional Engineers
PORTUForeign Language
PRMTWProject Management and Team Work
PROJEProject
RFRadio Frequency
SDSustainable Development
SDGSustainable Development Goal
SPESocio-professional ethics
UKECUnited Kingdom Engineering Council
USAUnited States of America

References

  1. American Management Association. Executive Summary: AMA 2012 Critical Skills Survey. 2012. Available online: https://playbook.amanet.org/wp-content/uploads/2013/03/2012-Critical-Skills-Survey-pdf.pdf (accessed on 5 May 2019).
  2. Cohen, S.; Grace, D. Engineers and social responsibility: An obligation to do good. IEEE Technol. Soc. Mag. 1994, 13, 12–19. [Google Scholar] [CrossRef]
  3. Staniškis, J.K.; Katiliūtė, E. Complex evaluation of sustainability in engineering education: Case & analysis. J. Clean. Prod. 2016, 120, 13–20. [Google Scholar] [CrossRef]
  4. Brundtland, G.; Khalid, M.; Agnelli, S.; Al-Athel, S.; Chidzero, B.; Fadika, L.; Hauff, V.; Lang, I.; Shijun, M.; de Botero, M.M.; et al. World Commission on Environment and Development: Our Common Future (Brundtland Report); Technical Report; United Nations: New York, NY, USA, 1987. [Google Scholar]
  5. Lozano, R.; Lukman, R.; Lozano, F.J.; Huisingh, D.; Lambrechts, W. Declarations for sustainability in higher education: becoming better leaders, through addressing the university system. J. Clean. Prod. 2013, 48, 10–19. [Google Scholar] [CrossRef]
  6. United Nations. United Nations Sustainable Development Agenda. 2016. Available online: http://www.un.org/sustainabledevelopment/ (accessed on 5 May 2019).
  7. Fleddermann, C.B. Engineering Ethics, 4th ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2012; Volume 4. [Google Scholar]
  8. Jesiek, B.K.; Zhu, Q.; Woo, S.E.; Thompson, J.; Mazzurco, A. Global Engineering Competency in Context: Situations and Behaviors. Online J. Glob. Eng. Educ. 2014, 8, 1–14. [Google Scholar]
  9. Zhu, Q.; Jesiek, B.K. Engineering Ethics in Global Context: Four Fundamental Approaches. In Proceedings of the 2017 ASEE Annual Conference & Exposition, Columbus, OH, USA, 25–28 June 2017; pp. 1–12. [Google Scholar]
  10. European Network for Accreditation of Engineering Education. EUR-ACE Framework Standards and Guidelines (EAFSG). 2015. Available online: http://www.enaee.eu/wp-assets-enaee/uploads/2012/02/EAFSG_full_nov_voruebergehend.pdf (accessed on 5 May 2019).
  11. Engineering Council. The Accreditation of Higher Education Programmes: UK Standard for Professional Engineering Competence—Third Edition. 2014. Available online: http://www.engc.org.uk/engcdocuments/internet/Website/Accreditation%20of%20Higher%20Education%20Programmes%20third%20edition%20(1).pdf (accessed on 5 May 2019).
  12. Accreditation Board for Engineering and Technology. Engineering Change: A Study of the Impact of EC2000. 2006. Available online: http://www.abet.org/wp-content/uploads/2015/04/EngineeringChange-executive-summary.pdf (accessed on 5 May 2019).
  13. Accreditation Board for Engineering and Technology. Going Global Accreditation Takes Off Worldwide—2008 Annual Report For ABET Fiscal Year 2007–2008. 2008. Available online: http://www.abet.org/wp-content/uploads/2015/04/2008-ABET-Annual-Report-.pdf (accessed on 5 May 2019).
  14. Accreditation Board for Engineering and Technology. Criteria for Accrediting Engineering Technology Programs, 2016–2017. 2017. Available online: http://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-technology-programs-2016-2017/ (accessed on 5 May 2019).
  15. National Academy of Engineering. Engineering Technology Education in the United States. 2016. Available online: https://www.nap.edu/catalog/23402/engineering-technology-education-in-the-united-states (accessed on 5 May 2019).
  16. Engineers Australia. Stage 1 Competency Standard For Professional Engineer. 2013. Available online: https://www.engineersaustralia.org.au/sites/default/files/content-files/2016-12/S02_Accreditation_Criteria_Summary.pdf (accessed on 5 May 2019).
  17. Engineers Australia. S02 Accreditation Criteria Summary, Rev. 2. 2008. Available online: https://www.engineersaustralia.org.au/sites/default/files/shado/Education/Program%20Accreditation/110318%20Stage%201%20Professional%20Engineer.pdf (accessed on 5 May 2019).
  18. Yuzainee, M.Y.; Zaharim, A.; Omar, M.Z. Employability skills for an entry-level engineer as seen by Malaysian employers. In Proceedings of the 2011 IEEE Global Engineering Education Conference (EDUCON), Amman, Jordan, 4–6 April 2011; pp. 80–85. [Google Scholar] [CrossRef]
  19. National Academy of Engineering. The Engineer of 2020: Visions of Engineering in the New Century; The National Academies Press: Washington, DC, USA, 2004. [Google Scholar] [CrossRef]
  20. National Academy of Engineering. Educating the Engineer of 2020: Adapting Engineering Education to the New Century; The National Academies Press: Washington, DC, USA, 2005. [Google Scholar] [CrossRef]
  21. National Academy of Engineering. Infusing Real World Experiences into Engineering Education; The National Academies Press: Washington, DC, USA, 2012. [Google Scholar] [CrossRef]
  22. Hackman, S.; Sokol, J.; Zhou, C. An Effective Approach to Integrated Learning in Capstone Design. INFORMS Trans. Educ. 2013, 13, 68–82. [Google Scholar] [CrossRef]
  23. Oladiran, M.T.; Uziak, J.; Eisenberg, M.; Scheffer, C. Global engineering teams—A programme promoting teamwork in engineering design and manufacturing. Eur. J. Eng. Educ. 2011, 36, 173–186. [Google Scholar] [CrossRef]
  24. Sheppard, K.G.; Nastasi, J.A.; Hole, E.; Russell, P.L. SE CAPSTONE: Implementing a Systems Engineering Framework For Multidisciplinary Capstone Design. In Proceedings of the 2011 ASEE Annual Conference & Exposition (ASEE 2011), Vancouver, BC, Canada, 26–29 June 2011; pp. 1–21. [Google Scholar]
  25. Stanford, M.S.; Benson, L.C.; Alluri, P.; Martin, W.D.; Klotz, L.E.; Ogle, J.H.; Kaye, N.; Sarasua, W.; Schiff, S. Evaluating Student and Faculty Outcomes for a Real-World Capstone Project with Sustainability Considerations. J. Prof. Issues Eng. Educ. Pract. 2013, 139, 123–133. [Google Scholar] [CrossRef]
  26. Palacin-Silva, M.V.; Seffah, A.; Porras, J. Infusing sustainability into software engineering education: Lessons learned from capstone projects. J. Clean. Prod. 2018, 172, 4338–4347. [Google Scholar] [CrossRef]
  27. Andersen, A. The European Project Semester: A Useful Teaching Method in Engineering Education. In Project Approaches to Learning in Engineering Education: The Practice of Teamwork; de Campos, L.C., Dirani, E.A.T., Manrique, A.L., van Hattum-Janssen, N., Eds.; SensePublishers: Rotterdam, The Netherlands, 2012; pp. 15–28. [Google Scholar] [CrossRef]
  28. Andersen, A. Preparing engineering students to work in a global environment to co-operate, to communicate and to compete. Eur. J. Eng. Educ. 2004, 29, 549–558. [Google Scholar] [CrossRef]
  29. Malheiro, B.; Silva, M.; Ribeiro, M.C.; Guedes, P.; Ferreira, P. The European Project Semester at ISEP: the challenge of educating global engineers. Eur. J. Eng. Educ. 2015, 40, 328–346. [Google Scholar] [CrossRef]
  30. Borzecka, A.; Costa, A.; Fagerström, A.; Gasull, M.D.; Malheiro, B.; Ribeiro, C.; Silva, M.F.; Caetano, N.; Ferreira, P.; Guedes, P. Educating global engineers with EPS@ISEP: The “Pet Tracker” project experience. In Proceedings of the 2016 2nd International Conference of the Portuguese Society for Engineering Education (CISPEE 2016), Vila Real, Portugal, 19–21 October 2016; pp. 1–10. [Google Scholar] [CrossRef]
  31. Borzecka, A.; Costa, A.; Fagerström, A.; Gasull, M.D. Pet Tracker—Final Report. European Project Semester, Instituto Superior de Engenharia do Porto. 2013. Available online: http://www.eps2013-wiki2.dee.isep.ipp.pt/doku.php?id=report (accessed on 5 May 2019).
  32. Statistics Finland. Official Statistics of Finland (OSF): Structure of Earnings. 2011. Available online: http://tilastokeskus.fi/til/pra/2011/pra_2011_2013-04-05_tie_001_en.html (accessed on 5 May 2019).
  33. The Finnish Kennel Club. Suomen Kennelliitto—Finska Kennelklubben—The Finnish Kennel Club. 2016. Available online: http://www.kennelliitto.fi/EN/kennelclub/kennelclub.htm (accessed on 5 May 2019).
  34. National Society of Professional Engineers. NSPE Code of Ethics for Engineers. 2016. Available online: http://www.nspe.org/Ethics/CodeofEthics/index.html (accessed on 5 May 2019).
  35. Keller, S.; Parker, C.M.; Chan, C. Employability Skills: Student Perceptions of an IS Final Year Capstone Subject. Innov. Teach. Learn. Inf. Comput. Sci. 2011, 10, 4–15. [Google Scholar] [CrossRef]
  36. Gilbert, G.; Wingrove, D. Students’ perceptions of employability following a capstone course. High. Educ. Skills Work-Based Learn. 2019. [Google Scholar] [CrossRef]
  37. Royal Academy of Engineering. Engineering Skills for the Future, The 2013 Perkins Review Revisited. 2019. Available online: http://www.raeng.org.uk/perkins2019 (accessed on 5 May 2019).
  38. Robinson, Z. Linking employability and sustainability skills through a module on ‘Greening Business’. Planet 2009, 22, 10–13. [Google Scholar] [CrossRef]
  39. Conlon, E. The New Engineer: Between Employability and Social Responsibility. Eur. J. Eng. Educ. 2008, 33, 151–159. [Google Scholar] [CrossRef]
  40. Robinson, S. Ethics and Employability; Learning & Employability, Series Two; The Higher Education Academy: Heslington, UK, 2005; Available online: https://www.qualityresearchinternational.com/esecttools/esectpubs/robinsonethics.pdf (accessed on 5 May 2019).
  41. DeAgostino, T.H.; Jovanovic, V.M.; Thomas, M.B. Simulating Real World Work Experience in Engineering Capstone Courses. In Proceedings of the 2014 ASEE Annual Conference & Exposition (ASEE 2014), Indianapolis, IN, USA, 15–18 June 2014; pp. 1–13. [Google Scholar]
Figure 1. EPS@ISEP learning process.
Figure 1. EPS@ISEP learning process.
Education 09 00119 g001
Figure 2. Evidences found in the weekly logbook.
Figure 2. Evidences found in the weekly logbook.
Education 09 00119 g002
Figure 3. EPS@ISEP process timeline.
Figure 3. EPS@ISEP process timeline.
Education 09 00119 g003
Table 1. Worldwide perspectives of engineering professional skills and 4C2S competencies.
Table 1. Worldwide perspectives of engineering professional skills and 4C2S competencies.
CompetencyBodyDesired Professional Skill
Critical Thinking and
Problem Solving
ABETability to understand the impact of engineering solutions (critical thinking)
ability to identify, formulate, and solve engineering problems
ability to recognise the need for and engage in life-long learning
EAability to undertake problem solving, design, and project work
ability to display critical reflection
capacity for lifelong learning and professional development
UKECability to critically evaluate, make judgements, and frame appropriate questions to achieve a solution to a problem
ability to manage their own learning
ENAEEability to make judgements, identify, formulate, and solve engineering problems as well as to manage complex technical or professional activities
ability to engage in independent life-long learning
Effective
Comm.
ABETability to communicate effectively and work in teams
EAability to display effective communication and pro-activity skills
UKECability to communicate with both specialist and nonspecialist audiences
ENAEEability to communicate effectively with the engineering community and society
Collaboration and
Team Building
ABETability to function on multidisciplinary teams
EAability to assume effective team membership and team leadership
UKECability to work as a member of an engineering team and awareness of team roles
ENAEEability to function in a national and international context, as an individual and as a member of a team, and to cooperate effectively with engineers and non-engineers
Creativity and
Innovation
ABETability to apply knowledge creatively in order to solve a problem
EAability to display effective creativeness, innovation, and pro-activity
UKECability to find creative solutions that are fit for purpose
ENAEEability to design innovative analysis and problem solving methods (master level)
Sustainable
Development
ABETability to consider economic, environmental, and sustainability constraints
EAability to accommodate the economic and environmental responsibilities
UKECability to identify environmental and sustainability limitations
ENAEEability to identify the environmental, economic, industrial, and managerial issues and to understand their implications
Socio-Professional
Ethics
ABETability to work with professional and ethical responsibility
EAability to accommodate social, cultural, ethical, legal, and political responsibilities as well as follow health and safety imperatives
UKECability to identify ethical, health, safety, security, risk, and intellectual property issues and to follow codes of practice and standards
ENAEEability to inform judgements that include reflection on relevant social and ethical issues, taking responsibility for decision making
Table 2. EPS@ISEP Syllabus.
Table 2. EPS@ISEP Syllabus.
ModuleAcronymECTU
ProjectPROJE20
Project Management and Team WorkPRMTW2
Marketing and CommunicationMACOM2
Foreign LanguagePORTU2
Energy and Sustainable DevelopmentESUSD2
Ethics and DeontologyETHDO2
Table 3. Mapping EPS@ISEP deliverables to 4C2S.
Table 3. Mapping EPS@ISEP deliverables to 4C2S.
DeliverablesCTPSECCTBCISDSPE
Interim Report
Interim Presentation
Final Report
Final Presentation
Paper
Poster
Leaflet
User Manual
Video
Table 4. Mapping EPS@ISEP aims to 4C2S.
Table 4. Mapping EPS@ISEP aims to 4C2S.
AimsCTPSECCTBCISDSPE
To train students in teamwork and to emphasise realistic and real-life situations
To demonstrate the ability to use modern design tools and techniques
To demonstrate the ability to plan and run a team-based project
To show the ability to communicate clearly in writing (a proper project report) as well as by other means

Share and Cite

MDPI and ACS Style

Malheiro, B.; Guedes, P.; Silva, M.F.; Ferreira, P. Fostering Professional Competencies in Engineering Undergraduates with EPS@ISEP. Educ. Sci. 2019, 9, 119. https://doi.org/10.3390/educsci9020119

AMA Style

Malheiro B, Guedes P, Silva MF, Ferreira P. Fostering Professional Competencies in Engineering Undergraduates with EPS@ISEP. Education Sciences. 2019; 9(2):119. https://doi.org/10.3390/educsci9020119

Chicago/Turabian Style

Malheiro, Benedita, Pedro Guedes, Manuel F. Silva, and Paulo Ferreira. 2019. "Fostering Professional Competencies in Engineering Undergraduates with EPS@ISEP" Education Sciences 9, no. 2: 119. https://doi.org/10.3390/educsci9020119

APA Style

Malheiro, B., Guedes, P., Silva, M. F., & Ferreira, P. (2019). Fostering Professional Competencies in Engineering Undergraduates with EPS@ISEP. Education Sciences, 9(2), 119. https://doi.org/10.3390/educsci9020119

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop