Abstract
In Industry 4.0, people need to be able to handle the vast amount of information from machines. In this sense, for Industry 4.0, higher education institutions play a fundamental role. The methodology of this article sought to identify the knowledge required by Industry 4.0 in the literature; carry out a diagnosis of the courses currently offered by Brazilian universities and the need to incorporate new knowledge, and validate the model in a real application. The survey is classified as exploratory, bibliographic and qualitative, supported by bibliometric research. As a result of the research, we identified that the technical content of courses needs to be reviewed to meet Industry 4.0 demand. Sixty-three per cent of respondents were production engineering course coordinators; 70% of respondents considered that the disciplines of current production engineering courses were not adequate to enable the production engineer to work in Industry 4.0. The priority knowledge for implementation in the curricula was identified as big data, advanced simulation, statistics for large amounts of information and virtual reality. It was also considered necessary to bring the industry closer to the universities. The model developed was applied and validated in a university that was evaluated with the highest score in the National Student Performance Exam (Enade).
1. Introduction
Innovation is the engine that allows industry to increase its competitiveness. A sustainable business provides for the integration of social, political, and economic development [1]. Over the decades, industrial revolutions have taken place, ranging from artisanal manufacture to the automation of processes.
In 2011, at the Hannover Fair (Germany), a new concept emerged as part of the German government’s strategy for the development of high technology for the country‘s manufacturing. Thus was born the term Industry 4.0, from the German “Industrie 4.0”. In Figure 1 we have a summary of the four industrial revolutions.
Figure 1.
Industrial revolutions–modified [2].
The concept of Industry 4.0 can be defined as the complete transformation of the entire sphere of industrial production through the fusion of digital technology and the internet with conventional industry [3].
Engineering 4.0 education should focus on the skills that lead to digitization in the manufacturing sector, with sensors embedded in practically all manufacturing components and manufacturing equipment, omnipresent cyber-physical systems and analysis of all relevant data [4].
Industry 4.0 creates the need for new multifunctional professionals. These new professionals will need to increasingly improve their knowledge about information technology and production processes where it can and should be used [5].
The study “High-tech skills and leadership for Europe” predicts a gap of 500,000 professionals in information technology in Europe in the year 2020 [6].
After the launch of Industry 4.0 in 2011, the number of papers and articles grew exponentially, jumping from 15 in 2011 and reaching 12,588 in 2015, according to the German Genios database [2].
From this context and based on bibliographic research, it was possible to realize that currently there are gaps in the formation of the future engineering professional who will work in the context of Industry 4.0 [7].
One of the areas of training for Industry 4.0 is the higher education institutions. Engineers need to be qualified to acquire the new knowledge required by Industry 4.0, so that they can act in this setting in search of solutions aligned with the new scenario. Identified in several reports as technology integrating engineering, automation and management, production engineering presents itself as one of the central points in providing professionals who will lead the implementation processes of Industry 4.0. Therefore, the objective of the article is to propose a model for evaluation of undergraduate courses in production engineering, aiming to meet the demand originated by Industry 4.0.
The originality of this study is related to the nature of the research, which is applied and which results in a product useful to managers and coordinators of production engineering courses by establishing the knowledge required by Industry 4.0 and its impact on the curricula of the production engineering courses at Brazilian universities.
2. Literature Review
2.1. Industry 4.0
As shown in Table 1, the theme Industry 4.0 has been discussed in the scientific community. The approach is aimed at characterizing this type of industry. However, there is already the beginning of a discussion on the need to introduce new knowledge for professionals who will work in this type of industry.
Table 1.
State of the art: articles in the literature referring to Industry 4.0 (author 2019).
2.2. Knowledge and Competence for Industry 4.0
In Table 2, the main challenges of Industry 4.0 are listed [22].
Table 2.
Industry 4.0 challenges matrix [22].
It is worth highlighting the training and development of professionals and the development of skills and competences. These challenges were raised considering the reports issued by the countries Germany and Brazil. Germany was chosen because it is the country where Industry 4.0 was born and Brazil because it is the target of this article. Other countries have also issued reports that outline a route plan for an Industry 4.0 implementation process.
These reports outline a roadmap for an Industry 4.0 implementation process.
The definitions of competence and skill are related [23]. Skill allows an individual to put into practice the theories that were learned. Competence is a wider concept as it covers three topics: knowledge, skills and attitudes (CHA).
In Industry 4.0, workers will need to acquire a whole new skill set. New educational systems must be introduced [17].
From a technological point of view, even 10 years ago, none of the technologies used in Industry 4.0 was mature enough to be included in any common training program [4]. Several authors cite in their articles the knowledge required for Industry 4.0. Table 3 presents the main knowledge and skills referenced in the literature.
Table 3.
Knowledge and skills for Industry 4.0 (author 2019).
Industry 4.0 deals with the massive manipulation and processing of information, where inferential statistics tools, machine learning, data mining and artificial intelligence should be included in the curriculum, at least in its fundamentals [27].
In the study conducted of engineering courses in South Africa, the areas that needed curriculum adjustments were advanced data analysis, real-time system simulation and human-machine interfaces [5]. There will be a need to create new subjects and adapt some existing ones in higher education [25].
In addition to technical knowledge, spirituality in the workplace also helps promote opportunities for personal growth, contributing positively to society [28].
2.3. Production Engineering Course Subjects – Brazilian Universities
By consulting the website of six public universities, we can see that the curricula of the production engineering courses has been changed in recent years, according to Table 4.
Table 4.
Last revision of the curricula frameworks of the production engineering courses (author 2019).
When we analyze the disciplines of these universities, divided into categories, according to Table 5, we see that, although the discipline could be related to Industry 4.0, the disciplines deal with processes, technologies and systems of traditional industries.
Table 5.
Main university disciplines that could include knowledge of Industry 4.0.
The menus do not yet include knowledge for Industry 4.0. When the disciplines related to information technology and statistics, which are pillars of Industry 4.0, are analyzed, it can be seen that they have been developed without taking into account the knowledge required by Industry 4.0.
3. Research Methods
This chapter aims to describe the methodological procedures that have been adopted for the development of this article.
In order to summarize the steps for this article with their respective objectives, Figure 2 was elaborated.
Figure 2.
Stages for research (author 2020).
- The first stage is related to the literature review. At this stage, the objective was, through a qualitative analysis, to identify the gaps in the scientific community regarding the training of the production engineer according to the knowledge required by Industry 4.0. This stage will be basically qualitative.
- On the second stage, a diagnosis was made, based on the gaps identified in the published articles. This step was supported by a survey through the application of a questionnaire. This stage is concluded with a critical analysis of the answers, so this is a quantitative stage.
- The last stage enabled the elaboration and validation of the model developed through application in a production engineering course
The product of this article is therefore a model for evaluating undergraduate courses in production engineering to meet the demand originated by Industry 4.0.
3.1. Qualitative Stage
In the qualitative phase, there was a predominance of scientific articles published in several countries, such as Germany, the United States, Italy and South Africa.
To access the subjects of the curricula, reviews were basically carried out on the universities’ websites, in addition to consulting the guidelines of the Ministry of Education and Culture and also of the Brazilian Association of Production Engineering (ABEPRO) [35]. By checking the curriculum of the production engineering courses of public universities in Brazil, we sought to show whether the subjects offered and their menus were aligned with the knowledge required by Industry 4.0. This step was essential to be able to assess the answers of the questionnaires that were applied in the quantitative stage.
3.2. Quantitative Stage
The quantitative step was carried out with the support of a survey. Surveying is a data collection technique that aims to describe, compare values and/or explain knowledge [36].
The survey research aimed to collect the perception of specialists in the field of production engineering regarding the need to incorporate knowledge in the menus of undergraduate courses in production engineering in Brazil, aiming at qualifying and training the production engineer in face of the requirements of Industry 4.0. The questionnaire was developed considering the knowledge that was required by Industry 4.0 according to the literature review (Supplementary Materials). In order to ensure the reliability of the study, we sought to compare the information obtained in the survey with the analysis of the menus of six Brazilian universities. Information from Brazilian universities was collected in the qualitative phase (Section 3.1)
3.3. Sample Size
The sample defined for the present study was non-probabilistic, as it was up to the researcher to define the sample size according to the specifications and restrictions of his project [37]. The sample of this study was the coordinators and professors of the production engineering courses who participated in the XXIV National Meeting of Graduate and Post-Graduate Production Engineering Course Coordinators of ABEPRO, held in 2019. For the selection of the sample, identified as non-probabilistic, we considered the knowledge, function and capacity of this group to initiate a more structured debate on the subject.
Initially, before the elaboration of the questionnaire, an interview with the technical director of ABEPRO was carried out in order to identify if the theme would make an academic contribution, so that the researcher could continue his research. Participation in the research was voluntary.
The choice of the course where the evaluation/diagnosis model was applied was justified because it is an institution with relevance among federal universities, having been evaluated with the highest score in the National Student Performance Exam (Enade) [38].
3.4. Pre-testing and Data Collection
The purpose of the pre-test was to highlight possible flaws in the writing of the questionnaire, such as: complexity of the questions, imprecision in the writing, lack of necessity of the questions, embarrassment to the informant, exhaustion etc. [37].
For this purpose, a pre-test was carried out with the participation of three coordinators of the production engineering undergraduate courses, and suggestions were made that were incorporated into the questionnaire. These questionnaires were not considered in the sample summing and in the final tabulation of the research.
This pre-test was also intended to verify the validity of the questionnaire. The validity of an instrument of this type is an important characteristic to evaluate its effectiveness.
3.5. Data Processing and Quality Verification of Results in Statistical Analysis
Since Rensis Likert introduced its method in 1932, several studies have used different scale formats of the Likert type.
Five-point scales are sufficient, since no reliability gain was observed in scales with more than five items [39,40]. The Cronbach alpha coefficient was used to evaluate the degree of reliability of the results obtained with the questionnaire. Values between 0.80 and 0.90 were preferred [41]. The Cronbach alpha coefficient was measured according to Equation [41]:
Cronbach’s alpha coefficient was calculated at 0.84. This demonstrates that the questionnaire was reliable.
3.6. Model for Graduation Course Evaluation in Production Engineering
In order to elaborate the model for evaluating undergraduate courses in production engineering (Figure 3), the results of the statistical analysis of the questionnaires applied was considered, as well as the information that was identified during the literature review.
Figure 3.
Disciplines that should be reviewed and/or modified (author 2019).
4. Discussion of Results
4.1. Questionnaire Response
The survey was comprised of 41 people who answered the questionnaire. Sixty-three per cent of the respondents were coordinators of production engineering courses. Regarding their degree of instruction, 78% of the respondents had a doctorate, 80% had a master’s degree and 97% had some kind of specialization.
4.2. Understanding of Industry Definitions 4.0
Regarding the understanding about the characteristics of Industry 4.0, 55% were able to identify the technologies and concepts that will be present in this industry. This result indicates that it is necessary to improve this debate in the academic environment so that the concepts of Industry 4.0 can be part of this community.
4.3. Curriculum and Menu of Current Course Subjects
Regarding the subjects currently offered in the production engineering courses of the educational institutions, 71% considered that they were not adequate to enable production engineers to work in Industry 4.0.
This result demonstrates the relevance of the theme of this article.
When asked if the disciplines offered in their educational institutions’ production engineering courses used appropriate technologies (3D printing/additive manufacturing, cloud data storage/cloud computing etc.) during the courses, 53% responded that they did not use them.
When asked if the statistics discipline of their educational institution provided enough information for graduates of production engineering to work with large amounts of data (big data analytics) in order to analyze and make decisions in real time, 63% answered no. This percentage reached 85% when questioning the offer of virtual reality training and increased reality during the disciplines.
Finally, 73% of respondents stated that the use of learning factories, where simulation of a real factory takes place using real-time data, was not used today in production engineering courses at their educational institutions.
It can be inferred from the above answers that production engineering courses in Brazil, for the most part, are still not offering disciplines and/or content capable of training production engineers with information about Industry 4.0.
4.4. Menu of Disciplines to be Reviewed
When we evaluated the experts’ perceptions regarding the disciplines that should be reviewed, establishing the priority disciplines, those with the highest percentage (Figure 3), classified as “very high” were production automation (41%), introduction to database design (41%) and computer programming (38%). The one with the highest percentage of “very low” was an introduction to production engineering (15%). This may indicate that disciplines that are related to information technology need to be adapted immediately.
4.5. Proposal for Inclusion of Themes in Existing Disciplines
Regarding the inclusion of subjects in existing disciplines (Figure 4), 50% of respondents considered that the subjects big data and advanced simulation were the ones that should be inserted with higher priority.
Figure 4.
Inclusion of themes in existing disciplines (author 2019).
Regarding the free field questioning related the role of the production engineer in Industry 4.0, most of the comments related to the role of the production engineer in management. Many also cited the role of the production engineer from the design phase to the implementation of this type of industry.
Some comments were broader, as summarized below:
- ✓ The role of the engineer will be much more than just the technical area, but working with people and their development is the great challenge;
- ✓ Integration (of processes and new technologies);
- ✓ Make quick decisions, adding value to the process;
- ✓ Design solutions for the ever-evolving industrial and service context.
4.6. Model for Evaluating a Degree Course in Production Engineering to Meet Industry 4.0-Driven Demand
Based on the answers to the questionnaires and also considering the literature review, a model was developed to evaluate production engineering courses (Table 6). This model was tested in a production engineering course at a Brazilian university, and validated. By answering the developed questionnaire, the coordinator was able to visualize the adherence of his production engineering course to the needs of Industry 4.0, as shown below.
Table 6.
Model for graduation course evaluation in production engineering.
5. Conclusions
The purpose of this work was to investigate in the literature the knowledge requirements of the production engineer as a function of Industry 4.0, in order to propose a model to evaluate the content of production engineering courses.
The literature review concludes that the Industry 4.0 theme has been widely discussed in the scientific community. Most of the discussions have been focused on the characterization of this type of industry. The articles refer to the need to review the content of the courses, especially the technical courses.
Although the theme Industry 4.0 has been discussed since 2011, it is still noticeable that the application/transmission of this knowledge to Brazilian universities has not yet been systematized.
It was verified in the survey that the existing curricula today are not yet adherent to Industry 4.0 mapping. Although some disciplines already exist, it is necessary to adapt their content so that graduates can have the knowledge that is required by Industry 4.0.
According to the criteria established by the researcher and according to the view of the respondents, the subjects that have priority in implementation in the curricula are:
- ✓ Big data;
- ✓ Advanced simulation;
- ✓ Learning factory;
- ✓ Statistics for large amounts of information;
- ✓ Virtual reality.
With the consolidation of the questionnaire answers, it was possible to elaborate a model for the evaluation of production engineering courses, aiming to meet the demand originated by Industry 4.0.
The model was applied in the production engineering course at the Fluminense Federal University. The result was quite positive, as the feedback received indicates that the model developed will encourage the coordinator of the course to think about the knowledge that can be formally inserted in the programmatic content of some disciplines and worked across several disciplines.
The proposed research objective was achieved, since the model elaborated can be applied in the real environment of a production engineering degree course, allowing coordinators to evaluate the adherence of the production engineering degree curriculum to the knowledge required by Industry 4.0.
The diagnostic model, after the application of the questionnaire and the case study, was considered satisfactory because it allowed the coordinator of the production engineering course to make a self-assessment of the knowledge taught in his course and also allows gaps to be identified.
The model developed is the result of findings in the literature review, elaboration and application of survey research, analysis of results and application in a graduate course in production engineering.
The originality lies in the development of a differentiated system, dedicated to the elaboration of an evaluation model for production engineering courses, in view of the knowledge required by Industry 4.0.
The application of this study in the management practices of production engineering courses is in the contribution to the revision of the production engineering courses’ curricula.
The main limitations present in the research are inherent to the methods chosen. That is, the systematic literature review did not cover all possibilities of skills and abilities of Industry 4.0. The research focused only on the knowledge required by Industry 4.0.
The proposal and validation of the model may reflect the subjectivity of the researcher in the elaboration of the instrument and the peculiarity of the sample profile used. Finally, the case study may reflect the subjectivity of the researcher, the participants and the peculiarities of the case.
The main applications of this work are focused on the support and guidance of management practices of undergraduate courses in production engineering.
In short, we can see that Industry 4.0 has brought new knowledge that needs to be incorporated into the curricula of production engineering courses to allow graduates to be prepared to act in this type of industry.
The main applications of this work are aimed at supporting and guiding the coordinators of undergraduate courses in production engineering, so that they can use the curriculum plan and educational strategies to meet the requirements of Industry 4.0.
This study does not conclude the discussion on the subject and does not address all the questions about knowledge that should be included in all engineering courses.
The survey research can also be extended to other engineering courses, with due adaptations, as well as the study of a research among the companies that have worked within Industry 4.0. In addition, it is suggested to continue studies seeking to verify if teachers are prepared to provide the knowledge required by Industry 4.0.
Supplementary Materials
The following are available online at https://www.mdpi.com/2071-1050/12/8/3471/s1.
Author Contributions
Conceptualization, R.G.d.S.; methodology, R.G.d.S. and O.L.G.Q.; software, R.G.d.S.; validation, R.G.d.S. and O.L.G.Q.; formal analysis, R.G.d.S. and O.L.G.Q.; investigation, R.G.d.S., O.L.G.Q.; resources, R.G.d.S. and O.L.G.Q.; data curation, R.G.d.S.; writing—original draft preparation, R.G.d.S. and O.L.G.Q.; writing—review and editing, R.G.d.S. and O.L.G.Q.; visualization, R.G.d.S.; supervision, O.L.G.Q.; project administration, R.G.d.S. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Arruda, L.R.; de Jesus Lameira, V.; Quelhas, O.L.; Pereira, F.N. Sustainability in the Brazilian Heavy Construction Industry: An Analysis of Organizational Practices. Sustainability 2013, 5, 4312–4328. [Google Scholar] [CrossRef] [Scilit]
- Wahlster, Wolfgang. Industrie4.0: Cyber-Physical Production Systems for Mass Customization. German-Czech Workshop on Industrie4.0-Prague, 11 April 2016. Available online: http://www.dfki.de/wwdata/German-Czech_Workshop_on_Industrie_4.0_Prague_11_04_16/Industrie_4_0_Cyber-Physical_Production_Systems_for_Mass_Customizations.pdf (accessed on 9 January 2018).
- European Parliament. Industry 4.0 Digitalisation for productivity and growth. Setembro de 2015. Available online: http://www.europarl.europa.eu/RegData/etudes/BRIE/2015/568337/EPRS_BRI%282015%29568337_EN.pdf (accessed on 29 December 2017).
- Assante, D.; Caforio, A.; Flamini, M.; Romano, M. Smart Education in the Context of Industry 4.0. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON), Dubai, United Arab Emirates, 9–11 April 2019; p. 1140. [Google Scholar]
- Sackey, S.M.; Bester, A. Industrial engineering curriculum in industry 4.0 in a South African context. S. Afr. J. Ind. Eng. 2016, 27, 101–114. [Google Scholar] [CrossRef] [Scilit]
- Ciolacu, M.; Svasta, P.M.; Berg, W.; Popp, H. Education 4.0 for Tall Thin Engineer in a Data Driven Society. In Proceedings of the 2017 IEEE 23rd International Symposium for Design and Technology in Electronic Packaging (SIITME), Constanta, Romania, 26–29 October 2017. [Google Scholar]
- Motyl, B.; Baronio, G.; Uberti, S.; Speranza, D.; Filippi, S. How will change the future engineers’ skills in the Industry 4.0 framework? A questionnaire survey. Procedia Manuf. 2017, 11, 1501–1509. [Google Scholar] [CrossRef] [Scilit]
- Sakuneka, T.; Marnewick, A.; Pretorius, J.-H. Industry 4.0 competencies for a control systems engineer. In Proceedings of the IEEE Technology & Engineering Management Conference (TEMSCON), Atlanta, GA, USA, 12–14 June 2019. [Google Scholar]
- Herter, J.; Ovtcharova, J. A Model based Visualization Framework for Cross Discipline Collaboration in Industry 4.0 Scenarios. Procedia CIRP 2016, 57, 398–403. [Google Scholar] [CrossRef] [Scilit]
- Meissner, H.; Ilsenr, R.; Aurich, J.C. Analysis of control architectures in the context of Industry 4.0. Procedia CIRP 2017, 62, 165–169. [Google Scholar] [CrossRef] [Scilit]
- Tupa, J.; Simota, J.; Steiner, F. Aspects of risk management implementation for Industry 4.0—Procedia Manufacturing. In Proceedings of the 27th International Conference on Flexible Automation and Intelligent Manufacturing, FAIM2017, Modena, Italy, 27–30 June 2017; pp. 1223–1230. [Google Scholar]
- Cohen, Y.; Faccio, M.; Galizia, F.G. Assembly system configuration through Industry 4.0 principles: The expected change in the actual paradigms. IFAC PapersOnLine 2017, 50, 14958–14963. [Google Scholar] [CrossRef] [Scilit]
- Bortolini, M.; Ferrari, E.; Gamberi, M.; Pilati, F.; Faccio, M. Assembly system design in the Industry 4.0 era: A general framework. IFAC-PapersOnLine 2017, 50, 5700–5705. [Google Scholar] [CrossRef] [Scilit]
- Peruzzini, M.; Grandi, F.; Pellicciari, M. Benchmarking of Tools for User Experience Analysis in Industry 4.0. Procedia Manuf. 2017, 11, 806–813. [Google Scholar] [CrossRef] [Scilit]
- Hecklau, F.; Galeitzke, M.; Flachs, S.; Kohl, H. Holistic Approach for Human Resource Management in Industry 4.0. Procedia CIRP 2016, 54, 1–6. [Google Scholar] [CrossRef] [Scilit]
- Wittenberg, C. Human-CPS Interaction – requirements and human-machine interaction methods for the Industry 4.0. IFAC-PapersOnLine 2016, 49, 420–425. [Google Scholar] [CrossRef] [Scilit]
- Sung, T.K. Industry 4.0: A Korea perspective. Technol. Forecast. Soc. Chang. 2018, 132, 40–45. [Google Scholar] [CrossRef] [Scilit]
- Baena, F.; Guarin, A.; Mora, J.; Sauza, J.; Retat, S. Learning Factory: The Path to Industry 4.0. Procedia Manuf. 2017, 9, 73–80. [Google Scholar] [CrossRef] [Scilit]
- Long, F.; Zeiler, P.; Bertsche, B. Modelling the flexibility of production systems in Industry 4.0 for analysing their productivity and availability with high-level Petri nets. IFAC-PapersOnLine 2017, 50, 5680–5687. [Google Scholar] [CrossRef] [Scilit]
- Erol, S.; Jäger, A.; Hold, P.; Ott, K.; Sihn, W. Tangible Industry 4.0: A scenario-based approach to learning for the future of production. Procedia CIRP 2016, 54, 13–18. [Google Scholar] [CrossRef] [Scilit]
- Mrugalska, B.; Wyrwicka, M.K. Towards Lean Production in Industry 4.0. Procedia Eng. 2017, 182, 466–473. [Google Scholar] [CrossRef] [Scilit]
- Volpe, W. Analysis of production engineering courses in the context of training engineers for the industry 4.0 environment. Dissertation presented to the Faculty of Applied Sciences, Institute of the State University of Campinas. Available online: http://repositorio.unicamp.br/bitstream/REPOSIP/335249/1/Volpe_Waini_M.pdf (accessed on 20 January 2020).
- Gale, A.; Brown, M. Project management professional development. J. Manag. Dev. 2003, 22, 410–425. [Google Scholar] [CrossRef] [Scilit]
- Karre, H.; Hammer, M.; Kleindienst, M.; Ramsauer, C. Transition towards an Industry 4.0 state of the LeanLab at Graz University of Technology. Procedia Manuf. 2017, 9, 206–213. [Google Scholar] [CrossRef] [Scilit]
- Benesova, A.; Tupa, J. Requirements for Education and Qualification of People in Industry 4.0. Procedia Manuf. 2017, 11, 2195–2202. [Google Scholar] [CrossRef] [Scilit]
- IFTF—Institute for the Future. Future Work Skills 2020. University of Phoenix Research Institute. 2011. Available online: http://www.iftf.org/uploads/media/SR-1382A_UPRI_future_work_skills_sm.pdf (accessed on 29 December 2019).
- Ramirez-Mendoza, R.A.; Menendez, R.M.; Iqbal, H.; Saldivar, R.P. Engineering Education 4.0—Proposal for a new Curricula. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), Santa Cruz de Tenerife, Canary Islands, Spain, 17–20 April 2018; p. 1273. [Google Scholar]
- Bella, R.L.F.; Quelhas, O.L.G.; Ferraz, F.T.; Bezerra, M.J.S. Workplace Spirituality: Sustainable Work Experience from a Human Factors Perspective. Sustainability 2018, 10, 1887. [Google Scholar] [CrossRef] [Scilit]
- Universidade Federal Fluminense (UFF). Curso de Engenharia de Produção. Available online: http://engenhariadeproducaoniteroi.sites.uff.br/sobre-o-curso/disciplinas/ (accessed on 19 August 2019).
- Universidade Federal De Pernambuco-Ufpe-Curso de Engenharia de Produção. Available online: https://www.ufpe.br/documents/480103/0/engenharia_producao_perfil_5203.pdf/a1ce1e7d-0ea2-4867-ac36-1f0c12182043 (accessed on 19 August 2019).
- Universidade Federal Do Rio De Janeiro—UFRJ. Curso de Engenharia de Produção. Available online: http://www.poli.ufrj.br/graduacao_cursos_engenharia_producao.php (accessed on 19 August 2019).
- Universidade Desao Paulo—USP. Curso de Engenharia de Produção. Available online: https://uspdigital.usp.br/jupiterweb/listarGradeCurricular?codcg=3&codcur=3083&codhab=3000&tipo=N (accessed on 19 August 2019).
- Universidadefederal Do Rio Grande Do Sul—UFRGS. Curso de Engenharia de Produção. Available online: http://www.ufrgs.br/producao/pagina/64 (accessed on 19 August 2019).
- Universidade De Campinas—UNICAMP. Curso de Engenharia de Produção. Available online: https://www.dac.unicamp.br/sistemas/catalogos/grad/catalogo2018/curriculoPleno/cp102.html (accessed on 19 August 2019).
- Associação Brasileira De Engenharia De Produção (ABEPRO). Referências de Conteúdos da Engenharia de Produção. Documento Elaborado pela Comissão de Graduação e referendado no GT de Graduação do Encep 08 e Enegep 08–16/10/08. Available online: http://www.abepro.org.br/arquivos/websites/1/Áreas%20da%20Engenharia%20de%20Produção.pdf (accessed on 27 December 2017).
- Fink, A. How to Conduct Surveys: A Step-By-Step Guide, 5th ed.; Sage Publications: Los Angeles, CA, USA, 2013. [Google Scholar]
- Gil, A.C. Como Elaborar Projetos de Pesquisa, 5th ed.; Atlas: São Paulo, Brazil, 2010. [Google Scholar]
- Exame Nacional de Desempenho dos Estudantes (ENADE). Available online: http://emec.mec.gov.br/emec/consulta-cadastro/detalhes-curso/d96957f455f6405d14c6542552b0f6eb/NTcy/9f1aa921d96ca1df24a34474cc171f61/MjIxOQ (accessed on 27 February 2019).
- Jenkins, G.D.; Taber, T.D. A Monte Carlo Study of Factors Affecting Three Indices of Composite Scale Reliability. J. Appl. Psychol. 1977, 62, 392–398. [Google Scholar] [CrossRef]
- Lissitz, R.W.; Green, S.B. Effect of the number of scale points on reliability: A Monte Carlo approach. J. Appl. Psychol. 1975, 60, 10–13. [Google Scholar] [CrossRef] [Scilit]
- Streiner, D.L. Being inconsistent about consistency: When coefficient alpha does and doesn´t matter. J. Personal. Assess. 2003, 80, 217–222. [Google Scholar] [CrossRef] [Scilit]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).









