For a Healthy (and) Higher Education: Evidences from Learning Outcomes in Health Sciences

: Increased recognition of outcomes, or competency-based education, has evolved across higher education on health sciences. However, there is signiﬁcant diversity in the current study of Portuguese programmes. Considering learning outcomes (LO) as indicators of knowledge, skills, abilities, attitudes and the understanding that the student will gain as a result of an educational experience, this study aims to explore which LO are emphasised on the study programmes of health sciences in Portugal. Through a qualitative methodology, carried out through MAXQDA software, all LO of all Portuguese health sciences study programmes submitted to quality accreditation to the Portuguese Agency for Assessment and Accreditation of Higher Education (A3ES) since 2009 until 2016 were analysed. Although speciﬁc knowledge was the most referenced LO, transversal skills were also emphasized, such as critical and reﬂexive analysis / critical thinking, research, ability to organize and plan and professional ethics. Signiﬁcant di ﬀ erences were found between LO selection when the analysis was made by comparing the diverse study programmes. This required assortment of knowledge and skills seems to reﬂect not only the speciﬁcities of each health science programme but also the challenging demands on professionals in the 21st century, along with the necessary changes imposed by society, fostering intercultural understanding, tolerance, mutual respect and an ethic of global citizenship and shared responsibility, crucial enablers of educational development for all in the scope of the 2030 Agenda for Sustainable Development.


Introduction
Learning outcomes (LO), defined as statements of what a learner is expected to know, understand and is capable of doing at the end of a period of learning [1], serve as a vital tool for implementation of the Bologna principles [2]. European educational systems are required to assure their curricula comparability in terms of structure, study programmes and pedagogical methodologies. Hence, assessment methodologies, curricula design procedures, scientific contents, and LO need to be aligned, mainly based on Bloom's taxonomy of learning objectives [19]. For Bloom [19], education should go further than simple factual knowledge and understanding. Academic outcomes such as practical application, synthesis, analysis and assessment should be considered learning objectives. Focusing on Nusche's [17] perspective, cognitive LO could be sub-divided into knowledge and skills acquirement. Knowledge acquirement focuses on a general field of know-how (that is, basic curricula) or on a specific scientific domain. In terms of skills acquirement, cognitive LO integrate, for instance, verbal and quantitative thinking, understanding, analytical processing, critical thinking or problem solving: i.e., skills that are transferable across different disciplinary subjects. On the other hand, non-cognitive skills refer to changes in beliefs or the evolution of certain standards and civic values [20]. Psychosocial development encompasses self-development, namely, maturing identity and self-esteem growth, as well as building up of relations among peers, with teachers, and with the institutions they are inserted into. Interpersonal and intercultural know-how, as well as autonomy and maturity, are also connected with psychological development. In turn, behaviours and ethics, despite being intimately connected, are two distinct dimensions. Behaviours are beliefs directed towards a specific object, while ethics are general patterns of action that transcend behaviours [21]. Social awareness, learning motivations, and respect for diversity are examples of non-cognitive LO concerning behaviours and ethics [14,22].
Considering the literature review and LO diversity in health courses, there is a gap in the identification of those that are most valued by colleges. Thus, this study aims to explore which LO are specifically emphasised in study programmes submitted to quality accreditation in the Portuguese Agency for Assessment and Accreditation of Higher Education (A3ES) [23], i.e., what knowledge, skills, competences and attitudes are reinforced as expected LO in the health education competency profiles. The identification of these LO can reduce the gap identified by Skochelak and Stack [24] by determining whether these LO meet the needs of future health professionals.

Learning Outcomes in Health Sciences
Despite a large consensus on necessary changes in healthcare education, most authors, such as Skochelak and Stack [24], agree that the gap between how health students are trained and the future needs of the health care system continues. According to Simões et al. [25], it is essential to humanize medical education and to deconstruct the image of a profession merely related to social prestige and financial return, with medical students being those who most value the social prestige and the economic components of the profession [26]. Increased recognition of LO-or competency-based education has evolved across health professions' education and training arenas [27]. However, there is a significant diversity among existing study programmes in terms of required courses, international field experiences, and thesis research projects [28]. Additionally, Metzel and Petty [29] advocate a pre-health major in medicine, health and society, which can incorporate structural competency concepts and skills into undergraduate courses. Therefore, Peluso et al. [30] presented the curriculum development working group, formed to assess the current state of curriculum design and implementation; the transformative learning working group, formed to examine how transformative learning theories could inform global health curricular recommendations; and the current status of general health education working group, which is using standardized questionnaire approaches to fill the knowledge gap, in line with Skochelak and Stack [24], who proposed the creation of medical schools of the future. Gonzalo et al. [31] also defined priority areas and potential solutions for the successful integration and sustainability of health systems science in undergraduate medical education: (1) partner with licensing, certifying, and accrediting bodies; (2) develop comprehensive, standardized, and integrated curricula; (3) develop, standardize, and align assessments; (4) improve the undergraduate medical education (UME) to graduate medical education (GME) transition; (5) enhance teachers' knowledge and skills and incentives for teachers; (6) demonstrate value added to the health system; and (7) address the hidden curriculum.
Brewer [32] stated that there are three potentially paradigm-shifting opportunities for undergraduate global health education: (i) the chance to substantially expand the pool of practitioners; (ii) engaging experts from disciplines beyond the health sciences to pursue global health careers; (iii) more public knowledge about global health issues. The partnerships between medical schools and health systems that are commonplace today also use health systems as a substrate for learning [33].
Different authors have studied different skills in tutors (foundation skills; Strauss and Hunter [34]) and in health students, namely, preparation and skilfulness to care for culturally diverse patients [35]; communication [36] and active listening [37]; reflective skills [38]; cross-disciplinary learning, role preparation, demonstrated skills in delivering effective integrated care [39]; specific therapies, case conceptualization, ethics and professional guidelines, governing laws and rules pertinent to behavioural health practice, and professional communication and record-keeping [40]; adolescent health and communication and interdisciplinary approach skills [41]; clinical skills laboratories and specific skills [42]; sensitivity to environmental sustainability [43]; health intervention research [44,45]; discovery and creation of new knowledge, assessment of health service needs, application of research, and dissemination of research findings through publications and participation in local, national and professional societies, research and evaluation projects [46]. Combined together, these different skills greatly contribute to successful LO.

Research Question
Considering the significant diversity among current Portuguese health sciences study programmes, will the LO (as indicators of knowledge, skills, abilities, attitudes and understanding that the student will gain as a result of an educational experience) emphasised in study programmes on health sciences in Portugal also differ?

Study Design
All LO from all health courses taught in Portugal were surveyed. These data are included in the description of each course at A3ES. After the data collection, LO were classified. A content analysis of LO was carried out, considering all LO included in the proposals of study programmes submitted to quality accreditation since 2009 until 2016. The analysis is based on the taxonomy of LO proposed by Dias and Soares [14], according to which LO are discriminated in terms of technical skills (specific knowledge) and generic skills (e.g., critical thinking, communication skills, leadership, or professional ethics). This technique of content analysis is more consistent with a theoretical thematic approach, in which "the researcher looks for themes with important messages inherent in the material" [47], trying to associate them with the taxonomy of LO. The number of incidences of each category mentioned in those study programmes was assessed, not only from an overall perspective but also reflecting each Health Science study programme.

Measures
At an initial stage of the accreditation process, higher education institutions submit an accreditation proposal to A3ES entitled "Request for Prior Accreditation of a New Study Cycle". In the description of each study cycle that is submitted, institutions must describe what the "intended LO" expected to be achieved by students at the end of a specific learning period are. Data analysis focused on information included in this specific question, which is limited by A3ES to 1000 characters. All LO of health sciences study programmes submitted to accreditation in A3ES [23]

Data Analysis
Data analysis was performed through MAXQDA (version 12), a software program designed for computer-assisted qualitative and mixed-methods data, text and multimedia analysis.

Sample Characterization
The sample included 140 health sciences study programmes (51 bachelor's, 70 master's and 19 Ph.D. degrees) from the Portuguese public (92) and private (48) higher education subsectors, as well as polytechnic (82) and university (58) higher education subsystems (Table 1), comprising all health sciences study programmes available. Since, in Portugal, there are no specific standards to be followed regarding LO, the examination of each LO study programme was based on the taxonomy proposed by Dias and Soares [14], according to which LO are characterized in terms of technical skills and generic skills and specifically designed for the Portuguese reality. The number of incidences of each category mentioned on those study programmes was assessed, and the results, translated in percentages, are related to the total of codified excerpts (competence representativeness).

General Results
Overall, specific knowledge is the most-mentioned LO in all categories. Critical and reflexive analysis/critical thinking, research and the ability to organize and plan are the next categories with the highest frequency. Next is professional ethics, followed by teamwork, written communication skills, troubleshooting, personal and social responsibility, concern about social, economic and environmental sustainability and development of autonomous work. Then, the LO of the studied courses value oral communication skills, ability in interpersonal relationships, general knowledge, motivation for excellence, motivation for continuous/long-life learning and decision making. In this alignment, practical knowledge, theoretical knowledge (i.e., innovation, leadership, ability to manage information, initiative and entrepreneurship), information technology, interdisciplinary teamwork and adaptation to new situations are also found. Finally, the less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills ( Table 2). Dentistry, as well as other healthcare study programmes, are the ones that make the most references to specific knowledge, followed by diagnostic and therapeutic technics and nursing study programmes and, finally, medicine, pharmaceutical sciences and therapy and rehabilitation programmes ( Table 2).
Dentistry study programmes are also the ones that most mention critical and reflexive analysis/critical thinking in their LO. Therapy and rehabilitation, nursing, and other health sciences study programmes follow. Finally, medicine, diagnostic and therapeutic technics and pharmaceutical sciences appear. The research category is mainly referred to by the medicine and therapy and rehabilitation. Following this are courses in nursing and pharmaceutical sciences, followed by health, diagnostic and therapeutic technics and dentistry courses. Ability to organize and plan is mostly referred to by dentistry, medicine, therapy and rehabilitation, nursing and health. At a great distance from these courses are those of pharmaceutical sciences and diagnostic and therapeutic technics. As far as professional ethics are concerned, the courses that referred to it most were dentistry and nursing. Following these are courses on therapy and rehabilitation, diagnostic and therapeutic technics, medicine, health and pharmaceutical sciences ( Table 2).

Results by Courses
The LO by groups of courses will be analysed next, and their hierarchical categorical valorisation presented.
Starting this analysis with the medicine courses (Table 3), it is possible to find that the LO of these courses essentially value, in descending order, specific knowledge, research, ability to organize and plan, development of autonomous work and critical and reflexive analysis/critical thinking. Next, distantly, comes concern about social, economic and environmental sustainability; still more distant are the categories of general knowledge, troubleshooting, domain of information technologies, teamwork, ability in interpersonal relationships and professional ethics. These courses do not make any reference to creativity, foreign language proficiency, ability to manage information, international teamwork, negotiation skills, conflict management, innovation, adaptation to new situations or respect for diversity and multiculturality.
The analysis of the nursing courses (Table 4) follows. The LO of these courses mainly value, in descending order, specific knowledge, research, critical and reflexive analysis, ability to organize and plan and professional ethics. More distantly, they value general knowledge, personal and social responsibility, teamwork, ability in interpersonal relationships, leadership, troubleshooting, motivation for excellence and concern about social, economic and environmental sustainability.
These courses do not refer to the foreign language proficiency, negotiation skills or conflict management categories.
Continuing the analysis with diagnostic and therapeutic technics courses (Table 5), it is possible to verify that the LO of these courses highlight, in descending order, specific knowledge, critical and reflexive analysis/critical thinking, research, troubleshooting, personal and social responsibility, professional ethics, and teamwork. These courses do not make any reference to foreign language proficiency, interdisciplinary teamwork, negotiation capacity, conflict management or respect for diversity and multiculturality.
Next, the therapy and rehabilitation courses are analysed (Table 6). It can be verified that the LO of these courses essentially value, in descending order, specific knowledge, critical and reflexive analysis, research, ability to organize and plan, and teamwork. More distantly, they value professional ethics, motivation for lifelong learning, general knowledge, theoretical knowledge, written communication skills, ability in interpersonal relationships, and personal and social responsibility. These courses do not make any reference to foreign language proficiency, international teamwork, conflict management or innovation.
The analysis of pharmaceutical sciences courses (Table 7) follows. The LO of these courses value, in descending order, specific knowledge, followed by motivation for excellence, concern about social, economic and environmental sustainability and research. Immediately following these, they value critical and reflexive analysis/critical thinking and oral communication skills and writing. Following at the same distance are troubleshooting and development of autonomous work.
These courses do not refer to general knowledge (practical and theoretical), decision making, foreign language proficiency, ability to manage information, domain of information technologies, international teamwork, negotiation capacity, conflict management, adapting to new situations, initiative and entrepreneurial spirit, leadership, personal and social responsibility, or respect for diversity and multiculturality. Continuing the analysis with dentistry courses (Table 8), the LO of these courses essentially value specific knowledge and critical and reflexive analysis/critical thinking. Following these, they value the ability to organize and plan, ability in interpersonal relationships, concern about social, economic and environmental sustainability and professional ethics. More distantly, they value practical knowledge, decision making, the domain of information technologies, motivation for lifelong learning, the development of autonomous work, and research. These courses do not make any reference to creativity, oral communication skills, foreign language proficiency, international teamwork, negotiation capacity, conflict management, initiative and entrepreneurial spirit, or leadership. This is followed by the analysis of health courses (Table 9). It can be found that the LO of these courses value above all, specific knowledge, critical and reflexive analysis/critical thinking, and research. Following are skills to organize and plan, written communication skills, concern about social, economic and environmental sustainability, oral communication skills, professional ethics, teamwork, troubleshooting and motivation for excellence. These courses do not refer to conflict management, negotiation capacity, international teamwork or interdisciplinary teamwork.

Discussion
In the context of the last decade, to achieve the SDGs of the 2030 Agenda, namely, SDG 4, i.e., education with quality, and considering LO as major indicators of the knowledge, skills, abilities, attitudes and understanding that students will gain because of an educational experience, this study aims to explore the LO emphasized in analysing health sciences study programmes in Portugal  (2009-2016).
The specific knowledge category is, by far, the most mentioned, followed by critical and reflexive analysis/critical thinking, research, ability to organize and plan and professional ethics. This means that the acquisition of specific knowledge is the most important expectation for the curricular designers of health sciences study programmes, according to the LO definition suggested by Marouchou [1].
This also reinforces that, from the healthcare student perspective, specific knowledge is the most expected and important LO [7], compared with all the other LO. Specific knowledge is thus not only seen as the major teaching aim to be achieved by academics but is, therefore, also seen by health students as the most important learning acquisition in higher education [7].
In terms of critical and reflexive analysis, Devi et al. [38] advocate reflection as an efficient tool to promote the engagement of students in thinking what they have learnt in a given context and how they will apply that learning in future. According to Jacob et al. [47], patient outcomes and safety are directly linked to healthcare workers' critical thinking skills. Other studies, such as the one by Soares et al. [48], have also emphasized both critical thinking and specific knowledge as nuclear dimensions of future Portuguese graduates' competency profiles. This contributes to contextualize and understand the relevance of the results obtained.
Regarding research skill, Basu et al. [49] described the transformation of the Cambridge Health Alliance internal medicine residency programme (2012) into a required social medicine and research-based health advocacy curriculum. One of the innovations consisted of a research-based health advocacy project group within the curricular year, where the students learn health services research methods. This project became a centrepiece of the residency programme, to "stimulate further synthesis of knowledge and skills and to provide residents with a first-hand experience of serving in the role of health advocate" (p. 518). This kind of research encourages the self-identification of the required knowledge [50]. Gonzalo et al. [31] recognized that health courses are encouraged to increase education and research into further integrating schools and student activities with academic health centres and community health programmes so that students could add value to the health system while they learn. This can be achieved by the LO to be valued in each course, thus contributing to a better education for all and a healthier education in health sciences. However, the perceived importance of clinical research skills by students, current residents, and residency programme directors is low [51] and, thus, the awareness of its importance must be better worked out near students.
Most authors include the ability to organize and plan in research-related competencies [31,33,51]. However, some authors considered this skill alone and applied it to other aspects of learning in health courses than research [52]. Safari and Meskini [53] considered that self-monitoring and planning are two important metacognitive skills for problem-solving.
Undergraduate medical students experienced ethical conflicts during their medical training, with a prevalence of conflict in the advanced years of the course [54]. Professional education programmes do not promote moral judgment development unless the programme contains an ethics curriculum [55]. This corroborates the data in this study. In fact, although there are health courses, namely dentistry and nursing, that value professional ethical issues, Portuguese medical courses seem to relegate these dimensions to a secondary plan, not contributing to a healthier higher education and to endowing future medical students with the necessary skills to deal with ethical issues.
The less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills. It is surprising, however, that creativity is the least referenced skill, when one of the most pointed out is critical thinking. Is it possible to develop critical thinking without creativity? Regarding international teamwork and foreign language proficiency being two of the least referenced skills, this also seems to contradict the fact that research is one of the most-mentioned skills. Again, is it possible to perform research with high quality standards without international teamwork and foreign language proficiency? These inconsistencies seem to suggest that decision-makers understand the need for certain LO and include them in their courses. However, they show difficulty operationalizing them. This is in fact difficult to do, and one must be cautious with the operationalisation of LO, particularly the less-mentioned categories, which must be considered in the context of LO found to be more relevant. Concerning the respect for diversity and multiculturalism skills' low focus, this result corroborates Crawford et al.'s [56] findings; i.e., health students' self-perceptions of competence related to cultural diversity were low. Although cultural competence is increasingly included in medical curricula to counter related health issues, Lu et al. [57] suggest that the training often risks reinforcing negative stereotypes. This could be problematic, especially when the paradigm of person-centered care [58] is widely used and understood as a reference framework, which is based on the respect for the individual's personal values and preferences. How is it possible to respect the other, even in a doctor-patient relationship, if respect for diversity is not recognized as an important health outcome is study programmes?
Regarding negotiation skills, almost all authors refer to them, although they were not relevant in the LO in this study. These results may suggest a huge gap between theory and practice with regard to the design of LO, which seems to be in line with reality.
Other health courses, as well as dentistry courses make the most references to specific knowledge. This can be explained, as related to the dentistry course, by the specific technology that is constantly changing and evolving in this area [59] and, concerning the other several courses included in the health category, for the same reason, given the specificity of some of the courses involved. The dentistry course is the one that most refers to critical and reflexive analysis/critical thinking in their LO. Technological development may justify a greater need for critical thinking skills in this course. The research category is mainly referred to by the medicine and therapy and rehabilitation courses, courses that need a greater investment in this area. This is one of the core competencies of medical schools in the employment of evidence-based practice, as medical information becomes quickly obsolete [60] and "does not prepare tomorrow's doctors to practice and learn in an ever-changing clinical environment" (p. 16). Ability to organize and plan is mostly referred to by dentistry, medicine, therapy and rehabilitation, and nursing and health, that is, for almost all health sciences courses. As far as professional ethics is concerned, the courses that referred to it most were dentistry and nursing. In dentistry and nursing, touch mediates professionals' interactions with patients [61], which may justify the concern of these courses with ethical questions. Some authors have called for touch to be included within formal curricula in nursing [62] and dentistry [63].
The results thus highlight that some courses do not value the most important LO to achieve SDG 4 of the 2030 Agenda. If respect for diversity and multiculturalism are neglected, this could mean that educational objectives and expected LO are not really universalist and do not support diversity in education. Education should be understood by all as not just a part of but also a key enabler of sustainable development. This mission, for a more equitable and fair society, is possibly the most relevant and ambitious part of SDG 4, focusing on LO both for the world of work as well as for better preparation for dealing with a diverse citizenship in a global and interconnected world. The LO of today's world deal with this specific goal and its outcomes as a reflection of the values of the current society. Consequently, the reported required assortment of knowledge and skills seems to reflect not only the particularities of each health science study programme but also the challenging demands on the professionals of the 21st century, along with the necessary changes imposed by society, as seen, fostering intercultural understanding, tolerance, mutual respect and an ethic of global citizenship and shared responsibility: crucial enablers of educational development for all.

Conclusions
This study aims to explore which LO are emphasised in health sciences study programmes in Portugal. The findings show that the specific knowledge category is, by far, the most-mentioned LO, followed by critical and reflexive analysis/critical thinking, research, ability to organize and plan and professional ethics. The less-mentioned categories were creativity, respect for diversity and multiculturalism, international teamwork, foreign language proficiency and negotiation skills. Future health professionals' profiles are characterized, above all, by the centrality assumed by specific knowledge (as a technical skill), critical thinking and ability to plan and organize (as soft skills). Success in any venture requires basic skills and knowledge, and this necessary variety of knowledge and skills seems to reflect the demands to the professionals of the 21st century and the specific characteristics of these scientific areas in health sciences, as well as the growing search for information and education with quality (SDG 4) by society, based on a global partnership where aspects formerly ignored are now valued in the context of the Sustainable Development of Agenda 2030, in a direct reflection of how the LO of heath sciences study programmes must be properly addressed, since they will affect the future professional activities of present students with consequences in society.
This article presents some limitations, namely, the non-distinction of LO in public and private universities. Furthermore, it was not possible to ascertain whether the proposed LO were in fact implemented and achieved.