Next Article in Journal
The Impact of COVID-19 on Faculty Technological Knowledge Development at an Academic Medical Center
Next Article in Special Issue
“A Lot Takes Place Digitally Now, So It Can Be Good to Train on It’’: A Large-Scale Repeated Cross-Sectional Study on Recording Live-Streamed Educational Activities among Health, Social, and Education Students
Previous Article in Journal
Providing Institutional Support for Academic Engagement in Online and Blended Learning Programs
Previous Article in Special Issue
Enhancing Learning in Tourism Education by Combining Learning by Doing and Team Coaching
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 12
Issue 10
10.3390/educsci12100642
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Viewpoint

Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs

by
Kayla Vieno
1,*,
Kem A. Rogers
1 and
Nicole Campbell
2
1
Department of Anatomy and Cell Biology, Western University, London, ON N6A 3K7, Canada
2
Department of Physiology and Pharmacology, Western University, London, ON N6A 3K7, Canada
*
Author to whom correspondence should be addressed.
Educ. Sci. 2022, 12(10), 642; https://doi.org/10.3390/educsci12100642
Submission received: 7 June 2022 / Revised: 7 September 2022 / Accepted: 19 September 2022 / Published: 22 September 2022
(This article belongs to the Special Issue Higher Education: Improvements for a Better System)
Browse Figure
Versions Notes

Abstract

:
Undergraduate and master’s programs—thesis- or non-thesis-based—provide students with opportunities to develop research skills that vary depending on their degree requirements. However, there is a lack of clarity and consistency regarding the definition of a research skill and the components that are taught, practiced, and assessed. In response to this ambiguity, an environmental scan and a literature search were conducted to inform the creation of a comprehensive list of research skills that can be applied across programs and disciplines. Although published studies directly comparing research skills in thesis and non-thesis programs were limited, the specific skills reported in each program type were similar. This viewpoint article identifies the following seven research skills that were most frequently reported across both thesis and non-thesis programs: critical appraisal, information synthesis, decision making, problem solving, data collection, data analysis, and communication. When contextualized appropriately, these skills can be useful for a student during their academic program and are transferable across a range of future career pathways. Broadening the definition of “research skills” can inform curricular updates and program development, independent of their program type, to ensure that students are presented with explicit opportunities to develop the skills needed to succeed in their educational and occupational endeavours.

1. Introduction

A common assumption across academia is that students develop research skills solely through the practice of hands-on research, such as performing experiments in a laboratory. However, the reality is that undergraduate and master’s students can develop their research skills through an array of experiences in thesis- and non-thesis-based programs. While pursuing a thesis and its related experiences, students complete various tasks related to a focused research project under the supervision of a faculty member [1,2]. This research occurs during the final year of undergraduate programs or throughout a graduate degree, with both experiences culminating in the writing and defense of the thesis [2]. A major difference between the thesis-based experiences of undergraduate and master’s programs is the level of depth and autonomy in which the student executes their project. In contrast, non-thesis research experiences can take a variety of formats. For example, undergraduate non-thesis research can be a laboratory-based research course, a literature review course, or inquiry-based assessment [1,3,4]. At the master’s level, non-thesis research experiences are generally incorporated into course-based programs such that the research can be executed as a major paper, or through coursework, activities, and assignments [5]. Despite the differences in the degree requirements between thesis and non-thesis programs, both provide students with distinct opportunities for research skill development to take place at the undergraduate and graduate levels.

2. Statement of the Problem

Across the literature, there is a lack of clarity and consistency surrounding the definition of research skills and which specific skills should be taught, practiced, and assessed at the undergraduate and master’s levels in both thesis and non-thesis programs. In addition, many research experiences involve the implicit development of research skills or assume that skills are naturally being developed simply by participation in research. However, this approach can lead to ineffective opportunities for skill development and student confusion as they may not be aware of the specific skills that they are developing or the experiences that contribute to said skill development. In fact, previous studies have found that many students who participate in research experiences struggle to understand the relevance of their education to their future aspirations [6,7]. These students often hold misconceptions about research that can hinder their learning not only at the beginning of a research experience but upon completion as well [6,8,9]. The lack of a clear definition of research skills and limited opportunities for explicit skill development can also contribute to greater variability in outcomes across programs and even across students within a program. The validated instruments for measuring development of research skills are limited and the skills included in each one are variable, as a standardized list of research skills does not exist in the literature. As such, misconceptions surrounding research skill development can be present in academia, which can hinder students’ success in their programs and beyond.

3. Theoretical Framework

Van Rossum and Schenk (1984) [10] state that learning is influenced by students’ conceptions of their studies. Extending this to research learning, these conceptions encompass the knowledge and beliefs that students have about research and are influenced by the research environments and experiences that they are involved in [7,11]. Focusing on knowledge development in relation to concept formation, students must learn procedural knowledge for research skill development to take place [12,13,14,15,16]. Thus, the theoretical framework for this work is informed by learning conceptions and procedural knowledge, which are applicable to research skill development in undergraduate and master’s programs.
Procedural knowledge is the understanding and ability to execute tasks applicable to a given domain [12,13,14,15,17]. Declarative knowledge, which is an understanding of relevant core facts, is crucial for procedural knowledge to develop [15,17]. Taken together, declarative and procedural knowledge contribute to structural knowledge, which is an understanding of the relationships between ideas within a domain [12]. Students may also form conceptions at the epistemic level, which describes the ability to understand the nature and sources of knowledge [15]. Despite the importance of procedural knowledge for skill development, previous research has revealed that students may struggle to form conceptions at this level. For example, a study by Salmento et al. (2021) [18] discovered that only 18.4% and 7.4% of students’ conceptions of theory were at the procedural and epistemic levels, respectively. Therefore, for students to develop their research skills, educators must also ensure that effective methods are implemented for the learning of basic research knowledge.
A study by Balloo et al. (2016) [16] explored the factors that contribute to the development of research methods-related knowledge and research skill development in undergraduate students. Their findings revealed that self-regulation and motivation were related to a higher degree of structural knowledge, implicating that these factors likely influence the development of research proficiency. Research self-efficacy and research interest also showed a significant positive correlation with knowledge, but the largest predictor of research knowledge and skills was the deliberate practicing of research methods. Many of these contributing factors are dependent on the individual student, yet the practice of research methods is a component that can be incorporated into the curriculum by educators to increase students’ knowledge and skill gains [16].
Students’ conceptions have been found to impact their approaches to learning, learning orientations, outcomes, and future skill applications [10,19,20,21,22,23]. Further, their conceptions of the importance of research for their future work can affect career readiness [24]. Yet, Murtonen et al. (2008) [24] discovered that about half of the students that participated in their study on research skill views were not convinced that research and statistics skills would be useful for their future work. If students do not understand the value of research for their futures, they may struggle with motivation when completing research tasks, which is a factor previously described to be correlated with the development of research proficiency [16,24]. Murtonen et al. (2008) [24] also found that students who valued research skills for their future work were more task-oriented, had low ego-defensiveness, used a deep approach to learning, and experienced fewer struggles in research methodology courses than students who did not see research skills as important for their careers. A different study by Shaw et al. (2013) [25] described that research-related self-efficacy, motivation, familiarity with the research environment, and a positive orientation towards research were key predictors that contribute to undergraduate students’ feelings of preparedness for future research.
Taken together, a variety of factors can influence the conceptions that students hold about research, thus affecting their understanding of procedural knowledge and development of research skills. By considering the factors that influence students’ conceptions and the effect that these conceptions can have on learning, educators can adapt their courses and programs for effective research teaching. Therefore, this theoretical perspective is appropriate for interpreting the literature on research skills in thesis versus non-thesis undergraduate and master’s programs.

4. Context and Methodology

In response to the ambiguity surrounding research skill development in undergraduate and master’s programs, an environmental scan and a literature search were conducted to inform the creation of a comprehensive list of research skills that can be applied across disciplines and programs (Figure 1). The scan was performed using the Journal Storage and Google Scholar databases. A variety of sources were included in the results, such as primary journal articles, book chapters, and meeting reports. The included sources were grouped into three categories based on the approach used to discuss research skills in undergraduate and master’s programs: targeted, perceived, or assessed research skills. Each of these categories presented a different approach to defining skill development. When combined, a comprehensive research skill list was created that considered the various perspectives and research findings of faculty, students, and alumni. The term “faculty” is used throughout this article to include the roles of professors, instructors, and research supervisors.
The three research skill categories used to guide the environmental scan and literature search can be defined as follows:
  • Targeted—research skills that faculty explicitly stated as a goal of their courses or programs, or that groups of experts considered to be important during their research experiences;
  • Perceived—research skills that students or faculty believed were developed during the course or program experience;
  • Assessed—research skills determined to have been developed during one’s program using objective measurements, such as assessments guided by in-depth rubrics.
Each of these categories made an important contribution to the creation of a comprehensive list for broadening the definition of research skills.
The sources that described research skills in general, without detailing specific skills, were excluded from the environmental scan. To account for differences in terminology across the sources, certain skills were grouped together. For example, both oral and written communication were collectively considered to be ‘communication’ skills. Similarly, the skill of critical thinking was grouped under the ‘problem solving’ category. The studies included in this review were from North America, the United Kingdom, and Australasia. Most papers focused on skill development in STEM or allied health science disciplines. The two exceptions were concerning research courses in a Master of Social Work program [26] and a paper about teaching research methods to undergraduate and graduate Sociology students [27]. Many of the STEM and allied health science papers also included other disciplines in their studies, such as social sciences, arts and humanities, and business and technology. Although most of the sources included in the environmental scan were based on STEM and allied health science degrees, it is suggested by the authors that these skills can be broadly applied across disciplines for incorporation into thesis and non-thesis research experiences at the undergraduate and master’s levels.

5. Seven Core Research Skills Transferable across Disciplines and Degrees

Research skills are defined as actions or tasks that can be taught, practiced, and performed to establish facts, postulate new ideas, test ideas to collect data, and analyze data to draw conclusions [28]. This definition highlights that research skills encompass more than just hands-on technical skills. This was also supported by the findings of this study’s environmental scan and literature search, which revealed seven research skills that were most frequently reported across both thesis and non-thesis programs at the undergraduate and master’s levels:
  • Critical appraisal—evaluating the methods, data, and conclusions of published research to determine its validity and reliability;
  • Information synthesis—combining information from various sources in a logical manner to draw conclusions;
  • Decision making—selecting and executing a specific course of action;
  • Problem solving—identifying sources of difficulty and finding reasonable and effective solutions to them;
  • Data collection—gathering information using structured methods to support the objectives of the study;
  • Data analysis—manipulating and modelling data to reveal trends and correlations to make conclusions related to a set of study objectives;
  • Communication—the sharing of information with others through either written or verbal means.
The results from this study’s environmental scan and literature search are presented in Table A1. The studies that directly compared thesis and non-thesis experiences were limited, such that most of the papers included in this literature search were focused on thesis or non-thesis research in specific programs or at individual institutions. These sources revealed many similarities between research skills in both thesis and non-thesis programs; thus, all the skills included in the research skill list were observed across both program types. As previously mentioned, these seven research skills were either targeted, perceived, or assessed across studies of research experiences in undergraduate and master’s programs. Therefore, including sources from all three of these categories within the results of the scan creates a definition of “research skills” that is current and comprehensive. Sources that are based on targeted research skill development represent the ideas and opinions of experts in the field regarding the goals and outcomes for students’ skill development. Perceived skill development portrays the lived experiences of students and faculty members within undergraduate and master’s level research experiences, illuminating their perspectives on skill development within these courses and programs. Finally, assessed skill development provides an objective measurement of students’ skill competencies, offering concrete evidence as to the efficacy of various research skill development opportunities. Each of these three categories of sources views research skills through a unique lens, but collectively, they comprise a thorough definition of “research skills” across undergraduate and master’s programs. The remainder of this section will provide specific examples of sources describing targeted, perceived, and assessed research skills across undergraduate and master’s programs.
Targeted skills were most frequently reported in sources such as secondary articles, guides, and meeting reports. Certain sources were based on established programs or courses that explicitly stated specific research skill development as a goal. For example, a review by Healey and Jenkins (2009) provided a guide for targeted research skill development by providing many examples of undergraduate courses across the United Kingdom, North America, and Australasia that have incorporated explicit research skill development opportunities into their curricula. Examples include a course where students were presented with experimental data and guided through the process of data analysis and presentation [29,30], and another where students practiced collecting data from their classmates to explore students’ quality of life [29]. Furthermore, other sources described the skills that educational organizations or associations deemed important for research experiences. One example is the Ontario Council on Graduate Studies’ ‘Principles for Graduate Study at Ontario Universities’, which described the skills and abilities that students should demonstrate upon receiving their master’s degree in both thesis and non-thesis programs [5]. The guide described that successful master’s students should demonstrate communication skills, the ability to apply a knowledge base to solve a problem, and autonomy, including within decision making, independent of the type of program that they are enrolled in. Taken together, these studies reveal that skills can be targeted at multiple levels across academia, from course activities and expectations to the defining characteristics of a degree type.
Although sources about targeted research skills are focused on the opinions of experts in the field, studies of perceived research skill development aim to understand student and faculty perspectives and lived experiences involving these research opportunities. An understanding of this category is important, as the conceptions that students hold can affect their skill utilization in the future [19,24]. Additionally, it has been found that educators’ conceptions of students’ capacity for learning align with their teaching conceptions [31], suggesting that the research skills that educators perceive students to be developing may be the skills that they value and emphasize in their teaching. An example of a perceived research skill development study compared student self-ratings before and after a thesis-based undergraduate research experience and found significant improvements in oral communication, written communication, data collection, and data analysis skills, as well as an effective usage of primary scientific literature [32]. A separate study of an interdisciplinary graduate course collected both student and faculty perspectives on the course’s first offering through surveys and an end-of-course meeting [33]. When comparing student self-evaluations and faculty ratings at the beginning and end of the course, significant improvements were reported in students’ communication, critical appraisal, and data analysis skills. The collection of student and faculty perspectives throughout the duration of a course or program can provide a greater understanding of students’ skill development based on feedback from those directly involved in the research experiences.
The sources describing assessed skill development were the least common across the literature. This finding was likely because certain skills can be difficult to assess objectively, and validated instruments do not exist to measure all research skills [34]. In one study of assessed skill development involving undergraduate students, a writing assignment was completed at the beginning and end of a course during which students were taught strategies for writing to the lay public [35]. Their assignments from both timepoints were then compared and they received grades from faculty as well as members of the public. Most students received higher grades for their paper written after the course, suggesting that the explicit teaching of writing skills had a positive effect on that skill development. In a similar study, thesis graduate students wrote a research proposal at the beginning (pre-proposal) and end (post-proposal) of their first year [36]. These proposals were then graded using a validated rubric and comparisons were made between the grades of students who had participated in undergraduate research and those who had not. Students who had been involved in undergraduate research received higher grades on both their pre-proposals and post-proposals in all categories except ‘testable hypothesis’ and ‘introduction and context.’ Although less common than perceived skill measurements, the methods for determining students’ assessed skill levels can be incorporated into courses and programs to determine the efficacy of research skill development opportunities. The seven skills previously described were those reported across studies of targeted, perceived, and assessed skill development (Table A1), thereby suggesting that they should be considered by faculty when creating research skill development opportunities in both thesis and non-thesis undergraduate and master’s curricula.

6. The Importance of Research Skill Development in Academia and Beyond

The development of research skills during both undergraduate and master’s programs offers a range of benefits for faculty and students. The positive effects for faculty include a narrowed divide between their teaching and research and a clarification of their teaching methods [37]. Faculty also reported that incorporating explicit research skill development into their courses supported their own research endeavours and led to changes in their understanding of their academic discipline [37]. These positive effects on faculty influenced students as they described an increased clarity of the purpose of learning tasks and assessments, a deeper understanding for their discipline, and an increased ability to think as a scientist would [3,37,38,39]. Students also reported that courses with explicit research skill development opportunities were characterized by an improved quality of feedback from their faculty and even contributed to higher grades [37]. Furthermore, in a study of undergraduate research outcomes, more than half of students agreed that it was more important for their future career to develop their skills and abilities than to learn content knowledge related to their discipline [40]. This finding reveals that students understand the value of skill development for their future success, encouraging the incorporation of opportunities for research skill development into undergraduate and master’s curricula.
Research experiences have also been shown to increase student excitement for and interest in continuing in academia, either through further educational pursuits or career aspirations [3,37,39,41,42]. A minor difference was found between thesis and non-thesis undergraduate programs, with the research experiences of thesis programs confirming a desire to continue in academia and non-thesis programs sparking a new interest for research [37,42]. The reason for this difference may be related to the limited number of students who can complete an undergraduate thesis within a given program, so these opportunities are often given to students who are at the top of their class or those who express a pre-existing interest in research [37]. In contrast, non-thesis research opportunities are often incorporated into courses such that all the enrolled students are given the same opportunity for research skill development whether they have an interest in future research or not [37]. The specific teaching techniques targeted towards increasing students’ interest, attitudes, and perceived relevance include active learning, problem-based learning, cooperative learning, service learning, and general experiential learning [6]. These techniques can be used to increase exposure to research experiences early on during one’s postsecondary education, potentially contributing to increased enrolment in graduate studies at the master’s and doctoral levels.
Beyond careers in academia, studies have shown that research skills are also transferable across a wide range of careers, providing evidence of clinicians, businesspeople, and lawyers who have all benefitted from acquiring a strong research skillset [34,40,43,44,45]. Although certain skills from the previously mentioned research skill list (Table A1) are traditionally considered technical research skills (critical appraisal, information synthesis, data collection, and data analysis) and others are considered transferable skills (communication, decision making, and problem solving), they can all be applied across various fields and situations. For example, in clinical settings, evidence-based practice involves the use of current scientific research to make decisions and solve problems in the field [46,47]. Although the clinician may not be performing novel research and conducting their own studies, they still require the skills to critically appraise the literature and effectively synthesize information to provide the best care for their patients [46]. Therefore, research skills are important across a range of career pathways to stay informed and up to date on best practices in the field. Additionally, across disciplines, studies have shown that the skills desired by employers extend beyond the technical skills required for the job to encompass a range of transferable skills, such as communication and problem solving [34,40]. In a society where career changes are becoming more common, interdisciplinary research courses can provide opportunities for universal skills to be developed [48]. Therefore, the implementation of research opportunities in undergraduate and master’s education that are guided by a comprehensive yet flexible skill list can provide students with important technical and transferable skills proven to be beneficial far beyond the walls of universities.

7. How Can Research Skills Be Explicitly Addressed in Undergraduate and Master’s Curricula?

Various methods are suggested throughout this section for the incorporation of explicit, hands-on research skill development opportunities into undergraduate and master’s courses and programs. These methods are suggested by the authors based on a synthesis of the existing literature and can be used to adapt existing curricula or during the design and development of new courses and programs. The first suggestion is to create learning outcomes, teaching and learning activities, and assessments to target specific research skills, while emphasizing the significance of these skills for students’ current learning and future work [7,49]. For example, a backward design approach, in which learning outcomes are used as the starting point for course design, has previously been used for the effective execution of non-thesis undergraduate courses [50]. This practice encourages constructive alignment between course components and could lead to the benefits of a deeper understanding of the purpose of learning tasks and assessments, and an improved quality of feedback [37,38,49]. Various studies have shown that students who view research skills as being valuable for their future careers reap a range of benefits for their current education as well as their careers [24,25]. An additional step that faculty can take towards increasing the transparency of research skills’ importance is including the list of skills that students will develop within their course on their syllabus and course website. Explicitly stating the targeted skills within a course will increase students’ awareness of, ability to articulate, and perceived importance of their skillsets. The alignment of skills with course components and explicit statements of targeted skills will also ensure that skill development is focused and purposeful, such that all students within a course or research experience will have similar, targeted opportunities for research skill development.
A second suggestion is for faculty to assess students’ skill development at certain timepoints throughout their study using a specified skill list. Furthermore, students could self-rate their own skill development on an ongoing basis during their research project. This practice can assist students to better understand their strengths and areas requiring improvement. In addition, providing a visualization of students’ skill development throughout the duration of their research experience can help them to articulate their skillset and the opportunities that led to their skill development [37]. This ability would be useful for students to align future skill development opportunities, such as workshops, to their needs. In addition, a strong understanding of their skillset and skill development experiences is valuable for future job and school applications and interviews. Similarly, the “Attitudes Toward Research” scale, developed by Papanastasiou (2005) [51], could be used by educators to collect students’ feelings towards research. This information could then provide real-time feedback to educators to redirect potential misconceptions and assist students’ learning [51,52].
Third, to visualize research skill development on a program-wide level, curriculum mapping can be adapted to include research skills across program components. Curriculum mapping is used to illustrate the relationships and intersections between various program features and goals [53,54]. This process allows for connections among teachings, assessments, and outcomes to be visualized. It can be adapted to illustrate targeted skill development and enables comparisons to be made across program features and between programs. It is suggested that research teaching be incorporated earlier in a student’s program rather than waiting until upper years when research courses are commonly offered. As previously described, it is critical that students learn the relevant concepts related to research and develop their declarative knowledge for procedural knowledge to be learned and relevant skills to be developed [12,13,14,15,17]. Introducing research concepts earlier in a student’s program also holds value because a student’s understanding of the purpose of research for their future has been found to take time to develop [7]. Therefore, designing courses to challenge students’ misconceptions and introducing the purpose and importance of research early on in their education can influence conceptual change and skill development such that they will occur more readily [7]. Developing skills earlier on in students’ education may also provide an avenue for them to reach the epistemic level of knowledge development, thus enabling an understanding that research leads to the continual production of knowledge in society [15]. Explicit exposure to opportunities for research skill development also increases students’ familiarity with the research environment and may inspire a more positive orientation towards research, contributing to feelings of preparedness for future research [25]. Further, an explicit research skill list could be used to develop a validated instrument for assessing students’ research skill competencies, as instruments of this type are limited and variable.
The strategies presented in this section are a guide to curricular updates or future program development suggesting the incorporation of explicit research skill development opportunities in a time-effective manner and without great alterations to existing curricula. These strategies are meant to increase the transparency of research skills’ importance and the frequency with which students have an opportunity to explicitly practice their research skills. The deliberate practice of research methods was found to be the most important factor contributing to the development of research skills and knowledge [16]. By adding these opportunities to both thesis and non-thesis research experiences, students will reap the benefits for their current programs and future careers.

8. Conclusions and Implications

In summary, this viewpoint article provides an insight into the incorporation of research skill development opportunities in thesis and non-thesis undergraduate and master’s programs. Seven core research skills (critical appraisal, information synthesis, decision-making, problem solving, data collection, data analysis, and communication) that are developed during, and are important for, research experiences were described. This list includes technical skills that are traditionally associated with research experiences, but also considers transferable skills. By encompassing transferable skills, the definition of “research skills” is broadened to be inclusive of a wide variety of research experiences in both thesis and non-thesis programs. This definition is further supported by the finding that similar skills were reported across both thesis and non-thesis research experiences, despite the lacking literature directly comparing these program types. Taken together, these findings imply that similar skill outcomes can be achieved across different programs at the same degree level, with the differences being the program features and teaching and learning activities that are used to achieve development of those skills.
Furthermore, this study supports the incorporation of explicit research skill development opportunities into undergraduate and master’s programs. The first step in incorporating these explicit opportunities is to minimize misconceptions as students’ conceptions can have an impact on their learning, motivation, and career readiness. By incorporating discussions and activities around research skills, students can understand which skills are developed during and are important for research experiences, as well as the value of developing these skills. As previously described, the assumption that skills are naturally developed during research participation can lead to confusion and variability in skill development among students. Rather, ensuring that core research concepts are taught in conjunction with relevant and explicit skill development opportunities can provide a range of benefits for students. These benefits include a deeper understanding of the purpose of learning tasks, an increased ability for skill articulation, and even stronger performance in research courses. Furthermore, providing frequent opportunities for the deliberate practice of research skills will increase students’ familiarity with, and potential interest in, research. These factors can thereby contribute to students’ knowledge and skill development as well as feelings of preparedness for future research endeavours. By ensuring an alignment between faculty and students’ definitions of a “research skill”, students’ research skills can be effectively developed and applied across a wide range of future educational and career pathways. Broadening the definition of “research skills” allows program development to be streamlined to offer explicit research experiences across disciplines in both thesis and non-thesis programs that are beneficial and applicable to students’ futures.

Author Contributions

Conceptualization, K.V., K.A.R. and N.C.; methodology, K.V.; investigation, K.V.; writing—original draft preparation, K.V.; writing—review and editing, K.V., K.A.R. and N.C.; visualization, K.V.; supervision, K.A.R. and N.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table
Table A1. Sources describing research skills in undergraduate and/or master’s programs.
Table A1. Sources describing research skills in undergraduate and/or master’s programs.
SourceCategoryScope: Targeted, Perceived, or Assessed Research Skills?Degree Level and DisciplineResearch Skills
Boyer Commission, 1998 [55].BookSkills targeted by the Boyer CommissionUndergraduate education; all disciplinesInformation synthesis, decision making, problem solving, communication
Healey, M., & Jenkins, A. 2009 [29].Secondary articleSkills targeted through current examples of undergraduate research experiences across North America, the UK, and OceaniaUndergraduate education; STEM, Social Sciences, Arts and Humanities, Education, Business and Technology, Interdisciplinary Studies, Environmental Studies, Social WorkInformation synthesis, decision making, problem solving, data collection, data analysis, communication
Laidlaw, A., Aiton, J., Struthers, J., & Guild, S. 2012 [28].GuideSkills targeted for undergraduate medical educationUndergraduate education; medicineCritical appraisal, information synthesis, decision making, problem solving, data collection, data analysis, communication
Auchincloss, L. C., Laursen, S. L., Branchaw, J. L., Eagan, K., Graham, M., Hanauer, D. I., Lawrie, G., McLinn, C. M., Pelaez, N., Rowland, S., Towns, M., Trautmann, N. M., Varma-Nelson, P., Weston, T. J., & Dolan, E. L. 2014 [3].Meeting reportSkills targeted by the Course-Based Undergraduate Research NetworkNon-thesis undergraduate education; STEMDecision making, problem solving, data collection, data analysis, communication
Bandaranaike, S. 2018 [56].Secondary articleSkills targeted according to the Work Skill Development FrameworkUndergraduate and master’s education; discipline not specifiedInformation synthesis, problem solving, data collection, communication
Gonzalez, 2001 [57].Viewpoint articleSkills targeted for undergraduate and thesis master’s researchUndergraduate and thesis master’s education; discipline not specifiedProblem solving, communication
Canadian Association for Graduate Studies. 2012 [58].GuideSkills targeted by the Canadian Association for Graduate StudiesMaster’s education; all disciplinesInformation synthesis, problem solving, data analysis, communication,
Ontario Council on Graduate Studies. 2017 [5].GuideSkills targeted by the Council of Ontario UniversitiesMaster’s education; all disciplinesDecision making, problem solving, communication
Sewall, J. M., Oliver, A., Denaro, K., Chase, A. B., Weihe, C., Lay, M., Martiny, J. B. H., & Whiteson, K. 2020 [59].Primary articleSkills targeted by the learning outcomes of the course; skill perceptions of studentsNon-thesis undergraduate education; STEMData analysis, problem solving, communication
Seymour, E., Hunter, A., Laursen. S, & DeAntonio, T. 2004 [39].Primary articleSkill perceptions of studentsThesis undergraduate education; STEMProblem solving, data analysis, communication
Sabatini, D. A. 1997 [38].Primary articleSkill perceptions of students and alumniThesis undergraduate education; STEMProblem solving, communication
Crebert, G., Bates, M., Bell, B., Patrick, C., & Cragnolini, V. 2004 [40].Primary articleSkill perceptions of alumniUndergraduate education; STEM, Social Sciences, Arts and HumanitiesDecision making, problem solving, communication
Bauer, K. W., & Bennett, J. S. 2003 [41].Primary articleSkill perceptions of alumniThesis undergraduate education; STEM, Social Sciences, Arts and HumanitiesCritical appraisal, problem solving, data analysis, communication
Hunter, A., Laursen, S. L., & Seymour, E. 2007 [42].Primary articleSkill perceptions of faculty and studentsThesis undergraduate education; STEMDecision making, problem solving, data analysis, communication
Kardash, C. M. 2000 [32].Primary article Skill perceptions of faculty and studentsThesis undergraduate education; STEMInformation synthesis, data collection, data analysis, data collection, communication
Lopatto, D. 2003 [60].Primary articleSkill perceptions of faculty and studentsThesis undergraduate education; STEMDecision making, communication
Shostak, S., Girouard, J., Cunningham, D., Cadge, W. 2010 [27].Primary articleSkill perceptions of studentsNon-thesis undergraduate and master’s education; Social SciencesCritical appraisal, decision making, problem solving, data collection, data analysis
Willison, J.W. 2012 [37].Primary articleSkill perceptions of faculty members and studentsNon-thesis undergraduate and master’s education; STEM, Arts and Humanities, BusinessCritical appraisal, information synthesis, communication
Bussell, H., Hagman, J., & Guder, C. S. 2017 [61].Primary articleSkill perceptions of studentsMaster’s education; STEM, Social Sciences, Arts and Humanities, Business, Education, Environmental StudiesData analysis
Anderson, S. G. 2003 [26].Primary articleSkill perceptions of studentsNon-thesis master’s education; Social WorkDecision making, problem solving, data collection, data analysis, decision making, problem solving
Wagner, H. H., Murphy, M. A., Holderegger, R., & Waits, L. 2012 [33].Primary articleSkill perceptions of faculty members and studentsNon-thesis master’s education; STEMCritical appraisal, problem solving, data analysis, communication
Feldon, D. F., Maher, M. A., Hurst, M., & Timmerman, B. 2014 [62].Primary articleSkill perceptions of faculty and studentsThesis master’s education; STEMProblem solving, data analysis
Hart, J. 2019 [34].Secondary articleSystematic search and review including studies of both perceived and assessed research skillsNon-thesis undergraduate education; STEMProblem solving, communication
Malotky, M. K. H., Mayes, K. M., Price, K. M., Smith, G., Mann, S. N., Guinyard, M. W., Veale, S., Ksor, V., Siu, L., Mlo, H., Nsonwu, M. B., Morrison, S. D., Sudha, S., & Bernot, K. M. 2020 [63].Primary articleSkill perceptions of students; skill assessment through pre- and post-examsNon-thesis undergraduate education; STEM, Social Sciences, Arts and Humanities, Business and Technology, Environmental Studies, Social WorkData analysis, communication
Gilmore, J., Vieyra, M., Timmerman, B., Feldon, D., & Maher, M. 2015 [36].Primary articleSkill perceptions of graduate students regarding their undergraduate research experiences; skill assessment through analysis of graduate students’ research proposalsUndergraduate and master’s education; STEMDecision making, problem solving, data collection, data analysis, communication
Si, J. 2020 [64]Primary articleSkill assessment of students’ research reports using a research skill rubricNon-thesis undergraduate education; STEMCritical appraisal, communication
Moni, R. W., Hryciw, D. H., Poronnik, P., & Moni, K. B. 2007 [35].Primary articleSkill assessment through undergraduate opinion editorial writing assignmentNon-thesis undergraduate education; STEMCommunication
Feldon, D.F., Peugh, J., Timmerman, B.E., Maher, M.A., Hurst, M., Strickland, D., Gilmore, J.A., & Stiegelmeyer, C. 2011 [65].Primary articleSkill assessment comparing written research proposals of students with and without teaching responsibilitiesThesis master’s education; STEMData analysis, communication
Timmerman, B. C., Feldon, D., Maher, M., Strickland, D., & Gilmore, J. 2013 [66].Primary articleSkill assessment through master’s students’ written research proposalsThesis master’s education; STEMInformation synthesis, data analysis, communication

References

  1. Wuetherick, B. Transforming undergraduate research at Canadian universities. In International Perspectives on Undergraduate Research, 1st ed.; Hensel, N.H., Blessinger, P., Eds.; Palgrave Macmillan: Cham, Switzerland, 2020; pp. 265–280. [Google Scholar]
  2. Vajoczki, S. From the International desk: A Canadian perspective on undergraduate research. CUR Q. 2010, 31, 41–47. [Google Scholar]
  3. Auchincloss, L.C.; Laursen, S.L.; Branchaw, J.L.; Eagan, K.; Graham, M.; Hanauer, D.I.; Lawrie, G.; McLinn, C.M.; Pelaez, N.; Rowland, S.; et al. Assessment of course-based undergraduate research experiences: A meeting report. CBE Life Sci. Educ. 2014, 13, 29–40. [Google Scholar] [CrossRef] [PubMed]
  4. Licht, F.; Milán, G.A.; Andreas, H.A. Bringing real-world energy-storage research into a second-year physical-chemistry lab using a MnO2-based supercapacitor. J. Chem. Educ. 2018, 95, 2028–2033. [Google Scholar] [CrossRef]
  5. Ontario Council on Graduate Studies. Principles for Graduate Study at Ontario’s Universities; Council of Ontario Universities: Toronto, ON, Canada, 2017; pp. 1–11. [Google Scholar]
  6. Earley, M.A. A synthesis of the literature on research methods education. Teach. High. Educ. 2013, 19, 242–253. [Google Scholar] [CrossRef]
  7. Balloo, K.; Pauli, R.; Worrell, M. Conceptions of research methods learning among psychology undergraduates: A Q methodology study. Cogn. Instr. 2018, 36, 279–296. [Google Scholar] [CrossRef]
  8. Meyer, J.H.F.; Shanahan, M.P.; Laugksch, R.C. Students’ conceptions of research. I: A qualitative and quantitative analysis. Scand. J. Educ. Res. 2005, 49, 225–244. [Google Scholar] [CrossRef]
  9. Murtonen, M. University students’ understanding of the concepts empirical, theoretical, qualitative and quantitative research. Teach. High. Educ. 2015, 20, 684–698. [Google Scholar] [CrossRef]
  10. Rossum, E.J.; Schenk, S.M. The relationship between learning conception, study strategy and learning outcome. Br. J. Educ. Psychol. 1984, 54, 73–83. [Google Scholar] [CrossRef]
  11. Vermunt, J.D.; Vermetten, Y.J. Patterns in student learning: Relationships between learning strategies, conceptions of learning, and learning orientations. Educ. Psychol. Rev. 2004, 16, 359–384. [Google Scholar] [CrossRef]
  12. Davis, F.D.; Yi, M.Y. Improving computer skill training: Behavior modeling, symbolic mental rehearsal, and the role of knowledge structures. J. Appl. Psychol. 2004, 89, 509–523. [Google Scholar] [CrossRef]
  13. Gagné, R.M. Learning outcomes and their effects: Useful categories of human performance. Am. Psychol. 1984, 39, 377–385. [Google Scholar] [CrossRef]
  14. Rickards, J.P.; Fajen, B.R.; Sullivan, J.F.; Gillespie, G. Signaling, notetaking, and field independence–dependence in text comprehension and recall. J. Educ. Psychol. 1997, 89, 508–517. [Google Scholar] [CrossRef]
  15. Salmento, H.; Murtonen, M. The roles of epistemic understanding and research skills in students’ views of scientific thinking. In Redefining Scientific Thinking for Higher Education, Higher-Order Thinking, Evidence-Based Reasoning and Research Skills; Murtonen, M., Balloo, K., Eds.; Palgrave Macmillan: Cham, Switzerland, 2019; pp. 31–57. [Google Scholar] [CrossRef]
  16. Balloo, K.; Pauli, R.; Worrell, M. Individual differences in psychology undergraduates’ development of research methods knowledge and skills. Procedia-Soc. Behav. Sci. 2016, 217, 790–800. [Google Scholar] [CrossRef]
  17. Abu-Zaid, A.; Khan, T.A. Assessing declarative and procedural knowledge using multiple-choice questions. Med. Educ. Online 2013, 18, 21132. [Google Scholar] [CrossRef]
  18. Salmento, H.; Murtonen, M.; Kiley, M. Understanding teacher education students’ research competence through their conceptions of theory. Front. Educ. 2021, 6, 763803. [Google Scholar] [CrossRef]
  19. Kawulich, B.; Garner, M.W.J.; Wagner, C. Students’ conceptions-and misconceptions-of social research. Qual. Sociol. Rev. 2009, 5, 5–25. [Google Scholar] [CrossRef]
  20. Vermunt, J.D. Metacognitive, cognitive and affective aspects of learning styles and strategies: A phenomenographic analysis. High. Educ. 1996, 31, 25–50. [Google Scholar] [CrossRef]
  21. Entwistle, N.J.; Ramsden, P. Understanding Student Learning; Croom Helm: London, UK, 1983. [Google Scholar]
  22. Marton, F.; Säljö, R. On qualitative differences in learning: I. Process and outcome. Br. J. Educ. Psychol. 1976, 46, 4–11. [Google Scholar] [CrossRef]
  23. Lonka, K.; Lindblom-Ylänne, S. Epistemologies, conceptions of learning, and study practices in medicine and psychology. High. Educ. 1996, 31, 5–24. [Google Scholar] [CrossRef]
  24. Murtonen, M.; Olkinuora, E.; Tynjala, P.; Lehtinen, E. “Do I need research skills in working life?”: University students’ motivation and difficulties in quantitative methods courses. High. Educ. 2008, 56, 599–612. [Google Scholar] [CrossRef]
  25. Shaw, K.; Holbrook, A.; Bourke, S. Student experience of final-year undergraduate research projects: An exploration of ‘research preparedness’. Stud. High. Educ. 2013, 38, 711–727. [Google Scholar] [CrossRef]
  26. Anderson, S.G. Engaging students in community-based research: A model for teaching social work research. J. Community Pract. 2003, 10, 71–87. [Google Scholar] [CrossRef]
  27. Shostak, S.; Girouard, J.; Cunningham, D.; Cadge, W. Teaching graduate and undergraduate research methods: A multipronged departmental initiative. Teach. Sociol. 2010, 38, 93–105. [Google Scholar] [CrossRef]
  28. Laidlaw, A.; Aiton, J.; Struthers, J.; Guild, S. Developing research skills in medical students: AMEE Guide No. 69. Med. Teach. 2012, 34, 754–771. [Google Scholar] [CrossRef] [PubMed]
  29. Healey, M.; Jenkins, A. Developing Undergraduate Research and Inquiry; The Higher Education Academy: York, UK, 2009; pp. 1–152. [Google Scholar]
  30. Edwards, A.; Jones, S.M.; Wapstra, E.; Richardson, A.M.M. Engaging students through authentic research experiences. In Proceedings of the Australian Conference on Science and Mathematics Education, Melbourne, VIC, Australia, 27–29 September 2007. [Google Scholar]
  31. Boulton-Lewis, G.M.; Smith, D.J.H.; McCrindle, A.R.; Burnett, P.C.; Campbell, K.J. Secondary teachers’ conceptions of teaching and learning. Learn. Instr. 2001, 11, 35–51. [Google Scholar] [CrossRef]
  32. Kardash, C.M. Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. J. Educ. Psychol 2000, 92, 191–201. [Google Scholar] [CrossRef]
  33. Wagner, H.H.; Murphy, M.A.; Holderegger, R.; Waits, L. Developing an interdisciplinary, distributed graduate course for twenty-first century scientists. Bioscience 2012, 62, 182–188. [Google Scholar] [CrossRef]
  34. Hart, J. Interdisciplinary project-based learning as a means of developing employability skills in undergraduate science degree programs. J. Teach. Learn. Grad. Employab. 2019, 10, 50–66. [Google Scholar] [CrossRef]
  35. Moni, R.W.; Hryciw, D.H.; Poronnik, P.; Moni, K.B. Using explicit teaching to improve how bioscience students write to the lay public. Adv. Physiol. Educ. 2007, 31, 167–175. [Google Scholar] [CrossRef]
  36. Gilmore, J.; Vieyra, M.; Timmerman, B.; Feldon, D.; Maher, M. The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. J. High. Educ. 2015, 86, 834–863. [Google Scholar] [CrossRef]
  37. Willison, J. When academics integrate research skill development in the curriculum. High Educ. Res. Dev. 2012, 31, 905–919. [Google Scholar] [CrossRef]
  38. Sabatini, D.A. Teaching and research synergism: The undergraduate research experience. J. Prof. Issues Eng. Educ. Pract. 1997, 123, 98–102. [Google Scholar] [CrossRef]
  39. Seymour, E.; Hunter, A.; Laursen, S.L.; DeAntoni, T. Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Sci. Educ. 2004, 88, 493–534. [Google Scholar] [CrossRef]
  40. Crebert, G.; Bates, M.; Bell, B.; Patrick, C.; Cragnolini, V. Developing generic skills at university, during work placement and in employment: Graduates’ perceptions. High. Educ. Res. Dev. 2004, 23, 147–165. [Google Scholar] [CrossRef]
  41. Bauer, K.W.; Bennett, J.S. Alumni perceptions used to assess undergraduate research experience. J. High. Educ. 2003, 74, 210–230. [Google Scholar] [CrossRef]
  42. Hunter, A.; Laursen, S.L.; Seymour, E. Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Sci. Educ. 2007, 91, 36–74. [Google Scholar] [CrossRef]
  43. Lai, W.W.; Vetter, V.L.; Richmond, M.; Li, J.S.; Saul, J.P.; Mital, S.; Colan, S.D.; Newburger, J.W.; Sleeper, L.A.; McCrindle, B.W.; et al. Clinical research careers: Reports from a NHLBI pediatric heart network clinical research skills development conference. Am. Heart J. 2011, 161, 13–67. [Google Scholar] [CrossRef]
  44. Klusek, L.; Bornstein, J. Information literacy skills for business careers. J. Bus. Finance Libr. 2006, 11, 3–21. [Google Scholar] [CrossRef]
  45. Hutchinson, T. Developing legal research skills: Expanding the paradigm. Melb. Univ. Law Rev. 2008, 32, 1065–1095. [Google Scholar]
  46. Willison, J.; Zhu, X.; Xie, B.; Yu, X.; Chen, J.; Zhang, D.; Shashoug, I.; Sabir, F. Graduates’ affective transfer of research skills and evidence based practice from university to employment in clinics. BMC Med. Educ. 2020, 20, 89–116. [Google Scholar] [CrossRef]
  47. Thomas, A.; Saroyan, A.; Dauphinee, W.D. Evidence-based practice: A review of theoretical assumptions and effectiveness of teaching and assessment interventions in health professions. Adv. Health Sci. Educ. 2011, 16, 253–276. [Google Scholar] [CrossRef] [PubMed]
  48. Wagner, C.; Garner, M.; Kawulich, B. The state of the art of teaching research methods in the social sciences: Towards a pedagogical culture. Stud. High. Educ. 2011, 36, 75–88. [Google Scholar] [CrossRef]
  49. Biggs, J. Enhancing teaching through constructive alignment. High. Educ. 1996, 32, 347–364. [Google Scholar] [CrossRef]
  50. Hills, M.; Harcombe, K.; Bernstein, N. Using anticipated learning outcomes for backward design of a molecular cell biology Course-based Undergraduate Research Experience. Biochem. Mol. Biol. Educ. 2020, 48, 311–319. [Google Scholar] [CrossRef]
  51. Papanastasiou, E.C. Factor structure of the “Attitudes toward Research” Scale. Stat. Educ. Res. J. 2005, 4, 16–26. [Google Scholar] [CrossRef]
  52. Walker, D.A. A confirmatory factor analysis of the Attitudes Toward Research Scale. Mult. Linear Regres. Viewp. 2010, 6, 18–27. [Google Scholar]
  53. Levin, P.F.; Suhayda, R. Transitioning to the DNP: Ensuring integrity of the curriculum through curriculum mapping. Nurse Educ. 2018, 43, 112–114. [Google Scholar] [CrossRef]
  54. Kopera-Frye, K.; Mahaffy, J.; Svare, G.M. The map to curriculum alignment and improvement. In Collected Essays on Learning and Teaching, 4th ed.; Raffoul, J., Ed.; Society for Teaching and Learning in Higher Education: Hamilton, ON, Canada, 2011; pp. 9–15. [Google Scholar]
  55. Boyer Commission. Reinventing Undergraduate Education: A Blueprint for America’s Research Universities; Stony Brook Press: New York, NY, USA, 1998. [Google Scholar]
  56. Bandaranaike, S. From research skill development to work skill development. J. Univ. Teach. Learn. Pract. 2018, 15, 108–126. [Google Scholar] [CrossRef]
  57. Gonzalez, C. Undergraduate research, graduate mentoring, and the university’s mission. Science 2001, 293, 1624. [Google Scholar] [CrossRef]
  58. Boland, S. Graduate Studies: A Practical Guide; Canadian Association for Graduate Studies: Ottawa, ON, Canada, 2012; pp. 1–19. [Google Scholar]
  59. Sewall, J.M.; Oliver, A.; Denaro, K.; Chase, A.B.; Weihe, C.; Lay, M.; Martiny, J.B.H.; Whiteson, K. Fiber force: A fiber diet intervention in an advanced course-based undergraduate research experience (CURE) course. J. Microbiol. Biol. Educ. 2020, 21, 110. [Google Scholar] [CrossRef]
  60. Lopatto, D. The essential features of undergraduate research. CUR Q. 2003, 24, 139–142. [Google Scholar]
  61. Bussell, H.; Hagman, J.; Guder, C.S. Research needs and learning format preferences of graduate students at a large public university: An exploratory study. Coll. Res. Libr. 2017, 78, 978–998. [Google Scholar] [CrossRef]
  62. Feldon, D.F.; Maher, M.A.; Hurst, M.; Timmerman, B. Faculty mentors’, graduate students’, and performance-based assessments of students’ research skill development. Am. Educ. Res. J. 2014, 52, 334–370. [Google Scholar] [CrossRef] [Green Version]
  63. Malotky, M.K.H.; Mayes, K.M.; Price, K.M.; Smith, G.; Mann, S.N.; Guinyard, M.W.; Veale, S.; Ksor, V.; Siu, L.; Mlo, H.; et al. Fostering inclusion through an interinstitutional, community-engaged, course-based undergraduate research experience. J. Microbiol. Biol. Educ. 2020, 21, 11. [Google Scholar] [CrossRef] [PubMed]
  64. Si, J. Course-based research experience of undergraduate medical students through project-based learning. Korean J. Med. Educ. 2020, 32, 47–57. [Google Scholar] [CrossRef] [PubMed]
  65. Feldon, D.F.; Peugh, J.; Timmerman, B.E.; Maher, M.A.; Hurst, M.; Strickland, D.; Gilmore, J.A.; Stiegelmeyer, C. Graduate students’ teaching experiences improve their methodological research skills. Science 2011, 333, 1037–1039. [Google Scholar] [CrossRef] [PubMed]
  66. Timmerman, B.C.; Feldon, D.; Maher, M.; Strickland, D.; Gilmore, J. Performance-based assessment of graduate student research skills: Timing, trajectory, and potential thresholds. Stud. High. Educ. 2013, 38, 693–710. [Google Scholar] [CrossRef]
Education 12 00642 g001 550
Figure 1. Overview of the process used for the literature search and environmental scan.
Figure 1. Overview of the process used for the literature search and environmental scan.
Education 12 00642 g001
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Vieno, K.; Rogers, K.A.; Campbell, N. Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs. Educ. Sci. 2022, 12, 642. https://doi.org/10.3390/educsci12100642

AMA Style

Vieno K, Rogers KA, Campbell N. Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs. Education Sciences. 2022; 12(10):642. https://doi.org/10.3390/educsci12100642

Chicago/Turabian Style

Vieno, Kayla, Kem A. Rogers, and Nicole Campbell. 2022. "Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs" Education Sciences 12, no. 10: 642. https://doi.org/10.3390/educsci12100642

APA Style

Vieno, K., Rogers, K. A., & Campbell, N. (2022). Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs. Education Sciences, 12(10), 642. https://doi.org/10.3390/educsci12100642

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop