Broadening the Deﬁnition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs

: Undergraduate and master’s programs—thesis- or non-thesis-based—provide students with opportunities to develop research skills that vary depending on their degree requirements. However, there is a lack of clarity and consistency regarding the deﬁnition of a research skill and the components that are taught, practiced, and assessed. In response to this ambiguity, an environmental scan and a literature search were conducted to inform the creation of a comprehensive list of research skills that can be applied across programs and disciplines. Although published studies directly comparing research skills in thesis and non-thesis programs were limited, the speciﬁc skills reported in each program type were similar. This viewpoint article identiﬁes the following seven research skills that were most frequently reported across both thesis and non-thesis programs: critical appraisal, information synthesis, decision making, problem solving, data collection, data analysis, and communication. When contextualized appropriately, these skills can be useful for a student during their academic program and are transferable across a range of future career pathways. Broadening the deﬁnition of “research skills” can inform curricular updates and program development, independent of their program type, to ensure that students are presented with explicit opportunities to develop the skills needed to succeed in their educational and occupational endeavours.


Introduction
A common assumption across academia is that students develop research skills solely through the practice of hands-on research, such as performing experiments in a laboratory. However, the reality is that undergraduate and master's students can develop their research skills through an array of experiences in thesis-and non-thesis-based programs. While pursuing a thesis and its related experiences, students complete various tasks related to a focused research project under the supervision of a faculty member [1,2]. This research occurs during the final year of undergraduate programs or throughout a graduate degree, with both experiences culminating in the writing and defense of the thesis [2]. A major difference between the thesis-based experiences of undergraduate and master's programs is the level of depth and autonomy in which the student executes their project. In contrast, non-thesis research experiences can take a variety of formats. For example, undergraduate non-thesis research can be a laboratory-based research course, a literature review course, or inquiry-based assessment [1,3,4]. At the master's level, non-thesis research experiences are generally incorporated into course-based programs such that the research can be executed as a major paper, or through coursework, activities, and assignments [5]. Despite the differences in the degree requirements between thesis and non-thesis programs, both provide students with distinct opportunities for research skill development to take place at the undergraduate and graduate levels.

Statement of the Problem
Across the literature, there is a lack of clarity and consistency surrounding the definition of research skills and which specific skills should be taught, practiced, and assessed at the undergraduate and master's levels in both thesis and non-thesis programs. In addition, many research experiences involve the implicit development of research skills or assume that skills are naturally being developed simply by participation in research. However, this approach can lead to ineffective opportunities for skill development and student confusion as they may not be aware of the specific skills that they are developing or the experiences that contribute to said skill development. In fact, previous studies have found that many students who participate in research experiences struggle to understand the relevance of their education to their future aspirations [6,7]. These students often hold misconceptions about research that can hinder their learning not only at the beginning of a research experience but upon completion as well [6,8,9]. The lack of a clear definition of research skills and limited opportunities for explicit skill development can also contribute to greater variability in outcomes across programs and even across students within a program. The validated instruments for measuring development of research skills are limited and the skills included in each one are variable, as a standardized list of research skills does not exist in the literature. As such, misconceptions surrounding research skill development can be present in academia, which can hinder students' success in their programs and beyond.

Theoretical Framework
Van Rossum and Schenk (1984) [10] state that learning is influenced by students' conceptions of their studies. Extending this to research learning, these conceptions encompass the knowledge and beliefs that students have about research and are influenced by the research environments and experiences that they are involved in [7,11]. Focusing on knowledge development in relation to concept formation, students must learn procedural knowledge for research skill development to take place [12][13][14][15][16]. Thus, the theoretical framework for this work is informed by learning conceptions and procedural knowledge, which are applicable to research skill development in undergraduate and master's programs.
Procedural knowledge is the understanding and ability to execute tasks applicable to a given domain [12][13][14][15]17]. Declarative knowledge, which is an understanding of relevant core facts, is crucial for procedural knowledge to develop [15,17]. Taken together, declarative and procedural knowledge contribute to structural knowledge, which is an understanding of the relationships between ideas within a domain [12]. Students may also form conceptions at the epistemic level, which describes the ability to understand the nature and sources of knowledge [15]. Despite the importance of procedural knowledge for skill development, previous research has revealed that students may struggle to form conceptions at this level. For example, a study by Salmento et al. (2021) [18] discovered that only 18.4% and 7.4% of students' conceptions of theory were at the procedural and epistemic levels, respectively. Therefore, for students to develop their research skills, educators must also ensure that effective methods are implemented for the learning of basic research knowledge.
A study by Balloo et al. (2016) [16] explored the factors that contribute to the development of research methods-related knowledge and research skill development in undergraduate students. Their findings revealed that self-regulation and motivation were related to a higher degree of structural knowledge, implicating that these factors likely influence the development of research proficiency. Research self-efficacy and research interest also showed a significant positive correlation with knowledge, but the largest predictor of research knowledge and skills was the deliberate practicing of research methods. Many of these contributing factors are dependent on the individual student, yet the practice of research methods is a component that can be incorporated into the curriculum by educators to increase students' knowledge and skill gains [16].
Students' conceptions have been found to impact their approaches to learning, learning orientations, outcomes, and future skill applications [10,[19][20][21][22][23]. Further, their conceptions of the importance of research for their future work can affect career readiness [24]. Yet, Murtonen et al. (2008) [24] discovered that about half of the students that participated in their study on research skill views were not convinced that research and statistics skills would be useful for their future work. If students do not understand the value of research for their futures, they may struggle with motivation when completing research tasks, which is a factor previously described to be correlated with the development of research proficiency [16,24]. Murtonen et al. (2008) [24] also found that students who valued research skills for their future work were more task-oriented, had low ego-defensiveness, used a deep approach to learning, and experienced fewer struggles in research methodology courses than students who did not see research skills as important for their careers. A different study by Shaw et al. (2013) [25] described that research-related self-efficacy, motivation, familiarity with the research environment, and a positive orientation towards research were key predictors that contribute to undergraduate students' feelings of preparedness for future research.
Taken together, a variety of factors can influence the conceptions that students hold about research, thus affecting their understanding of procedural knowledge and development of research skills. By considering the factors that influence students' conceptions and the effect that these conceptions can have on learning, educators can adapt their courses and programs for effective research teaching. Therefore, this theoretical perspective is appropriate for interpreting the literature on research skills in thesis versus non-thesis undergraduate and master's programs.

Context and Methodology
In response to the ambiguity surrounding research skill development in undergraduate and master's programs, an environmental scan and a literature search were conducted to inform the creation of a comprehensive list of research skills that can be applied across disciplines and programs ( Figure 1). The scan was performed using the Journal Storage and Google Scholar databases. A variety of sources were included in the results, such as primary journal articles, book chapters, and meeting reports. The included sources were grouped into three categories based on the approach used to discuss research skills in undergraduate and master's programs: targeted, perceived, or assessed research skills. Each of these categories presented a different approach to defining skill development. When combined, a comprehensive research skill list was created that considered the various perspectives and research findings of faculty, students, and alumni. The term "faculty" is used throughout this article to include the roles of professors, instructors, and research supervisors.
The three research skill categories used to guide the environmental scan and literature search can be defined as follows: • Targeted-research skills that faculty explicitly stated as a goal of their courses or programs, or that groups of experts considered to be important during their research experiences; • Perceived-research skills that students or faculty believed were developed during the course or program experience; • Assessed-research skills determined to have been developed during one's program using objective measurements, such as assessments guided by in-depth rubrics.
Each of these categories made an important contribution to the creation of a comprehensive list for broadening the definition of research skills. The three research skill categories used to guide the environmental scan and literature search can be defined as follows: • Targeted-research skills that faculty explicitly stated as a goal of their courses or programs, or that groups of experts considered to be important during their research experiences; • Perceived-research skills that students or faculty believed were developed during the course or program experience; • Assessed-research skills determined to have been developed during one's program using objective measurements, such as assessments guided by in-depth rubrics.
Each of these categories made an important contribution to the creation of a comprehensive list for broadening the definition of research skills.
The sources that described research skills in general, without detailing specific skills, were excluded from the environmental scan. To account for differences in terminology across the sources, certain skills were grouped together. For example, both oral and written communication were collectively considered to be 'communication' skills. Similarly, the skill of critical thinking was grouped under the 'problem solving' category. The studies included in this review were from North America, the United Kingdom, and Australasia. Most papers focused on skill development in STEM or allied health science disciplines. The two exceptions were concerning research courses in a Master of Social Work program [26] and a paper about teaching research methods to undergraduate and graduate Sociology students [27]. Many of the STEM and allied health science papers also included other disciplines in their studies, such as social sciences, arts and humanities, and The sources that described research skills in general, without detailing specific skills, were excluded from the environmental scan. To account for differences in terminology across the sources, certain skills were grouped together. For example, both oral and written communication were collectively considered to be 'communication' skills. Similarly, the skill of critical thinking was grouped under the 'problem solving' category. The studies included in this review were from North America, the United Kingdom, and Australasia. Most papers focused on skill development in STEM or allied health science disciplines. The two exceptions were concerning research courses in a Master of Social Work program [26] and a paper about teaching research methods to undergraduate and graduate Sociology students [27]. Many of the STEM and allied health science papers also included other disciplines in their studies, such as social sciences, arts and humanities, and business and technology. Although most of the sources included in the environmental scan were based on STEM and allied health science degrees, it is suggested by the authors that these skills can be broadly applied across disciplines for incorporation into thesis and non-thesis research experiences at the undergraduate and master's levels.

Seven Core Research Skills Transferable across Disciplines and Degrees
Research skills are defined as actions or tasks that can be taught, practiced, and performed to establish facts, postulate new ideas, test ideas to collect data, and analyze data to draw conclusions [28]. This definition highlights that research skills encompass more than just hands-on technical skills. This was also supported by the findings of this study's environmental scan and literature search, which revealed seven research skills that were most frequently reported across both thesis and non-thesis programs at the undergraduate and master's levels: • Critical appraisal-evaluating the methods, data, and conclusions of published research to determine its validity and reliability; • Information synthesis-combining information from various sources in a logical manner to draw conclusions; • Decision making-selecting and executing a specific course of action; • Problem solving-identifying sources of difficulty and finding reasonable and effective solutions to them; • Data collection-gathering information using structured methods to support the objectives of the study; • Data analysis-manipulating and modelling data to reveal trends and correlations to make conclusions related to a set of study objectives; • Communication-the sharing of information with others through either written or verbal means.
The results from this study's environmental scan and literature search are presented in Table A1. The studies that directly compared thesis and non-thesis experiences were limited, such that most of the papers included in this literature search were focused on thesis or non-thesis research in specific programs or at individual institutions. These sources revealed many similarities between research skills in both thesis and non-thesis programs; thus, all the skills included in the research skill list were observed across both program types. As previously mentioned, these seven research skills were either targeted, perceived, or assessed across studies of research experiences in undergraduate and master's programs. Therefore, including sources from all three of these categories within the results of the scan creates a definition of "research skills" that is current and comprehensive. Sources that are based on targeted research skill development represent the ideas and opinions of experts in the field regarding the goals and outcomes for students' skill development. Perceived skill development portrays the lived experiences of students and faculty members within undergraduate and master's level research experiences, illuminating their perspectives on skill development within these courses and programs. Finally, assessed skill development provides an objective measurement of students' skill competencies, offering concrete evidence as to the efficacy of various research skill development opportunities. Each of these three categories of sources views research skills through a unique lens, but collectively, they comprise a thorough definition of "research skills" across undergraduate and master's programs. The remainder of this section will provide specific examples of sources describing targeted, perceived, and assessed research skills across undergraduate and master's programs.
Targeted skills were most frequently reported in sources such as secondary articles, guides, and meeting reports. Certain sources were based on established programs or courses that explicitly stated specific research skill development as a goal. For example, a review by Healey and Jenkins (2009) provided a guide for targeted research skill development by providing many examples of undergraduate courses across the United Kingdom, North America, and Australasia that have incorporated explicit research skill development opportunities into their curricula. Examples include a course where students were presented with experimental data and guided through the process of data analysis and presentation [29,30], and another where students practiced collecting data from their classmates to explore students' quality of life [29]. Furthermore, other sources described the skills that educational organizations or associations deemed important for research experiences. One example is the Ontario Council on Graduate Studies' 'Principles for Graduate Study at Ontario Universities', which described the skills and abilities that students should demonstrate upon receiving their master's degree in both thesis and non-thesis programs [5]. The guide described that successful master's students should demonstrate communication skills, the ability to apply a knowledge base to solve a problem, and autonomy, including within decision making, independent of the type of program that they are enrolled in. Taken together, these studies reveal that skills can be targeted at multiple levels across academia, from course activities and expectations to the defining characteristics of a degree type.
Although sources about targeted research skills are focused on the opinions of experts in the field, studies of perceived research skill development aim to understand student and faculty perspectives and lived experiences involving these research opportunities. An understanding of this category is important, as the conceptions that students hold can affect their skill utilization in the future [19,24]. Additionally, it has been found that educators' conceptions of students' capacity for learning align with their teaching conceptions [31], suggesting that the research skills that educators perceive students to be developing may be the skills that they value and emphasize in their teaching. An example of a perceived research skill development study compared student self-ratings before and after a thesisbased undergraduate research experience and found significant improvements in oral communication, written communication, data collection, and data analysis skills, as well as an effective usage of primary scientific literature [32]. A separate study of an interdisciplinary graduate course collected both student and faculty perspectives on the course's first offering through surveys and an end-of-course meeting [33]. When comparing student self-evaluations and faculty ratings at the beginning and end of the course, significant improvements were reported in students' communication, critical appraisal, and data analysis skills. The collection of student and faculty perspectives throughout the duration of a course or program can provide a greater understanding of students' skill development based on feedback from those directly involved in the research experiences.
The sources describing assessed skill development were the least common across the literature. This finding was likely because certain skills can be difficult to assess objectively, and validated instruments do not exist to measure all research skills [34]. In one study of assessed skill development involving undergraduate students, a writing assignment was completed at the beginning and end of a course during which students were taught strategies for writing to the lay public [35]. Their assignments from both timepoints were then compared and they received grades from faculty as well as members of the public. Most students received higher grades for their paper written after the course, suggesting that the explicit teaching of writing skills had a positive effect on that skill development. In a similar study, thesis graduate students wrote a research proposal at the beginning (preproposal) and end (post-proposal) of their first year [36]. These proposals were then graded using a validated rubric and comparisons were made between the grades of students who had participated in undergraduate research and those who had not. Students who had been involved in undergraduate research received higher grades on both their pre-proposals and post-proposals in all categories except 'testable hypothesis' and 'introduction and context.' Although less common than perceived skill measurements, the methods for determining students' assessed skill levels can be incorporated into courses and programs to determine the efficacy of research skill development opportunities. The seven skills previously described were those reported across studies of targeted, perceived, and assessed skill development (Table A1), thereby suggesting that they should be considered by faculty when creating research skill development opportunities in both thesis and non-thesis undergraduate and master's curricula.

The Importance of Research Skill Development in Academia and Beyond
The development of research skills during both undergraduate and master's programs offers a range of benefits for faculty and students. The positive effects for faculty include a narrowed divide between their teaching and research and a clarification of their teaching methods [37]. Faculty also reported that incorporating explicit research skill development into their courses supported their own research endeavours and led to changes in their understanding of their academic discipline [37]. These positive effects on faculty influenced students as they described an increased clarity of the purpose of learning tasks and assessments, a deeper understanding for their discipline, and an increased ability to think as a scientist would [3,[37][38][39]. Students also reported that courses with explicit research skill development opportunities were characterized by an improved quality of feedback from their faculty and even contributed to higher grades [37]. Furthermore, in a study of undergraduate research outcomes, more than half of students agreed that it was more important for their future career to develop their skills and abilities than to learn content knowledge related to their discipline [40]. This finding reveals that students understand the value of skill development for their future success, encouraging the incorporation of opportunities for research skill development into undergraduate and master's curricula.
Research experiences have also been shown to increase student excitement for and interest in continuing in academia, either through further educational pursuits or career aspirations [3,37,39,41,42]. A minor difference was found between thesis and non-thesis undergraduate programs, with the research experiences of thesis programs confirming a desire to continue in academia and non-thesis programs sparking a new interest for research [37,42]. The reason for this difference may be related to the limited number of students who can complete an undergraduate thesis within a given program, so these opportunities are often given to students who are at the top of their class or those who express a pre-existing interest in research [37]. In contrast, non-thesis research opportunities are often incorporated into courses such that all the enrolled students are given the same opportunity for research skill development whether they have an interest in future research or not [37]. The specific teaching techniques targeted towards increasing students' interest, attitudes, and perceived relevance include active learning, problem-based learning, cooperative learning, service learning, and general experiential learning [6]. These techniques can be used to increase exposure to research experiences early on during one's postsecondary education, potentially contributing to increased enrolment in graduate studies at the master's and doctoral levels.
Beyond careers in academia, studies have shown that research skills are also transferable across a wide range of careers, providing evidence of clinicians, businesspeople, and lawyers who have all benefitted from acquiring a strong research skillset [34,40,[43][44][45]. Although certain skills from the previously mentioned research skill list (Table A1) are traditionally considered technical research skills (critical appraisal, information synthesis, data collection, and data analysis) and others are considered transferable skills (communication, decision making, and problem solving), they can all be applied across various fields and situations. For example, in clinical settings, evidence-based practice involves the use of current scientific research to make decisions and solve problems in the field [46,47]. Although the clinician may not be performing novel research and conducting their own studies, they still require the skills to critically appraise the literature and effectively synthesize information to provide the best care for their patients [46]. Therefore, research skills are important across a range of career pathways to stay informed and up to date on best practices in the field. Additionally, across disciplines, studies have shown that the skills desired by employers extend beyond the technical skills required for the job to encompass a range of transferable skills, such as communication and problem solving [34,40]. In a society where career changes are becoming more common, interdisciplinary research courses can provide opportunities for universal skills to be developed [48]. Therefore, the implementation of research opportunities in undergraduate and master's education that are guided by a comprehensive yet flexible skill list can provide students with important technical and transferable skills proven to be beneficial far beyond the walls of universities.

How Can Research Skills Be Explicitly Addressed in Undergraduate and Master's Curricula?
Various methods are suggested throughout this section for the incorporation of explicit, hands-on research skill development opportunities into undergraduate and master's courses and programs. These methods are suggested by the authors based on a synthesis of the existing literature and can be used to adapt existing curricula or during the design and development of new courses and programs. The first suggestion is to create learning outcomes, teaching and learning activities, and assessments to target specific research skills, while emphasizing the significance of these skills for students' current learning and future work [7,49]. For example, a backward design approach, in which learning outcomes are used as the starting point for course design, has previously been used for the effective execution of non-thesis undergraduate courses [50]. This practice encourages constructive alignment between course components and could lead to the benefits of a deeper understanding of the purpose of learning tasks and assessments, and an improved quality of feedback [37,38,49]. Various studies have shown that students who view research skills as being valuable for their future careers reap a range of benefits for their current education as well as their careers [24,25]. An additional step that faculty can take towards increasing the transparency of research skills' importance is including the list of skills that students will develop within their course on their syllabus and course website. Explicitly stating the targeted skills within a course will increase students' awareness of, ability to articulate, and perceived importance of their skillsets. The alignment of skills with course components and explicit statements of targeted skills will also ensure that skill development is focused and purposeful, such that all students within a course or research experience will have similar, targeted opportunities for research skill development.
A second suggestion is for faculty to assess students' skill development at certain timepoints throughout their study using a specified skill list. Furthermore, students could self-rate their own skill development on an ongoing basis during their research project. This practice can assist students to better understand their strengths and areas requiring improvement. In addition, providing a visualization of students' skill development throughout the duration of their research experience can help them to articulate their skillset and the opportunities that led to their skill development [37]. This ability would be useful for students to align future skill development opportunities, such as workshops, to their needs. In addition, a strong understanding of their skillset and skill development experiences is valuable for future job and school applications and interviews. Similarly, the "Attitudes Toward Research" scale, developed by Papanastasiou (2005) [51], could be used by educators to collect students' feelings towards research. This information could then provide real-time feedback to educators to redirect potential misconceptions and assist students' learning [51,52].
Third, to visualize research skill development on a program-wide level, curriculum mapping can be adapted to include research skills across program components. Curriculum mapping is used to illustrate the relationships and intersections between various program features and goals [53,54]. This process allows for connections among teachings, assessments, and outcomes to be visualized. It can be adapted to illustrate targeted skill development and enables comparisons to be made across program features and between programs. It is suggested that research teaching be incorporated earlier in a student's program rather than waiting until upper years when research courses are commonly offered. As previously described, it is critical that students learn the relevant concepts related to research and develop their declarative knowledge for procedural knowledge to be learned and relevant skills to be developed [12][13][14][15]17]. Introducing research concepts earlier in a student's program also holds value because a student's understanding of the purpose of research for their future has been found to take time to develop [7]. Therefore, designing courses to challenge students' misconceptions and introducing the purpose and importance of research early on in their education can influence conceptual change and skill development such that they will occur more readily [7]. Developing skills earlier on in students' education may also provide an avenue for them to reach the epistemic level of knowledge development, thus enabling an understanding that research leads to the continual production of knowledge in society [15]. Explicit exposure to opportunities for research skill development also increases students' familiarity with the research environment and may inspire a more positive orientation towards research, contributing to feelings of preparedness for future research [25]. Further, an explicit research skill list could be used to develop a validated instrument for assessing students' research skill competencies, as instruments of this type are limited and variable.
The strategies presented in this section are a guide to curricular updates or future program development suggesting the incorporation of explicit research skill development opportunities in a time-effective manner and without great alterations to existing curricula. These strategies are meant to increase the transparency of research skills' importance and the frequency with which students have an opportunity to explicitly practice their research skills. The deliberate practice of research methods was found to be the most important factor contributing to the development of research skills and knowledge [16]. By adding these opportunities to both thesis and non-thesis research experiences, students will reap the benefits for their current programs and future careers.

Conclusions and Implications
In summary, this viewpoint article provides an insight into the incorporation of research skill development opportunities in thesis and non-thesis undergraduate and master's programs. Seven core research skills (critical appraisal, information synthesis, decision-making, problem solving, data collection, data analysis, and communication) that are developed during, and are important for, research experiences were described. This list includes technical skills that are traditionally associated with research experiences, but also considers transferable skills. By encompassing transferable skills, the definition of "research skills" is broadened to be inclusive of a wide variety of research experiences in both thesis and non-thesis programs. This definition is further supported by the finding that similar skills were reported across both thesis and non-thesis research experiences, despite the lacking literature directly comparing these program types. Taken together, these findings imply that similar skill outcomes can be achieved across different programs at the same degree level, with the differences being the program features and teaching and learning activities that are used to achieve development of those skills.
Furthermore, this study supports the incorporation of explicit research skill development opportunities into undergraduate and master's programs. The first step in incorporating these explicit opportunities is to minimize misconceptions as students' conceptions can have an impact on their learning, motivation, and career readiness. By incorporating discussions and activities around research skills, students can understand which skills are developed during and are important for research experiences, as well as the value of developing these skills. As previously described, the assumption that skills are naturally developed during research participation can lead to confusion and variability in skill development among students. Rather, ensuring that core research concepts are taught in conjunction with relevant and explicit skill development opportunities can provide a range of benefits for students. These benefits include a deeper understanding of the purpose of learning tasks, an increased ability for skill articulation, and even stronger performance in research courses. Furthermore, providing frequent opportunities for the deliberate practice of research skills will increase students' familiarity with, and potential interest in, research. These factors can thereby contribute to students' knowledge and skill development as well as feelings of preparedness for future research endeavours. By ensuring an alignment between faculty and students' definitions of a "research skill", students' research skills can be effectively developed and applied across a wide range of future educational and career pathways. Broadening the definition of "research skills" allows program development to be streamlined to offer explicit research experiences across disciplines in both thesis and non-thesis programs that are beneficial and applicable to students' futures.

Conflicts of Interest:
The authors declare no conflict of interest.