Next Article in Journal
Cultivating Global Citizenship through Higher Education: A Reflection on the Development from Civic to Global Engagement
Previous Article in Journal
A Study of Primary Students’ Technology Acceptance and Flow State When Using a Technology-Enhanced Board Game in Mathematics Education
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 12
Issue 11
10.3390/educsci12110765
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics

by
Víctor Raul Ferrer-Pardo
1,
Irene Jimenez-Perez
1,2,*,
Marina Gil-Calvo
1,3,
Pedro Pérez-Soriano
1,2 and
Jose Ignacio Priego-Quesada
1,2
1
Research Group in Sports Biomechanics (GIBD), Department of Physical Education and Sports, Faculty of Physical Activity and Sport Sciences, University of Valencia, 46010 València, Spain
2
Research Group in Medical Physics (GIFIME), Department of Physiology, Faculty of Medicine and Dentistry, University of Valencia, 46010 València, Spain
3
Department of Physical Education and Sports, Faculty of Physical Activity and Sports Sciences, Universidad de León, 24004 León, Spain
*
Author to whom correspondence should be addressed.
Educ. Sci. 2022, 12(11), 765; https://doi.org/10.3390/educsci12110765
Submission received: 6 September 2022 / Revised: 8 October 2022 / Accepted: 26 October 2022 / Published: 28 October 2022
Versions Notes

Abstract

:
The use of rubrics in the evaluation of oral presentations has been associated with several benefits for students. However, it is unknown whether students with better academic marks and greater self-regulation find the use of rubrics more useful or not. This paper aims to assess the relationship between how students perceive the use of a rubric and their academic characteristics, and to analyze the congruence between the professor’s and students’ evaluations when using the rubric. Eighty-five students studying for a Degree in Sport Sciences participated in this study. A rubric for oral presentations was used to assess the students’ performance. The students then filled out a questionnaire about their perception of the validity of the rubric, an assessment of academic performance, and a self-regulation questionnaire. Inverse correlations were observed between the academic record and two items of the rubric validity perception (r < −0.24). Direct correlations were also found between learning oriented self-regulation and four items of the rubric validity perception (r > 0.22). There was very good congruence between the professor’s and students’ marks when using the rubric (ICC = 0.78). The results suggest that the rubric used is a good instrument to ensure fair and consistent evaluations, despite possible differences between evaluators.

1. Introduction

Traditionally, university education has been considered an instructional phase exclusively directed at checking the degree of knowledge acquired by the student, without considering the student’s ability to learn or the learning process [1]. However, active student-focused teaching methods aim to enhance the learning process and develop transversal skills such as communication skills or teamwork [1,2,3,4]. One of the new university educational model objectives is to prepare students for their possible future job roles, including an improvement in communication skills [5,6]. Although oral presentations are the most common method employed for developing these communication skills, teaching and learning how to do this properly is a complex task because of the number of different aspects involved (content, formal, temporal, and others) and the variety of applicable evaluation criteria [6,7,8]. There is, therefore, a need to unify criteria and appraisals in this field. Establishing and using rubrics is a useful tool for evaluating these skills, since they allow us to align the teaching, learning and evaluation processes [9,10].
A rubric can be defined as a guide that clarifies the criteria used to evaluate the quality and level of skills development achieved by the students [11]. In general, rubrics are made up of three essential elements: evaluation criteria, descriptors and/or quality levels, and rating or scoring strategies [12]. Some of the benefits associated with the use of rubrics are: (a) higher quality learning, given the focus of attention on positive criteria in preparing work; (b) a simpler and more accurate self-evaluation of students; (c) more valid evaluations by teachers on the level of skills acquisition; and d) being able to offer more specific feedback to students on the quality of their work or performance, and so highlighting the aspects to be improved [13,14]. Moreover, it has also been observed that the use of rubrics reduces students’ complaints about their marks, reduces their anxiety, and increases their perception of the quality of their work [15,16,17].
However, although the use of rubrics has begun to increase in the university context, it is still scarce in relation to other methods [10], due to professors’ distrust of their reliability and validity in evaluating the academic performance of students [12,14,18]. Although various studies have shown that students positively perceived the use of rubrics [15,16,19], in some cases, students did not believe that their performance had improved, but that they had performed the task more in accordance with the rubric’s demands [15]. On this issue, of whether rubrics can be more beneficial for some kinds of students than others there is still scarce knowledge. Moreover, Priego Quesada et al. [20] observed that a group with a higher academic mark perceived the use of a rubric more positively. It has also been suggested that there is a relationship between the co-creation of rubrics and the improvement of self-regulation and self-efficacy towards learning and academic performance [21]. Self-regulation is a student’s self-initiated and cyclic process in which they self-represent a task, plan how to carry it out depending on their expectations, assess its realization, be faced with difficulties and emotions, and make attributions concerning the origin of the outcomes [22,23]. Alonso-Tapia et al. [24] define two styles of self-regulation: (1) learning-oriented (implying positive emotions), when students focus on the process and experience progress, leading them to maintain motivation to continue with the task; and (2) performance/avoidance-oriented (implying negative emotions), when students are not able to manage stress due to the difficulty of the task, thus leading them to avoid it. In this sense, it is known that higher self-regulation enhances learning motivation and students’ satisfaction with their own work [24]. However, it is unknown whether students with greater self-regulation find the use of rubrics more useful or not.
Various studies have evaluated the reproducibility of a rubric between peer evaluators [1,8,25]. This is an important aspect for ensuring the quality of rubric use by different professors of the same subject. Unifying the criteria can be favorable for examiners, instructors, and learners because it can increase the quality of instruction, and their functionality as a guideline in terms of learning expectations and fostering critical reflection on performance [25,26]. However, there is sometimes a shared evaluation between the professor and students, and in this case, the professor–student congruence/agreement could be lower because both profiles have different knowledge, experiences, expectations, etc.
The objectives of this study were to assess the relationship between the perception of using a rubric and the academic characteristics of students (academic mark performance and self-regulation), and to analyze the congruence between professor’s and students’ evaluations when using a rubric. We hypothesized that students with better self-regulation and academic marks would also have better marks using rubrics. We also hypothesized that professor’s and students’ evaluations would show a high level of agreement due to the clarity of the rubric’s criteria.

2. Materials and Methods

2.1. Participants

The intervention was carried out during the 2019–2020 academic year, in the subject “Methodology of the Teaching of Physical Activity and Sport” of the Degree in Sciences of Physical Activity and Sport. This degree course is taught at the Department of Physical Education and Sports of the University of Valencia (Valencia, Spain). The subject is a core course and is taught in the first semester of the third year of the degree. That academic year, 123 students enrolled in this subject. However, for our analysis, only the 85 participants who reported their perception of the validity of the rubric were considered (63 males and 22 females, 21 ± 2 years old). The professor who carried out the intervention was a male of 33 years old, with 5 years of experience teaching at the university.

2.2. Design of the Intervention

The intervention was put into practice in the unit of the subject where the different teaching styles applicable to the sciences of physical activity and sport take place [27]. In this unit, the professor introduced the topic in two sessions (the first session was an introduction, and in the second one he trained the students in the use of the rubric). Several sessions were then held where the students presented the different teaching styles through oral presentations. Finally, the professor provided conclusions to close the unit in a final session. The students made a total of 18 oral presentations. The regulations for oral presentations consisted of the following points:
  • The composition of the teams was made by the professor in accordance with the criterion of alphabetical order on the list.
  • The teams were composed of between 5 and 7 students.
  • The duration of the presentation had to be between 25 and 35 min.
  • On the same day as the evaluation, the teams were given a sheet with the presentation divided into parts in line with the number of participants, and the professor decided which member should present each of the parts. The aim of this was that all the members had to know the whole presentation.
  • In each oral presentation, both the professor and the students who were not making the presentation had to evaluate the team using a rubric. The final mark of the oral presentation was made up 60% by the professor, 30% by the students, and 10% in the form of self-evaluation.
  • Before the students were given their mark, they were asked to fill out an anonymous questionnaire on their perception of the validity of the rubric and another on their self-regulation.
The temporal distribution of the intervention is shown in Table 1.

2.3. Rubric

A modified version of the rubric, developed and evaluated by García-Ros [1] (Table 2), was used as a rubric to assess the oral presentations. The only aspect modified from the original version was the inclusion of an extra score of 0.5 points if students incorporated creative and innovative elements in their presentation.

2.4. Rubric Validity Perception Questionnaire

A questionnaire, designed with Google Forms (Google, Mountain View, CA, USA), was used to evaluate the students’ perception of the validity and usefulness of the rubric. This questionnaire consisted of 9 items (see items in Table 4), using a 5-level Likert scale, from “not at all agree” to “strongly agree”. In addition, the total validity perception score was considered as the sum of the scores of all the items. The questionnaire was carried out by a total of 85 participants anonymously. The questionnaire was previously validated [20] and was delivered via online format.

2.5. Assessment of Academic Performance

The academic performance of the 85 students was characterized by different marks: the mark given by the students that attended the oral presentation, the mark given by the professor for the oral presentation, the final mark for the oral presentation (considering students and professor’s marks), the final mark for the subject, and the mean mark in the degree obtained at the moment of the questionnaire.

2.6. Self-Regulation Questionnaire

The “Emotion and Motivation Self-Regulation Questionnaire” (EMSR-Q) was used to analyze student self-regulation [24]. This questionnaire includes 20 items that are answered on a 5-Likert scale, from “never” to “always”. The items are grouped into two scales: learning oriented self-regulation (with 8 items) and performance/avoidance oriented self-regulation (with 12 items) (Table 3). The first scale includes actions that are oriented to learning objectives, so the higher the value on this scale, the more positive the effect of learning oriented self-regulation (reliability α = 0.84) [24]. The second scale includes actions that show a lack of self-regulation or performance orientation, so the higher the value on this scale, the more negative the actions will be for learning performance (reliability α = 0.77) [24]. This questionnaire was also designed with Google Forms (Google, Mountain View, CA, USA) and was delivered via online format.

3. Statistical Analysis

The statistical analysis was performed using SPSS 26 (SPSS Inc., Chicago, IL, USA). Data were reported as mean (standard deviation). Relationships between students’ perception of the validity and usefulness of the rubric and their academic characteristics (academic marks performance and self-regulation) were assessed using bivariate Pearson correlations. Significant correlations (p < 0.05) were classified as weak (±0.2 < r < ± 0.5), moderate (±0.5 ≤ r < ± 0.8), or strong (r ≥ ±0.8) [28]. In addition, the intraclass correlation coefficient (ICC) based on a mean rater measurement, absolute agreement, and 2-way random-effects model was calculated between the professor’s and students’ (those who attended the oral presentation) marks. The following classification of ICC values was used [29]: values 1.00 to 0.81 (excellent reproducibility), 0.80 to 0.61 (very good), 0.60 to 0.41 (good), 0.40 to 0.21 (reasonable), and from 0.20 to 0.00 (poor). Finally, as the marks for oral presentations provided by the students and the professor presented a non-normal distribution (Kolmogorov–Smirnov test; p < 0.05), the differences between both marks were assessed by Wilcoxon test. The effect size (ESr) of the difference obtained was calculated as Z statistic divided by square root of the sample size and classified as small (ESr ≥ 0.1), moderate (ESr ≥ 0.3), or large (ESr ≥ 0.5) [30]. The level of statistical significance was set at p < 0.05.

4. Results

4.1. Relationship between Students’ Perception of the Validity and Usefulness of the Rubric and Academic Characteristics of Students

Table 4 shows the values obtained about the students’ perception of the validity and usefulness of the rubric. The rubric was valued in most of the items with average values higher than four (corresponding to the valuations between “Agree” and “Strongly agree”).
Table
Table 4. Mean (standard deviation) of the students’ perception of the validity and usefulness of the rubric.
Table 4. Mean (standard deviation) of the students’ perception of the validity and usefulness of the rubric.
ItemMean (SD)
1. I have found it very appropriate to assess the presentations.4.3 (0.6)
2. I found it very easy to use.4.5 (0.7)
3. The number of items was adequate.4.1 (0.9)
4. It was easy for each item to know the assessment of each group.4.0 (0.7)
5. Having the rubric it is easier to prepare a presentation to get a good mark.4.3 (0.8)
6. With the rubric you can evaluate more fairly.4.1 (0.7)
7. The rubric has helped us to prepare the presentation.3.9 (1.0)
8. The rubric includes the most important aspects of the presentation to evaluate.4.1 (0.7)
9. I recommend using the rubric in future years4.4 (0.6)
10. Total rating (sum of all items; maximum rating = 45).37.7 (4.0)
Table 5 shows the mean and SD values of the academic characteristics and self-regulation of the students assessed.
Table 6 shows Pearson’s correlation between rubric perception, academic performance, and self-regulation. Weak inverse correlations were observed between the academic record and the appropriateness item on the use of rubrics to evaluate a speech (r = −0.28), and the ease-of-use rubric item (r = −0.24). Moreover, weak direct correlations were found between learning-oriented self-regulation and the appropriateness of items in the rubric (r = 0.22), the fairness of evaluations using the rubric (r = 0.24), the satisfaction with the rubric (r = 0.34), and the total rating (r = 0.29). Another important point to observe is the correlation between the final oral presentation mark and the perception of obtaining a good mark with the rubric (r = 0.22).

4.2. Professor–Student Congruence of Evaluation

Very good congruence was observed between the professor’s and students’ (those who attended the oral presentation) marks when using the rubric (ICC = 0.78). The professor’s mark was higher than the mark provided by the students, with a moderate effect size (8.7 (0.9) vs. 8.5 (0.6) points respectively; p < 0.01, ESr = 0.32).

5. Discussion

The purpose of this research was to evaluate the relationship between the perception of the use of rubrics and the academic characteristics of students (academic marks performance and self-regulation), and to analyze the congruence between the professor’s and the students’ evaluations when using the rubric. The main finding of the study was that the academic record was negatively correlated with some of the items in the perception of the use of rubrics. Additionally, a positive correlation between the learning oriented self-regulation and some of the items in the perception of the use of rubrics was observed. Finally, there was a very good congruence between the professor’s and the students’ evaluations.
It is important to understand the relationship between the perception of the validity of the rubrics and the performance of the student [6], both in the case of the whole academic record and in the case of the subject where the rubric was applied. It has been suggested that the use of the rubrics improves the educational process using alternative and novel forms of evaluation, and it is also consistent and precise [12,14,31]. In this context, in the present study, no clear relationship was observed between the perception of validity of the rubric and academic performance. In the first place, no validity perception variable of the rubric was related to the subject mark. It is important to highlight that most of the items of the perception of validity of the rubric focus on oral presentation (specifically items one, five, seven, and eight), whereas the subject mark depends on more parts of the subject than just the oral presentation (e.g., exams, class tasks, practical sessions, etc.). Therefore, since the subject mark is so broken down, there may be a significant number of students who have obtained high marks for the subject for being outstanding in other aspects than the oral presentation, which would explain the lack of correlation with these items. Moreover, it is possible that this lack of correlation on all the items could be due to the small variability between the marks, so that stronger correlations could not exist [32]. It has been found that the use of rubrics enhances the performance of students [33,34,35,36], which could explain the higher marks observed in our study. In this sense, the students who obtained a higher mark in the oral presentation did value the use of the rubric more positively for achieving a better grade. This can be explained by the fact that the rubrics are not only used to assess but also to teach/learn how to perform a task properly [34]. Therefore, special care must be taken to avoid the danger of using a rubric in teaching solely for a test, without learning in general [37]. Future studies should investigate these aspects with higher variability of marks between students.
Nevertheless, the results of the present study also showed a negative correlation between the academic record and the appropriateness of the use of rubrics for evaluating a speech, and also between academic record and the ease of use of the rubric. This means that students with higher academic records valued the use of rubrics as a tool to evaluate their presentations more negatively, as well as their ease of use. These results, however, should be treated with caution as they present a weak correlation (±0.2 < r < ±0.5). A possible explanation could be that students with lower marks are often in greater need of guidance to improve their qualifications and students with better marks are already aware of these strategies for studying and undertaking academic tasks [38,39].
It was also observed that students with higher learning-oriented self-regulation found the number of items of the rubric more appropriate, perceived that the evaluation can be fairer thanks to the use of the rubric, would further recommend the use of the rubric in future years, and were generally more satisfied with the use of the rubric. As was observed with the academic record, these correlations were also weak. However, the fact that in this case they were repeated in three items and also in the total sum of the items provides greater consistency to the result. Therefore, as in a previous study [40], self-regulation of learning does seem to have a direct relationship with the perception of validity of the rubric. This relationship of ideas could be due to the fact the students have a behavior based on learning instead of based on performance (the marks they get). While an excessive number of items in a rubric could lead to difficulties in their learning, having an adequate number makes students focus better on those items and learn the important information to be successful in these items [15,41]. In addition, according to various studies [12,13,42], students perceive the evaluation using the as rubric fairer, as they know how they will be evaluated, so they can use strategies to emphasize their efforts on the most important aspects and avoid losing time on non-essential aspects.
On the other hand, no correlation was observed between the perception of validity of the rubric and performance/avoidance-oriented self-regulation. This result was positive since it may indicate that the rubric is easy to use, and is not better or worse valued by the students who have a worse or better performance/avoidance-oriented self-regulation [34].
Finally, although the professor’s evaluations were slightly higher than the students’ evaluations, high congruence was found between the professor’s and the students’ marks when using the rubric (ICC = 0.78). In this respect, it has been demonstrated that rubrics are a good tool for evaluation because of their good inter-subject reproducibility [1,25]. In the present study, the degree of agreement between the evaluation of the professor and the students remained strong even though the evaluators do not have the same academic experiences, nor can they be included within the same academic figure. Therefore, a well-constructed rubric is a good tool for fairer and more consistent assessments [9,34,41], and also allows a shared evaluation between professor and students. In this sense, it should be noted that the results found in this study may not be generalizable for all rubrics, since the different elements involved in the design of a rubric (e.g., the experience in the use of rubrics of the creator and of the users; the form of writing; the organization and presentation of the text; or the previous instructions provided to the users) can make one rubric very different from another [41,42].
One limitation of this study was that it was performed with only one subject and with one professor. In addition, due to the weak correlations and little variability between marks, these results must be interpreted with caution. Future studies with more professors and subjects, and greater variability should corroborate these findings. As it remains unknown, it would also be interesting to explore the behavior of the students’ evaluations of their classmates based on their academic record. Students with a better academic record could evaluate their classmates differently based on the effort they make to obtain a good result; however, this hypothesis needs further investigation.

6. Conclusions

The results of the present study suggest that the rubric used is a good instrument for ensuring fair and consistent evaluations of oral presentations, despite the possible differences between evaluators. A weak negative correlation was observed between academic record and the perception of the usefulness of rubric; maybe because students with better marks already know how they will be evaluated, and they do not need the guidance that rubrics offer as much. In addition, a higher learning-oriented self-regulation correlated positively with a good perception of use of the rubrics, especially in the number of items and fairness in the evaluations. Further investigations should check whether these results tend to repeat, mirroring to the lack of variability in the marks of this particular study.

Author Contributions

Conceptualization, I.J.-P. and J.I.P.-Q.; formal analysis, J.I.P.-Q.; investigation, M.G.-C.; methodology, J.I.P.-Q.; project administration, P.P.-S.; supervision, M.G.-C. and J.I.P.-Q.; validation, I.J.-P. and P.P.-S.; writing—original draft, V.R.F.-P.; writing—review and editing, I.J.-P., M.G.-C., P.P.-S. and J.I.P.-Q. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding but was part of the Educational Innovation Projects of the University of Valencia (ref. UV-SFPI_PID19-1096320).

Institutional Review Board Statement

The Institution where the study was conducted did not require approval from the ethics committee for this research because when students begin their studies they authorize the use of their academic data anonymously for research.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. García-Ros, R. Análisis y Validación de Una Rúbrica Para Evaluar Habilidades de Presentación Oral En Contextos Universitarios. Electron. J. Res. Educ. Psychol. 2011, 9, 1043–1062. [Google Scholar] [CrossRef] [Green Version]
  2. Cooper, K.M.; Ashley, M.; Brownell, S.E. A Bridge to Active Learning: A Summer Bridge Program Helps Students Maximize Their Active-Learning Experiences and the Active-Learning Experiences of Others. CBE Life Sci. Educ. 2017, 16, ar17. [Google Scholar] [CrossRef] [PubMed]
  3. Knight, J.K.; Wise, S.B.; Rentsch, J.; Furtak, E.M. Cues Matter: Learning Assistants Influence Introductory Biology Student Interactions during Clicker-Question Discussions. CBE—Life Sci. Educ. 2015, 14, ar41. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Mann, K.V. Theoretical Perspectives in Medical Education: Past Experience and Future Possibilities. Med. Educ. 2011, 45, 60–68. [Google Scholar] [CrossRef] [PubMed]
  5. Živković, S. The Importance of Oral Presentations for University Students. Mediterr. J. Soc. Sci. 2014, 5, 468. [Google Scholar] [CrossRef]
  6. Verano-Tacoronte, D.; González-Betancor, S.M.; Bolívar-Cruz, A.; Fernández-Monroy, M.; Galván-Sánchez, I.; Verano-Tacoronte, D.; González-Betancor, S.M.; Bolívar-Cruz, A.; Fernández-Monroy, M.; Galván-Sánchez, I. Valoración de la competencia de comunicación oral de estudiantes universitarios a través de una rúbrica fiable y válida. Rev. Bras. Educ. 2016, 21, 39–60. [Google Scholar] [CrossRef] [Green Version]
  7. Dasí, A.; Iborra, M. La Evaluación de Los Estudiantes En La Educación Superior. LA Mejora de La Comunicación Oral: Una Perspectiva de Proceso; Servei de Formació Permanent; Universitat de València: Valencia, Spain, 2007. [Google Scholar]
  8. Murillo-Zamorano, L.R.; Montanero, M. Oral Presentations in Higher Education: A Comparison of the Impact of Peer and Teacher Feedback. Assess. Eval. High. Educ. 2018, 43, 138–150. [Google Scholar] [CrossRef]
  9. Dunbar, N.E.; Brooks, C.F.; Kubicka-Miller, T. Oral Communication Skills in Higher Education: Using a Performance-Based Evaluation Rubric to Assess Communication Skills. Innov. High. Educ. 2006, 31, 115. [Google Scholar] [CrossRef]
  10. Van Ginkel, S.; Laurentzen, R.; Mulder, M.; Mononen, A.; Kyttä, J.; Kortelainen, M.J. Assessing Oral Presentation Performance: Designing a Rubric and Testing Its Validity with an Expert Group. J. Appl. Res. High. Educ. 2017, 9, 474–486. [Google Scholar] [CrossRef]
  11. Popham, W.J. What’s Wrong–and What’s Right–with Rubrics. Educ. Leadersh. 1997, 55, 72–75. [Google Scholar]
  12. Reddy, Y.M.; Andrade, H. A Review of Rubric Use in Higher Education. Assess. Eval. High. Educ. 2010, 35, 435–448. [Google Scholar] [CrossRef]
  13. Jonsson, A.; Svingby, G. The Use of Scoring Rubrics: Reliability, Validity and Educational Consequences. Educ. Res. Rev. 2007, 2, 130–144. [Google Scholar] [CrossRef]
  14. Panadero, E.; Jonsson, A. A Critical Review of the Arguments against the Use of Rubrics. Educ. Res. Rev. 2020, 30, 100329. [Google Scholar] [CrossRef]
  15. Andrade, H.; Du, Y. Student Perspectives on Rubric-Referenced Assessment. Educ. Couns. Psychol. Fac. Scholarsh. 2005, 10, 3. Available online: https://scholarsarchive.library.albany.edu/edpsych_fac_scholar/2 (accessed on 8 April 2022).
  16. Barney, S.; Khurum, M.; Petersen, K.; Unterkalmsteiner, M.; Jabangwe, R. Improving Students with Rubric-Based Self-Assessment and Oral Feedback. IEEE Trans. Educ. 2012, 55, 319–325. [Google Scholar] [CrossRef]
  17. Brookhart, S.M.; Chen, F. The Quality and Effectiveness of Descriptive Rubrics. Educ. Rev. 2015, 67, 343–368. [Google Scholar] [CrossRef]
  18. Contreras-Higuera, W.E.; Martínez-Olmo, F.; José Rubio-Hurtado, M.; Vilà-Baños, R. University Students’ Perceptions of E-Portfolios and Rubrics as Combined Assessment Tools in Education Courses. J. Educ. Comput. Res. 2016, 54, 85–107. [Google Scholar] [CrossRef] [Green Version]
  19. Eshun, E.F.; Osei-Poku, P. Design Students Perspectives on Assessment Rubric in Studio-Based Learning. J. Univ. Teach. Learn. Pract. 2013, 10, 8. Available online: https://eric.ed.gov/?id=EJ1005281 (accessed on 20 April 2022). [CrossRef]
  20. Priego Quesada, J.; Requena-Bueno, L.; Jiménez-Pérez, I.; García-Ros, R. ¿Implicar a Los Estudiantes En La Modificación de Rúbricas Influye Sobre Su Percepción de Validez? In V Congreso de Innovación Educativa y Docencia en Red; Editorial Universitat Politècnica de València: Valencia, Spain, 2019; pp. 329–342. [Google Scholar]
  21. Fraile, J.; Panadero, E.; Pardo, R. Co-Creating Rubrics: The Effects on Self-Regulated Learning, Self-Efficacy and Performance of Establishing Assessment Criteria with Students. Stud. Educ. Eval. 2017, 53, 69–76. [Google Scholar] [CrossRef]
  22. Panadero, E.; Tapia, J.A.; Huertas, J.A. Rubrics and Self-Assessment Scripts Effects on Self-Regulation, Learning and Self-Efficacy in Secondary Education. Learn. Individ. Differ. 2012, 22, 806–813. [Google Scholar] [CrossRef]
  23. Efklides, A. Interactions of Metacognition with Motivation and Affect in Self-Regulated Learning: The MASRL Model. Educ. Psychol. 2011, 46, 6–25. [Google Scholar] [CrossRef]
  24. Alonso-Tapia, J.; Panadero Calderón, E.; Díaz Ruiz, M.A. Development and Validity of the Emotion and Motivation Self-Regulation Questionnaire (EMSR-Q). Span. J. Psychol. 2014, 17, E55. [Google Scholar] [CrossRef] [PubMed]
  25. Keshmiri, F.; Ponzer, S.; Sohrabpour, A.; Farahmand, S.; Shahi, F.; Bagheri-Hariri, S.; Soltani-Arabshahi, K.; Shirazi, M.; Masiello, I. Contextualization and Validation of the Interprofessional Collaborator Assessment Rubric (ICAR) through Simulation: Pilot Investigation. Med. J. Islam. Repub. Iran 2016, 30, 403. [Google Scholar]
  26. Curran, A.; Klein, M.; Hepokoski, M.; Packard, C. Improving the Accuracy of Infrared Measurements of Skin Temperature. Extrem. Physiol. Med. 2015, 4, A140. [Google Scholar] [CrossRef] [Green Version]
  27. Mosston, M.; Ashworth, S. The Spectrum of Teaching Styles: From Command to Discovery; Longman: Harlow, UK, 1990; ISBN 978-0-8013-0350-0. [Google Scholar]
  28. O’Rourke, N.; Hatcher, L.; Stepanski, E.J. A Step-by-Step Approach to Using SAS for Univariate & Multivariate Statistics; SAS Institute: Cary, NC, USA, 2005; Available online: http://books.google.es/books?hl=es&lr=&id=pfUfzykTZ1AC&oi=fnd&pg=PP1&dq=A+Step-by-step+Approach+to+Using+SAS+for+Univariate+%26+Multivariate+Statistics&ots=8z_RqQ9Ktd&sig=q68wV7c90KexLUL5MvI6wJs3xSw (accessed on 15 May 2022).
  29. Weir, J.P. Quantifying Test-Retest Reliability Using the Intraclass Correlation Coefficient and the SEM. J. Strength Cond. Res. Natl. Strength Cond. Assoc. 2005, 19, 231–240. [Google Scholar] [CrossRef] [Green Version]
  30. Tomczak, M.; Tomczak, E. The Need to Report Effect Size Estimates Revisited. An Overview of Some Recommended Measures of Effect Size. Trends Sport Sci. 2014, 1, 19–25. [Google Scholar]
  31. Andrade, H.G. Teaching with Rubrics: The Good, the Bad, and the Ugly. Coll. Teach. 2005, 53, 27–31. [Google Scholar] [CrossRef]
  32. Bennett, A.F. Interindividual Variability: An Underutilized Resource. New Dir. Ecol. Physiol. 1987, 19, 147–169. [Google Scholar]
  33. Reitmeier, C.A.; Svendsen, L.K.; Vrchota, D.A. Improving Oral Communication Skills of Students in Food Science Courses. J. Food Sci. Educ. 2004, 3, 15–20. [Google Scholar] [CrossRef]
  34. Panadero, E.; Romero, M. To Rubric or Not to Rubric? The Effects of Self-Assessment on Self-Regulation, Performance and Self-Efficacy. Assess. Educ. Princ. Policy Pract. 2014, 21, 133–148. [Google Scholar] [CrossRef]
  35. Andrade, H.G. The Effects of Instructional Rubrics on Learning to Write. Educ. Theory Pract. Fac. Scholarsh. 2001, 6, 1–21. Available online: http://scholarsarchive.library.albany.edu/etap_fac_scholar/6 (accessed on 2 February 2022).
  36. McCormick, M.J.; Dooley, K.E.; Lindner, J.R.; Cummins, R.L. Perceived Growth versus Actual Growth in Executive Leadership Competencies: An Application of the Stair-Step Behaviorally Anchored Evaluation Approach. J. Agric. Educ. 2007, 48, 23–35. [Google Scholar] [CrossRef]
  37. Copp, D.T. Teaching to the Test: A Mixed Methods Study of Instructional Change from Large-Scale Testing in Canadian Schools. Assess. Educ. Princ. Policy Pract. 2018, 25, 468–487. [Google Scholar] [CrossRef]
  38. Ramírez, A.G. Language Learning Strategies Used by Adolescents Studying French in New York Schools. Foreign Lang. Ann. 1986, 19, 131–138. [Google Scholar] [CrossRef]
  39. Zulaihah, S.; Harida, R. Autonomous Learning Strategy of the Successful Nontraditional Students. Eltin J. J. Engl. Lang. Teach. Indones. 2017, 5, 71–84. [Google Scholar] [CrossRef]
  40. Nicol, D.J.; Macfarlane-Dick, D. Formative Assessment and Self-Regulated Learning: A Model and Seven Principles of Good Feedback Practice. Stud. High. Educ. 2006, 31, 199–218. [Google Scholar] [CrossRef]
  41. Dawson, P. Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice. Assess. Eval. High. Educ. 2017, 42, 347–360. [Google Scholar] [CrossRef]
  42. Humphry, S.M.; Heldsinger, S.A. Common Structural Design Features of Rubrics May Represent a Threat to Validity. Educ. Res. 2014, 43, 253–263. [Google Scholar] [CrossRef]
Table
Table 1. Temporal distribution of the intervention.
Table 1. Temporal distribution of the intervention.
TaskDate
Rubric training10 October 2019
Oral presentationsFrom 22 October 2019 to 7 November 2019
Completion of the questionnaire on perception of the validity of the rubric and on self-regulation7 November 2019
Publication of oral presentation marks10 December 2019
Table
Table 2. Rubric used for the oral presentations. Modified and translated from García-Ros [1].
Table 2. Rubric used for the oral presentations. Modified and translated from García-Ros [1].
1. Basic Principles (30%)
Inadequate 0To Improve 1Appropriate 2Very Appropriate 3
Mastery and understanding of the topicDoes not master or understand the subjectShows a good understanding of parts of the topic, but not some of themShows a good understanding of parts of the topicExpresses a complete and deep understanding of the subject
Content planning and organizationHave not plannedDifficult to follow speech and inconsistent connections, indicating poor planning and little effortFollows a proper general outline, even if he/she gets lost or needs to pay more attention to detailsWell planned speech, links topics logically and coherently
2. Application and exemplification (30%)
Inadequate 0To improve 1Appropriate 2Very appropriate 3
Consistency with the theoretical modelThe developed example is not consistent with the modelThe example developed is consistent with the model, although various aspects are not adequately capturedThe example developed is consistent with the theoretical model, although it would need to refine some specific aspectThe example developed is consistent with the theoretical model
Model clarification and audience motivationSelected example is confusing and does not fit the modelThe example does not clarify adequately and is not motivating for the audienceThe example does not clarify adequately OR is not motivating for the audienceThe example is clarifying and motivating for the audience
3. Visual support (20%)
Inadequate 0To improve 1Appropriate 2Very appropriate 3
Number and explanation of slidesIt does not meet the number and adequacy requirements in its explanation.Uses an inappropriate number of slides, but tries to explain themAdequate number of slides, although imprecise explanation of some of themAdequate number of slides and relevant explanation of them
Suitability of the slidesIt incorporates slides that are not relevant to the topic, they do not facilitate the understanding or motivation of the audience.Although most of them are related to the topic, they do not favor their understanding or the motivation of the audienceAll are related to the topic and facilitate material understanding, although some are not connected with speech or suffer from amenityAll are related to the topic and connection with the discourse, facilitating the
understanding of the material, and are entertaining/attention-grabbing
ReadabilityIllegible, overwhelming use of colors, abuse of text and paragraphsHandwriting readable by approximately 85% audience, excessive length in mostly slides. Inadequate color contrastHandwriting readable for the entire audience, more than 5 lines per slide. Adequate contrast of colorsNo more than 5 lines per slide, font legible for the entire audience, adequate contrast of colors
Relevance and appropriateness of images/diagramsDoes not use or distractAccording to content, although not relevant and low resolutionRelevant, consistent with content on most occasions, but little resolutionRelevant, consistent with content, and with adequate resolution
Sequencing, animations, slide transitionsNo order, excessive or no use. Animation and transition that facilitate distractionLogical order, although animation is missing and only applies transitions on some slide, or does both improperlyLogical order, animation of unimportant points or transitions that impede attentionAnimation of important points, helps focus attention on what is important and avoids being distracted
4. Communication skills (20%)
Inadequate 0To improve 1Appropriate 2Very appropriate 3
Volume and tone of voiceToo low to be heard by all and excessively monotonousLoud enough to be heard around 70% of the time by everyone and/or inappropriate tone (monotonous, boring)Loud enough to be heard by all and adequate tone of voice around 90%Loud enough and with an appropriate tone of voice to be heard by the entire audience
Clarity in speechLittle clarity. Often mumbles and mispronounces wordsSpeaks clearly 85–90% of the time and mispronounces numerous wordsHe speaks clearly 90–95% of the time, although he mispronounces some wordsSpeak clearly practically all of the time.
Pauses and use of taglinesDoesn’t use pauses on purpose. Tagline abuse. Large number of incomplete sentencesInadequate number of pauses –by excess or defect-, introduce some at unexpected moments or frequently use taglines (um…, ehhh…, well… etc.). Uses a lot of incomplete sentencesUses pauses correctly but introduces some in an unexpected/inappropriate way. Uses taglines on several occasions (um…, ehhh…, good… etc.). Uses incomplete sentences more than 5% of the timeUse pauses correctly and at the end of sentences. Use complete sentences more than 95% of the time
Posture and eye contactDoes not maintain eye contact and body posture is inadequateSometimes maintains an upright posture and maintains eye contact with the audience, but less than 90% of the timeUpright posture and makes eye contact with everyone, although some indicators of tension/disinterest are observedStraight back, relaxed and confident posture, make eye contact with everyone
5. Innovation and creativity (+0.5)
If creative and innovative elements have been incorporated, +0.5 can be added to the final grade. In the case of peer evaluation, this +0.5 will be considered if it is requested by more than 50% of the evaluations.
Table
Table 3. Emotion and motivation self-regulation questionnaire (EMSR_Q) [24].
Table 3. Emotion and motivation self-regulation questionnaire (EMSR_Q) [24].
Items of the Emotion and Motivation Self-Regulation Questionnaire
Learning-oriented self-regulation (α = 0.84)
1This is going O.K.! … It seems that I understand it.
2Calm down… “Do not hurry, do not stop” … You’ll get it.
3Well… It seems that every time I do it better… I’m progressing…
4How interesting! It seems to me that I understand it.
5How difficult, but how interesting!… I have to understand how to do it.
6This is not right… I’m going to check it step by step.
7How complicated!… Well, I’ll go on… it is important to learn how to solve it.
8Here was the mistake! Great! Next time I will know how to do it.
Performance/avoidance oriented self-regulation (α = 0.77)
9This is not worth my time… Let’s try to finish it as soon as possible
10This task is a complete loss of time!
11What instructions so long! They only make me confused.
12What a boring task! Let’s see if I finish and leave.
13I’m dead tired… Well, I had to go on to pass.
14I must go on… if I do not, I’ll fail.
15What a mess! Well… Go on… if not you won’t pass the exam.
16What a tiring task!… But I have to pass… Let’s continue.
17What a stressful task! I’m doing it very bad… It’s so difficult!
18This is so difficult… I am not going to be able to make it right.
19I am not made for doing this. If I could, I would give it up.
20I am getting nervous… I’m not able to do it.
Table
Table 5. Mean and standard deviation (SD) of the academic characteristics and self-regulation of the students assessed.
Table 5. Mean and standard deviation (SD) of the academic characteristics and self-regulation of the students assessed.
Academic CharacteristicMean (SD)
Oral presentation mark provided by students (max 10 points)8.5 (0.6)
Oral presentation mark provided by professor (max 10 points)8.7 (0.9)
Final oral presentation mark (max 10 points)8.8 (0.7)
Subject mark (max 10 points)8.2 (1.3)
Academic record (max 10 points)7.7 (0.6)
Self-regulationMean (SD)
Learning oriented self-regulation (max 40 points)28.9 (4.5)
Performance/avoidance oriented self-regulation (max 60 points)35.7 (7.4)
Table
Table 6. Pearson’s correlation between rubric perception, academic performance, and self-regulation.
Table 6. Pearson’s correlation between rubric perception, academic performance, and self-regulation.
Pearson’s Correlation (p-Value)
ItemFinal Oral Presentation MarkSubject MarkAcademic RecordSelf-Regulation LearningSelf-Regulation Performance
1. I have found it very appropriate to assess the presentations.0.01 (0.94)−0.15 (0.16)−0.28 (0.01)0.08 (0.45)0.01 (0.90)
2. I found it very easy to use.−0.04 (0.74)−0.15 (0.16)−0.24 (0.03)0.08 (0.45)−0.06 (0.59)
3. The number of items was adequate.0.01 (0.96)−0.05 (0.64)−0.19 (0.08)0.22 (0.04)−0.04 (0.74)
4. It was easy for each item to know the assessment of each group.0.01 (0.97)0.04 (0.73)−0.12 (0.29)0.10 (0.34)−0.04 (0.72)
5. Having the rubric it is easier to prepare a presentation to get a good mark.0.22 (0.04)0.04 (0.69)−0.10 (0.39)0.19 (0.08)−0.01 (0.93)
6. With the rubric you can evaluate more fairly.0.04 (0.71)0.01 (0.92)−0.13 (0.23)0.24 (0.03)−0.03 (0.79)
7. The rubric has helped us to prepare the presentation.0.13 (0.25)0.06 (0.56)0.06 (0.60)0.13 (0.23)−0.07 (0.52)
8. The rubric includes the most important aspects of the presentation to evaluate.−0.15 (0.16)−0.18 (0.11)−0.02 (0.86)0.20 (0.07)−0.12 (0.26)
9. I recommend using the rubric in future years.−0.14 (0.21)−0.14 (0.20)−0.16 (0.16)0.34 (0.002)−0.09 (0.41)
10. Total rating (sum of all items; maximum rating = 45).0.03 (0.79)−0.08 (0.47)−0.20 (0.07)0.29 (0.008)−0.08 (0.45)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ferrer-Pardo, V.R.; Jimenez-Perez, I.; Gil-Calvo, M.; Pérez-Soriano, P.; Priego-Quesada, J.I. Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics. Educ. Sci. 2022, 12, 765. https://doi.org/10.3390/educsci12110765

AMA Style

Ferrer-Pardo VR, Jimenez-Perez I, Gil-Calvo M, Pérez-Soriano P, Priego-Quesada JI. Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics. Education Sciences. 2022; 12(11):765. https://doi.org/10.3390/educsci12110765

Chicago/Turabian Style

Ferrer-Pardo, Víctor Raul, Irene Jimenez-Perez, Marina Gil-Calvo, Pedro Pérez-Soriano, and Jose Ignacio Priego-Quesada. 2022. "Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics" Education Sciences 12, no. 11: 765. https://doi.org/10.3390/educsci12110765

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop