Assessing Massive Open Online Courses for Developing Digital Competences among Higher Education Teachers
Abstract
:1. Introduction
2. Materials and Methods
2.1. Overview of the Methodology
- Online course collection: A group of three (3) specialized scientists was created that would act as the reviewers of the online courses. Three digital platforms, namely Coursera, edX, and Udemy were selected to find available online courses. Udemy is often categorized as an MOOC platform, since it contains a very large number of online courses, so we chose to include it for the purposes of this study. These platforms were selected for our assessment because they are among the most popular worldwide. The search was conducted manually and was focused using particular keywords: “teaching” or “education” or “competences” or “pedagogy” or “instruction” or “online teaching” or “digital” or “course development” or “online tools” or “online classes” or “core competences” or “usage competences” or “online teaching strategies” or “ensuring positive online student experiences” or “assessment strategies” or “physical health” or “data protection” or “identity management” or “law” or “storage” or “search” or “active participation” or “collaboration” or “production and sharing” or “creation and change” or “understand” or “design and structure” or “achieve” or “digital delivery” or “structured learning” or “self-regulated learning” or “top-down” or “collaborative” or “inspire” or “reflection” or “knowledge gaps” or “formative” or “summative” or “scaffolding” or “group working” or “peer assessment” or “data analysis” or “identity material” or “modify material” or “communicate value” or “interactivity” or “engaging static content”.
- Online course assessment: A dedicated model (described in Section 2.1) was used to assess the collected online courses regarding their potential in covering the development of a range of digital competences of higher education teachers.
- Peer review scheme: Once an expert performed an assessment, two additional experts were asked to perform their own independent assessment of the same online courses in a blind manner (i.e., without knowing the initially assigned scores). In cases of differences in the assessment of a particular course, the median values were calculated and assigned as final scores. This review scheme was randomized so that the three experts had equal chances of serving in the roles of first, second, and third reviewer for each course.
- Final filtering: In cases where the consolidated experts’ opinion pointed out that an online course was relevant to computer science field and the teaching of at least one digital competence of higher education teachers, then the online course was considered for the final step of results analysis. Otherwise, if the course did not cover any of the digital competences, then it was excluded from the last and final step.
- Results analysis: The results of the assessment were subjected to statistical analysis so as to provide insights into the strengths and weaknesses of the available online courses for enhancing higher educators’ digital skills.
2.2. Online Course Assessment Model
- (i)
- Categories with five attributes (“Core Competences”, “Usage Core Competences”, “Ensuring Positive Online Student Experiences”, “Creating Digital Content”) can reach a maximum total score of 5 × 3 = 15, so the levels are defined as follows: 0 ≤ low < 5; 5 ≤ medium < 10; and 10 ≤ high ≤ 15.
- (ii)
- The “Online Teaching Strategies” category, which has six attributes, can reach a maximum total score of 6 × 3 = 18, so the levels are defined as follows: 0 ≤ low < 6; 6 ≤ medium < 12; and 12 ≤ high ≤ 18.
- (iii)
- The “Assessment Strategies” category, which has seven attributes, can reach a maximum total score of 7 × 3 = 21, so the levels are defined as follows: 0 ≤ low < 7; 7 ≤ medium < 14; and 14 ≤ high ≤ 21.
2.2.1. Descriptive Categories and Their Attributes
2.2.2. Assessment Category Attributes
3. Results
3.1. Online Course Collection
3.2. Assessment of Online Courses
3.2.1. Descriptive Category Attributes
3.2.2. Assessment Category Attributes
4. Discussion
5. Conclusions
- For policy-makers:
- o
- There is a noticeable lack of online courses that adequately align with current digital competence frameworks. Existing online courses fail to cover the wide range of necessary digital skills.
- o
- There is a lack of online courses explicitly targeted for the needs of higher educators (rather than general audiences).
- For higher education teachers and their institutes:
- o
- The top five digital competences addressed by existing online courses are ”data analysis”, “data protection”, “understand”, “digital delivery”, and “structured learning”, meaning that there are good chances of finding relevant online courses to cover them.
- o
- The bottom five digital competences addressed by existing online courses are: “physical health”, “storage”, “peer assessment”, “collaboration”, and “inspire”, meaning that there are small chances of finding relevant online courses to cover them.
- For developers of online courses and platforms offering massive open online courses:
- o
- The average degree of coverage of digital competences offered by existing online courses is low (ranging from 15 to 24, depending on the platform, on a scale from 0 to 99).
- o
- There is need for better coverage of both individual digital competences (see the bottom five competences identified above) as well as of entire categories (refer to Table 2), by developing new courses or expanding existing ones.
- o
- Digital skills falling into the categories of core competences, ensuring positive online student experiences, and assessment strategies are covered to a mostly low degree by existing online courses. Digital skills related to usage core competences and creating digital content are better-covered, although still to a low to medium degree. Digital skills belonging to the category of online teaching strategies are, overall, covered to a more satisfactory degree at the moment.
- o
- Innovative courses need to be meticulously crafted from the outset, specifically tailored for higher educators, and be based on comprehensive digital competence frameworks. Simultaneously, to address the fragmented nature of current courses, it is essential to establish learning pathways. These pathways, should combine different course collections for more effective skill enhancement.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Banović, A.; Mladenović, M.; Boljat, I. Informatics teacher’s perspective on distance learning during COVID-19 lockdown in Croatia. In Proceedings of the 14th Annual International Conference of Education, Research and Innovation, Online Conference, 8–9 November 2021. [Google Scholar]
- Cui, Y.; Ma, Z.; Wang, L.; Yang, A.; Liu, Q.; Kong, S.; Wang, H. A survey on big data-enabled innovative online education systems during the COVID-19 pandemic. J. Innov. Knowl. 2023, 8, 100295. [Google Scholar] [CrossRef]
- El Masri, A.; Sabzalieva, E. Dealing with disruption, rethinking recovery: Policy responses to the COVID-19 pandemic in higher education. Policy Des. Pract. 2020, 3, 312–333. [Google Scholar] [CrossRef]
- Karalis, T. Planning and evaluation during educational disruption: Lessons learned from COVID-19 pandemic for treatment of emergencies in education. Eur. J. Educ. Stud. 2020, 7, 125–142. [Google Scholar]
- Daniel, S.J. Education and the COVID-19 pandemic. Prospects 2020, 49, 91–96. [Google Scholar] [CrossRef] [PubMed]
- Aldulaimi, S.H.; Abdeldayem, M.M.; Keir, M.A.; Al-Sanjary, O. E-learning in higher education and COVID-19 outbreak: Challenges and opportunities. Psychol. Educ. J. 2021, 58, 38–43. [Google Scholar]
- Krishnamoorthy, R.; Keating, K. Education crisis, workforce preparedness, and COVID-19: Reflections and recommendations. Am. J. Econ. Sociol. 2021, 80, 253–274. [Google Scholar] [CrossRef] [PubMed]
- Edem Adzovie, D.; Jibril, A.B. Assessment of the effects of COVID-19 pandemic on the prospects of e-learning in higher learning institutions: The mediating role of academic innovativeness and technological growth. Cogent Educ. 2022, 9, 2041222. [Google Scholar] [CrossRef]
- Alsoud, A.R.; Harasis, A.A. The impact of COVID-19 pandemic on student’s e-learning experience in Jordan. J. Theor. Appl. Electron. Commer. Res. 2021, 16, 1404–1414. [Google Scholar] [CrossRef]
- Zhao, Y.; Llorente, A.M.P.; Gómez, M.C.S. Digital competence in higher education research: A systematic literature review. Comput. Educ. 2021, 168, 104212. [Google Scholar] [CrossRef] [PubMed]
- Núñez-Canal, M.; de Obesso, M.D.L.M.; Pérez-Rivero, C.A. New challenges in higher education: A study of the digital competence of educators in Covid times. Technol. Forecast. Soc. Chang. 2022, 174, 121270. [Google Scholar] [CrossRef]
- Tejedor, S.; Cervi, L.; Pérez-Escoda, A.; Jumbo, F.T. Digital literacy and higher education during COVID-19 lockdown: Spain, Italy, and Ecuador. Publications 2020, 8, 48. [Google Scholar] [CrossRef]
- Tondeur, J.; Howard, S.; Van Zanten, M.; Gorissen, P.; Van der Neut, I.; Uerz, D.; Kral, M. The HeDiCom framework: Higher Education teachers’ digital competencies for the future. Educ. Technol. Res. Dev. 2023, 71, 33–53. [Google Scholar] [CrossRef]
- Masoumi, D.; Noroozi, O. Developing early career teachers’ professional digital competence: A systematic literature review. Eur. J. Teach. Educ. 2023. [Google Scholar] [CrossRef]
- Smestad, B.; Hatlevik, O.E.; Johannesen, M.; Øgrim, L. Examining dimensions of teachers’ digital competence: A systematic review pre-and during COVID-19. Heliyon 2023, 9, e16677. [Google Scholar] [CrossRef] [PubMed]
- European Commission. Directorate-General for Education, Youth, Sport and Culture, Key competences for lifelong learning. Publications Office, 2019. Available online: https://data.europa.eu/doi/10.2766/569540 (accessed on 20 June 2023).
- Redecker Christine. European Framework for the Digital Competence of Educators: DigCompEdu. Available online: https://policycommons.net/artifacts/2163302/european-framework-for-the-digital-competence-of-educators/2918998/ (accessed on 20 June 2023).
- Coursera|Degrees, Certificates, & Free Online Courses. Available online: https://www.coursera.org (accessed on 28 July 2023).
- edX. edX|Build New Skills. Advance Your Career. Available online: https://www.edx.org (accessed on 28 July 2023).
- Udemy. Online Courses-Learn Anything. On Your Schedule|Udemy. Available online: https://www.udemy.com (accessed on 28 July 2023).
- Baturay, M.H. An overview of the world of MOOCs. Procedia-Soc. Behav. Sci. 2015, 174, 427–433. [Google Scholar] [CrossRef]
- Bitakou, E.; Demestichas, K.; Ntaliani, M.; Costopoulou, C. An assessment of online courses for supporting Greek higher education teachers’ digital competence. In Proceedings of the 15th International Conference on Education and New Learning Technologies (EDULEARN23), Palma de Mallorca, Spain, 3–5 July 2023. [Google Scholar]
- ADVICE Project–Advancing Digital Competence in Higher Education. Available online: https://advice.civitas.edu.pl (accessed on 27 August 2023).
- McClelland, M. Metadata standards for educational resources. Computer 2003, 36, 107–109. [Google Scholar] [CrossRef]
- Basilotta-Gómez-Pablos, V.; Matarranz, M.; Casado-Aranda, L.A.; Otto, A. Teachers’ digital competencies in higher education: A systematic literature review. Int. J. Educ. Technol. High. Educ. 2022, 19, 8. [Google Scholar] [CrossRef]
- Guillén-Gámez, F.D.; Cabero-Almenara, J.; Llorente-Cejudo, C.; Palacios-Rodríguez, A. Differential analysis of the years of experience of higher education teachers, their digital competence and use of digital resources: Comparative research methods. Technol. Knowl. Learn. 2022, 27, 1193–1213. [Google Scholar] [CrossRef]
- Krippendorff, K. Content Analysis: An Introduction to Its Methodology; Sage Publications: Los Angeles, CA, USA, 2019. [Google Scholar]
Assessment Category | Digital Skill | Online Courses That Teach This Digital Skill | |
---|---|---|---|
No. | % | ||
Core Competences | Physical health | 1 | 0.62 |
Data protection | 85 | 54.47 | |
Identity management | 18 | 11.11 | |
Law | 20 | 12.34 | |
Storage | 2 | 1.23 | |
Usage Core Competences | Search | 46 | 28.39 |
Active participation | 52 | 32.10 | |
Collaboration | 15 | 9.26 | |
Production and sharing | 40 | 24.69 | |
Creation and change | 32 | 19.75 | |
Online Teaching Strategies | Understand | 84 | 51.85 |
Design and structure | 74 | 45.68 | |
Achieve | 21 | 12.96 | |
Digital delivery | 83 | 51.23 | |
Structured learning | 78 | 48.15 | |
Self-regulated learning | 57 | 35.18 | |
Ensuring Positive Online Student Experiences | Top-down | 64 | 39.50 |
Collaborative | 18 | 11.11 | |
Inspire | 16 | 9.87 | |
Reflection | 17 | 10.49 | |
Knowledge gaps | 23 | 14.20 | |
Assessment Strategies | Understand | 19 | 11.73 |
Formative | 18 | 11.11 | |
Summative | 29 | 17.90 | |
Scaffolding | 24 | 14.81 | |
Group working | 20 | 12.34 | |
Peer assessment | 12 | 7.41 | |
Data analysis | 97 | 59.87 | |
Creating Digital Content | Identify material | 47 | 29.01 |
Modify material | 35 | 21.60 | |
Communicate value | 37 | 22.84 | |
Interactivity | 35 | 21.60 | |
Engaging static content | 25 | 15.43 |
Assessment Categories | Level of Coverage by Courses | |||||
---|---|---|---|---|---|---|
Low | Medium | High | ||||
No. | % | No. | % | No. | % | |
Core Competences | 139 | 85.80 | 23 | 14.20 | 0 | 0.00 |
Usage Core Competences | 116 | 71.60 | 35 | 21.60 | 12 | 7.41 |
Online Teaching Strategies | 75 | 46.30 | 65 | 40.12 | 22 | 13.58 |
Ensuring Positive Online Student Experiences | 143 | 88.27 | 16 | 9.88 | 3 | 1.85 |
Assessment Strategies | 146 | 90.12 | 15 | 9.26 | 1 | 0.62 |
Creating Digital Content | 114 | 70.37 | 44 | 27.16 | 4 | 2.47 |
Platform | Mean Score | Max Score | Min Score |
---|---|---|---|
Coursera | 24.02 | 56 | 12 |
edX | 15.61 | 48 | 2 |
Udemy | 16.83 | 49 | 2 |
Platforms | Level of Coverage by Courses | |||||
---|---|---|---|---|---|---|
Low | Medium | High | ||||
No. | % | No. | % | No. | % | |
Coursera | 28 | 17.28 | 8 | 4.94 | 0 | 0.00 |
edX | 24 | 14.81 | 2 | 1.23 | 0 | 0.00 |
Udemy | 90 | 55.56 | 10 | 6.17 | 0 | 0.00 |
Total | 142 | 87.65 | 20 | 12.35 | 0 | 0.00 |
Correlations | |||||||
---|---|---|---|---|---|---|---|
Core Competences | Usage Core Competences | Online Teaching Strategies | Ensuring Positive Online Student Experiences | Assessment Strategies | Creating Digital Content | ||
Core Competences | Pearson Correlation | 1 | −0.580 ** | −0.528 ** | −0.234 ** | −0.258 ** | −0.470 ** |
Sig. (2-tailed) | 0.000 | 0.000 | 0.003 | 0.001 | 0.000 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 | |
Usage Core Competences | Pearson Correlation | −0.580 ** | 1 | 0.591 ** | 0.571 ** | 0.293 ** | 0.553 ** |
Sig. (2-tailed) | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 | |
Online Teaching Strategies | Pearson Correlation | −0.528 ** | 0.591 ** | 1 | 0.572 ** | 0.255 ** | 0.530 ** |
Sig. (2-tailed) | 0.000 | 0.000 | 0.000 | 0.001 | 0.000 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 | |
Ensuring Positive Online Student Experiences | Pearson Correlation | −0.234 ** | 0.571 ** | 0.572 ** | 1 | 0.109 | 0.278 ** |
Sig. (2-tailed) | 0.003 | 0.000 | 0.000 | 0.165 | 0.000 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 | |
Assessment Strategies | Pearson Correlation | −0.258 ** | 0.293 ** | 0.255 ** | 0.109 | 1 | 0.266 ** |
Sig. (2-tailed) | 0.001 | 0.000 | 0.001 | 0.165 | 0.001 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 | |
Creating Digital Content | Pearson Correlation | −0.470 ** | 0.553 ** | 0.530 ** | 0.278 ** | 0.266 ** | 1 |
Sig. (2-tailed) | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 | ||
N | 162 | 162 | 162 | 162 | 162 | 162 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bitakou, E.; Ntaliani, M.; Demestichas, K.; Costopoulou, C. Assessing Massive Open Online Courses for Developing Digital Competences among Higher Education Teachers. Educ. Sci. 2023, 13, 900. https://doi.org/10.3390/educsci13090900
Bitakou E, Ntaliani M, Demestichas K, Costopoulou C. Assessing Massive Open Online Courses for Developing Digital Competences among Higher Education Teachers. Education Sciences. 2023; 13(9):900. https://doi.org/10.3390/educsci13090900
Chicago/Turabian StyleBitakou, Effrosyni, Maria Ntaliani, Konstantinos Demestichas, and Constantina Costopoulou. 2023. "Assessing Massive Open Online Courses for Developing Digital Competences among Higher Education Teachers" Education Sciences 13, no. 9: 900. https://doi.org/10.3390/educsci13090900