Assessing Massive Open Online Courses for Developing Digital Competences among Higher Education Teachers

Abstract

The outstanding growth in digital technologies has raised higher education teachers’ attention towards developing digital competences for effectively fulfilling students’ learning requirements and keeping up with the rapidly evolving global education system. Nowadays, higher education teachers can find a plethora of online courses provided by educational organizations, universities, and businesses for reskilling and upskilling. The objective of this paper is to provide results from the assessment of existing massive open online courses regarding their potential to cover a large range of digital competences that higher education teachers should possess. A total of one hundred and sixty-two (162) online courses from three prominent international digital platforms, offering massive open online courses, were subjected to evaluation using a model incorporating six categories of attributes aligned with pertinent digital competences. The results indicate that the majority of the existing online courses lack sufficient coverage of key digital competences required by higher education teachers. Notably, these inadequately addressed competences regard evaluating students’ performance in the digital environment and ensuring positive online student experiences through collaborative engagement and self-reflection. The findings of this study offer valuable insights not only to higher education teachers, but also to online course developers and education policy-makers. The identified gaps in digital competences underscore the pressing need for substantial improvements in existing online courses or the development of new courses to effectively bridge these competence gaps. By addressing these shortcomings, higher education institutions can better equip their teachers to navigate the digital realm and elevate the overall quality of education in the digital era.

1. Introduction

The COVID-19 outbreak had a significant impact on educational systems across the world [1,2] by disrupting educational activities [3,4] and creating severe challenges for learners and educators [5]. With the support of Information and Communication Technology (ICT), education has experienced an unprecedented shift towards digital learning platforms, electronic learning, and distance education [6], used as countermeasures during this destabilizing learning crisis [7].

In higher education, this digital transformation was, in many cases, abrupt, but serving as the only alternative for continuing academic activities [8]. Digital tools, such as teleconferencing and online lessons, were used to replace face-to-face practices for teaching, exams, communication, and collaboration among students and academics [9]. Given that twenty-first-century-generation students have experienced the rapid development of digital technology (e.g., Internet of Things, virtual reality, and artificial intelligence), the emphasis on digital competence in higher education continues to grow in popularity [10]. Higher education teachers need a comprehensive set of competences in a range of areas in order to make digital education for students equal and effective. Under these circumstances, current studies make evident that higher education teachers’ digital competence cannot longer be considered only as a tool, but also as an essential pedagogical element [11], and a key point in new learning scenarios [12]. It is noteworthy that there is a positive relation between the development of teachers’ digital competence with students’ digital competence quality [13].

Different terms, such as digital literacy, ICT literacy, and digital skills, have been used to discuss digital competence in education. The competences and skills necessary to comprehend and apply digital technology in educational practices have more recently been referred as professional digital competence. There are various theoretical views and conceptual frameworks that describe how digital technologies are used in educational activities and that define the attributes of instructors who are digitally competent [14]. The state-of-the-art of digital competence among university teachers, and the training needed for enhancing teachers’ digital competence are discussed in recent review articles. Nevertheless, there is a lack of understanding regarding the conceptualization of teachers’ digital competence inside the research literature. More specifically, teachers’ digital competence has frequently been defined and articulated in terms of the practical user abilities that are regarded as necessary to operate a digital classroom. This is why the term “skills” is more often mentioned explicitly by research works that deal with teachers’ digital competence, while “knowledge” and “attitude” come second and third in popularity. Nonetheless, there has been a small increase in publications referring to “attitude” during COVID-19, which shows how the increased usage of technology during COVID-19 appears to have impacted researchers’ interest in instructors’ attitudes about technology. Teachers’ attitudes may have changed as a result of being required to utilize technology, yet for some, this may have had a motivating effect [15].

The European Commission (EC) defines the term digital competence as “the confident, critical and responsible use of the technologies from the society of information for work, entertainment and education”. Also, the EC has included digital and technology-based competences in the eight key competences of lifelong learning, along with literacy; multilingualism; numerical, scientific, and engineering skills; interpersonal skills and the ability to adopt new competences; active citizenship; entrepreneurship; and cultural awareness and expression [16]. Moreover, the EC has developed the European Framework for the Digital Competence of Educators (DigCompEdu). The framework proposes 22 competences categorized in six areas, namely professional engagement, digital resources, teaching and learning, assessment, empowering learners, and facilitating learners’ digital competence. It aims at educators at all levels of formal education, as well as vocational education and training, special needs education, and non-formal learning contexts, to support them in developing educator-specific digital competences. It acknowledges that the innovation of education requires educators to master a set of digital competences specific to their profession [17].

Based on all of the above, we can deduce that the current streams of research in the area of digital competences have shed light on numerous aspects, including the following: (a) Definitions and frameworks: digital competences extend beyond mere technological proficiency, encompassing a range of abilities from using digital tools to critically evaluating digital information and understanding online ethics. Renowned frameworks, such as the DigCompEdu mentioned above, offer structured categorizations, breaking down digital competence into key components like information literacy, safety, and problem-solving. (b) Importance in modern society: research has consistently shown the central role of digital competences in various sectors, from education and employment to civic participation. Those lacking these competences are at a distinct disadvantage, enhancing the digital divide. (c) Assessment and Metrics: efforts have been made to measure digital competence, with several tools and rubrics developed. These assessments typically consider a blend of theoretical understanding and practical skills.

In parallel, according to the cited literature, certain topics in this area are still insufficiently explored: (a) Long-term evolution: as technology continuously evolves, it is challenging to predict the future requirements concerning digital competences. Existing frameworks might be less suited to address future digital landscapes, particularly with advancements like quantum computing or augmented reality on the horizon. (b) Universal and equal access: while the importance of digital competences is recognized, research has yet to offer comprehensive solutions for global disparities in digital access and education. The true depth of the digital divide, particularly in under-researched areas and demographics, remains a puzzle. (c) Integration challenges in education: while there is a consensus on the need for digital competences in curricula, the most effective ways to integrate them across diverse educational contexts, age groups, and disciplines remain underexplored.

In light of the above, our present study investigates the adequacy of existing massive open online courses (MOOCs) in equipping higher education teachers with sufficient teaching digital competences, thus contributing to the investigation of topics related to universal and equal access as well as to integration in education, as defined above. In particular, in recent years, there has been a significant increase in the availability of online courses covering a wide range of subjects, providing educators with ample opportunities to develop their digital competences. There are various online platforms, like Coursera [18], edX [19] and Udemy [20], providing online courses. Early in 2012, Coursera emerged as an independent for-profit technology. Several independent non-profit initiatives like Udacity and Udemy were created in the same year. Following that, edX adopted the MITx platform created by MIT and Harvard [21]. At present, with either free or paid access, the Coursera platform provides more than 7.000 online courses, edX provides over 4.000, and Udemy provides over 210.000. These platforms cover different subjects, such as health, science, and information technology. Typically, individuals enrolling in an online course are often required to pay for additional benefits.

Although there are available online courses and resources for the development of digital competences, there is preliminary, yet still limited, evidence (for instance, refer to our study from Greece [22]) suggesting that these are not provided in a structured and integrated way to support higher education teachers in identifying specific digital competence gaps and achieve efficient upskilling. The objective of this paper is to present the outcomes of an assessment study conducted on existing massive open online courses to evaluate their potential in addressing the diverse digital competences essential for higher education teachers. More specifically, the focus of this paper is to provide answers to the following questions regarding the digital competences of higher education teachers:

RQ1: Which digital competences of higher education teachers are better covered by the available online courses?

RQ2: To what extent do existing online courses cover the development of the broad spectrum of digital competences required by higher education teachers?

To the best of our knowledge, this is the first assessment study reported in the literature that investigates the potential and relevance of massive open online courses in covering the needs of higher education teachers in terms of digital competences. The structure of this paper is as follows: Section 2 analytically presents the methodology followed, as well as the model used for the assessment of online courses. Section 3 presents the results of the assessment of online courses. Finally, concluding remarks and future work are provided in Section 4.

2. Materials and Methods

2.1. Overview of the Methodology

In this section, the steps followed, as well as the model used for the assessment of online courses, are presented in detail. More specifically, as depicted in Figure 1, the design of this evaluation study comprised the following steps:

• Online course collection: A group of three (3) specialized scientists was created that would act as the reviewers of the online courses. Three digital platforms, namely Coursera, edX, and Udemy were selected to find available online courses. Udemy is often categorized as an MOOC platform, since it contains a very large number of online courses, so we chose to include it for the purposes of this study. These platforms were selected for our assessment because they are among the most popular worldwide. The search was conducted manually and was focused using particular keywords: “teaching” or “education” or “competences” or “pedagogy” or “instruction” or “online teaching” or “digital” or “course development” or “online tools” or “online classes” or “core competences” or “usage competences” or “online teaching strategies” or “ensuring positive online student experiences” or “assessment strategies” or “physical health” or “data protection” or “identity management” or “law” or “storage” or “search” or “active participation” or “collaboration” or “production and sharing” or “creation and change” or “understand” or “design and structure” or “achieve” or “digital delivery” or “structured learning” or “self-regulated learning” or “top-down” or “collaborative” or “inspire” or “reflection” or “knowledge gaps” or “formative” or “summative” or “scaffolding” or “group working” or “peer assessment” or “data analysis” or “identity material” or “modify material” or “communicate value” or “interactivity” or “engaging static content”.
• Online course assessment: A dedicated model (described in Section 2.1) was used to assess the collected online courses regarding their potential in covering the development of a range of digital competences of higher education teachers.
• Peer review scheme: Once an expert performed an assessment, two additional experts were asked to perform their own independent assessment of the same online courses in a blind manner (i.e., without knowing the initially assigned scores). In cases of differences in the assessment of a particular course, the median values were calculated and assigned as final scores. This review scheme was randomized so that the three experts had equal chances of serving in the roles of first, second, and third reviewer for each course.
• Final filtering: In cases where the consolidated experts’ opinion pointed out that an online course was relevant to computer science field and the teaching of at least one digital competence of higher education teachers, then the online course was considered for the final step of results analysis. Otherwise, if the course did not cover any of the digital competences, then it was excluded from the last and final step.
• Results analysis: The results of the assessment were subjected to statistical analysis so as to provide insights into the strengths and weaknesses of the available online courses for enhancing higher educators’ digital skills.

2.2. Online Course Assessment Model

The model employed in this study was based on the DigCompEdu framework and was developed in the context of the “Advancing Digital Competence in Higher Education-ADVICE” Erasmus + project for supporting higher education teachers in tackling their digital competence gaps and mismatches [23]. The aim of the model is to assess available online courses in terms of their ability to develop a range of digital competences required by higher education teachers for making digital education more accessible and effective for their students.

More specifically, the model involves fifty-five (55) attributes for the description and assessment of each online course. These attributes are categorized in fifteen (15) categories, namely nine (9) descriptive categories and six (6) assessment categories. In essence, the attributes of the descriptive categories represent specific key meta-data of an online course, while the attributes of the assessment categories represent key digital competences that a higher education teacher might need to develop.

The descriptive categories are the following: (1) content description, (2) subject, (3) intellectual property, (4) identifiers, (5) resource, (6) language, (7) audience, (8) sources and relations, and (9) rating. These descriptive categories include twenty-two (22) attributes in total; in each one, the experts are asked to record specific meta-data regarding the online course they are viewing.

The assessment categories are the following (for ease of reference, we use consecutive numbering): (10) core competences, (11) usage core competences, (12) online teaching strategies, (13) ensuring positive online student experiences, (14) assessment strategies, and (15) creating digital content. These assessment categories include thirty-three (33) attributes in total. In each of these attributes and for each online course under investigation, the experts are asked to provide an assessment, using a scale from 0 to 3, whereby 0 represents no coverage of the development of the corresponding digital skill, 1 represents low coverage, 2 represents medium coverage, and 3 represents high coverage.

Consequently, by summing up the individual scores of its attributes, each of the assessment categories can be characterized as low, medium, or high coverage. Low coverage means that a category of digital competences is not satisfactorily covered or not covered at all in the online course, whereas high coverage means that a category of digital competences is very satisfactorily or fully covered in the online course. Since the assessment categories have different numbers of attributes, the levels of coverage are defined as follows:

(i)

Categories with five attributes (“Core Competences”, “Usage Core Competences”, “Ensuring Positive Online Student Experiences”, “Creating Digital Content”) can reach a maximum total score of 5 × 3 = 15, so the levels are defined as follows: 0 ≤ low < 5; 5 ≤ medium < 10; and 10 ≤ high ≤ 15.

(ii)

The “Online Teaching Strategies” category, which has six attributes, can reach a maximum total score of 6 × 3 = 18, so the levels are defined as follows: 0 ≤ low < 6; 6 ≤ medium < 12; and 12 ≤ high ≤ 18.

(iii)

The “Assessment Strategies” category, which has seven attributes, can reach a maximum total score of 7 × 3 = 21, so the levels are defined as follows: 0 ≤ low < 7; 7 ≤ medium < 14; and 14 ≤ high ≤ 21.

Finally, since the assessment categories include 33 attributes in total, the total score that an online course can gain ranges from 0 to 99. Thus, an online course can be characterized as low, medium, or high coverage, based on the sum of the individual scores of these attributes (0 ≤ low < 33; 33 ≤ medium < 66; 66 ≤ high ≤ 99).

2.2.1. Descriptive Categories and Their Attributes

In the following, the attributes of each descriptive category are analyzed in detail. The selection of the attributes included in our model was based upon existing meta-data standards, such as the Dublin Core Metadata Element Set and the IEEE Learning Object Metadata (LOM) [24].

(1) Content Description

Title: the descriptive name given to an online course (e.g., course/unit/lecture, certificate) to identify or describe it.

Abstract: a summary providing a general description and the most important information about the course.

(2) Subject

Broad: a wide area of knowledge that the course belongs to (e.g., Law, Art).

Narrow: the specific subject that the course belongs to (e.g., Data Protection Legislation).

(3) Intellectual Property

Creator: the person that has created the course.

Contributor: someone who has contributed to the course (e.g., in the form of additional content, articles, reviews, etc.).

Publisher: person or organization who is responsible for publishing the course online.

License type: the type of license of the course (e.g., commercial, public domain).

License cost: each course can be characterized as “free”, “paid”, or “mixed”. The “mixed” attribute refers to a combination of “free” and “paid” courses.

(4) Identifiers

Identifier: a Uniform Resource Locator (URL), commonly known as Web address, is the unique identifier used to locate a course on the Internet.

Alternative identifier: an additional URL or other type of unique code or number which can be used to uniquely refer to the course.

(5) Resource

Type: the types of learning resources that are included in an online course. These types are article/reference, assessment (e.g., exam, quiz), course/unit/lecture, image/visuals/PowerPoint, software, textbook, or other.

Format: the format of learning resources that are included in an online course. These are document, image, audio, video, multimedia, or other.

Length: the duration of the online course in various metrics, e.g., teaching hours and video projections, the expected duration in weeks and months, and the number of sections, lectures, and questions.

(6) Language

Language: the language in which the course is provided (e.g., English, French, etc.).

(7) Audience

Audience: the groups of people that the course targets. In the present study, there are two types of audiences: “higher level educators” and “general public”, excluding students at schools.

Education level: the level of difficulty of the course, namely beginner, intermediate, advanced, or other.

(8) Sources and Relations

Requires: specific background knowledge/skills that participants should have to successfully attend the course.

References: any information necessary for identifying and retrieving relevant works.

(9) Rating

User rating: the average evaluation of the course by people who have already attended it. This attribute typically uses a five-star rating system.

Official rating: a rating provided by an official authority, e.g., a quality assurance organization, a peer reviewer, a university, etc.

Other rating: any other type of rating provided for this course that does not belong to the two aforementioned types.

2.2.2. Assessment Category Attributes

In the following, the attributes of each assessment category are analyzed in detail. In principle, each attribute represents a different digital skill that higher education teachers need to learn and that an online course might teach (to some extent, on a scale from 0 to 3). As noted in the beginning of Section 2.2, the model used was developed by the ERASMUS+ Project “ADVICE” [23], as an extension and adaptation of the DigCompEdu framework of the European Commission.

(10) Core Competences: digital skills related to safe digital environment navigation.

Physical health: does the course teach how to maintain physical health (e.g., protecting eyesight) when participating in digital learning?

Data protection: does the course teach how to ensure the protection of third-party data?

Identity management: does the course teach how to ensure the participant’s online identity?

Law: does the course teach how to ensure data protection and privacy?

Storage: does the course teach how to store data and information securely?

(11) Usage Core Competences: digital skills for retrieving and adapting information and collaborating with others online.

Search: does the course teach how to find and identify required information online?

Active participation: does the course teach how to engage and participate in online communities?

Collaboration: does the course teach how to work with colleagues in collaborative online platforms (e.g., Slack, MS Teams, etc.)?

Production and sharing: does the course teach how to create digital resources to present ideas online?

Creation and change: does the course teach how to adapt self-created digital resources to different audience needs (e.g., improve their accessibility)?

(12) Online Teaching Strategies: digital skills related to teaching strategies in online digital spaces.

Understand: does the course teach how to introduce new online teaching methods? (In particular, this refers to the adaptation of teaching methods according to the specific objectives of the course.)

Design and structure: does the course teach how to structure lessons or teaching processes in general?

Achieve: does the course teach how to deliver hybrid (online and offline) lessons?

Digital delivery: does the course teach how to adapt digital lessons to different types of students?

Structured learning: does the course teach how to structure online lessons?

Self-regulated learning: does the course teach how to plan lessons, monitor, and reflect on their delivery, and use this reflection to inform the planning of the next lesson?

(13) Ensuring Positive Online Student Experiences: digital skills for improving the experience of online students.

Top-down: does the course teach how to create interactive and interesting lectures online for students?

Collaborative: does the course teach how to collaborate with students in an online space?

Inspire: does the course teach how to build safe online spaces for students?

Reflection: does the course teach how to encourage positive behaviors in students in academic spaces (such as self-refection, self-critique, and discipline)?

Knowledge gaps: does the course teach how to identify gaps in students’ knowledge using online tools?

(14) Assessment Strategies: digital skills that help participants to assess their students in the digital environment.

Understand: does the course teach how to assess students’ work/comprehension of the topics online?

Formative: does the course teach how to monitor students’ work and offer meaningful feedback in the digital environment?

Summative: does the course teach how to test student skills and knowledge?

Scaffolding: does the course teach how to provide the appropriate amount of online support to students to enable them to independently complete tasks beyond their initial capacity?

Group working: does the course teach how to evaluate student group work online?

Peer assessment: does the course teach how to encourage student peer review and assessment?

Data analysis: does the course teach how to analyze and assess different students’ learning behaviors?

(15) Creating Digital Content: digital skills that help participants to develop digital content for online lessons.

Identify material: does the course teach how to identify useful online and offline digital content?

Modify material: does the course teach how to modify pre-existing digital content?

Communicate value: does the course teach how to communicate the value of content to students in the online environment?

Interactivity: does the course teach how to develop interactive online content for students (e.g., interactive models)?

Engaging static content: does the course teach how to transform static content into interactive content (e.g., book excerpts)?

3. Results

3.1. Online Course Collection

The evaluation study was conducted from August 2021 to December 2022. From the search of online courses, in total, one hundred and sixty-two (162) online courses were found to be relevant. Specifically, twenty-six (26) courses were found inside the edX platform, one hundred (100) inside Udemy, and thirty-six (36) inside Coursera. Three reviewers from Greece participated in the evaluation of the online courses and were asked to use and apply the aforementioned assessment model (described in Section 2.2). The experts used Excel files and Google sheets to fill in the values of the attributes of the descriptive and assessment categories. In the following, the results regarding each attribute of the descriptive and assessment categories are presented in detail.

3.2. Assessment of Online Courses

3.2.1. Descriptive Category Attributes

(1) Content Description

Title: courses are related to strategies for online teaching and learning (50), data protection (32), privacy (18), identity management (10), cyber security (6), and the rest (46) to other relevant topics.

Abstract: all online courses provide an abstract.

(2) Subject

Broad: broad subjects include education and teacher training courses (46); information technology (IT) and software (27); social sciences (20); law (10); business strategy (9); computer science (9); network and security (9); data analysis and statistics (8); management (6); database design, app and web development (5); operating systems and servers (3); personal productivity (2); software engineering (2); e-commerce (1); communication courses (1); programming languages (1); career development (1); general health (1); and social media marketing (1).

Narrow: most of the courses (98) have indicated specific narrow subjects. These subjects refer to several different topics, with the majority focusing on online teaching (30) and data protection (15).

(3) Intellectual Property

Creator: all courses provide the name of the creator (professor, university, or company).

Contributor: all courses provide the name of the contributor (professor, university, or company).

Publisher: the name of the publisher is typically the platform name (edX, Udemy, Coursera).

License type: only two courses explicitly provide this type of information (commercial license).

License cost: courses are either “paid” (84), “free” (38), or “mixed” (40).

(4) Identifiers

Identifier: all courses have a Web address (URL).

Alternative identifier: no alternative identifiers were found for the courses under investigation.

(5) Resource

Type: the types of learning resources included in courses are courses/units/lectures (158) and assessments (e.g., exam, quiz) (4).

Format: the formats are multimedia (138), videos (20), documents (3), and other (1).

Length: the majority of courses (138) provided this information in number of hours, while some courses (82) provided this information in number of modules, as well.

(6) Language

Language: courses are provided mainly in English (161), and, additionally, in French (11), German (10), Italian (11), Portuguese (11), Spanish (10), Polish (10), Russian (9), Arabic (8), Vietnamese (8), Indonesian (2), Romanian (2), Greek (1), Japanese (1), and Thai (1).

(7) Audience

Audience: courses are mainly addressed to the general public (149), while a few (13) are specifically targeted to higher-level educators.

Education level: most courses (120) target the beginners’ level, while fewer target the intermediate level (37) or the advanced level (3).

(8) Sources and Relations

Requires: in total, 120 courses require basic knowledge of information technology; the rest do not have any prerequisites.

References: no course explicitly provides this information.

(9) Rating

User rating: only courses provided by the Udemy (99) and Coursera (34) platforms provide a five-star rating system. The max rating noted is 4.8 and the min rating is 4.1.

Official rating: no course provides this value.

Other rating: no course provides this value.

3.2.2. Assessment Category Attributes

As already mentioned, in each attribute of the assessment categories, an online course was evaluated using a scale from 0 to 3. A value of 0 suggests that the online course is not relevant to the teaching of the corresponding digital skill, while a value of 1, 2, or 3 suggests that the online course is relevant to the teaching of that digital skill to a low (1), medium (2), or high (3) degree, respectively.

The key results of this study are presented through the following bar charts. In particular, Figure 2 presents the assessment of the attributes of the core competences category. It can be noted that, with the exception of the “data protection” attribute or digital skill, the rest of the attributes of the core competences category are not adequately covered by the offered courses. In particular, approximately half of the investigated courses cover the teaching of data protection to either a high (38.9%) or medium (11.7%) degree. The skill of “identity management” is covered by 9.9% of the courses to a high degree and by 13% of the courses to a medium degree. The remaining attributes (physical health, storage, and law) are covered by only a small part of the courses (for instance, 0.6% in the case of physical health).

Figure 3 presents the assessment of the attributes of the usage core competences category. It shows that the majority of the courses does not sufficiently cover the skills in this category. This is especially true for the “collaboration” skill, which is covered by less than 1 out of 10 courses. On the other hand, “search” and “active participation” are the best-covered skills in this category, with approximately 3 out of 10 courses being relevant (to a medium or high degree) to their teaching.

Figure 4 presents the assessment of the attributes in the online teaching strategies category. It can be noted that, in particular, the “achieve” skill is not sufficiently covered, in comparison with the others. On the other hand, “digital delivery”, “understand”, and “structured learning” are among the best-covered skills in this category, with approximately one out of two courses being relevant (to a medium or high degree) to their teaching.

Figure 5 presents the assessment of the attributes related to ensuring positive online student experiences. It shows that the majority of the courses does not sufficiently cover the skills in this category. This is especially true for the “collaborative”, “inspire”, “reflection”, and “knowledge gaps” skills, which are covered by less than 10–15% of the investigated courses. On the other hand, the “top-down” skill is the most widely covered in this category, with approximately 4 out of 10 courses being relevant (mostly to a medium degree) to its teaching.

Figure 6 presents the assessment of the attributes in the assessment strategies category. It shows that the majority of the courses does not sufficiently cover the skills in this category. This is especially true for the “understand”, “formative”, “summative”, “scaffolding”, “group working”, and “peer assessment” skills. On the other hand, the “data analysis” skill is the most widely covered in this category, with one out of two courses being relevant (to a medium or high degree) to its teaching.

Figure 7 presents the assessment of the attributes related to the creating digital content category. It shows that the majority of the courses does not sufficiently cover the skills in this category. This is especially true for the “engaging static content” skill. On the other hand, the “identify material” skill is the best-covered in this category, with almost 3 out of 10 courses being relevant (to a medium or high degree) to its teaching.

For each digital skill,

Table 1. Provision of assessment category attributes.

Assessment Category	Digital Skill	Online Courses That Teach This Digital Skill
		No.	%
Core Competences	Physical health	1	0.62
	Data protection	85	54.47
	Identity management	18	11.11
	Law	20	12.34
	Storage	2	1.23
Usage Core Competences	Search	46	28.39
	Active participation	52	32.10
	Collaboration	15	9.26
	Production and sharing	40	24.69
	Creation and change	32	19.75
Online Teaching Strategies	Understand	84	51.85
	Design and structure	74	45.68
	Achieve	21	12.96
	Digital delivery	83	51.23
	Structured learning	78	48.15
	Self-regulated learning	57	35.18
Ensuring Positive Online Student Experiences	Top-down	64	39.50
	Collaborative	18	11.11
	Inspire	16	9.87
	Reflection	17	10.49
	Knowledge gaps	23	14.20
Assessment Strategies	Understand	19	11.73
	Formative	18	11.11
	Summative	29	17.90
	Scaffolding	24	14.81
	Group working	20	12.34
	Peer assessment	12	7.41
	Data analysis	97	59.87
Creating Digital Content	Identify material	47	29.01
	Modify material	35	21.60
	Communicate value	37	22.84
	Interactivity	35	21.60
	Engaging static content	25	15.43

presents the number of online courses that are relevant to the teaching of the particular digital skill, which means that they have achieved a score from one to three (i.e., above zero).

As may be observed, the digital skill that is covered by most of the courses is “data analysis” (59.87%). Moreover, about half of the courses cover the development of the following digital skills: “data protection” (54.47%); “understand” (51.85%) and “design and structure” (45.68%) from the online teaching strategies category; “digital delivery” (51.23%); and “structured learning” (48.15%). The least-addressed digital competences, namely those that are covered by less than 10% of courses, are as follows: “physical health” (0.62%); “storage” (1.23%); “peer assessment” (7.41%); “collaboration (9.26%); and ”inspire” (9.87%).

Table 2. Courses by coverage level for each of the assessment categories.

Assessment Categories	Level of Coverage by Courses
	Low		Medium		High
	No.	%	No.	%	No.	%
Core Competences	139	85.80	23	14.20	0	0.00
Usage Core Competences	116	71.60	35	21.60	12	7.41
Online Teaching Strategies	75	46.30	65	40.12	22	13.58
Ensuring Positive Online Student Experiences	143	88.27	16	9.88	3	1.85
Assessment Strategies	146	90.12	15	9.26	1	0.62
Creating Digital Content	114	70.37	44	27.16	4	2.47

shows, for each of the assessment categories, the number and percentage of online courses, split in terms of coverage level (low/medium/high). As mentioned, a category is characterized to provide a low, medium, or high level of coverage, based on the sum of the individual scores of its attributes.

According to Table 2, all of the assessment categories are covered by the courses we investigated, at least to a low degree. With the exception of the online teaching strategies, all the other assessment categories are characterized by a low level of coverage. Specifically, the category of online teaching strategies is the one addressed in the most sufficient manner compared with the rest, and presents 46.30% low coverage, 40.12% medium coverage, and 13.58% high coverage. Notably, the category of core competences does not present high coverage at all.

Subsequently, for each online course, we calculated the total score of the course as the sum of the attribute scores. As explained in Section 2.2, based on this total score, a course can be considered to provide a low, medium, or high coverage of digital competences needed by higher education teachers (0 ≤ low < 33; 33 ≤ medium < 66; 66 ≤ high ≤ 99).

Table 3. Mean, maximum, and minimum of total scores of courses by platform.

Platform	Mean Score	Max Score	Min Score
Coursera	24.02	56	12
edX	15.61	48	2
Udemy	16.83	49	2

depicts the mean, maximum, and minimum total scores of all courses by platform.

According to the mean score in Table 3, the courses provided by the three platforms present, on average, a low overall degree of coverage of digital competences. The maximum total score, which is 56, is presented by only one course offered through the Coursera platform. The minimum total score, which is two, is presented by two courses, one offered through the edX platform and one through Udemy.

Table 4. Number of courses by coverage level for each platform.

Platforms	Level of Coverage by Courses
	Low		Medium		High
	No.	%	No.	%	No.	%
Coursera	28	17.28	8	4.94	0	0.00
edX	24	14.81	2	1.23	0	0.00
Udemy	90	55.56	10	6.17	0	0.00
Total	142	87.65	20	12.35	0	0.00

shows, for each of the three platforms, the number and percentage of courses by coverage level.

In total, there are no courses presenting a high degree of coverage of digital competences. Moreover, 87.65% of the courses are characterized by low coverage, whereas the remaining 12.35% are characterized by medium coverage.

Subsequently, a correlation analysis was performed, employing the Pearson correlation coefficient, to investigate whether there is a relationship among the assessment categories. According to

Table 5. Correlation of assessment categories.

Correlations
		Core Competences	Usage Core Competences	Online Teaching Strategies	Ensuring Positive Online Student Experiences	Assessment Strategies	Creating Digital Content
Core Competences	Pearson Correlation	1	−0.580 **	−0.528 **	−0.234 **	−0.258 **	−0.470 **
	Sig. (2-tailed)		0.000	0.000	0.003	0.001	0.000
	N	162	162	162	162	162	162
Usage Core Competences	Pearson Correlation	−0.580 **	1	0.591 **	0.571 **	0.293 **	0.553 **
	Sig. (2-tailed)	0.000		0.000	0.000	0.000	0.000
	N	162	162	162	162	162	162
Online Teaching Strategies	Pearson Correlation	−0.528 **	0.591 **	1	0.572 **	0.255 **	0.530 **
	Sig. (2-tailed)	0.000	0.000		0.000	0.001	0.000
	N	162	162	162	162	162	162
Ensuring Positive Online Student Experiences	Pearson Correlation	−0.234 **	0.571 **	0.572 **	1	0.109	0.278 **
	Sig. (2-tailed)	0.003	0.000	0.000		0.165	0.000
	N	162	162	162	162	162	162
Assessment Strategies	Pearson Correlation	−0.258 **	0.293 **	0.255 **	0.109	1	0.266 **
	Sig. (2-tailed)	0.001	0.000	0.001	0.165		0.001
	N	162	162	162	162	162	162
Creating Digital Content	Pearson Correlation	−0.470 **	0.553 **	0.530 **	0.278 **	0.266 **	1
	Sig. (2-tailed)	0.000	0.000	0.000	0.000	0.001
	N	162	162	162	162	162	162

** Correlation is significant at the 0.01 level (2-tailed).

, ensuring positive online student experiences is not related with assessment strategies. All the other categories are related, either in a weak (i.e., where the Pearson coefficient ranges from 0.25 to 0.50) or moderate manner (i.e., where the Pearson coefficient ranges from 0.50 to 0.75). For instance, there is a moderate positive relation between the online teaching strategies and the usage core competences, suggesting that courses that perform well in the development of online teaching strategies also do well in the development of usage core competences, and vice versa. Notably, there is a negative relation (of weak or moderate strength) among core competences and all the other categories.

4. Discussion

Higher education teachers are faced with a complex, dynamic, and uncertain educational panorama, in which digital competences are considered as key competences that they have to master to further evolve their teaching [25,26].

This work has presented the results from assessing existing massive open online courses provided by popular international platforms, so as to investigate whether they cover a range of digital competences for higher education teachers. In total, one hundred and sixty-two (162) courses were assessed. The majority of them are courses/units/lectures, supported with learning resources in multimedia formats, and they are targeted to the general public. Also, most of the courses are mainly targeted for beginners’ and require basic knowledge of information technology. More than half require payment for enrollment and/or accessing more modules.

According to the results, the majority of the online courses (87.65%) is characterized by a low degree of coverage of key digital competences, and the rest (12.35%) by a medium degree of coverage, meaning that they do not sufficiently cover the broad range of digital competences required by higher education teachers. Notably, we identified no courses presenting a high degree of coverage of digital competences. Although all of the assessment categories are covered by the courses, at least to a low degree, the core competences, usage core competences, ensuring positive online experiences, assessment strategies, and creating digital content categories are characterized by low coverage, which denotes that these categories are not satisfactorily covered or not covered at all in the courses under study. The least-covered categories are core competences and assessment strategies. On the other hand, the category of digital competences that is most covered is online teaching strategies, presenting a high coverage of 13.58%.

As far as the specific digital skills (attributes) are concerned, none of them are covered by more than 60% of the courses. The attribute that is covered by most of the courses is “data analysis” (59.87%), which belongs to the assessment strategies category. The least-covered attribute is “physical health” (0.62%), which is part of the core competences category.

Moreover, the correlation analysis has revealed that there is either a weak or moderate positive relation between most of the assessment categories. For instance, there is a moderate positive relation between the online teaching strategies and the usage core competences, suggesting that the performance of a course in the former assessment category is an indicator of its performance in the latter, and vice versa. Notably, there is a negative relation (of weak or moderate strength) among core competences and all the other categories.

As far as the limitations of this study are concerned, since the assessment of online courses was realized by three reviewers, it also entails an element of subjectivity. We made an effort to address this through the peer review scheme included in our methodology. We acknowledge that, if we employed more reviewers with different backgrounds, we could reduce the risk of bias even more. Another limitation is the number of MOOC platforms that we investigated; although we selected the most popular ones, Coursera, and edX, as well as, the Udemy platform. It is worthy noted that there are additional MOOC platforms present in the market that we have not investigated in this study. Moreover, we have to acknowledge that, although the reviewers performed their best to evaluate each course accurately and reliably, they relied on the description and metadata (curriculum) provided by the courses themselves, in conjunction with a quick cross-check of their content. Thus, since there is the possibility that the content of some courses is not well-reflected in their description, this could have introduced some degree of inaccuracy in our study. Nonetheless, through the clear and robust evaluation design specified in Section 2.1, we have strived to reduce the effects of these limitations so that we can draw useful conclusions for the stakeholders involved. This type of inter-rater reliability check is consistent with best practices in both quantitative and qualitative research to reduce individual biases [27].

5. Conclusions

This work provides insights not only for higher education teachers, but also for developers of online courses and decision-makers in education. The added value of this evaluation study is that it provides higher education teachers with a structured way when searching for online courses to identify digital competence attributes that are suitable for them. Also, this study has revealed gaps in existing online courses regarding digital competences that developers of online courses should bear in mind for improving existing courses or developing new ones so as to cover higher education teachers’ needs.

In a nutshell, according to the results of this work, the following are suggested:

• For policy-makers:

  o

  There is a noticeable lack of online courses that adequately align with current digital competence frameworks. Existing online courses fail to cover the wide range of necessary digital skills.

  o

  There is a lack of online courses explicitly targeted for the needs of higher educators (rather than general audiences).

• For higher education teachers and their institutes:

  o

  The top five digital competences addressed by existing online courses are ”data analysis”, “data protection”, “understand”, “digital delivery”, and “structured learning”, meaning that there are good chances of finding relevant online courses to cover them.

  o

  The bottom five digital competences addressed by existing online courses are: “physical health”, “storage”, “peer assessment”, “collaboration”, and “inspire”, meaning that there are small chances of finding relevant online courses to cover them.

• For developers of online courses and platforms offering massive open online courses:

  o

  The average degree of coverage of digital competences offered by existing online courses is low (ranging from 15 to 24, depending on the platform, on a scale from 0 to 99).

  o

  There is need for better coverage of both individual digital competences (see the bottom five competences identified above) as well as of entire categories (refer to Table 2), by developing new courses or expanding existing ones.

  o

  Digital skills falling into the categories of core competences, ensuring positive online student experiences, and assessment strategies are covered to a mostly low degree by existing online courses. Digital skills related to usage core competences and creating digital content are better-covered, although still to a low to medium degree. Digital skills belonging to the category of online teaching strategies are, overall, covered to a more satisfactory degree at the moment.

  o

  Innovative courses need to be meticulously crafted from the outset, specifically tailored for higher educators, and be based on comprehensive digital competence frameworks. Simultaneously, to address the fragmented nature of current courses, it is essential to establish learning pathways. These pathways, should combine different course collections for more effective skill enhancement.

According to recent research [25], there has been a notable exponential increase in the occurrence of issues and gaps concerning the competences of higher education teachers. This requires continuous training and effort. However, since most of the existing courses require payment, work needs to be undertaken by decision-makers towards the provision of high-quality and certified online courses that are cost-free for higher educators. This could be achieved through the establishment of collaboration networks among European universities and research institutions.

In the future, the established methodology can be used for the assessment and comparison of additional online courses related to the development of digital competences. Thus, we aim to maintain and update our dataset containing the assessment of online courses, which can prove to be useful for third-party researchers and course developers alike. Moreover, a study has to be realized in order to investigate the reasons why there are not many online courses dedicated to higher-level educators (i.e., not to the general public), taking into consideration their particular needs and characteristics. Lastly, future studies can be focused on evaluating in more detail the quality of the content and learning methodology of the existing courses, thus further deepening the results of this study.