Promoting Digital Skills for Austrian Employees through a MOOC: Results and Lessons Learned from Design and Implementation

Abstract

Digital skills are now essential, not only in information and communications technology (ICT) jobs, but for employees across all sectors. The aim of this article is to detail how employees’ digital skills can be fostered through a Massive Open Online Course (MOOC), how such an offer is used and what the effects of such a measure are. Using an approach oriented at action research and design-based research activities, the authors describe the basics of their finding on existing European competence frameworks for digital skills and European projects that used MOOCs, the development and design of the MOOC, the evaluation on the basis of learning analytics insights and a questionnaire, as well as a reflection. The MOOC was offered as Open Educational Resources (OER) on the Austrian MOOC platform iMOOX.at from March to April 2021, with 2083 participants, of whom 381 fully completed the course (at end of June 2021) and 489 filled out the final questionnaire.

1. Introduction: The Need for Austrian Employees to Acquire Digital Skills

Information and communications technology (ICT) skills are required in more and more workplaces, even in jobs not traditionally associated with ICT skills or investment. Digital competences, or digital skills, have become a new kind of basic competence for the 21st century, much like writing, reading and math [1]. The skills and jobs survey (ESJS) from the European Centre for the Development of Vocational Training (CEDEFOP), for example, has shown that fundamental ICT skills are required for seven out of ten adult employees in the EU. At the same time, according to the survey, “about one in three of these employees is at risk of digital skill gaps” [2], p. 52. A survey on behalf of the Austrian Labor Market Service (AMS) confirms these findings for Austria and adds that digitization of the workplace will also create new job profiles that do not yet exist, or for which there is not yet any specific training. Employees must therefore learn general skills that will allow them to flexibly address future challenges in the workplace and embrace future developments quickly [3]. However, current offers for employees are often only comprised of “training sessions” in using specific programs or systems, and neglect more general digital competence that can be transferred to other settings [1]. In this context, the project “Digital Skills for 500 private employees in Austria”, a cooperation of ABIF (an independent social science research and consulting institute with a focus on applied research), GPA-djp (The Union of Private Sector Employees, Printing, Journalism, and Paper) and Graz University of Technology (TU Graz) aimed to provide training in basic, general digital skills for Austrian employees in the private sector using an eight-week Massive Open Online Course (MOOC). The purpose of this contribution and research is to explore and systematically describe how a MOOC for digital skills for employees can be planned and implemented to promote digital skills among employees in the Austrian private sector and beyond, how it may be used and what its effects on participants are.

This contribution will answer the following two research questions:

• How can a MOOC be planned and implemented to support the development of digital skills among employees in the Austrian private sector?
• Which of our experiences should similar future projects consider?

To answer the question, we used a research and action approach that intertwines research activities with the development and implementation of an activity—in our case, a MOOC. This is related to an action research design, where the people directly responsible for a pedagogical intervention are—at least partly—involved in the research process [4]. In general, we followed an “action–reflection cycle”, a recursive process of planning, implementation (acting), observation and reflection [5]. The approach is also influenced by the discussion of design-based research [6], in which a sound, research-based development of an educational intervention is accompanied by an evaluation and then critically reviewed.

Figure 1 details the two interconnected activities of research and MOOC development and their phases from April 2020 to October 2021.

From a research perspective, we first searched the scientific literature and project reports for existing implementations and experiences with digital skills development in Europe, focusing on implementations with MOOCs. This is not a systematic, worldwide literature search, which we could not find, but a targeted analysis of relevant EU publications.

For the description of the planning and development phase of the MOOC, we used the notes and records of the project team for this article. These were prepared in such a way that the most important planning and development steps, as well as the design of the MOOC itself, are as comprehensible as possible. For the evaluation of the MOOC itself, we collected data and made different evaluations using a learning analytics system (“Creator’s Dashboard”) that was developed at TU Graz and is directly integrated into the MOOC platform iMooX.at. It collects, analyzes and visualizes learners’ data, for example about enrolments, activity and interactions. In addition, protocols were written during the live sessions of the MOOC. A third form of data collection on the MOOC was done through a written online survey at the end of the course of all participants who wanted to download a certificate of participation. The 32-item questionnaire consists of general questions on the background of the participants (single-choice) and a second part where participants are asked to rate their MOOC experience on a five-item Likert scale. Several questions offer an additional free-text field for explanations or comments. The original questionnaire is publicly available (see Supplementary Materials). The answers to the questionnaires are analyzed using descriptive methods.

Finally, the procedure also includes a (critical) reflection on the implementation. For this purpose, the research team reviewed and discussed the results of the evaluation. In this way, a collection of possible adjustments and changes was made for a renewed implementation of such a project.

In the following section, we first describe the related work on MOOCs in adult education, existing concepts and experiences for the promotion of digital skills throughout Europe; then, following the steps of our action research design, we describe the design and planning phase of the MOOC, its implementation, the results of the evaluation and, finally, a reflection on the project.

2. Related Work

Our project is not the first of its kind. There have been several practical and scientific research projects on MOOCs in adult education and, more specifically, for furthering digital skills. In the following sections, we describe the related work.

2.1. MOOCs as Further Education for Employees

Massive Open Online Courses (MOOCs) are online courses for many people (“massive”), i.e., more than 150 participants [7]. The term “open” is used because the first MOOCs were offered by university members, but were accessible online without formal restrictions, such as a university entrance qualification, and usually free of charge [7,8]. MOOCs were never meant for students only; rather, they were always intended for other target groups, such as employees, seniors, employees or “lifelong learners” in general. The model of so-called “xMOOCs”, which are presentation-oriented online courses for many individuals [9], has become widely established. In this type of MOOC, learning videos and material for self-regulated learning are offered in course form and exchange between learners is supported by forums. Unsurprisingly, this learning setting has been of interest for employees from the beginning. In German-speaking Europe, among the first MOOCs for employees were, for example, a MOOC about open educational resources (COER13.de) [10], the vhsMOOC 2013 for adult educators [11] or the Magenta MOOC for employees of the Deutsche Telekom company [12]. Schulmeister [13] anticipated MOOCs as a method for professional development. Nevertheless, a survey in Germany in 2018 (n = 1003) showed that only very few individuals (10%) know what a MOOC is, and only 1% have already used them for private or professional purposes [14]. In the past few years, more research was carried out in the field of MOOCs, for example on various teaching scenarios [15]. To more effectively address the needs of adults, we developed a design approach of “inverse blended learning” that tries to enrich the online course setting with “analog” materials, such as a printed handbook or learner meet-ups [16]. To sum up, we have determined that MOOCs are a helpful measure to reach and support the learning of adults and employees, but it is far from being well established.

2.2. Concepts of Digitals Skills for European Citizens and Current Approaches for Their Development

In Europe, the needs of citizens with regards to digital skills have been addressed for several years now. In order to define the specific competences that are needed, competence frameworks have been developed to define such learning goals as a basis for interventions and training. The current developments concerning a digital competence framework, including the Austrian adaptation, and existing approaches to foster them amongst European citizens in member states are presented in the following section.

2.2.1. Existing Competence Frameworks (DigComp 2.1 and DigComp AT)

The Digital Competence Framework for Citizens (for short, DigComp) was first developed and published in 2013 by the European Commission, and is acknowledged as the reference tool for a common understanding of digital competence for citizens. It provides an overall, complete and shared understanding for multiple stakeholders of what digital competences are [17].

There have been several follow-up publications to the original framework [17], translating the DigComp for use in the workplace and in the labor market, such as ‘DigComp into Action’ [18], a user guide for anyone seeking to promote digital competences in the workplace; ‘Developing digital competence for employability’ [19], a report of a consultation workshop with stakeholders; and the ‘DigComp at Work Implementation Guide’ [20], which offers specific guidelines for the development of training offers in digital competences.

These follow-up publications encourage the use of the DigComp framework for the development and strategic planning of digital competence initiatives, both at European and at the member state level [19]. Training and assessment based on DigComp can be shared and expanded more easily to other EU countries thanks to a shared framework and vocabulary [18].

The “DigComp 2.1 Framework” [17] consists of five parts, which are:

• Competence areas identified to be part of digital competence; 5 areas articulated into a total of 21 competences;
• Competence descriptors and titles that are pertinent to each area in general terms,
• Proficiency levels for each competence, ranging from “foundation” to “highly specialized”;
• Knowledge, skills and attitudes applicable to each competence;
• Examples of DigComp use.

The ‘DigComp at Work Implementation Guide’ [20] suggests using DigComp in the respective language version and reusing existing resources if they are accessible and have a similar purpose. Such a transfer was done for Austria: the Austrian national framework, DigComp 2.2 AT [21], is, in large part, a translation of the European DigComp 2.1 framework into German. However, some parts have been adapted or added to. The most important addition is an additional competence area numbered “zero”. Competence area zero describes first access to the field of digital competence and the requirements for access and the start of participation in the field of digital competence. “Zero” was chosen so that the other competence areas would retain the same numbers and therefore would be directly comparable with the European counterpart. Further additions to the Austrian version of DigComp 2.2 AT are “Buying and selling” (2.4) and “Protecting yourself from fraud and infringement of consumer rights” (4.4). Competence area 5, called “Problem solving” in the European framework, was expanded to “Problem solving and further education” to emphasize the importance of lifelong learning [21].

2.2.2. Supporting the Development of Digital Skills through MOOCs: Existing Experiences

To locate the Digital Skills MOOC within a wider European perspective, an overview of DigComp 2.1-based MOOCs for employees from other member states is given in this section. The ‘DigComp into Action’ guide [18] already provides an extensive overview of projects based on DigComp 2.1 for various target groups and in various settings, so the following examples in

Table 1. Overview of other member states’ approaches for the furthering of digital skills of adult employees. Source: own selection and visualization based on [18]. An “X” indicates that this criterion is fulfilled, a “-“ means it is not.

Project Name	Main Target Group	OER	Digital Competence Profile for Employees	Supplemented by Face-to-Face Offer
MU.SA	Museum professionals	X	X (for employees)	X
INTEF	Teachers in Spain	Open access (but no open licenses)	X (for teachers)	-
Elene4Work	Young adults entering the labor market	Some of the recommended MOOCs are OER, others not	-	-

are extracted from this report. The examples were chosen based on the criteria that the target group of this offer is (among others) adult employees and a MOOC is used. Table 1 gives an overview of the member states’ approaches for the development of digital skills of adult employees using MOOCs and shows whether these MOOCs are available as OER, are complemented by job profile descriptions and their respective digital competences and whether the offers are provided as a blended MOOC with additional face-to-face units.

MU.SA is a training project for museum professionals, developed in Italy, Portugal and Greece. At first, the job roles for different museum professionals and their necessary digital skills were defined. The first stage of the training consists of a general non-job-specific training phase using digital OER in the form of a MOOC titled “Essential digital skills for museum professionals”, focusing on transversal digital skills in 22 modules. The materials offered consisted of interactive images and forums for discussion and socializing [22]. The second stage of the training consists of a specialization training course in a blended learning format, including self-study phases and face-to-face lectures. The third stage involves workplace learning in museums and cultural organizations affiliated with the project. An additional goal of MU.SA was to establish communities of practice to ensure that the results are carried on beyond the end of the project [18]. An analysis showed that the MOOC also attracted participants from other countries than the project members, mainly other EU member states [22].

In Spain, the INTEF MOOC was developed to train teachers in one or more competence areas. After offering three pilot MOOCs in 2014, the project evolved into a catalog of “Nano Open Online Learning Experiences”. The contents are based on the 2017 Spanish Common Digital Competence Framework for Teachers, which is an adaptation of DigComp to the teaching profession. The MOOCs take between 1 and 20 h of work to finish. If a teacher completes several courses, they can receive a digital micro-credential.

One program featuring MOOCs, but not with adult professionals as a target group, is eLene4work. It was an Erasmus+ project from 2015–2017. The program is directed towards young adults entering the labor market. Part of the project includes MOOCs for enhancing participants’ digital and soft skills. The participants have the option to take a self-assessment test to find out which soft skills they might want to develop further. They can then select and participate in some of over 200 MOOCs selected and classified by soft skill. The participants are also given advice on how to make the most of their MOOC experience, such as how to develop soft skills by using the learning activities embedded in social and connectivist-type MOOCs [18]. They are encouraged to keep a record of their learning in a personal journal in case they are part of the eLene4work field study.

There are other projects geared towards employees’ digital competencies described in the publication, but numerous tools only focus on a self-assessment of one’s digital competences, often with subsequent links to suitable training offers. In Austria, the Fit4Internet project (part of the wider Austrian digitization strategy for all citizens) provides a self-assessment tool for digital competence based on DigComp 2.2 AT (Fit4Internet). It aims to assess the digital competence of all Austrian citizens, comprising, but not limited to, private sector employees. Fit4Internet also provides an online catalog of training offers (MOOCs, but also face-to-face training) categorized according to the six competence areas and eight competence levels defined by the Digital Competence Framework for Austria. However, the training offers listed are not all openly licensed and free of charge. Some of the offers are online courses and others are face-to-face courses.

With this is mind, the DigiSkills project coordinators decided against developing and providing another assessment tool and instead opted for a MOOC.

3. Results

3.1. Planning Phase: Design Considerations and Marketing

3.1.1. Motives

A MOOC was chosen for the DigiSkills training in order to easily reach the large target group of Austrian private sector employees. MOOC participants can further train their digital skills flexibly and independently of time and place, which is suitable for people working part or full time. It is assumed that most employees have at least a smartphone, tablet or computer, so they will be able to follow a MOOC. There are no financial obstacles to participation, as the MOOC is free of charge.

3.1.2. Target Group

The main target group of the MOOC are Austrian private sector employees, but since the MOOC is offered as an OER on iMooX.at, another possible target group are German-speaking adults beyond the private sector interested in furthering their digital skills, which is a potentially quite large and heterogeneous group. Since the MOOC is potentially open to everyone interested in furthering their digital competences, the DigiSkills project did not include extensive descriptions of job profiles and their respective digital skill requirements, as was included in other member states’ projects. When planning a MOOC for private sector employees, it must be considered that they are not typical MOOC participants. Studies have shown that participants in MOOCs still tend to be individuals with prior experience in higher education and do not automatically tend to attract people who are distanced from (online) education [23].

3.1.3. Goals of the MOOC

The goal of the MOOC is, first and foremost, to promote digital skills among employees in the Austrian private sector by allowing them to take part in the freely available online course, especially amongst employees who do not usually work using a computer and who display no or few digital skills. On the level of labor market stakeholders, the project is also meant to empower the works council as an interest group by allowing them to contribute to the creation and the content of this program. Lastly, the goal is to future-proof this self-study MOOC so it can be used by employees free of charge on the iMooX platform, even after the end of the guided phase, as a way of furthering their own education independent of time and place. It is also planned to repeat the MOOC every two years as a guided MOOC by the project partners.

A quantitative goal of the project is to gather at least 500 participants who would finish the MOOC and obtain a participant certificate [1]. To reach this quantitative goal, it must be considered that the completion rate for MOOCs is often much lower than the number of people registered. Some participants may only register to try out online learning. For others, their learning outcomes can be met without completing a whole MOOC and obtaining a certificate, so they could drop out without certificates as well.

3.1.4. The MOOC Platform

In Austria, there is a MOOC platform that was chosen for the implementation. The Austrian national MOOC platform, iMooX.at, is hosted by the University of Technology of Graz (TU Graz) and is committed to offering open educational resources (OER). All iMooX.at courses are offered free of charge and only a valid e-mail address is required to sign in [24]. Courses can be followed at everyone’s own pace and via any device with Internet access, which is especially beneficial to the target group of the DigiSkills project—employees working various shifts and not necessarily living close to the nearest training center.

3.1.5. Co-Design with Stakeholders

During the planning phase of the MOOC, six workshops with members of the works council were organized to find out which specific challenges and changes arise due to digitalization at the workplace and which specific content would be necessary in a training offer for employees. The invitation was sent to 13,000 members of the works council in different economic sectors in Austria [1]. However, the size of the workshops had to be reduced due to the measures against COVID-19. These workshops were a first step to establishing collaboration and continuous dialogue between the different labor market stakeholders collaborating on this MOOC.

Three workshops with works councils were held online via Zoom at the beginning of 2020, during the first lockdown in Austria. The first workshop focused on the competence requirement in modules 0 and 1 of the DigComp model. The second workshop was dedicated to modules 2 and 3, and the third workshop to modules 4 and 5. In September 2020, a fourth workshop was held face-to-face, where the competence requirements in all modules were the topic (see Figure 2). The participants in the workshops were works councilors recruited by the GPA. Between 15 and 20 participants took part in each workshop. The first three workshops took place without the specific MOOC instructional design background, and it proved difficult to take the results into account, as the competences required by the representatives of the different sectors were too diverse, yet specific.

Ultimately, topics had to be found that would appeal to all adult professionals in Austria. Therefore, in the summer of 2020, a rough MOOC concept and the basic content of all modules were defined in an ABIF-internal workshop. The participants of the last workshop were then confronted with these ideas, where they could rate the topics, or add new topics or examples from everyday work life.

3.1.6. MOOC Design and Overview

This MOOC was designed to run as a “Blended MOOC” and later as a self-study MOOC without online live sessions. Figure 3 gives an impression of the online course. The online phase of the MOOC is divided into eight units, with a new unit being made available each week:

• 101 of the Internet;
• Safe surfing and downloading;
• Digital collaboration;
• Digital tax returns, signatures, and official services;
• Digital job applications: MS Word and Europass (the European online CV);
• Social Media: posts, privacy and copyright;
• How to spot fakes and false information online;
• Problem solving and continuing to learn.

Each unit consists of one or more video(s) and documents for further reading.

The forum was used for reflection and to share one’s own experiences. The participants were also able to ask questions about the various MOOC topics and modules. The videos gave an overview of the topics and provided guidance, for example, on how to get a digital signature or how to look for a job online. In this sense, the entire MOOC was very low-threshold, which was also its goal. The participants were also able to choose which modules they wanted to complete and were given badges for each completed module, even if they did not want to complete the whole MOOC.

As with other MOOCs on the platform, there are some support options to guide the learners: participants can mark completed activities to record their own progress within a unit. Additionally, at the end of each unit, there is a single- or multiple-choice quiz. The participants have five attempts to take each quiz and need to answer at least 75% of the questions correctly to pass the quiz.

Furthermore, the participants were invited to engage with the MOOC topics in an offline setting in the form of a specially designed workbook that could be obtained from stakeholders or the organizers before the start of the MOOC. The printed booklet is designed to help participants who do not yet possess a great deal of digital skills. It contains the MOOC content as texts and exercises (gap fill, reflection questions) with solutions for self-assessment and to repeat the content. The idea for the workbook was based on the booklet accompanying the MOOC “Gratis Online Lernen” and the idea of the didactical concept of “inverse blended learning” [15].

The self-study phase is complemented by three non-compulsory 1.5 h live sessions via Zoom at the beginning, in the middle and at the end of the MOOC as in a “Blended MOOC” [16]. These live events are supposed to foster interest and a connection between the participants and allow them to ask questions and give feedback.

For each completed quiz, the participants receive a digital badge. If the participants have completed all of the quizzes and filled out the compulsory questionnaire at the end, they can download a PDF certificate attesting “Basic Digital Skills” [1]. For professionals, such a certificate may be important to use as evidence of further education to provide to their employers or during job applications [25].

3.1.7. Marketing Issues and Activities

Apart from stakeholders learning about the MOOC during the workshops, the course was promoted via mailing lists to members of GPA-djp and directly to members of the works council. ABIF promoted the MOOC in a mailing list with 3000 recipients and AMS (the Austrian Labor Market Service) promoted it in their research network with 8000 members. In addition, iMooX.at promoted the MOOC on its social media channels. The above-mentioned workbooks were also handed out to potential participants at the stakeholder institutions as part of the marketing activities.

3.2. Implementation Phase

Between 1 March and 25 April 2021, the MOOC was offered on iMooX.at as a blended MOOC [16], which means that the asynchronous online course was enriched with face-to-face activities in regular intervals.

As of May 2021, after the end of the guided MOOC phase, the online course remains open and accessible for everyone free of charge and as an OER, but as a self-study MOOC without live sessions. The data gathered via the creator’s dashboard as well as the questionnaire results presented below represent all MOOC activities from 1 March 2021 to 30 June 2021—in other words, the whole guided phase and the beginning of the following self-study phase.

3.2.1. MOOC Participants and Activities

According to the data gathered via the creator’s dashboard, during this time frame till end of June 2021:

• 2083 participants signed up for the DigiSkills MOOC on iMooX.at;
• 4765 badges were issued (one is available per unit);
• 381 participants fully completed the MOOC;
• 369 certificates were issued.

Looking at an activity overview provided by the Creator’s Dashboard in Figure 4, most participants enrolled onto the course at the beginning of the course, and those enrolments then slowed down with a number of spikes in later weeks. The first interaction graph shows that most people who enrolled onto the course also interacted, i.e., accessed activities in the course, directly at the start of the course or in the first few weeks after the course start date.

Most participants completed the course towards the end of April 2021, which coincided with the end of the guided MOOC phase. There were no graduations prior to this date since the units were released on a weekly basis and it is necessary to complete all of the quizzes to complete the MOOC. The visits graph provides information about when people accessed the course. Most visits to the course were taken during the guided MOOC phase, with spikes each week that coincide with the weekdays when the new weekly unit was released.

3.2.2. Quiz Activities

The quiz of unit 1 was completed by 769 participants by 30 June. The quiz of unit 8 was only completed by 416 participants. As Figure 5 shows, the number of completed quizzes slightly decreased each week, which might hint at participants dropping out or completing the course slower than others.

3.2.3. Time Spent in the MOOC

The Creator’s Dashboard also gives insight into where participants spend the most time in a MOOC. It can be seen from Figure 6 that the participants spent most of their time on “assignments”, i.e., quizzes and activities. A total of 21% of the time was spent watching the educational videos provided. “Navigation” made up 28% of the time spent in the MOOC. This category comprises all activity within the MOOC that cannot be narrowed down to specific exercises, such as reading texts and descriptions provided in the MOOC.

3.3. Evaluation

3.3.1. Questionnaire and Participants

A total of 489 participants filled out the questionnaire, which is 23% of all registered participants (n = 2083). This is higher than the number of participants who finished the course (n = 381).

The participants in the survey were 29% male and 71% female. The largest share of participants was in Austria (76%), however there was also a significant share of participants from Germany (20%) and another 2% from Switzerland. With regards to the educational background of the participants as shown in Figure 7, approximately one third (32%) held a master’s degree or equivalent; 23% completed A-levels or equivalent; and another 21% completed a vocational school or apprenticeship. Finally, 13% of participants completed elementary school or middle school education and 2% had not completed any schooling (so far).

Most participants worked full-time (185) or part-time (134) at the time of the survey. The third largest group consisted of people currently looking for work (98). Thirty-three participants were in training, 32 were self-employed and 32 were students. Twenty-one participants were marginally employed, 15 were still in school and 18 answered that they were retired. Five participants were on maternal/paternal leave and three were on disability. The survey asked the participants about which area they were currently employed in or in which area their last employment was. Most of the participants (31%) were or had been employed in the educational sector. The second largest sector was office/commerce/management (26%). The third largest group of participants was or had been employed in the social and health sector (17%). Another 6% were or had been working in trades or technology. Lastly, 20% of the surveyed participants chose “other”. The sectors most frequently mentioned in the corresponding free-text field were retail, catering and the insurance sector.

3.3.2. Reasons for Attending the MOOC

The survey participants were asked to rate different statements on a five-point Likert scale (“strongly agree” through “strongly disagree”). To give more concise results, “strongly agree” and “agree” is pictured as one category in the graphs. The same goes for “disagree” and “strongly disagree”.

As can be seen in Figure 8, 85% of the respondents stated that they were interested in the content and topic of the course, and 24% took on the course because of who was teaching it. The names and roles of the teachers were not explicitly used to promote the course, which might explain why almost half of the surveyed disagreed with the statement.

A total of 45% of the participants agreed that the MOOC content complemented their current training or education; 29% disagreed or strongly disagreed. Over half (55%) answered that the content complemented their current job, while 20% disagreed with the statement. Most participants agreed with the statement that they were following the MOOC to gain experience with online education (65%), with 27% answering that the MOOC was part of their training or education. A much larger portion of those surveyed, however, stated that the MOOC was simply additional training or further education (66%). In addition, 26% of the participants mentioned reorientation in the job market as a reason for attending the MOOC.

The participants were asked to rate which benefits an online education in the form of a MOOC brings them. As can be seen in Figure 9, 66% agreed or strongly agreed that a MOOC is an especially useful format for someone currently in employment; 31% agreed or strongly agreed that an online course is ideal for them because of their place of residence; 26% agreed or strongly agreed that online education is useful because of care responsibilities. A much smaller but noteworthy share (10%) agreed or strongly agreed that online courses are ideal for them because of a disability.

3.3.3. Passing on the Knowledge

In the next section, the participants were asked to rate how likely they were to pass on the knowledge obtained in the MOOC to other people, which is the principal idea of OER. Over a third of the respondents (34%) agreed or strongly agreed that they were planning to pass on the knowledge to coworkers or employees. Another third of those surveyed (32%) reported wanting to pass on the knowledge to clients. An almost equally large share (36%) planned to pass on the new knowledge to their family or circle of friends.

3.3.4. Rating the MOOC Contents and Instructional Design Aspect

The participants were asked to rate the content and the methodological and instructional aspect of the course, again using a five-point scale in accordance with the Austrian grading system used in an educational context, where 1 is the best grade (“very good”) and 5 is the worst (“insufficient”). The content of the MOOC received an average grade of 1.68 (~0.76 standard deviation); the methodological aspect was given an average of 1.57 (~0.71 standard deviation). An open-ended question asked the participants to mention aspects that they particularly liked. The aspects that were mentioned most often were the videos, which were seen as enjoyable and easy to follow; the simple explanations and practical examples; and the combination of different activities, including the booklet. It also becomes evident that the flexibility and possibility to learn anytime and anywhere was seen as a positive aspect of this MOOC. Another open-ended question invited the participants to share aspects they were not satisfied with. The most frequent criticism regarded the sign-up for the course, which posed a challenge for some participants. Several participants also seemed to have trouble navigating the platform and were confused by the possibility of ticking items to mark their own learning process. Some participants complained about the fact that all of the examples were focused on the Windows operating system. The participants were asked about what they felt should be changed, should the course be revised and relaunched in the future. A total of 186 participants stated that there is no need for revision. The most frequently given answer concerned the speed of the videos, which was seen as too fast, especially when screencasts or examples were given. Some participants found the content too simplistic or too easy to follow and would have liked more detailed explanations or links for further reading. Some participants would have enjoyed more diverse quiz formats in addition to the multiple-choice quizzes. Another request was to have transcripts and subtitles for the videos and possibly offer the course in other languages in a future relaunch.

3.4. Reflection

3.4.1. Reaching the Target Group

In an openly licensed and publicly available MOOC, the participants will be heterogeneous. However, it is necessary to keep a specific target group in mind when planning the content and methodology. Information about this “ideal” target group was provided by the stakeholders in preliminary workshops. A closer look at the demographics of the participants can give us an answer to the question of how such an offer is used. The first thing that stands out here is the large proportion of women on the course (71%). This could be related to a high share of women in the field of office administration [26] and might be partly related to the known higher proportion of females working as adult educators [27]. A significant share of the participants reported working full-time or part-time in the private sector, so it can be assumed that the MOOC reached its main target group of private sector employees. Additionally, the fact that a large share of the participants reported having a master’s degree and working in the educational sector leads to the assumption that the MOOC was extensively used by trainers and coaches in adult education, either for their own furthering of digital skills or along with their group, which is in the spirit of OER. Even if the target group was not reached directly, they should still benefit indirectly from the MOOC.

3.4.2. MOOC Contents and Instructional Design Considerations

When planning and implementing a MOOC to promote Austrian employees’ digital skills, DigComp 2.2 AT offers a sound basis for creating and structuring the content. Differently from an ongoing face-to-face course, the content and methods of a MOOC must be agreed on well before the start of the MOOC, as there is little flexibility to change the basic structure or methodology of the course once it is active. The MOOC was taken by participants with varied educational backgrounds and from many different sectors, so it can be assumed that participants’ previous knowledge varies significantly. This was reflected in the open-ended feedback form: some respondents stated that the content was too simple, while for others, simply signing up for the course was challenging. To answer the question of how the offer was used and what its effect are, it can be said that only a fraction of the enrolled users took part in the (non-compulsory) live sessions. However, the option was appreciated by the participants since it provides the opportunity to ask additional questions and get to know other participants, which, in turn, might create a sense of community.

3.4.3. Potential Adaptations for Future DigiSkills MOOCs

One way of dealing with the different levels of prior knowledge and interest is to focus the course more strongly and to differentiate in the course. The prerequisites of the course could be clarified, for example, with a self-assessment test or a survey: only those people who have minimal prerequisites (e-mail address) but can still learn essential things will be recommended for the course. One solution to this problem might be a preliminary self-assessment or entry exam to determine the level of the participants and then presenting them with relevant videos or suggesting an entry point into the course based on the results. Another possibility within the course is to describe more differentiating offers, e.g., advanced courses or different tasks depending on personal interests. Such differentiations are always connected with the problem that they can make the course more complicated.

4. Discussion

We would now like to critically discuss the results described in the article and compare them with other projects and implementations.

First, the analysis of the digital skills frameworks and European projects has shown that there are some, but not many, training measures using a MOOC for employees on a large scale in Europe; i.e., this project was not a novel innovation, but is still unique. During the planning of the MOOC, the need for corresponding educational offers became very clear, e.g., in the workshops with stakeholders, not least since the offer of a MOOC was seen as thoroughly modern, but also caused uncertainty as to whether it was the right instrument, because it already requires initial digital skills. We know this discussion from other MOOCs aimed at beginners in digital learning [15]. In a way, COVID-19 was helpful here: due to the changes involved—restrictions on social contact and a ban on meetings in spring 2021—a MOOC that can be completed fully online was just the right thing to offer.

The activities of the participants during the MOOC, the completion rate and also the presented interaction courses—higher activities at the beginning of the course, with a downturn in the course—follow the usual patterns that we know from other MOOCs and do not show any conspicuous features. This is particularly pleasing, since this MOOC is a voluntary offer and participation is not integrated into a degree course, for example, and the participants do not yet have much experience with digital technologies. The positive feedback from the participants, taken from the evaluation, shows that with the MOOC, we have developed and implemented a very good basic measure to promote digital skills among employees in Austria.

In the reflection section, we have elaborated on some further reflections and insights that should be critically monitored in future implementations. It turned out that quite a few adult educators finished the course, which are not the main target group. This could also lead, for example, to the creation of targeted educational offers for this target group. The Mu.Sa project has also expanded its offer with specialization modules, so this need for differentiating offers is also evident in other projects.

As always, in researching MOOCs, we cannot make assumptions about participants who did not enroll, are not active in the MOOC, or those who did not fill out the questionnaire or take part in the live sessions. While the Learning Analytics Creator’s Dashboard does give us information about the number of inactive participants, we can only cautiously assume their motives and reasons for not participating or not finishing the MOOC, which might include such factors as lack of motivation, time constraints or technical issues. We assume that not all of the dropouts are necessarily an issue that needs to be solved, as some might indicate, for example, that the participants never wanted to complete the course or only sought specific information that was important for them. At the same time, we are aware of the fact that a MOOC is a very good, but not sufficient, measure to promote digital skills among employees.

5. Conclusions and Outlook

The Digital Skills MOOC, based on DigComp 2.2 AT, was designed to upskill 500 Austrian private sector employees and complements the European strategy for meeting the needs created by the digitalization of the labor market. This paper answered the questions of how the MOOC was developed and implemented, how it was used and what the potential effects of such an educational intervention are, using a four-step action research design as the background for the description, analysis and reflection. In summary, our research shows very satisfactory results and we see a MOOC as a powerful measure to support the development of digital skills among employees.

The MOOC itself will be updated within a planned second round based on the findings and reflections presented in this paper and will be relaunched regularly as a guided MOOC after another promotion phase. As all the content in the MOOC is openly licensed with a Creative Commons license, the course as a whole, or parts of it, may be reused by trade unions or adult education centers if they would like to offer seminars or courses based on this content. In these future training, the online content may very well be enriched with face-to-face classes instead of online live sessions [1]. To improve the MOOC experience for learners and course creators, the Creator’s Dashboard that provides learning analytics will be integrated further and new features will be added to reveal how particular activities engage different types of learners. Learning analytics will also identify distractions and help automate mundane organizational tasks, so that participants can focus on learning.

Concerning future research and development, we see the practical need to adapt examples and practice examples concerning the digitals skills framework to a special target group and their settings and conditions. Thus, for future projects for slightly different target groups or other countries, it is recommended to follow the national frameworks. You can orientate yourself on our developments and topics, but it is highly likely that your target group uses other applications—for example, to book a train or for digital signatures. Here, the projects must also plan and carry out longer development work, ideally with the target group and stakeholders.