Next Article in Journal
Teaching Management System with Applications of RFID and IoT Technology
Next Article in Special Issue
Distance Learning—Predictions and Possibilities
Previous Article in Journal
Socially Challenged Collaborative Learning of Secondary School Students in Singapore
Previous Article in Special Issue
Distance Learning and Assistance Using Smart Glasses
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The Flipped MOOC: Using Gamification and Learning Analytics in MOOC Design—A Conceptual Approach

Welten Institute, Open University of The Netherlands, 6419 AT Heerlen, The Netherlands
School of Digital Technologies, Tallinn University, Narva Road 25, 10120 Tallinn, Estonia
Author to whom correspondence should be addressed.
Educ. Sci. 2018, 8(1), 25;
Submission received: 30 November 2017 / Revised: 5 February 2018 / Accepted: 6 February 2018 / Published: 11 February 2018
(This article belongs to the Special Issue Challenges and Future Trends of Distance Learning)


Recently, research has highlighted the potential of Massive Open Online Courses (MOOCs) for education, as well as their drawbacks, which are well known. Several studies state that the main limitations of the MOOCs are low completion and high dropout rates of participants. However, MOOCs suffer also from the lack of participant engagement, personalization, and despite the fact that several formats and types of MOOCs are reported in the literature, the majority of them contain a considerable amount of content that is mainly presented in a video format. This is in contrast to the results reported in other educational settings, where engagement and active participation are identified as success factors. We present the results of a study that involved educational experts and learning scientists giving new and interesting insights towards the conceptualization of a new design approach, the flipped MOOC, applying the flipped classroom approach to the MOOCs’ design and making use of gamification and learning analytics. We found important indications, applicable to the concept of a flipped MOOC, which entails turning MOOCs from mainly content-oriented delivery machines into personalized, interactive, and engaging learning environments. Our findings support the idea that MOOCs can be enriched by the orchestration of a flipped classroom approach in combination with the support of gamification and learning analytics.

1. Introduction

“Education is a fundamental human right and essential for the exercise of all other human rights. It promotes individual freedom and empowerment and yields important development benefits” [1]. With the aim of bringing and democratizing education for all, phenomena such as Open Education and in particular Massive Open Online Courses (MOOCs) have started to grow in recent years; MOOCs could hypothetically bring education to everyone and for free. Despite this potential, MOOC studies have highlighted several limitations, such as the lack of users’ engagement and their high dropout and consequently low completion rates [2].
Recent studies suggest looking at a completion rate as an indicator of MOOC learners’ intentions [3,4], and until now the success of MOOCs has been mainly measured in terms of course completion, counting each participant who has not completed the whole course with its assignments and exams as a dropout factor. We argue that studying completion in MOOCs requires a different perspective. MOOCs are not traditional courses. The fact that they potentially can reach a massive number of people means dealing with and catering for an enormous variety of needs. From this perspective, the user interest and intention have to be considered in the design phase.
MOOCs that take the diversity of learners into account need to be personalized, engaging and motivating. Various concepts have been proposed [5,6] to improve MOOCs using community-oriented approaches. Despite the availability of different types of MOOCs, however, the majority of them still deliver learning content via video and as a result MOOCs have become content containers and repositories of material. If content is delivered in a monotonous way without taking advantage of advanced technological and didactical approaches it will be hard for users to remain engaged.
Consequently, one of the key challenges in MOOC design is to overcome the problem of being attractive to mass audiences and at the same time it has to be relevant to a large variety of individual needs, requirements, and expectations. Therefore, we conceptualize the idea of a flipped MOOC based on the concepts of flipped classroom, gamification, and learning analytic. In line with the technology-enhanced learning perspective in this article, we aim to address the issues of low student engagement, low completion rates, and high dropout rates by proposing a MOOC design solution that makes use of successfully applied pedagogical concepts known from other educational fields: flipped classroom, gamification, and learning analytics. We will briefly introduce these concepts and their current state as reported in literature in the following paragraphs.
The “flipped classroom” is a model within the blended learning delivery where direct instruction happens online and interaction is planned for classroom activities. This approach is often described in terms of Bloom’s taxonomy [7] where the first two phases (remembering and understanding) happen online through interaction with the content and direct instruction. Flipping a classroom implies focusing on interactive and active support of teacher and active learning from the students’ side. In this context, students are required to engage with the content independently. While student perception reports on the use of flipped classroom are mixed, we know that video lectures are not preferred to in-person lectures [8]. On the other hand, students do prefer interactive classroom activities. Therefore, what is mainly flipped is the active role of the teacher in passing the concepts and content knowledge, which is left up to the student, at his/her own pace. It is not the teacher that lectures and transmits his/her own vision on the specific topic but the student by reading and studying the material, while the teacher provides help to the student to develop his/her own understanding during the in-class time, where there is time to discuss.
There is some evidence suggesting that the student learning is improved by the flipped mode and that it leads to higher cognitive outcomes [9,10,11]. Other findings suggest that a MOOC-based flipped class is a good solution to promote students’ motivation and learning, but the implementation of this teaching strategy is delicate and must be very well planned [12]. Some authors indicate that the use of flipped MOOCs is promising not only in terms of effectiveness but also the motivation of the students and teachers [13] and increased satisfaction [14]. Meanwhile some authors are more skeptical and suggest that flipped learning (FL) effectiveness is yet to be proven [11]. It is our aim to transfer the concept of the flipped classroom to MOOC design in order to achieve MOOCs that are more engaging, interactive, and that provide a more personalized learning experience. A MOOC designed in such a way can be called a “flipped MOOC”. The following two concepts described here represent the building blocks for the first concept (flipped classroom), which we aim to explore to design flipped MOOCs.
“Gamification” is the application of game elements in a non-game scenario to solve a problem or induce a change in the behavior of the target population [15]. Gamification has a lot of potential in education [16]. Its effectiveness has been studied by several authors and from these studies it has been suggested that the gamified approach can generate several effects, for instance, on users’ performance [17,18,19,20,21], motivation [22,23,24], engagement and enjoyment [25,26,27]. In the last 10 years gamification has been applied in several fields, such as: marketing, trading, education, training, fitness, software engineering, etc. The highest number of applications can be found in education [28,29].
Conceptual examples of gamification applied in MOOC environments are also available, such as the approach by Chang and Wei [26], who try to address the problem of the most suitable game mechanics to enhance users’ engagement. However, as other authors highlighted, the game elements most commonly found in implemented gamification solutions are points, badges and leaderboards (known as PBL), which are rewards given to participants who mainly derive their motivational potential from social comparison theory [27,30].
Similar to flipped learning (FL) approaches, gamification also requires conscious design decisions. In particular, it is fundamental to know the problem that characterizes the context of application, in our case MOOCs, and the awareness of the expected impact that determinate game elements can generate on users’ behavior.
The third and last concept “learning analytics” (LA), is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Society of Learning Analytics Research (SoLAR)). Learning analytics here is introduced as an instrument that supports game elements applied in a gamification approach to deliver personalized and interactive learning experiences. As a rapidly developing field, LA promises to offer new types of data analytics, including new assessment methods [31]. On the one hand, students have increasing needs to interpret activities given by teachers and engage with study material; on the other hand, teachers need to make informed decisions about the classroom phase activities and, overall, inform the learning design [32]. Within the context of our research, LA use has twofold aims: (1) to envision how LA can support learners through meta-cognitive capabilities since students in a FL setting have more elevated necessity to reflect on their own learning than in a traditional setting [33]; and (2) to support teachers to create a feedback loop between online and classroom phases and assess the learning of students. This can be done through dashboards or retrospective analysis [34]. In the context of our approach towards designing flipped MOOCs, the role of learning analytics is to explore the potential of feeding learner-generated data into the selected game elements to foster personalization, interactivity, and engagement.
Flipping a MOOC requires planning, explicit design and orchestration of several elements (such as gamification and the learning analytics enrichment) to achieve the desirable results it can offer [12]. Our idea of flipping a MOOC has been presented during a workshop in an international conference. This idea was developed by the authors by analyzing the concepts of flipped classroom, gamification and learning analytics.
In the context of a flipped classroom, active learning strategies are required from students during the pre-classroom activities enabling them to engage and fully participate then in the classroom. In this way, discipline and self-directed learning skills are required from the student to remain engaged in these pre-classroom activities [35,36]. “Many students have under-developed self-regulation skills and need support and scaffolding to manage their learning in less familiar and more intensive settings that often characterize FL designs” [34]. This statement is even truer in the context of MOOCs in general, and the flipped one in particular. We believe that gamification can help users stay engaged and manage learning in accordance with their intentions.
In previous studies [2,29], we have involved experts from different disciplines (game design, learning sciences, and technology-enhanced learning) to assess the first selection of game elements [2,30] suitable for MOOC. As a result, we have identified ten game elements suitable to be applied in the MOOC environment designed for FL. These game elements are: (g1) empowerment; (g2) smooth learning curves; (g3) communication channels; (g4) levels; (g5) clues; (g6) goal indicators; (g7) skills tree; (g8) guilds; (g9) storytelling; and (g10) stimulated planning.
We have given these game elements to our workshop participants and presented them with the concept of flipped classroom and learning analytics. We have asked them to design a flipped MOOC and to apply the concepts explained above in their design. In this paper, we report on this study, describing the method applied and the results achieved. We then briefly discuss these results as well as the limitations of the study conducted and then draw some conclusions.

2. Materials and Methods

In the context of in an international conference dedicated to MOOC stakeholders, we have used the outcomes of the previous research, as indicated in the introduction, to design a creative expert design study, in which educational experts were briefly introduced to the concepts of MOOCs, flipped classroom, gamification, and learning analytics. Furthermore, we have presented the results of our previous work, which lead to the list of game elements presented above. We then asked the participants to apply the presented concepts in the design of a flipped MOOC aiming at creating an engaging and personalized learning experience.

2.1. Participants

Nineteen professionals with several years of experience in education participated in our study. The participants were all either practicing teachers in higher education or involved in educational research, therefore we refer to them as educational experts; however their backgrounds differ significantly with respect to the design of MOOCs, the concept of flipped classroom, or the application of gamification concepts in educational design.
About one third of the participants had prior experience in designing MOOCs or comparable experience in online learning content creation, two had experience applying a flipped classroom approach in MOOCs, and one participant had experience in an on-campus use of MOOCs. Others were more experienced in classroom-based teaching. Two of the participants had active experience in gamification, a few more were familiar with the concept of gamification previously, none were acquainted with learning analytics application.
All participants were informed that they were taking part in a study, that we were collecting the results of their participation and that their anonymized outcomes will be used to prepare a scientific publication.

2.2. Procedure

We organized the participants into small groups (4–5 participants per group) and assigned them the task of designing and conceptualizing a flipped MOOC using gamification and learning analytics applications (see Figure 1).
The participants were asked to first select a topic for their MOOC and specify how would they roughly organize the content into main sections and how much content would be required for their MOOCs.
As a next step, participants had to discuss which part of the MOOC they wanted to flip, which game elements presented to them (see Table 1) they deemed most suitable with respect to their selected topic and their specified size of the MOOC (in terms of content and structure). Based on this selection, we also asked the participants to conceptualize the game elements in order to give a first impression of a possible implementation of the game elements in the MOOC.
Finally, we asked our participants to connect their design to learning analytics. We asked them to indicate what kind of learner data is required to support the selected game elements and how this learner data could be acquired from the learners and teachers.
The whole study lasted three hours, with 30 min spent on the introduction, assignments, and group forming. Participants had then 105 min of time to work on their assignment in the groups and to prepare their mock-up. The final 45 min were used for group presentations, follow-up discussions, and a short wrap-up session.
In order to prepare the participants for their assignment, in addition to the initial presentations, we distributed information on different game elements proposed and added guiding questions, which aimed at helping the participants design or conceptualize the selected game elements in a concrete way (see Table 1).
Furthermore, participants received guiding questions for applying learning analytics to their design and for designing the whole course in terms of domain, topic, and target group (see Table 2). Participants were asked to document and present their findings as part of the group assignment by answering the guideline questions.

3. Results

Four participant groups delivered answers to the guideline questions stating their flipped MOOC concept based on the application of gamification elements and learning analytics concepts. All the results received are provided as Supplementary Materials (see link to Supplementary Materials at the end of this article).
Each group selected their own topic, covering: “Engaging students in the 21st century” (Group 1), “Organizational models” (Group 2), “How to write a dissertation” (Group 3), and “Basic Japanese” (Group 4). All groups selected higher-education target groups including teacher training cohorts (group 1), engineering students (Group 2), undergrad and/or post grad students (Group 3), and students visiting Japan (Group 4). The chosen domains where indicated as “Education for the future” (Group 1), “Dissertation” (Group 3), and “Language Learning” (Group 4), group 2 did not indicate the domain (see Table 3).
Due to the limited time of the study, only two groups worked on the LA-related part of the assignment, while all four groups worked on the gamification part. For the same reason, the groups were not able to provide a more detailed conceptualization of their MOOCs or to dive into content related designs beyond high-level considerations.

3.1. Game Elements’ Selected

All four groups chose the game elements “Levels” and “Goal indicators” in their concepts. Three of the four groups each decided to use the game elements “Empowerment”, “Smooth learning curves”, and “Communication channels” in their concepts. Two groups decided for “Guilds”. One group has chosen “Skills tree” or “Storytelling” respectively. None of the groups picked “Clues” or “Stimulated planning” (see summary on Table 4). All groups decided to apply four to six of the game elements from the list (Table 1) to their gamification concept.

3.2. Game Element Design Implication

Game element: Levels. All four groups applied the game element “Levels”. However, the way levels are conceptualized vary strongly among the different design concepts. Group 1 used successful quiz results as an enabler for accessing new content boxes (and, therefore, level). Group 2 conceived these as challenges of increasing complexity. Group 3 aligned levels to educational levels (under-graduate/post-graduate) and defined metaphors as cake ingredients, which all have to be mastered to be able to bake the cake (= graduation) at the end. Group 4 assigned levels to an initial assessment, which enables learners to choose topics of interest.
Game element: Goal indicators. Also, all four groups used goal indicators. Group 1 defined a set of different goal categories (intermediate goals, personal goals, community goals), which impose different activities to be achieved. Furthermore, the group connected goal indicators to the game element “Empowerment” by allowing learners to choose their own goals and consequently individualize their learning path. Feedback on the goal achievement status should be visible to learners. Group 2 related goal indicators to challenges and linked real world organizational systems to the goal indicators. Group 3 linked goal indicators to the core learning objectives of the MOOC, where each learning session corresponds to reaching a specific goal. Group 4 linked goal indicators with levels, where each level has a specific set of goals.
Game element: Empowerment. Three groups used this game element. Group 1 designed various input and interaction methods for different learners, allowing the learners to individualize their access to the course content. Group 2 envisioned a content related simulation system, where learners can manipulate core simulation parameters. Group 4 related empowerment to the learning path by allowing learners to freely navigate the whole course after performing the initial assessment.
Game element: Smooth learning curves. Three groups decided to use smooth learning curves as a part of their design. Group 2 applied smooth learning curves to the varying and increasing complexity of the simulation system. Group 3 provided different levels of quizzes, which can be taken. Group 4 bases their smooth learning curve approach on initial basic assessments.
Game element: Communication channels. Three groups have chosen this game element. Group 1 did not however, provide a design consideration for this game element. Group 2 aimed to enable forum and chat-based communication channels in their MOOC. Group 4 proposed separate topic-related discussion rooms. Based on participant interests, different discussion rooms can be offered to the learners.
Game element: Guilds. Two of the four groups included guilds in their concept. Group 2 aimed to let learners self-organize into groups, which have to solve specific challenges together. The whole group would have received public feedback to their challenge result. Group 3 rather proposed an automatic assignment of participants to guilds based on subject chosen.
Game element: Skills tree. One group opted for this game element by providing a visualization of acquired skills to the learner according to a chosen scenario.
Game element: Storytelling. One group aimed to use storytelling for language learning, by envisioning stories to be developed by the participants in a collaborative manner. This way, the group connected storytelling with communication channels.
Game elements: Clues, Stimulated Planning. None of the groups selected one of these patterns and consequently no design concept has been provided.

3.3. Learning Analytics Concepts

Two of the groups worked on the learning analytics part of the assignment (Group 1 and Group 2) and specified what data can be used to support selected game elements and how the game elements can be affected by learning analytics.
Group 1 specified a number of data sources, which can be used for learning analytics, mainly based on course interactions (content access, time spent, activity completion, interaction, and communication), and which can support personalization of learning processes. Furthermore, this data can be used to adapt the game elements in order to address engagement and retention issues. The data can also support reflection and debriefing phases.
Group 2 aimed to support the game elements “Empowerment”, “Communication Channels”, “Levels”, “Goal indicators”, and “Guild” with specific learning analytics approaches. For the game element “Smooth Learning Curves” group 2 does not foresee learning analytics support.
Empowerment is supported by connecting simulation data with real-world data to generate immediate feedback on key performance parameters of the simulation. In the case of communication channels, learning analytics helps to assess individual contributions in relation to overall group activities. The process of leveling up is supported through the analysis of learner progress data (“Levels”), which also fosters the display of current goal achievement status (“Goal Indicators”). For “Guilds”, measuring intra-group interaction (based on chat activity) as well as the challenge completion rate is proposed.

4. Discussion

Selecting and designing game elements and applying them to educational contexts are interdisciplinary, creative, and complex processes. Game elements allow for various ways of implementation and visualization but also generate interdependencies and possible conflicts. Some game elements, such as “Empowerment”, allow for learner control in process and progress and thus support personalization, while others, i.e., “Storytelling”, rather require the following of a predefined path to provide a meaningful story. While both game elements can be motivating for learners, it is difficult to combine both game elements in one MOOC [29].
In order to assess the possible value of specific game elements for their application in flipped MOOCs, we have followed a multi-step research approach involving a literature review, expert studies with experts of various domains (learning sciences, game design, technology-enhanced learning) to narrow down the amount of game elements to a promising selection [30,37].
This previous work lead to the selected list of game elements and also resulted in the design guidelines used to utilize these elements in the context of MOOCs. Clearly, the selected game elements are not exclusive and the design guidelines given to the participants are not the only way to reflect on specific game elements and their use in MOOCs. We are aware that the guidelines might produce a specific bias to the participants. However, since the selection of game elements and the corresponding guidelines are based on previous studies, we support that they help inform the participants reflect on the utilization of specific game elements and provide orientation considering rather short timeframe of the study [2,30,37].
We are also aware that we combined complex concepts into a single intervention (MOOCs, flipped classroom, gamification, learning analytics), and thus ran the risk of acquiring hard-to-compare and hard-to-measure results. However, since the aim of this study was to understand the concept of a flipped MOOC and derive corresponding design implications from the point of view of educators, we believe that our study still contributes to the understanding of the key concepts and their application.
Based on the pre-selection, we here reported a further study with educational practitioners and educational researchers to understand better, which particular design considerations can help to implement our concept of flipped MOOCs.
With nineteen participants split in four groups, our expert study cannot, of course, conclusively rate specific game elements as better apt for the purpose than others. However, we can use the expert recommendations and their design input as indicators for the design of a flipped MOOC concept based on gamification and learning analytics. As such, this study does not contribute empirical evidence about the potential effects of the flipped MOOC approach and it does not claim to deliver the best possible conceptualization of a flipped MOOC based on the underlying gamification and learning analytics concepts. However, we argue that this study contributes to the understanding of the design of flipped MOOCs as a conceptual idea to create more interactive learning and personalized experiences with MOOCs. Based on the expertise of a group of educational experts with various backgrounds, we could indicate ways towards the design of flipped MOOCs.
With levels, goal indicators, empowerment, smooth learning curves, and communication channels as most chosen game elements, our experts clearly voted for game elements, which are engaging (levels, communication channels), motivating (goal indicators, smooth learning curves), and personalization oriented (empowerment, goal indicators). These game elements are expected to foster interaction, communication, and self-organization in the learning processes.
Our concept of flipped MOOC relies on turning MOOCs from mainly content-oriented delivery machines into personalized, interactive, and engaging learning environments with support from selected game elements and learning analytics. This comes along with a shifted perspective on success in a MOOC: while previous measures mainly accounted for completion rates or certificates earned, modern approaches take user intentions into account and calculate individual goal achievement rates [2,38,39], only with this shift in success measurement, the flipped MOOC is possible: individualizing the learning experience does not necessarily deliver comparable results in terms of completion rates. Consequently, a personalized MOOC would be rated as less successful. Taking the individual intentions, expectations, and goals into account may balance this view [39]. By selecting gamification elements, which foster these personalized experiences, our participants confirm this view.
The outcomes produced by the educational experts included in our expert design study mainly indicate the relevance of this new concept. The participants were also able to apply the presented concepts (MOOC, gamification, learning analytics, flipped classroom) into their conceptualization and design of a flipped MOOC for a self-chosen topic. All groups mainly followed the guideline questions given to them (Table 1 and Table 2) and answered in textual form. Group 3 additionally created a visual presentation of their findings presented during the discussion round.
The groups spent less of their time on the aspects related to learning analytics. This is, as participants reported, partly due to short time given to them but also due to learning analytics being introduced in relation to the gamification concepts selected. This way, participants had to first tackle the gamification design before they could address the LA-related issues involved.
The differences found in the various concepts mainly support the statement that designing game elements for learning environments is still a creative and complex routine, which is open to variations. Also, we can see that game elements chosen have been selected partly with the domain and topic of the MOOC in mind: group 1, focusing on student engagement, mainly selected game elements for communication and personalization (empowerment, communication channels, skills tree, goal indicators, and levels). Group 2, working on simulations for organizational models related their choice of game elements to the simulation experience and connected the design of their MOOC to real-world data giving feedback on simulation progress and success. Group 3 organized their MOOC about writing a dissertation on levels according to the academic procedures and thus designed a strong connection of learning process and game element. Group 4, dealing with learning the Japanese language, applied storytelling as a key game element to learning scenarios. They have incorporated collaborative approaches to develop and tell these stories as a part of the community approach, which is, in fact, one possible way to solve the abovementioned conflict between empowerment and storytelling. All of the four groups claimed that they have enhanced student engagement, community building, and interactivity was their guiding principle. Yet the selected topic and domain yielded varying concepts and designs.

5. Conclusions

As a key contribution of this article, we aim to introduce the concept of a flipped MOOC based on gamification and learning analytics as a way to provide more engaging, personalized, interactive, and community-oriented learning experiences in MOOCs. Based on a series of expert studies, we presented and discussed a set of design concepts for flipped MOOCs utilizing the mentioned concepts serving as input for the participants of our study.
While there is a clear overlap in the selected game elements educators used to design their MOOC, the concrete design considerations vary according to topic and domain considerations but also according to individual creativity and design ideas.
As a matter of fact, our study is clearly limited due to the short period of time used and due to the limited number of participants included. Nevertheless, we believe this large variety is also a confirmation of the complexity involved in applying game elements to educational processes. Furthermore, we believe that this variety is an indicator, that gamification should not be applied in a content- or topic-agnostic way, but rather support the concrete educational approach chosen by the designer of a MOOC. Exactly at this point, we see a major shift away from previously dominant content-agnostic gamification approaches that rather relied on extrinsic motivational factors triggered by game elements such as points, badges, and leaderboards, for which long-term effects are at least questionable [40]. The set of expert studies we conducted confirm our view that gamification concepts should also support intrinsic motivational factors, engagement, community-building and personalization. In line with previous results [37,41,42,43,44,45], the designs created by our participants indicate that gamification can be helpful in education only if it is carefully selected and designed chosen according to the application scenario at hand (problem to be solved/addressed, target group, topic, domain, content).
In the context of our flipped MOOC concept, we thus believe that a well-balanced application of game elements, selected to match the topic, the domain, and the target group of the MOOC and in particular the problem that needs to be solved, can deliver an engaging, personalized learning experience. Learning analytics serves as an underlying key concept to relate individual learner data to overall learner activity, which is required for many of the game elements to reveal their full potential.
As a next step, we plan to design a MOOC in accordance with the instructional design concept presented in this paper. Within that flipped MOOC we will experimentally evaluate effects on users’ engagement, community-building, and goal achievement.

Supplementary Materials

The following are available online at Presentation S1: Briefing Presentation, Table S2: Assignment Template, Tables S3–S6: Group results for each of the four groups, Presentation S7: Result presentation by group 3.


The research reported here has been funded on internal cost of the Open University of the Netherlands and Tallinn University. Costs for publishing open access are also covered on internal funds. We specifically thank the participants of our study for their contributions.

Author Contributions

Alessandra Antonaci and Maka Eradze conceived and designed the study; Alessandra Antonaci, Maka Eradze, and Roland Klemke performed the study; Roland Klemke and Maka Eradze analyzed the data; Roland Klemke, Maka Eradze, and Alessandra Antonaci wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.


  1. The Right to Education|Education|United Nations Educational, Scientific and Cultural Organization. Available online: (accessed on 1 March 2017).
  2. Antonaci, A.; Klemke, R.; Stracke, C.M.; Specht, M. Gamification in MOOCs to enhance users’ goal achievement. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON 2017), Athens, Greece, 25–28 April 2017. [Google Scholar]
  3. Kalz, M.; Kreijns, K.; Walhout, J.; Castaño-Munoz, J.; Espasa, A.; Tovar, E. Setting-up a European Cross-Provider Data Collection on Open Online Courses. Int. Rev. Res. Open Distrib. Learn. 2015, 16, 62–77. [Google Scholar] [CrossRef]
  4. Reich, J. Learner Intention Recasts “Low” MOOC Completion Rates|HarvardX. Available online: (accessed on 4 October 2016).
  5. Meinel, C.; Schweiger, S. A Virtual Social Learner Community—Constitutive Element of MOOCs. Educ. Sci. 2016, 6, 22. [Google Scholar] [CrossRef]
  6. Karlsson, N.; Godhe, A.-L. Creating a Community Rather Than a Course—Possibilities and Dilemmas in an MOOC. Educ. Sci. 2016, 6, 18. [Google Scholar] [CrossRef]
  7. Toto, R.; Nguyen, H. Flipping the work design in an industrial engineering course. In Proceedings of the 39th IEEE Frontiers in Education Conference FIE’09, San Antonio, TX, USA, 18–21 October 2009; pp. 1–4. [Google Scholar]
  8. Gooch, D.; Vasalou, A.; Benton, L. Exploring the use of a gamification platform to support students with dyslexia. In Proceedings of the IISA 2015—6th IEEE International Conference on Information, Intelligence, Systems and Applications, Corfu, Greece, 6–8 July 2015. [Google Scholar]
  9. Utomo, A.Y.; Santoso, H.B. Development of gamification-enriched pedagogical agent for e-Learning system based on community of inquiry. In Proceedings of the International HCI and UX Conference in Indonesia, Bandung, Indonesia, 8–10 April 2015; pp. 1–9. [Google Scholar]
  10. Hamari, J.; Koivisto, J.; Sarsa, H. Does gamification work?—A literature review of empirical studies on gamification. In Proceedings of the 7th IEEE Hawaii International Conference on System Sciences (HICSS), Waikoloa, HI, USA, 6–9 January 2014; pp. 3025–3034. [Google Scholar]
  11. Sarawagi, N. A flipped CS0 classroom: Applying Bloom’s taxonomy to algorithmic thinking. J. Comput. Sci. Coll. 2014, 29, 21–28. [Google Scholar]
  12. Bishop, J.L.; Verleger, M.A. The flipped classroom: A survey of the research. In Proceedings of the ASEE National Conference, Atlanta, GA, USA, 23–26 June 2013; pp. 1–18. [Google Scholar]
  13. Sergis, S.; Sampson, D.G.; Pelliccione, L. Investigating the impact of Flipped Classroom on students’ learning experiences: A Self-Determination Theory approach. Comput. Human Behav. 2017, 78, 368–378. [Google Scholar] [CrossRef]
  14. O’Flaherty, J.; Phillips, C. The use of flipped classrooms in higher education: A scoping review. Internet High. Educ. 2015, 25, 85–95. [Google Scholar] [CrossRef]
  15. Rodríguez, M.F.; Correa, J.H.; Pérez-Sanagustín, M.; Pertuze, J.A.; Alario-Hoyos, C.A.; Alario-Hoyos, C. A MOOC-Based Flipped Class: Lessons Learned from the Orchestration Perspective. In European Conference on Massive Open Online Courses; Springer: Cham, Switzerland, 2017; pp. 102–112. [Google Scholar]
  16. Sancassani, S.; Casiraghi, D.; Corti, P.; Trentinaglia, N. MOOC, OER e l’approccio “flipped classroom”: Due case study di transizione in ambito scolastico e aziendale. Form@re 2014, 14, 49. [Google Scholar]
  17. Murphy, J.; Tracey, J.B.; Horton-Tognazzini, L. MOOC camp: A flipped classroom and blended learning model. In Information and Communication Technologies in Tourism; Springer: Cham, Switzerland, 2016; pp. 653–665. [Google Scholar]
  18. Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L. From Game Design Elements to Gamefulness: Defining “Gamification”. In Proceedings of the 15th ACM International Academic MindTrek Conference: Envisioning Future Media Environments, Tampere, Finland, 28–30 September 2011; pp. 9–15. [Google Scholar]
  19. Dicheva, D.; Dichev, C. Gamification in Education: Where Are We in 2015? In Proceedings of the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Kona, HI, USA, 19–22 October 2015; pp. 1445–1454. [Google Scholar]
  20. Hamari, J. Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electron. Commer. Res. Appl. 2013, 12, 236–245. [Google Scholar] [CrossRef]
  21. Hamari, J.; Koivisto, J. Why do people use gamification services? Int. J. Inf. Manag. 2015, 35, 419–431. [Google Scholar] [CrossRef]
  22. De-Marcos, L.; Domínguez, A.; Saenz-De-Navarrete, J.; Pagés, C. An empirical study comparing gamification and social networking on e-learning. Comput. Educ. 2014, 75, 82–91. [Google Scholar] [CrossRef]
  23. Huang, B.; Hew, K.F. Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification. In Proceedings of the 23rd International Conference on Computers in Education, Hangzhou, China, 30 November–4 December 2015; pp. 275–280. [Google Scholar]
  24. Mazarakis, A. Using Gamification for Technology Enhanced Learning: The Case of Feedback Mechanisms. Bull. IEEE Tech. Comm. Learn. Technol. 2015, 17, 6–9. [Google Scholar]
  25. Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in Education: A Systematic Mapping Study Gamification in Education: A Systematic Mapping Study. Educ. Technol. Soc. 2015, 18, 75–88. [Google Scholar]
  26. Chang, J.W.; Wei, H.Y. Exploring engaging gamification mechanics in massive online open courses. Educ. Technol. Soc. 2016, 19, 177–203. [Google Scholar]
  27. Christy, K.R.; Fox, J. Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Comput. Educ. 2014, 78, 66–77. [Google Scholar] [CrossRef]
  28. De-Marcos, L.; Garcia-Lopez, E.; Garcia-Cabot, A. On the effectiveness of game-like and social approaches in learning: Comparing educational gaming, gamification & social networking. Comput. Educ. 2016, 95, 99–113. [Google Scholar]
  29. Björk, S.; Holopainen, J. Patterns in Game Design, Game Development Series; Charles River Media: Newton Centre, MA, USA, 2004. [Google Scholar]
  30. Antonaci, A.; Klemke, R.; Stracke, C.M.; Specht, M. Identifying game elements suitable for MOOCs. In Proceedings of the 12th European Conference on Technology Enhanced Learning, Tallinn, Estonia, 12–15 September 2017. [Google Scholar]
  31. Lang, C.; Siemens, G.; Wise, A.; Gasevic, D. (Eds.) Handbook of Learning Analytics; Society for Learning Analytics Research: Beaumont, AB, Canada, 2017. [Google Scholar]
  32. Kennedy, G.; Corrin, L.; Lockyer, L.; Dawson, S.; Williams, D.; Mulder, R.; Khamis, S.; Copeland, S. Completing the loop: Returning learning analytics to teachers. In Proceedings of the 31st Annual Conference of the Australian Society for Computers in Tertiary ducation, ASCILITE 2014, Dunedin, New Zealand, 23–26 November 2014; pp. 436–440. [Google Scholar]
  33. Frederickson, N.; Reed, P.; Clifford, V. Evaluating Web-supported learning versus lecture-based teaching: Quantitative and qualitative perspectives. High. Educ. 2005, 50, 645–664. [Google Scholar] [CrossRef]
  34. Jovanovic, J.; Mirriahi, N.; Pardo, A.; Dawson, S.; Gašević, D. An Analytics-based Framework to Support Teaching and Learning in a Flipped Classroom. In Learning Analytics in the Classroom: Translating Learning Analytics Research for Teachers; Routledge: Oxon, UK, 2017. [Google Scholar]
  35. Lai, C.-L.; Hwang, G.-J. A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Comput. Educ. 2016, 100, 126–140. [Google Scholar] [CrossRef]
  36. Mason, G.S.; Shuman, T.R.; Cook, K.E. Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Trans. Educ. 2013, 56, 430–435. [Google Scholar] [CrossRef]
  37. Antonaci, A.; Peter, D.; Klemke, R.; Bruysten, T.; Christian, M.; Specht, M. GMOOCs—Flow and Persuasion to Gamify MOOCs. In Proceedings of the Games and Learning Alliance Conference (GALA 2017), Lisboa, Portugal, 5–7 Dcecember 2017. [Google Scholar]
  38. Henderikx, M.A.; Kreijns, K.; Kalz, M. Refining success and dropout in massive open online courses based on the intention—Behavior gap. Distance Educ. 2017, 38, 353–368. [Google Scholar] [CrossRef]
  39. Henderikx, M.; Kreijns, K.; Kalz, M. To change or not to change? That’s the question. On MOOC-success, barriers and their implications. In Digital Education: Out to the World and Back to the Campus; Delgado Kloos, C., Jermann, P., Pérez-Sanagustin, M., Seaton, D.T., White, S., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 210–216. [Google Scholar]
  40. Kyewski, E.; Krämer, N.C. To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Comput. Educ. 2017, 118, 25–37. [Google Scholar] [CrossRef]
  41. Kelle, S.; Klemke, R.; Specht, M. Design patterns for learning games. Int. J. Technol. Enhanc. Learn. 2011, 3, 555–569. [Google Scholar] [CrossRef]
  42. Kelle, S.; Klemke, R.; Gruber, M.; Specht, M. Standardization of Game Based Learning Design. In Computational Science and Its Applications—ICCSA 2011—Lecture Notes in Computer Science; Murgante, B., Gervasi, O., Iglesias, A., Taniar, D., Apduphan, B.O., Eds.; Springer: New York, NY, USA, 2011; pp. 518–532. [Google Scholar]
  43. Kelle, S.; Klemke, R.; Specht, M. Effects of Game Design Patterns on Basic Life Support Training Content. Educ. Technol. Soc. 2013, 16, 275–285. [Google Scholar]
  44. Antonaci, A.; Klemke, R.; Specht, M. Towards Design Patterns for Augmented Reality Serious Games. In The Mobile Learning Voyage—From Small Ripples to Massive Open Waters; Springer: Cham, Switzerland, 2015; pp. 273–282. [Google Scholar]
  45. Schmitz, B.; Kalz, M.; Klemke, R.; Specht, M. In the eye of the beholder: Promoting learner-centric design to develop mobile games for learning. In International Conference on Mobile and Contextual Learning; Springer: Cham, Switzerland, 2014. [Google Scholar]
Figure 1. Workshop assignment: towards designing a flipped MOOC.
Figure 1. Workshop assignment: towards designing a flipped MOOC.
Education 08 00025 g001
Table 1. Game element descriptions and design guideline questions.
Table 1. Game element descriptions and design guideline questions.
Game ElementDescriptionHow Do We Use It?
Enables a user to perceive that he/she can impact the situation, have a sense of controlWhat are the decisions an empowered learner can take?
Smooth Learning Curves
Keeps the user in a continuous learning progress and in mastering his/her skillsHow can we smoothen learning curves for different learners?
Communication Channels
Enables communication via chats, forum, etc.Which communication channels are specifically needed?
Part of the game in which all player actions take place until a certain goal has been reached or an end condition has been fulfilledHow do we distinguish between the different levels? What is specific for each level?
Hints given to users to solve a problem and/or to achieve a goal and unblock the next levelHow do we balance clues (not too many, not too few, being informative without giving away too much information)?
Goal Indicators
Indicators, which make the goals explicitWhich goals can be selected/defined?
Skills tree
Enables custom configurations of a character’s abilities, often organized in branchesWhich skills can we distinguish? How can we organize them in a hierarchy?
Associations of players who chose to come together to achieve a common goalHow can learners team up in guilds? What is the goal of each guild? How does it compete with other guilds?
The act of telling stories within the gameWhich story do we want to tell?
Stimulated Planning
Enabling and stimulating users to plan future actions or goals and thus guiding them to define the strategy to follow in order to complete a level or the game itselfIn a flipped MOOC how do you foresee the implementation of Stimulated Planning, a game elements that invite the users to plan their strategy?
Table 2. Guiding questions for learning analytics and the course design.
Table 2. Guiding questions for learning analytics and the course design.
Learning Analytics Conceptualization
For each game element selected:Why do we use it?
For whom?
How do we use it?/how do we implement it?
What data do we need to support that?
Design Phase
DomainIn which domain is our MOOC placed?
TopicWhich topic(s) should it cover?
Target group in mindFor which target group do we design it?
Table 3. Groups, topics, and domains.
Table 3. Groups, topics, and domains.
Group 1Engaging students in the 21st centuryEducation for the future
Group 2Organizational models-
Group 3How to write a dissertationDissertation
Group 4Basic JapaneseLanguage Learning
Table 4. Use of game elements by the groups.
Table 4. Use of game elements by the groups.
Game Elements 1234Total
1. EmpowermentYYNY3
2. Smooth Learning CurvesNYYY3
3. Communication ChannelsYYNY3
4. LevelsYYYY4
5. CluesNNNN0
6. Goal IndicatorsYYYY4
7. Skills treeYNNN1
8. GuildNYYN2
9. StorytellingNNNY1
10. Stimulated PlanningNNNN0
Total Patterns per Group5646-

Share and Cite

MDPI and ACS Style

Klemke, R.; Eradze, M.; Antonaci, A. The Flipped MOOC: Using Gamification and Learning Analytics in MOOC Design—A Conceptual Approach. Educ. Sci. 2018, 8, 25.

AMA Style

Klemke R, Eradze M, Antonaci A. The Flipped MOOC: Using Gamification and Learning Analytics in MOOC Design—A Conceptual Approach. Education Sciences. 2018; 8(1):25.

Chicago/Turabian Style

Klemke, Roland, Maka Eradze, and Alessandra Antonaci. 2018. "The Flipped MOOC: Using Gamification and Learning Analytics in MOOC Design—A Conceptual Approach" Education Sciences 8, no. 1: 25.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop