Towards an Engaging and Gamified Online Learning Environment—A Real CaseStudy ^†

Abstract

Currently, remote work is common, and this trend has come to several areas and processes, such as education and teaching. Regarding higher education, universities have several challenges to overcome, the most challenging being transforming teaching to be more digital and engaging. Therefore, TechTeach has arisen as a new teaching paradigm that creates a unique learning environment and satisfies students’ and professors’ expectations. After the success of the b-learning approach, professors created new experiences utilizing an entirely online learning environment following this paradigm. This article shows the work performed through a real case study, explains the strategy used to implement this paradigm, provides students’ opinions, and analyses the results achieved. The results demonstrated that, while the effort was tremendous, the result was beneficial to all. After 208 online hours of classes, 11,173 downloads, 15,224 messages, 200,000 sessions, 3 rescues requests, and 28t cards, 98.15% of the active participants gave it their approval, 96.53% considered this subject equal to or better than the others, and 85% of accepted the gamification system. These results show that a class can be an engaging environment where students can learn and enjoy it regardless of whether it is physical or not.

1. Introduction

Universities are experiencing several changes induced by the technological and social trend towards digitisation, with students and professors being the main stakeholders of the ecosystem. On the one hand, professors have well-defined teaching strategies, and on the other, students are becoming even more demanding and engaging more in digital platforms. The coronavirus pandemic (COVID-19) has helped spur these changes [1] and brought new challenges to education, by which students, schools, and professors have been led to this sudden digital transformation without being ready for it [2].

Since 2017, the leading professor of this study has given out a questionnaire every year at the beginning of class. Until the present study, almost 400 (399) students were asked if they preferred an interactive or traditional way of learning/teaching. Most students (86.22%) chose interactive classes over formal lessons. Since then, several methods have been developed, and some technologies have been tested. This research process led to the development of TechTeach, a new teaching paradigm that uses technology and combines several methods to create a unique learning ecosystem.

TechTeach started as a b-learning approach and evolved into a fully online experience. Therefore, to test it online, the author of this approach created a proof of concept. The professor planned the classes, defined the strategy, and implemented it using a real case study. He deployed the case study at the University of Minho in a Curricular Unit (CUnit) of Web Programming with theEuropean Credit Transfer and Accumulation System (ECTS) with 166 students in the year 2020/2021. This case study also included an actual project to develop the students’ (soft and technical) skills and create a possible solution to support the work of the Portuguese Intervention and Civil Forces.

The primary goals of this case study were to assess the viability of TechTeach in an e-learning context, monitor all the activities performed online, and collect the students’ opinions. It used team- and project-based learning and flipped and interactive classes and explored tools such a Zoom, ioEduc, ioChat, Kahoot, and HackerRank to ensure the proper execution of the paradigm. The case study also promoted gamification activities to increase students’ engagement and assess their performance. This article extends the previous work that introduced the online-teaching environment with gamification [3], adds more details about and process, and presents more results.

The paper is divided into six sections: the Introduction, which presents the work, and the Background, providing the paper’s topics and related works. Then, Section 3 exposes all the methods and tools used in this case study. The case study is described in Section 4, and the results achieved are analysed in the Discussion Section. Finally, the conclusion of the work is provided, and the future work is explained.

2. Background

This provides the case study’s theoretical context and explains some of the concepts addressed.

2.1. e-Learning

e-learning is an alternative teaching method for existing learning activities [4]. This concept emerged with the appearance of online technologies, paradigms, and ideas. In this work, the professors adapted the traditional theories of distance learning to the online world. The content was organised according to the conventional models and delivered online to a group of students. Typically, the professor leads the process, following a specified curriculum to be completed at a predetermined pace [5].

According to Wiharko et al. [4], “The implementation of e-learning requires the belief of all parties that e-learning has the power to improve a learning environment that is beneficial for students, in addition to mastery and maturity of concepts and accuracy in choosing strategies and evaluations.” In order for the stakeholders to enjoy a complete experience of e-learning, it is essential to have the right conditions to promote online classes. e-learning and, in particular, online platforms allow studying a particular area or discipline as a whole by promoting self-education, which must be safe and of high quality [6].

e-learning platforms are essential to help answer the observed changes to distance teaching and learning during the COVID-19 pandemic. This situation has created a real challenge for both instructors and students in the promotion of online learning environments [7].

2.2. Online Learning

Classes with online learning have several characteristics that impact faculty definition and course [8]. Converting a classroom from face-2-face to online is one of the biggest challenges in education. According to a study [9], online and remote learning in higher education institutes is necessary during COVID-19. This approach came to stay, and it is a real possibility from now to the future. Wahab Ali [9] also mentioned that the massive technological advance represents a shift in educational goals and aspirations. Online classes can have better results with bigger classes [10]; however, the typical conversion of teaching environments is not the best solution.

One of the challenges of online learning is to apply the Discover, Learn, Practice, Collaborate, and Assess (DLPCA) strategy, as can be observed from some recent studies [7,11,12]. One of the challenges was aligning DLPCA layers with online teaching and learning constraints/plans to reduce the impact of the issues. Thus, and according to [7], a DLPCA proposal should consider:

• Educational theory;
• Socio-technical constraints in online teaching and learning;
• The role of instructor, student and technologies;
• Revised course plan and checklists;
• Asynchronous and Synchronous teaching and learning tasks;
• Assessments and learning outcomes.

Although this work did not follow a DLPCA presentation, it was concerned about addressing all aspects of it. The DLPCA strategy was helpful to understand whether TechTeach is or is not suitable for online learning.

2.3. Digital Classrooms

A digital transformation process can exploit wisdom and promote new practices of learning, teaching, schoolwork, and communicating [13,14] among the stakeholders. The digital skills can be identified and attributed to someone who can use, without support, digital devices (e.g., laptops) and several digital learning resources [15]. Besides that, the context/environment sets the digital competencies of the participants (e.g., professors), which take decisions based on their values, principles, approaches, and frameworks. The process of digital transformation in the classrooms should not forget the principles of instruction [16]:

1.

Students are engaged in solving real-world problems.

2.

Available knowledge is activated as a base of wisdom.

3.

New knowledge is

• applied by the professor;
• demonstrated to the student;
• integrated into the participant (e.g., student) world.

After ensuring the principles are accomplished, the instructor, student, and technology will have a vital role in the online classroom [7]. It is the critical point to have a good learning environment.

2.4. Related Works

This paper’s main goal is to show obtained results with the online TeachTeach study. This article does not address or compare it with other teaching methods. Regarding TechTeach, the author already exposed the paradigm and its application in a b-learning environment [17,18]. As related work should be considered, the article published in ICPEC 2021 [3] primarily presented the case study and addressed the achieved results. This extended version details the case study, shows new achievements, and provides a depth analysis of them.

3. Material and Methods

This section presents the TechTeach paradigm, related approaches, strategies, and techniques explored.

3.1. TechTeach

Originally, TechTeach was defined as a teaching paradigm that combines a set of topics such as flipped classrooms, team-based learning, project-based learning, gamification, blended-learning, soft skills, quizzes, and surveys to provide the student with the best learning environment possible [18]. According to the author [17], TechTeach is able to adapt to different contexts. It wants to make the learning environment more digital, fun, and attractive by increasing student engagement and using gamification.

After the first successful experience in a face-to-face setting (2019/2020) with good results (the approval rate by the students was 81%) [17], it was time to challenge this paradigm in a fully online mode using the same CUnit and course but in a different year (2020/2021). The case study presented in this article explored this approach in a non-face-2-face setting and tried to show/assess its suitability for online teaching environments, providing all the achieved results. The following subsections explain the topics addressed and implemented during the classes.

3.1.1. Project-Based Learning (PBL)

PBL is a pedagogical approach centred on the student [19,20,21] and explores the use of exercises to provide the most practical learning. It involves a dynamic classroom approach where professors exploit real-world problems and challenges and stimulate students to increase their knowledge [21].

3.1.2. Team-Based Learning

(TBL) TBL represents a form of learning in groups and an excellent complement to PBL. It combines small and large learning groups [22] and is an application-oriented teaching method. TBL also shows the ability to work well within a team and the ability to learn in a group [23].

3.1.3. Gamification

Gamification is the use of “game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems” according to Kapp et al. [24]. The use of gamification in learning environments can positively impact the learning outcome [25]. It is being used to motivate students to realise specific tasks and improve their motivation and concentration [26]. In this case, the use of a narrative is essential to measure the engagement and impact of gamification activities [27].

3.1.4. Flipped Classes

This instructional approach is used to support teaching [25] and allow students to have their learning environments outside of the classroom [28]. This class type is different and more practical because professors can use contact time to discuss the most relevant topics and clarify students’ doubts instead of having monotonous exposition lessons.

3.1.5. Skills

It is a natural process of human development [29]. This process can be divided into two types: technical and soft. The first type is concerned with all technical and professional knowledge that a human can learn. Soft-skills represents human actions, behaviour, and beliefs. Adapting and optimising skills is demanded in the modern world [30]. Regarding the educational field, this process elevates the cognitive and constructionist pedagogical perspectives [29]. This case study trained the students in two types of skills: technical (web programming and similar) and soft (argumentation, adaptability, leadership, resilience, communication, entrepreneurship, innovation, among others).

3.2. Tools

This subsection presents the tools used in this case study and explains how they enhanced the use of TechTeach.

3.2.1. Teach Supporting

This case study used three tools to support the teaching process and ensure the correct execution of online classes:

• ioEduc helps teachers and students with their duties [31]. It is an excellent tool to support e-learning courses because it has several web-based features (e.g., attendances, live classes, chat, slides, quizzes, drive, FAQs, surveys, peer-assessment). This tool also integrates ioChat, among other features.
• ioChat arose as an internal communication platform of IOTech, and they then explored it in the educational context. It is based on RocketChat, a fully customised open-source software. It has many features, such as messages/conversations (text, video or audio), rooms (public, group, private, or discussion), notifications (message, email, online, push), plugins (polls, drive, calendar, reminder), among others.
• Zoom allows people to maintain high-quality meetings with many participants. It is one of the most recognised video-conferencing platforms and allows, for example, creating an online teaching environment and splitting students into teams (breakout rooms). Zoom is accessible and can connect long-distance participants across different rooms systems, desktops, and mobile devices [32].

These tools are pervasive systems [33] and are available to anyone anywhere, anytime, and from any device (e.g., mobile, tablet, or desktop) after using the authentication system.

3.2.2. Collaborative and Competitive

In this field, professors explored two applications: Kahoot and HackerRank. Both tools help to realise the assessment processes online.

• Kahoot! is characterised as a response system that engages participants through game-like pre-made or impromptu quizzes, discussions, and surveys [34]. Its game-based learning allows professors to support their education activities [35]. According to Carolina et al. [36], “Instructional games are gaining acceptance in the classroom as the e-learning merits of student engagement, and immediate feedback is recognised”.
• HackerRank [37] is a classic competitive programming platform [38] with several different types of exercises (e.g., problem-solving or challenges). People can use it to recruit workers or test their knowledge.

3.2.3. Programming

This case study was performed in a programming course, so professors used several tools to support the learning process. Most of the applications were available online or run locally with online data. The list of explored tools is the following:

• AWS C9;
• GitHub;
• GitHub Pages;
• Heroku;
• MySQL Workbench;
• Visual Studio Code.

Figure 1 shows the tools used in the context of this CUnit. As can be observed in the figure, the tools are divided by front-end and development.

3.2.4. Data Analytics

The last tool was essential for measuring achievements and the paradigm’s impact. Google Analytics is a web tool that helps to analyse websites’ traffic [39]. This tool is essential for understanding the tools/websites’ impact/usage.

3.3. Research Process

As this article stated, it presents a case study, so the case study methodology was applied following the process detailed in the Figure 2.

The case study research method is described as a generic term for works that commonly decide to focus on an investigation around a specific instance or event [41], such as TechTeach. It allows the researcher to focus on particular cases to identify interactive processes that may be crucial but transparent in large-scale research [41], such as online learning. The principal researcher defined the following strategy/plan for this case study:

1.

Developed Theory—TechTeach [17].

2.

Select cases—Face-to-Face and Online.

3.

Data Collection—The acquisition process was designed to include only digital data automatically collected from online platforms, including Kahoot (spreadsheets), BlackBoard (spreadsheets), ioEduc (database), ioChat (database), and Google Analytics (reports), and the answers provided by the students in the surveys (spreadsheets). All data were made anonymous and followed the General Data Protection Regulation rules as closely as possible. For example, students were asked to allow data collection since the beginning of the CUnit.

4.

Case Study 1—The first case study was realised in 2019/2020, and all the details and individual reports are available in previous works [17,18]. TechTeach had minor improvements based on the feedback received and results achieved in the first study. Then, the 2nd case study was designed to assess the new developments released.

5.

Case Study 2—The second case study was performed in 2020/2021, and this article presents the individual case report containing some of the previous results [3].

6.

Final cases and cross-case conclusions—The researcher selected the second case study as the most relevant because it addresses a tendency: “the remote work”. The analysis of it is available in the discussion section. As a result, it was possible to observe that the theory (paradigm) suited the two environments (face-to-face and online). Still, professors can improve it to achieve a better fit in the online mode.

The present work is in the modify theory phase, where both case studies will be dissected, and a new updated version of the TeachTeach paradigm will be released. The following section presents the second case study performed online to assess the TechTeach adequacy.

4. Case Study

This section explains the context, challenges, and approaches used in the case study. The case study explored the applicability of TeachTeach in an online environment to answer two questions:

• “Is TeachTeach ready or adapted to classes 100% online?”;
• “What is the effort needed? Is it worth it?”.

In the beginning, the coordinator professor and mentor of TechTeach defined two practical goals:

1.

Innovating and guaranteeing the success of a 100% Online CUnit;

2.

Using a set of trends/concepts: team and project-based learning, gamification, quizzes, flipped classes, and emerging tools for providing the best online learning environment to the students.

4.1. Context

The Curricular Unit (CUnit) used in this case study was part of the Information System Integrated Master at the University of Minho, run online, and had the following features:

• European Credit Transfer and Accumulation System (ECTS): 10 (Ten);
• Degree: Integrated Master;
• Year of study plan: 2nd;
• Online period: from 5 of October 2020 and 29 of January 2021;
• Registered students: One hundred and sixty-eight (168 (90% active participants, i.e., attended more than 50% of classes and 5% of inactive, i.e., the attendance rate was less than 5%));
• Professors: 5 (Five);
• Formats of classes: 3 (Three)—Theoretical (T), Theoretical-practice (TP), laboratory practices (LP);
• Weeks of work: 15 (Fifteen);
• Weeks of contact classes: 13 (Thirteen);
• Hours of work by week/student: 14 (Fourteen);
• Hours of contact by week: 6 (Six);
• Tutorial Working Time (OT) hours: 2 (Two).

The teaching team designed the working/teaching plan of Web Programming based on the following premises:

• Full online classes (100%);
• Gamification mechanisms to assess student’s performance;
• An Online communication platform that allows simulating the classroom environment;
• Tests and assessments—dynamic, inclusive, and online;
• Preparation of learning content/tutorials to be carried out outside of classes (OT);
• Online repository (drive) with complementary information (books, articles, videos, or tutorials);
• FAQs with the most common issues;
• Recording of theoretical classes;
• Project-based learning with the application of a real case;
• Room for doubts and online chat;
• Continuous assessment of the CUnit and direct contact with the class delegates;
• Exploring soft skills (e.g., entrepreneurship, cross-learning, communication, adaptability, resilience, and leadership).

4.2. Challenges

The main challenge of teaching is motivating the students in the classes and creating an engaging environment. Thus, it was essential to study the context and anticipate possible issues in order to address the challenges and guarantee TeachTeach’s suitable transition to an online environment. In this context,

Table 1. Challenges of an online CUNit.

Challenges Issue	Mitigation Plan
Answers and support delay	Keep the Drive and a FAQ updated Use a direct communication system—ioChat
Individual assessment copy	Time-limited exercises—Accomplish quick and short activities under pressure (without time to think—10 to 60 s questions) Questions had personal attributes, which made each exercise unique (e.g., first and middle name as the function’s name) Random Locked exercises opened according to the student knowledge
Demotivation of the students	Implement all the gamification systems designed Create an extra CUnit survey Re-think assessment and gamification methods Talk with delegates to understand the reasons
Bad grades	Allow students to show the reason and allow them to use the rescue system Revise questions and answers
Technical Issues (e.g., cannot make attendances, and internet issues, login not available)	Create the students’ accounts at the platforms Prepare the environment and test all the platforms used Create a room for technical support.
Classes or evaluation misses due to external factors (e.g., health, family, devices, internet issues)	Prepare an individual assessment path with online challenges Create extra opportunities for assessment Record and store classes online

shows an overview of the main challenges identified at the beginning and the mitigation plan/tasks defined. For example, if students verify some delay by the professors, they must look for support in the Drive or FAQs and insist on communication through ioChat.

4.3. Approach

This approach was intended to be explored by any professor who wants to have a different teaching view and innovate their classes. The idea behind online practical classes (TP and LP) was to use a live (online) coding approach to design and implement a coding project during the lecture period [42]. In particular, professors adopted the concept and developed a new approach: team coding, where professors made the students program in groups using an online programming tool (code editor). They also could discuss their work/code with colleagues and professors using a collaborative environment.

Thus, to ensure the correct implementation of this paradigm, professors used several approaches/techniques, and a weekly plan (Figure 3) was designed. In Figure 3, it is possible to understand the schedule by week and observe the students’ assessment moments: individual—three Mini-Tests (MT1, MT2, and MT3) and group project: two Control Points (CP1 and CP2) with a final presentation (FA) of the work. As can be observed, this subject had a strong focus on the project component with three assessment phases.

4.3.1. Classes

Both types of classes took place online, and the teaching team used three platforms to support the teaching process and one to promote online learning and team coding. The following list explains how professors applied it in the case study:

• Zoom—At the beginning of each theoretical class, the professor started the recording and shared the ioEduc live system (containing the slides and other features) in Zoom. After a brief explanation of the week’s main topics, the professor created several breakout rooms, and the students were randomly distributed. Students were faced with some challenges to solve in a group according to the topics addressed in this phase. They could ask for support in Zoom or ioChat, and professors jumped between each room to help them. Each lesson had 20 rooms with an average of 5–7 students each. Before classes ended, all the students returned to the main room, and the professor performed a raffle among the attendees to determine which students had a bonus in the weekly quiz. Finally, the professor put the lesson video on the ioEduc drive so it was available for the students.
• ioChat—It was the primary communication tool and was selected because it is easy to customise and is based on the open-source platform rocket-chat. Before starting the CUnit, the professor created several conversation rooms according to classes, teams, and groups. Then all the students and professors were registered in ioEduc and had access to this platform. At the beginning of each lesson (TP and LP), the professor started a video call with all the students. This first video call was used to talk a little about the class goals. Then, all the students went to their particular room (team or group) and initiated another video call. Then, professors visited each room to explain possible doubts and provide support in project development. This tool allowed professors to share knowledge and have parallel classes in the same place at the same time. In the TP classes, students were grouped into teams according to their role (front-end, back-end, or full-stack). In the LP classes, the students were organised by group, following the allocation method in Figure 4. ioChat also had a set of creation rules (rooms name, username, photo, among other details), quickly identifying students, groups, projects, and professors. Stakeholders used this tool to promote and facilitate communication between them, inside and outside of the classes.
• ioEduc—This platform was chosen as the Learning Platform instead of Moodle or BlackBoard. The justification for the choice is easy; the author of this article is the same who developed the tool and included some gamification tasks (e.g., Cards and Rescue System). This choice puts ioEduc in the centre of the process, allowing a real connection between professors, students, and the subject. All the slides were available on this platform; students could, for example, ask questions, answer the quizzes, consult FAQs, or perform a peer assessment. In all classes, students registered their presence on ioEduc through the attendance system. ioEduc’s cloud drive contained recorded classes and many books, tutorials, and videos to support the OT work. Theoretical classes also used the ioEduc live system to share the slides and interact with students. Students could also consult their grades, assess group members, or rescue an assessment. The professors could realise quizzes or surveys, consult all the students’ performance details, or assign a card. ioEduc also connects many complementary tools useful for helping the classes (e.g., Cloud 9 AWS, GitHub, or Heroku).
• Cloud9—Although this unit used several programming tools, students explored joint programming online through AWS Cloud 9 (C9). All the students programmed their projects and exercises in groups using a shared repository (GitHub), i.e., they edited their files in real-time using (C9) similar to a google doc file, and the program stored all the code online. Students shared the repository and C9 account with all team members and professors. Therefore, this strategy allowed professors to see the evolution of the project and help when needed.

It is important to note that this team selected these tools as you would choose any other. The idea was to test the TechTeach paradigm with platforms already explored and adapted to the environment.

4.3.2. Project

The project developed by the students in this case study brought some innovation and entrepreneurship to the entire process. They were asked to plan, design, and develop a solution to a real case. The following points are a brief presentation of the project requirements (terms of reference):

1.

“Secure4All” is responsible for the operations coordination of public entities. They need a web solution that helps them manage each of the occurrences received and a web/mobile app that allows their operation in the field.

2.

The goal is to develop a web/mobile solution (PWA) able to support the management of public intervention units.

3.

The company defined a digital transformation strategy to some of its units, namely: ASAE (Food Inspection), Firefighters (Fire), GNR (Events), INEM (Accident), Maritime Police (Drowning), Municipal Police (Robberies), PJ (Cyber-Attacks), and PSP (Disturbances).

4.

Students must create teams of three groups and select the project area. The group is responsible for developing a set of features/components.

5.

Each team has total working autonomy and must define the development plan for their group. The group must develop the front-end (visual) and back-end (server) layer of the assigned component.

6.

Stakeholders should use the ioChat platform to meet, ask for help, and/or explain parts of the project.

Each team was composed of three or four groups (areas) following the distribution:

• Group I—Operations Centre;
• Group II—Operations manager/Administrators;
• Group III—Field Operations Officer;
• Group IV—Audit (external entity).

Figure 4 outlines the student distributions by each class type and how professors organised them by teams and groups.

As can be observed in Figure 4, the groups and teams had distribution rules at classes to promote the cross-learning. The rules were:

1.

The teams were divided by TP classes shifts;

2.

There were two teams by TP class: each one with three or four groups and a specific theme;

3.

Each group had six elements (±1 depending on the number of shift students) and had to include at least two front-end students, two back-end students, and one full-stack student.

4.

Each student (group element) was responsible for developing a set of features/interactions.

5.

The project’s features were split between front-office (without login) and back-office (with login).

Each team had to elaborate the terms of reference (specifications) to shape the work and ensure the correct execution of the project. This document shows the team’s innovative and entrepreneurial vision to solve the problem and define: project requirements, roles and task distribution by group and elements, and a projected project cost (i.e., the grade they want to achieve if they meet all the requirements). It represented the commitment between the team (students) and the client (teaching team).

As stated before, this project aimed to train technical and soft skills and promote a professional learning environment. All the work followed a set of working good practices and included a couple of challenges such as

1.

Leadership (manage teams, groups and areas);

2.

Cross-work (listen from one side—“Team”—and transmit to another—“Group”);

3.

Arguing (discuss and sustain the decisions);

4.

Healthy competition (make a difference and look for extra features to improve the quality of project);

5.

Efficiency and simplicity (create a minimalist solution that is able to solve a real problem).

4.3.3. Assessment

This case study promoted an across and continuous evaluation method to evaluate professors, students, and CUnit. The students’ assessment was divided into three types:

• A quiz with a bonus (weekly) to reward engagement, motivation, work, and participation;
• A cards system to evaluate skills in the project;
• Three synchronised Mini-Tests to evaluate different types of knowledge (front-end, back-end, and full-stack).

Students evaluated the CUnit and professors at two different times: the middle and end of the semester.

Students could also assess and contribute to the implementation of this approach by answering some questions following this plan:

• Q1—At the beginning of the class: first week;
• Q2—At the middle of the semester: week 7;
• Q3—At the end of the final week.

Thus, professors invited the students to evaluate the CUnit and participate in the definition of this CUnit in three distinct phases (Q1–Q3). Figure 5 shows each quiz’s relevance, schedule, goals, and actions. Students participated in interactive surveys (Kahoot!) and answered questions about professors’ performances, type of classes, motivation, expectations, etc. Both surveys had a participation rate greater than 80%.

These surveys assessed several factors from professors (knowledge and teaching skills) and CUnit (plan and classes). The feedback received in Q1 is used to define the strategy. The feedback of Q2 is used to adjust the process in the middle of the semester, correct some bad decisions, and avoid a lack of motivation. Q3 feedback is always used to improve this CUnit in the next edition (year).

4.3.4. Gamification

Regarding gamification, the teaching team attributed positive and negative points to the students during the classes using several mechanisms. Professors designed a narrative for each method and applied gamification in the following contexts/exercises:

• MT1 and MT3—Students grades were obtained through a metric based on their knowledge, questions weight, and quickness, i.e., answering time according to the average time of respondents (3rd quartile).
• MT2—Exercise one included code gaps, where students must complete them with their individual data (e.g., birthday, name). Then, they had to use their knowledge to unlock the second part of the exercise. In the end, students could achieve one of three grades (with a percentage):

  -

  −1 Student fails—does not know the basics (negative grade)—0%;

  -

  0 Student can pass—knows the basics—100%;

  -

  1 Student approved—knows the basics and overcame all the challenges (e.g., unlock the second part of the exercise)—110%.

  The combined grade of all MTs was calculated by the formula FMT = avg(MT1, MT3) × MT2.
• Quiz with bonus—During this assessment method (weekly), students had the possibility to duplicate their results (extra). This approach only awarded students in attendance. At T classes, professors made a selection of 10–15 students. The rules were simple: only the attendants of that T class were eligible, and the system could select no one more than two times without all students having been assigned at least one bonus (B). After the last quiz (final week), professors collected all the individual results (I) and applied the following rules:

  -

  20—Student with the most points without a bonus (K);

  -

  20—Students with points plus bonus (I + B) higher than K;

  -

  10 to 20—Students with points plus bonus (I + B) less than K. Grade achieved by the formula: ((I + B)/K).

  This exercise aimed to reward students who attended classes and answered more questions.
• Rescue System—Students could only use this method in MT2. The rescue system was available to failed students (grade = −1) who thought they deserved more. The professor analysed the student process and, in the case of acceptance, they allowed them to continue with a penalty of fifteen per cent (15%) in the final MT grade (i.e., MT2 equal to 85%).
• Cards System—The card system was available, and it was used during the project by team leaders and professors. They could attribute white (positive) or yellow (negative) cards to the students. When the limit number of positive or negative cards was achieved, professors applied the rules defined in the narrative.

At the beginning, professor presented all the rules (tasks, goals, results) to the students.

4.4. Numbers

The following list helps show the impact and endeavour necessary to have a CUnit with a complete online-learning context with TechTeach and using e-learning strategies:

• 13 classes shifts (1T, 4TP, 8LP);
• 1 Real project with 3 evaluation phases (CP1, CP2, FA);
• 8 teams (±20 students) and 27 groups (±6 students);
• 3 synchronous online Mini-Tests (2 Kahoot, 1 HackerRank);
• 2 CUnit and professors evaluations (middle and end);
• 38 questions in FAQ;
• 50 learning documents (slides and resumes)
• 76 files in drive (books, videos, Cheat Sheet, examples, video classes, OT exercises)
• The average number of responses in CUnit final evaluation was 115;
• 85% of students accept the CUnit gamification system;
• 80% was the minimum attendance of T Classes;
• The number of bonuses attributed was 144;
• 28 students received a card (8 yellow, 8 white, 6 blue, 1 orange, and 3 red) in a total of 39 interactions;
• The number of individual assessments received was 1881;
• 3 students activated the rescue system.

In the end, the achieved results using online platforms should be mentioned:

• 208 (two hundred and eight) hours of online classes with parallel sessions, with a total of 338 (three hundred and thirty-eight) hours;
• 80 (eighty) chat rooms;
• 4528 (four thousand five hundred and twenty-eight) online classes attendances;
• ∼ 220 k page views;
• ∼ 25 k access to the CUnit support platform (sessions);
• ∼ 16 k access to the chat (sessions);
• ∼ 15 k messages exchanged in the chat;
• ∼ 11 k file downloads;
• ∼ 106 uninterrupted days online (sum of all active sessions time).

In the first lesson of the CUnit, survey Q1 asked students their opinion on having classes online (Figure 6) and why they were there (Figure 7). As observed in both figures, when the class started, students showed that they were afraid of having online courses, regardless of whether they wanted to learn web programming. The following images (Figure 6 and Figure 7) and results can help to assess, in the end, if the environment confirmed the students’ initial intentions.

Between Q1 and Q3, Q2 was performed. This short, intermediate questionnaire was used to understand students’ opinions about classes and the professor’s performance. Almost one hundred students participated in Q2, which was accomplished in week 7. (The average number of respondents was 92.) Regarding Q2, some results can be highlighted:

1.

Of all the students, 98.86% were enjoying online classes;

2.

Of all the students, 76.53% affirmed they learned a lot;

3.

Only 2.17 % of students stated that they do not appreciate CUnit.

A last open question was performed about what should be changed in the second part of the semester. Most students (>75%) did not like having random groups at T classes, so the professor removed this "feature" of theoretical classes. Students also asked professors to make minor improvements regarding online class organisation by providing more examples and increasing project support. Professors achieved both requests.

Figure 8, Figure 9, Figure 10 and Figure 11 show some Q3 achieved results about the TechTeach approach in terms of gamification and online component. The figures confirm the results mentioned before and sustain the assumptions taken with this case study.

Finally, the primary tool, ioEduc (with ioChat), was assessed (Figure 11) about its value and whether it should continue to support the learning process. More than 85% of students considered it essential to the goal and helped the learning (yes or maybe).

Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16 show a visual analysis of the paradigm’s impact on the students’ online performance and their assessments. These figures summarise some of the achieved numbers obtained from ioEduc and ioChat (such as events, sessions, page views, access, devices, among others).

4.5. Academic Quality CUnit Report

The university also has an official mechanism of receiving feedback and preparing the quality report of each CUnit. At the end of the CUnits, all students are invited to evaluate the CUnit and professors. This questionnaire is available to all the courses and students of the University of Minho.

Figure 17 shows the results collected from 135 answers. This survey analysed four dimensions: pedagogical dimension (TechTeach Paradigm), working conditions (online classes and tools), teacher performance, and student motivation. All the aspects studied achieved more than 90% of positive answers. These results sustain internal evaluations, clearly show the strategy’s success, and give the confidence to continue this work in the future.

4.6. Word Cloud

It is an excellent method to show the opinions of students. After students’ submit their opinions (free text), is it possible to observe that the most positive words (Figure 18) are dynamic classes and professors. On the negative side, they identified (Figure 19) “nothing” or CUnit issues, such as matter and materials.

5. Discussion

The case study being analysed was the first approach to test TeachTeach in an online environment using an e-learning strategy. Students demonstrated their concern about having online classes (greater than 50%). However, professors did not observe most of the students’ preoccupations during the lessons. Globally, they enjoyed the learning environment, even though they had some online issues. The achieved results are impressive and allow professors to have an in-depth analysis of interactions and engagement verified between students and professors. In the end, Web Programming classes reached two hundred thousand (232,433) page views, fifteen thousand messages exchanged (15,224), sessions with an average of fifteen minutes (15 m 25 s), and eleven thousand (11,173) downloads.

This case study also measured the students’ online activity (presence). A curious fact is that only two periods did not have anyone online—Monday from 4 a.m. to 5 a.m. and Tuesday from 5 a.m. to 6 a.m. In the other periods, at least one user was online, with class hours on Wednesday and Friday having the most.

Students did at least fifteen thousand accesses in both platforms during the semester. If we sum all the online sessions’ time (professors and students), we can achieve an incredible number of one hundred and six (106) online days without interruption, i.e., more than two thousand and five hundred hours).

In order to assess the relevance and application of gamification, it was possible to measure the implementation’s impact and students’ opinions at the end. During the lessons, more than 90% of students received a bonus (144), twenty-eight (28) students received a card from a total of 39 warnings (positive or negative), and three students asked for redemption (both were then approved). In the end, 85% of the students liked the gamification approach.

Regarding the Mini-Tests, the strategy implemented had simple exercises under pressure (easy questions without much time to think)—MT1 and MT3—and exercises containing personal attributes that made each activity unique—MT2. In both types of exams, professors used the last two questions of the tests to ask the students if the test was easy and if they liked it. On MT1 and MT3, more than sixty per cent of students (62.39%) approved the model; however, more than sixty per cent considered the test hard or too hard (63.52%). In terms of MT2, more than fifty per cent (50%) of students disliked MT2 and thought it too hard. Both results showed the success of the strategy applied. MT1 and MT3 were easy (more than 35% of students had the same opinion) and allowed them to understand the basics. After analysing the results, professors understood why students refused the MT2 approach. Some students did not understand and could not do it because the Mini-Test was challenging to copy, think about, or comprehend if they did not know the subjects. Another curious fact is that after students received the results of MT2, they told professors that they had not understood the exercise goal at the beginning. They were surprised by the grades (positive) and affirmed that the exercise was interesting.

Analysing students’ opinions (text), they consider the Teaching Team and TechTeach as one whole, professors’ availability, and their dynamism as the most positive aspects. On the other side (negative), many students had “nothing” to appoint; however, they highlight the quantity of matter and materials taught in the theoretical classes as the less positive aspects. It is a recurrent issue because this CUnit is the only one to address all the fundamental themes, and the time is not enough to explain all with time, so the subjects must be compressed. This problem is being solved, and this CUnit will be split in two: the front-end and back-end (optional).

The students’ opinions were very positive and motivated a global and compelling analysis. More than ninety-five per cent of students (96.53%) considered this CUnit to be the average or higher than the average of other online subjects, and ninety per cent of students considered ioEduc valuable. Most of the students (60%) liked online lessons, but 40% answered negatively. It is an excellent point to notice that this result was below what was initially indicated by the students (Figure 6). The negative answer was justified by the working conditions (e.g., one computer at home to many users/students or lousy internet). The quality report of the University of Minho attested to the answers provided by the students through the university’s own assessment mechanisms. All the topics, including pedagogical dimension, working conditions, professors performance, and student motivation, achieved more than ninety per cent of positive answers. As mentioned before, TechTeach has the primary goal of motivating students. Thus, and as can be observed in Figure 17, almost 80% of students had good or excellent motivation, and less than 5% did not feel motivated to participate in classes. These results prove the effectiveness of this approach.

Although this article presents a rigorous case study, anyone can explore this approach. For example, the TechTeach paradigm is tool-less, so the professor can keep interaction and engagement by creating their rules and replacing the tools used for other well-known platforms:

• ioChat –> (Slack or Discord) + Zoom or Team;
• ioEduc –> (Moodle or Blackboard) + Google Forms (Gamification);
• Kahoot –> Voxvote or Mentimeter.

6. Conclusions

This article detailed a case study performed to assess the suitability of TechTeach in an online context. All the assumptions made in this section are regarding the environment explored and presented in Section 4.1. Other contexts could have different results and performances. The present case study allowed answering the two research questions:

1.

“Is TeachTeach ready or adapted to classes 100% online?”

• The answer is Yes.

  -

  After finishing the teaching process, more than 80% of stakeholders (students and professors) considered this approach a success. The achieved values corroborate good results. The classes attained more than 220 k page views, 40 k sessions, 11 k downloads of files, and 15 k messages. The sum of all online sessions represented more than 106 days online.

  -

  In terms of gamification, 85% of students approved the system. In this approach, almost 150 students received a bonus, nearly 2 k of individual assessments were realised, 28 students received a card, and 3 students used the rescue system. Moreover, the class’ attraction level was high, with an average attendance of T classes over 80%.

2.

“What is the effort needed? Is it worth it?”

• The answer is Yes.

  -

  It is essential to note that it was the first time using TechTeach in this context (online), so the effort was considerable. For example, the coordinator’s effort was ∼ 250% multiplied by their teaching hours (8) for an average of 20 h a week.

  -

  The point to highlight is that the professor can reuse most of the work the next time. This possibility will reduce the implementation time and can increase the results.

  -

  It is essential to have some “free” time to apply this methodology and keep students engaged when a professor has the first experience with TechTeach.

  -

  It is necessary to put in extra effort, have a good teaching team, and help the students, as the word cloud shows.

In a global analysis of RQ, this case study proved that the combination of factors such as time and professor’s availability had a strong implication in the strategy’s success.

The word cloud showed a global image of how TechTeach worked in the online environment and the students’ opinions. It positively highlights the importance of professors, and, on the negative side, it has a high number of matters addressed in theoretical classes.

This work displayed that TechTeach is adequate for adapting to the constantly changing teaching system and had a favourable acceptance from students. Unfortunately, this study also showed a relevant gap. In some cases, and although the results are motivating, students fear and distrust online classes. In the beginning, more than 65% were preoccupied. At the same time, 73% wanted to learn web programming. In the end, 85% of students liked the pedagogical dimension (it means 12% more than the beginning). Still, their limitations became evident. For example, 40% of students did not want to have online lessons because they felt they did not have the proper conditions. Thus, if we are going to have online classes, we must create the right conditions; otherwise, we cannot expect student participation or good grades.

Finally, by analysing the students’ opinions, more than ninety per cent (96.53%) of the students considered this CUnit equal or better than the other online subjects. Their grades were positive, and 98.15% of the students passed (i.e., professors failed of only three of the students who attended more than 25% of classes). Thus, balancing the effort and achieved results, professors concluded that the achievements were much more rewarding and offset the endeavour. As such, the main contributions of this article are:

• A detailed explanation of how gamification can be used and applied in an online environment;
• A complete guide of the strategies/tools needed to have a fully online class;
• A real case study with all the details that allow measuring students’ receptivity to the TechTeach approach.

To summarise, analysing TeachTeach following the DLPCA strategy makes it possible to observe an almost perfect match. TechTeach has a well-defined plan where the rules of professors and students and the type of tools are clearly defined. The theory of education is based on the narrative created by the course. Professors identified some socio-technical constraints (e.g., internet, computers, health and time). The case study promoted online async and sync activities for their students (e.g., Online MT, Tutorials, Challenges). It allowed professors to assess many learning outcomes (e.g., participation rate, online times, individual and collective results, among others). Further, the case study also accomplished Merrill’s principles—professors engaged the students to learn, apply, and integrate the obtained knowledge (web programming) to a real case. Students demonstrated their achievements by developing a group project adapted to their reality/world. Therefore, this case study showed TechTeach’s adaptability to online environments considering the DLPCA strategy and Merrill’s principles, which proved that this approach could increase student engagement and promote innovation and training skills at a distance (online). This work will jump to the next level now. Thus, professors will analyse collected data and apply a Web Mining process to correlate the students’ performance with their activity. Then, the paradigm will be tuned, and some new features added.