The study findings that include the course features and the counter student behavior were laid out to analyze the effectiveness of the course design and its implementation, thereby addressing all four research questions. The following data collection tools were used in drawing out the findings of the study.
The analysis of the course features was performed by examining the student behavior and feedback towards accomplishing each course activity as well the asynchronous assignments. The following sections detail the findings based on different factors that contributed to the course’s successful design and implementation.
4.1. Assessment of Course Content and Design
The topic, ‘force and motion’, was carefully considered for the course, as per the QSC requirements. As the course design implemented different hands-on activities, it was quintessential for the developers to overlook the safety guidelines, ensuring that the materials and activity procedures were safe to be implemented at home. Moreover, the materials’ sourcing was restricted, as the course was provided to the students when the entire country was under lockdown due to the pandemic. The situational crisis prevented the students from leaving their houses or accessing school labs or material stores, thereby forcing the developers to create content that was fully functional from their home premises.
The content was designed in such a way to make sure to retain the students interested during the sessions in a cordial manner, by providing a wide variety of activities that offered dynamic and interactive content to engage and learn. The course successfully applied various digital resources (refer
Table 5 and
Figure A1a–d from
Appendix A) like different learning videos, real-time simulations, hands-on activities, and multiple games to understand the concepts of force and motion. As games are an effective strategy to engage students on online platforms, numerous interactive games were introduced to grab their attention. All the activities except for the assignments were executed live in the facilitators’ presence during the virtual sessions, thereby visualizing their commitment throughout the session. The course content was successful to a significant extent in integrating STEM subjects, thereby correlating forces to real-life applications. In the due process, it was also observed that students exhibited enhanced teamwork through online communication on social platforms like WhatsApp and MS teams, thereby marching forward to social learning.
Some of the online games (for example,
https://www.gameflare.com/online-game/seesaw-ramp-car-balance/, (accessed on 23 February 2021)) included are concise and function as a single level task of asking the participant to predict the title of the lesson or activity. Such short games are highly effective in creating short-term excitement in students. The virtual simulation (for example,
https://phet.colorado.edu/sims/html/forces-and-motion-basics/latest/forces-and-motion basics_en.html, (accessed on 23 February 2021)) was integrated into a part of four-day lessons to adhere the students to the program through invoking student interest and engagement. The developers also ensured that the simulation resources are not repeated in the consecutive sessions, to avoid students getting bored quickly. The respective simulations were chosen, as they easily assisted the students to attempt multiple times, varying the inputs and observing the corresponding outputs. The 3D simulations also gave them a holistic view of the concept, offering them situations they can easily imprint on, thereby improving students’ engagement. Zacharia and other researchers [
32,
33,
34,
35] found that when the teachers interacted with the computer-based simulations, the explanations they constructed were more fluent, more detailed, scientifically more accurate, and involved more formal reasoning. Similarly, games involving puzzles and crosswords were complimentary online engaging resources that were utilized to address the students’ reasoning skills and problem-solving competences.
The facilitators had ensured that the implemented videos were finely audible with attractive sound modulation and understandable for students. The videos were created either by the facilitators or from other existing educational resources. The course also made room for animation videos [
36] to convey the lesson objectives, as the children exhibited more interest in animation for educational purposes. The videos were further implemented as an exceptional input to assessing the students, as they were requested to observe and analyze a situational problem from the provided video as in problem-solving activity. The respective activity helped the facilitators interpret their observational skills and their application of critical thinking to the corresponding situation. Importantly, the videos were short-timed so that the participants could not divert their attention. The course developers ensured that the inclusion of the digital resources was balanced and rightly timed according to the background study conducted on the behavior of students on online learning.
During the sessions, the students zealously engaged in the course, actively participating in all the online and offline activities. Amid the online session activities, extroverted students with excellent leadership qualities and social learning skills were often observed to provide oral answers quickly. Meanwhile, most of the participants were introverts, who acknowledged considerable activity through meeting chats to address the facilitators’ questions. However, the introverted students did seem to perform relatively better along with the extrovert students on hands-on activities, as they were accomplished individually from their homes. They recorded videos using mobile phone cameras and thrived on WhatsApp group conversations with their team members, wherein most of the course driven group discussions were facilitated. Few students had difficulty approaching group-oriented activities and discussions and exhibited poor socializing skills either due to their behavior or due to lack of associating with the digital transformation. As the participants were from middle school, most of them also had to access their parents’ guidance for gadgets or course-related follow-up activities.
The hands-on scientific experiments were crucially important in the course content for attracting the students to participate in the courses without any compulsion. The students were requested to perform activities as a part of live sessions and assignments, attributing to reflections of the corresponding live session (refer
Figure 4a,b and
Figure A2a–c from
Appendix A). They consistently performed these activities and recorded their act as a part of creating evidence and documenting the result of their work as videos (student video links are attached in the
Appendix B). The students’ documentation was performed with their parents’ assistance in holding the camera or measuring the distance traveled by their toy car and/or arranging the setup. The videos were also promoted to be published on social platforms such as YouTube [
37], thereby using their personal inclination towards social media platforms. Resultantly, the students were self-motivated and encouraged to improvise their work with each attempt, thereby learning and self-assessing themselves in the meantime. Moreover, the course developers also ensured that the experiments were conducted with readily available materials like cloth hangers, plastic cups, rope, paper, plastic bottles, and bottle caps, etc., guaranteeing material access to every student at home. Though the materials were commonly found in homes, the facilitators also tested the student’s capability to replace these materials in case of a deficit. However, to their surprise, the students exhibited critical thinking and problem-solving competencies by successfully accomplishing their target with limited materials, guidance, and workspace.
The design project activity, included in the course, was the course’s key highlight in deriving student creativity, problem-solving, and reasoning potential, as the students successfully transferred the knowledge for creating the product. The students built a toy car (refer
Figure 4c–e) by satisfying a set of criteria mentioned in the methodology section on page 7.
The facilitators were keen to include opportunities for the students to make predictions and interpretations. The exercises on predictions and interpretations are critical factors in activating and refining the student’s prior knowledge and enhancing their engagement. Each activity satisfied the lesson plan’s core objective, as most of the students responded positively to the UG mentors, who kept tabs on their assignments and technical glitches, and guided them during group discussions. The facilitators also notified that all the course certificates were provided to those who completed their assignments without any fail.
The students exhibited enhanced participation for hands-on activities despite the restrictions of accessing the materials on their own in contrast to having been accustomed to settings with readily available materials at their school classrooms or labs. They even showcased their problem solving and critical thinking dispositions while accessing the required materials by replacing them with better alternatives in case of deficit. During the design project assignment, which was to be accomplished as teamwork, each member designed and developed a product independently, as they did not have the opportunity to gather around due to social distancing norms. Once they succeeded, they tested their products in front of their teammates via a web conference call. They displayed the product to each other and compared the results to choose the best design according to the criteria, in order to compete with other groups. Some groups even resorted to improving their final product even after meeting the requisite conditions. For example, in the case of one girls’ group, student 1 built a balloon car (as in
Figure 4c) that was able to satisfy all the preset criteria. However, student 2 made a toy car, which also exhibited similar performance concerning speed and distance (as in
Figure 4d). However, student 3 changed the design of the vehicle (car) to function itself as a boat as well, powered by a battery source, which was tested in a tub filled with water (as in
Figure 4e) and chosen to be best in performance and design (videos of testing is available on request).
4.3. Upgradation of Course Content
The effectiveness of the course was assessed by the results from the feedback mechanism implemented in the daily session. During the five days of the course, the students were required to provide feedback for the following three open-ended questions.
What did you like the most from the session?
What did you like the least from the session?
What would you like to change in the session?
The analysis of the three open ended feedback questions was carried out daily by the UG mentors, who notified the facilitators to improve the teaching approach, learner-centered instruction, hence implementing the changes according to the student choice. This method led to providing the students with a voice and equally participating in the pedagogical change that improves the teaching approach with time.
The course was improved in the instructional approach owing to the recommendations of the students. Some of the students requested more activities, whereas others asked for time to complete the sessions’ online challenges. Some students even asked to include more days of activities to the course schedule; meanwhile, others did not have any suggestions at all. One of the students did suggest that the mentors did not give fair chance to all. This feedback was rebounded by introducing an online interactive tool, “wheel of names”, which chooses students randomly from a list of entries, thereby solving their problem. Though the facilitators considered providing more time for the challenges, they could not accommodate more activities within the designed course due to the time allotted for the course. The recommendations put forward by some of the students, on giving equal chances to all participants to answer queries, for which they were awarded points on a daily score-chart, were also considered on the following days. The graphical representation of student feedback that portrays satisfaction with respect to each course feature is also displayed hereafter in
Figure 6. The feedback from the students was analyzed, and the frequently recurring course feature was chosen as a keyword. The keywords derived from their course include activities, challenges, teamwork, interaction, etc. While the students stated the teaching approach of the mentors, for example, “the mentors should give a fair chance”, “the mentors should mute participants”, or “the mentors should give more time to finish challenges”, the keyword represented is “mentor approach”. In case of stating “nothing” for their dislikes or recommendations, we represent the keyword as fully satisfied. The
Figure 6 presents the graphical representation of participant preferences for each of the mentioned keywords or course features.
As the course program was based in Arabic language, the data representation was based on feedback stated in the
Appendix C attached, and contains transcripts translated by the “Google translate” application (a sample of original WhatsApp chats is provided in
Appendix D).
All digital resources were included in the course by cautiously considering the preferences of the students and balancing time management. They demanded activities based on simulations and games in their daily feedback, thereby inciting the developers and facilitators to include more weightage for the same. Videos were provided either to deliver content knowledge or to introduce a game and hence were mostly short timed to play as 1-min videos, limiting their maximum daily playtime to 3 min. Since PowerPoint presentations were used to deliver the whole session course content in the form of videos, games, simulations, etc., they were accounted to be implemented for the maximum duration. The online engaging resources include interactive applications like whiteboard for illustration and the wheel of names that was applied to select students on a fair basis for each game, thereby avoiding conflicts in the midst of session. The graphical representation of digital tool distribution that displays the daily usage of each digital tool during the respective session is provided below in
Figure 7a–f.