Next Article in Journal
Perspectives on Socially Intelligent Conversational Agents
Previous Article in Journal
Design Considerations for Immersive Virtual Reality Applications for Older Adults: A Scoping Review
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
MTI
Volume 6
Issue 8
10.3390/mti6080061
mti-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Learning Management System Analytics on Arithmetic Fluency Performance: A Skill Development Case in K6 Education

by
Umar Bin Qushem
1,2,*,
Athanasios Christopoulos
1,* and
Mikko-Jussi Laakso
1,2
1
Centre for Learning Analytics, University of Turku, 20014 Turku, Finland
2
Department of Computing, Faculty of Technology, University of Turku, 20014 Turku, Finland
*
Authors to whom correspondence should be addressed.
Multimodal Technol. Interact. 2022, 6(8), 61; https://doi.org/10.3390/mti6080061
Submission received: 15 June 2022 / Revised: 8 July 2022 / Accepted: 19 July 2022 / Published: 22 July 2022
(This article belongs to the Special Issue Effective and Efficient Digital Learning)
Browse Figures
Versions Notes

Abstract

:
Achieving fluency in arithmetic operations is vital if students are to develop mathematical creativity and critical thinking abilities. Nevertheless, a substantial body of literature has demonstrated that students are struggling to develop such skills, due to the absence of appropriate instructional support or motivation. A proposed solution to tackle this problem is the rapid evolution and widespread integration of educational technology into the modern school system. To be precise, the Learning Management System (LMS) has been found to be particularly useful in the instructional process, especially where matters related to personalised and self-regulated learning are concerned. In the present work, we explored the aforementioned topics in the context of a longitudinal study in which 720 primary education students (4th–6th grade), from United Arab Emirates (UAE), utilised an LMS, at least once per week, for one school year (nine months). The findings revealed that the vast majority (97% of the 6th graders, 83% of the 4th graders, and 76% of the 5th graders) demonstrated a positive improvement in their arithmetic fluency development. Moreover, the Multiple Linear Regression analysis revealed that students need to practice deliberately for approximately 68 days (a minimum of 3 min a day) before seeing any substantial improvement in their performance. The study also made an additional contribution by demonstrating how design practice compliance with gamification and Learning Analytics in LMS may lead children to be fluent in simple arithmetic operations. For educators interested in LMS-based intervention, research implications and directions are presented.

1. Introduction

Developing a strong numeracy and fluency skillset is widely regarded as a significant component of human potential [1]. Contrarily, failure to develop computational fluency (i.e., the ability to perform arithmetic operations with accuracy, flexibility, and efficiency), from as early as the primary education levels, might affect individuals’ attitude toward mathematics for the rest of their education and, ultimately, lives [2]. These facts are true in all countries, including the United Arab Emirates (UAE). Previous research in the field of education has revealed that the development of mathematical understanding requires a combination of effortless treatment for low-level processes (such as arithmetic operations) and systematic practice for more complex tasks which require higher-order thinking and advanced cognitive abilities [3]. To assess these outcomes, researchers (e.g., [4,5]) have concluded that students show mathematical cohesiveness when they display the following: (a) flexibility in chosen computational techniques, (b) sufficient capacity to explain and support decisions, and (c) appropriateness of responses.
The traditional teaching-and-learning methods enable students to conceptualise basic mathematics skills but often lead to the creation of a repetitive routine, which hinders knowledge development or fluency advancement [6]. An approach that has been evolved over recent years to mitigate these issues and prevent such negative outcomes is called ‘deliberate practice’ [7]. The core philosophy of this approach positions individuals at the center of attention, while also allowing the identification of the cohort’s differences in skill acquisition and knowledge capacity. Accordingly, educators and instructional designers can provide learners with goal-oriented activities aligned to their individual needs, while students maintain complete (self-)control over their educational goals [6,7]. However, in order to increase the efficiency of this approach, guided practice, as well as immediate feedback and opportunities for reflection, are essential; a fact that also applies in the context of mathematics education [8].
The abovementioned conclusions, as well as the notion that the development of mathematics fluency requires deliberate practice, motivated the design and conduct of this longitudinal experiment. Attaining fluency in key mathematical procedures is prevalent to the development of students’ mathematical creativity [9,10]. Besides, an agreement across different studies argues that, in order for students to develop clear understanding of numbers and their potential application, it is essential to first develop solid cognitive schemas of basic arithmetic procedures [11,12].
Although some individuals are considered ‘gifted’, based on their demonstratable mathematical competencies which come with little to no effort, most students usually face difficulties in mathematics, primarily linked to lack of attention or absence of appropriate instructional techniques [13,14,15]. The longitudinal study that Rinne, Ye and Jordan [16] conducted aimed at identifying the trajectories behind students’ arithmetic fluency development (addition, subtraction, and multiplication). The main findings suggested that prior to developing arithmetic fluency pupils should have achieved a reasonable degree of reading fluency. Others [17] have concluded that cognitive assistance is required for the development of mathematical fluency. Barnes et al. [14] emphasise the importance of identifying mathematical difficulties and comorbid mathematics and reading difficulties that learners may face, as this would facilitate the provision of special, precision-based, treatment. The latter also contradicts the common practice approach, which averages low-performing students with large classroom audiences. Whatever the case is, researchers agree that the key to success for any kind of fluency development is practicing in various forms [18], as this enables learners to develop multiple levels of procedural and conceptual understanding [9].
The abovementioned findings framed the direction of the present study which comes with a twofold aim: on the one hand, to explore the effects of an adaptive Learning Management System (LMS) which has been utilised both inside and outside the classroom context in a course of mathematics and, on the other, to identify whether a curriculum-driven personalised learning path can facilitate students’ arithmetic fluency development. Under this notion, we matched with our sample selection the recommendations made by Rinne et al. [16] concerning the age maturity that reading fluency requires, whereas the implications made by Lee and Choi [19] informed the instructional design elements that a digital learning intervention should present when supported by a modern LMS. Subsequently, we empirically explored and examined the abovementioned topics in view of the following Research Questions (RQs) and under the aid of modern Learning Analytics (LA) practices:
RQ1. How does digital deliberate practice impact student’s arithmetic fluency development?
RQ2. How does students’ digital practice behavior influence arithmetic fluency development?

2. Related Work

2.1. Arithmetic Fluency

The study and the respective initiatives concerning the development of fluency have become core areas of interest in the field of educational research, with multiple efforts examining these concepts from different disciplines and viewpoints. For instance, Cui et al. [20], utilising the Online Experimental Psychological System (OEPS), a web-based application, have concluded that the Rapid Automated Naming (RAN) approach, a method that examines individuals’ capacity to name as many types of highly recognizable visual information (such as colors, items, numbers, words) as possible, is one of the main indicators to determine fluency in reading or arithmetic (basic tasks, such as subtraction and addition). It. While the study of the latter was carried out with only 160 third-year kindergarten students, it revealed highly significant correlations between the RAN approach and arithmetic fluency development, and, thus, challenged other predictors, such as cognitive processing skills. A possible limitation from this study is the fact that the researchers focused explicitly on kindergarten individuals, excluding pupils of later grades (e.g., K6, K9).
The study that Whitney et al. [21] performed assessed the impact of the ‘Great Leaps Mathematics’ curriculum which was designed as a fluency intervention tool to examine and improve students’ fundamental knowledge of mathematical facts (arithmetic addition). The study results revealed that participants’ verbal and written mathematical fluency skills were improved and so were their problem-solving abilities. However, their study was limited to ‘addition facts’ with numbers ranging from 0 to 9, whilst their experiment involved only three participants. The present study extended this attempt by including a larger population with more comprehensive exercises and arithmetic operations.
Considering the development of mathematical creativity, arithmetic operations like addition and subtraction are particularly easier than multiplication and division operations [16]. On these grounds, Russo and Hopkins [22] developed a novel approach to measure learners’ mental computational fluency with particular emphasis on the exercises that concern addition. However, both studies failed to relate students’ fluency development and advancement with their practicing behaviours, which was one of the main objectives of the present exploratory study. Indeed, attaining computational or arithmetic fluency is much less about how much one knows and more about how flexible, efficient, strategic, and accurate one is in solving arithmetic operations [22]. Although the findings of the latter pilot study were restricted by the relatively small sample (169 elementary school pupils with 3–4 grades), the authors were confident about the key findings and further recommended the examination of this model in additional contexts.
The longitudinal study conducted by [21] explored the difficulties that students face when constructing mathematics competencies. Based on future work recommendations, the authors urge future researchers to focus on the examination and analysis of the diversified factors which impact the development of arithmetic fluency, as it starts from an early-age and can greatly influence pupils’ future success; be it in the educational context or the professional world. This conclusion is also linked to the role that critical thinking skills and analytical abilities play on the future success of individuals [23]. It can, therefore, be assumed that, by supporting pupils’ exposure to contexts that support the development of such cognitive norms, this can also affect their future success.

2.2. Mathematics Interventions in Learning Management Systems

Research related to instructional practices that aim at improving mathematical fluency has verged for many years. Research covers areas from traditional instructional approaches to the utilisation of LMS, which alleviates interaction by introducing pedagogically sound and technically executable learning designs, to more advanced tutoring systems, which allow for the identification and analysis of patterns that contribute toward the deconstruction of learners’ difficulties.
The meta-analysis that Benavides-Varela et al. [24] conducted, regarding the impact of Technology Enhanced Learning (TEL) on mathematics education, found significant correlations in learners’ performance and even more so in cases where the students demonstrated mathematical disabilities. The authors [24] placed particular emphasis on the aspect of instructional design and the nature of the instructional strategies that were utilised, especially in digital-based interventions, as these can impact the obstruction that mathematical disabilities cause. However, they did not lay out the direct association or additional benefits of using computer games with regards to the digital practicing strategy. Other studies related to digital instruction [23] demonstrate the importance of incorporating problem-solving activities and interventions into students’ routine and further recommend the use of well-designed, interactive, and ‘intelligent’ digital learning tools to enable learners to achieve their fullest potential. The LMS-based intervention utilized in this study also attempted to shed light in this regard.
An experiment to explore this matter in greater depth is demonstrated by Lee and Choi [19] who utilised a tablet-based game intervention to improve students’ numeracy skills, without, however, introducing games dedicated to basic arithmetic operations.
Burte et al. [25] applied the ‘Make a Dice’ concept to assess learners’ performance in accordance with their spatial thinking. Kormos and Willby [26] iterated fluency development based on student motivation, performance, and self-efficacy, whereas Begeny et al. [27] integrated different motivation strategies in fluency interventions which were later evaluated primarily with students who had shown low or little interest in problem-solving. According to [27], such interventions should include a set of strategic motivation elements, such as interactivity, personalised feedback, automated assessment (correctives), and adaptivity. In the present study, the provided digital learning and practicing system was designed with these features in mind, as a means of supporting students’ participation to the greatest possible extent.

3. Research Context

In the recent educational transformation that the government of the UAE has undertaken (strategic plan for 2017–2021) special emphasis was given to the elimination of dropout rates, as this would lead to the graduation of more highly qualified professionals who could then support the development of a knowledge-based economy [28].
According to the WENR reviews [29], the strategic advancement plan that the UAE has employed is already demonstrating positive results with graduation rates reaching up to 96.7% whereas, for 2021, the driving indicators were related to matters that concern Science, Technology, and Innovation. To this end, the Ministry of Education has already promoted the integration of STEM (Science, Technology, Engineering, Mathematics) education subjects across all the country’s schools and further recommended the use of the English language for the delivery of the course material [30]. Even though the curriculum changes were adopted by all the country’s schools, a portion of private schools maintained the use of the local language (Arabic) for the delivery of the course material across all subjects [31].
Regarding mathematics education, a shift toward the integration of educational technology is identified, wherein both the national curriculum and the respective activities across all the grades and topics, are digitalized in accordance with completive competitions, such as SAT or Olympiad [32]. According to Woolsey et al. [33], primary school teachers are required to hold a four-year academic degree in Education which also includes courses related to mathematics (e.g., number theory, algebra, probability and statistics, geometry). This wider knowledge base enables prospective instructors to facilitate active learning and fosters learner engagement [33]. However, pre-service teachers are also faced with significant challenges, as they are required to integrate innovative pedagogical approaches, integrated via digital learning tools, into their practices without, however, having adequate preparation or training [34]. From the students’ perspective, the main issues that have been identified concern the difficulties that learners have in understanding the ‘language’ of mathematics which, consequently, prevents them from grasping the respective concepts [30]. However, as Chi Hyun et al. [35] recommended, digital learning environments enable learners to work flexibly and independently which, in turn, promotes knowledge advancement and mastering of subjects.
Under the consideration of the presented barriers and recommendations, the UAE Ministry of Education invited educational developers and technologists to support the proposed transformation by providing digital instructional tools and expertise in different nation-wide case studies. Following consideration of this invitation, researchers from the University of Turku (Centre for Learning Analytics) aided the development plan of UAE by facilitating the didactic of mathematics. Precisely, a digital tutoring system dedicated to mathematics education was utilised as the medium to support learners’ fluency development, whereas the built-in LA features allowed the key stakeholders (researchers, educational technologists, educators) to explore instructional design strategies and techniques that were more appealing and influential to their learners. The integrated LMS offers a wide diversity of educational games, with customised difficulty levels, automated feedback, and personalised learning paths. According to [9] the presence of such features increases the incentives for engagement and the interest in pursuit of new problem-solving challenges. Similarly, Walkington and Bernacki [36] investigated two major personalisation factors of students learning algebra problems in the context of Precision Education (PE), both of which may affect outcomes, and which are: (a) learner characteristics and (b) personalised activities [37]. The present study is part of a larger project related to the creation of personalised digital learning paths for mathematics education [38]. In this work, we explore and elaborate on the elements and conditions which influence learners’ arithmetic fluency development and advancement.

4. Research Methodology

As mentioned above, the present work investigated the elements that influence arithmetic fluency development. To achieve this goal, the research team, in close collaboration with local teachers, developed a personalized learning path tailored to the needs of the integrated context. The intervention was carried out with primary school pupils (4th–6th grade) who utilised the integrated LMS for a total duration of nine months. During this time, their academic performance and learning behaviour were continuously monitored using the available LA tools.

4.1. Intervention Desgin

For the needs of this study, a curriculum-driven, school-grade specific, personalised learning path for arithmetic fluency development was prepared. The exercises were developed in close collaboration with the teachers as recommended by Hoyles [39], whereas the integrated learning path constituted a comprehensive model for integrating TEL into schoolwork, which was further supported by modern LA practices.
Table 1 provides an overview of all the topics that were covered during this experiment. In the context of the study, teachers and students were requested to utilise the platform in one of their weekly lessons (in-class), including, also, respective homework. Other than this condition, no other restrictions were applied. In other words, teachers and students were free to utilise the platform for additional exercising/practicing at their own discretion. It should be noted that all the participating schools were equipped with the necessary infrastructure; thus, ruling out technical barriers or limitations that other countries face (e.g., [40]).
Prior to the conduct of the intervention, participating teachers and students were given clear information about the nature of the study and the reasons governing the initiative. Subsequently, the guardians of the students were requested to countersign a consent form which detailed all the information about the study, including the protection of students’ personal information in accordance with the General Data Protection Regulation (GDPR) guidelines. Upon completion of the experiment, the tests were marked by the researchers and linked to the students’ digital profiles.
Unlike the traditional, standardised assessments, which include exercises of few variations (i.e., addition and subtraction or multiplication and division, or two among them) (c.f., [18,41]), in this study we focused on the examination and evaluation of pupils’ mathematic fact fluency on all basic arithmetic operations. Precisely, a total of 160 calculations were presented to the participating students in both the pre- and the post-intervention assessments (performed using pen and paper). The total duration of each test had a fixed limit of 180 s, and each correct answer was counted as one point without negative marking. Subsequently, the total score (i.e., correct answers) was used as the dependent variable to classify and compare pupils’ fluency development.

4.2. Participants and Setting

Prior to the conduct of the intervention, the research team invited several public and private schools across the UAE to participate in the empirical study. Interestingly, only four public schools volunteered to participate (primary education, 4th–6th grade). Each of the participating schools had been employing different educational strategies and approaches but all of them had adopted the national curriculum. The distribution of the cohorts for each school was as follows: School 1: 4th and 5th grades, School 2: 4th grade, School 3: 5th and 6th grades, and School 4: 4th grade. In total, 776 pupils participated in the experiment, but only 720 of them met the inclusion criteria (e.g., participation in the pre-/post-intervention knowledge assessments, consistency with online practicing) (Table 2). Considering the strict exclusion criteria that were applied, we can confidently argue that any differences observed on the performance variations, after the conduct of the intervention, were dependent to the digital instruction system and linked to the practices followed.

4.3. The Integrated Learning Management System

The collaborative digital learning tool ‘ViLLE’ was utilised in the context of this study [42]. The initial development of the platform started in 2005, as a programming visualisation tool for Higher Education Computer Science students, and a couple of years later was employed and rigorously evaluated in other educational levels (i.e., primary and secondary education) and subjects (e.g., mathematics, languages) [12]. For the time being, the platform is utilised by more than half of Finnish schools (58%) with approximately 300,000 students and more than 14,000 registered teachers who have created collaboratively more than 4000 courses and 45,000 exercises via the ‘From Teachers to Teachers’ initiative. Thanks to these teachers and the university development team, the platform also presents a set of customised learning paths, covering topics related to mathematics, programming, and languages. Below we present some of the key technical and instructional design features of the platform:
  • The main operations (i.e., automated assessment, computerized exam administration, peer review function, construction or editing of new exercises) are performed on dedicated servers, thereby making the platform accessible via any modern web-browser without the need to install specialized software or browser-based plugins.
  • The system has been designed with the following principles in mind: (a) instructor–student interaction, (b) student–student interaction, (c) automatic assessment, and (d) immediate feedback.
  • An extensive set of premade exercise types and learning materials that can be deployed with little to no effort are readily available, thus making the tool as versatile as possible. To this end, a large variety of assignment delivery methods has also been included to facilitate personalised learning paths (i.e., acquisition of prerequisite knowledge) or the contact of exams.
  • To further support teachers’ missions, a built-in tool that facilitates the creation of new exercises, or the customisation of existing exercises, is also in place. Content created by individual teachers (including diversified difficulty levels, gamification elements, marking scales and so on) can be shared (copied) and edited by other community members.
Although the platform is operating-system and device-agnostic, teachers and students are encouraged to use it mainly with computers, due to the advantages that large screen monitors bring. Teachers are free to create their own learning paths, based on the available premade materials, in accordance with their students’ needs or create their own exercises using the built-in exercise editors.
As of 2021, engaged learners complete more than 20,000,000 gamified tasks (Figure 1) per month which are accompanied by immediate feedback and reflection hints. In addition, more than 95% of these submissions are graded automatically, thereby allowing teachers to spend more time in supporting students. Behind the platform’s user interface different Machine Learning and Educational Data Mining algorithms are exploring and interpreting diverse sets of data deriving from students’ real-time practicing. The outcomes of these analyses are utilised to provide educators and instructional designers detailed insights into the learners’ progress, difficulties, and misconceptions.
Moreover, the freedom offered to teachers to create and share their own exercises, tailored to their teaching approach, differentiates the platform from other available solutions that are more restrictive when it comes to content customisation and personalization. To this end, the inclusion of the LA dashboard further enables teachers to identify students who are not making adequate progress and are, thus, in need of additional support.

4.4. Data Analysis

The single group quasi-experiment design approach [43,44] was utilised for the exploration and evaluation of the RQs put forward. To identify trends and differences across the participating cohorts’ digital practicing and assessment results, the Exploratory Data Analysis approach [45] was utilised. In greater detail, the gathered data were initially investigated using descriptive statistics, followed by paired t-tests to investigate the statistical significance of the observed differences. Consequently, the log-data from the LMS was explored in a similar fashion to that described in the work conducted by Lee and Choi [19], while exploring pupils’ development of numeracy skills. Upon completion of this process, pupils’ practicing behaviour was analysed in accordance with the guidelines and techniques detailed by Haelermans and Ghysels [1]. In pursuit of identifying the impact associated with student practice and reflection, the exercise types were critically examined and searched for correlations, both on the individual and on the cohort levels. Lastly, a Multiple Regression Analysis was carried out to identify the relation between student post-assessment performance improvement and deliberate practice in the LMS.

5. Results

5.1. Practicing Impact on Students’ Performance

The personalised learning path included various exercises (e.g., algebra, geometry, calculations) aligned to the school level of each cohort and the national curriculum guidelines. Teachers were given the autonomy to make any kind of modifications to the provided courses (e.g., inclusion/exclusion of specific topics, introduction of new content, alterations to the difficulty level) and students the freedom to undertake exercises beyond the predetermined ones in accordance with their needs or discretion. During the online practicing time in the LMS, students’ digital traces were recorded, classified, and analysed for informational (teachers’ perspective) and research (instructional designers’ perspective) purposes. In the present study, we drew data from all the fluency-related exercises and associated them with the students’ knowledge assessment tests. Accordingly, we explored and discussed the most and the least favorable instructional design approaches that impacted learners’ fluency performance.
A total of fifteen (15) fluency development exercise types were available during the conduct of this experiment (Table 3). Table 3 enlists all the provided exercises and connects them with the respective knowledge development areas. In addition, it displays information related to the frequency at which each exercise was undertaken as well as the scores that the students achieved in each one of them (for comparison purposes). However, prior to diving into the specifics, a breakdown of the key-features of the exercise types is provided:
Mathematics Quiz: knowledge development related to arithmetic operations (addition, subtraction including decimal and negative numbers (linked to the ‘ARI’ lessons).
Mathematics Calculations (Forms): knowledge development related to problem solving such as identification of missing number or objects required to form the expected outcome (linked to the ‘NUM’ lessons).
Mathematics Calculations (Rows): knowledge development related to columnar addition and subtraction involving large numbers (linked to the ‘ADD/SUB’ lessons).
Calculation Order: knowledge development related to the choice of order in arithmetic’s problems (linked to ‘ARI’ lessons).
Audio Arithmetic: knowledge development related to the basic arithmetic operations through multimedia and visualisations (linked to ‘ARI’ lessons).
Mathematics Driller: knowledge development related to quartile values (linked to ‘QUA’ lessons).
Number Composition: knowledge development related to number identification and positioning (linked to ‘NUM’ lessons).
During the intervention period, 4th grade students showed a clear preference (54% of the total submissions) for the Mathematics Driller, the Mathematics-Quiz, and the Calculation Order exercises. In contrast, exercises like Mathematics Decimals, Calculations Fractions, Cards Game, and Bubble Mathematics were not so appealing to them (less than 1% of the total submissions). Similarly, the later exercises maintained an equally low practice share as far as the 5th and the 6th grade students were concerned. However, following the similar trends, Mathematics Driller was, once again, the most preferred exercise type in the 6th grade and the second most practiced exercise type in the 5th grade. The exercise type Audio Arithmetic was also amongst the most practiced events sharing a total of 30% over the overall submissions.
By considering the nature of the most practiced exercises (i.e., additions and subtractions), as well as the statistically positive improvement that most students across all the grades demonstrated in the fluency tests (Tables 4 and 5), a positive correlation was identified.

5.2. Practicing Behaviour and Fluency Development

Given that students’ practicing behaviour was positively correlated with improvement in their fluency performance, an additional effort was made to explore the threshold of time and its impact on fluency advancement. It should be noted that for this analysis the time measurement metric was chosen to be in seconds, as it aligned to the fast-paced nature that such exercises have and the limited timespan that students were given to complete the arithmetic fluency assessments (180 s).
Figure 2 illustrates the time that participants spent undertaking exercises on the digital platform which further conferred their practicing behaviour. The most notable difference was observed across the extremes (i.e., junior versus senior grade students) as the initial assumption was that students from the lower grades would require more practicing time than the senior ones. In greater detail, 4th grade students demonstrated small differences as far as their practicing time was concerned, though one of these cohorts (School 2) demonstrated stronger competences, as seen from the number of completed exercises. On the other hand, 5th grade students had greater variation in their practicing behaviour and outcomes which, in turn, prohibited any reliable conclusions from being drawn. Finally, students from the 6th grade outperformed all the other cohorts in terms of practiced exercises, while also being the second-best performing group in terms of time.
In addition to the previous observations, a new measurement was introduced to compare students’ performance (score) against the practicing time. The main incentive behind this decision was the identification of the optimal practicing time before no further (demonstratable) improvement in participants’ fluency skills could be identified. For this comparison, quartiles from the total number of the digital submissions were taken into consideration and linked to the individuals’ assessment scores. Figure 3 illustrates the correlation between student performance over time. For demonstration purposes this variable was called ‘effort’. Based on the LMS log-data, students’ effort could be categorised into the following distinct categories: (1) High Practice, (2) Moderately High Practice, (3) Moderately Low Practice, and (4) Low Practice.
By linking students’ performance with their practicing effort, we further classified them in accordance with their learning status as follows: (a) those whose performance improved, (b) those whose performance remained unchanged, and (c) those whose performance decreased. Based on these clusters, the following observations were made:
  • Students from the 6th grade outnumbered other groups both in fluency improvement and in terms of completed exercises. Moreover, there were no students in this cohort whose performance decreased.
  • Students from the 4th grade demonstrated a reasonable improvement over time, although they had been practicing considerably less, having less exercises completed.
  • Students from the 5th grade had too intensified variety in their performance and, therefore, we could not draw any definite conclusions regarding their performance.
Considering the above, students could be further categorized into the following clusters:
  • Students whose fluency skills improved after practicing for a considerably small amount of time or, otherwise, completing only a small number of exercises.
  • Students who spent a considerable amount of time practicing, yet their performance not only did not improve but, instead, was negatively impacted.
Although the primary data available in this study did not allow for any further examination, this outcome raised important inquiries, which future studies should consider exploring.

5.3. Evaluation of Students’ Performance

By narrowing it down to the cohort-level (Figure 4) it was revealed that pupils’ mathematics fluencies differed across the educational levels, as the more senior students displayed better performance results in both tests. On the grounds of these results and the subsequent cohort-specific analysis (Table 4) a preliminary validation of our hypothesis could be made regarding the potential of the digital learning tool to promote and support pupils’ fluency development across all the educational levels and knowledge bases.
Following the summative data analysis, a statistically significant difference (with 99% Confidence Interval—CI) between the samples’ performance (pre/post intervention) was identified (Table 5).
While considering the improvement identified in participants’ performance, we progressed further and explored students’ practicing habits with the integrated digital system. To examine this matter, we performed a Multiple Linear Regression (MLR) in consideration of the following variables (Table 6): (i) students’ average practicing time per day (X1), (ii) the number of exercises each student performed (X2), and (iii) the total number of days spent practicing (Y). Equitation (1) describes the MLR analysis formula used in this study.
Y = a + b 1 X 1 + b 2 X 2
where, a is the constant value, b 1 the coefficient of X 1 and b 2 the coefficient of X 2 , respectively.
To determine the total number of days students spent practicing (Y) we correlated students’ average practicing time per day ( X 1 ) and the number of exercises each student performed ( X 2 ). In view of this outcome, we accounted the mean values of students’ average practicing time (exercises explicitly related to fluency development) and the average number of practiced times. The conclusive analysis revealed that for the students to observe any substantial improvement in their performance, deliberate practice was required for at least 68 days, with a minimum of 3 min a day.

6. Discussion

The primary goal of this study was to determine whether digital tutoring systems can support students’ arithmetic fluency development in view of the deliberate practicing notion and, further, to explore how LMS-generated data provides genuine opportunities for investigating student performance.
The findings emerged from a representative sample consisting of primary school students, located in the same geographical location but in different school contexts. The results indicated a positive association between the instructional approach that the integrated digital learning tool offers and its impact on students’ arithmetic fluency development. To this end, exploration of links between the learners’ background, intrinsic learning motivation, and the associated practicing behaviors in an LMS should be investigated in future works to facilitate the personalization of students’ learning experience, utilising multimodal data (e.g., digital traces, psychometrics), as also indicated by [46].
The average practicing time to average practicing days ratio can be considered a highly impactful indicator of student performance among schools. In addition, it justifies how deliberate practicing impacts students’ mathematics fluency development. These findings were consistent with recent research efforts [2,36], thus, contributing to the wider body of knowledge related to instructional science.
In mathematics education, the term ‘success’ is frequently associated with the “achievement of set goals”. These goals are further categorized into the following categories: (a) master-oriented goals and (b) performance-oriented goals [47,48]. This classification can also be applied in this study after considering that fluency is synonymous to performance. The latter is also aligned to the definition that Cartwright [49] offers regarding the essence of mathematical fluency. According to the author [49] an individual is considered mathematically fluent after being able to utilise appropriate strategies and reasoning power to develop his/her conceptual understanding. Students from the 4th grade of our sample, for example, chose to practice with exercises that were related to quartiles and arithmetic operations. Associating these higher levels of practicing with their fluency improvement, it can be concluded that those students are mathematically fluent in these areas (Table 3). However, given the demanding and complex societal system we are living in, it would be reasonable to claim that prospective professionals need to demonstrate both performance-oriented and mastery-oriented skills (i.e., in-depth understanding of the philosophy behind mathematics).
The role of educational technology during these challenging and demanding times has been more important than ever. As societies shift over to the new norm it is imperative to ensure that the available multimodal educational technology solutions can fulfill not only the short-term needs (e.g., content delivery, provision of exercises) but also the long-term precision educational needs (e.g., personalised learning paths, monitoring of students’ progression, adaptive support and guidance) [37]. Scholars consider the contribution of LMS to the modern educational system as a key component, since these technologies extend students’ self-efficacy, support the development of their cognitive capacity, and promote the development of fluency [50].
These findings agree with the conclusions drawn by López-Pernas et al. [51] who utilised an escape room, in conjunction with a digital learning tool, to teach programming and mathematics. The results, in both cases, demonstrated positive outcomes in both participants’ computational thinking (CT) abilities and fluency competence.
In addition, the integrated LMS brings together various gamification elements as described in [52] and illustrated in Figure 5. The presence of such instructional features can potentially increase learners’ extrinsic motivation, promote self-efficacy, and, thus, facilitate self-regulation of learning. Lastly, the inclusion of such a tool from as early as the primary school level has been correlated positively to the prevention of misconceptions, while also setting the foundations for continuous and sustainable knowledge development.

7. Conclusions

Early solid foundations in fundamental mathematics is critical for developing thinking abilities in young children’s daily lives, which they will experience as they progress. Our major goal was to build the groundwork for practical intervention employing an LMS to improve ability in arithmetic operations. In the context of this work, our exploration of the impact that LMS has on learners’ arithmetic fluency development and advancement was achieved. The key findings showcased that most of the students (n = 720) demonstrated a positive improvement in their arithmetic fluency development (97% of the 6th graders, 83% of the 4th graders, and 76% of the 5th graders) with a medium-size effect (Cohen’s d = 0.51). This outcome could be justified either under the consideration of the cognitive mindset that individuals naturally develop as their age matures [53] or in accordance with the findings of Rinne et al. [16] who linked reading fluency, which is naturally more advanced as school grades progress, with the calculations’ performance.
The current findings also coincide with previous research work related to the opportunities for undisrupted and unconditional practicing that modern LMS offers [19,24,27,54], as well as the importance and the effectiveness of automated feedback which compliments learners’ reflection [55,56]. Nonetheless, while considering the limitations of these studies, the present work broadens the possibilities of scaling the proficiency and numeracy in all essential arithmetic operations in an LMS. Moreover, the substantial improvement in pupils’ arithmetic fluency, which has been correlated to their practicing behaviours in the provided digital learning system, further validates the findings of Sun and Xie [57], who concluded that computer-supported education can be particularly effective for goal-oriented students.
Furthermore, the present study, confirms and further expands the key findings of Lee and Choi [19] who demonstrated significant improvements in the development of basic numeracy skills, under the aid of a game-based intervention. To this end, the importance of providing learners not only with a variety of gamified exercises but also doing so in a structured/personalised form emerges. Consequently, this was particularly important regarding both students’ performance growth rates as well as the appropriateness and effectiveness of the examined intervention. Student improvements observed in pupils’ fluency skills over the number of exercises can be seen in Figure 6. This study also made additional contribution by demonstrating how design practice compliance with gamification (Figure 1) and LA (Figure 6) in an LMS may lead to not only enhancing numeracy skills, but also allowing children to be fluent in simple arithmetic operations; a finding which contradicts the conclusions drawn by the authors in [24].
In view of the outcomes of the present work, the following implications are made regarding the theory of fluency development and the importance of precision education (PE) in mathematics.
  • To facilitate the development of students’ mathematic fluency, educational technologists and instructional designers are highly recommended to consider the wider adoption of a curriculum-driven and school-grade specific personalised learning path when integrating digital learning tools. Under this consideration, this study has demonstrated how deliberate and planned practice with self-regulated learning and automatic feedback can promote students’ mathematics competencies and maintain their interest in the subject. This becomes even more crucial when considering the recent pandemic outbreak and the lack of physical interaction between teachers and students.
  • Considering the preferences students showed in specific exercises, we can tentatively assume that not all instructional approaches are equally efficiently in training a target skill. Therefore, additional research on the instructional design features and the respective game elements is desired.
  • In view of the above, establishing a validated inventory (battery) of tasks for arithmetic fluency development, especially for primary school students, could be particularly helpful for educators and educational technologies.

8. Limitations and Future Directions

As with every research study, we have also faced several challenges which set the limitations of this work. Conducting this experiment in the UAE highlighted the impact that the language barrier has on non-native English students (e.g., usability issues with the user interface). In addition, the relatively young age group of the participants brought considerable difficulties when interpreting the physical assessment forms as their personal identification information was not always clearly displayed. As a result, those individuals whose identity could not be confidently matched to the LMS database were excluded from the study. Furthermore, only one cohort of 6th graders was included in the experimental study and, thus, their outcomes could not be examined in comparison to other equivalent groups. Therefore, future studies should emphasise both the strategic recruitment of the sample (i.e., at least two cohorts from different school districts) and the physical data collection approaches (e.g., premade identifiers for every student). Finally, although an improvement in pupils’ arithmetic fluency could be identified, follow-up assessments examining whether this improvement in fluency is short- or long-term should also be considered. Finally, using this study as an example, we wish that educators be motivated to conduct similar, or even larger-sized interventions, so that greater advantage of the LA features can be taken, both within the national and the international contexts.

Author Contributions

Conceptualization, U.B.Q., A.C. and M.-J.L.; methodology, U.B.Q. and A.C.; software, U.B.Q.; validation, U.B.Q., A.C. and M.-J.L.; formal analysis, U.B.Q.; investigation, U.B.Q.; resources, U.B.Q. and M.-J.L.; data curation, U.B.Q.; writing—original draft preparation, U.B.Q.; writing—review and editing, U.B.Q., A.C. and M.-J.L.; visualization, U.B.Q.; supervision, A.C. and M.-J.L.; project administration, M.-J.L.; funding acquisition, M.-J.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national).

Informed Consent Statement

School principals, teachers, and participants’ parents/guardians have declared and countersigned their consent for students’ involvement in the study.

Data Availability Statement

Anonymized data that support the findings of this study are available from the corresponding author upon reasonable request.

Acknowledgments

The authors would like to thank all the school principals, teachers, students, and parents who supported the conduct of this initiative.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Haelermans, C.; Ghysels, J. The Effect of Individualized Digital Practice at Home on Math Skills–Evidence from a Two-Stage Experiment on Whether and Why It Works. Comput. Educ. 2017, 113, 119–134. [Google Scholar] [CrossRef]
  2. Musti-Rao, S.; Telesman, A.O. Comparing the Effects of Two Practice Conditions on the Subtraction Fact Fluency of Fifth-Grade Students. J. Behav. Educ. 2020, 1–19. [Google Scholar] [CrossRef]
  3. Tanujaya, B. Development of an Instrument to Measure Higher Order Thinking Skills in Senior High School Mathematics Instruction. J. Educ. Pract. 2016, 7, 144–148. [Google Scholar]
  4. Anthony, G.; Walshaw, M. Effective Pedagogy in Mathematics. Educational Practices Series-19; UNESCO International Bureau of Education: Geneva, Switzerland, 2009; pp. 1–30. [Google Scholar]
  5. Shaw, S.T.; Pogossian, A.A.; Ramirez, G. The Mathematical Flexibility of College Students: The Role of Cognitive and Affective Factors. Br. J. Educ. Psychol. 2020, 90, 981–996. [Google Scholar] [CrossRef] [PubMed]
  6. Lehtinen, E.; Hannula-Sormunen, M.; McMullen, J.; Gruber, H. Cultivating Mathematical Skills: From Drill-and-Practice to Deliberate Practice. ZDM Math. Educ. 2017, 49, 625–636. [Google Scholar] [CrossRef]
  7. Ericsson, K.A. Summing Up Hours of Any Type of Practice Versus Identifying Optimal Practice Activities: Commentary on Macnamara, Moreau, & Hambrick. Perspect. Psychol. Sci. 2016, 11, 351–354. [Google Scholar] [CrossRef]
  8. Kourgiantakis, T.; Sewell, K.M.; Bogo, M. The Importance of Feedback in Preparing Social Work Students for Field Education. Clin. Soc. Work J. 2019, 47, 124–133. [Google Scholar] [CrossRef]
  9. Foster, C. Developing Mathematical Fluency: Comparing Exercises and Rich Tasks. Educ. Stud. Math. 2018, 97, 121–141. [Google Scholar] [CrossRef] [Green Version]
  10. Tubb, A.L.; Cropley, D.H.; Marrone, R.L.; Patston, T.; Kaufman, J.C. The Development of Mathematical Creativity across High School: Increasing, Decreasing, or Both? Think. Ski. Creat. 2020, 35, 100634. [Google Scholar] [CrossRef]
  11. Kaskens, J.; Segers, E.; Goei, S.L.; van Luit, J.E.H.; Verhoeven, L. Impact of Children’s Math Self-Concept, Math Self-Efficacy, Math Anxiety, and Teacher Competencies on Math Development. Teach. Teach. Educ. 2020, 94, 103096. [Google Scholar] [CrossRef]
  12. Kurvinen, E.; Kaila, E.; Laakso, M.-J.; Salakoski, T. Long Term Effects on Technology Enhanced Learning: The Use of Weekly Digital Lessons in Mathematics. Inform. Educ. 2020, 19, 51–75. [Google Scholar] [CrossRef]
  13. Geary, D.C.; Hoard, M.K.; Byrd-Craven, J.; Nugent, L.; Numtee, C. Cognitive Mechanisms Underlying Achievement Deficits in Children with Mathematical Learning Disability. Child. Dev. 2007, 78, 1343–1359. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Barnes, M.A.; Clemens, N.H.; Fall, A.-M.; Roberts, G.; Klein, A.; Starkey, P.; McCandliss, B.; Zucker, T.; Flynn, K. Cognitive Predictors of Difficulties in Math and Reading in Pre-Kindergarten Children at High Risk for Learning Disabilities. J. Educ. Psychol. 2020, 112, 685–700. [Google Scholar] [CrossRef]
  15. Huijsmans, M.D.E.; Kleemans, T.; van der Ven, S.H.G.; Kroesbergen, E.H. The Relevance of Subtyping Children with Mathematical Learning Disabilities. Res. Dev. Disabil. 2020, 104, 103704. [Google Scholar] [CrossRef] [PubMed]
  16. Rinne, L.F.; Ye, A.; Jordan, N.C. Development of Arithmetic Fluency: A Direct Effect of Reading Fluency? J. Educ. Psychol. 2020, 112, 110–130. [Google Scholar] [CrossRef]
  17. Brown, R.D.; Schmithorst, V.J.; Kroeger, L. Calculation. In Neuroscience of Mathematical Cognitive Development: From Infancy through Emerging Adulthood; Brown, R.D., Ed.; Springer International Publishing: Cham, Switzerland, 2018; pp. 59–77. [Google Scholar] [CrossRef]
  18. Reed, H.C.; Gemmink, M.; Broens-Paffen, M.; Kirschner, P.A.; Jolles, J. Improving Multiplication Fact Fluency by Choosing between Competing Answers. Res. Math. Educ. 2015, 17, 1–19. [Google Scholar] [CrossRef] [Green Version]
  19. Lee, H.K.; Choi, A. Enhancing Early Numeracy Skills with a Tablet-Based Math Game Intervention: A Study in Tanzania. Educ. Tech. Res. Dev. 2020, 68, 3567–3585. [Google Scholar] [CrossRef]
  20. Cui, J.; Georgiou, G.K.; Zhang, Y.; Li, Y.; Shu, H.; Zhou, X. Examining the Relationship between Rapid Automatized Naming and Arithmetic Fluency in Chinese Kindergarten Children. J. Exp. Child. Psychol. 2017, 154, 146–163. [Google Scholar] [CrossRef]
  21. Whitney, T.; Hirn, R.G.; Lingo, A.S. Effects of a Mathematics Fluency Program on Mathematics Performance of Students with Challenging Behaviors. Prev. Sch. Fail. Altern. Educ. Child. Youth 2016, 60, 133–142. [Google Scholar] [CrossRef]
  22. Russo, J.; Hopkins, S. Measuring Mental Computational Fluency with Addition: A Novel Approach; Mathematics Education Research Group of Australasia: Adelaide, Australia, 2018. [Google Scholar]
  23. Cozad, L.E.; Riccomini, P.J. Effects of Digital-Based Math Fluency Interventions on Learners with Math Difficulties: A Review of the Literature. J. Spec. Educ. Apprenticesh. 2016, 5, 19. [Google Scholar]
  24. Benavides-Varela, S.; Zandonella Callegher, C.; Fagiolini, B.; Leo, I.; Altoè, G.; Lucangeli, D. Effectiveness of Digital-Based Interventions for Children with Mathematical Learning Difficulties: A Meta-Analysis. Comput. Educ. 2020, 157, 103953. [Google Scholar] [CrossRef]
  25. Burte, H.; Gardony, A.L.; Hutton, A.; Taylor, H.A. Make-A-Dice Test: Assessing the Intersection of Mathematical and Spatial Thinking. Behav. Res. 2019, 51, 602–638. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Kormos, J.; Wilby, J. Task Motivation. The Palgrave Handbook of Motivation for Language Learning; Lamb, M., Csizér, K., Henry, A., Ryan, S., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 267–286. [Google Scholar] [CrossRef]
  27. Begeny, J.C.; Codding, R.S.; Wang, J.; Hida, R.M.; Patterson, S.L.; Kessler, S.; Fields-Turner, F.; Ramos, K.A. An Analysis of Motivation Strategies Used within the Small-Group Accelerating Mathematics Performance through Practice Strategies (AMPPS-SG) Program. Psychol. Sch. 2020, 57, 540–555. [Google Scholar] [CrossRef]
  28. Nichols, E.G.; Kohn, A. UAE: Brief Review of Education in the United Arabic Emirates. Math. Its Teach. Muslim World 2020, 265–275. [Google Scholar]
  29. Kamal, K. Education in the United Arab Emirates. WENR 2018. Available online: https://wenr.wes.org/2018/08/education-in-the-united-arab-emirates (accessed on 14 June 2022).
  30. Jarrah, A.M. The Challenges Faced by Pre-Service Mathematics Teachers during Their Teaching Practice in the UAE: Implications for Teacher Education Programs. Int. J. Learn. Teach. Educ. Res. 2020, 19, 23–34. [Google Scholar] [CrossRef]
  31. Eltanahy, M.; Forawi, S.; Mansour, N. STEM Leaders and Teachers Views of Integrating Entrepreneurial Practices into STEM Education in High School in the United Arab Emirates. Entrep. Educ. 2020, 3, 133–149. [Google Scholar] [CrossRef]
  32. Edugain. Edugain UAE: Math Worksheets, Online Tests and Practice. Natl. Curric. 2021. Available online: https://ae.edugain.com/ (accessed on 14 June 2022).
  33. Woolsey, M.L.; Tennant, L.; Kelly, M.; Rashad, F. Emirati Pre-Service Teachers in Mathematics and Science: An Ecobehavioral Assessment. In Proceedings of the 10th Annual International Conference of Education, Research and Innovation, Seville, Spain, 16–18 November 2017; p. 560. [Google Scholar] [CrossRef]
  34. Jarrah, A.M. Elementary Schools Mathematics and Science Teachers’ Perspectives on Using English as a Medium of Instruction. Online Submiss. 2020, 8, 473–482. [Google Scholar] [CrossRef]
  35. Chi Hyun, C.; Asbari, M.; Wijayanti, L.; Santoso, P.; Pramono, R.; Bernarto, I.; Purwanto, A. Implementation of Contextual Teaching and Learning (CTL) to Improve the Concept and Practice of Love for Faith-Learning Integration. Int. J. Control Autom. 2020, 13.1, 365–383. [Google Scholar]
  36. Walkington, C.; Bernacki, M.L. Personalizing Algebra to Students’ Individual Interests in an Intelligent Tutoring System: Moderators of Impact. Int. J. Artif. Intell. Educ. 2019, 29, 58–88. [Google Scholar] [CrossRef]
  37. Qushem, U.B.; Christopoulos, A.; Oyelere, S.S.; Ogata, H.; Laakso, M.-J. Multimodal Technologies in Precision Education: Providing New Opportunities or Adding More Challenges? Educ. Sci. 2021, 11, 338. [Google Scholar] [CrossRef]
  38. Christopoulos, A.; Kajasilta, H.; Salakoski, T.; Laakso, M.-J. Limits and Virtues of Educational Technology in Elementary School Mathematics. J. Educ. Technol. Syst. 2020, 49, 59–81. [Google Scholar] [CrossRef]
  39. Hoyles, C. Transforming the Mathematical Practices of Learners and Teachers through Digital Technology. Res. Math. Educ. 2018, 20, 209–228. [Google Scholar] [CrossRef]
  40. Christopoulos, A.; Sprangers, P. Integration of educational technology during the Covid-19 pandemic: An analysis of teacher and student receptions. Cogent. Educ. 2021, 8, 1964690. [Google Scholar] [CrossRef]
  41. Koponen, T.; Mononen, R. The 2-Minute Subtraction Fluency Test. In Unpublished Test Material; Niilo Mäki Instituutti: Jyvaskyla, Finland, 2010. [Google Scholar]
  42. Laakso, M.-J.; Kaila, E.; Rajala, T. ViLLE—Collaborative Education Tool: Designing and Utilizing an Exercise-Based Learning Environment. Educ. Inf. Technol. 2018, 23, 1655–1676. [Google Scholar] [CrossRef]
  43. Carr, J.E. Recommendations for Reporting Multiple-Baseline Designs across Participants. Behav. Interv. 2005, 20, 219–224. [Google Scholar] [CrossRef]
  44. Mc Caleb, K.N.; Andersen, A.; Hueston, H. An investigation of school violence and pre-service teachers. Curr. Issues Educ. 2008, 10, 11. [Google Scholar]
  45. Tukey, J.W. Exploratory Data Analysis; Addison Wesley: Reading, PA, USA, 1977; Volume 2, pp. 131–160. [Google Scholar]
  46. Qushem, U.B.; Christopoulos, A.; Laakso, M.-J. The Value Proposition of An Integrated Multimodal Learning Analytics Framework. In Proceedings of the 45th Jubilee International ICT Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 23–27 May 2022; pp. 666–671. [Google Scholar] [CrossRef]
  47. Dweck, C.S. Motivational Processes Affecting Learning. Am. Psychol. 1986, 41, 1040–1048. [Google Scholar] [CrossRef]
  48. Senko, C.; Miles, K.M. Pursuing Their Own Learning Agenda: How Mastery-Oriented Students Jeopardize Their Class Performance. Contemp. Educ. Psychol. 2008, 33, 561–583. [Google Scholar] [CrossRef]
  49. Cartwright, K. “Because 7 and 8 Are Always in All of Them”: What Do Students Write and Say to Demonstrate Their Mathematical Fluency? Mathematics Education Research Group of Australasia: Adelaide, Australia, 2019. [Google Scholar]
  50. Hwang, G.-J.; Wang, S.-Y.; Lai, C.-L. Effects of a Social Regulation-Based Online Learning Framework on Students’ Learning Achievements and Behaviors in Mathematics. Comput. Educ. 2021, 160, 104031. [Google Scholar] [CrossRef]
  51. López-Pernas, S.; Gordillo, A.; Barra, E.; Quemada, J. Analyzing Learning Effectiveness and Students’ Perceptions of an Educational Escape Room in a Programming Course in Higher Education. IEEE Access 2019, 7, 184221–184234. [Google Scholar] [CrossRef]
  52. Arnab, S.; Lim, T.; Carvalho, M.B.; Bellotti, F.; Freitas, S.; Louchart, S.; Suttie, N.; Berta, R.; De Gloria, A. Mapping learning and game mechanics for serious games analysis. Br. J. Educ. Technol. 2015, 46, 391–411. [Google Scholar] [CrossRef] [Green Version]
  53. Seminelli, M.D. Interventions to Develop a Growth Mindset in a Remedial Math Classroom. Math. Mil. 2018, 23, 3. [Google Scholar]
  54. Maki, K.E.; Zaslofksy, A.F.; Knight, S.; Ebbesmeyer, A.M.; Chelmo-Boatman, A. Intervening with Multiplication Fact Difficulties: Examining the Utility of the Instructional Hierarchy to Target Interventions. J. Behav. Educ. 2021, 30, 534–558. [Google Scholar] [CrossRef]
  55. Kochmar, E.; Vu, D.D.; Belfer, R.; Gupta, V.; Serban, I.V.; Pineau, J. Automated Personalized Feedback Improves Learning Gains in An Intelligent Tutoring System. In Artificial Intelligence in Education; Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2020; pp. 140–146. [Google Scholar] [CrossRef]
  56. Erümit, A.K.; Çetin, İ. Design Framework of Adaptive Intelligent Tutoring Systems. Educ. Inf. Technol. 2020, 25, 4477–4500. [Google Scholar] [CrossRef]
  57. Sun, Z.; Xie, K. How Do Students Prepare in the Pre-Class Setting of a Flipped Undergraduate Math Course? A Latent Profile Analysis of Learning Behavior and the Impact of Achievement Goals. Internet High. Educ. 2020, 46, 100731. [Google Scholar] [CrossRef]
Mti 06 00061 g001 550
Figure 1. Indicative examples of the platform’s gamified exercises’ user interface.
Figure 1. Indicative examples of the platform’s gamified exercises’ user interface.
Mti 06 00061 g001
Mti 06 00061 g002 550
Figure 2. Variations in participants’ practicing behaviour (n = 720).
Figure 2. Variations in participants’ practicing behaviour (n = 720).
Mti 06 00061 g002
Mti 06 00061 g003 550
Figure 3. Clusters of effort-performance.
Figure 3. Clusters of effort-performance.
Mti 06 00061 g003
Mti 06 00061 g004 550
Figure 4. Distribution of the fluency assessment results.
Figure 4. Distribution of the fluency assessment results.
Mti 06 00061 g004
Mti 06 00061 g005 550
Figure 5. ViLLE LMS design influence determinants (* sign) in the LM-GM model.
Figure 5. ViLLE LMS design influence determinants (* sign) in the LM-GM model.
Mti 06 00061 g005
Mti 06 00061 g006 550
Figure 6. Overview of students’ performance variation over time.
Figure 6. Overview of students’ performance variation over time.
Mti 06 00061 g006
Table
Table 1. Overview of the curriculum structure.
Table 1. Overview of the curriculum structure.
GradeArithmetic LessonsOther Featured Lessons
Additions/SubtractionsMultiplications/Divisions
4thAddition and Subtraction; Addition and Subtraction with like fractions; Addition and Subtraction with unlike fractions; Addition and Subtraction with decimal numbers; Decimal numbers: Columnar addition and subtraction, comparing; Addition and subtraction with negative numbers.Multiplication and division, order of operations; Multiplication tables; Division: partition; Columnar multiplication and long division; Remainders.Expressions; Geometry: Lines, triangles and quadrilaterals; Coordinate plane; Fractions and mixed numbers; Decimal numbers; Hundredths; Units of length and mass; Time; Comparing and rounding integers; Negative numbers; Charts and graphs; Equations; Large numbers; Calculations with large numbers.
5thBasic Arithmetic Operations; Reducing fractions, adding and subtracting mixed numbers; Addition and subtraction with decimal numbers.Multiplication, and Division; Multiplying and dividing fractions; Multiplying and dividing a decimal number.Equations, problem solving and order of operations; Percentage, Fractions, integers and decimal numbers; Basics of geometry; Circles, and Triangles; Quadrilaterals and solid figures; Large, and mixed numbers; Tables, charts and diagrams; Mean, median and mode; Probability and Statistics; Measurement, Units of mass and volume; Time and speed; Area: Exponents and units; Area: Triangles and parallelograms; Similarity and scale; Reflection, Estimating, Coordinate system and sets.
6thBasic Arithmetic Operations; Addition and Subtraction with decimal numbers; Racer: Addition and subtraction.Multiplication, and Division; Multiplication and division with decimal numbers; Multiplication and division with fractions.Large, decimal numbers and fractions; Scales, geometry, measurement and maps; Triangles and Rectangular cuboids; Quadrilaterals and solid figures; Reducing fractions; Mixed numbers with common denominators; Expanding unlike fractions to common denominators; Calculating percentages; Prices, Time and units of time; Number line and equations; Negative numbers; Integers and the coordinate plane; Divisibility and factors; Functions; Calculating time intervals; Speed and time zones; Tables, graphs and charts; Probability; Racers; Problem solving.
Table
Table 2. Sample’s demographics.
Table 2. Sample’s demographics.
SchoolsTypeNumber of StudentsTotal
Grade 4Grade 5Grade 6
School 1Mixed236172-408
School 2Mixed161--161
School 3Girls-292958
School 4Mixed93--93
Table
Table 3. Exercises with cumulative percentage scores.
Table 3. Exercises with cumulative percentage scores.
Exercises *Lessons4th Grade5th Grade6th Grade
f%Scoref%Scoref%Score
Mathematics QuizARI65410.09%15740.46%4430.31%26
Mathematics Calculation FormsNUM901.39%18470.81%5510310.76%146
Mathematics Calculation RowsADD/SUB5268.11%20212814.87%15512412.96%371
Mathematics DecimalsDEC30.05%44333.83%130222.30%141
Calculation FractionsFRA40.06%59192.21%75575.96%179
Fill in ExerciseSUB1372.11%4700.00%000.00%0
Audio ArithmeticARI5368.27%17925729.85%258687.11%261
Match PairsMAO1221.88%122212.44%45121.25%72
Mathematics DrillerQUA206031.78%41913315.45%25616417.14%644
Cards Game-00.00%000.00%010.10%30
Calculation OrderARI77711.99%616576.62%12221322.26%559
Number ExerciseARI5979.21%129273.14%54959.93%36
Bubble Mathematics-00.00%000.00%030.31%22
RunnerADD/SUB6139.46%12713115.21%101868.99%89
Number CompositionNUM3635.60%50445.11%8760.63%60
Total-6482100.00%2331861100.00%1383957100.00%2637
* Exercises: (Category 1) ARI: Arithmetic operations, Sum and Difference, Basic Arithmetic operations, Columnar Addition and Subtraction; ADD/SUB: Addition and subtraction; (Category 2) Listen and Visualisation; (Category 3) NUM: Fill in missing number, Number identifications, Placing numbers; (Category 4) DEC: Decimals; FRC: Fractions; SUB: Subtractions; (Category 5): MAO: Matching Arithmetic Operations; (Category 6): QUA: Quartiles.
Table
Table 4. Statistical analysis of participants’ pre- and post-assessment results.
Table 4. Statistical analysis of participants’ pre- and post-assessment results.
CohortsPre-AssessmentPost-Assessmentt-Test
SchoolGradeN x ¯ s x ¯ stp
School 1 423646.1416.9753.7216.069.200.00001
School 2 416144.6417.5356.6519.5710.90.00001
School 4 49346.4117.6558.6019.7311.190.00001
School 1 517253.4920.6462.1317.977.580.00001
School 3 52950.3413.8058.3417.234.970.00003
School 3 62957.8322.3573.8628.546.800.00001
Table
Table 5. Overview of participants’ performance.
Table 5. Overview of participants’ performance.
EvaluationN x ¯ st-TestpSig.Cohen’s d
Pre-assessment 72048.2418.6019.692.2 × 10−160.010.51
Post-assessment 72058.0118.98
Table
Table 6. Association between performance and digital deliberate practicing.
Table 6. Association between performance and digital deliberate practicing.
VariablesCoefficientStandard ErrortpCI (95%)
Constant26.3434.6565.658 < 2.21 × 10 − 8 * 0.000
X1 0.061 0.022 2.740 0.0063 * 0.001
X2 0.405 0.022 18.847 < 2 × 10 − 16 * 0.000
* The coefficient represents the strength of association; Residual standard error is 50.04 on 717 degrees of freedom; The model can be examined using the coefficient of determinations (R2), where the R2 value accounts for the 33% of the predictors in the model. Adjusted R2 = 0.330; F-count = 177.7; p < 0.01; α = 0.05
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Qushem, U.B.; Christopoulos, A.; Laakso, M.-J. Learning Management System Analytics on Arithmetic Fluency Performance: A Skill Development Case in K6 Education. Multimodal Technol. Interact. 2022, 6, 61. https://doi.org/10.3390/mti6080061

AMA Style

Qushem UB, Christopoulos A, Laakso M-J. Learning Management System Analytics on Arithmetic Fluency Performance: A Skill Development Case in K6 Education. Multimodal Technologies and Interaction. 2022; 6(8):61. https://doi.org/10.3390/mti6080061

Chicago/Turabian Style

Qushem, Umar Bin, Athanasios Christopoulos, and Mikko-Jussi Laakso. 2022. "Learning Management System Analytics on Arithmetic Fluency Performance: A Skill Development Case in K6 Education" Multimodal Technologies and Interaction 6, no. 8: 61. https://doi.org/10.3390/mti6080061

Article Metrics

Back to TopTop