A Current Overview of the Use of Learning Analytics Dashboards

: The promise of Learning Analytics Dashboards in education is to collect, analyze, and visualize data with the ultimate ambition of improving students’ learning. Our overview of the latest systematic reviews on the topic shows a number of research trends: learning analytics research is growing rapidly; it brings to the front inequality and inclusiveness measures; it reveals an unclear path to data ownership and privacy; it provides predictions which are not clearly translated into pedagogical actions; and the possibility of self-regulated learning and game-based learning are not capitalized upon. However, as learning analytics research progresses, greater opportunities lie ahead, and a better integration between information science and learning sciences can bring added value of learning analytics dashboards in education.


Introduction
The promise of Learning Analytics (LA) in education is to collect and analyze data with the ultimate ambition of improving students' learning.Over the past decade, LA has transitioned from a mere concept to a solid area of study, encompassing a worldwide network of scholars and professionals.LA is defined as: "the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" [1] (pg. 252).Students and teachers interact with LA to process and consume educational data often through dashboards, also called Learning Analytics Dashboards (LADs).LADs are control panels that can be tailored to display LA components that update according to the learning processes, also in real time.Therefore, dashboards can serve as a medium employing LA to monitor, assess, and visualize learners' online behaviors related to learning, and other educational activities, such as study progress, grades, and group performance [2].A subset of technical methods can be used for visualizing data in LADs.Examples of advanced LADs are presented in our latest research [3,4].The visualizations mainly exemplify descriptive information, such as time spent on an online task, access to various online resources, and learning progression in a course or subject/task, but also provide means to compare user results.The dashboard visualizations of user data aim to increase pupil motivation, self-regulation, educational performance, and, furthermore, teacher engagement [5][6][7] on all education levels.With this potential, LADs are rapidly becoming integral assessment instruments for teaching and learning, and especially so in higher education, rather than at lower educational levels.This is because institutions within higher education often have more centralized data and technology infrastructure [8].The COVID-19 pandemic functioned as a catalyst for an exponential growth in the use of digital technology in classrooms, Educ.Sci.2024, 14, 82 2 of 15 and the creation of data from this and its subsequent visualizations.Therefore, there is a clear need for the use of LADs in the classroom, especially if they can support teachers' pedagogical decision-making.
With the aim of understanding how far we have come and what the future reserves in this field, in this article, the authors synthesize and provide an overview of the latest systematic reviews examining the use of learning analytics and its dashboard visualization, identifying several trends and key areas for future research opportunities.

Methods
We started by checking if LA research was trending by simply conducting a search in Scopus for the term "learning analytics" from 2010 to 2023.This included 7496 publications related to LA. 'Title', 'Abstract', and 'Keywords' were considered as field tags.We examined the following document types: article, conference paper, conference review, book, book chapter, review, editorial, and data paper.With the knowledge that there was a large number of publications in this field, we then performed a so called 'umbrella review' [9], to compile evidence only from multiple systematic reviews, that is, literature bearing a high level of evidence, into a comprehensive article that allowed us to gain a broader perspective on the use of LADs in education and helped us identify trends.
We then followed the Joanna Briggs Institute's manual for evidence synthesis and followed the steps required in a systematic review of any evidence type [10], which includes: (1) formulating the research question, (2) identifying relevant studies and defining inclusion and exclusion criteria, (3) study selection (4), charting the data, and (5) collating, summarizing, and reporting the results.

Research Questions
With the aim of providing an overview of the latest systematic reviews examining the use of LADs, the main research question that guides this study was: What is the current use of learning analytics dashboards in education?
The main research question is operationalized by a series of sub-questions designed to describe contextual and design aspects related to the use of LADs: Who are the users or target users?What are the visualization techniques?What design frameworks are used?What are the target outcomes?Are ethical issues discussed?And are strategies in play?
Finally, one last sub-question is about understanding the evidence of the effects of LADs: What is the research evidence about the use of LADs?

Identify the Relevant Studies
We searched for relevant systematic reviews in Scopus, Web of Science, and Scholar.Google.se.In those databases we searched for the terms: "systematic review" or "systematic literature review", and "learning analytics" in the title of articles, and "dashboard" in all fields.We also conducted a manual search on Google to complete the study's dataset, choosing all systematic reviews that met the criteria listed below.
The selection of papers was performed by the first and second authors and was based on the following criteria: Included studies

1.
That contained "learning analytics" and "systematic literature review" or "systematic review" in the title.

3.
That were published in English.

4.
That were published only between 2019 and April 2023.

1.
That did not consider "learning analytics" as one subject or considered the two, each into a separate subject, that is, "learning" and "analytics".

Study Selection
The entire process of article selection returned 63 records published during the last 5 years, between 2019 and April 2023, in the three databases.Google search took into account all document types and year range, returning over 100,000 records.The two first authors then removed the duplicates and all articles that were outside the scope of this study.The majority of records in Google were also quickly removed since it returned the current systematic reviews within the first few screens or number of records.Lastly, 22 records were included in this study.

Data Charting
Table A1 in the Appendix A provides an overview of the included 22 systematic reviews about the use of LADs in education.

Reporting the Results
The final step is about reporting the results of the search process.As we mentioned, before starting the review work, we engaged in a simple search exercise to see if the terms learning analytics was trending.The returned 7496 publications related to LA resulted in Figure 1.As shown in Figure 1a, over the past years, there has been a notable upward trend in publications related to LA.In 2010, only one publication was recorded in Scopus, and since then, there has been a consistent increase, reaching a peak of 983 publications in 2022.This growth reflected a growing interest in and recognition of the significance of LA, not just in education.The numbers indicated a nearly tenfold increase over the course of a decade, highlighting the expanding body of knowledge and research contributions in the field.The numbers of publications during the last five years further emphasized the sustained momentum and attention dedicated to LA in recent years.As can be seen in Figure 1b, research from the USA, Spain, Australia, the UK, Germany, and China represented the bulk of publication relevant to LA.
Table 1 illustrates a summary of the 22 reviews, considering the year of publication, country of study, the number of articles reviewed in the publication, and citations.As can be seen, Finland, with four articles, and the UK, USA, and Germany, with three articles each, have the most reviews on LADs.Moreover, the reviews published in 2019 and 2020 have already reached 193 and 290 citations, respectively.
related to LA resulted in Figure 1.As shown in Figure 1a, over the past years, there has been a notable upward trend in publications related to LA.In 2010, only one publication was recorded in Scopus, and since then, there has been a consistent increase, reaching a peak of 983 publications in 2022.This growth reflected a growing interest in and recognition of the significance of LA, not just in education.The numbers indicated a nearly tenfold increase over the course of a decade, highlighting the expanding body of knowledge and research contributions in the field.The numbers of publications during the last five years further emphasized the sustained momentum and attention dedicated to LA in recent years.As can be seen in Figure 1b, research from the USA, Spain, Australia, the UK, Germany, and China represented the bulk of publication relevant to LA.Table 1 illustrates a summary of the 22 reviews, considering the year of publication, country of study, the number of articles reviewed in the publication, and citations.As can be seen, Finland, with four articles, and the UK, USA, and Germany, with three articles each, have the most reviews on LADs.Moreover, the reviews published in 2019 and 2020 have already reached 193 and 290 citations, respectively.

Ethics of LA
As student data are increasingly being collected, processed, and visualized on dashboards, ethical issues are becoming prominent, making the use of student data a work in progress, both technically and pedagogically, as well as juridically.Among the ethics areas, privacy is a major concern [2,15,16,18,22,[24][25][26][27][28]30,31] in relation to access, identification, and data governance of student data.Transparency is also a significant issue [15,22,28] in terms of source, analysis, and presentation of data.Other ethical issues are related to informed consent [15,22,28], i.e., raising awareness among students and teachers about the fact that their data is being tracked and analyzed; responsibility [22], that is, personal and institutional responsibility towards the use of student data; and validity [22,28] in relation to the accuracy of the algorithms used to interpret student data.However, despite the importance of the ethical issues, only a small percentage of LA research consider the relevance of ethics in this field [34].

Lacking Institutional Strategies
LA research has focused on the individuals, that is, the educator/teacher and student who are supposed to make sense of the analytics to contribute to teaching and learning, respectively [16].Studies have shown that the responsibility to change learning behaviors when prompted by LA is put upon the higher education students, while to better reap the benefits of LA systemic interventions should be developed at the institutional level [11,14,18,21,22,31].However, even though the legal, ethical, and effective utilization of LA is, indeed, the responsibility of the educational institution [15,21,22,30], a better interplay between the institution and students/teachers should foster better implementation, accountability [11,22], equity [24], and academic planning [23].

LAD Effectivenness
According to the reviews, the design of LADs should be guided by educational theories and pedagogical practices to be effective in helping educators and students to enhance teaching and learning [2,[11][12][13][14][20][21][22][23]26].The reviews also state that studies showing effectiveness in terms of reliably improved learning outcomes are steadily increasing yet few [15,20], corroborating earlier reviews [34], while the evidence of the effectiveness is growing in terms of motivation, engagement, and system usage behaviors [15,16,18,[26][27][28]31]. One major obstacle is the methodologies of much of the research presented in the reviews, which relied on small convenience samples and simple correlational pre/post intervention designs.Ideally, and where possible, studies should also include experimental research methods like randomized control trials.Therefore, one way to increase effectiveness is by filling the gap between learning science and information science, that is, dashboard design led by learning and cognitive theories [31], and include systematic experimental research design [11], as earlier research also has pointed out, respectively [35,36].

Trends Based on the Research Reviews
The systematic review articles make an important contribution to a broader and general understanding of the affordances of LADs.Specifically, the authors can extract five trends in connection to the use of LADs that can be of value to practitioners and researchers, and ultimately learners.Those are presented below.

Trend 1. LADs Research Is Rapidly Evolving
The simple fact that only one review (within our timeframe) on this topic was published before 2020, while all others were published after (see Table 1), led us to believe that LADs' time is now.Perhaps one reason for this is that COVID-19 pandemic functioned as catalysator for an exponential use of educational technology in classrooms, and the exploitation of their generated data thereafter.This is in accordance with one of the reviews included here [2] and a recent article focusing on the challenges and limitations concerning LA [37].This indicates that LADs research has been expanding rapidly, especially during the last few years, and the trend is expected to grow and evolve even more rapidly at all levels of education.This notion is also put forward in the yearly Horizon Report (Educause.https://www.educause.edu/;accessed on 4 December 2022) that annually identifies key educational technology trends.It has listed LA and its dashboard visualizations as an emerging technology.What this means is that more and more educational institutions are betting on the notion that LADs could help in determining several target outcomes based on data to help the appropriate decision-making process in regard to students and teachers, and teaching and learning.Yet, there are only limited examples of large-scale and well-developed LADs implementations across educational institutions [38].This leaves a door wide open for developers to capitalize on this trend.

Trend 2. Inequalities and Inclusiveness
The papers on LA are published in educational technology, education, and engineering journals [2].As can be seen from Figure 1b, the origins of the papers are heavily concen-trated in North America, Europe, and Australia, and Asian countries also have a large footprint.This means that the development of LADs and its research is carried out in specific contexts.In a long-term perspective, this can become a bias.In fact, what and how data is visualized, is influenced by software developers and the specific context into which the dashboards-through various algorithms-are developed.This limits the broader implementation in other countries without a thorough local evaluation, which could result in problems, such as embedded racial or gender stereotypes, or unjust predictions [24,39].This could be amplified by the fact that LA research is lacking institutional strategies, exemplified in many of the reviews included here.Also, background information about learners like demographic data (gender, sex, age, race etc.) is not often reported in the included studies [15,24,26].Moreover, broader aspects of inclusiveness and disability are seldom researched [18,23].In addition, governments are lagging behind in comprehending the use of analytics, for example in K12 education [32].Verbert et al. [7] state that it is necessary to address the implications of predictive systems and bias in LA research, and making the visualizations transparent can offer a better understanding of the decision-making model for the users.This trend run contrary to the United Nations' Sustainable Development Goal 4-Quality Education-which aims to "ensure inclusive and equitable quality education and promote lifelong learning opportunities for all" (https://sdgs.un.org/goals/goal4; accessed on 24 October 2023).

Trend 3. Data Privacy, Ownership, and Legal Framework
The challenges of data privacy and ownership are, according to the reviews [2,15,16,18,22,[24][25][26][27][28]30,31], not sufficiently researched and discussed, as also pointed out in recent research literature [37,40].However, those are issues that could ultimately end up being politicized, possibly discouraging further development of LADs, if not properly attended at the right level, i.e., between students or legal guardians, teachers, and the respective educational institutions [15,31].Legal frameworks are proposed to mitigate concerns over LA [37,40], although this is a relatively new phenomenon with few examples in practice, which leaves room for interpretation of legal frameworks in different contexts [39].A key ethical concern is data ownership."Those who own the data own the future", writes Harari [41] (pg 89).According to the author, data ownership is going to bring even further inequalities into the world [41].This and the previous trend seem to be interrelated.The tech giants like Facebook, Google, and Amazon are amassing profits based on our personal data use, and they feed us with information based on that data to capture our attention.Those same tech giants are also part of the education domain.What stops them from steering our learning behaviors for their own profit, monetary or otherwise?Here, we argue that each of us should own our own data, period!

Trend 4. Prediction and Effectiveness-Lost in Translation
The reviews are in agreement about the notion that LA seems to provide efficient predictive models [11][12][13]15,18,21,22,25], as research has already indicated [37,40].Prediction algorithms are highly specialized and can predict retention [11,13,18,20,21,25,31,34], study success, test score, drop-out, and students' well-being behavior [11,13,25].Those predictions are turned into visualization elements to raise teachers' and students' awareness so that they can act upon said elements.Therefore, the trend is to design dashboards to be more effective and efficient by connecting learning theories to dashboard design [2,12,13,20,21,23].However, that in itself is not sufficient for either teachers or students to enhance the effectiveness of dashboards and visualizations elements, or as Sønderlund et al. [11] put it: "how this knowledge is translated into interventions".What this means is that a LAD in itself is not a necessary condition to bring behavioral changes to educational processes that can be solved using traditional means.As with all educational technology, regardless of type or purpose, we need to devise a method for translating the possible effect into practice.According to the reviews, the research evidence of their effectiveness in improving teaching or learning is growing [11,16,25,26], yet still limited [11,15,20,21].Indeed, there is a scarcity of credible studies in the peer-reviewed LA literature that demonstrate significant impact on learning or student success [38].

Trend 5. Self-Regulated Learning and Game-Based Learning
SRL refers to learning guided by intrinsic or extrinsic motivation to learn and to plan, monitor, and assess personal progress and understanding to adjust for the next learning task [42].Game-based learning refers to learning and instruction through the use of a digital application that combines aspects of video games, [43], gamification, and has the potential to enhance learners' knowledge gain and comprehension, motivation, and performance [26].
There are several SRL models proposed in the literature [44][45][46][47][48][49], and many have been incorporated into MOOCs (Multiple Online Open Courses), or online courses with a selfpaced structure, like those exemplified in the reviews [2,12,14,20,23,27].A similar situation is reached with game-based learning, that is, gamification concepts are introduced in many classroom and online learning situations and users of diverse demographics [15,17,18,23,26,30].In the articles included in the reviews, the authors visualize students' progress, i.e., course or module completion, by using progress bars or charts.Unfortunately, visualization per se is not sufficient for effective feedback either for self-regulation or game-based learning [23,26].SRL guiding rules, i.e., metacognition, motivation, and emotion, also need to be accommodated into LADs [23], as well as demographic data and previous game experience [26].The reviews have highlighted that existing LADs are rarely grounded in learning theories [23,24].In this study though, we have demonstrated that there are various frameworks to guide the design of LADs.In addition, there is an immense opportunity to incorporate SRL and game-based learning into LADs.SRL provides an opportunity, especially in higher education, where online, self-paced learning is growing, and students have recognized a possible value of LADs suited for SRL [22].Furthermore, game-based learning could provide an opportunity to support a whole range of intellectual disabilities in lower grades [18].

Conclusions
The latest systematic review articles on LA and dashboards have different focuses, therefore there is a variety of contextual and design aspects in connection to LADs.The reviews report the bulk of the use of LA within higher education, and the use of simple statistical graphs as visualization elements.The reviews also address the use of educational theories to guide the design of LA dashboards, with SRL and game-based learning being prevalent theories, and the tracking of outcomes such as users' performance, behaviors, and effective measures.Furthermore, the reviews note a raising awareness of ethical issues aligned with the growing use of LADs, and the lack of organizational policies to make the use of LA more efficient, as well as accountable and equitable.Last, but not least, the reviews reveal a sparsity of studies that show effectiveness of improved learning outcomes-the promise of LA-even though the evidence is growing.
Based on the research evidence exemplified in the review articles, the researchers have identified five emerging trends in connection to LADs.The first and most obvious one is that LA and its visualization through dashboards is being increasingly used, especially in higher education and western countries, but also growing at lower educational levels and Asian countries.This means that LADs afford individuals or organizations to train educational decision-making in a hybrid format, where the 'human' is aided by the 'machine'.The second emerging trend concerns the fact that the development of LA and dashboards occurs in specific regions of the world, often omitting demographics and disability aspects, which projects development inequalities and exclusiveness in the LA algorithms used in LADs.The third trend highlights the ethical and legal complexity related to the analytics of educational data, pinpointing the absolute necessity for a deeper discourse with all stakeholders, and a clearer way forward about a LA legal and ethical framework on all levels of education.The fourth trend is all about prediction, and little about learning effectiveness.LA algorithms are effective at predicting.However, the disconnection to practical pedagogical interventions limits the educational value of LA.The final and fifth emerging trend argues about the potential of LA built into LADs to support SRL and game-based learning.However, there is a need for a clearer connection between the design of LADs and what educational science asserts works for learning.
We are aware that our overview is limited by the fact that we only chose to look at systematic reviews of different quality and available data, raising the issue of heterogeneity when combining reviews with different conditions.Nonetheless, the authors believe that this overview is useful in giving an intuitive analysis of this emerging field of research.As time and development of LADs progresses, we will discover new trends.However, the authors believe that researchers and developers need to team up to collaboratively understand better the processes involved in learning and teaching practices and bring them into the design of LADs to expand the possibilities and value of analytics for practitioners in education [50,51].We offer a number of key areas for future research opportunities at the intersection of the fields of LA and education science:

•
Establish an international research agenda to test and develop LA and LADs in a cross-cultural and cross-language manner to maximize possible benefits.

•
Move the evaluation of LADs beyond functionality and usability aspects and assess the impact on the usefulness of LADs to increase understanding not only on outcomes but also learning processes.

•
Examine predictive analytics of LADs and the self-reflection that they elicit in learners which results in positive behavioral and cognitive adjustments, incorporate learning sciences into the design of LADs, and use systematic experimental research methods for learning at scale.

•
Examine and address possible biases towards different user groups and demographics in the design of LA and LADs.

•
Consider ethical aspects of educational and personal data protection such as privacy, security, and transparency.Develop national and international legislation methods for data collection and analysis.
The promise of LA and respective LADs seems to be "alive and kicking".Identified research types, methodologies, and trends in the area of MMLA, revealing an increasing interest in these technologies.The authors reviewed 57 studies and reviews and highlighted 14 topics under four themes-learning context, learning processes, systems and modality, and technologies-that can contribute to the development of MMLA.
[28] Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2023).Correction to: Artificial intelligence and learning analytics in teacher education: A systematic review Discussed the integration of artificial intelligence (AI) and LA in teacher education.They analyzed 30 studies.Key findings from this review reveal a concentration on examining the behaviors, perceptions, and digital competencies of pre-service and in-service teachers regarding AI and LA in their teaching practices.The primary types of data utilized in these studies include behavioral data, discourse data, and statistical data.Furthermore, machine learning algorithms are a common feature in most of these studies.However, only a few studies mention ethical considerations.
Reviewed empirical LA studies conducted in K-12 education with a focus on pedagogically innovative approaches involving technology-mediated learning, such as knowledge building and maker-centered learning.The review identified 22 articles with an emphasis on constructivist pedagogies.A majority (eight) of the included articles took place in the secondary school (grades 7-9, age 13-15).Four articles took place in primary school (grade 1-6) and four in high school.
Another four articles had a variety of age groups ranging between 10 and 18 years.Finally, two articles had teachers or pre-service teacher participants.
Analyzed the impact of LA research in the context of elementary level teaching.The review focused on highly cited articles retrieved from the Scopus database, with a total of 33 papers meeting the criteria.The analysis revealed that LA research in elementary education is relatively limited, especially in terms of highly cited studies.The research within this field is fragmented, with varying pedagogical goals and approaches.It was noted that LA is used both as a means for analyzing research data and as a tool to support students' learning.The review emphasized the importance of developing more robust theoretical foundations for LA in elementary school contexts.

Study-[Reference Number] Authors, Year and Title Summary
[31] Kew, S. N., & Tasir, Z. (2022).Learning analytics in online learning environment: A systematic review on the focuses and the types of student-related analytics data.
Presented a comprehensive overview of LA in online learning environments, drawing from a review of 34 articles published between 2012 and 2020.The review highlighted the key themes in LA research, such as monitoring/analysis, prediction/intervention, and the types of student-related analytics data commonly used.Additionally, it addressed challenges in LA, including data management, dissemination of results, and ethical considerations.

Figure 1 .
Figure 1.LA-related publications: (a) from 2010 to 2023; (b) in the first 30 countries with the highest number of publications.

Figure 1 .
Figure 1.LA-related publications: (a) from 2010 to 2023; (b) in the first 30 countries with the highest number of publications.

Table 1 .
Summary of the 22 systematic review articles, year, country of publication, number of articles reviewed, and citations (in ascending reference number order).

Table 1 .
Summary of the 22 systematic review articles, year, country of publication, number of articles reviewed, and citations (in ascending reference number order).

Table A1 .
Cont.Focused on the field of LA and its impact on educational interventions.It highlighted the growing interest in LA but noted the lack of empirical evidence on the effects of such interventions.The review included 52 papers.The findings indicated that student-facing dashboards are the most commonly employed LA-based intervention.Additionally, the review discussed the methodological aspects of these interventions, noting that data distillation for human judgment is the most prevalent approach.In terms of outcomes, the review looked at learning outcomes, user reflections, motivation, engagement, system usage behaviors, and teachers' performance monitoring.Examined the current research on LA and collaboration in online laboratory environments.A total of 21 articles between 2015 and 2021 were included.Half of the studies were within the higher education and medical field, but also lower level of education.There were a broad variety of lab platforms and LA was derived from students' log files.LA was used to measure performance, activities, perceptions, and behavior of students in virtual labs.The landscape was fragmented, and the research did not focus on learning and teaching.Standards and protocols are needed to address factors of collaboration, learning analytics, and online labs.Examined the role of LA in promoting inclusiveness for students with disabilities.A total of 26 articles were analyzed, and the results indicated that while LA began in 2011, discussions on inclusiveness in education started only in 2016.LA has the potential to foster inclusiveness by reducing discrimination and supporting marginalized groups.However, there are gaps in realizing this potential.Reviewed the use of LA within seamless learning environments, analyzing 27 journal articles.While seamless learning, which focuses on inquiry-based and experiential learning, has grown in popularity, the integration of LA to assess student progress in these environments has also increased.However, there's a gap in comprehensive reviews on this integration.Presented the latest evidence on the ethical aspects in relation to the use of LA in higher education.They reviewed 21 publications (16 journal articles and five proceedings), with a concentration of the research mostly in a few countries: the USA, Australia, and the UK.Those aspects concerned mainly the transparency, privacy, and informed consent that if not properly attended, according to the evidence, can prevent the development and implementation of LA.

Table A1 .
Cont.Discussed issues such as justice, equity, diversity, and inclusion in relation to LA dashboards research in higher education.The review included 45 relevant publications from journals and two conference proceedings, mostly from North America, Europe, and Australia.The authors identified four themes: participant identities and researcher positionality, surveillance concerns, implicit pedagogies, and software development resources.All were needed to avoid the risk of reinforcing inequities in education, according to the authors.The review also considered how to address those issues in future LA dashboards research and development.Explored the use of Serious Educational Games (SEGs) and the application of Game Learning Analytics (GLA) in formal education settings.The review included 80 relevant studies, and the key findings were: (1) GLA emerged as a powerful tool to assess and improve the usability of educational games.It has the potential to enhance education by improving learning outcomes, early detection of students at risk, increasing learner engagement, providing real-time feedback, and personalizing the learning experience).(2)Multidimensional taxonomy to categorize and understand different aspects of SEGs in formal education.