Predicting the Intention to Use Learning Analytics for Academic Advising in Higher Education

Abstract

Learning analytics (LA) is a rapidly growing educational technology with the potential to enhance teaching methods and boost student learning and achievement. Despite its potential, the adoption of LA remains limited within the education ecosystem, and users who do employ LA often struggle to engage with it effectively. As a result, this study developed and assessed a model for users’ intention to utilize LA dashboards. The model incorporates constructs from the “Unified Theory of Acceptance and Use of Technology”, supplemented with elements of personal innovativeness, information quality, and system quality. The study utilized exploratory research methodology and employed purposive sampling. Participants with prior experience in LA technologies were selected to take part in the study. Data were collected from 209 academic staff and university students in Malaysia (59.33% male) from four top Malaysian universities using various social networking platforms. The research employed “Partial Least Squares Structural Equation Modeling” to explore the interrelationships among the constructs within the model. The results revealed that information quality, social influence, performance expectancy, and system quality all positively impacted the intention to use LA. Additionally, personal innovativeness exhibited both direct and indirect positive impacts on the intention to use LA, mediated by performance expectancy. This study has the potential to offer valuable insights to educational institutions, policymakers, and service providers, assisting in the enhancement of LA adoption and usage. This study’s contributions extend beyond the present research and have the potential to positively impact the field of educational technology, paving the way for improved educational practices and outcomes through the thoughtful integration of LA tools. The incorporation of sustainability principles in the development and deployment of LA tools can significantly heighten their effectiveness, drive user adoption, and ultimately nurture sustainable educational practices and outcomes.

1. Introduction

Recent technological developments have presented a chance to collect and monitor students’ learning patterns in digital environments, storing them as extensive sets of data [1]. Therefore, through learning analytics (LA), higher education in the twenty-first century persists in promoting discoveries in the field [2]. LA is commonly defined as measuring, storing, analyzing, and reporting data on students’ progress as well as the environments in which they learn, aiming to better comprehend and improve both the learning environment and learning itself [3]. Research has revealed several advantages of LA, such as the provision of customized course offerings, superior learning outcomes, curriculum design, teaching execution, and greater post-educational employment options [4].

Utilizing data gathered from learners’ interactions with a “Learning Management System” (LMS) to make predictions about the factors that contribute to increased student retention is one example of learning analytics. According to the findings of [5], students who are actively engaged by participating in more discussions, sending more messages, and completing more assessments tend to achieve greater overall grades in the course. The results highlight the potential role that learning analytics can play in guiding students’ academic success and growth [6].

Today’s instructors need a learning system that offers thorough and instructive LA for their online courses [7]. By accessing the LA of students’ lesson completion status and quiz results, educators may better understand students’ ability to follow and grasp the course contents, the themes they found problematic, their social relationships, and their knowledge gains [8]. LA has shown that it can detect learners’ unique requirements and learning challenges in addition to predicting student success. This knowledge might be used to build flexible educational structures with tailored directions for specific students [9]. Thus, when properly applied, LA could result in greater accountability at all educational levels.

The early promise of LA to enhance learning and its settings has not been fully realized [10]. There is currently little evidence demonstrating how LA services affect outcomes of student learning, processes of instruction and learning, as well as institutional decision-making [11]. Although numerous tools and methods for LA have been presented, there is little empirical evidence of the factors impacting this new technology’s potential adoption [8]. To handle the difficulties of LA adoption, previous researchers have developed numerous instruments and frameworks to guide the implementation of LA technologies in Australia, Europe, and North America [3]. Although those studies have enabled academics to pinpoint crucial factors that generally impact the LA services’ acceptance, there are still several complicated silos, competing leadership agendas, and institution-specific problems [12]. There have been few efforts to develop theoretical models and evaluate the variables determining the intention to utilize LA technologies [13].

The integration of learning analytics (LA) in the educational system is presently limited, and users often face challenges in effectively utilizing LA technologies [14]. This highlights a critical research problem: understanding and improving users‘ willingness to use LA dashboards, ultimately enabling smooth integration of LA tools in educational environments [15]. Learning analytics has substantial potential to revolutionize teaching methods and enhance student learning and success [16]. However, their underutilization and ineffective engagement hinder the realization of these advantages [17]. Addressing this research problem is crucial for optimizing the incorporation of LA tools, thereby enhancing educational methods and results. Hence, this study aimed to create and validate a model for users‘ intention to use LA dashboards by integrating elements from the “Unified Theory of Acceptance and Use of Technology” (UTAUT), supplemented with aspects of “system quality” (SQ), “personal innovativeness” (PI), and “information quality” (IQ). The study hypothesized that integrating these additional variables, along with the UTAUT construct, would positively influence the intention to use LA among academic staff and university students in four leading Malaysian universities.

This study makes a significant contribution by elucidating crucial factors that influence users‘ intention to utilize LA, including information quality, social influence, performance expectancy, and system quality. Furthermore, it emphasizes the direct and indirect positive effects of personal innovativeness on the intention to use LA, mediated by performance expectancy. These findings offer valuable insights for educational institutions, policymakers, and service providers to enhance LA adoption and usage. The study‘s contribution extends beyond the current research, positively impacting the educational technology field. By advocating for the integration of sustainability principles in the development and implementation of LA tools, it suggests a path to enhance effectiveness, drive user adoption, and ultimately foster sustainable educational practices and outcomes.

The paper is organized as follows: Section 2 provides a review of related works, while Section 3 presents the conceptual framework and hypotheses. In Section 4, we delve into the research methodology, and Section 5 covers the results of the data analysis. Section 6 discusses the key findings of the study, including theoretical and managerial implications. Finally, in Section 7, we conclude the study, address limitations, and consider future avenues of research.

2. Literature Review

2.1. Learning Analytics Dashboards

Technology has evolved into a crucial tool that helps teachers and students create more effective learning environments over the past few years. The proliferation of online learning environments has substantially increased, expanding the amount of data created about the educational process [18]. Mitchell and Costello coined the term “Analytics of Learning” in 2000, presenting it as an emerging concept in their analysis of the potential prospects in the global market for the development and implementation of educational services via the network [19].

Stakeholders in higher education are increasingly employing “Learning Analytics” (LA) dashboards for various purposes, such as tools for learners to assess their progress [20], tools for administrators to support and control instructors, students, and staff [21], and tools for faculty to assess students’ performance and provide feedback on their teaching exercises [22].

LA is a field that focuses on gathering, analyzing, and sharing data related to learning environments and learners, evolved to realize the promise of this data analysis. LA is intended to address retention concerns and preserve the effectiveness of academic achievement [23]. LA was shown to be the most effective tool for growing awareness of the importance of bringing the ideas of information technologies and education closer together in the context of advancing higher education and, most significantly, learning professional and personal development based on the theoretical assumption of [5].

Numerous frameworks have been suggested to facilitate the adoption of LA and address the challenges it encounters. There are, however, few empirical studies on the variables impacting the potential adoption of LA technologies [8]. Klein et al. conducted qualitative research to comprehend organizational challenges, rewards, and opportunities relating to faculty and professional advising staff usage of LA techniques [24]. The results indicate that both organizational commitment and the organizational context, encompassing structures, processes, leadership, and policies, influence individuals’ choices to utilize and place trust in LA technologies. A study was performed [3] in European higher education organizations by interviewing institutional leaders. They identified context, challenges, strategy, and people as factors of LA adoption. Results suggested routine assessments of LA adoption to guarantee the desired changes and alignment of strategies.

Dawson et al. utilized “complexity leadership theory (CLT)” to identify the interaction between key dimensions of LA adoption. Interviews were conducted in Australian universities, and results suggested that to advance from the small-scale course stages to a more integrated and comprehensive organizational level, research on LA adoption models needs to extend [5]. Based on the UTAUT, Herodotou et al. analyzed the involvement patterns of university teachers using LA dashboards through in-depth interviews [25]. Results revealed that “social influence” (SI), “performance expectancy” (PE), and “effort expectancy” (EE) were among the elements promoting engagement with PLA. PE-facilitated conditions (FC) and a lack of knowledge of predictive data were factors that prevented PLA involvement. Based on the “technology acceptance model” (TAM), Ali et al. provided a model of the variables impacting instructors’ opinions on the use of LA tools [8]. The usage beliefs regarding an LA tool are related to the intention to employ the technology, according to the model. The study identified analytics categories that are the main interpreters of “perceived ease of use” and “perceived usefulness’ (interactive visualization).

Malaysia is known as one of the largest educational providers in Asia. The Malaysian government has consistently created efficient policies to improve the educational system. The newest but most crucial measure to enhance the educational system is thought to be the implementation of LA [23]. Malaysia is just beginning to investigate LA’s possibilities to aid student retention. The “Ministry of Higher Education” (MOHE) intends to emphasize LA to incorporate the learning and teaching transformation in higher education institutions, shifting the emphasis from retention to better satisfy the present changes in the industry known as the “fourth industrial revolution” (IR 4.0) [26]. To follow the IR 4.0 revolution, the MOHE advised a focus on four key areas: reforming learning classrooms, integrating 21st-century teaching methods, utilizing a flexible curriculum to address new developments and fields of knowledge, and utilizing the most recent teaching and learning technologies [27]. Studies confirmed academics’ high interest in utilizing LA for learning and teaching and its positive role in learners’ performance in Malaysian higher learning institutions [23,26]. Zaki et al. proposed an LA conceptual model in serious games for education in Malaysia [28]. Ismail et al. identified the LA as a powerful technique for exploring the data generated from the LMS and highlighted the likelihood of handling the LA technique with the LMS engagement in educational institutions [29]. However, LA implementation in Malaysia is fraught with difficulties. As a vital tool for the management and operation of educational institutions in Malaysia, LA is still not frequently utilized [27]. There is a lack of theoretical models that examine the determinants of LA tools’ usage in higher education in Malaysia. Therefore, the objective of this research was to construct a conceptual framework addressing the question: “What factors influence users’ intention to utilize LA dashboards within Malaysian higher education institutions?”.

2.2. Applications of LADs in Educational Settings

“Learning Analytics Dashboards” (LADs) have evolved in sync with the advancements in educational technology and the growing abundance of educational data. While the roots of learning analytics can be traced back to the early 2000s, progress in data analytics, visualization techniques, and digital learning platforms has significantly impacted the creation and usage of LADs in higher education [30].

Learning analytics utilize methodologies from data science to analyze data and present the resulting analysis using diverse textual and visual approaches [31]. In the domain of LA, dashboards have gained significant attention as tools that can provide users with relevant insights, encourage self-reflection, and potentially guide interventions to optimize learning and improve the quality of the student experience [32]. As defined by Schwendimann et al., LADs are described as “a unified display that consolidates various indicators about the learner(s), learning process(es), and/or learning context(s) into one or multiple visual representations” [33] (p. 8).

These dashboards aggregate and synthesize diverse educational data, offering insights into student performance, engagement, behavior, and learning patterns [34]. LADs are designed to assist educators, administrators, and students in making informed decisions, optimizing teaching and learning strategies, and improving overall educational outcomes [35].

LADs provide a real-time view of individual and group academic performance [36]. Consequently, educators can monitor student progress, identify areas for improvement, and tailor instructional strategies accordingly. LADs analyze student data to craft personalized learning pathways based on individual weaknesses, strengths, and learning styles [37]. This facilitates a customized learning experience, enhancing student engagement and comprehension [38].

LADs can detect early signs of academic struggles or disengagement, enabling timely intervention and support for at-risk students [39]. This proactive approach can boost student retention rates [39]. Educators can utilize LADs to evaluate the effectiveness of courses and curricula [32]. Insights from the dashboards can guide adjustments to course content, assessments, and instructional strategies for better learning outcomes.

LADs can employ predictive models to forecast student success, aiding institutions in identifying students who may require additional support [40]. They evaluate student engagement and motivation levels using data on participation, interactions, and feedback [41]. This information empowers educators to implement strategies that enhance engagement and motivation [42]. Institutions can optimize resource allocation, including faculty time and support services, based on LAD insights [43]. This data-driven approach facilitates efficient planning and allocation of educational resources [33].

In summary, LADs are pivotal in contemporary higher education as they harness the power of data and analytics to improve teaching, enrich learning experiences, and promote student achievement [44]. They offer a multifaceted view of educational data, enabling stakeholders to make data-informed decisions and nurture a more effective and efficient educational ecosystem [42]. Integrating learning analytics into academic advising not only enhances the advising process but also contributes to student success and retention [39]. By harnessing the power of data analytics, academic advisors can provide timely, personalized guidance, ultimately supporting self-regulated learning [38].

2.3. Research Gap

In a thorough examination of existing literature, researchers [45] identified critical factors predicting the adoption of “Learning Analytics” (LA) within higher education institutions (HEIs). These factors encompassed user support, effective communication with users, comprehensive end-user training, and the establishment of standards for LA tools. Another systematic review focused on enhancing students’ learning performance through improved engagement in learning, encompassing cognitive, behavioral, and emotional dimensions [46].

A separate study [47] explored trust’s influence on the integration of LA within higher education. The results showed that educators displayed substantial trust in the competence of HEIs and the effectiveness of LA. However, their trust in technology vendors concerning privacy and ethical aspects was relatively diminished. Another investigation [48] aimed to grasp students’ privacy anxieties regarding the effective deployment of LA tools in HEIs. This study revealed aspects that heightened students’ concerns regarding the gathering, utilization, and disclosure of the data for learning analytics.

Furthermore, a study [49] investigated differences in teachers’ utilization of LA and identified that early adopters, classified as “innovators”, displayed significantly higher engagement during remote education weeks compared to their counterparts. This underscored the critical role of personal innovativeness in LA tool adoption. Additionally, recent research [50] examined LA adoption in higher education and emphasized “perceived usefulness” and “ease of use” as significant determinants.

Another study [51] explored the effect of a comprehensive LA approach on learning performance in online collaborative learning. The outcomes indicated a significant enhancement in group performance, collaborative knowledge building, metacognitive learning engagement, social interaction, and coregulated behaviors compared to traditional online collaborative learning. Moreover, research [52] evaluated the effectiveness and utility of an LA dashboard in HEIs, demonstrating its efficacy in aiding students in informed decision-making regarding their learning approach.

Integrating AI-based performance prediction models with LA methodologies, a study [53] endeavored to enhance student learning outcomes within a collaborative learning environment. The integrated approach exhibited notable improvements in collaborative learning performance, increased student engagement, and heightened student satisfaction with the learning process. Additionally, research [54] investigated how students utilize “Learning Analytics Dashboards” (LADs) in higher education. The findings highlighted that engagement with LADs supported students’ metacognitive and time management strategies during learning, particularly during out-of-class learning sessions, with greater benefits observed for higher-performing students.

In previous research, the “Technology Acceptance Model” (TAM) and the “Unified Theory of Acceptance and Use of Technology” (UTAUT) were predominantly employed to elucidate “Learning Analytics” (LA) adoption within “Higher Education Institutions” (HEIs).

Table 1. Studies on LA Adoption in HEIs.

Source	Model	Key Findings
[55]	TAM	PU and PEOU were positively associated with training satisfaction.
[56]	TAM	PU, PI, and PEOU were positively associated with BI.
[7]	UTAUT	Use of PLA was positively associated with PE, less experience, self-efficacy, attitudes, and low anxiety, while associated with lack of FC and low EE.
[57]	TAM	Teachers’ perceived usefulness was positively associated with attitudes toward use.
[58]	UTAUT	PE, EE, and SI were positively associated with BI.
[8]	TAM	PU and PEOU were positively associated with BI.

succinctly presents the key findings from studies on LA adoption within HEIs. These findings consistently demonstrate robust correlations among factors such as “Perceived Usefulness” (PU), “Perceived Ease of Use” (PEOU), “Behavioral Intention” (BI), “Performance Expectancy” (PE), “Effort Expectancy” (EE), “Social Influence” (SI), and others. These factors are shown to have positive or negative associations with one another or with specific outcomes, including training satisfaction and attitudes toward use.

While the literature review offers valuable insights into the essential factors that influence the adoption of “Learning Analytics” (LA) and its influence on student learning, a gap persists in achieving a comprehensive understanding of the interconnections among these factors and their collective impact on student learning. This study took a deeper dive into exploring how information quality, system quality, and personal innovativeness converge to influence LA adoption. To fill this research gap and gain a more holistic perspective, further exploration of the viewpoints and experiences of both educators and students concerning LA adoption is warranted, providing a more encompassing view of this subject.

3. Theoretical Background

3.1. Conceptual Framework

One of the frequently cited theories in the literature on technology acceptance and information systems (IS) is the UTAUT [59]. The UTAUT has been repeatedly replicated with success and applied in research on a wide range of technologies and even circumstances outside of adoption [60,61]. The complete range of innovation adoption, from initial adoption to post-adoption use, has been studied using UTAUT [61]. UTAUT was confirmed to surpass earlier adoption theories [59]. Moreover, UTAUT stood out as the most employed model in a recent literature review performed to investigate technology acceptance models from 2010 to 2020 [62]. UTAUT is a comprehensive and successful theory examining the adoption and use of different technologies in the education sector [63]. Hence, UTAUT was adopted as the foundation of the current research. UTAUT consists of “performance expectancy” (PE), “effort expectancy” (EE), “social influence” (SI), and “facilitating condition” (FC) constructs. In this study, UTAUT has been employed to assess the intentions to use LA (IULA). Additionally, the UTAUT’s basic constructs, “personal innovativeness” (PI), “information quality” (IQ), and “system quality” (SQ) constructs, were included in the model as they were recommended to be significant for technology acceptance in educational environments [64,65,66]. Figure 1 shows the developed conceptual framework in this study.

The study employed UTAUT as a theoretical framework to understand users’ intention to utilize LA dashboards. UTAUT is a well-established model that integrates several key constructs such as “performance expectancy”, “effort expectancy”, “social influence”, and “facilitating conditions”, providing a comprehensive understanding of user acceptance and use of technology. The decision to use UTAUT in this study is justified by its widespread recognition and adoption in technology acceptance research [67].

Additionally, the study integrated specific variables from the DeLone and McLean IS Success Model, “information quality” (IQ) and “system quality” (SQ)—as well as the “personal innovativeness” (PI) construct. “Information quality” (IQ) assesses the information quality provided by the system, while “system quality” (SQ) evaluates the overall system attributes and capabilities [68]. “Personal innovativeness” (PI) measures an individual’s openness to try out novel technologies and innovations [69].

SQ and IQ are widely acknowledged as pivotal factors influencing the acceptance and utilization of information systems, including learning analytics platforms [70]. SQ encompasses the overall performance and reliability of the system, while IQ encompasses the accuracy, relevance, and completeness of the information provided by the system [71]. These two factors are crucial in determining the effectiveness of learning analytics from the users’ perspective. PI is also considered a relevant variable due to its connection with technology adoption. It signifies an individual’s willingness to embrace and utilize new technologies, making it highly relevant in the context of learning analytics adoption [56]. Prior research, including studies utilizing the “Diffusion of Innovations theory,” underscores the significance of PI as a determinant of technology acceptance and adoption [72].

The integration of these variables from the D&M “IS Success Model” is justified by their relevance to the context of LA adoption and usage. “Information quality” (IQ) and “system quality” (SQ) are critical factors influencing users’ perception and acceptance of LA dashboards, aligning with the study’s objective to investigate factors impacting LA adoption. “Personal innovativeness” (PI) complements UTAUT constructs by assessing the individual’s openness to adopting recent technologies, which is crucial in the context of rapidly evolving educational technologies like LA.

3.2. Hypothesis Development

The term “performance expectation” (PE) refers to “the degree to which an individual believes that utilizing the system will assist them in achieving improvements in job performance” [59]. The degree to which using the system is simple and uncomplicated is referred to as “effort expectancy” (EE) [59]. Handoko confirmed the significant influence of PE and EE on students’ perceptions of technology adoption [73]. The positive impact of PE on teachers’ adoption of MOOCs has been supported in [74]. With UTAUT, the objective of [25] was to identify the factors predicting university teachers’ engagement with “predictive learning analytics” (PLA). The study’s findings indicated that key facilitators for engagement with PLA included PE and EE. Therefore, in this study, we expect the following:

H1: 

PE positively influences IULA.

H2: 

EE positively influences IULA.

The term “social influence” (SI) refers to “the extent to which an individual believes that significant others believe they should use the new system” [59]. Handoko found a positive effect of SI on students’ behavioral intentions to use technology [73]. Abd Rahman et al. confirmed the positively significant effect of SI on the intent to utilize flipped learning among learners [75]. Herodotou et al. discovered that social influence significantly and positively impacts university teachers’ engagement with “predictive learning analytics” (PLA) [25]. In this study, users’ desire to learn and adopt LA is influenced by the opinions and actions of their colleagues and peers. When they observe their peers using LA, they are more likely to use it themselves.

H3: 

SI positively influences IULA.

The term “facilitating conditions” (FC) refers to “the level of confidence an individual has in the existence of an organizational and technical infrastructure to support system usage” [59]. According to UTAUT, the FC factor significantly impacts the ultimate acceptance and use of an innovation [25]. The influence of FC on students’ and educators’ behavior toward technology adoption has been confirmed in [63,74]. Lecturers and students are expected to be more willing to use LA for educational purposes if sufficient technical and organizational infrastructure is provided to support them.

H4: 

FC positively influences IULA.

In the field of general diffusion of innovations research, it has long been widely accepted that highly innovative individuals are active information seekers about new concepts (Lu et al., 2005). They can manage a great deal of ambiguity and develop more positive attitudes towards acceptance. “Personal innovativeness” (PI) denotes a proclivity for experimenting with cutting-edge information technology, showcasing a positive correlation with the utilization and adoption of emerging technologies [67,76,77]. According to Blut et al., PI has a strong impact on technology usage [60]. Previous studies have indicated that PI positively impacts the intentions to use m-learning applications [56,76,78], cryptocurrencies [79], and cloud computing systems [80]. Moreover, recent findings confirmed the effects of PI on PE and EE related to e-learning and e-book adoption by students [81,82]. Therefore, in this study, we hypothesize the following:

H5: 

PI positively influences IULA.

H6: 

PI positively influences PE.

H7: 

PI positively influences EE.

The quality of the content and information offered on the online platform is evaluated as “information quality” (IQ). The provided information inevitably influences user satisfaction and intentions to use [66]. In this study, IQ assesses the quality of the information generated with LA tools and its effectiveness for users. “System quality” (SQ) refers to the extent of a system’s availability, speed of feedback, user-friendliness, and screen features (interface)—all indicators of its usability—influencing the user’s behavioral intentions to use the innovation [68]. E-learning studies have shown that IQ and SQ have a substantial positive effect on users’ intentions to use [64,65]. Both information and system quality have a positive and significant effect on both usage and user satisfaction [83]. Thus, in this study, we hypothesize the following:

H8: 

IQ positively influences IULA.

H9: 

SQ positively influences IULA.

4. Research Method

4.1. Procedure and Participants

This study was conducted at four top Malaysian universities using purposive sampling. The universities were specifically chosen due to their status as comprehensive institutions with a large population of students in blended and online courses. They were also among the top 200 universities in the world according to international rankings. The survey targeted academic staff (lecturers or tutors) as well as students. The choice of academic staff (lecturers or tutors) and students as respondents in this study stems from the study’s focus on “learning analytics” (LA) and its potential to improve teaching methods and student learning outcomes. Academic staff and students are primary users of LA tools, making their perspectives and intentions crucial to understanding the effective implementation of LA dashboards. The inclusion of both groups allows for a comprehensive exploration of the factors influencing LA usage and adoption, providing diverse perspectives from teaching and learning angles.

Respondents were encouraged to participate if they had experience using LA technologies for educational purposes. The questionnaire was administered online using various social media platforms such as Facebook and email. Over 3 months, a total of 209 complete questionnaires were collected and utilized for the data analysis in the present study. Given the exclusion of 120 participants without prior experience with LA from the main analysis, the overall response rate for the study was 63.5%.

The sample size for this study was established following the rule of thumb proposed in [84], which recommends that the minimum sample size should be 10 times larger than the number of structural paths in the structural model directed at a specific construct. As there were a maximum of seven structural paths directing LA adoption in this study, a minimum sample size of 70 respondents would be needed. Therefore, the sample size of 209 exceeded the recommended minimum and is highly adequate to validate the developed model.

Table 2. Respondents’ personal information.

Information of Respondents		Frequency (Percentage)
Gender	Female	85 (40.67%)
	Male	124 (59.33%)
Current position	Lecturer	30 (14.35%)
	Student	179 (85.65%)
Age	Below 20	35 (16.75%)
	21–30 years old	89 (42.58%)
	Above 30	85 (40.67%)
Technology skills	Not bad	9 (4.31%)
	Sufficient	170 (81.34%)
	Very good	30 (14.35%)

displays that the majority of the respondents were male (59.33%). Nearly three-quarters of the respondents were students (85.65%). Furthermore, most respondents (81.34%) believed they had sufficient skills in using technology.

4.2. Research Instruments

In this study, the research instrument was segmented into two main sections. The initial segment gathered personal information from the respondents. The second section comprised questions rated on a 5-point Likert-type scale, ranging from “strongly disagree = 1” to “strongly agree = 5”, for assessing the influence of EE, PE, SI, FC, PI, IQ, and SQ constructs on IULA. Questions related to EE, PE, SI, FC, and intention to use LA were adopted from [59]. For assessing PI, four questions were adopted from Agarwal and Prasad (1998). Four IQ items were adopted from [64,85,86], while three SQ items were adopted from [66,87] (see Appendix A).

Content validity was ensured by inviting four experts to confirm the simplicity and relevance of the scale items. Furthermore, a pilot study involving 70 respondents was carried out to evaluate the validity and reliability of the survey instrument. After incorporating suggested rewordings and modifications, the instrument was confirmed for use in large-scale data collection for the main study.

4.3. Research Design and Data Analysis

The cross-sectional study utilized an exploratory research design. The statistical method known as “Partial Least Squares Structural Equation Modeling” (PLS-SEM) was utilized to assess and estimate relationships using causal hypotheses and statistical data [88,89]. Since it is a powerful statistical approach to assess complex relationships, PLS-SEM is extensively employed in numerous social science fields, such as “Management Information Systems” [90], hospitality [91], and strategic management [92]. This method reconciles the apparent opposition between explanation and prediction, which forms the foundations for creating managerial implications, a crucial aspect often emphasized in academic research [93]. PLS-SEM’s statistical power is particularly valuable for exploratory research that delves into less-developed or still-emerging theories [88]. Therefore, PLS-SEM was selected for the data analysis in this study. SmartPLS 4 was the software used to run PLS-SEM. The following subsections first assessed the measurement model, and then the acceptance/rejection of hypotheses was evaluated in the structural model.

5. Results

5.1. Common Method Bias

Since self-reported data were utilized in the data collection process, there is a potential risk of encountering “common-method bias”. To address this concern, a “Harman’s one-factor test”, following the approach proposed in [94], was conducted on the factors included in the theoretical model. The findings revealed that the initial factor accounted for the largest share of covariance at 22.7%, falling below the recommended threshold of 50%. This result indicates that the risk of “common-method bias” in the present study is not significant. Moreover, the “Variance Inflation Factor” (VIF) was computed to evaluate the extent of multicollinearity among the predictors employed in the regression model. The results demonstrate that the VIF values for the majority of predictors are in proximity to 1, signifying low multicollinearity. Nevertheless, the VIFs for the independent variables PI, PE, and EE exceed the recommended threshold. Due to the relatively small sample size, it is plausible that these elevated VIFs are a consequence of this factor.

5.2. Evaluation of the Measurement Model

The measurement model was evaluated using the following criteria: 1. Indicator reliability, evaluated through outer loading values. 2. Construct reliability, evaluated through “Composite Reliability” (CR) and “Cronbach’s alpha” (CA). 3. Discriminant validity, examined using “Average Variance Extracted” (AVE), Fornell–Larcker Criterion, cross-loadings, and “Heterotrait–Monotrait” Ratio (HTMT) [84].

Under outer loading, the robustness of the association between each indicator and its items is assessed. Based on

Table 3. Validity and Reliability.

Constructs	Items	Factor Load	CA	rho_A	CR	AVE
EE	EE1	0.684	0.768	0.801	0.849	0.585
	EE2	0.825
	EE3	0.802
	EE4	0.741
FC	FC1	0.793	0.863	0.870	0.907	0.710
	FC2	0.882
	FC3	0.882
	FC4	0.808
IQ	IQ1	0.890	0.909	0.909	0.936	0.785
	IQ2	0.898
	IQ3	0.883
	IQ4	0.873
PE	PE1	0.849	0.877	0.880	0.916	0.731
	PE2	0.839
	PE3	0.895
	PE4	0.837
PI	PI1	0.897	0.857	0.861	0.904	0.703
	PI2	0.776
	PI3	0.772
	PI4	0.899
SI	SI1	0.876	0.880	0.883	0.917	0.735
	SI2	0.875
	SI3	0.821
	SI4	0.855
SQ	SQ1	0.867	0.781	0.785	0.873	0.697
	SQ2	0.868
	SQ3	0.765
IULA	IULA1	0.909	0.920	0.922	0.943	0.806
	IULA2	0.897
	IULA3	0.924
	IULA4	0.860

, the factor loadings ranged between 0.684 and 0.924, affirming strong construct validity [87]. For construct reliability, the values of CR and CA must exceed 0.7 to ensure reliability. In this study, construct reliability was confirmed, with CR values ranging between 0.849 and 0.943 and CA coefficients ranging from 0.768 to 0.920, all falling within the acceptable range. AVE was examined to establish convergent validity. As all constructs demonstrated values above 0.50, convergent validity was affirmed [95].

Discriminant validity ensures that no two constructs are highly correlated. In the model, any pair of constructs should have correlations that are lower than √AVE values. As depicted in

Table 4. Fornell–Larcker Criterion.

	EE	FC	IQ	IULA	PE	PI	SI	SQ
EE	0.765
FC	0.369	0.842
IQ	0.548	0.452	0.886
IULA	0.377	0.481	0.534	0.898
PE	0.459	0.515	0.530	0.524	0.855
PI	0.389	0.267	0.375	0.417	0.423	0.838
SI	0.374	0.506	0.447	0.480	0.460	0.265	0.857
SQ	0.349	0.483	0.390	0.456	0.426	0.323	0.399	0.835

, the √AVE values (diagonal in bold) for each factor are higher than the maximum correlation with any other factor in both the row and column [95]. Additionally, as demonstrated in

Table 5. Cross-Loadings.

	EE	FC	IQ	IULA	PE	PI	SI	SQ
EE1	0.684	0.311	0.415	0.180	0.226	0.226	0.224	0.237
EE2	0.825	0.349	0.483	0.388	0.435	0.332	0.336	0.285
EE3	0.802	0.301	0.448	0.305	0.393	0.357	0.286	0.307
EE4	0.741	0.147	0.312	0.224	0.297	0.243	0.284	0.225
FC1	0.336	0.793	0.439	0.407	0.487	0.246	0.513	0.406
FC2	0.296	0.882	0.399	0.418	0.424	0.250	0.369	0.375
FC3	0.338	0.882	0.420	0.441	0.458	0.236	0.406	0.461
FC4	0.267	0.808	0.244	0.344	0.354	0.155	0.421	0.379
IQ1	0.443	0.396	0.890	0.455	0.455	0.334	0.383	0.306
IQ2	0.493	0.401	0.898	0.459	0.478	0.307	0.394	0.336
IQ3	0.532	0.430	0.883	0.495	0.514	0.397	0.429	0.414
IQ4	0.471	0.374	0.873	0.482	0.431	0.289	0.374	0.320
IULA1	0.354	0.448	0.483	0.909	0.498	0.410	0.447	0.494
IULA2	0.312	0.377	0.440	0.897	0.411	0.392	0.417	0.369
IULA3	0.307	0.466	0.492	0.924	0.478	0.345	0.451	0.404
IULA4	0.379	0.431	0.502	0.860	0.490	0.351	0.405	0.361
PE1	0.404	0.457	0.523	0.460	0.849	0.389	0.426	0.368
PE2	0.338	0.371	0.362	0.449	0.839	0.331	0.376	0.321
PE3	0.440	0.447	0.502	0.470	0.895	0.387	0.403	0.365
PE4	0.386	0.488	0.416	0.411	0.837	0.336	0.365	0.407
PI1	0.344	0.209	0.305	0.400	0.339	0.899	0.226	0.244
PI2	0.337	0.215	0.311	0.400	0.341	0.897	0.227	0.249
PI3	0.267	0.209	0.315	0.320	0.380	0.776	0.195	0.269
PI4	0.355	0.263	0.329	0.270	0.362	0.772	0.239	0.325
SI1	0.303	0.465	0.383	0.442	0.394	0.280	0.876	0.370
SI2	0.305	0.414	0.330	0.400	0.358	0.187	0.875	0.326
SI3	0.315	0.364	0.420	0.375	0.343	0.199	0.821	0.279
SI4	0.361	0.482	0.400	0.423	0.476	0.234	0.855	0.385
SQ1	0.289	0.429	0.320	0.391	0.377	0.219	0.396	0.867
SQ2	0.271	0.333	0.335	0.391	0.346	0.303	0.370	0.868
SQ3	0.315	0.452	0.321	0.359	0.345	0.287	0.225	0.765

, all elements of a construct exhibit higher factor loadings than the related cross-loadings in both rows and columns.

The “Heterotrait–Monotrait Ratio” (HTMT) was employed, as proposed in [96], to validate “discriminant validity.” It assumes that the HTMT conservative criterion should be below the threshold value of 0.85. As presented in

Table 6. HTMT.

	EE	FC	IQ	IULA	PE	PI	SI	SQ
EE
FC	0.440
IQ	0.645	0.503
IULA	0.425	0.535	0.583
PE	0.535	0.589	0.590	0.581
PI	0.465	0.308	0.426	0.468	0.489
SI	0.448	0.580	0.500	0.531	0.521	0.303
SQ	0.446	0.590	0.462	0.536	0.517	0.399	0.476

, the HTMT values are all less than 0.85. The results affirm the discriminant validity of the model.

5.3. Estimation of the Structural Model

In this step, the significance of the p-values and t-values for each hypothesized path was evaluated. Bootstrapping was conducted for this purpose. The model’s ability to predict the endogenous constructs was evaluated using R² to determine its explanatory power. The results are detailed in

Table 7. The outcome of the hypotheses testing.

	Mean	STDEV	t-Value	p-Value	VIF	Result
PE → IULA	0.156	0.073	2.151	0.032	1.0437	Supported
EE → IULA	−0.037	0.073	0.502	0.615	1.0005	Rejected
SI → IULA	0.160	0.068	2.349	0.019	1.0589	Supported
FC → IULA	0.118	0.065	1.819	0.069	1.0300	Rejected
PI → IULA	0.159	0.059	2.698	0.007	1.0789	Supported
PI → PE	0.423	0.063	6.679	0.000	5.0947	Supported
PI → EE	0.389	0.063	6.223	0.000	5.0140	Supported
IQ → IULA	0.233	0.065	3.590	0.000	1.2365	Supported
SQ → IULA	0.139	0.068	2.042	0.041	1.0513	Supported

and Figure 2.

PE (β = 0.158, p-value = 0.032), SI (β = 0.157, p-value = 0.019), PI (β = 0.160, p-value = 0.007), IQ (β = 0.232, p-value = 0.000), and SQ (β = 0.139, p-value = 0.041) were found to have a positively significant effect on IULA, providing support for H1, H3, H5, H8, and H9, respectively. However, EE (β = −0.032, p-value = 0.615) and FC (β = 0.118, p-value = 0.069) were not found to have a significant impact on IULA, not providing support for H2 and H4. PI showed a positive influence on EE (β = 0.395, p-value = 0.000) and PE (β = 0.428, p-value = 0.000), supporting H7 and H6.

In summary, the developed model explained 45.6% of the variance in IULA, 15.2% of the variance in EE, and 17.9% variance in PE. Model fit was confirmed by evaluating the “standardized root mean square residual” (SRMR) with a value of 0.062, falling below the threshold of 0.08. This indicates a good fit for the developed model in the study [97].

6. Discussion

6.1. Key Findings

The findings indicated that users’ perception of performance expectancy, social influence, personal innovativeness, system quality, and information quality was positively linked to their intention to use “learning analytics” (LA). This affirms that when users perceive that utilizing LA will enhance their performance and are influenced by social factors, they are more inclined to adopt these tools. Previous research on LA use in higher education underscored the critical role of perceived usefulness as a primary determinant [50]. Additionally, [98] found that “perceived usefulness” significantly impacts the success of LA dashboards. Furthermore, both “perceived ease of use” and “perceived usefulness” showed positive correlations with satisfaction regarding LA dashboards [55].

The study highlighted the pivotal role of system quality and information quality in positively influencing users’ intention to utilize learning analytics (LA) dashboards. This observation is supported by [83], which illustrated that quality aspects such as technical systems, educational systems, instructors, support service, and course content quality exert a direct positive effect on students’ satisfaction, perceived usefulness, and the use of e-learning systems in higher education. Likewise, a different study found evidence supporting a positive association between the service quality of e-learning analytics and the system’s ease of use [99].

Unexpectedly, variables such as facilitating conditions and effort expectancy did not demonstrate a notable impact on users’ intention to use LA dashboards. This result may be attributed to the distinctive characteristics and technological familiarity of the sampled academic staff and university students in Malaysia. It is plausible that their prior experiences and comfort with technology have diminished the perceived effort typically associated with adopting new technological tools. Additionally, the provision of sufficient training and support to participants may likely equip them with the necessary knowledge and skills to navigate LA dashboards effectively. These factors collectively might have mitigated the anticipated impact of facilitating conditions and effort expectancy on their intention to use LA dashboards.

The study also revealed a strong positive impact of personal innovativeness on both effort expectancy and performance expectancy, highlighting the role of individuals’ willingness to embrace innovation in shaping their engagement with LA. Likewise, a study [49] delved into variations in teachers’ utilization of LA, revealing that those identified as early adopters or “innovators” demonstrated notably heightened engagement during remote education weeks in comparison to their peers. This highlights the pivotal role of personal innovativeness in the adoption of LA tools. Overall, the findings provide crucial insights for educational institutions and stakeholders, emphasizing the significance of improving information quality, and system quality, and fostering a culture of innovation to drive effective adoption and utilization of LA tools in educational settings.

6.2. Managerial Implications

This study contributes valuable insights for academic instructors, LA tool developers, and administrators to understand crucial determinants when implementing LA tools in their institutions. Understanding the key factors that impact users’ behavioral intention to use LA tools is essential for the successful adoption and utilization of these tools in educational settings.

The significant positive effects of PE, SI, PI, IQ, and SQ on IULA emphasize the importance of these factors in driving users’ intention to use LA tools. LA tool developers should focus on enhancing these aspects to increase users’ willingness to adopt and use their tools. Specifically, the study highlights the importance of automation in analytical tool design to reduce human involvement and streamline decision-making processes, enhancing the effectiveness of LA tools.

Moreover, the findings underscore the significance of providing reliable, secure, user-friendly, and responsive LA tools. Developers should prioritize creating systems that deliver accurate and timely information in an easy-to-understand format, ensuring a positive user experience and encouraging tool adoption.

The influence of PI on PE, EE, and IULA suggests that individuals with a higher inclination for experimenting with recent technologies are more likely to adopt LA tools. Developers should consider designing LA tools that cater to the characteristics of innovative and adventurous users, making the tools engaging and exciting for experimentation.

However, the study also revealed that FC and EE did not have a significant impact on IULA. Understanding the reasons behind these findings and addressing any barriers to their influence on user intention could lead to improvements in LA tool design and implementation. Additionally, considering the specific context of developing nations and adapting LA tools to align with users’ familiarity and flexibility preferences is crucial for successful adoption.

Overall, this research provides actionable insights that can guide the development, marketing, and promotion of LA tools, ultimately contributing to the enhancement of educational experiences and outcomes in HEIs.

6.3. Theoretical Implications

This research successfully formulated a comprehensive model based on the UTAUT to assess users’ intention to use “Learning Analytics” (LA) dashboards in HEIs. By extending UTAUT with additional constructs including “Personal Innovativeness” (PI), “Information Quality” (IQ), and “System Quality” (SQ), this study provided a more nuanced understanding of the key factors predicting the adoption and utilization of LA tools. This integration allowed for a comprehensive exploration of users’ intention to utilize LA dashboards, encompassing both established technology acceptance factors and domain-specific variables relevant to the educational context. The study aimed to uncover how these constructs collectively influence users’ intention to use LA, contributing to a more nuanced understanding of LA adoption and usage within the educational ecosystem.

The empirical assessment of this model represents a notable contribution to the field, as there has been a lack of extensive empirical studies evaluating theoretical models concerning LA dashboard usage. The outcomes of this research are expected to shed light on users’ behavior towards LA, offering insights into the key factors that drive their intentions to use these tools.

The implications of this model are far-reaching. Educational institutions, instructors, and LA tool developers can leverage the insights gained from this study to strategize and enhance the adoption and usage of LA tools. By understanding the determinants outlined in the model, decision-makers can make informed choices regarding the selection and implementation of LA tools, aligning them with the specific needs and contexts of their institutions.

Moreover, this research lays the foundation for future studies, serving as a foundational basis for developing a comprehensive theory for the use of LA tools. The identified significant variables and their interactions can be further explored and refined in subsequent research, advancing our understanding of the complex dynamics predicting the adoption and effective utilization of LA in educational settings.

The contributions discussed in the study have notable implications for sustainability within educational technology. A central strategy highlighted is the integration of automation in the design of LA tools. By automating analytical procedures and minimizing human intervention, this approach can potentially optimize resource utilization and operational efficiency, thereby reducing the environmental impact. Additionally, prioritizing user-friendliness, reliability, and responsiveness in LA tool design aligns with sustainable principles, ensuring long-term viability and scalability. Adapting LA tools to specific contexts, particularly in developing nations, is vital for their continued relevance and utilization, embodying a commitment to educational sustainability. Furthermore, a strong focus on data accuracy and quality within LA tools supports well-informed and sustainable decision-making in educational institutions. In summary, integrating sustainability principles in developing and implementing LA tools can significantly enhance their effectiveness, drive user adoption, and ultimately foster sustainable educational practices and outcomes.

7. Concluding Remarks

Compared to developing countries, developed economies are leaders in adopting and utilizing recent technologies to improve teaching and learning procedures. Learning analytics (LA) is a new trend, the utilization of which can bring advantages to educational institutions [56]. However, not much is known about the usefulness of LA tools or their effects on individual cognitive capacities, despite their predicted benefits for enterprises [16,100]. Due to obstacles and a lack of expertise, many higher education institutions might not be prepared to begin adopting LA [101]. Efforts that aim to validate theoretical models of the determinants influencing intention to use LA tools are limited. Therefore, this study aimed to propose and empirically validate a model for the intentions to use LA dashboards in Malaysian HEIs. This study drew upon the UTAUT as a theoretical base and three additional factors (PI, IQ, and SQ). Data were obtained from academic staff and students at selected universities in Malaysia.

The results revealed that the “Intention to Use Learning Analytics” (IULA) is formed with “Performance Expectancy” (PE), “Social Influence” (SI), “Information Quality” (IQ), and “System Quality” (SQ) factors. Personal innovativeness (PI) showed a direct positive influence and an indirect influence through PE on the IULA. However, “Effort Expectancy” (EE) and “Facilitating Conditions” (FC) did not indicate any significant impact on the IULA. The study provides robust implications in academia, as educational institutions in developing nations should enhance the implications of LA tools by motivating users to acquire knowledge through flexible modules.

Limitations and Future Research Recommendations

While this study improves the existing understanding of the phenomena, there are also some limitations. Firstly, the target respondents in the present study were from selected top universities in Malaysia. Future studies can replicate this research by collecting data from a broader range of public and private universities and providing an in-depth comparison between findings. Secondly, it may be exciting to use the created model in other nations and even under different circumstances. Acknowledging the limitations of the study, including the specific focus on selected top universities in Malaysia, opens up possibilities for broader and more diverse future research. Replicating the study across a broader range of institutions, potentially in different countries, can enhance the generalizability and applicability of the findings.

A notable limitation in this study is the presence of multicollinearity, evident from the VIFs of the independent variables PI, PE, and EE surpassing the recommended threshold. To address the impact of multicollinearity resulting from a limited sample size, future research should prioritize expanding the sample size to enhance the robustness and representativeness of the dataset. Additionally, considering interviews with professionals and exploring moderating and mediating roles of variables can further enrich the understanding of LA tool adoption. Further, the study aimed to comprehensively explore the factors influencing LA usage and adoption by considering both academic staff and student perspectives. However, it is important to acknowledge that not treating these perspectives as separate units for the analysis might present a limitation. While this approach allows for a focused examination, analyzing them separately may provide a more in-depth understanding of each viewpoint. Lastly, this study performed a statistical analysis using PLS-SEM. Future studies can integrate statistical and multi-criteria decision-making approaches to acquire more exciting results.