AI-Based Chatbots Adoption Model for Higher-Education Institutions: A Hybrid PLS-SEM-Neural Network Modelling Approach

Abstract

Chatbot implementation for assisting customers as a virtual agent can be seen as a tool in helping an organisation to serve better customer service. Malaysia is among the countries forging ahead with the Fourth Industrial Revolution. One of the core technologies mentioned is adopting artificial intelligence tools such as chatbots. In the last few years, there has been a growing interest in AI-based chatbot adoption in the non-HEI context. However, most higher-education institutions (HEIs) are reported not ready to adopt AI-based chatbots as one of the solutions for virtual student services support. The research of chatbot adoption in the HEI context is still new and is a less explored and examined topic in the information systems domain. Moreover, most of the existing research regarding chatbot adoption in the HEI context focuses more on the benefit of chatbot usage and is not specialised in a student services solution perspective. Furthermore, most of the studies were not guided by the information systems (IS) theories. Therefore, this study aims to identify factors that influence the effectiveness of chatbot adoption in the HEI context by adapting the UTAUT2 model as the IS theory reference. A survey method was applied using the purposive sampling technique. For 3 months, data were collected online from 302 users of Malaysia’s HEI postgraduate students from various public and private universities. A two-stage analytical procedure (SEM-ANN) was used to validate the research model and assess the presented research hypotheses. This research reveals that perceived trust is influenced by interactivity, design, and ethics. Meanwhile, behavioural intention is influenced by perceived trust, performance expectancy, and habit towards the use of chatbot applications in the HEI context. Lastly, the findings of this study can be helpful to the HEI student services unit and can be a guide towards productivity and marketing strategy in serving the students better.

1. Introduction

AI-based chatbots have changed the customer communication landscape and have become quite a marketing buzzword. Chatbots are chat robots that interact online with humans, simulating people’s interactions with each other [1]. In business sectors that have customer service support, they are used to address thousands of frequently asked questions (FAQ) that may be repeated more than one time a day. These conversational agents are bringing a new element to a business’s website and to the way their clients communicate with the company. Expectations for the consumer journey are getting higher, and, by introducing a chatbot as a virtual agent in customer service, it is important to maintain an upward trend [2].

In this 21st century, higher-education institutions (HEIs) are going the digital route. However, what we might not have predicted is its acceleration in the last decade [3]. In the era of AI, chatbots are being utilised more frequently in HEIs for instructional objectives by offering prompt and individualised services to everyone in the industry, including institutional staff and students [4]. Over the past few years, leading colleges and universities around the world have adopted AI-based chatbots for their websites for college inquiry. Such chatbots will serve as university guides round-the-clock, all at a fraction of the cost of recruiting multiple human employees [5]. The availability of this technology will be easier for the students since they do not need to meet the staff at the information counter. However, it is important to analyse the acceptance and use of this technology in universities because it helps to predict the attitude of students towards new technologies [6,7]. Adoption of new technologies can increase educational and scientific results [8]. Substantial research has, therefore, been carried out to define the critical variables affecting the implementation of different technologies in the educational context, using different technology adoption models and frameworks [9]. For example, opportunities provided by several bots with AI features enable universities to access advanced software, updated platforms, and high-tech technology without spending a great deal of money on building and maintaining a large and expensive IT infrastructure while increasing user satisfaction due to the increase in employee performance with the help of AI machine or systems to cover the daily tasks [10]. In the context of HEIs, it gives students and the university staff the opportunity to search for info quickly and economically by using chatbots. Further work has already been performed on chatbots in the private and public sectors. However, most of the studies regarding chatbot adoption in Malaysia’s HEIs combining the successful factors, strategic adoption, development approaches, and chatbot characteristics for HEI were not guided by IS theories.

There are many studies guided by IS theory, but they are not limited to the context of HEI alone [1,4,11,12,13]. For instance, the authors of [14] mentioned that most of the computing platforms related to chatbots in the HEI context in Malaysia did not report an IS theoretical basis for factors influencing the usage of chatbots. Even though those studies provided a theoretical basis, very few attempted to evaluate any of the theoretical component’s hypothesised to be affected by the usage [14]. Chatbots have different levels of understanding depending on their level of AI. Chatbots are nothing more than software with which users interact in natural language. A chatbot (also known as a talkbot, bot, IM bot, interactive agent, or artificial conversational entity) is a computer program that conducts a natural language conversation using auditory or textual methods, understands the user’s intention, and sends a reply based on business rules and data of the organisation [15]. It performs different substitution roles for a man and mimics human behaviour. Together with subsequent experience, it learns while extending its communication skills and develops knowledge about how to customise a message and improve its own communication and reaction rules. Consequently, chatbots can answer questions, provide answers, and solve problems by understanding the intentions of the users. Therefore, a chatbot becomes a man’s technological reflection leading to dehumanising what is real and humanising science with its (more human than human) manifestations [13,16].

With the new normal due to the recent world pandemic, less face-to-face communication has been encouraged. In the problem background, researchers have mentioned that there are a few other choices of virtual customer services such as emails, chat support, and phone calls that can also support the new procedure of social distancing. However, as time goes by, the solution is no longer considered the best choice, especially when it comes to real-time response, handling contentious conversations between consumers and customer service agents, and effectively handling the daily FAQ routine. Furthermore, chatbots make fewer errors while answering customer inquiries. Thus, chatbots are among the virtual customer service platforms that can be considered by organisations for adoption. At the same time, the chatbot’s availability gives another level of choice to the users to communicate seamlessly despite not approaching customer services physically.

According to [17], chatbot implementation in the HEI context has been studied. The key limitation of this research is that the model is insufficient to cover the effectiveness of chatbot adoption. Most of the chatbot adoption feedback from students in developed countries is related to the inability to answer the questions precisely, which leads to an insufficient knowledge base. They are aware of the benefits of using a chatbot and are comfortable using it for the first time since they can access the info anytime and anywhere without waiting for a reply compared to communication via emails, phone calls, and customer service counters. However, knowing that the chatbot applications could not fulfil their needs, they feel disbelief in using the service. On the basis of the above introductory study, we can see the potential of chatbot applications in helping universities’ administration processes to improve customer service. At the same time, a chatbot can be used to ease the administration staff’s burden to attend to the same repetitive administrative questions from students while they can allocate the time to do other things. Therefore, the main aim of this study was to develop a model for chatbot adoption in the Malaysian HEI context through the extension of the UTAUT 2 model with new constructs, which are perceived trust, design, perceived interactivity, and ethics. This study is important to be addressed to the Malaysia HEI organisations that have the intention to adopt chatbot usage as their customer service platform for students. This study also addresses the IR4.0 goal for organisations, which is to be better than the average human at making mistakes and to implement AI, as well as IoT, to drive some human tasks in the organisation.

The remainder of this paper is organised as follows: Section 2 provides the research model and hypothesis development details followed by the research methodology. In Section 4, the results describe the measurement model assessment and the ANN analysis approach. Section 5 provides a discussion of the positive hypothesis and the variables that are not positively significant to the study. Section 6 presents the conclusion of the discussion, which contains the theoretical contribution, practical implementations, limitations of the study, and future work.

2. Research Model and Hypothesis Development

This study proposes a theoretical model by extending the UTAUT2 model based on the constructs (perceived expectancy, hedonic motivation, effort expectancy, social influence, habit, facilitating conditions, behavioural intention, and use) combined with the chatbot adoption success factor framework in the private service sector (design, ethics, and interactivity) to encourage students to adopt the chatbot in the HEI context. The model was also developed on the basis of the integrated model [17] for continuance learning of IS model, which contributes to perceived trust as one of the constructs in this research study. Since chatbots are still in their early stages in the HEI context, it is hoped that using those factors will provide an insightful understanding of students’ long-term use of chatbots. Chatbot adoption success factors are included in this model due to the effectiveness and efficiency of the chatbot usage in the organisation in terms of conversation, communication, and responsiveness between bots and users that are influenced by the constructs of design, ethics, and interactivity. However, according to [18], future studies should collect empirical evidence, and the results should be tested according to the IS theory. Thus, this study includes those constructs to be tested in the UTAUT2 extended model.

The model presented in Figure 1 depicts 12 relationships (represented by path diagrams) that correspond to the study’s stated hypotheses. The proposed model suggests that chatbot adoption is supported by UTAUT 2 model extended with constructs that include design, interaction, and ethics of chatbot usage that have a significant impact on the perception of trust in chatbot usage. The constructs extracted from the previous study for this research were interactivity, design, ethics, and perceived trust.

Table 1. The combination of chatbot adoption success factor variables (referring to education in both public and private sectors).

No	New Variables	Is Model	Author	Details of Variables
1.	Procedures and security	N/A	[19]	The procedures and security are the potential variables that can help to improve users of a Peruvian educational institution in using chatbot.
2.	Usability characteristics and usability techniques	N/A	[20]	The characteristics include effectiveness, efficiency, and satisfaction.
3.	Design and security	TAM	[21]	The design and security are the stimuli that represent the technology adoption and feature capabilities.
4.	Attitude and security	TAM & DOI	[22]	The results of this study may provide insights and understanding on the attitude of the user towards chatbot developers, researchers, and organisations and the intention to use messenger chatbots.
5.	Attitude	TAM	[23]	Attitude of consumers towards E-CRM behavioural intention is significantly affected by perceived quality, perceived usefulness, and perceived ease of use, and an extended TAM model framework is proposed and empirically tested using data collected from the survey.
6.	Efficiency, the expectation, expectation of effort and habit significantly	UTAUT2	[24]	Three primary constructs, namely, efficiency, expectation of effort, and habit, significantly predicted the behavioural intention (BI) of students to use chatbot technology.
7.	Self-management of learning	UTAUT2	[25]	The performance expectation and self-management of learning affect a behavioural intention to implement the CRM m-learning.
8.	Data security, compatibility and the relationship with the technology.	UTAUT2	[26]	The results show that six out of seven determinants are important when looking at the adoption of chatbot technology leaving the updated UTAUT2 model with hedonic motivation.
9.	Perceived trust and perceived service quality	UTAUT2	[27]	The results prove that perceived trust and perceived service quality improved the explanatory and predictive power of the original model.

shows the eight studies related to this research area.

2.1. Design

Design refers to the characteristic property of the technology used for chatbot development. According to a literature review, the authors of [28] categorised the design into two subcategories which are functionalities and security. They also confirmed that the design of the chatbot-related features could positively impact the trust of the users in choosing chatbot as a platform for customer service. Meanwhile, according to [21], design and security are the stimuli that reflect the system and its capabilities. A previous study determined the significant impact of design on perceived trust in the private service sector [27]. Previous studies confirmed the beneficial effect of expectation confirmation on perceived utility in the setting of intelligent chatbots [12]. However, this previous research was in a different context (online marketing). Accordingly, we suggest that students have more trust in using chatbots as their virtual agent service if the design can easily solve their inquiries when using the chatbot. Hence, the following hypothesis is formulated:

H1: 

Design positively influences perceived trust in chatbot adoption.

2.2. Perceived Interactivity

Interactivity refers to users’ communication and response in order to ensure high-quality interaction between students and the chatbot [18,29]. Interactivity has been used and tested in a variety of contexts in previous studies (e.g., [30]). The authors of [28] studied interactivity related to users’ communication and response in order to ensure high-quality interaction between students and the chatbot. Meanwhile, the authors of [18] categorised interactivity including conversation and language, as well as communication and response.

The influence of interactivity on the intention to adopt AI-based chatbots has been discussed in a previous research context. It is important to lead to a better understanding of the user’s problem. Communication and understanding between user and chatbot are important to lead to a better understanding of the user’s problem [31,32,33]. From here, this indicates the effectiveness and efficiency of chatbot usage in the organisation in terms of conversation, communication, and responsiveness between bots and the users or people. In the same vein, the present study defines perceived interactivity as the students’ communication and response with regard to high-quality interaction between students and the chatbot. On the basis of the previous study, we agree that high-quality interaction can provide the best possible solution to influence users to adopt chatbot usage. Accordingly, the following hypothesis is formulated:

H2: 

Interactivity positively influences perceived trust in the adoption of the chatbot.

2.3. Ethics

Ethics refer to users’ acceptable and unacceptable conduct of chatbots towards students [17]. The same thought was explained by [18], whereby ethics consider the criteria that differentiate between acceptable and unacceptable conduct. Ethics count as a consideration in the adoption of new technology and have a direct impact on the trust in the technology adoption. In this study, the perception of ethics is tested using a chatbot in the HEI context. It is considered as the user’s acceptable and unacceptable conduct of the chatbot towards students. This study proposes that, if students agree that using the chatbot allows them to communicate properly and contribute to a positive value, they would have a higher chance of continuing use in the future as a preferred solution to prevent ethics issues in customer service. Accordingly, the following hypothesis is formulated:

H3: 

Ethics positively influence perceived trust in chatbot adoption.

2.4. Perceived Trust

Perceived trust refers to users’ perceptions about the expected reliability and integrity of the chatbot platform. According to previous studies [34,35], perceived trust and behavioural intention have a direct relationship, whereby trust in using or adopting technology will lead to individual dedication to participating in specific activities. Previous research has offered some hints as to the crucial elements in figuring out whether or not users trust chatbots. However, given that chatbots include a number of extremely unique qualities, it is necessary to specifically examine trust in relation to this interactive technology.

In this study, perceived trust can help students commit to using a chatbot in daily campus lifestyle whenever needed. Previous experience in using chatbots might have a relationship with continuous usage. A negative chatbot experience might lead to trust issues in using a chatbot in the future. Thus, we propose that students prefer to use a chatbot if their trust issues do not impact them in adopting chatbot usage as one of the preferred solutions for customer service. Accordingly, the following hypothesis is formulated:

H4: 

Perceived trust positively influences behavioural intention in chatbot adoption.

2.5. Performance Expectancy

On the basis of the literature, the present study defines performance expectancy as users’ feelings about the chatbot that will help them to achieve the exact answer. Performance expectancy is modified from the study by [11]. Performance expectancy was studied as a construct that influences individuals’ belief in utilising the technology that would increase their results in using the technology. The authors of [25] concluded that performance expectancy is related to the user’s feelings in using a chatbot to get the exact answer in the right time manner [36]. They mentioned that performance expectancy was taken originally from the UTAUT2 framework by Venkatesh [37]. The authors of [25] confirmed the hypothesis that performance expectancy influences university students’ behavioural intention to use chatbots. The authors of [24] used a UTAUT2 model to identify the user’s perception about chatbot usage in customer relationship management (CRM) adoption in the higher-education context. They found that the construct performance expectancy significantly predicted the behavioural intention (BI) of students to use chatbot technology. The authors of [24,25] agreed that this indicator should focus on the user’s perception of technology adoption in the community. Accordingly, the following hypothesis is formulated:

H5: 

Performance expectancy positively influences behavioural intention in chatbot adoption.

2.6. Effort Expectancy

According to previous research, effort expectancy is defined as the user’s intention to use the chatbot effortlessly or how easy it is to use a chatbot. In various studies, effort expectancy and its latent variables were proven to be strong predictors of a user’s willingness to accept new technologies [38]. According to [25], effort expectancy offers a beneficial impact on university students’ willingness to adopt chatbots in the future. Meanwhile, the authors of [39] mentioned that effort expectancy refers to the user’s perceptions of the technology platform’s ease of use or the projected effort required to use it. The hypothesis from the study showed that the user attitude towards technology is positively influenced by the technological platform’s effort expectancy. In this study, effort expectancy measures the relationship in adopting a chatbot for students since it helps in measuring expected performance and effort as one of the key factors. Accordingly, the following hypothesis is formulated:

H6: 

Effort expectancy positively influences behavioural intention in chatbot adoption.

2.7. Social Influence

The social Influence factor has been demonstrated to influence one’s behaviour [37]. The term social influence refers to the degree to which an individual perceives that important others believe they should use the technology [40]. In a variety of studies, social influence evolved as a key factor in determining whether or not a user intends to adopt a particular technology [41]. A study by [25] showed that social influence has an impact on the students’ willingness to use chatbots in a positive way. It showed that this instance has a direct impact on the students’ behavioural intention in using a chatbot for student engagement purposes in universities. The results showed that students who receive positive reinforcement for using a chatbot become more committed to using it on a daily basis. In line with the previous study, we suggest that social influence triggers students’ willingness to use a chatbot in a positive way.

Accordingly, the following hypothesis is formulated:

H7: 

Social influence positively influences behavioural intention in chatbot adoption.

2.8. Facilitating Conditions

Facilitating conditions are defined as the extent to which an individual believes that an organisation and technical infrastructure exists to support the use of the system. The authors of [42] proposed that facilitating conditions refer to an individual’s belief that the technology and organisational infrastructure are in place to enable the use of the technology [37]. The hypothesis from [25] showed that facilitating conditions have a beneficial impact on the university students’ willingness to use chatbots in the future. This study examines students’ belief with respect to whether the infrastructure readiness is in place while adopting the chatbot as one of the student engagement platforms in the HEI context. Accordingly, the following hypothesis is formulated:

H8: 

Facilitating conditions positively influence behavioural intention in chatbot adoption.

2.9. Hedonic Motivation

Hedonic motivation refers to the user’s perception that motivation plays a positive role in determining technology acceptance and usage. The authors of [43] defined hedonic motivation as a feeling that arises from employing technology such as joy or happiness. In the context of student engagement, intrinsic aspects such as fun and enjoyment were found to have a substantial impact on the student’s attitude towards a new technology [34]. Despite the performance implications that could be anticipated, this variable is about a sense of pleasure when using the chatbot. According to a previous study conducted by [25], it was demonstrated that this variable has a favourable impact on technology acceptance and its utilisation by students. The belief is that hedonic motivation could be another positive influence that motivates the students to adopt a chatbot. Accordingly, the following hypothesis is formulated:

H9: 

Hedonic motivation positively influences behavioural intention in chatbot adoption.

2.10. Habit

In the context of IS and technology, habit is defined as the level at which people tend to conduct behaviours (use IS) automatically as a result of learning. There are two definitions of habit: prior behaviour and automatic behaviour [37]. Meanwhile, the authors of [34,37] mentioned that habit could be viewed in two ways: as an example of a previous action or as a habitual pattern. The UTAUT2 model states that habit has both a direct and an indirect impact on the use of technology. According to a previous study conducted by [25], it was demonstrated that university students’ behavioural intentions to use chatbots are positively influenced by habit. Therefore, this study aims to comprehend the suitability of the habit construct among empirical investigations of students’ chatbot adoption in the HEI context. Accordingly, the following hypothesis is formulated:

H10: 

Habit positively influences behavioural intention in chatbot adoption.

2.11. Use

Use refers to the user’s utilisation of the chatbot to probe the intention and usage of the platform. According to [18], ease of use is related to the learnability of technology. The organisation in which the new change is implemented must be prepared, and the end user’s acceptance must be evaluated. In this study, the user’s utilisation of the chatbot is measured to probe the intention and usage of the platform. Accordingly, the following hypothesis is formulated:

H11: 

Behavioural intention positively influences students to use the chatbot.

3. Research Methodology

3.1. Sample and Data Collection Procedure

The data in this study were collected from postgraduate students in various universities in Malaysia who previously had experience in using chatbot applications. Gathering of the data was undertaken via a study questionnaire by employing a purposive sampling approach. This sampling method was built on the presumption that the nominated sample elements represent the population of interest and are likely to support the research goals. The reason for choosing this sampling approach is because the adoption of chatbot in the HEI context is still new in Malaysia and there are a small number of experienced or non-experienced users consisting of students in the universities who use chatbots developed and adopted for academic administration purposes.

This study established a set of criteria in order to choose suitable respondents. These criteria aimed to ensure that potential respondents met the demands of this study. The first criterion applied was related to the HEI student’s characteristics. Meanwhile, the second criterion was related to the chatbot type. The requirements for individuals and the chatbot type chosen for this study are outlined in detail below.

• The person must have prior experience with chatbot usage.
• The chatbot type can be task-oriented (two-way interaction between chatbot and users) or non-task-oriented (static information provided by the chatbot only).

A sample size of 300 participants was required after the calculation was made using G*Power software 3.1. The statistics of intake for postgraduate students in Malaysia referred to the Ministry of Higher Education statistics from 2019 to 2021. From the statistics, the number of participants suggested was reliable and adequate. The postgraduate students from private and public universities in Malaysia were participants in this research study. The participants were given a set of questions online, and they were expected to read the instructions carefully before responding to the questions. The data of at least 300 students were collected within 3 months after distributing the survey link through email by the university’s faculty administrator.

3.2. Research Instrument

Two sections of an online survey were used to collect data from the target respondents. The questions in the first section were intended to gather data on the demographics of the students, and the questions in the second section were intended to measure the variables in the research model. The elements of each category were measured using a five-point Likert scale with values ranging from “1 = strongly disagree” to “5 = strongly agree”. While the items used to measure perceived trust were modified from [29], the items used to test ethics, interactivity, and design were taken from [18], and the items used to test use, behavioural intention, habit, hedonic motivation, facilitating conditions, social influence, effort expectancy, and performance expectancy were taken from [11]. The constructs and their associated elements are described in depth in Appendix A.

3.3. Data Analysis

For data analysis, a two-stage analytical procedure (PLS-SEM–ANN) was used to validate the research model and assess the presented research hypotheses. First, the PLS-SEM approach was applied by examining the reliability and validity of the indicators and constructs. PLS-SEM is mainly used for investigation rather than confirmation [44,45,46]. Prior to validating the structural model, the measurement model was evaluated in terms of the relationships between constructs. Then, the ANN was used to investigate the key elements influencing chatbot usage over time. This research used the ANN method to extensively apply linear and nonlinear interactions between the variables of the suggested model [4,47,48,49]. It is crucial to note that ANN can make predictions that are more accurate than the majority of the current regression techniques [49,50].

4. Result

4.1. Measurement Model Assessment

Through assessing internal consistency reliability, convergent validity, and discriminant validity, the measurement model was created to examine the validity and reliability of the constructs and indicators [44,45,46]. To assess the internal consistency reliability of the constructs, Cronbach’s alpha (CA) and composite reliability (CR) were used.

Table 2. Reliability and convergent validity results.

Construct	Indicators	Loadings	CR	Cronbach’s Alpha	AVE
Performance expectancy	PE1	0.839	0.933	0.911	0.736
	PE2	0.872
	PE3	0.858
	PE4	0.85
	PE5	0.87
Effort expectancy	EE1	0.872	0.929	0.905	0.724
	EE2	0.842
	EE3	0.861
	EE4	0.866
	EE5	0.813
Social influence	SI1	0.809	0.916	0.887	0.686
	SI2	0.831
	SI3	0.844
	SI4	0.821
	SI5	0.837
Facilitating conditions	FC1	0.795	0.882	0.834	0.601
	FC2	0.809
	FC3	0.777
	FC4	0.787
	FC5	0.705
Hedonic motivation	HM1	0.826	0.853	0.742	0.659
	HM2	0.795
	HM3	0.816
Habit	HT1	0.751	0.796	0.618	0.566
	HT2	0.74
	HT3	0.767
Interactivity	INT1	0.818	0.835	0.759	0.507
	INT2	0.781
	INT3	0.653
	INT4	0.577
	INT5	0.704
Design	DE1	0.857	0.852	0.781	0.539
	DE2	0.695
	DE3	0.603
	DE4	0.786
	DE5	0.704
Ethics	ET1	0.772	0.548	0.72	0.549
	ET2	0.634
	ET3	0.848
	ET4	0.687
Perceived trust	PT1	0.802	0.878	0.819	0.647
	PT2	0.83
	PT3	0.775
	PT4	0.809
Behavioural intention	BI1	0.837	0.858	0.752	0.669
	BI2	0.812
	BI3	0.804
Use	USE1	0.828	0.868	0.80	0.621
	USE2	0.74
	USE3	0.758
	USE4	0.824

shows that the CA values were between 0.72 and 0.912, whereas the CR values were between 0.827 and 0.933. As for the habit construct, the CA value reported was 0.6178, which is still acceptable. According to [44,45,51], both CA and CR values exceeded the benchmark level of 0.70, thus demonstrating the measures’ high dependability. The outer loadings of the indicators were used to examine the convergent validity and to extract the average variance (AVE). As shown in Table 2, the study’s convergent validity was achieved because the factor loadings were >0.708 and the AVE values were >0.50. [45]. The degree of construct separation is explained by the discriminant validity. In this paper, the heterotrait–monotrait (HTMT) ratio was used to evaluate the discriminant validity [52]. The HTMT criterion was satisfied, as indicated in

Table 3. HTMT results.

	BI	DE	EE	ET	FC	HM	HT	INT	PE	PT	SI	USE
BI
DE	0.4422
EE	0.5345	0.3418
ET	0.7601	0.5642	0.6344
FC	0.5892	0.4012	0.8362	0.7074
HM	0.5204	0.3851	0.6204	0.6324	0.7867
HT	0.8413	0.4609	0.5521	0.7075	0.6905	0.7233
INT	0.5208	0.7667	0.4209	0.6570	0.5573	0.4375	0.5703
PE	0.5561	0.2503	0.8891	0.6023	0.7757	0.5552	0.5521	0.3542
PT	0.7526	0.6301	0.5830	0.8396	0.6199	0.5760	0.6141	0.6121	0.5633
SI	0.5117	0.3150	0.7487	0.6004	0.8374	0.7034	0.5981	0.4688	0.7235	0.5875
USE	0.7921	0.5262	0.5943	0.8445	0.6653	0.6053	0.6669	0.4870	0.6163	0.7950	0.5757

, since all values were below the 0.85 ceiling value. Figure 2 shows the structural model based on the PLS-SEM.

4.2. Structural Assessment Model

The structural model was then evaluated by obtaining the path coefficient (beta), t-values, coefficient of determination (R²), effect sizes (f²), and predictive relevance (Q²) using a bootstrapping technique with 5000 resamples [44,45].

To ensure there were no lateral collinearity difficulties with the structural model, the collinearity between the variables was initially assessed using the variance inflation factor (VIF) before the structural model was evaluated [45,46].

The findings in

Table 4. Hypotheses testing results.

Hypothesis	Structural Path	Path Coefficient	t-Value	p-Value	f2	R2	Q2	VIF	Empirical Evidence
H1	BI -> USE	0.637	16.0967	0.0000	0.6817	0.4033	0.238	1.0000	Supported
H2	DE -> PT	0.212	3.0419	0.0012	0.0548	0.4972	0.313	1.6455	Supported
H3	EE -> BI	−0.007	0.0696	0.4722	0.0000	0.4778	0.314	3.6039	Not supported
H4	ET -> PT	0.492	7.7573	0.0000	0.3545	0.4972	0.313	1.3730	Supported
H5	FC -> BI	0.041	0.5381	0.2953	0.0011	0.4778	0.314	3.1176	Not supported
H6	HM -> BI	−0.03	0.4344	0.3320	0.0009	0.4778	0.314	1.8396	Not supported
H7	HT -> BI	0.356	4.7188	0.0000	0.1646	0.4778	0.314	1.5089	Supported
H8	INT -> PT	0.142	1.9141	0.0278	0.0227	0.4972	0.313	1.7802	Supported
H9	PE -> BI	0.134	1.6662	0.0479	0.0110	0.4778	0.314	3.1940	Supported
H10	PT -> BI	0.385	6.1198	0.0000	0.1832	0.4778	0.314	1.5849	Supported
H11	SI -> BI	−0.026	0.4372	0.3310	0.0005	0.4778	0.314	2.5796	Not supported

show that seven of the suggested hypotheses were accepted and four of the suggested hypotheses were rejected. The behavioural intention positively influenced the use in chatbot adoption (H1: β = 0.637, t = 16.096). Meanwhile, design positively influenced perceived trust (H2: β = 0.212, t = 3.042), effort expectancy did not positively influence behavioural intention (H3: β = −0.007, t = 0.069), ethics positively influenced perceived trust (H4: β = 0.492, t = 7.757), facilitating conditions positively influenced behavioural intention (H5: β = −0.041, t = 0.538), hedonic motivation did not positively influence behavioural intention (H6: β = −0.03, t = 0.434), habit positively influenced behavioural intention (H7: β = 0.356, t = 0.472), interactivity positively influenced perceived trust (H8: β = 0.142, t = 1.914), perceived expectancy positively influenced behavioural intention (H9: β = 0.134, t = 1.666), perceived trust positively influenced behavioural intention (H10: β = 0.385 t = 6.119), and social influence did not positively influence behavioural intention (H11: β = −0.026, t = 0.437). Because the coefficients of determination (R²) for the three endogenous constructs accounted for a sizable amount of total variance (R² = 0.4778 for behavioural intention, R² = 0.4972 for perceived trust, and R² = 0.4033 for use), the PLS-SEM results in Table 4 show that the theorised model was statistically meaningful [53].

As shown in Table 4, the effect sizes (f²) were also estimated using Cohen’s [53] guidelines. According to the f² results in Table 4, effort expectancy had a small effect size of 0.000 on behavioural intention. Furthermore, with the value of 0.681, behavioural intention reported the largest effect size on use. Meanwhile, the f² result showed a medium effect size of ethics on perceived trust with a value of 0.354. The f² values with a small effect size of 0.054, 0.001, 0.0009, 0.1646, 0.0227, 0.0110, 0.1832, and 0.0005 were observed for design, facilitating conditions, hedonic motivation, habit, interactivity, perceived expectancy, perceived trust, and social influence. The structural model also achieved predictive relevance (Q²) using the cross-validated redundancy results of the three endogenous variables (based on the blindfolding procedure with an omission distance of 7). According to Table 4, the Q² values for behavioural intention, perceived trust, and use were Q² = 0.314, 0.313, and 0.239, respectively (higher than zero).

4.3. ANN Results

The first stage of this study applied PLS-SEM to test the correlations and determine the factors influencing students’ chatbot adoption in the HEI context. The ANN analysis approach was used in the second stage to rank the factors affecting chatbot adoption. This study chose the ANN because, as previously reported [4,43,47], it outperforms more conventional statistical tools such as multiple linear regression, binary logistics regression, and SEM in detecting both linear and nonlinear relationships. An input layer, one or more hidden layers, and an output layer are typical neural network components. The sigmoid function was used as the activation function for the output and hidden neurons. The sigmoid function was used as the activation function for the output and hidden neurons.

For improved model performance, the range of the input and output neurons was kept within [0, 1] [43]. Overfitting was avoided by using a tenfold cross-validation technique in the ANN models, in which the training phase used 90% of the data and the testing phase used the remaining 10% [41,43]. Three endogenous constructs, namely, behavioural intention, perceived trust, and use, were included in the research model created for this study. Consequently, as shown in Figure 3, Figure 4 and Figure 5, the research model was divided into three ANN models. The first ANN model comprised three input layers (as shown in Figure 3), i.e., interactivity, design, and ethics, with one output layer, i.e., perceived trust. As shown in Figure 4, the second ANN model had two input layers, i.e., habit and perceived trust, and one output layer, i.e., behavioural intention. The third ANN model contained one input layer (behavioural intention) as shown in Figure 5 and one output layer (use).

In order to assess the accuracy of the three constructed ANN models, the root-mean-square of error (RMSE), a standard accuracy metric employed in prior research [48,50], was used. The RMSE represents the error during the training and testing phases. The average RMSE of the three neural network models was 0.12, 0.122, and 0.131 for the training data and 0.119, 0.121, and 0.133 for the testing data according to the readings in

Table 5. RMSE values of ANN models.

Network	Model 1		Model 2		Model 3
	RMSE (Training)	RMSE (Testing)	RMSE (Training)	RMSE (Testing)	RMSE (Training)	RMSE (Testing)
1	0.119	0.119	0.118	0.131	0.133	0.140
2	0.117	0.128	0.109	0.149	0.129	0.145
3	0.113	0.129	0.119	0.121	0.153	0.154
4	0.125	0.106	0.130	0.102	0.134	0.133
5	0.122	0.122	0.130	0.106	0.130	0.110
6	0.113	0.136	0.121	0.124	0.122	0.119
7	0.123	0.109	0.122	0.119	0.120	0.127
8	0.126	0.103	0.119	0.131	0.134	0.134
9	0.120	0.119	0.117	0.127	0.117	0.131
10	0.118	0.122	0.131	0.107	0.135	0.134
Mean	0.120	0.119	0.122	0.121	0.131	0.133
Standard deviation	0.005	0.011	0.007	0.014	0.010	0.012

. This shows that the models had a high level of prediction accuracy for a variety of endogenous categories. As a result, it is thought that the ANN models created for this study produced dependable and accurate findings.

The normalized importance is computed by considering the average of each predictor against the highest mean value, which is expressed as a percentage. The mean and normalized importance of all the employed predictors during the ANN modeling process are presented in

Table 6. Sensitivity analysis for model 1.

Neural Networks	INT	DE	ET
1	0.380	0.172	0.448
2	0.380	0.172	0.448
3	0.283	0.172	0.545
4	0.321	0.267	0.412
5	0.404	0.156	0.439
6	0.188	0.215	0.597
7	0.188	0.215	0.597
8	0.300	0.171	0.529
9	0.104	0.272	0.624
10	0.348	0.181	0.470
Mean importance	0.290	0.199	0.511
Normalized Importance %	0.602	0.395	1.000
Ranking	2	3	1

. As per the results of sensitivity analysis in Table 6, it can be noticed that ET is the most important input that is associated with chatbot adoption in the HEI context, followed by INT with a relative importance of 0.602. Further, the DE has the lowest influence on chatbot adoption with a relative importance of 0.395.

5. Discussion

This research evaluated the intention to adopt a chatbot in the HEI context. Organisations are becoming more innovative in their competition strategies, increasing the need for breakthrough innovation that imposes business differentiation, provides unprecedented value to customers, and creates intangible resources [54]. Chatbot implementation for assisting students as a virtual agent can be seen as a tool in helping an organisation to serve better customer service. The authors of [20] mentioned that successful chatbot adoption has to be useful, relatable, accurate, trustworthy, and likeable, and the characteristic must keep pace with the factors, which represents why reasonable consumers would consider using a chatbot as their preferred solution for customer service assistance. This research aimed to examine the factors that influence chatbot usage for administration inquiry and information purposes among students in the context of HEI. To accomplish this, an integrated model was created on the basis of constructs extracted from the previous UTAUT2 model of chatbot adoption by [25]. Using a hybrid SEM–ANN methodology, the model was then validated using information gathered from university students.

The results show that the design positively influences perceived trust in chatbot adoption. The hypothesis supports the results found in previous research [18]. This explains that the design of chatbot-related features positively impacted the trust of users in choosing chatbot as a platform for customer service. This could be explained by the fact that a chatbot’s adoption affects how the chatbot is being designed, and that the design could impact their trust in using the chatbot. This result could address the issue highlighted by [17], whereby an effective chatbot design can centralise the information needed by students from different age groups, educational backgrounds, and walks of life.

The result also supports the positive impact of interactivity on perceived trust in chatbot adoption. This result aligns with the previous research reporting that high-quality interaction could provide the best possible solution to influence users to adopt chatbot usage. The quality of how the chatbot reacts to the inquiry will build students’ trust in using the chatbot from time to time. In other words, students prefer to use chatbots when they feel that the interactivity between the bot and themselves is meaningful and could lessen their burden in getting the information needed promptly. This result could address the issue highlighted by [17] whereby an interactive chatbot will be able to overcome the absence of direct interaction with a human that handles enquiries for customer service.

Ethics were found to significantly influence students’ trust in using a chatbot. This hypothesis clarifies that the consideration of accepting the chatbot adoption has a direct impact on the trust in adopting the chatbot. This means that, if students agree that using a chatbot will allow them to communicate properly and contribute to a positive value, they will have a higher chance of continuing to use it in the future as a preferred solution to prevent ethics issues in customer service. This result could address the issue highlighted by [21], whereby chatbot adoption can motivate the students by eliminating unfriendly and abusive customer service.

This study also found that perceived trust positively influences behavioural intention in chatbot adoption. This hypothesis explains the previous studies provided by [17], i.e., that perceived trust and behavioural intention have a direct relationship, whereby trust in using or adopting the technology leads to the individual dedication to participating in specific activities. In other words, perceived trust helps students commit to using a chatbot in the daily campus lifestyle whenever needed. This result could address the issue highlighted by [17], whereby students prefer to use a chatbot if their trust issues do not impact them adopting chatbot usage as a preferable solution for customer service.

The performance expectancy was found to be positively influenced by behavioural intention in chatbot adoption. This result confirms that the performance expectancy relatively impacts the user’s feeling when using a chatbot to get the exact answer correctly. This result is in line with previous research conducted by [24,25], whereby they agreed that this indicator should focus on the user’s perception of technology adoption in the community. This result could address the issue in previous studies, whereby, no matter how easy it is to use the chatbot, it would not be adopted if it is not deemed valuable. Users adopt a chatbot when they believe it is helpful to them.

The effort expectancy was found to not positively influence behavioural intention in chatbot adoption. The result shows that the technological platform’s effort expectancy does not positively influence the students’ attitude towards technology. This means that, if using a chatbot requires great effort, the students may be discouraged from adopting the chatbot usage. The result differs from the study conducted by [17], whereby users believed that chatbots are easy to use and understandable, and that they could become skilful at using them. It can be inferred that when users find it easy to get a chatbot to answer their inquiries, it helps users accomplish things faster.

Social influence is another construct that does not positively influence behavioural intention in chatbot adoption. Previous research showed that social influence influences one’s behaviour. Somehow, in this study, it was reported to not positively impact behavioural intention in chatbot adoption. This shows its indirect impact on students’ behavioural intention in using a chatbot for student engagement purposes in universities. On the other hand, students do not require social reinforcement to use the chatbot. Users are not influenced by the opinions, suggestions, and recommendations of important others who think they should adopt a chatbot. Facilitating conditions were reported to be insignificant towards behavioural intention in chatbot adoption. The authors of [24,31] defined facilitating conditions as an individual’s belief that the technological and organisational infrastructure is in place to enable the use of technology. This means that students’ belief in infrastructure readiness is not positively impacted their behaviour towards adopting the chatbot in the HEI context. This means that students would not be concerned by the infrastructure behind the chatbot as long as the technology could improve their experience in using the chatbot, with no issues continuing to use it in the future. When users find a chatbot easy to use, it eliminates the need for support infrastructure. This explains why facilitating conditions were found to be insignificant in predicting adoption intention.

Hedonic motivation was reported to not positively influence behavioural intention in chatbot adoption. According to previous research, hedonic motivation is defined as a feeling that arouses as a result of employing technology such as joy or happiness. In this study, it was related to the enjoyment or pleasure brought on by students using the chatbot. In this context, intrinsic aspects such as fun and enjoyment were found to have an insignificant impact on the student’s attitude towards new technology. This means that, when the students use a chatbot, the probability of feelings or excitement would not influence them using it in the future. Therefore, hedonic motivation could be another insignificant influence that motivates students to adopt a chatbot. Habit was reported to have a positive influence on behavioural intention in chatbot adoption. A previous study mentioned that habit has both direct and indirect impacts on the use of technology. This study demonstrated that university students’ behavioural intentions to use chatbots are positively influenced by habit. The usage of chatbots by students was significantly predicted by habit, which suggests that, as this technology is used more frequently, students are more likely to want to use it as one of the customer service solutions. The effect of habit is consistent with previous investigations conducted by [25] with a conclusion that habit positively influences behavioural intention in chatbot adoption. This result explains the findings from previous research conducted by [17]. They mentioned that, when the use of a chatbot becomes routine, habit becomes an additional force that increases the behavioural intention to use the technology. This research revealed that behavioural intention had a major impact on use. This implies that students intend to use the chatbot if needed, and that their utilisation of the chatbot was positively in place in order to probe the intention and usage of the chatbot. Thus, the hypothesis that behavioural intention positively influences use in chatbot adoption was accepted. Deriving from the empirical data, the ANN results showed that ethics was the most influential factor in predicting the sustainable use of chatbots. When chatbots enable students to access and apply the acquired information in the university administration tasks, they are more likely to use these intelligent agents in the future.

6. Conclusions

6.1. Theoretical Contributions

This study established a few significant theoretical contributions. The authors of [17] recently investigated chatbot usage in the HEI context. The scope of the study was limited to computer science students at Brunel University, United Kingdom. They used the UTAUT2 model in their study to look at the elements that lead to the intention to adopt chatbots in the HEI context by taking three main predictors from the original UTAUT2 model, namely, perceived expectancy, effort expectancy, and habit, and removing four insignificant predictors from the original UTAUT2 model, namely, effort expectancy, social influence, facilitating conditions, and hedonic motivation. The study’s initial theoretical contribution was to look into the elements that influence chatbot adoption by extending four other constructs of perceived trust, interactivity, design, and ethics. The possibility for perceived trust, design, and ethics to become chatbot adoption constructs was investigated in this research study, which resulted in the development of a new extended theoretical model for chatbot adoption in the HEI context. A previous study explicitly focused on computer science students at Brunel University, United Kingdom. In their context, chatbot adoption is a tailored service delivered through a suitable platform suggested by the developer to enable users to access correct information via chatbot information. On the other hand, this research focused on chatbot usage, allowing users to ask frequent questions and other general information regarding academic and university general matters.

6.2. Practical Implications

The factors that influence users’ decisions to adopt the chatbot technology in HEI were revealed in this study. Researchers can use the findings of this study to evaluate the proposed model in another context outside of Malaysia. According to this study, the new construct constituting design, interactivity, ethics, and perceived trust was proven to have an impact on the users’ intention to use a chatbot in HEI, and the findings were based on a survey of 284 postgraduate students from various education backgrounds in Malaysia. Chatbot developers can use the findings from this study to design a marketing approach tailored to HEI. Meanwhile, in the HEI context, the findings of this study may be used to plan for a better user experience with customer service. The findings are also envisaged to assist HEI management in developing a plan to increase the likelihood of a successful chatbot implementation in the HEI environment. As a result, the study’s findings have significant ramifications for both university administration and users. It is hoped that future adoption of a chatbot implemented in the HEI based on this model is capable of assisting users keen to use the chatbot as an option of virtual customer service but without the opportunity to use it due to a lack of implementation in the HEI.

6.3. Limitations and Future Work

In terms of limitations, the results of this model were proposed on the basis of the responses from postgraduate students in the HEI. Other constructors and moderators (educational level, engagement level, and others) should be included in future work to increase the use of chatbots in many situations. To better understand chatbot adoption intentions in diverse contexts, future studies should include more respondents from a multi-country comparative study.

Thus, future studies should include evaluating the proposed paradigm for different education levels, such as undergraduate, and the results can be extended to university management in order to get feedback from academic staff regarding chatbot adoption in the HEI. Future research can extend the model used in this study to predict the intention of continuous usage and compare the results to the intention of adoption. Follow-up research should build a more comprehensive model by integrating moderating variables to predict adoption intentions. The research findings could be investigated in a future study using qualitative methods. Different paradigms may produce different results, which may be valuable in explaining chatbot adoption in HEIs.

Furthermore, comparing the results using quantitative and qualitative methods may give researchers information to improve the quality of the data collection instrument such as focus group interviews. The longitudinal technique should also be used by future researchers to forecast adoption intention over time. As a result, the model needs to be evaluated over time. Future research should, for instance, examine adoption intention in different periods such as before and after the adoption of a chatbot.