Adopting Artificial Intelligence and Artificial Reality in an Interactive Sign Language Learning System: Acceptance of Interactive Technology ^†

Abstract

Growing awareness of inclusivity for people with disabilities has led the National Taiwan Library to promote digital and interactive approaches for improved communication between sign language users and the hearing community. The “Interactive System of Digital Sign Language Picture Books”, developed with augmented reality (AR) and artificial intelligence (AI), enhances inclusive learning experiences. In this study, 82 participants with hearing disabilities aged 31–60 used the system and completed a technology acceptance model questionnaire. Results indicate that ease of use positively influences perceived usefulness, attitude toward use, and intention to use, supporting enriched learning and fostering social communication.

1. Introduction

The awareness and acceptance of being inclusive toward individuals with disabilities have been increasing. Taiwanese Sign Language has been officially recognized as a tool for communication as it safeguards cultural and linguistic heritage and raises the awareness of culture and language for individuals with hearing disabilities. Beyond the essential support, the public must understand the value of inclusivity.

1.1. Technology in Education

The acceptance of educational technologies, particularly those to enhance language learning, has been a subject of research recently. Technology acceptance models (TAMs) and their applications provide frameworks for understanding users’ attitudes and intentions toward new technologies. Perceived ease of use (PEOU) and perceived usefulness (PU) are key determinants of user acceptance [1]. When users perceive a technology as easy to use and useful, they are more likely to adopt it for educational purposes [2,3].

Innovations in sign language technology improve communication, accessibility, and inclusivity for the hearing-impaired community, underscoring the goal to enhance sign language learning and support the hearing-impaired [4]. Incorporating the perspectives of the hearing-impaired is crucial; their engagement with the community is beneficial to them [5].

1.2. Technological Innovations in Sign Language Learning

Augmented reality (AR) and artificial intelligence (AI) are used to create innovative tools for learning sign language [6,7,8,9]. Such advanced technology offers unique opportunities to enhance the interactivity and effectiveness of educational resources, particularly for individuals with hearing impairment. For example, AR has been adopted in learning sign languages in English [10], Indonesian [7], and Peruvian [11]. Several applications have been developed to facilitate communication between the hearing and the hearing-impaired by adopting AI technologies and sign language capturing. Yet, it has been developed in English, Hindi, Thai, Greek, Korean, and Cantonese [12]. However, state-of-the-art technologies have not been adopted to fulfill the needs of the hearing-impaired in Taiwan.

Therefore, we developed an “Interactive System of Digital Sign Language Picture Books” that integrates cutting-edge technologies to enhance the learning experience for the hearing and the hearing-impaired. The system uses AR to enable users to interact with animated content more engagingly. AI technology focuses on sign language recognition to provide learners with immediate feedback on the accuracy of their sign language. This real-time feedback helps learners understand and correct their signs, thereby enhancing the learning experience. By combining AR and AI, the system improves the visual and interactive aspects of sign language education and supports an inclusive reading environment that caters to diverse user needs and abilities.

2. Materials and Methods

To foster a sustainable learning environment, the system was developed in Taiwanese Sign Language (Figure 1). For testing the system, 82 participants were recruited, including sign language instructors and students. The participants included the hearing and the hearing-impaired who had an interest in learning sign language.

Participants began by selecting a story of interest from the main page of the system. Upon selection, they proceeded to watch a video that featured storybooks accompanied by sign language representations embedded within the content. Following the viewing, participants engaged in sign language practice by signing in front of the camera, which was initiated by clicking a designated green button on the user interface. The AI-driven recognition system subsequently analyzed their signing performance and provided immediate feedback, presenting the accuracy of their signed vocabulary as a percentage (Figure 2).

TAM was employed to examine learners’ acceptance of the system and identify the perceived usefulness and perceived ease of use and their influence on users’ attitudes toward and intentions. The results were used to predict users’ behavior in the successful implementation of technology [1].

Before using the system, the participants were informed about the study’s objectives and procedures. Each participant used the system for 10−15 min. Then, they completed a questionnaire to assess their acceptance of the system. The collected data were analyzed using structural equation modeling (SEM) to conduct factor and path analyses to predict the user acceptance of the system. A survey questionnaire was developed for this study, encompassing four constructs and 14 items. The items were tailored to align with the objectives of this research. The theoretical framework of TAM includes four constructs: perceived usefulness (PU), perceived ease of use (PEOU), attitude toward using (ATU), and behavioral intention to use (BIU), which are used to predict the actual system use (ASU) by the users.

In the questionnaire, the perceived usefulness (PU) construct included four items (1-1 to 1-4); the perceived ease of use (PEOU) construct included five items (2-1 to 2-5); the attitude toward using (ATU) construct included three items (3-1 to 3-3); and the behavioral intention to use (BIU) construct included two items (4-1 to 4-2). A five-point Likert scale ranging from (1) “Strongly Disagree” to (5) “Strongly Agree” was used to score the items. The constructs and items are detailed in

Table 1. Constructs and items in the questionnaire on this study.

TAM Construct		Items
Perceived usefulness	PU1	This sign language learning system helps me learn sign language better.
	PU2	This sign language learning system allows me to learn sign language more quickly.
	PU3	This sign language learning system makes it easier for me to learn sign language.
	PU4	I find this sign language learning system very helpful for learning sign language.
Perceived ease of use	PEOU1	Learning to operate this system is easy for me.
	PEOU2	Interacting with this sign language learning system is clear and easy to understand.
	PEOU3	I find this sign language learning system to be very versatile in its interactivity.
	PEOU4	Mastering the use of this sign language learning system is easy for me.
	PEOU5	Learning sign language using this system is easy for me.
Attitude toward using	ATU1	I think using this sign language system to learn sign language is a great way.
	ATU2	This system makes learning sign language more enjoyable.
	ATU3	I enjoy using this system to learn sign language.
Behavioral intention to use	BIU1	I plan to continue using this system for teaching.
	BIU2	I will continue using this system for teaching.

. Data collection was conducted electronically through an online survey.

The participants completed questionnaires using a computer, tablet, or mobile phone. The collected data were analyzed using JASP and SmartPLS 4 for descriptive statistics and structural equation modeling (SEM) to explore relationships among latent variables of TAM. SmartPLS 4 was chosen for SEM analysis due to its availability for small sample sizes and its capability to handle complex causal models involving multiple constructs. Partial Least Squares (PLS), a method within SEM, was appropriate for this study as it has predictive power and model interpretability.

3. Results

In SmartPLS, the measurement model represents the first stage of testing. In this stage (the outer model), convergent validity and discriminant validity are determined to see whether the items in each construct reflect that construct. The second stage involves testing the inner model to verify the proposed hypotheses and evaluate the explanatory power of the constructs.

The descriptive statistics are presented in

Table 2. Demographic Analysis.

		Participants (N = 82)	Percentage (%)
Gender	Male	21	25.6
	Female	61	74.4
Age	Under 18	3	3
	18–30	3	6
	31–40	19	23.2
	41–50	29	35.4
	51–60	24	29.3
	61+	4	4.9
Hearing Status	Deaf	41	50
	Hard of Hearing	1	1.2
	Hearing	40	48.8
Occupation	Sign Language Teacher	62	75.6
	General Public	20	24.4

. Females outnumbered males at 74.4% and 25.6% of the total number of participants. The majority of participants were 31−60 years old. The ratio of the hearing and hearing-impaired was equal.

Descriptive statistics for the four constructs in this study are presented in

Table 3. Descriptive statistics for each construct.

TAM Constructs	Mean	SD
Perceived usefulness	4.637	0.523
Perceived ease of use	4.593	0.566
Attitude toward using	4.707	0.504
Behavioral intention to use	4.537	0.659

. The average scores for each construct are all above 4.5, with the highest average of 4.707 observed in “Attitude Toward Using” and the lowest average of 4.537 in “Behavioral Intention to Use”.

3.1. Outer Model

In the outer model, internal consistency was assessed using Cronbach’s alpha (

Table 4. Reliability and validity analysis values of construct items.

TAM Constructs	Mean	SD	Factor Loadings	Cronbach’s Alpha (α)	CR	AVE
PU1	4.70	0.46	0.832	0.877	0.887	0.730
PU2	4.63	0.48	0.894
PU3	4.61	0.56	0.846
PU4	4.61	0.58	0.845
PEOU1	4.59	0.60	0.892	0.905	0.908	0.726
PEOU2	4.54	0.61	0.862
PEOU3	4.60	0.54	0.768
PEOU4	4.59	0.56	0.859
PEOU5	4.69	0.50	0.874
ATU1	4.67	0.52	0.861	0.889	0.892	0.819
ATU2	4.77	0.42	0.936
ATU3	4.68	0.56	0.915
BIU1	4.57	0.63	0.924	0.822	0.822	0.849
BIU2	4.50	0.69	0.919

). A reliability level of 0.7 or higher is recommended [13]. Cronbach’s alpha of the constructs exceeded 0.7, ranging from 0.877 to 0.905. Composite reliability (CR represents the reliability of all measurements. The higher the CR value of a latent variable, the better the items measure the latent variable, indicating higher internal consistency of the construct [13]. In this study, the CR of all constructs exceeded 0.7, ranging from 0.822 to 0.908.

CR and average variance extracted (AVE) were used to assess the reliability of constructs. If these values meet the standard thresholds, it indicates good construct reliability [14]. AVE measures the extent to which a latent variable explains the variance of its indicators. A higher AVE value indicates a greater degree of variance explained by the latent variable [15]. AVE must be greater than 0.5, with 0.36−0.5 being an acceptable range. A higher AVE value signifies a higher convergent validity of the construct [16]. In this study, AVEs were higher than 0.5, indicating sufficient convergent validity for each construct (

Table 5. Analysis Values of Cronbach’s α for each construct.

TAM Constructs	Numbers of Items	Cronbach’s α Value
Perceived usefulness	4	0.872
Perceived ease of use	5	0.877
Attitude toward using	3	0.879
Behavioral intention to use	2	0.925

).

Discriminant validity is used to assess the extent to which measurement variables distinguish between different constructs, indicating low correlation between items from different constructs. According to the Fornell–Larcker criterion, the square root of the average variance extracted (AVE) for each latent construct should be greater than the correlation coefficients between that construct and other constructs, demonstrating discriminant validity between constructs [16,17]. The discriminant validity of the four constructs investigated in this study is shown in

Table 6. Discriminant validity for each construct (Fornell–Larcker criterion).

	ATU	BI	PEOU	PU
ATU	0.905
BI	0.815	0.921
PEOU	0.839	0.726	0.852
PU	0.659	0.620	0.687	0.854

. The square root of AVE for each latent construct exceeds the correlation coefficients between that construct and other constructs, indicating discriminant validity among the constructs.

However, Henseler et al. [18] argue that the Fornell–Larcker criterion may lack sensitivity, thus proposing the heterotrait–monotrait ratio of correlations (HTMT) as an alternative index to assess discriminant validity between constructs. According to their recommendation, the HTMT values between constructs should be less than 1 to indicate discriminant validity. In this study, all HTMT values between constructs are below 1, demonstrating discriminant validity (as shown in

Table 7. Discriminant validity for each construct (HTMT criterion).

	ATU	BI	PEOU	PU
ATU
BI	0.951
PEOU	0.934	0.840
PU	0.737	0.721	0.757

).

3.2. Structural Model/Inner Model: Hypothesis Verification and Analysis Test

Structural model (inner model) analysis includes (1) model fit analysis, (2) overall model explained variance (R²), (3) path coefficients, and (4) collinearity assessment.

3.2.1. Model Fit Analysis

Model fit analysis assesses the goodness of fit using the standardized root mean square residual (SRMR), where a good fit is indicated by an SRMR value below 0.08 or 0.1 [19]. In this study, the SRMR value for the model is 0.075 (see

Table 8. Model fit.

	Saturated Model	Estimated Model
SRMR	0.075	0.075
D_ULS	0.590	0.592
NFI	0.781	0.782

), indicating a good model fit.

3.2.2. The Explained Variance (R²) of the Overall Research Model

The explained variance (R²), also known as the coefficient of determination, is used to explain the goodness of fit in a linear regression model. It indicates the causal relationships between the latent variables in the model, reflecting the extent to which the independent variables can explain the dependent variable. This metric can represent the in-sample predictive power and is commonly used as a criterion to judge the quality of a model. However, since R² can be influenced by the number of paths from other constructs, it is prone to bias. Therefore, the adjusted R² is often used for evaluation. The R² value ranges from 0 to 1, where a smaller value indicates poorer explanatory power, and a larger value indicates better explanatory power. A value of 0 means the independent variables are completely unrelated to the dependent variable, while a value of 1 means the independent variables can fully explain the dependent variable. The critical values for R² are 0.67, 0.33, and 0.19, which represent different levels of explanatory power for the model: substantial, moderate, and weak, respectively [20,21].

In the context of this research model, the constructs of attitude toward use (ATU) and behavioral intention (BI) exhibit substantial explanatory power, while the construct of perceived usefulness (PU) demonstrates moderate explanatory power (

Table 9. Explained Variance of the Structural Model.

Construct	R2	Adjusted R2
ATU	0.717	0.710
BI	0.676	0.668
PU	0.473	0.466

).

3.2.3. Path Coefficient

Collinearity assessment utilizes the variance inflation factor (VIF) to evaluate collinearity among constructs. A VIF value should be less than 5 to indicate that there is no collinearity problem among the constructs. In this study, the VIF values range between 1 and 1.89, all of which are below 5, indicating no collinearity issues (see

Table 10. Structural equation modeling results.

Hypothesis	Hypothesis Paths	Results
H1	PEOU → PU	Supported
H2	PEOU → ATU	Supported
H3	PU → ATU	Not supported
H4	PU → BIU	Not supported
H5	ATU → BI	Supported

).

3.2.4. Collinearity Assessment

Collinearity assessment utilizes the variance inflation factor (VIF) to evaluate collinearity among constructs. A VIF value should be less than 5 to indicate that there is no collinearity problem among the constructs. In this study, the VIF values range between 1 and 1.89, all of which are below 5, indicating no collinearity issues (see

Table 11. The structural equation modeling results of this study.

Hypothesis	VIF	Path Coefficients	SD	T Statistics	p Value	Confidence Interval
H1	1.000	0.687	0.074	9.280	0.000	[0.536, 0.825]
H2	1.896	0.733	0.094	7.762	0.000	[0.531, 0.905]
H3	1.896	0.155	0.110	1.414	0.157	[−0.049, 0.380]
H4	1.767	0.147	0.086	1.720	0.086	[−0.020, 0.318]
H5	1.767	0.718	0.072	9.910	0.000	[0.566, 0.850]

).

The research hypotheses and their path coefficients are shown in Figure 3. The arrows on the figure represent the paths, and the numbers on the paths indicate the path coefficients. An asterisk following a path coefficient indicates that the path has a significant difference.

The research results show that the path coefficients for H1, H2, and H5 are positive. Perceived ease of use (PEOU) positively influences perceived usefulness (PU), as shown by path H1 (p < 0.001), and perceived ease of use (PEOU) positively influences attitude toward use (ATU), as shown by path H2 (p < 0.001). Thus, hypotheses H1 and H2 are supported. Additionally, attitude toward use (ATU) positively influences behavioral intention (BIU), as shown by path H5 (p < 0.01), supporting hypothesis H5.

However, perceived usefulness (PU) does not have a significant positive influence on attitude toward use (ATU), path H3 (p > 0.01), nor on behavioral intention (BIU), path H4 (p > 0.01). Therefore, hypotheses H3 and H4 are not supported. This indicates that perceived usefulness (PU) does not have a significant positive influence on attitude toward use or behavioral intention.

4. Discussion

The research results indicate multiple positive relationships regarding learners’ acceptance of technology when interacting with the sign language picture book system developed in this study. Firstly, three hypotheses are supported: the perceived ease of use of the system for learning sign language positively influences perceived usefulness (H1), perceived ease of use positively influences attitude toward use (H2), and attitude toward use positively influences behavioral intention (H5). This suggests that if learners find the system easy to use, they will also perceive it as useful. Moreover, when learners consider the system useful, it positively affects their attitude toward using it and their behavioral intention to use it. However, the study results show that the system’s perceived usefulness (PU) does not significantly positively impact attitude toward use (ATU) and behavioral intention (BIU). The possible reasons for this are as follows: (1) the sign language recognition accuracy of the system needs improvement, (2) the content of the sign language materials should provide thematic dialogue learning scenarios, and (3) a search function should be provided to allow users to look up content by entering keywords. Additionally, because the current teaching content is more suitable for learners with some foundational knowledge, it is recommended that the materials be categorized by different levels of difficulty. For beginners, the sign language actions and speed in the videos might be too fast, so it may be necessary to slow down the speed and provide more demonstrations in the video content. These suggestions aim to enhance users’ perceived usefulness of the system.

5. Conclusions

The development of the Interactive System of Digital Sign Language Picture Books contributes to enriching the reading and learning experience of deaf and hearing-impaired communities. Furthermore, it could promote social inclusion by facilitating better communication between deaf and hearing individuals and raising awareness of sign language in broader society. Evaluating the interactive system using the technology acceptance model (TAM) has provided us with valuable insights into how target users perceive the system for learning and reading through Taiwanese Sign Language.

The limitations of this study include the lack of examination of external variables influencing the acceptance of the sign language picture book system. Examples of such variables could include instructional content design, information or system quality, self-efficacy, enjoyment, or gamification, among others. Therefore, future research is recommended to incorporate these external variables to further explore the system’s acceptance.

Additionally, since participants were all first-time users of the system developed in this study, acceptance may vary over time with prolonged system use. Factors such as improvement in sign language proficiency or increased familiarity with system usage could influence acceptance. Therefore, future studies could also delve deeper into the impact of external variables on the acceptance of this system.