The Construction of an Evaluation Index System for Assistive Teaching Robots Aimed at Sustainable Learning

Abstract

A typical example of a robot used in education is the assistive teaching robot, which has much potential to enhance teaching and learning as well as to promote sustainable learning. However, there needs to be formalized selection and evaluation procedures for robotic teaching assistants. To address this need, this paper presents a function evaluation system framework for assistive teaching robots, which includes four dimensions: system structure, appearance interface, teaching function, and auxiliary support. This study employed the framework of the DANP method to examine the extent of influence of various indicators. The analysis focused on determining the critical components of the function evaluation system for assistive teaching robots. Ultimately, the study concluded that two crucial factors in this evaluation system emerged: teaching function and auxiliary support. These two aspects are also key elements in promoting sustainable learning. Moreover, recommendations are made for designing and selecting suitable assistive teaching robot products, aiming to serve as an exemplary framework for future product development and implementing educational activities within school settings, while further contributing to the realization of sustainable learning.

1. Introduction

With the arrival of ChatGPT 4.0, people have felt the power and convenience of natural language processing technology. The popularity and advancement of artificial intelligence technology have been significantly aided by the open source and promotion of ChatGPT [1]. Because ChatGPT is a substantial language model based on deep learning technology, it is regarded as a significant advancement in artificial intelligence techniques. Moreover, since robots themselves belong to the broad realm of AI technology applications, they will rightfully benefit from the technological progress [2]. To develop students’ analytical, creative, and practical skills, robots suitable for structured and practical disciplines such as Science, Technology, Engineering, and Mathematics (STEM) fields are called educational robots. These robots have the qualities of teaching applicability, openness, scalability, and friendly human–computer interaction [3], and their introduction into the educational field, which focuses on structured learning and the development of practical skills, can contribute to the development and transformation of education to a greater extent [4], and also help to promote the achievement of sustainable learning [5]. For example, in English classes, robots are used to help students practice speaking, listening, and grammar, and to provide personalized learning advice [6]; Cozmo robots, designed by Anki, provide students with an open programming environment [7]. Google Classroom has begun to use educational robotics to provide students with online learning tools and resources [8]. These robots are able to provide a more personalized learning experience by automatically adapting the learning material and level of difficulty based on students’ performance and need as they interact with them. This approach not only makes it easier for students to acquire knowledge but also enhances their ability to learn independently and continuously. Social education has long promoted a commitment to developing people’s willingness and ability to learn, so that they can continually adapt to new demands for knowledge and skills and improve themselves. The integration of technology through artificial intelligence can bring new opportunities and challenges to the education field, which focuses on structured learning and developing practical skills [9], offering new possibilities for the development of sustainable learning and helping society to support the growth of the younger generation.

At the same time, it is important to note that while educational robots can enhance students’ interest and ability to learn, they cannot completely replace the human interaction and emotional communication provided by human teachers [10]. Educational robots excel at providing personalized learning support, helping students understand concepts, answering questions, and automating assessment and feedback. However, the scenarios in which robots can be used are limited when it comes to complex emotional support, ethics and moral education, and other domains [10]. These domains typically require deep human interaction and emotional communication from human teachers, as well as understanding and respect for individual differences.

Since they make up the majority of robots used in educational robotics applications, assistive teaching robots will receive most of the users’ and makers’ attention [11]. Assistive teaching robots are also known as robots that assist teachers with classroom support or repetitive tasks [12]. Humanoid robots such as NAO, SAYA, Bioloid, Aisa (ASIMO), Eddie, RoboTutor, and so on, play a significant role as teaching assistants in academic environments [13,14]. At present, robots come in many different varieties, with a wide range of products on the market and very different areas of application. From the point of view of robot manufacturing, although there is a flourishing trend in the market, there is an obvious homogenization in terms of the underlying technology, modeling, functions, and so on [15,16], which leads users to wonder which robot to use in the teaching practice process or whether they should just heedlessly follow the trends. As a result, it is not possible to effectively meet the market’s diverse needs. The inability to meet different demands has a negative impact on the development and proper use of educational robots. Additionally, it is not conducive to enhancing the sustainable learning ability of the educated. More options are available due to the growing use of assistive teaching robots in education today [17]. While the use of educational robots in relatively structured and explicit educational environments, such as STEM fields and basic subjects, can have a positive impact, we still need to be aware of their limited nature. When evaluating the use of educational robots, we need to focus on both their positive and potential negative impacts. So an evaluation framework that can support efficient development and assessment is needed to help with the selection. However, the lack of a framework with extensive coverage and a high degree of application and the fragmented functional evaluation criteria in the current literature have prevented this issue from being fully addressed [12,18,19].

This study proposed a functional evaluation paradigm with broad applicability to close this research gap. The Decision-Making Trial and Evaluation Laboratory (DEMATEL) and the Analysis Network Process (ANP) method, also known as the DANP method, were combined in this study’s hybrid, multi-criteria decision-making (MCDM) model to create interdependencies between evaluation dimensions and criteria [20,21,22]. Although the use of robots in the classroom has been studied in the past, such as the use of the NAO robot to help children learn English [23], Zhexenova et al. (2020) [24] found that students’ writing skills and confidence improved after interacting with the CoWriter robot. There are also studies that have introduced the SAYA robot into the classroom as a support for a teleclassroom system [25]. In general, these studies have focused on specific applications and there is still a lack of methods to support their development and evaluation. At the same time, with continuous learning as the key to personal growth and development, and its close relationship to the sustainable development of society and the long-term educational goals of social education, the ability of educational robots to promote continuous learning has become a major concern. It is necessary to organize and develop a set of functional evaluation metrics to direct the development of assistive teaching machine goods and to guide the selection of robots in schools.

The following sections make up the remaining portions of this study. Section 2 describes the creation of the evaluation metrics, and the pertinent literature is reviewed. Section 3 explains how the influential network relationship map (INRM) impacts relationships, and how criterion weights were generated using the DANP method. The data are analyzed in Section 4. The relevant discussion is found in Section 5, and some concluding remarks are presented in Section 6.

2. Literature Review

The advent of the digital era and developments in information technology and engineering science has led to the widespread use of robots and related products in various industries. Robots can perform tedious, complicated, and even dangerous tasks more accurately than people [26]. In recent years, robots have also begun to be progressively used in education [27]. While there is an intersection between robotics and its related components and products and the sustainable development of education [28], robots positively impact the ability of students to continue learning in a shorter period of time by providing personalized learning support [8], assessment of learning outcomes and massive learning resources [29,30], robots improve students’ learning effectiveness in a shorter period of time [31], thus actively promoting the sustainable development of education in the whole society. Correspondingly, the improvement of educational effectiveness also gives impetus to the continuous iteration and rapid development of robotics [32]. These two elements mutually stimulate and reinforce each other, creating a symbiotic relationship that drives progress.

2.1. Current Status of Research

Educational robots are intelligent systems or bodies of intelligence that integrate information technology and innovative technologies like computing, sensing, networking, and artificial intelligence to achieve digital modeling and computation of knowledge systems, educational participants, educational scenarios, and educational processes [33]. With the ability to help with teaching, manage to teach, handle teaching, and even host teaching [34], educational robots can create an engaging and interactive learning environment by offering students intriguing activities and unique engagement opportunities for class participation [35]. Interacting with educational robots can meet the different learning needs of students as the robots can provide personalized educational content and support [8]. As a result, educational robots are seen as a valuable tool for raising students’ motivation, interest, and academic performance [36].

With the rapid development of artificial intelligence and natural language processing technologies, educational robots in teaching can be broadly classified into three categories: assistive teaching robots, companion robots, and teaching tools [37].

Robots can perform on a par with or even better than humans in perception, memory, arithmetic, continuous work, and concentration [38]. By storing student learning data in the system, robots can pace teaching based on a student’s proficiency, helping human teachers with labor-intensive repetitive tasks. Robots are thus most frequently used as teaching assistants in educational settings. Using assistive robots in the teaching and learning process helps to alter some of the work teachers typically do. When we look at students’ access to knowledge, it is easy to see that the opportunities for one-to-one interaction between students and teachers are quite limited in the traditional general classroom environment. This limitation restricts students’ individuality and their ability to develop a deeper understanding of knowledge. In a class with many students, the teacher has to cover a large amount of content in a limited amount of time, making it difficult to provide enough individualized instruction and interaction for each student. Assistive teaching robots can solve teacher shortages [38]. According to Robaczewski and other scholars, NAO robots have been widely employed in education and can aid novices in learning through several built-in programs [39]. They also observed that NAO robots contain motor, functional, and emotional qualities enabling them to perform well as assistive robots. On the one hand, some researchers used the Android Robot SAYA and the Madatech robot to compare the effectiveness of robot-assisted teachers within the same course [40]. They ultimately discovered that most students could actively participate in learning activities and interact with the robot assistants throughout the system with the help of the robot. By employing the ASIMO robot to teach storytelling to students, Costa, and other scholars have discovered that the humanoid robot’s gaze behavior significantly aided in developing the students’ narrative abilities [41]. Some scholars use robots as a tool to assist classroom teaching, using the robot’s language recognition and facial expression recognition functions to help students and teachers complete classroom interactions.

In education, the nature of teaching and learning is much more than the transfer of knowledge. One of the aims of education is to develop students’ generic skills, which include critical thinking, creative thinking, problem-solving [42], and a sense of self-directed learning. Although robots excel in perception, memory, persistence, and concentration, they rarely cover all dimensions of education, such as emotions, morality, and social interaction [43]. This is precisely why robots can only be used as tools to assist teachers in the teaching and learning process, where teachers are able to impart human values, inspire students to think creatively, and guide them to become socially responsible citizens. Assistive teaching robots support teachers in terms of knowledge transfer and practice correction, but their greater significance lies in freeing up teachers’ time and energy to focus on those aspects of education that require the involvement of human emotion, intelligence, and flexibility. Robots do not exist to replace teachers [44], but rather to play to their strengths in education and provide more support to teachers, enabling them to guide the development of their students in a more profound way.

Meanwhile, in modern society, knowledge is updated at a rapid pace and students need to have the ability to learn continuously and adapt to changes [45]. Assistive teaching robots help to develop students’ sense of independent learning and continuous learning ability through personalized learning plans, continuous learning tracking, and resource support [46]. This is a gradual process toward lifelong learning, enabling students to adapt to future changes and challenges. Therefore, the use of assistive teaching robots in education is not only to improve efficiency, but also to better meet the diversity of education and the comprehensive needs of students. As part of educational technology, they work with teachers to create a more dynamic and interactive learning environment, helping to cultivate future talents with comprehensive skills.

While robotics facilitates sustainable learning [47], and the number of situations where assistive teaching robots are used in the classroom is growing, necessitating suitable methods to create and assess these robots [19]. There needs to be more information regarding the options for assistive teaching robots in the research on developing systems for evaluating the functions of robots. Most of the existing content primarily concentrates on the application of industrial robots [48]. Therefore, it is hoped that by drawing on the development of an evaluation system for industrial robots, an evaluation system that is practical and appropriate for evaluating educational robots can be explored.

It is not easy to choose the best industrial robot because of the increasing complexity of robotic systems and the growing diversity of robots with varied roles, features, and specifications on the market [49]. Most currently used selection techniques for creating functional evaluation metrics in the literature on industrial robots use the Multi-Criteria Decision-Making (MCDM) model. An MCDM refers to a situation with a limited or infinite collection of decisions that cannot be shared and have conflicting solutions. The fundamental goal of this strategy is to arrive at a value judgment that is based on consensus [50,51]. The Analytic Hierarchy Process (AHP) is a multi-criteria technique that is among the most well-known and significant. Numerous factors, including positioning accuracy, cost, flexibility, load capacity, human–machine interface, and vendor service quality [48], must be taken into account during the industrial robot selection process [52,53]. Therefore, the selection of industrial robots, in general, can be considered a multi-criteria decision problem, and manufacturers and business organizations can use the MCDM approach to resolve various problems with robot selection evaluation [54]. The most well-known MCDM methodology, AHP, was first put forth by Saaty (1980) [55] for assessing, rating, ranking, and evaluating decision selection of one of the most widely used MCDM techniques. The fundamental steps of the AHP method include laying out a complex decision problem as a hierarchy, using pairwise comparison techniques to estimate the relative importance of each element at each level, and finally integrating these significant issues to create an overall evaluation system for choosing a decision solution [56].

By determining the weights of various levels of criteria through AHP to determine the performance of various robot designs, Goh (1997) [57] and Geng (2013) [58] evaluated the value of various industrial robot alternatives. An integrated model combining Analytic Hierarchy Processing (AHP) and Quality Function Deployment (QFD) was presented to determine whether the performance of the functional deployment of robots in the industry improves from the standpoint of needs [56]. Wang et al. (2022) [59] used a linguistic assessment scale based on a spherical fuzzy set, SF-AHP, to allow decision makers to freely express their significance in decision making. Kapoor and Tak (2005) [60] proposed a method of using “fuzzy linguistic variables” instead of numbers to solve the industrial robot selection problem by combining the Analytic Hierarchy Process (AHP) and Multi-Attribute Utility Theory (MAUT), two conventional, multi-criteria decision-making techniques. They used a fuzzification process by tying linguistic variables to the values of membership functions and developing suitable decision rules. Defuzzification, the final step in the process, converts fuzzy outputs into exact values and produces results as fuzzy scores.

AHP is frequently utilized as an MCDM technique for evaluating industrial robot functions since it is predicated on the idea that the dimensions and criteria of a system are independent [61]. The dimensions or criteria of the evaluation system rarely function independently in practice; instead, they frequently interact. The Analytical Network Process (ANP), a development of the AHP methodology, has also successfully resolved several real-world decision problem-solving processes, including project selection, product planning, supply chain management, and optimal scheduling. A better-organized functional evaluation and selection metrics system must choose the best robot for a particular domain and analyze fuzzy selection choices. DEMATEL can be used by producers, decision makers, and users as a field decision-making guide when combined with ANP approaches.

2.2. The Functional Evaluation Index System of Assistive Teaching Robots

In order to select suitable robots for teaching and to evaluate the effectiveness of robots in facilitating sustainable learning, it is necessary to consider robotics in different educational scenarios [30] and the corresponding teaching standards, teaching objectives, content and teaching environments. Even though earlier studies have emphasized educational level and learning subjects as two crucial factors in selecting educational robots, there is still a lack of an implementable evaluation framework to guide future research [21,22,23,24,25,26,27]. Most studies have concentrated on theory, design, development, practice, and reflection, but there has yet to be much academic research on the functional analysis and evaluation of educational robots. The significance of the educational purpose is the aspect that needs to be focused on in the case of constrained educational costs for assistive teaching robots, according to some related scholars who have made this point in their research [38]; Chang et al. (2010) [62] argue for focusing on the appearance and body construction of teaching assistants robots, with movable robotic arms or suitable display positions designed to enable a variety of interactions between teachers and students; Tsiakas et al. (2018) [63] divided the four functional evaluation metrics for assistive teaching robots into perceptual functions, behavioral control, classroom assistance, and personalization to assess the effects of robot appearance, verbal and nonverbal behavior, service, and communication style on educational instruction. To further improve the autonomous capabilities of the assistive teaching robot, Cooney and Leister (2019) [38] built a more comprehensive functional evaluation system through an exploratory experimental study of the robot’s capabilities. Other researchers used a structural equation modeling approach to design a quantitative model of the instructional design model and proposed a functional design framework for robot-assisted instruction based on constructivism [64]. The network relationships of the assessment system and the effect relationships between evaluation indicators, however, have not been taken into consideration by related research, which proposes a few evaluation dimensions on the surface.

The quality evaluation index system for the assistive teaching robot was developed in this study based on a substantial amount of domestic and international literature on the test evaluation and related design of the assistive teaching robot in four aspects: system structure, appearance interface, teaching function, and auxiliary support. The 17 secondary indicators under the dimension level are as follows:

2.2.1. System Structure (X₁)

A robotic system comprises hardware, operating system software, and a physical body. The robot is a typical system, and although it should be highly reliable in theory, Merlet (2009) claimed that variations in how the mechanical components of the robot were made inherently cause uncertainty in the robot. As a result, some analytical methods are required to guarantee that the robot is reliable in a pedagogical use environment [65]. Functional diversity and ease of use can improve teaching effectiveness and expand the range of services teaching assistants offer, making the robot more manageable and structurally compact. According to Yoshino and Zhang (2018) [66], the reliability and security of the system should be the primary considerations when evaluating the indicators, because they can improve the user experience when the operation of the robot system is stable, the authentication service system is comprehensive, and the firewall has security protection. The user experience is impacted by how quickly system functions respond, according to Yu et al. (2020) [67] and a related article published by the China Education Equipment Industry Association.

Furthermore, easy-to-use systems and programs can maintain their appeal to students and their interest and motivation to learn. When utilizing the ROBOSEM robot as an example to emphasize the relevance of switching between remote operation and autonomous control features given by assistive teaching robots, Park et al. (2011) [68] underlined that system flexibility may lead to a superior qualitative experience. As a result, according to the literature mentioned above, the system structure dimension’s metrics include system reliability, security, flexibility, functional diversity, and operational convenience.

2.2.2. Appearance Interface (X₂)

Robots have been divided by Fong and other scholars into four categories based on their appearance, namely, anthropomorphic, exaggerated, caricatured, and functional [12]. It turns out that children under the age of 9 pay close attention to the appearance of robots. Children’s attitudes toward robots that resemble humans are better than their attitudes toward robots that are simply machines [69], but support for those robots that closely resemble humans has declined dramatically. Ryu et al. (2007) [69] explored the design function of assistive teaching robots and found that the robot’s physical characteristics and role models significantly impacted educated people. As a result, the esthetically pleasing, comprehensive, and compact design of the outer structure is a critical assessment criterion. According to Yang and Wang’s (2020) [70] hypothesis, the interactive interface is the element that people find most intuitive in determining the level of service quality and customer happiness, and the ease with which an interface’s features may be used directly impacts the user experience. According to a related article by Tsiakas et al. (2018) [63], custom extension plugins can increase the diversity of the educational process. According to the Technical Specification for Teaching Robots in Primary and Secondary Schools, teaching assistants should have a modular structure and flexibly mountable and expandable external device interfaces. To better assess the appearance of the user interface, the degree of expansion of the equipment is also taken into account in this study.

2.2.3. Teaching Function (X₃)

Huijnen et al. (2017) [71] suggest that assistive robots need to provide students with content that is compatible with their learning methods and study habits, improve student learning through personalized instruction, and get them more actively involved in learning. Appropriate use of robotics in the teaching and learning process is expected to improve the effectiveness of teaching and learning, enhance students’ motivation and interest in learning [72], and ultimately contribute to the promotion of lifelong sustainable learning. Therefore, the teaching function of assistive teaching robots has become one of the most important dimensions to measure its functionality. Many schools have introduced educational robots as teaching aids, such as the Pepper robot, which is able to interact with students, introduce course content and answer questions [73]. Teaching functionality, UI appearance, and assistive support are the three main factors that determine the quality of an assistive teaching robot. As a result, the instructional design content that the assistive teaching robot holds must follow the rules of effective teaching and learning, such as normality, applicability, and completeness. According to Wu et al. (2015) [74], because educational robots must have higher levels of intelligence and a greater variety of knowledge, assistive teaching robots must be able to create various lesson plans to suit curriculum variations. To be an effective teaching assistant, robots should assist teachers in the classroom by adjusting the pace of teaching and learning to create a more efficient and effective teaching environment [75]. They should also be able to meet the needs and preferences of different students according to their diverse characteristics. According to the literature listed above, the teaching function dimensions include teaching content standardization, teaching content applicability, teaching content integrity, and teaching process control.

2.2.4. Auxiliary Support (X₄)

According to Hong and Huang (2016) [76], the RALL system framework, which offers teachers aids, scripted materials for writing, scripted materials that can be easily imported into multimedia resources, and scripted presentations for anthropomorphic robots, can improve the resource management capabilities of the assistive teaching robot. According to Cross et al.’s (2009) [77] theory, a robot’s expression, language, actions, emotions, and ability to communicate pain are distinctive and dynamic in human–robot interaction. This ability to express verbal engagement, in particular, impacts the entire classroom. According to Louie et al. (2017) [75], assistive teaching robots are created to address the need for teacher support in the school, particularly in circumstances where teachers struggle to handle things on their own and where teaching assistants are required to be able to perform repetitive tasks and manage sizable amounts of redundant data resources instead of teachers. Park et al. (2011) [68] use ROBOSEM assistive teaching robots to pronounce and assess pronunciation, identify students with RFID tags or markers, or record student credits while retrieving portfolios and voices using microphone radios. It also provides interaction and real-time feedback with students to enhance the learning experience. As a result, regulatory capability, voice interaction capability, resource management capability, and learning support capability are all part of auxiliary support.

The specific reference

Table 1. Literature sources for the assistive teaching robots’ functional evaluation index system.

Dimensions	Indicators	Literature Sources
System Structure (X1)	System Reliability (X11)	[67,68,78]
	System Security (X12)	[69,78]
	System Flexibility (X13)	[66,68,69]
	Functional Diversity (X14)	[6,65,69]
	Operational Convenience (X15)	[66,67,68]
Appearance interface (X2)	Esthetics of the Exterior Structure (X21)	[12,68,79]
	Clarity of Interface Presentation (X22)	[69,70]
	Simplicity of Interface Interaction (X23)	[12,63,70]
	Degree of Equipment Expansion (X24)	[19,63]
Teaching function (X3)	Teaching Content Standardization (X31)	[66,71,72]
	Teaching Content Applicability (X32)	[71,73,74]
	Teaching Content Integrity (X33)	[66,74]
	Control of the Teaching Process (X34)	[71,72,75]
Auxiliary Support (X4)	Regulatory Capability (X41)	[68,75]
	Voice Interaction Capability (X42)	[66,68,77]
	Resource Management Capability (X43)	[76,77]
	Learning Support Capability (X44)	[63,77,78]

for this article’s dimensions and index sources is below.

3. Methods

To provide a system of robotic solutions and selection indicators that meet different educational needs and are oriented towards sustainable learning. This study refers to the relevant research results of Hsu (2012) [80] and Govindan (2015) [81] in this section. In this study, the DEMATEL method is extended by incorporating the network analysis method (ANP) and utilizing the DANP method. This approach allows for a more detailed quantification of factor weights using a comprehensive influence matrix. Additionally, the study investigates the relationships among the elements within the system, thereby providing a deeper understanding of their interconnections. The network relationships between different assessment dimensions are made clear, successfully extending the analytical reach of the DEMATEL methodology and simplifying pairwise comparisons using the ANP method. The steps in its computation process are described as follows.

3.1. The DEMATEL Method

This study uses the DEMATEL method to determine the interdependence of variables/criteria and to reduce the dependency on features representing the underlying system and development patterns. The interrelationships between the elements that went into creating the INRM can also be determined through it [82,83]. The steps of the method are described as follows:

Step 1: Calculate matrix $A$ of the direct relationship average. The average matrix of direct links was calculated using values from 0–4 for “no impact”, “low impact”, “average impact”, “strong impact”, and “very strong impact”. For the expert questionnaire, the average matrix of direct links was utilized. Respondents were asked to rate the degree of direct influence $a_{ij}$ between the two paired criteria using the following scale, and expert ratings were used to assess the relationships between the elements. Afterwards, the same criteria from the matrix of $f$ respondents were averaged to produce the direct relationship averaging matrix. Equation (1) is the matrix $A$, where $n$ is the number of criteria. Thus

$$A=\left[\begin{array}{ccccc}{a}_{11}& \cdots & a_{1j}& \cdots & a_{1n}\\ ⋮& \cdots & ⋮& \cdots & ⋮\\ a_{i1}& \cdots & a_{ij}& \cdots & a_{in}\\ ⋮& \cdots & ⋮& \cdots & ⋮\\ a_{n1}& \cdots & a_{nj}& \cdots & a_{nn}\end{array}\right]$$

(1)

Step 2: The product of $A$ and $v$ yields the initial direct influence matrix $S={\left[s_{ij}\right]}_{n\times n}$, where $n$ represents the total number of influencing factors in the system and $a_{ij}$ represents the strength of $i$ influencing a factor’s impact over $j$.

$$S=v\times A$$

(2)

$$\mathit{v}=\mathit{max}\left[\underset{\mathrm{i}}{\mathit{max}}{\sum }_{\mathit{j}}^{\mathit{n}}\left|\mathit{dij}\right|,\underset{\mathit{j}}{\mathit{max}}{\sum }_{\mathit{i}}^{\mathit{n}}\left|\mathit{dij}\right|\right]$$

(3)

Step 3: Equation (4) is used to generate the total impact matrix $Z$, and the member $Z_{ij}$ represents the indirect influence of criterion $i$ on criterion $j$. Therefore

$$Z=S+S^2+\cdots +S^k=S{\left(I-S\right)}^{-1}$$

(4)

where, $Z={\left[Z_{ij}\right]}_{n\times n}$ and $I$ is the unit matrix.

Step 4: The matrix $Z$’s column sums ($c_i$) and row sums ($r_i$) are calculated as follows:

$$c_i={\left(c_j\right)}_{n\times 1}={\left(c_j\right)}_{1\times n}^{\prime }={\left[{\displaystyle \sum _{i=\mathit{1}}^n{z}_{ij}}\right]}^{\prime }$$

(5)

$$r_i={\left(r_i\right)}_{n\times 1}=\left[{\displaystyle \sum _{j=\mathit{1}}^n{z}_{ij}}\right]$$

(6)

The overall influence effect that criterion $j$ receives from other criteria is represented by element $c_j$ in vector $c$. Similar to this, $r_i$ indicates how factor $i$ impacts other criteria, both directly and indirectly.

Step 5: Both matrix $Z_C$ and matrix $Z_D$ are derived using the criteria and dimensionality, respectively. Based on the criterion and dimensionality, matrix $Z$ can be distinguished as $Z_C$ and $Z_D$, respectively. Matrix $Z_D$ is produced by averaging the criterion’s level of influence in each dimension.

$${\mathrm{Z}}_{\mathrm{C}}=\begin{array}{c}{D}_1\\ \\ D_2\\ ⋮\\ D_n\end{array}\begin{array}{c}{C}_{11}\\ C_{12}\\ ⋮\\ C_{1m_1}\\ C_{21}\\ C_{22}\\ ⋮\\ C_{2m_2}\\ ⋮\\ C_{n1}\\ C_{n2}\\ ⋮\\ C_{n{m}_n}\end{array}\stackrel{\begin{array}{ccccc}\begin{array}{l}{D}_1\\ C_{11}\cdots C_{1m_1}\end{array}& & \begin{array}{l}{D}_2\\ C_{21}\cdots C_{2m_2}\end{array}& \begin{array}{l}\cdots \\ \cdots \end{array}& \begin{array}{l}{D}_n\\ C_{n1}\cdots C_{n{m}_n}\end{array}\end{array}}{\left[\begin{array}{cccc}{Z}_c^{11}& Z_c^{12}& \cdots & Z_c^{1n}\\ Z_c^{21}& Z_c^{22}& \cdots & Z_c^{2n}\\ ⋮& ⋮& \cdots & ⋮\\ Z_c^{n1}& Z_c^{n2}& \cdots & Z_c^{nm}\end{array}\right]}$$

(7)

Step 6: Get INRM. As a result, while $r_i$ − $c_i$ displays the net influence of factor on the other components, $r_i$ + $c_i$ depicts the intensity of giving and receiving the power of factor $i$. It is obvious that factor $i$ is the causative component if $r_i$ − $c_i$ is positive, and that factor $i$ is the impacted component if $r_i$ − $c_i$ is negative. Therefore, the data set ($r_i$ + $c_i$, $r_i$ − $c_i$) can be mapped to create the INRM.

3.2. Getting ANP Weights Using the DEMATEL Method

This study uses the DEMATEL approach to establish the interrelationships between the elements required to produce the INRM. The steps are briefly listed below. ANP is the procedure for creating unweighted supermatrices and for determining factor weights. ANP, as opposed to AHP, considers the connections and interdependencies between numerous aspects or criteria [84]. The original ANP strategy, however, suffers from three major problems. Before utilizing the network analysis method, one must assume a relational structure of the evaluation system. Second, the complexity of the questionnaire will make it difficult to compare two indications simultaneously [85], which may make the results challenging to grasp. Thirdly, each cluster having the exact same weight is implausible, given that the effect level varies between dimensions or clusters [86].

It should be noted that the DEMATEL-based ANP approach can be used to address these three problems. The unweighted supermatrix is then normalized using the ANP method after the DEMATEL method determines the extent of influence between dimensions. These are the precise steps:

Step 7: Obtain the supermatrix that is not weighted. A matrix $Z_C^{\sigma }$ is obtained by normalizing $Z_C$.

$$Z_c^{\delta }=\begin{array}{c}{D}_1\\ \\ D_2\\ ⋮\\ D_n\end{array}\begin{array}{c}{C}_{11}\\ C_{12}\\ ⋮\\ C_{1m_1}\\ C_{21}\\ C_{22}\\ ⋮\\ C_{2m_2}\\ ⋮\\ C_{n1}\\ C_{n2}\\ ⋮\\ C_{n{m}_n}\end{array}\stackrel{\begin{array}{ccccc}\begin{array}{l}{D}_1\\ C_{11}\cdots C_{1m_1}\end{array}& & \begin{array}{l}{D}_2\\ C_{21}\cdots C_{2m_2}\end{array}& \begin{array}{l}\cdots \\ \cdots \end{array}& \begin{array}{l}{D}_n\\ C_{n1}\cdots C_{n{m}_n}\end{array}\end{array}}{\left[\begin{array}{cccc}{Z}_c^{\delta 11}& Z_c^{\delta 12}& \cdots & Z_c^{\delta 1n}\\ Z_c^{\delta 21}& Z_c^{\delta 22}& \cdots & Z_c^{\delta 2n}\\ ⋮& ⋮& \cdots & ⋮\\ Z_c^{\delta n1}& Z_c^{\delta n2}& \cdots & Z_c^{\delta nm}\end{array}\right]}$$

(8)

A submatrix of $Z_C$ called $Z_C^{pq}$, for instance, can be normalized to $Z_C^{\delta pq}$ as follows:

$$Z_c^{pq}=\begin{array}{c}{c}_{p1}\\ ⋮\\ c_{pi}\\ ⋮\\ c_{p{m}_p}\end{array}\stackrel{\begin{array}{ccccc}{C}_{q1}& \cdots & C_{qj}& \cdots & C_{q{m}_q}\end{array}}{\left[\begin{array}{ccccc}{z}_{11}^{pq}& \cdots & z_{1j}^{pq}& \cdots & z_{1m_q}^{pq}\\ ⋮& & ⋮& & ⋮\\ z_{i1}^{pq}& \cdots & z_{ij}^{pq}& \cdots & z_{i{m}_q}^{pq}\\ ⋮& & ⋮& & ⋮\\ z_{m_{p}1}^{pq}& \cdots & z_{m_{p}j}^{pq}& \cdots & z_{m_p{m}_q}^{pq}\end{array}\right]}\to z_i^{pq}={\displaystyle \sum _{j=1}^{m_q}{z}_{ij}^{pq}}$$

(9)

where $I$ = 1, 2… $m_p$

$$Z_C^{dpq}=\begin{array}{c}{c}_{p1}\\ ⋮\\ c_{pi}\\ ⋮\\ c_{p{y}_p}\end{array}\stackrel{\begin{array}{ccccc}{C}_{q1}& \cdots & C_{qj}& \cdots & C_{q{y}_q}\end{array}}{\left[\begin{array}{ccccc}\raisebox{1ex}{$z_{11}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_1^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{1j}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_1^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{1y_q}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_1^{pq}$}\right.\\ ⋮& & ⋮& & ⋮\\ \raisebox{1ex}{$z_{i1}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_i^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{ij}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_i^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{i{y}_q}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_i^{pq}$}\right.\\ ⋮& & ⋮& & ⋮\\ \raisebox{1ex}{$z_{y_{p}1}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_{y_p}^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{y_{p}j}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_{y_p}^{pq}$}\right.& \cdots & \raisebox{1ex}{$z_{y_p{y}_q}^{pq}$}\!\left/ \!\raisebox{-1ex}{$z_{y_p}^{pq}$}\right.\end{array}\right]}=\left[\begin{array}{ccccc}{z}_{11}^{dpq}& \cdots & z_{1j}^{dpq}& \cdots & z_{1y_q}^{dpq}\\ ⋮& & ⋮& & ⋮\\ z_{i1}^{dpq}& \cdots & z_{ij}^{dpq}& \cdots & z_{i{y}_q}^{dpq}\\ ⋮& & ⋮& & ⋮\\ z_{y_{p}1}^{dpq}& \cdots & z_{y_{p}j}^{dpq}& \cdots & z_{y_p{y}_q}^{dpq}\end{array}\right]$$

(10)

The matrix $Z_C^{\delta pq}$ transposes the unweighted matrix supermatrix $W_{pq}$ as follows:

$$\frac{z_D^{xy}={\displaystyle \sum _{i=1}^{i_x}{\displaystyle \sum _{j=1}^{j_y}{z}_{ij}}}}{i_x\times j_y}$$

(11)

$$W_{pq}^{}=\begin{array}{c}{c}_{q1}\\ ⋮\\ c_{qj}\\ ⋮\\ c_{q{m}_q}\end{array}\stackrel{\begin{array}{ccccc}{C}_{p1}& \cdots & C_{pi}& \cdots & C_{p{m}_p}\end{array}}{\left[\begin{array}{ccccc}{z}_{11}^{pq}& \cdots & z_{i1}^{pq}& \cdots & z_{m_{p}1}^{pq}\\ ⋮& & ⋮& & ⋮\\ z_{ij}^{pq}& \cdots & z_{ij}^{pq}& \cdots & z_{m_{p}j}^{pq}\\ ⋮& & ⋮& & ⋮\\ z_{1m_q}^{pq}& \cdots & z_{i{m}_q}^{pq}& \cdots & z_{m_p{m}_q}^{pq}\end{array}\right]}={\left(Z_c^{dpq}\right)}^{\prime }$$

(12)

As shown in Equation (11), $D_i$ denotes the $i$th dimension; $C_{ij}$ denotes the $j$th criterion of the $i$th dimension.

Step 8: A weighted supermatrix is derived. As seen in step 5, the matrix $Z_D$ is generated by averaging the degree of influence of the criteria on each dimension.

$$\frac{z_D^{nm}={\displaystyle \sum _{i=1}^{i_n}{\displaystyle \sum _{j=1}^{j_m}{z}_{ij}}}}{i_n\times j_m}$$

(13)

where $i_n$ is the number of criteria in dimension $n$ and $j_m$ is the number in dimension $m$.

$$Z_D=\left[\begin{array}{cccc}{z}_D^{11}& z_D^{12}& \cdots & z_D^{1n}\\ z_D^{21}& z_D^{22}& \cdots & z_D^{2n}\\ ⋮& ⋮& & ⋮\\ z_D^{n1}& z_D^{n2}& \cdots & z_D^{nn}\end{array}\right]\to d_2={\displaystyle \sum _{j=1}^n{z}_D^{d2j}}$$

(14)

The weighted supermatrix $W^{\delta }$ is created by multiplying the unweighted supermatrix as $W$ by $Z_D^{\delta }$ as follows:

$$Z_D^d=\left[\begin{array}{cccc}\frac{z_D^{11}}{d_1}& \frac{z_D^{12}}{d_1}& \cdots & \frac{z_D^{1n}}{d_1}\\ \frac{z_D^{21}}{d_2}& \frac{z_D^{22}}{d_2}& \cdots & \frac{z_D^{2n}}{d_2}\\ ⋮& ⋮& & ⋮\\ \frac{z_D^{n1}}{d_n}& \frac{z_D^{n2}}{d_n}& \frac{z_D^{n3}}{d_n}& \frac{z_D^{nn}}{d_n}\end{array}\right]=\left[\begin{array}{cccc}{z}_D^{d11}& z_D^{d12}& \cdots & z_D^{d1n}\\ z_D^{d21}& z_D^{d22}& \cdots & z_D^{d2n}\\ ⋮& ⋮& & ⋮\\ z_D^{dn1}& z_D^{dn2}& \cdots & z_D^{dnn}\end{array}\right]$$

(15)

$$W^{\delta }=W\times Z_D^{\delta }$$

(16)

Step 9: Determine the weights of DANP. Continue to impose restrictions on the weighted supermatrix $W^{\delta }$ until it converges and stabilizes. The DANP weights can be multiplied by the matrix to reach a steady state.

$$W_W=\underset{\lambda \to \infty }{\lim }{\left(W\right)}^{\lambda }$$

(17)

4. Data Analysis

4.1. Relationship between Metric Dimensions and Indicators

According to

Table A1. Questionnaire on the impact of each indicator.

Level of Influence	X11	X12	X13	X14	X15	X21	X22	X23	X24	X31	X32	X33	X34	X41	X42	X43	X44
X11
X12
X13
X14
X15
X21
X22
X23
X24
X31
X32
X33
X34
X41
X42
X43
X44

Note: X₁₁: System Reliability, X₁₂: System Security, X₁₃: System Flexibility, X₁₄: Functional Diversity, X₁₅: Operational Convenience, X₂₁: Esthetics of the Exterior Structure, X₂₂: Clarity of Interface Presentation, X₂₃: Simplicity of Interface Interaction, X₂₄: Degree of Equipment Expansion, X₃₁: Teaching Content Standardization, X₃₂: Teaching Content Applicability, X₃₃: Teaching Content Integrity, X₃₄: Control of the Teaching Process, X₄₁: Regulatory Capability, X₄₂: Voice Interaction Capability, X₄₃: Resource Management Capability, X₄₄: Learning Support Capability. Black cells indicate that there is no need for the questionnaire to ask about the impact of the indicator on itself.

(see Appendix A), a questionnaire was developed to gauge the initial direct influence matrix by comparing the level of influence between any two indicators. A total of 10 experts, including university professors, specialists in robotics-related fields, and primary and secondary school teachers with some familiarity with assisted robotics, were invited to form an advisory group to score the indicators of the influencing factors at each level. By using the maximization of affiliation, this study totaled the results to determine the value of the level of influence between each indicator. The direct impact matrix was produced by applying Equation (2) to convert the fuzzy triangular numbers of the impact factors of the functional assessment indicators into precise values after compiling the questionnaire responses from 10 experts, as shown in Table A1. Even though the 10 experts did not represent all relevant beneficiaries, the results show good consistency and can, in some cases, reflect reality.

The normalized directed relationship matrix is obtained from Equations (2) and (3), and is calculated using Equation (4) (refer to

Table 2. Total influence matrix.

Z	X11	X12	X13	X14	X15	X21	X22	X23	X24	X31	X32	X33	X34	X41	X42	X43	X44
X11	0.20	0.28	0.31	0.31	0.35	0.17	0.18	0.24	0.31	0.30	0.30	0.30	0.35	0.34	0.28	0.31	0.34
X12	0.24	0.16	0.22	0.26	0.28	0.13	0.14	0.20	0.26	0.25	0.26	0.26	0.30	0.30	0.23	0.26	0.27
X13	0.22	0.21	0.18	0.30	0.32	0.15	0.15	0.23	0.27	0.26	0.26	0.25	0.30	0.29	0.23	0.26	0.29
X14	0.26	0.24	0.29	0.26	0.35	0.17	0.19	0.27	0.31	0.29	0.31	0.30	0.35	0.33	0.27	0.31	0.34
X15	0.22	0.21	0.25	0.28	0.24	0.15	0.16	0.24	0.25	0.24	0.26	0.25	0.31	0.28	0.23	0.27	0.29
X21	0.12	0.11	0.14	0.17	0.22	0.08	0.16	0.19	0.16	0.17	0.17	0.17	0.19	0.17	0.13	0.16	0.17
X22	0.16	0.15	0.17	0.22	0.26	0.19	0.11	0.23	0.20	0.21	0.23	0.22	0.24	0.22	0.18	0.21	0.23
X23	0.21	0.19	0.25	0.28	0.32	0.19	0.19	0.18	0.23	0.25	0.27	0.25	0.31	0.26	0.22	0.25	0.28
X24	0.27	0.27	0.29	0.35	0.36	0.17	0.17	0.26	0.24	0.27	0.30	0.29	0.33	0.32	0.28	0.30	0.34
X31	0.15	0.16	0.16	0.21	0.23	0.12	0.13	0.18	0.18	0.17	0.25	0.24	0.26	0.23	0.18	0.24	0.25
X32	0.14	0.14	0.16	0.20	0.23	0.10	0.11	0.18	0.18	0.23	0.18	0.24	0.26	0.24	0.18	0.24	0.27
X33	0.16	0.15	0.18	0.22	0.25	0.11	0.13	0.19	0.21	0.25	0.26	0.19	0.27	0.25	0.19	0.26	0.28
X34	0.16	0.16	0.19	0.22	0.26	0.11	0.13	0.19	0.20	0.25	0.26	0.26	0.22	0.28	0.21	0.26	0.29
X41	0.19	0.21	0.20	0.26	0.27	0.12	0.13	0.20	0.22	0.26	0.27	0.26	0.30	0.22	0.23	0.26	0.29
X42	0.20	0.19	0.22	0.28	0.31	0.12	0.14	0.22	0.25	0.24	0.26	0.25	0.29	0.28	0.18	0.25	0.28
X43	0.22	0.22	0.24	0.29	0.30	0.13	0.15	0.22	0.25	0.26	0.27	0.27	0.28	0.28	0.22	0.22	0.30
X44	0.20	0.20	0.22	0.27	0.28	0.13	0.14	0.20	0.24	0.24	0.26	0.25	0.27	0.27	0.22	0.26	0.22

). The influence matrix $Z$ within each dimension is then averaged to yield the total influence matrix $Z_D$ for each dimension, as shown in

Table 3. Influence matrix by dimension.

${\mathit{Z}}_{\mathit{D}}$	X1	X2	X3	X4
X1	0.26	0.21	0.28	0.29
X2	0.23	0.18	0.24	0.23
X3	0.19	0.15	0.24	0.24
X4	0.24	0.18	0.26	0.25

. Equations (5) and (6) of Step 4 were completed to produce the sum of provided and received impacts between criteria and dimensions, as shown in

Table 4. Comprehensive influence matrix.

INRM	X11	X12	X13	X14	X15	X21	X22	X23	X24	X31	X32	X33	X34	X41	X42	X43	X44
r	4.83	4.04	4.15	4.81	4.14	2.68	3.42	4.14	4.81	3.31	3.27	3.54	3.67	3.88	3.96	4.14	3.86
c	3.35	3.27	3.61	4.36	4.82	2.33	2.49	3.62	3.97	4.12	4.37	4.24	4.82	4.54	3.67	4.31	4.74
r + c	8.17	7.31	7.77	9.17	8.96	5.01	5.91	7.76	8.77	7.44	7.63	7.78	8.49	8.42	7.63	8.46	8.59
r − c	1.48	0.77	0.54	0.45	−0.68	0.34	0.92	0.52	0.84	−0.81	−1.10	−0.70	−1.16	−0.66	0.29	−0.17	−0.88

.

4.2. Identification and Statistics of Functional Evaluation Indicators for Assistive Teaching Robots

The entire influence matrix of functional assessment impact indicators can be created using the calculation in Step 5. The impact, influence, centrality, and causation degrees of each factor at all levels are calculated, as seen in

Table 5. Influence index and ranking of each dimension.

Dimensions	Influence Degree/Rank	Influenced Degree/Rank	Centrality /Rank	Cause Degree /Rank
System Structure (X1)	1.034(1)	0.907(3)	1.941(1)	0.128(2)
Appearance interface (X2)	0.884(2)	0.724(4)	1.607(4)	0.160(1)
Teaching function (X3)	0.815(3)	1.026(1)	1.841(3)	−0.211(3)
Auxiliary Support (X4)	0.930(4)	1.007(2)	1.938(2)	−0.077(4)
System Reliability(X11)	4.826(1)	3.345(14)	8.171(8)	1.481(1)
System Security(X12)	4.042(8)	3.27(15)	7.312(15)	0.772(4)
System Flexibility(X13)	4.152(4)	3.615(13)	7.767(10)	0.537(5)
Functional Diversity(X14)	4.811(2)	4.361(6)	9.172(1)	0.449(7)
Operational Convenience(X15)	4.138(7)	4.821(2)	8.959(2)	−0.683(12)
Esthetics of the Exterior Structure(X21)	2.675(17)	2.331(17)	5.006(17)	0.345(8)
Clarity of Interface Presentation(X22)	3.418(14)	2.493(16)	5.91(16)	0.925(2)
Simplicity of Interface Interaction(X23)	4.142(6)	3.621(12)	7.764(11)	0.521(6)
Degree of Equipment Expansion(X24)	4.807(3)	3.966(10)	8.773(3)	0.841(3)
Teaching Content Standardization(X31)	3.314(15)	4.123(9)	7.437(14)	−0.809(14)
Teaching Content Applicability(X32)	3.266(16)	4.367(5)	7.632(12)	−1.101(16)
Teaching Content Integrity(X33)	3.541(13)	4.243(8)	7.785(9)	−0.702(13)
Control of the Teaching Process(X34)	3.667(12)	4.825(1)	8.492(5)	−1.158(17)
Regulatory Capability(X41)	3.88(10)	4.544(4)	8.424(7)	−0.665(11)
Voice Interaction Capability(X42)	3.96(9)	3.666(11)	7.625(13)	0.294(9)
Resource Management Capability(X43)	4.143(5)	4.313(7)	8.456(6)	−0.17(10)
Learning Support Capability(X44)	3.857(11)	4.735(3)	8.592(4)	−0.879(15)

.

The main influencing factors for the functional evaluation of the assistive teaching robot can be initially identified based on the centrality ranking in Table 5 (continued), including functional diversity (X₁₄), ease of use (X₁₅), device expansion (X₂₄), learning assistance capability (X₄₄), teaching process control (X₃₄), and resource management capability (X₄₃). To more intuitively demonstrate the mutual influence and influenced relationship of each influencing factor in the functional evaluation of the assistive teaching robot, this study combines the data in Table 5, projects the statistical results of the cause degree and centrality of each key influencing factor into a two-dimensional coordinate system, and obtains the cause–effect diagram of the influencing factor of the functional evaluation of the assistive teaching robots as shown in Figure 1. Meanwhile, the dimensional and systematic influence network relationship map (INRM) can be drawn from Table 5, as shown in Figure 2.

According to the four-quadrant diagram of causality, the factor in the first quadrant is the Driving Factor of the entire evaluation model, having the most significant impact on the functional evaluation index system and playing an important role in the whole system [87]. The factors in the second quadrant are known as Supporting factors (Voluntariness), as they play a supporting role in the model; the factors in the third quadrant are known as Independent factors, as they have a more significant influence on the model; none of the factors in this study are located in the third quadrant, demonstrating the interdependence of all the factors in the functional index system; and the factor in the fourth quadrant is known as the Threshold factor.

Table 6. Four-quadrant index factor relationship chart.

Quadrants	Title	Factors (Sorted by Centrality)	Features
1	Driving Factor	X11 X12 X13 X14X23 X24 X42	The reason degree is positive; the centroid is above 7
2	Voluntariness	X21 X22	The reason degree is positive; the centroid is below 7
3	Independent	/	The reason degree is negative; the centroid is below 7
4	Core Problem	X15 X31 X32 X33X34 X41 X43 X44	The reason degree is negative; the centroid is above 7

presents the association between the variables in the four quadrants.

The unweighted supermatrix $W_{pq}$ of the 17 secondary influencing factors was constructed in this study and is displayed in

Table 7. Unweighted supermatrix.

Z	X11	X12	X13	X14	X15	X21	X22	X23	X24	X31	X32	X33	X34	X41	X42	X43	X44
X11	0.14	0.20	0.19	0.22	0.25	0.19	0.20	0.27	0.34	0.24	0.24	0.24	0.28	0.27	0.22	0.24	0.27
X12	0.21	0.14	0.19	0.22	0.24	0.18	0.19	0.27	0.36	0.23	0.25	0.24	0.28	0.28	0.22	0.25	0.26
X13	0.18	0.17	0.15	0.24	0.26	0.19	0.19	0.29	0.33	0.24	0.24	0.23	0.28	0.27	0.22	0.24	0.27
X14	0.19	0.17	0.20	0.18	0.25	0.18	0.20	0.29	0.33	0.23	0.25	0.24	0.28	0.26	0.21	0.25	0.27
X15	0.19	0.17	0.21	0.23	0.20	0.19	0.20	0.30	0.31	0.23	0.25	0.24	0.29	0.26	0.22	0.25	0.27
X21	0.16	0.15	0.18	0.22	0.28	0.14	0.27	0.32	0.27	0.24	0.25	0.24	0.27	0.27	0.21	0.25	0.27
X22	0.17	0.16	0.17	0.23	0.27	0.26	0.15	0.32	0.28	0.24	0.25	0.24	0.27	0.26	0.21	0.25	0.28
X23	0.17	0.15	0.20	0.23	0.26	0.24	0.24	0.23	0.29	0.23	0.25	0.23	0.29	0.26	0.21	0.25	0.28
X24	0.17	0.17	0.19	0.23	0.23	0.20	0.21	0.31	0.28	0.22	0.25	0.25	0.28	0.26	0.23	0.24	0.27
X31	0.17	0.18	0.17	0.23	0.25	0.19	0.21	0.30	0.30	0.18	0.27	0.27	0.28	0.25	0.20	0.26	0.28
X32	0.17	0.16	0.18	0.23	0.26	0.18	0.19	0.31	0.32	0.25	0.20	0.27	0.29	0.26	0.20	0.26	0.29
X33	0.17	0.16	0.18	0.23	0.26	0.18	0.20	0.30	0.32	0.25	0.27	0.19	0.28	0.25	0.20	0.26	0.28
X34	0.16	0.16	0.19	0.22	0.26	0.17	0.20	0.30	0.32	0.25	0.27	0.26	0.22	0.27	0.20	0.25	0.28
X41	0.17	0.19	0.18	0.23	0.24	0.17	0.19	0.30	0.34	0.24	0.25	0.24	0.27	0.22	0.23	0.26	0.29
X42	0.17	0.16	0.19	0.23	0.26	0.17	0.19	0.30	0.34	0.24	0.25	0.24	0.28	0.28	0.18	0.25	0.29
X43	0.17	0.17	0.19	0.23	0.23	0.17	0.20	0.29	0.34	0.24	0.25	0.25	0.26	0.28	0.22	0.21	0.30
X44	0.17	0.17	0.19	0.23	0.24	0.18	0.20	0.28	0.34	0.24	0.25	0.25	0.27	0.28	0.23	0.27	0.23

to more precisely quantify the weights of the influencing factors for the functional evaluation of the assistive teaching robot.

Based on this, Equation (6) was used to calculate and weigh the comprehensive influence matrix $Z_C^{\delta }$ of four first-level influence factors. The weights of every part of the system were then derived by performing limit operations on the weighted supermatrix of the influence factors to assess the assistive teaching robot’s function, as shown in

Table 8. Polarized supermatrix.

Z	X11	X12	X13	X14	X15	X21	X22	X23	X24	X31	X32	X33	X34	X41	X42	X43	X44
X11	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04
X12	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04
X13	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05	0.05
X14	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
X15	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
X21	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04
X22	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04
X23	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
X24	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
X31	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
X32	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
X33	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
X34	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08
X41	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
X42	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06	0.06
X43	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
X44	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08	0.08

.

Although most other research constructed robot indicators using the ANP approach [88,89], this study opted to employ the DANP method to determine the significance of indicators. The DANP technique utilizes the DEMATEL and ANP results, in addition to examining the interactions between internal elements of the system, to calculate the criteria weights. The degree of influence ($Z_C$) between the indicator and the dimension is obtained from DEMATEL, and $Z_C$ is normalized by considering the intensity of the effect of each indicator within its dimension. As a result, the weighted supermatrix (Step 8 of Section 3) takes each criterion’s percentage within its dimension and the extent to which it influences the other dimensions into account. Therefore, the weights of the indicators can be obtained first, then we determine the consequences of each dimension by adding the corresponding importance of the indicators within the same dimension. Consistent results can be obtained while avoiding the time-consuming, pairwise comparisons in the original ANP.

Table 9. Functional evaluation impact factor weights.

Factor	X34	X44	X41	X32	X43	X33	X31	X24	X15	X42	X23	X14	X13	X11	X12	X22	X21
Weight	0.077	0.076	0.072	0.070	0.069	0.068	0.066	0.063	0.061	0.058	0.058	0.056	0.046	0.042	0.041	0.039	0.036
Rank	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17

and

Table 10. Overall weighting table for functional evaluation impact factors.

T	Local Weight	Ranking	T	Local Weight	Ranking	Global Weight
X1	0.246	3	X11	0.172	14	0.042
			X12	0.168	15	0.041
			X13	0.186	13	0.046
			X14	0.226	12	0.056
			X15	0.248	9	0.061
X2	0.196	4	X21	0.186	17	0.036
			X22	0.201	16	0.039
			X23	0.294	11	0.058
			X24	0.320	8	0.063
X3	0.281	1	X31	0.235	7	0.066
			X32	0.249	4	0.070
			X33	0.243	6	0.068
			X34	0.273	1	0.077
X4	0.276	2	X41	0.262	3	0.072
			X42	0.211	10	0.058
			X43	0.251	5	0.069
			X11	0.172	14	0.042

show the component weight values and the total of each component’s influences. This study has now quantified the weights of the factors impacting the functional evaluation of the assistive teaching robot.

5. Discussion

5.1. Cause Factor Analysis

The appearance interface (X₂), which has the most considerable cause degree ($r_i$ − $c_i$) value (0.160) and is consequently the most influential dimension, is shown in Table 5 and Figure 1 through a study of how the influencing elements interact with one another and their positions within the system. This indicates the importance of the appearance interface in the operation of the educational robot, aligning with Ryu et al.’s (2007) [69] assertion that a robot’s external features significantly influence educated individuals. At the same time, the appearance interface will dramatically impact the other dimensions and could have the most significant impact on the overall functional evaluation system [12]. The main causal factors (causal degree values greater than 0) of the operational evaluation influencing factors of the assistive teaching robot are system reliability (X₁₁), clarity of interface interaction (X₂₂), degree of equipment expansion (X₂₄), system security (X₁₂), system flexibility (X₁₃), simplicity of interface presentation (X₂₃), functional diversity (X₁₄), the esthetic of the exterior structure (X₂₁), and voice interaction capability (X₄₂). These findings indicate that the above factors are more likely to influence the other factors in the system during the application of the assistive teaching robot, which plays a fundamental and guaranteed role.

Additionally, the system structure (X₁) has the highest centrality ($r_i$ + $c_i$) (1.941), which indicates that it has the most outstanding overall degree of impact inside the dimension, and has the most ability to influence the other dimensional indicators. Furthermore, Yoshino and other researchers (2018) noted in their publication that a system’s general stability enhances the user’s experience when using it [66,67], which is consistent with the findings of this study. We are aware that when selecting a robot that is not yet fully understood, the attractiveness of the appearance can affect people’s initial willingness to use it, which is referred to as the first-cause effect. The stable use experience that the robot itself can bring plays a crucial role in the user’s long-term use as the length of use increases and the use deepens [66]. As a result, the system structure will influence the user experience brought by other dimensions in the process. System security and flexibility are necessary for delivering high-quality services; device expansion, interface presentation clarity, and functional diversity offer crucial hardware and software guarantees for the assistive teaching robot; and whether or not the user will accept the assistive teaching robot depends on the esthetic appeal of the exterior structure and its voice interaction capability. User acceptance of the assistive teaching robot is directly correlated with system dependability and the clarity of interface interaction. The decision makers’ and users’ choice of the robot will ultimately be influenced most by the robot’s appearance interface, and its system structure.

5.2. Outcome Factor Analysis

Similarly, it can be deduced that the capability of resource management (X₄₃), regulatory capability (X₄₁), operational convenience (X₁₅), teaching content applicability (X₃₂), teaching content standardization (X₃₁), simplicity of interface interaction (X₂₃), learning support capability (X₄₄), teaching content integrity (X₃₃), and control of the teaching process (X₃₄) are the outcome factors (reason degree value less than 0) of the factors influencing the functional evaluation of the assistive teaching robot. The components primarily dealing with teaching function and auxiliary support directly impact the evaluation system’s system structure and appearance interface components. The size of the interface or the system design will influence the richness of teaching-related functions that can be provided on the operating interface. In a related study, Yang and Wang (2020) [70] also suggested that the neatness of the interface features affects the range of functions and assistive support that assistive teaching robots can offer users. Teaching function and auxiliary support are two crucial aspects of the functional evaluation system of assistive robots and are key elements in promoting sustainable learning.

5.3. Importance Analysis of Influencing Factors

According to the ranking results of the weight values of the factors influencing the functional evaluation of the assistive teaching robot in Table 10 in the order of highest to lowest, the factors are teaching function (X₃), auxiliary support (X₄), system structure (X₁), and appearance interface (X₂). From the criteria, five indicators, namely control of the teaching process (X₃₄), learning support capability (X₄₄), regulatory capability (X₄₁), teaching content applicability (X₃₂), and resource management capability (X₄₃), have a significant influence on the application evaluation of assistive teaching robots. This finding is in line with the need for assistive teaching robots to have a higher level of intelligence and a wide range of knowledge for the teaching field [74], while the esthetic of the exterior structure (X₂₁), clarity of interface presentation (X₂₂), and system security (X₁₂) are the minor essential indicators in the system.

These results support the findings of the DEMATEL method study, which discovered that the teaching function and auxiliary support are two crucial parts of an assistive teaching robot’s functional evaluation system. The teaching function and auxiliary support are vital to the functional evaluation of an assistive teaching robot. Still, simultaneously, these two dimensions are easily influenced by both the system structure and the appearance interface. Although this result contrasts with the results of the centrality ranking of the factors presented in Table 5, it indicates this fact. The finding that robots with diverse teaching resources, standardized content settings, personalized learning for teachers and students, and convenient support are readily preferred by users is also consistent with the expectations of customized teaching that teaching assistants can provide in related studies [71,75]; the four dimensions intertwine to influence the selection of an assistive teaching robot as a whole. Given that teaching is the main application scenario for teaching robots, the design of teaching functions and the reserve of teaching resources require special consideration during the design and development process, as shown by the ranked factors in Table 5 and the computation of the index weights in Table 9. For the assistive teaching robot to be a valuable tool for teachers and a helpful learning partner for students, it is also important to consider how the system is built and how the interface’s appearance is designed during development.

5.4. Theoretical and Practical Implications

In this study, based on integrating the existing literature and verifying the validity of the evaluation indicators, we successfully constructed a set of functional evaluation indicator systems for assistive teaching robots to promote sustainable learning. By applying the DANP method of hybrid, multi-criteria decision making, we deeply explored the associations among the indicator dimensions and their mutual influence relationships in the complex evaluation system. This not only provides an implementable and widely adaptable evaluation framework for related research, but also provides a useful reference for the analysis and evaluation of educational robot functions in educational environments that emphasize structure and operability, presenting the network associations of the evaluation system and the complex influences among evaluation indicators from a new perspective.

In practical applications, this study provides a practical evaluation tool within a specific educational field, especially for relatively structured and explicit educational environments such as STEM fields and basic subjects. In these educational environments, the functionality of assistive teaching robots is compatible with the nature of the subject, and their features such as personalized support, content delivery, and learning pace adjustment can significantly enhance teaching and learning.

According to the data analysis findings, it is evident that auxiliary support and teaching functions are the main criteria for judging the functionality of the assistive teaching robot. The design and development of assistive teaching robots can be guided by the core issues of evaluation systems to meet the needs of teachers and students in educational fields that emphasize structured learning and the development of practical skills. The teaching function includes the teaching methods, techniques, and strategies that the assistive robot can provide to the teacher, as well as the ability to personalize the teaching to the student’s needs. Auxiliary support focuses on the support and assistance that the robot can provide in the teaching process beyond what is necessary for the course to proceed, such as answering questions and solving problems, and recommending learning resources. The robot assistant should be able to obtain the latest teaching resources and knowledge in a timely manner and integrate them into the teaching content to meet the evolving learning needs and knowledge updating requirements. Therefore, the evaluation system can provide guidance for the design and development of assistive robots in terms of focusing attention, as well as a basis for selecting robots that promote sustainable competence enhancement to meet the different learning needs of teachers and students, and to support the whole society in developing sustainable learning competence of the next generation of young people.

The final results of the study may, in certain cases, provide targeted recommendations for the development of appropriate robotic products to support the use of assistive teaching robots in instruction and education. The relevant results from the data analysis can address the homogenization problem with educational robot production, and provide a more robust logical basis for the robots selected by manufacturers, decision makers, and users. When developing and designing assistive teaching robots, the emphasis should be on providing practical teaching functions and tailored assistive support to meet the needs of teachers and students for sustainable learning in education. Even though an assistive teaching robot’s operator interface, appearance, and practicality impact the user experience, a greater focus should be placed on its individualized assistance support and practical teaching functions. Suppose assistive teaching robots are to be successful in the classroom. In that case, they must have access to a wide range of teaching resources, standardized teaching topic environments, and the ability to support individualized learning for teachers and students. Robot manufacturers should concentrate on improving teaching resources, monitoring the quality of teaching content, and improving personalized learning support capabilities during the development process. At the same time, decision makers and users can also use these factors as the primary basis for selecting robots.

6. Conclusions

This study has built an assistive teaching robot functional evaluation indicator system aimed at sustainable learning, that can direct product development and serve as a guide for schools in selecting robots. Relevant functional evaluation is increasingly crucial as artificial intelligence technology continuously upgrades and educational robot applications in real-world settings increase in number. This study built a functional evaluation indicator system for assistant teaching robots using 17 indicators, including resource management skills, system dependability, and degree of equipment expansion, and four dimensions: system structure, appearance interface, teaching function, and auxiliary support. The weights of each dimension indicator that influence the functional evaluation are then determined using the DANP method. Finally, the pertinent influence, result, and causal factors were examined. According to the findings, the teaching function is the key evaluation factor that significantly impacts deciding which assistive teaching robot functions to use.

The research results can provide the following contributions: (1) identify suitable functional evaluation indicators for assistive teaching robots; (2) determine the weights of each dimension and indicator in the evaluation system by using advanced models; and (3) provide targeted suggestions for the development and selection of assistive teaching robots based on expert judgments.

6.1. Suggestions for Future Research and Practice

By analyzing cause–effect relationships and influencing factors, and considering the ranking of influencing factors, this study makes the following recommendations for the design and selection of future assistive teaching robots:

6.1.1. Enrich the Teaching Resource Library and Enhance the Ability to Assist in Teaching

Robots with an extensive library of teaching resources can aid in improving teaching in educational settings. The teaching function is the primary determining factor when choosing assistive teaching robots. Teaching robots should follow national or regional education and teaching standards, ensure that the lesson content satisfies the requirements, and provide a library of teaching resources that may include various educational applications, interactive multimedia teaching materials, online courses, and so on, to meet the needs of multiple learners and promote their participation. The integrated teaching resource library should also cover the fundamental and advanced knowledge of the pertinent subjects in-depth and methodically. Teachers can fully utilize the courseware and teaching resources in the teaching resource library to prepare lessons, saving them time and allowing them to focus more on assisting students with their learning. Assistive teaching robots must also offer teaching materials that can be individually adjusted and optimized according to the needs of various learners to promote students’ learning progress and improve their learning effectiveness. Assistive teaching robots should also have the function of effectively managing learning resources, including collecting, classifying, storing and sharing learning resources, ensuring that students can conveniently access the required learning materials, and providing a diversified choice of resources to meet the learning needs of different students. With the rapid development of information technology and the rapid updating and innovation of educational resources, educational robots, as tools capable of providing rich educational resources, have the potential to create a richer educational experience for learners. By providing students with diverse learning materials, real-time feedback, and personalized support, educational robots are expected to effectively enhance their motivation and interest in learning, thus helping to strengthen their ability to learn in a sustainable way.

6.1.2. Combine Core Technologies and Facilitate Supportive Support

Teaching assistants’ supportive assistance in the classroom can impact the entire teaching process. Robots must have sensors that can provide a variety of application scenarios, natural language processing capabilities, and potent perceptual information processing technologies since the diversity of robot functions must rely on integrating numerous essential technologies. For instance, the Alpha Egg from China’s iFlytek combines speech recognition and sound source localization technology to enable face-to-face communication with users. Due to the variety of data encountered in the practical application of assistive teaching robots in education, there is a need for collaborative efforts among more sophisticated technologies to enhance the pedagogical suitability of educational robots. At the same time, in case of potential misbehavior, the assistive robot needs to have a monitoring and warning function to ensure the order of the educational process and the safety of the learning environment. In practice, assistive teaching robots can use advanced recommendation algorithms and machine learning technologies to recommend learning content and resources tailored to students’ learning needs and interests. Through in-depth analysis of students’ learning history and behavior patterns, it understands and grasps students’ learning tendencies and knowledge needs, and provides them with personalized learning suggestions, thus stimulating students’ active learning interests and sustained motivation. Currently, technologies that can emulate human perception are still in the development stage, and one important technology is multimodal fusion perception. This technology enables robots to obtain more comprehensive and accurate information to support more efficient scientific decision making by assistive teaching robots.

6.1.3. Pay Attention to the Design of Robot Appearance and Interaction Interface

The appearance of the robot has a long-term direct impact on the user’s in-use behavior regarding the robot. To lower the barriers to human–computer interaction and to enhance the learners’ sense of affinity, it is important to essentialize the robot’s appearance, expressions, and movements specifically for the user’s age group. The interactive interface is the first barrier to human–robot interaction, and slick interface animations and clear instructions greatly impact the user’s experience. However, the design of robot products is frequently uniform, both domestically and internationally, in terms of appearance and user interface, and robots from large manufacturers have a similar appearance and the same operation methods, quickly making users feel fatigued. The emphasis on appearance and interactive interface design will help to resolve this issue and enable learners to interact with the robot more thoroughly and productively. Additionally, robots must be more appealing and targeted to learners’ unique requirements and cognitive variations at various levels. In addition, as related artificial intelligence technologies, like bionic technology and intelligent manufacturing technology mature, the anthropomorphic appearance design of assistive teaching robots will become a significant development trend in the future.

6.2. Limitations

A hybrid of DEMATEL and ANP methods was used to build and analyze the indicators in this study. The DEMATEL method, based on the expert analysis and selection of indicator relationships, was used to confirm the interdependence of variables/criteria, so the data obtained have some subjectivity. This limitation issue can be resolved if future research includes fuzzy set comparison analysis.