Estimating the Artificial Intelligence Learning Efficiency for Civil Engineer Education: A Case Study in Taiwan

Abstract

The civil engineering educators focused on implementing interdisciplinary learning in artificial intelligence (AI) based on a more innovative application of construction requirements. However, only a few pieces of literature discussed the educational learning efficiency and feedback for this trend. Hence, this study surveyed the 237 data from eight universities that issued the interdisciplinary courses. The factors were modified from the scales in science, technology, engineering, and mathematics education. Further, the descriptive analysis was used to explain this situation in Taiwan. A novel approach based on data envelopment analysis and Mahalanobis distance approaches was proposed to solve this problem. The advantages of the proposed approach were discussed and compared with traditional method. Based on the student gains in the interdisciplinary courses, three groups were clustered and compared. The feedback of a high-input and low-efficiency student group was suggested for improving learning strategies. The sensitivity analysis of this special group showed that effective teaching practice is the key factor in the artificial intelligence courses for civil engineering students. These students may increase technical efficiency by 37% by paying 21% inputs. Therefore, this paper provided a useful and easy approach to make learning strategies for non-informatics students in AI learning.

1. Introduction

One of the recent educational questions asked is whether construction industry sectors can successfully operate in the digital environment and face future artificial intelligence (AI) challenges. Although the implementation of AI eases work on all construction procedures, it concurrently causes many new problems. With the extensive adoption of AI, construction engineering and management are experiencing a rapid digital transformation. Noteworthily, Pan and Zhang (2021) presented a systematic review under both scientometric and qualitative analysis to illustrate the current state of AI adoption in construction engineering and management. The characteristics of keywords, journals, and clusters based on 4473 journal articles published in 1997–2020 were surveyed and discussed. The results showed that there had been an explosion of relevant papers, especially in the past 10 years. The AI implementation topics, including (i) knowledge representation and reasoning, (ii) information fusion, (iii) computer vision, (iv) natural language processing, (v) intelligence optimization, and (vi) process mining, have an ample advantage in the construction industry [1]. Due to the values revolutionizing the construction industry, leading to a more reliable, automated, self-modifying, time-saving, and cost-effective process of civil engineering and management, the interdisciplinary learning of AI and civil engineering is also a key factor in graduate and undergraduate education for better research and development [2,3,4,5]. At the same time, Science, Technology, Engineering, and Mathematics (STEM) education is improving the implementation of the AI topics mentioned above in the civil engineering industry. Borrego and Newswander (2010) compared literature on interdisciplinary studies with a content analysis of 129 successful proposals (including civil infrastructure) in the U.S. National Science Foundation. The results identified and discussed five categories of learning outcomes for interdisciplinary graduate education: (i) disciplinary grounding, (ii) integration, (iii) teamwork, (iv) communication, and (v) critical awareness [6]. On the other hand, Akinosho et al. (2020) carefully analyzed previous researches that have implemented each of these deep learning algorithms as deep neural network, convolutional neural network, recurrent neural network, auto-encoders, restricted Boltzmann machines, deep belief networks, and generative adversarial network. They believe that there are currently insufficient applications of deep learning in this domain compared to other digital technologies, such as building information modeling (BIM) and other machine learning algorithms [7]. The results indicated that graduate and undergraduate education might focus on AI implementation in the construction industry.

In recent literature, interdisciplinary education is focused on the different fields of higher education. Further, the interdisciplinary education perception scale (IEPS) is studied and proposed in several pieces of literature [8,9,10]. However, only a few studies proposed the performance between inputs and outputs based on the Data Envelopment Analysis (DEA) method. The DEA method is a nonparametric method used in operations research and economics to estimate production frontiers [11]. An et al. (2021) first considered subordinates’ strategy behaviors to realize the actual “best practice” during the benchmarking process and employed the agency theory to reveal their strategy behaviors. Additionally, a reimbursement scheme is proposed to motivate decision-making units in realizing their “best practices” [12].

Velasquez and Hester (2013) discussed the multiple attributes utility theory and focused on the comparison of DEA, Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), Simple Multi-Attribute Rating Technique (SMART), and others [13]. However, one of the significant limitations is the independent characteristics of the attributes. In recent research, some papers discussed the dependent attributes of decision-making problems using the Mahalanobis distance instead of the Euclidean distance. Vega et al. (2014) proposed a TOPSIS-Mahalanobis approach to incorporate the correlations among the attributes. Meanwhile, the Manhattan distance, Euclidean distance, and Tchebycheff in TOPSIS were compared and discussed [14]. On the other hand, Sheikh et al. (2019) compared the TOPSIS-Mahalanobis approach, TOPSIS, and Simple Additive Weight methods for prioritizing the districts of the Golestan Province with respect to all three susceptibility maps. The results have shown that the TOPSIS-Mahalanobis approach was chosen as the well-performing ranking method for further environmental managerial actions, mainly due to its consideration of the strong correlations among the criteria. The authors believed that the TOPSIS-Mahalanobis approach framework merits more studies and is applicable to any multi-criteria decision-making issue in any branch of science [15]. Moreover, these articles show that the Mahalanobis distance is suitable and useful in the utility theory.

This study aimed to survey the situation and efficiency of interdisciplinary education on AI for civil engineering students in Taiwan. Particularly, three questions were posed:

i.

What is the course name and number of students of the interdisciplinary education on AI for civil engineering students in Taiwan?

ii.

How does the proposed DEA-Mahalanobis distance approach estimate the performance of inputs and outputs in the interdisciplinary learning system? What is the difference between the traditional DEA and the proposed approach?

iii.

How does the efficiency analysis feedback to these students?

2. Materials and Methods

2.1. Research Design

A descriptive research design, paired with traditional data envelopment analysis (DEA) and integration approach based on the DEA and Mahalanobis distance were used. In addition, the benchmark clustering method was also used.

2.2. Sample

All civil or construction engineering education, universities in general (n = 17), and science and technology (n = 18) offering civil education programs in Taiwan were noted. Based on the 2020 year course maps of all noted universities in the Ministry of Education—Taiwan R.O.C., only eight general universities offered courses, including artificial intelligence, machine learning, deep learning, or artificial neural network. Furthermore, students enrolled in schools or graduated in 3 years from these eight general universities were selected.

2.3. Data Collection

The survey datasets are from two parts as the Course Information Website database (https://ucourse-tvc.yuntech.edu.tw/ accessed on 20 August 2021) and questionnaire that modified from the measurement of interdisciplinary competence for engineers [16], the National Survey of Student Engagement [17], and IEPS [9]. The aim of questionnaire is to estimate the input and output of civil students’ subjective and objective scores. Lattuca et al. (2013) used interdisciplinary skills, reflective behavior, and recognizing disciplinary perspectives to build a framework for estimating the measure of interdisciplinary competence [9]. However, this framework focuses on the subjective output and still need objective output and the input data for efficiency estimation. Thus, some questions for students’ objective scores and subjective input from references [16,17] were used to complement these parts. Data were collected from August 2021 to September 2021.

2.4. Ethical Considerations

All information analyzed were data obtained from the open survey website. On the other hand, the ethical approval for this study was obtained from the Human Research Ethics Council (HREC code: 110-331).

2.5. The Proposed Approach

Data envelopment analysis (DEA), also called frontier analysis, was first put forward by Charnes, Cooper and Rhodes (CCR ratio model) in 1978. DEA is a performance measurement technique that can be used to evaluate the relative efficiency of decision-making units (DMUs) in multiple organizations [18]. A DMU is a distinct unit within an organization. Students in different universities are the DMUs in this research. Furthermore, the advantages of the DEA method are as follows: (i) it is not necessary to explicitly specify a mathematical form for the production function; (ii) it is capable of handling multiple inputs and outputs; (iii) it is capable of being used with any input-output measurement, although ordinal variables remain tricky; (iv) the sources of inefficiency can be analyzed and quantified for every evaluated unit; (v) using the dual optimization problem, it can identify which DMUs are evaluating itself against which other DMUs [18]. Remarkably, many pieces of literature discussed the DEA application in education [19,20,21,22].

For selecting the suitable model in this variable return to scale (VRS) education case, the Banker Charnes Cooper (BCC) model [23] was used to distinguish between technical and scale inefficiencies by (i) estimating pure technical efficiency at the given scale of operation and (ii) identifying whether the possibilities of increasing, decreasing or constant returns to scale are present for further exploitation. The procedures of the BCC model are as follows. Assume that there are n DMUs (DMUj: j = 1, 2, …, n), which consume m inputs (xi: i = 1, 2, …, m) to produce s outputs (yr: r = 1, 2, …, s). The BCC input-oriented model evaluates the efficiency of DMUo, under consideration, by solving the following linear program.

$$\begin{array}{c}\mathrm{Max} {\displaystyle \underset{s}{\overset{r=1}{{{\displaystyle \sum }}^{ }}}}{u}_r{y}_{rj}-u_0\\ \mathrm{s}.\mathrm{t}. {\displaystyle \underset{m}{\overset{i=1}{{{\displaystyle \sum }}^{ }}}}{w}_i{x}_{io}=1\\ {\displaystyle \underset{s}{\overset{r=1}{{{\displaystyle \sum }}^{ }}}}{u}_r{y}_{rj}-u_0-{\displaystyle \underset{m}{\overset{i=1}{{{\displaystyle \sum }}^{ }}}}{w}_i{x}_{ij}⩽0, j=1,2,\dots ,n\\ u_0, \mathrm{free}\\ w_i⩾\epsilon , i=1,2,\dots ,m\\ u_r⩾\epsilon , r=1,2,\dots ,s\end{array},$$

(1)

where x_ij and y_rj are the inputs and outputs of the jth DMU, w_i and u_r are the input and output weights (also referred to as multipliers). x_io and y_ro are the inputs and outputs of DMUo.

However, one of the disadvantages of DEA is the limitation of orthogonal attributes. The problem of dependent variables is considered in many pieces of literature [24,25]. Particularly, the traditional DEA method is hard to use in learning with orthogonal attributes. Hence, the Mahalanobis distance is used in rebuilding the measurement of DMUs, which Vega et al. (2014) and Sheikh et al. (2019) found to be a suitable and useful method for solving this limitation. Further, De Maesschalck et al. (2020) explained and discussed the application of Mahalanobis distance and proposed its comparisons with the Euclidean distance [26]. The Mahalanobis distance between two series is defined as:

$$d={({(x_A-x_B)}^T\times C^{-1}\times (x_A-x_B))}^{0.5},$$

(2)

where x_A and x_B are a pair of series, and C is the sample covariance matrix. Further, the covariance matrix C is a measure of how much x_A and x_B variables move together in the same direction.

In this study, the series are the inputs and outputs of DMUs.

3. Results

The courses, number of student, and compulsory in Taiwan AI education for civil engineering students in 2020 were shown in

Table 1. The courses, number of student, and compulsory in Taiwan AI education for civil students in 2020.

Course Name	Number of Student	Compulsory	Undergraduate	Graduate
The application of artificial neural network in civil and hydraulic engineering	6	yes	yes	yes
The analysis and application of deep learning for disaster prevention and information management	13	no	no	yes
Introduction to machine learning and deep learning	190	no	yes	no
The discussion and application for artificial intelligence engineering	10	no	no	yes
Scientific computing and artificial intelligence platform	26	no	yes	no
The application of artificial intelligence in civil engineering	109	no	yes	yes
Internet of Things and smart monitoring technology of structures	15	no	no	yes
Data exploration of intelligent transportation system	20	yes	yes	no
Artificial intelligence	33	no	yes	no
Special topics on smart city	20	no	no	yes

. The application of AI and machine learning are almost all for undergraduates. The deep learning, Internet of Things, and sensors are for graduates. The accumulation of student number is 436 per year. Only two courses are compulsory in some departments.

A descriptive analysis of 237 surveyed records was carried out. For the internal consistency reliability, the total Cronbach’s α is 0.758. The details of the alpha if items deleted values are shown in

Table 2. Factor analysis results of the survey related to interdisciplinary competence.

DMUs	Factors	Item To What Extent do you Agree or Disagree with Each of the Statements Below. 1 (Code)	Alpha if Item Deleted	Item Means (Std. Dev.) 2
Inputs	Student Engagement (SE)	I am not late and leave early in the course. (SE1)	0.75	3.39 (0.73)
		I focus on teacher explanation in the course. (SE2)	0.75	3.55 (0.88)
		I do not use cellphone in the course. (SE3)	0.74	2.56 (0.92)
		I am happy to attend expert meetings or seminars. (SE4)	0.74	3.30 (0.82)
		I learn hard with peers. (SE5)	0.72	2.66 (0.84)
		I learn from YouTube or others. (SE6)	0.72	2.94 (0.94)
		I am happy to discuss with faculty. (SE7)	0.72	2.62 (0.76)
		I have effective teaching practices. (SE8)	0.75	3.99 (0.83)
	College Engagement (CE)	I think that the courses have quality of interactions. (CE1)	0.74	4.37 (0.70)
		I think that the courses have supportive environments. (CE2)	0.74	3.14 (0.88)
Outputs	Interdisciplinary Skills (IS)	I value reading about topics in AI and civil engineering. (IS1)	0.74	3.30 (0.69)
		I enjoy thinking about how different fields approach the same problem in different ways. (IS2)	0.75	2.68 (0.90)
		Not all engineering problems have purely technical solutions. (IS3)	0.75	3.26 (0.83)
		In solving civil engineering problems I often seek information from AI fields. (IS4)	0.71	2.67 (0.81)
		Given knowledge and ideas from different fields, I can figure out what is appropriate for solving a problem. (IS5)	0.72	2.92 (0.68)
		I see connections between ideas in civil engineering and AI. (IS6)	0.71	3.03 (0.70)
		I can take ideas from outside engineering and synthesize them in ways that help me better understand. (IS7)	0.72	3.85 (0.61)
		I can use what I have learned in one field in another setting. (IS8)	0.73	3.67 (0.63)
	Reflective Behavior (RB)	I often step back and reflect on what I am thinking to determine whether I might be missing something. (RB1)	0.74	4.27 (0.50)
		I frequently stop to think about where I might be going wrong or right with a problem solution. (RB2)	0.72	3.92 (0.61)
	Recognizing Disciplinary Perspectives (RDP)	If asked, I could identify the kinds of knowledge and ideas that are distinctive to civil engineering and AI fields of study. (RDP1)	0.72	4.14 (0.53)
		I recognize the kinds of evidence that different fields of study rely on. (RDP2)	0.75	3.12 (0.89)
		I’m good at figuring out what experts in different fields have missed in explaining a problem/solution. (RDP3)	0.75	4.02 (0.80)
	Incentive Outcomes (IO)	After interdisciplinary education, I am more interesting in AI and civil engineering. (IO1)	0.75	3.39 (0.73)
		After interdisciplinary education, I believe that I will be more competitive in the job. (IO2)	0.75	3.55 (0.88)

¹ Scale: 1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree; ² Values in the right column denote the mean and standard deviations in parentheses for each item.

.

The survey questions and scales were modified from the literature [9,16,17] and judged by three associate professors studying education. The validity is sufficient to establish the reliability. The traditional DEA was used to analyze the efficiency of the 25 variables. The outputs and inputs of DMUs were analyzed based on the BCC model, output-oriented, and VRS. The results are shown in

Table 3. DEA analysis results of 25 variables.

Statistical Parameters	Technical Efficiency from CRS	Technical Efficiency from VRS	Scale Efficiency
Mean	0.996	1	0.996
Standard deviation	0.017	0.005	0.014

. Further, the mean and standard deviation were almost one and zero, implying that the DEA failed in the parameter setting.

Thus the factors of the inputs were set as student engagements and college engagements, while the outputs were set as interdisciplinary skills, reflective behavior, recognizing disciplinary perspectives, and incentive outcomes. The mean and standard deviation were shown in

Table 4. The statical parameters of six factors.

Statistical Parameters	IS	RB	RDP	IO	SE	CE
Mean	2.998	3.759	4.111	4.034	3.002	4.179
Standard deviation	0.378	0.454	0.410	0.565	0.401	0.724

. Meanwhile, the correlation coefficients of six factors are shown in

Table 5. The correlation coefficient of six factors.

Correlation Coefficient	IS	RB	RDP	IO	SE	CE
IS	1	-	-	-	-	-
RB	0.537	1	-	-	-	-
RDP	0.667	0.593	1	-	-	-
IO	0.012	0.007	0.093	1	-	-
SE	0.801	0.414	0.503	0.003	1	-
CE	0.075	0.002	0.258	0.168	0.043	1

. Notably, the correlation between the parameters was significant.

After reducing the number of variables (from 25 to 6), the results of the DEA analysis are shown in

Table 6. DEA analysis results of six variables.

Statistical Parameters	Technical Efficiency from CRS	Technical Efficiency from VRS	Scale Efficiency
Mean	0.858	0.897	0.958
Standard deviation	0.076	0.083	0.046

.

Based on the Mahalanobis distance, the matrix of the original six factors can be transferred to a new matrix with eliminating the correlation of factors. The results of the DEA-Mahalanobis analysis are shown in Table 6.

By accumulating the outputs (IS, RB, RDP, and IO) and inputs (SE and CE) of DMUs, the scatter diagram of efficiency, output, and input can be shown in Figure 1. Figure 1a is the proposed approach, while Figure 1b is the traditional DEA method. The characteristics of high output students are discussed in the next section.

4. Discussion

The descriptive analysis of surveyed data, calculated results, and the diagram are shown. The discussions of three study purposes are as follows.

4.1. The Situation of Interdisciplinary Education for AI and Civil Engineering Fields in Taiwan

The 8/35 universities issued interdisciplinary AI courses for civil engineering in undergraduate and graduate programs in Taiwan. Comprising student engagement items, the mean of SE7 is 2.62. These data show that students were less accustomed to discussing with the faculty, similar to Chang et al. (2014) [27]. The problem of using mobile phones in the course still affected the students’ engagement, as Kuznekoffc and Titsworth (2013) found [28]. The score of SE8 is 3.99, which is different from the traditional courses in civil engineering. Noteworthily, effective teaching practice (SE8) is the most important item. In the interdisciplinary course on AI and civil engineering, programming procedures make students understand the kernel concept of the industry requirements. Self-discipline and adherence to classroom norms, such as arriving in class on time and not using mobile phones, can improve learning efficiency. The learning environment and channels of peers and other personnel are easy to understand. However, based on the analysis of the questionnaire content, the effect of self-disciplined learning is good among peers and other personnel. Furthermore, students can learn and improve a lot by studying with a professional. Appropriate learning and practice can bring back and rehabilitate previous learnings, which may happen in investigative learning. It is important and necessary for students to understand other fields of knowledge and technology to solve civil engineering problems. It may also be valuable to ask students about their understanding of the different fields or ways to solve civil engineering problems and explore different solutions. Moreover, students are willing to apply cross-field learning and practical applications, as they think both are helpful.

4.2. The Comparison of the Proposed Approach and DEA-PCA Method

Principal component analysis (PCA) is a common statistical tool in computing the principal components and performing a change of basis on data. PCA is used for dimensionality reduction by projecting each data point only on the first few principal components to obtain lower-dimensional data while preserving as much of the data’s variation as possible. Several pieces of literature discussed the combination or integration of the DEA and PCA. Particularly, Ueda and Hoshiai (1997) discussed the usage of PCA as a means of weighing inputs and outputs and parsimoniously summarizing rather than selecting them. The basic model and its modifications were proposed [29]. Furthermore, the hybrid approaches of the DEA and PCA are used in supplier selection, performance estimation, and score ranking problems [30,31,32,33,34].

The theory and applications of the Mahalanobis distance were discussed in many works of literature [14,15,26]. However, only a few articles discussed the hybrid approaches of the DEA and Mahalanobis distance. Therefore, the proposed approach focused on this problem.

The comparison of the DEA with non-orthogonal variables, DEA-PCA, and the proposed approach is shown in Figure 2. Figure 2a has two non-orthogonal variables X1 and X2. Based on DEA-PCA, the original coordinate axis X1 and X2 was transferred to Y1 and Y2. However, the meaning of variables (Y1 and Y2) was different from X1 and X2, as in Figure 2b. By the proposed approach, the non-orthogonal coordinate axis was transferred to orthogonal X1 and X2, as in Figure 2c. In explaining the variables, the proposed approach was clearer than the DEA-PCA method. Besides, the traditional DEA model could not easily explain the right-down point in Figure 1b through the education theory. The students’ efficiency (input = 2 and output = 16) is very low.

4.3. The Efficiency Analysis Feedback to the Students

The primary aims of efficiency and performance analysis of education were the feedback to the students. By ranking the output grades, three groups (outputs ≧ 10, 10 > outputs ≧ 5, and 5 > outputs) were clustered. The mean test of input and efficiency by t-test and analysis of variance (ANOVA) is shown in

Table 7. DEA -Mahalanobis analysis results.

Statistical Parameters	Technical Efficiency from CRS	Technical Efficiency from VRS	Scale Efficiency
Mean	0.291	0.487	0.600
Standard deviation	0.326	0.402	0.306

. The result showed that the differences among groups 1 to 3 are significant. Figure 3 shows the diagram of the inputs and VRS efficiency of different groups. In particular, the result showed that group 1 has a higher input and efficiency. Although in groups 2 and 3, the input of some students (n = 20, 8.4% of samples) is high (>4), their efficiency is notably low.

DMUs can be divided into strong efficiency (efficiency ≧ 0.8), middle efficiency (0.8 > efficiency ≧ 0.2), and low efficiency (efficiency < 0.2). The slack variable is a variable added to an inequality constraint to transform it into an equality constraint. Introducing a slack variable replaces an inequality constraint with an equality constraint and a non-negativity constraint on the slack variable. In the DEA method, the slack variable analysis can help DMUs increase the students’ efficiency. The technical efficiency mean and standard deviation of a special group (N = 20, output < 10, and input > 4) are 0.261 and 0.316. These data show that this special group may adjust inputs for better efficiency. Based on slack variable analysis of this special group, student engagement is the key factor. The suggestions are as follows:

i.

Conducting effective teaching practices;

ii.

Discussion with faculty frequently;

iii.

Attending expert meetings or seminars; and

iv.

Learning hard with peers.

The sensitivity analysis of this special group showed that effective teaching practice is the key factor in the artificial intelligence course for civil engineering students. These students may have 37% more technical efficiency by paying 21% more inputs.

5. Conclusions

The artificial intelligence education for civil engineering is significant, based on the industry requirements and development in the near future. The course names, number of students, and related factors for AI learning in Taiwan were described. The input and output factors for estimating efficiency were defined by modifying the useful and common scales in literature. The 237 datasets from students who accept AI courses were surveyed. Due to the non-orthogonal attributes in datasets, it was hard to estimate the efficiency using a traditional DEA method. Many studies estimated students’ efficiency based on the DEA-PCA method. However, the variables may be changed by the procedures of the axis transformation in PCA. Hence, a new DEA-Mahalanobis distance approach was proposed to solve this problem. Moreover, the method’s advantages were discussed, where the improving learning strategy in interdisciplinary learning was suggested to increase students’ efficiency. If the attributes are independent and orthogonal, the proposed approach is similar to DEA-PCA method. This may be a limitation of the proposed approach. Based on the sensitivity analysis of survey datasets, the result showed that effective teaching practice is the key factor in the artificial intelligence courses for civil engineering students. In the future, the relationship of students’ subjective and objective grades will be collected for long-term research.