SeisTutor: A Custom-Tailored Intelligent Tutoring System and Sustainable Education

Abstract

Education is the cornerstone of improving people’s lives and achieving global sustainability. Intelligent systems assist sustainable education with various benefits, including recommending a personalized learning environment to learners. The classroom learning environment facilitates human tutors to interact with every learner and obtain the opportunity to understand the learner’s psychology and then provide learning material (access learner previous knowledge and well-align the learning material as per learner requirement) to them accordingly. Implementing this cognitive intelligence in Intelligent Tutoring System is quite tricky. This research focused on mimicking human tutor cognitive intelligence in the computer-aided system of offering an exclusive curriculum or quality education for sustainable learners. The prime focus of this research article was to evaluate the proposed SeisTutor using Kirkpatrick four-phase evaluation model. The experimental results depict the enhanced learning gained through intelligence incorporated SeisTutor against the intelligence absence, as demonstrated.

1. Introduction

Quality education is a core element of sustainable development, intended to ensure equitable and inclusive quality education and encourage opportunities for lifelong learning for all. Research claims that education is critical to livelihood and the world’s long-term viability [1,2]. New digital technologies are transforming education in both informal and formal learning environments. This overarching goal is linked to several general issues influencing education in the digital era. Some of the significant developments and their impact on education can be stated as follows: (a) educational objectives; (b) learning environments and ecosystems; (c) learning and teaching processes; and (d) educational policy and governance.

According to the Sustainable Development Goal, equality (male and female) to access excellent technical learning and training practices is anticipated by 2030 [3]. Education is the cornerstone of improving people’s lives and achieving global sustainability [1,2]. Furthermore, equitable and inclusive education equips people to devise innovative solutions for their day-to-day lives. Incorporating digital inclusion and good quality educational practice enables the learners to deliver motivation, knowledge, and opportunities; offer skill training to employ SDG explication; and resolve the SDG challenges [4,5].

Human, cultural, ethical, and ecological principles are described as sustainable education development. It offers good teaching practices to societies that aid the growth of communities, institutions, organizations, and the competitive environment. Thus, it enhances the economic and social life [6,7]. Intelligent systems assist sustainable education with a wide variety of benefits, including recommending a personalized learning environment to learners [8].

The value of utilizing automated systems over long-duration teaching practices is becoming more apparent. Incorporating Artificial intelligence and expert systems in education helps gauge real-world scenarios. E-learning, technology, applications, e-teaching principles, and sustainable development are the most vital parts of a sustainable e-learning environment. The principles of intelligent learning or e-learning comprise the framework, models, and theories, whereas the principle of intelligent tutoring or e-teaching comprises a program, curriculum, and pedagogy. Good teaching practices and evaluation models, collaborative, and custom-tailored or personalized learning approaches work together to determine and recommend a custom-tailored pedagogy or tutoring strategy [9].

Personalization emphasizes self-development and the academic realm. The learners’ self-growth depends on the skill and knowledge gained through online courses and internet communities. A custom-tailored or personalized learning environment encompasses learning tools, services, and application or system that recommends learning per learners’ needs based on learner capabilities, a low-cost tutoring environment, and a personalized learner profile [10].

In order to effectively meet this learner-centric requirement, an intelligent tutoring system can offer personalized learning experiences to learners. The intelligent tutoring system (ITS) is an artificial intelligence technique that offers the learner exclusive learning material, aligned and gathered as per the learner’s grasping ability and preferred media of learning.

SeisTutor was developed for learning “Seismic Data Interpretation” as a subject domain of this research. This research aims to provide pastoral care to learners by cost-effectively offering one-to-one, customized learning material. SeisTutor offers a personalized learning environment. It brings personalization in identifying tutoring strategy (based on pretest (learning style test and domain knowledge test)), exclusive curriculum design, and observation of learner psychological state of mind during learning sessions. The feature of personalization is built on many aspects, such as accuracy of predicting tutoring strategy (based on pretest), curriculum design, and psychological parameters. The only technique to determine the performance of the personalization facility of SeisTutor is to appraise the system in actual circumstances (learners who are learning the course material). Appraise acts as a critical element for quality assurance because it enables the learner to provide valuable feedback on the learning experience and on learning content, which further helps to understand the learner’s perspective and makes the learning better.

The assessment of SeisTutor was accompanied in the 2018–2019 scholastic year. The objective of the evaluation was to examine how effectively SeisTutor personalized itself to fulfill the learner’s needs and whether SeisTutor helps enhance the learning gain for learning the “Seismic Data Interpretation” domain. In order to accomplish this, the following program was outlined:

• Perform a schematic literature review on how to evaluate the efficacy of learning;
• Generate an assessment for determining the efficacy of learning;
• Perform analysis to identify how efficiently learners learn with SeisTutor.

This article is organized into five sections. It begins with a schematic literature review on the evaluation of learning with ITS (Section 2). Further, Section 3 introduces the architecture of SeisTutor in detail, which includes the general description and mathematical justification of its functionalities. Section 4 illustrates the experimental analysis performed to evaluate the effectiveness of SeisTutor systematically. Section 5 elaborates on the comparative analysis with current work, and in Section 6, we finally conclude the implication of this work.

2. Background and Preliminaries

In this section, comprehensive coverage of the development of a sustainable, intelligent tutoring system (ITS) from traditional computer-aided instruction (CAI) to artificial intelligence techniques embedded tutoring systems and the evaluation prototype are discussed.

The academic realm provides a framework for considering long-term sustainable education [11]. Incorporating AI and expert systems in education plays a vital role in offering learning (in the form of blended, online, virtualization, visualization, etc.) at work, home, coffee shops, museums, city streets, and parks. Another trend is shifting our perceptions of context and spaces of learning. Finally, it considers the potential of digitally mediated learning to be accessible anytime and anywhere [12]. Here the teaching environment expanded temporally and spatially. Ubiquitous learning abandons the discrepancy between formal and informal learning. A planned curriculum and competent facilitators play a role in keeping individuals actively engaged; however, the learning is more inclined to need, purpose, and context [13]. These necessitate a paradigm shift from “curriculum-based” to “problem-based” learning, requiring a rethinking of learning motivations, processes, and content. One of the most remarkable features of providing learners with online learning possibilities is that they have significantly more autonomy and choice in pursuing paths of study that are fascinating and meaningful to them. Learning evolves from a recipient, passive model to a more self-directed, collaborative, and active model [14,15]. Teachers can collaborate with domain experts and technology to develop a learning environment that appropriately recognizes learner challenges and recommend an adaptive instructional strategy that can help understand the concept effectively. The goal of a “flipped” learning environment is not to replace teachers; instead, it is to automate the learning processes; enable the teachers to focus their talents, energies, experience, and creativity on tutoring.

Computer-aided instruction (CAI) systems store the learning material, which the learner further uses in different ways (representation) [16]. These systems have several limitations. These systems mainly focused on quantitative education (teacher-centric) rather than qualitative nature (learner-centric) education. The CAI systems adopt a highly primitive tutoring strategy, which leads to less interaction between the CAI tutor and the learner. Further advancement led to the origin of the field of Intelligent Computer-Aided Instruction (ICAI) or Intelligent Tutoring System (ITS) [17].

In 1950, The first ITS was technologized as CAI [18]. McDonald, Woolf developed intelligence embedded computer-aided instruction, which delivers learning material by establishing effective interaction with the learner. Thus, these systems emulate the cognitive intelligence of humans only to a small extent, i.e., provide necessary guidance based on the learner’s action.

ITS, developed by [19], creates questions on vocabulary and arithmetic but cannot adapt and model learner needs. In advancement to this, several adaptive systems were developed by the researchers [20,21,22]. Advancement in the learning systems led to significant modifications in the architecture of ITS. Further improvements in Pedagogy Model led to adapting the learning material to learner competency. Thus, incorporating AI techniques shifted to fine-tune pedagogical recommendation and learner feedback.

Physics is a well-documented subject domain. An intelligent tutoring system developed by the Andes comprises physics as the knowledge domain [23,24]. The Andes incorporates an intelligent feature of determining performance parameters, predicting learners’ subsequent actions, and identifying and recommending a suitable strategy. For decision-making, they utilize a Bayesian network.

In order to adjudge previous learner knowledge, pretest plays an important role. ITS developed by [25] offers a personalized learning environment by offering learning material to the learner based on previous knowledge about the course. The pathfinder technique was used to determine the previous knowledge of the learner. ITS developed by [26] proposed a solution for determining the learners’ curriculum by utilizing a profile framework.

Further advancement in this field uses a data mining technique to mine the meaningful learning path for the learner. This system tracks learner activity during learning and recommends the most suitable learning path. ITS developed by [27] incorporated two methodologies: determining the learning path and recommending the learning path. Initially, the system utilizes the apriori algorithm to generate an initial course coverage plan; they used course-concept analysis to determine the association among the course-concept and then adjudged the preferable course coverage plan.

Another [28] proposal uses a fuzzy rule base technique to determine the association between the list of materials and learner requirements based on web navigation. In the recent development in technologies, some concepts of ontologies, genetic-based algorithms, and artificial neural networks are used to recommend a suitable course coverage plan [29,30].

ITS developed by [31] utilizes the nature-inspired algorithm to adjudge the custom-tailored learning path. They used two critical parameters for fitness function; one is difficulty level and other associations between the course concepts. Another research work on e-learning systems makes use of the nature-inspired algorithm for determining optimal course coverage plans based on the incorrect response on the pretest [32,33].

In [34], the author used ACO techniques to recommend an adaptive learning path by considering the learner’s learning style. The authors of [35] utilized self-organizing techniques to recommend optimal course coverage plans to the learner. Similar practices can be applied through a probabilistic technique. In which nodes indicate the pedagogy items and edges indicate the hypertext links (preferred probabilities), and learners act as ants who have to traverse all the nodes [36,37,38] (Skinner. et al.; Brusilovsky. et al., 1996; Karampiperis. et al., 2004; Dhabi. et al., 2006; Bert van den Berg. et al., 2005).

Evaluating and validating an ITS is a challenging task due to the lack of standard agreement and procedure. The primarily used prototype for evaluating the training program of ITS is the prototype established by Donald Kirkpatrick [39,40]. Many researchers revised this prototype, but its basic architecture is still the same [41,42].

Kirkpatrick’s prototype comprises four stages of evaluation, shown in Figure 1 and briefly discussed in Figure 2.

Figure 1. Kirkpatrick’s four stages of evaluation.

Figure 2. Brief illustration of Kirkpatrick’s four phases or stages of assessment. Evaluation of Reaction [43,44,45], Evaluation of Learning [42,46,47], Evaluation of Results [45].

Computing results is the best way to quantify the effectiveness of any learning program, but it is challenging to conduct. The authors of Ref. [48] provided a statement regarding the evaluation that not all learning programs focus on impacting the learning performance of a learner; instead, they can perform for a purpose. Other researcher uses the Kirkpatrick model as a base model. The author of Ref. [49] introduced the fifth level of the Kirkpatrick model named Return of Investment (ROI), which is used to measure the effectiveness of learning or investment.

Kirkpatrick’s model is considered a base model for judging the efficacy of a training program by Ref. [48]. Their suggestion is to set the initial objective (from the learning program) and then monitor the fulfillment of objectives after the learning program. The authors of Ref. [50] disapprove of Kirkpatrick’s model, giving the following reason:

• Offline Test (Written Test) lacks validity and reliability in quantifying knowledge, skill, and attitude (KSA);
• The 100% response rate is idealistic;
• Control groups are not feasible in the learning program context.

Furthermore, educational organizations recommend considering the merits and demerits of various evaluation prototypes and methodologies to build an organizational-specific evaluation prototype that fulfills their requirements [51]. Furthermore, it was proposed that the evaluation prototype prioritizes both education practices and findings [52]. Valid, reliable, inexpensive, and acceptable is the feature of an ideal evaluation prototype. Moreover, the evaluation prototype may encompass quantitative, objective, subjective, and qualitative methods. Thus, evaluation results are advantageous for determining the learning attainments of the learning program [53,54,55,56].

The conclusion drawn from the literature recommends that the learning program include three levels of Kirkpatrick’s prototype. Kirkpatrick Level 3 and level 4 are very challenging to observe in an educational learning program [57] because level 1 and level 2 can be quantified during an ongoing learning session. However, levels 3 and level 4 require post-assessment analysis. Level 4 needs the rigorous observation of the inference of the learning program. For evaluation, there is no articulated framework. As per [57], continuous feedback and instruction help the learner achieve the best learning skills [58,59,60,61,62].

3. Proposed Prototype

This section illustrates the SeisTutor architecture and briefly discusses functionality incorporated in SeisTutor. SeisTutor is an ITS explicitly designed for the “Seismic Data Interpretation” subject domain. SeisTutor recommends learning contents as per learner performance in pretest (prior knowledge assessment test). In addition, SeisTutor keeps track of the learner’s behavior (psychological state) during the entire learning session and conducts a test to determine the degree of understanding of the topic. SeisTutor is adaptive, i.e., content and link-level [63]. Adaptive content level indicates that the learners with different performances in the pretest (prior knowledge test) obtain different learning material. Before indulging learners in the learning session, SeisTutor enables learners to go through a pretest. There are two kinds of assessments performed in pretest: a learning style test and a prior knowledge test. The current focus of this research is on the prior knowledge test. SeisTutor observes the learner’s performance during the tests and aligns the learning material accordingly.

For link elimination, ideally, the curriculum covers all the links of sub-topics to learn—however, the links eliminated are not in the determined curriculum for the learner. SeisTutor continually observes and stores the learner’s action, performance, and behavior. This information is further utilized for making intelligent strategic plans for recommendation and evaluation work.

3.1. SEISTUTOR Architecture

SeisTutor follows all the guidelines for implementing an intelligent tutoring system. The critical functional model of an ITS is the domain model, learner model, pedagogy model, and learner interface. SeisTutor architecture comprises a learner interface, pedagogy model, learner model, and domain model [64,65,66]. The following subsection briefly illustrates the various components involved in SeisTutor.

3.1.1. Domain Model

This model is described as a cluster of concepts. Here concept terminologies indicate the single topic. Other terminologies are used in the different research papers, such as knowledge element, object, learning outcome, and attribute. In the current context, concepts possess a prerequisite relationship to each other. Each concept is further segregated into learning units. SeisTutor utilizes unit variant technologies to attain content-level adaptation. Content level adaptation indicates that the system has alternative units and recommends the learning units based on learner grasping and learning style [67,68,69].

3.1.2. Pedagogy Model

This model consists of various rules and logic that build a knowledge infrastructure essential for adapting the learning materials as per the learner’s:

Curriculum Planner

The planner generates a curriculum as a sequence of learning units covered during the learning session [64];

Learning Assessment

Learning assessment is the process of determining the learner’s learning process. The accuracy of this adjudging model acts as a critical factor that affects the adaptation practice;

Understanding Assessment

The understanding assessment identifies the learner’s degree of understanding of the concept. SeisTutor makes any decision by referring to all the models, such as determining the curriculum for the learner and offering the learning content as per the learner’s grasping level and learning style.

3.1.3. Learner Model

SeisTutor captures three characteristics of a learner. This model captures the learner’s activity during the learning session and stores and updates the learner’s information for making the decision. This information is fruitful for the system to adapt based on learner characteristics (grasping level and learning style).

Learner Demographic Information

Learner demographic information includes learner’s basic details, such as name, learner username, and email id. This information is used to create the learner profile and collected when the first-time learner signup with SeisTutor.

Psychological State

It recognizes the learner’s emotions during the tutoring session. Recognizing the psychological state of mind is essential because it helps determine how far the learner liked the learning contents during the learning session. SeisTutor determines six emotions, i.e., smile, sad, neutral, surprise, fear, and anger.

This psychological state recognition module becomes triggered as soon as the learner starts the learning session, as shown in Figure 3. Initially, participants belong to both the studies and undergo the initial assessment phase (Pretest); after the pretest, their learner profile and learning style are determined. The I2A2 learning style prototype is utilized to ascertain the learning style [67]. Here I2A2 is the name of the learning style, i.e., imagistic (learning through observing), intuitive (learning through interpreting written words), acoustic (learning through listening), active (learning through action). Based on tutoring strategy (study 2 (learner study without customized learning) and study 1 (learner study with customized learning)) and course coverage plan, the suitable mode of learning (study 1 (learner study with customized learning)) is chosen by the learner and pedagogy model and offers customized content through learner interface. Thus, SeisTutor examines the participant’s attainment in the pretest and predicts the individualized tutoring strategy (study 2 (learner study without customized learning) and study 1 (learner study with customized learning)) and custom-tailored curriculum for the applicants (study 1 (learner study with customized learning)).

Figure 3. Flow Diagram of CNN-based Emotion Recognition module.

As illustrated in Figure 3, the CNN-based emotion recognition module is instantiated as soon as the learner begins the learning session. This emotion recognition module takes the snap of the learner via webcam, which acts as an input to the CNN-based emotion recognition module. This module determines the learner’s psychological (emotion) state, which is gauged for future analysis (phase 1: Evaluation of Reaction). The gathering psychological (emotion) state is repeated until the learner completes all the learning contents (topics) associated with all the weeks.

Learner Performance

It evaluates the learner’s performance by quantifying their learning by organizing quizzes and tests. Learner performance was quantified based on one parameter, i.e., the number of correct responses. As soon as learners finish the learning content of each week, they have to complete a quiz and understanding test. One quiz is associated with every week (shown in Figure 4). Each quiz contains five questions, and each question contains only one hint. Hints appear to the learner based on the learner’s request to seek help to solve the question. SeisTutor asks the learner to summarize the learned concepts in understanding the test. Based on the user entered information, dictionary-based sentimental analysis is performed. The result gives a score out of 100, which tells the learner’s overall understanding of the concepts.

Figure 4. Quiz and Understanding Test Representation.

In order to determine the overall learning gain, SeisTutor has pretest and post-test assessment scores. The average learning gain is computed (using Equation (1)).

$$Learning\_Gain=\left(PostTest\_Scor{e}_L-PreTest\_Scor{e}_L\right)$$

(1)

Learning History

It keeps track of the learner’s activity, such as login time, total time spent, etc., and its interaction details during the entire tutoring session. SeisTutor utilizes this information to make necessary action, which further helps to make the whole learning process effective.

In order to start tutoring with SeisTutor, learners need to register themselves first (see Figure 5). After registration, SeisTutor creates a learner account and instructs the learner to complete a pretest. Pretests are comprised of two tests:

Figure 5. Flow diagram of learner characteristics model.

• Prior Knowledge Test;
• Learning Style Test.

Prior Knowledge Test

The prior or Previous Knowledge test is the preliminary test used to identify the domain’s elementary knowledge to know the learner’s initial learning level. This test is responsible for analyzing the learner’s knowledge about the domain. The domain model examined the test result and categorized the learner into three learning profiles, i.e., “beginner, intermediate and expert” (see Figure 5). A further outcome of this result is also responsible for determining the curriculum, which is exclusively designed for the learner. This test comprises twenty questions, and all the questions are verified by the domain expert of seismic data interpretation.

Learning Style Test

This test is responsible for determining preferred media for learning. The research has noticed that learner performance is gradually increased if learning material is provided as per preferred learning media (see Figure 5). Thus, this test analyses the learn-er-preferred media. The learner model examines the test result and categorizes the learner into four learning styles, i.e., “Imagistic, Acoustic, Intuitive, and Active”.

Based on these two tests, the pedagogy model determines the tutoring strategy (the result of learning profile and learning style). A tutoring session then begins with the determined tutoring strategy. As soon as learning begins, psychological features are triggered, capturing learner emotions and saving the results in the database for future reference. After completion of every week, the learner is tested. Based on the test result, learner performance and degree of understanding are computed.

3.1.4. Learner Interface Model

This model helps to establish communication between the learning system and the learner via multiple visualization tools (audio, video, textual, navigation, etc.).

4. Evaluation of SeisTutor

After tutoring, post-tutoring assessments of learners are performed by SeisTutor. This section depicts the analysis used to determine the impact of both the studies, i.e., study 1 (learner study with customized learning) and study 2 (learner study without customized learning) learning methods with SeisTutor. Study 1 (learner study with customized learning) exercises learning practices with features such as personalized curriculum design, recognition of the psychological state of the learner during the learning process, and learner’s degree of understanding of taught concepts. When it is not practiced, it is characterized as study 2 (learner study without customized learning). An experimental comparison of both the study groups was tabulated (see Figure 6 and Table 1). These variances help to identify the discrepancy in the learning experiences.

Figure 6. Flow of accomplishing the evaluation process.

Table 1. Feature of study 1 (learner study with customized learning) and study 2 (learner study without customized learning) evaluation groups.

	Study 2 (Learner Study without Customized Learning)	Study 1 (Learner Study with Customized Learning)
Personalized Tutoring contents	Offer learning content (similar curriculum) based on tutoring strategy.	Offer personalized learning content (different curricula) based on the determined prior knowledge level and tutoring strategy [64,65,66].
Psychological State tracking	Psychological state of the learner is not tracked during an ongoing learning session.	Determine psychological state of the learner during an ongoing learning session.
Degree of Understanding computation	Learner’s understanding of the concept is not adjudged.	Quantify learner’s understanding of the concept.

4.1. Experimental Design and Methodology

The SeisTutor evaluation is the fundamental piece of the development of this framework. In order to quantify the adequacy and effectiveness of SeisTutor, assessment tests were conducted. The SeisTutor was tested on a selected population of students, teachers, and both (teachers and students) from an anonymous university. A total of 60 learners was volunteered in the evaluation process. Based on their compliance in the participation, a compliance agreement form was issued that demonstrates essential details related to the assessment process. It is requisite for each applicant to give their approval for participation in the assessment process. Applicants were haphazardly assigned one of the groups. Thirty-two applicants were in study 1 (learner study with customized learning), and the remaining were in study 2 (learner study without customized learning).

Out of 60 learners, 30% are learners pursuing graduation, 17% are graduates, 35% are post-graduates, and the remaining minimum qualifications are doctorate (Ph.D.).

Twelve percent of learners were in the 18–20 age group, 18% of learners were in the 20–22 and 22–24 age group, 5% of learners were in the 24–28 age group, 12% of learners were in the 28–32 age group, 22% of learners were in the 32–34, and the remaining were in thh above 34 age group (shown in Figure 7. SeisTutor is explicitly created for the “Seismic-Data-Interpretation” domain. As a result, it is intended to be used by participants or learners belonging to the petroleum engineering and exploration domain. Thus, to quantify the effectiveness of SeisTutor, undergraduate learners (B.Tech/B.E. Petroleum engineering), teachers (Petroleum Engineering dept), and others (government exploration industry) are taken into consideration (refer to Figure 7).

Figure 7. Learner Demographic Characteristics.

The learner underwent a pretest as soon as they were registered with SeisTutor. Their learning style and grasping level (learning level) were adjudged as 28 learners, as mentioned above, were in study 1 (learner study with customized learning). Based on their responses in the pretest, the custom-tailored curriculum was determined, which is realigned and reorganized from the domain (content) capsule. SeisTutor examines every learner involved in study 1 (learner study with customized learning), identifies their psychological state (emotions) during the learning session, and quantifies the degree of understanding of the concepts. The remaining learners in study 2 (learner study with customized learning) follow the standard curriculum for learning, i.e., contents in the same sequence (irrespective of their prior knowledge about the domain). Thus, all the learners follow the same learning path. The point to underline is that their pretest performance is not used for exclusive learning path recommendations. The learning session begins weekly for both the study groups and subsequent post-assessment tests.

4.2. Data Preparation

Before analysis, obtained data underwent a data screening phase. In this phase, the elimination of missing values and data normalization was performed. For deducing conclusion about the effectiveness of learning through SeisTutor, learner’s performances, i.e., prior knowledge test (pretest), understanding test, psychological state results, and quiz test (post-test) during learning, were taken into consideration. SPSS version 25 was used for accomplished analysis.

4.3. Min Max Normalization

The learner classification parameters (prior knowledge test (0–5), post-assessment test (0–5), learning gain (0–5), learner emotion (0–100)) of each learner were used and normalized using the methods called Min–Max. It converts a value of $Y=\left\{y_1,y_2,y_3,\dots \dots \dots ,y_n\right\}$, and converges in the range of [A,B]. The formula for score standardization is specified below, where A is the lowest range; B is the highest range. In our case [A,B] is [0, 10];

$$Z=\left\{\frac{y_i-Lowest value in Y }{Highest value in Y-Lowest value in Y }\right\}\times \left(B-A\right)+A$$

(2)

5. Result and Discussions

The evaluation of SeisTutor was performed using the Kirkpatrick evaluation model. As discussed in Section 2, the Kirkpatrick evaluation prototype comprises of four-phase shown in Figure 1 and Figure 2.

5.1. Kirkpatrick Phase 1: Evaluation of Reaction

Evaluation of reaction as its name indicates the reaction (emotion) of the learner during learning or how far the learner likes the learning content and teaching process, i.e., pedagogy. SeisTutor incorporates an emotion recognition module and an open-end questionnaire (Learner Feedback).

The Min–Max normalization utilizes to maintain the uniformity, which converges the original values in the scope of [0, 10].

As mentioned above in Table 1, the psychological state of the learner was determined only for the applicants involved in study 1 (learner study with customized learning). Thus, their descriptive states are shown in Figure 8. From the stats shown in Figure 9, the average mean score percentage among 28 applicants is 44% for the emotion happy, 40% for the emotion neutral, 36% for the emotion angry, 32% for the emotion surprise, 30% for the emotion fear, and 24% for the emotion sad. Thus, one can deduce with confidence that, on average, learners are happy with the learning content and teaching process, i.e., pedagogy.

Figure 8. Descriptive statistics of psychological parameter of learner for study 1 (learner study with customized learning).

Figure 9. Learning gain of study 1 (learner study with customized learning) and study 2 (learner study without customized learning).

5.2. Kirkpatrick Phase 2: Evaluation of Learning

Evaluation of learning, as its name indicates, is how effectively learners grasp the learning content. SeisTutor conducts a small quiz and degree of the understanding test to adjudge the learner’s overall learning.

The average learning gain of applicants involved in study 1 (learner study with customized learning) is 22%, and in study 2 (learner study without customized learning), it is 12%. Thus, it is concluded that if learning material is offered as per the learner’s inclination with an exclusively designed curriculum based on the learner’s previous knowledge, then the proposed SeisTutor succeeds in enhancing the learner’s curiosity and interest, which indirectly enhance the overall learning gain (refer Figure 9 and Figure 10).

Figure 10. Learner’s Learning Gain.

These tests were performed on learning gain and degree of understanding of study 1 (learner study with customized learning) through SeisTutor. Figure 10 illustrates the progressive learning gain of 22% discovered among learners, those applicants who participated in study 1 (learner study with customized learning) SeisTutor. Furthermore, this information (statistical) was used in the analysis, i.e., Bivariate Pearson Correlation.

Understanding tests were designed and conducted only for study 1 (learner study with customized learning) because it strongly proves the effectiveness of learning gain achieved by intelligent features incorporated SeisTutor. Here, the correlation of learning gain with itself is one because a variable or parameter is perfectly interrelated. The Pearson correlation of learning gain with a degree of understanding is 0.484, and the significance is two-tailed, i.e., the p-value is less than 0.01 (refer to Table 2 and Table 3). Thus, one can confidently say that Learning gain and degree of understanding are statistically significant linear relationships. (see Figure 11).

Table 2. Average mean score of learning gain and degree of understanding.

Study 1 (Learner Study with Customized Learning) Parameters	No. of Participants (n)	Learning Gain
		Mean (/10)	Standard Deviation	Mean%
Learning Gain	28	2.2170	1.02795	22%
Degree of Understanding	28	2.5467	1.31201	25%

Table 3. Correlation matrix between learning gain and degree of understanding.

Parameters		Learning Gain	Degree of Understandings
Learning Gain	Pearson_correlation	1	0.484 **
	Sig._(2-tailed)		0.009
	N	28	28
Degree of Understanding	Pearson_correlation	0.484 **	1
	Sig._(2-tailed)	0.009
	N	28	28

** Correlation is significant at the 0.01 level (2-tailed).

Figure 11. Linear relationship with learning gain and degree of understanding.

5.3. Kirkpatrick Phase 3: Evaluation of Behaviour

Evaluation of behavior is quite tuff to quantify. In order to measure this, SeisTutor collects feedback from the applicants. As the learner completes all the learning concepts every week, SeisTutor requests the learner’s feedback. In this section, a conclusion from the learner feedback is drawn.

Learners who were the impeccable part of this evaluation of SeisTutor had a good perception of the system, and their feedbacks were very encouraging. It was shown in their reactions whether they would like to recommend the SeisTutor to others who need to take this study. Around 93% of the learners showed that they would recommend it to others, out of which 48% showed strong agreement, and the remaining 45% agreed on a recommendation as well (see Table 4 and Figure 12). The overall satisfaction with SeisTutor was around 93%, out of which 45% were strongly satisfied, and 48% were satisfied as well. It was also observed that learners’ studies became productive with the SeisTutor.

Table 4. Learner feedback on effectiveness of SeisTutor.

	Questions	Degree					Applicant Evaluation Group
		Strongly Satisfy	Satisfy	Neutral	Dissatisfy	Strongly Dissatisfy
System Effectiveness	What is your overall level of satisfaction with SeisTutor?	27	23	6	3	1	Study 1 Study 2
	Learning through this tutoring system (SeisTutor) was easy.	27	26	5	1	1	Study 1 Study 2
	Did you feel that you were achieving learning outcomes?	30	21	6	3	0	Study 1 Study 2
	I would recommend a course through SeisTutor with no instructor help	29	24	3	4	0	Study 1 Study 2
	Would you recommend SeisTutor to individual who needs to take another course?	25	27	5	3	0	Study 1 Study 2
	Did SeisTutor support you to make your study productive?	28	27	3	0	2	Study 1 Study 2
	How well does this system deliver on your learning intentions?	31	21	5	2	1	Study 1 Study 2

Figure 12. Evaluation of learner behavior based on learner feedback.

Few questions were asked on the impact of the intelligent features provided by SeisTutor and were collected and summarized in Table 4 As some intelligent features are not provided for study 2 (learner study without customized learning) applicants, 28 effective feedback was taken into consideration from study 1 (learner study with customized learning) participants.

Most of the participants were happy with the tutoring strategy provided by the system, with 86% satisfaction, which includes 46% who were satisfied and 40% who were strongly satisfied. Eighty-five percent of participants felt that learning from their own learning experience made them perform better, with 40% who were strongly satisfied and 45% were satisfied. The participants were happy with the recommended exclusive curriculum by the system with 85% satisfaction, which includes 35% who were satisfied and 50% who were strongly satisfied. Most of the participants were happy with the recommended custom-tailored curriculum provided by the system, with 85% satisfaction, which includes 39% who were satisfied and 46% strongly satisfied. Ninety-two percent of participants agreed that the understanding test each week corresponds to the lesson taught, 39% strongly agreed, and 53% also agreed. Lastly, 82% of students agreed with the psychological parameter accurately determined with SeisTutor, with 39% who strongly agreed to this and 43% who were satisfied (see Table 5 and Figure 12).

Table 5. Learner Feedback on Adaptivity of SeisTutor.

	Questions	Degree					Applicant Evaluation Group
		Strongly Satisfy	Satisfy	Neutral	Dissatisfy	Strongly Dissatisfy
Adaptivity/Personalization	Did SeisTutor satisfy you with dynamic creation of your learning profile?	27	21	7	1	4	Study 1 Study 2
	Were you convenient and satisfied with the tutoring strategy presented to you by SeisTutor?	24	19	9	6	2	Study 1 Study 2
	The information provided by SeisTutor is at a level that you understand.	29	17	12	0	2	Study 1 Study 2
	The tutoring session was at the right level of difficulty for me.	26	23	9	2	0	Study 1 Study 2
	As a learner, did you feel that your learning style was appropriately judged?	29	25	3	2	1	Study 1 Study 2
	Once tutoring began, and you were tutored, were your learning preferences sufficiently satisfied?	24	24	9	2	1	Study 1 Study 2
	Did the experience of learning by your own learning preference make you perform better?	24	21	6	7	2	Study 1 Study 2
	Based on your prior subject knowledge, has SeisTutor accurately determined an exclusive curriculum for you?	14	8	2	3	1	Study 1
	How satisfied are you with the exclusively determined curriculum?	13	7	4	3	1	Study 1
	As a learner, did you feel learning material enabled you to improve your ability to formulate and analyze the problem?	10	14	1	3	0	Study 1
	Are you satisfied with the sequencing of learning content?	14	09	3	2	0	Study 1
	Does sequencing of learning material relate to your previous knowledge? (Give Rating)	12	11	2	3	0	Study 1
	Does the learning content formulated under various learning levels and styles satisfactorily justify itself? (Give Rating)	17	7	2	0	2	Study 1
	Has this learning session been successful in improving your knowledge in the subject domain? (Give Rating)	12	11	2	3	0	Study 1
	Did this learning material fulfill your expectations?	11	13	2	1	1	Study 1
	Does the understanding test at the end of each week correspond to the lessons taught?	11	13	2	0	2	Study 1
	SeisTutor compels and supports me to complete the quizzes, understanding tests, and lessons.	13	12	2	0	1	Study 1
	The post-tutoring evaluation system (weekly understanding) as it exists is:	14	10	1	2	1	Study 1
	How do you rate the sequence of the lessons in the course?	18	8	0	0	2	Study 1
	Has SeisTutor accurately determined your psychological (emotional) state during the tutoring sessions? (Give Rating)	11	7	5	5	0	Study 1
	Do you feel recognition of emotion during ongoing tutoring is indicative of empathy for the system?	13	12	2	0	1	Study 1
	Is the course content relevant and well organized?	14	10	1	2	1	Study 1

The overall impact of the support provided by SeisTutor on the learning process was assessed through the learner’s feedback questionnaire answered by 60 participants (see Table 6 and Figure 12). The analyzed results showed that 87% of the students were happy with SeisTutor supports, with 47% satisfied and 40% strongly satisfied. In addition, 78% of the students were happy with the system navigation support to find the needed information, with 43% satisfied and 35% strongly satisfied. Lastly, out of 80% of the students, 42% had a strong agreement, and 38% agreed that the SeisTutor pre-learning procedure was beneficial for learning.

Table 6. Learner feedback on SeisTutor ongoing learning support.

Questions	Degree					Applicant Evaluation Group
	Strongly Satisfy	Satisfy	Neutral	Dissatisfy	Strongly Dissatisfy
How are you satisfied with the system support?	24	17	11	6	2	Study 1 Study 2
The system navigation support enabled finding the needed information easily.	21	17	9	11	2	Study 1 Study 2
Was the pre-learning procedure available in SeisTutor helpful to you?	25	17	6	9	3	Study 1 Study 2
Were you able to understand the language used to explain the lessons in SeisTutor?	33	21	6	0	0	Study 1 Study 2
The tutoring was flexible to meet my learning requirements.	30	21	7	2	0	Study 1 Study 2

The usefulness of the lesson components such as lesson explanations, revisions, presented quizzes, and the question hints in the learning process were evaluated in Table 7. The questionnaire feedback results show that 85% of students were happy with the content explained by SeisTutor, with 47% satisfied and 38% strongly satisfied. Moreover, 78% of students showed interest and agreed that the tutoring resources were adequate, with 35% strongly satisfied and 43% satisfied. It is clear that the quizzes and hints were realistic and focused on the learning contents provided by SeisTutor.

Table 7. Learner Feedbacks on learning material, quizzes and overall SeisTutor support.

Questions	Strongly Satisfy	Satisfy	Neutral	Dissatisfy	Strongly Dissatisfy	Applicant Evaluation Group
SeisTutor explained the content correctly.	23	25	3	8	1	Study 1 Study 2
SeisTutor made the course as interesting as possible.	31	19	9	1	0	Study 1 Study 2
The tutoring resources were adequate.	21	19	7	9	4	Study 1 Study 2
The presentation of the course content stimulated my interest during the learning session.	32	24	2	1	1	Study 1 Study 2
The course content is relevant and well organized.	29	25	3	2	1	Study 1 Study 2
SeisTutor supported me in understanding the content, which I found confusing?	27	26	6	1	0	Study 1 Study 2
Did the quiz at the end of each week correspond to the lessons taught?	28	27	3	1	1	Study 1 Study 2
The question-wise hints were helpful.	27	26	6	0	1	Study 1 Study 2
Did the SeisTutor react decidedly to your necessities?	26	21	11	1	1	Study 1 Study 2
Was the learning provided sufficiently to take the quiz?	36	18	4	2	0	Study 1 Study 2
During ongoing tutoring, assessments are a fair test of my knowledge and learning preferences.	32	21	5	2	0	Study 1 Study 2

The impact of the interactive graphical user interface, content organization, and design features of SeisTutor in the learning process was evaluated through the learner questionnaire responses described in Table 8. The questionnaire results revealed that 77% of the learners were 40% strongly satisfied and 37% satisfied with the interactive GUI and content organization of SeisTutor. Eighty percent of the learners were 44% strongly satisfied and 36% satisfied with SeisTutor to compel and support to complete the quizzes and lessons. Finally, students were happy with the account setup process with the system, which maintains the learners learning progress, grades, and basic account information.

Table 8. Learner Feedback on learning material presentation and on overall SeisTutor’s look and feel.

Questions	Strongly Satisfy	Satisfy	Neutral	Dissatisfy	Strongly Dissatisfy	Applicant Evaluation Group
How satisfied are you with the look and feel (user interface design) of this system?	32	19	8	1	0	Study 1 Study 2
How satisfied are you with the account setup experience of this system?	31	21	5	3	0	Study 1 Study 2
How pleasing is the color scheme used in this system?	28	21	11	0	0	Study 1 Study 2
How user-friendly is this system? Give a rating	27	21	8	2	2	Study 1 Study 2
SeisTutor compels and supports me to complete the quizzes and lessons.	25	27	6	1	1	Study 1 Study 2
How satisfied are you with the organization/customization of contents feature of the system?	29	26	4	0	1	Study 1 Study 2

The learner’s overall evaluation of the SeisTutor showed that 82% of learners agreed that tutoring should begin based on the learner profile considering their learning style and previous knowledge. Most of the students were unaware of their learning style, and about 80% of students never knew about it. Most of the students liked the artificial intelligence features such as automatic selection of the tutoring strategies, dynamically assessing the learner attainment, and flipping the tutoring plan or strategy.

The learner’s feedback questionnaire responses were retrieved and analyzed in an accessible fashion. Some learners put their suggestions to improve the productivity of SeisTutor. Most of the suggestions were general and related to the improvement of the system, and few were pessimistic regarding improvement of the quality of learning contents, improving the quality of the video lessons, and hints provided by the system. At last, through the overall evaluation of SeisTutor, 87% of learners agreed that they improved their learning performance and outcomes.

5.4. Kirkpatrick Phase 4: Evaluation of Results

Evaluation of results illustrates the overall impact of learning on the learner. In order to quantify the effectiveness of learning, paired wise sample T-test was performed on existing information, i.e., pretest and post-test results of participants involved in both the studies (study 1 (learner study with customized learning) and study 2 (learner study without customized learning)). Here two cases are taken into consideration.

Case_1: 

A Paired_Sampled_T-Test performed on study 1 (learner study with customized learning) consists of intelligent features depicted in Table 1;

Hypothesis_Case_1.0: 

Let the participants involved in study 1 (learner study with customized learning) have similar pretest and post-test mean scores;

Hypothesis_Case_1.1: 

Let the participants involved in study 1 (learner study with customized learning) not have similar pretest and post-test mean scores;

Case_2: 

A Paired-Sampled-T-Test performed on study 2 (learner study without customized learning), which consist of feature depicted in Table 1;

Hypothesis_Case_2.0: 

Let the participants involved in study 2 (learner study without customized learning) have similar pretest and post-test mean scores;

Hypothesis_Case_2.1: 

Let the participants involved in study 2 (learner study without customized learning) not have similar pretest and post-test mean scores.

The calculated T value ($T_{Stats}$,) for study 1 (learner study with customized learning) is 11.410, $p<0.01$ (see Table 9 and Table 10. On average post-test scores were 2.21786 points higher than pretest scores. Here the calculated $T_{Stats}$ is greater that $T_{critical}$, thus hypothesis 1.0 is rejected. From Table 9 and Table 10 one can deduce with confidence that there is a significant difference between pretest and post-test scores. The calculated T value ($T_{Stats}$,) for study 2 (learner study without customized learning) is 5.312, $p<0.01$ (see Table 11 and Table 12). On average, post-test scores were 1.24719 points higher than pretest scores. Here the calculated $T_{Stats}$, is greater than $T_{critical}$, thus hypothesis 2.0 is rejected. From Table 11 and Table 12, one can deduce with confidence that there is a significant difference between pretest and post-test scores.

Table 9. Statistical results of paired sample T-test of study 1 (learner study with customized learning).

Comparison_Item	Learning-Mode
	Mean	N	Std._Deviation	Std._Error_Mean
Post-test_of_Study1_Applicants	3.9375	28	0.39455	0.07456
Pretest_of_Study1_Applicants	1.7196	28	0.99740	0.18849

Table 10. Paired sample T-test results of study 1 (learner study with customized learning).

	Mean	Std._Deviation	Std._Error_Mean	95% Confidence Interval_of_the Difference		T	df	Sig (2 Tailed)
				Lower	Upper
Pair_1: Post-test_of_Study_1–Pretest_of_Study 1 (learner study with customized learning)	2.21786	1.02856	0.19438	1.81902	2.61669	11.410	27	0.000

Table 11. Statistical results of paired sample T-test of study 2 (learner study without customized learning).

Comparison_Item	Learning-Mode
	Mean	N	Std._Deviation	Std._Error_Mean
Post-test_of_Study2_Applicants	3.6525	32	0.58915	0.10415
Pretest_of_Study2_Applicants	2.4053	32	1.39565	0.24672

Table 12. Paired sample T-test results of study 2 (learner study without customized learning).

	Mean	Std._Deviation	Std._Error_Mean	95% Confidence Interval_of_the Difference		T	df	Sig (2 Tailed)
				Lower	Upper
Pair_1: Post-test_of_Study_2–Pretest_of_Study_2	1.24719	1.32804	0.23477	0.76838	1.72600	5.312	31	0.041

Both the studies reject the null hypothesis, which means both the studies provide effective training. However, this research aims to identify which study has a high impact on enhancing learning gain. In conclusion, the aim of both the studies was compared. Study 1 (learner study with customized learning) is higher than Study 2 (learner study without customized learning). Thus, study 1 (learner study with customized learning) has a significant difference in the post tutoring and pretest scores and provides more effective training than study 2 (learner study without customized learning).

From the research, it was observed that quality education plays a vital role in providing sustainable development of society and its need. Quality education includes a personalized learning environment and teaching process.

Learners of all ages are empowered with the knowledge and skills to confront the linked global challenges that we face, including climate change, environmental degradation, and the loss of biodiversity. Education for sustainable development empowers learners of all ages.

An intelligent tutoring system named SeisTutor was developed to accomplish the objective of personalized learning, which observes the learner’s preferences, grasping level, and knowledge level and recommends a personalized curriculum (learning environment) and personalized teaching process. This analysis concludes that the intelligent incorporated SeisTutor used in study 1 (learner study with customized learning) outperforms the SeisTutor used in study 2 (learner study without customized learning) in providing custom-tailored intended curriculum, identifying learner sentiments while learning, and computing the learner’s overall degree of knowledge that meets the learner’s sustainable needs. Thus, SeisTutor used in study 1 can provide effective quality education for sustainability.

Practicing sustainability empowers humans to construct knowledge, explore values, and develop an appreciation of the environment and its relationship to their worlds. This lays the foundations for sustainable education.

6. Conclusions

This article demonstrates the proposed sustainable personalized intelligent tutoring system prototype, named as SeisTutor. From the research, it was noted that a learner receives repetitive learning content, which indirectly disorients the learner. Thus, to address this issue, a bug model was utilized, which analyzes the bugs and recommends the custom-tailored curriculum to the learner. This technique helps to bring empathy to ITS. SeisTutor is not a passive tutor; it also analyses the learner’s behavior, i.e., the psychological state of the learner during learning, which helps to understand the learner’s experience with SeisTutor (about learning content). The experimental results revealed that SeisTutor utilized by participants in study 1 (learner study with customized learning) provided a customized learning sequence or path of learning material that endorses effective learning. Experimental analysis reveals the effective learning gain (when the learner receives the custom-tailored sequenced learning material) of 44.34% compared to SeisTutor used in study 2 (learner study without customized learning) (not sequenced learning material). In order to evaluate the overall effectiveness of SeisTutor, the Kirkpatrick four-phase evaluation model was utilized. The analysis reveals that the participants involved in study 1 (learner study with customized learning) attain, while study 2 (learner study without customized learning) attains study 1 (learner study with customized learning) provides effective learning.