Enhancing the Efficiency of Massive Online Learning by Integrating Intelligent Analysis into MOOCs with an Application to Education of Sustainability

Abstract

Massive Open Online Courses (MOOCs) is an innovative method in modern education, especially important for autonomous study and the sharing of global excellent education resources. However, it is not easy to implement the teaching process according to the specific characters of students by MOOCs because the number of participants is huge and the teacher cannot identify the characters of students through a face to face interaction. As a new subject combined with different areas, such as economics, sociology, environment, and even engineering, the education of sustainability-related courses requires elaborate consideration of individualized teaching for students from diverse backgrounds and with different learning styles. Although the major MOOC platforms or learning management systems (LMSs) have tried lots of efforts in the design of course system and the contents of the courses for sustainability education, the achievements are still unsatisfied, at least the issue of how to effectively take into account the individual characteristics of participants remains unsolved. A hybrid Neural Network (NN) model is proposed in this paper which integrates a Convolutional Neural Networks (CNN) and with a Gated Recurrent Unit (GRU) based Recurrent Neural Networks (RNN) in an effort to detect individual learning style dynamically. The model was trained by learners’ behavior data and applied to predicting their learning styles. With identified learning style for each learner, the power of MOOC platform can be greatly enhanced by being able to offer the capabilities of recommending specific learning path and the relevant contents individually according to their characters. The efficiency of learning can thus be significantly improved. The proposed model was applied to the online study of sustainability-related course based on a MOOC platform with more than 9,400,000 learners. The results revealed that the learners could effectively increase their learning efficiency and quality for the courses when the learning styles are identified, and proper recommendations are made by using our method.

1. Introduction

Due to the increasing impact of population growth, environmental degradation, and resource depletion, sustainable development has gradually become a worldwide focus and received great concerns from most countries. Promoting educational equity is the one of 17 key goals in the Sustainable Development Goals (SDGs), officially known as “Transforming our world: the 2030 Agenda for Sustainable Development” issued by United Nations in 2015 [1], i.e., improving people’s overall quality, ensure inclusive and equitable quality education for boys and girls, man and woman, eliminate gender disparities and promote lifelong learning opportunities for all. Plenty of efforts have been dedicated to realizing these goals, including various innovative ideas, advanced teaching methods and facilities, and indispensable infrastructure. Among those, the open education service platform equipped with massive learning resources, personalized learning plan and instructions, and efficient measurement and evaluation technologies, is one of the most critical strategic elements for effective learning, especially in higher education and continuous education. The importance of this platform not only lies in promoting and integrating the global education resources, but offers the equal chances to all learners to access the most excellent subjects and instructors.

Various schemes and their technical supports have been designed since the idea of open education was concerned, among which the Massive Open Online Courses (MOOCs), emerging in 2012, has attracted wide attention and been adopted by a lot of educational organizations [2]. MOOCs start a new era of open education which makes lifelong learning and widely sharing of high-quality education resources being feasible and acceptable via the power of the internet. Different from to Open Courseware (OCW) [3], initiated by MIT in 2001 which provides global learners and instructors free access to its undergraduate and postgraduate course materials through the internet, MOOCs not only offer learning resources and instructional services for mass learners, but also provides user forums to support community interactions among students, professors, and teaching assistants, making it easy to communicate with course instructors and discuss learning issues with fellow learners. Aiming at providing low-cost and equal learning opportunities to more people, MOOC and its service providers obtain widespread support from governments, different social organizations, and even venture capitals [4]. According to an incomplete statistic from Class Central, 2600 new courses were announced in 2016 (up to 1800 in 2015), and the total number of MOOCs reaches 6850 contributed from over 700 universities. The total number of students who registered at least one subject of MOOC is 58 million, while the number for 2015 is only 35 million [5]. Traditionally, Coursera, edX, and Udacity were top three MOOC providers in the world, but in 2016, xuetangX.com from China has exceeded Udacity in user base and courses hosted. The fast growth of participants and courses shows that MOOC has gradually become an essential component of the modern education system which can effectively facilitate the people to accept high-class education in a more efficient and equitable manner.

Yet, on the other hand, there are still some problems with MOOCs which affect the expected performance of it. Although nowadays MOOCs are well designed and operated by outstanding teaching teams, even the completion rate for most courses is below 13%, typically range from 2% to 10%, in view of thousands of participants enrolling in these courses [6,7]. Investigations have been conducted to explore the possible resolutions. Some authors argued that student intentions should be taken into account [6], but problems about learning experience and styles have been less discussed. For example, many of the MOOC courses are still simply organized in static mode, and teach all the learners in the same way, lack of consideration about their different needs and characteristics, such as prior knowledge and learning patterns. In fact, many courses, especially those from interdisciplinary subjects, require higher individualization in teaching and learning to enhance the learning performance. Sustainability-related subjects typically fall into this category.

Sustainability education involves the integration of social, environmental, and technological elements with economic considerations, it is a broad concept and difficult to precisely define. The concept of “sustainability education” first emerged in “Our Common Future”, also known as the Brundtland Report, issued by the United Nations World Commission on Environment and Development (WCED) in 1987 [8]. UNESCO (United Nations Educational, Scientific, and Cultural Organization) also launched Education for Sustainable Development (ESD) as a guide to teaching and learning that promotes sustainable development growing from an idea into a global movement [9,10]. Nowadays, there are two main research fields in the education of sustainable development: (1) learning contents: what kind of topics and contents should be taught in courses? How to properly represent these broad-scale and cross-disciplinary contents in sustainability? (2) Pedagogical methods: how should we teach students the sustainability-related courses, especially in an open and distant learning environment, facing the large-scale population of learners? Lots of efforts have been exerted on this issue, and researches on pedagogical methods are diverse from nationwide experiments [11,12] to learning specific courses with MOOCs. Quite some investigations indicate that individualized teaching and learning under the massive online learning environment is a significant way to increase the performance of sustainability education. This conclusion has greatly boosted the online education of Sustainability. For example, there is none of Chinese university providing sustainability-related online courses in 2015 [13], while there are nine courses on xuetangX.com in 2017.

In recent years, increasing attention has been paid to the characteristics of learners such as learning styles [14,15,16,17,18,19], including the impact on learning performance and how such individual characteristics should be supported by adaptive systems. Students' learning styles can be determined in many different ways [20].But other scientists state that there is no adequate evidence base to justify incorporating learning styles assessments into general educational practice [21], thus this mismatch between practice and evidence has provoked controversy, and some have labeled Learning Styles a ‘myth’ [22]. However, all these studies or discussion were based on experiments and evidence in traditional classroom, which means, learning style measurement or tailoring teaching to a student's preferred style are low cost-effective. By contrast, with the rapid development of artificial intelligence and internet technology, online massive individualized education becomes possible, educational interactions can be controlled by system with an acceptable cost. Researches on providing adaptive learning materials with different learning styles in online learning environment [23] and employing fuzzy logic to determine users’ quality of interaction [24] indicates the key success factors could be different between online and traditional offline classroom, demonstrating the potential of new technology.

The first and crucial step to implementing individualized learning is to identify the personal characteristics of the learners. Graf and Kappel introduced the learning styles to learning management systems in an effort to increase the adaptivity of the system, in which learning styles were defined by adopting the Silverman learning style model [25]. With their approach, however, a survey including 44 questions which takes more than 10 minutes has to be completed in order to induce the individual learning style, this is quite boring and inconvenient to most of the learners and even lead to erroneous or unfaithful responses. Moreover, the learning style will change in case of learning pressure or situation changes in the learning process, therefore the static model cannot accurately reflect the individual characters in a changing environment.

The primary purpose of this work is to design a real-time intelligent interaction mechanism which can automatically and dynamically identify the learning style of the learners, and then to provide proper instruction methods and materials in accordance to the personal characteristics of different users for improving their learning efficiency in massive online learning environment. Sustainability subject is chosen as the case to demonstrate the effectiveness of the method in this paper because it is a typical interdisciplinary subject and more suitable for individualized learning as mentioned before. In addition, sustainability-related courses have been adopted by more and more educational organizations, how to teach the courses in a more effective and efficient manner is with particular significance.

The contents of the paper are organized as follows. The problem of individualized learning is briefly introduced in the next section. And the method of how to dynamically identify the learning styles is presented in Section 3. To demonstrate the performance of the proposed method, it is applied to the online study of sustainability-related courses based on a MOOC platform with more than 9,400,000 learners, and the results are analyzed in Section 4. Finally, some conclusions are summarized and possible future works are recommended.

2. Individualized Learning Problem

In traditional education, instructors need to prepare the course contents and design the teaching process. Besides, they have to realize the students’ background and pay close attention to students’ behavior to understand their characters in learning and provide special arrangements after class. These interactions are quite essential for achieving better performance in study. Nowadays in massive learning era, online education becomes one of the most important components of modern education system. With the power of computers and internet, the system can provide 7 × 24 h personalized service and treat learners with great patience. And with the great progress in artificial intelligence individualized learning also becomes possible. One of the important ways to implement the individualization is to integrate the adaptivity to learning management systems. The following three fundamental problems have to be addressed for this design.

2.1. Learning Style Modeling

Various learning style models were proposed in the last 50 years, with concerns on different aspects, such as student characters, emotional situations, cognitive styles, and even environments. Among the existing studies, Felder-Silverman Learning Style Model (FSLSM) is the one which has been most wildly adopted, especially in adaptive learning systems [26]. FSLSM took many advantages of previous work, and the design of each of the four dimensions was greatly inspired by other learning style models, e.g., the learning style model by Kolb [27], Pask [28], as well as the Myers-Briggs Type Indicator [29]. With FSLSM, learners are characterized from four dimensions. These dimensions are based on the main concerns in the field of learning styles and can be viewed independently from each other, indicates how learners prefer to processing (Active/Reflective), perception (Sensory/Intuitive), input (Verbal/Visual), and understanding (Sequential/Global) information [26]. Learning styles and their main characteristics are summarized in

Table 1. FSLSM Learning style dimensions and keywords group of learners’ behavior.

Dimensions	Style	Keywords Groups
Processing	Active	Try something out Social preferred
	Reflective	Thinking more detail Impersonal preferred
Perceive	Sensing	Existing ways Concrete material Careful with details
	Intuitive	New ways Abstract material Less attention to details
Receiving	Visual	Graphics
	Verbal	Oral and written words Difficulty with visual style
Understanding	Sequential	Detailed oriented Sequential progress From parts to the whole
	Global	Overall picture Non-sequential progress Relations/connections

.

To identify the learning styles by FSLSM, Felder and Soloman developed the Index of Learning Styles (ILS), which includes a 44-item questionnaire [30]. The learners express their preferences by using values between +11 to −11 per dimension with step of 2, in completing the questionnaire. The values can well classify the learners into proper categories regarding learning styles, and can even identify the extent to certain category.

2.2. The Challenge of Initializing Learning Style

FSLSM considers learning styles as “flexible stable”, arguing that the learning styles of students come from their previous learning experiences together with other environmental factors [15]. That is to say, learning styles tend to be more or less stable but can change over time, so we have to keep on the identification process. The first step of study with LMSs is to identify the learning style which is the start of enhancing the learning process. As mentioned in Section 2.1, the traditional measuring method for ISL is to finish a questionnaire, which is convenient to carry out in a closed environment (e.g., a classroom) because the education institutes can easily arrange the students who are using the system to fill it out. However, it is almost impossible to ask every user to finish such a survey in massive online learning environment for time and cost concern. Hence the system has to face the challenge of initializing learning style, or so-called “Cold Start”.

2.3. Dynamic Detecting

It is very critical to introduce the adaptivity to the LMSs by developing the mechanism which is capable of dynamically identifying the learning styles to competently meet the requirement of FSLSM. However, it is impossible to trace the changing of styles by repeatedly asking the learners to complete the questionnaire. Therefore, an automated dynamic learning style tracking mechanism is the essential part of an efficient massive online learning environment.

Currently, there are two different approaches adopted for learning styles dynamic detecting based on FSLSM: data-driven approach and the literature-based approach. The data-driven approach aims at building a model that imitates the ILS questionnaire. It uses sample data to construct a model and derives the proper patterns for identifying learning styles from the behavior of learners. Then Decision Trees and Hidden Markov Models [31] or Bayesian Networks [32] are employed to obtain the parameters of the model. This approach only uses the fixed and specific data relevant to the particular course. Hence it strongly depends on the available data and the course characteristic, and is less flexible for the variety of subjects or when the student’s behavior style changes.

The literature-based approach is to use the typical behavior of students to get hints about their learning style preferences, then apply a simple rule-based method to deduce their learning styles based on the number of patterns matched. This approach is more generalized and applicable to the data gathered from any course. However, the approach might have problems in accuracy, because it is hard to estimate the importance of different hints used for calculating the learning styles.

The process of learning styles detection can be demonstrated in Figure 1.

Although the above two approaches can detect and predict learning styles to some extent with certain accuracy, there are obvious limitations when apply. For example, learning behavior could change in different courses due to the knowledge backgrounds and learning habits of learners; learning style could even change in different parts of the same course, for the difficulty level or knowledge type changes. This problem can be easily solved in traditional classrooms because the teacher can identify the changes and make the relevant adjustment timely. In a machine-based learning environment, however, the current system is not capable to detect such changes in learning styles of learners and hence might make the improper recommendations. Therefore, dynamic detecting is an essential requirement for modern massive online learning facilities.

3. Dynamic Identification of Learning Styles

About thirty years ago, Learning Management Systems (LMSs), also referred to as Virtual Learning Environments, was developed in the UK, comprising a collection of software and web applications that enable the online delivery of course materials as well as the tracking and reporting of student participation. Recently, Next-generation LMS has been proposed [33], also called Next-Generation Digital Learning Environments (NGDLE), referring to the development of more flexible systems that support personalization and follow the universal design standards. Although both systems have been widely used in technology enforced education they provide only a little, or even none in most cases, the capability of adaptivity for the learning process. To enhance the capability of LMS, an adaptivity mechanism is designed by introducing the dynamic identification for learning styles of learners based on FSLSM in this section.

3.1. Framework for LMS with Adaptivity

The framework of the enhanced adaptive LMS is proposed as Figure 2.

Learning style identification is initialized in User Modeling Process, and a basic user model is set up by user profiling based on demographic and user tracking data. The identified learning styles in terms of FSLSM can be dynamically updated by Behavior Analyzing module when users are interacting with the system, and new characteristics have been captured. On another side, learning materials are no longer simple static videos and texts. In Content Analyzing Process, various kinds of videos, texts, and subtitles are highly organized so that concepts can be extracted to become the nodes of knowledge graph by Concept Extraction module, and Concept Relation Mapping module builds a prerequisite relationship matrix between segments and courses. Finally, the Interacting Process provides adaptive learning path routing to learners based on their learning styles kept in the knowledge base.

One of the core functions of above system is to detect the learning styles dynamically. This can be realized by adopting intelligent analysis technologies. Recently, a large volume of methods about intelligent analysis has been developed, among which some, e.g., Neural Network and Deep Learning, have achieved great success and applied to various fields such as image and speech recognition. The main advantage of these methods lies in the strong capability of processing unstructured data, which also makes it possible to detect learning styles dynamically via the behavior sequence of learners in LMS.

By the identified learning styles, the system can provide individualized learning contents, learning path, and suitable method, which will lead to higher efficiency and more satisfying performance in learning.

3.2. User Modeling

According to FSLSM, learners can be categorized using Active/Reflective, Sensing/Intuitive, Visual/Verbal, Sequential/Global dimensions, ranked in different levels, and the adaptive learning system then provides appropriate contents and learning path correspondingly. For example, the system will provide Reflective users the method based on Problem-Based Learning Approach which employs a series of questions to lead the students finding the solution to the problems. And for the Active users who tend to get information aggressively, it will continuously push the latest ideas on Course Interactive Forum to them so as to inspire a more in-depth and efficient study.

Firstly, we set up a basic leaner property model for user i at time t as expression (1):

$$U_{i,t}=U\left(D_i,L_{i,t}\right)$$

(1)

where $D_i$ denotes demographical distribution of user i, and $L_{i,t}$ represents the learning style based on FSLSM with four dimensions defined as expression (2).

$$L_{i,t}=L\left(\left(AR,W_{AR}\right),\left(SI,W_{SI}\right),\left(VV,W_{vv}\right),\left(SG,W_{SG}\right)\right)$$

(2)

The initial assignment of learning style for user i $L_{i,0}$ can be determined either by the final state of his immediately previous course, or based on the demographical distribution if there is not a predecessor. Having initiated, the system will continuously observe and analyze the user behavior to detect the changes in learning style, as depicted in Figure 3.

3.3. Behavior Features Extraction

Based on lecture study on FSLSM, dimension values and rankings of learning style can be determined by the means of using user behavior sequences, which can be measured with ILS questionnaire. However, different knowledge structure and study pressure for different courses might lead to varying results in ILS, hence the static ILS cannot precisely reflect the behavior characteristics of learners. A more appropriate candidate is to differentiate the learners in terms of learning style by proper classification scheme based on their behavior features, instead of using the exact index values.

Graf proposed a kind of approach to automatically detect the learning styles with LMS [34], by which threshold-based detecting behavior patterns was calculated relatively in percentage after learning process. The thresholds model can indicate the trend of learning style by combining the behavior patterns and making the relevant analysis. However, it needs the data after the learning process and hence is not applicable to dynamic detection.

The main idea of the method in this paper is to take the advantages of both thresholds-based approach and FSLSM by setting up thresholds for user behavior patterns and mapping user behavior logs to four learning style dimensions of FSLSM as indicated in

Table 2. Behavior logs for learning style dimensions of FSLSM.

Channel	Learning Style	Featured Behavior Logs
1	Active/Reflective	Effective learning points density log
2		Forum visit density log
3		Forum post density log
4	Sensing/Intuitive	Content web page scrolling log
5		Video seeking log (Tracking forward/rewind clicks and timeframe)
6	Visual/Verbal	Picture stays timing log
7	Sequential/Global	Navigation Log
8		Effective learning points density log

.

Learning style detection module will continuously track the user behavior records and take time frame into consideration as well. Each log(channel), is mapped into a time(by day)-content(by chapter) based 2-dimensions array, the nodes of each feature events are associated with a timestamp and normalized to the same level. For example, effective learning point density log $L{E}_{i,j}$ including absolute learning time and relative learning progress can be mapped into day based time serials and content as expressed in Formula (3)

$$L{E}_{i,j}=LE\left(\left(\frac{t{v}_i}{T},t_i\right)·C_j\right)$$

(3)

where $t{v}_i$ is the playing time of video clip for learning action i, and T is total video content time, $t_i$ is absolute time of learning, and $C_j$ is index of the jth segment of course content. For the non-content related logs: channel 2 and 3, the node was filled with 0 for the lines does not contains data.

As an example, logs from “Data Structure” (Part 1, 2017), one of most popular MOOC courses on xuetangX.com with more than 23,000 students enrolled is examined. The course includes six chapters and lasts 73 days. Two learners are randomly selected and the logs for their effective learning points density are visualized as Figure 4.

Based on the aforementioned FSLSM summarized in Table 1 we can infer the difference in learning styles of the two users: (a) User #4441580 learning one by one chapter in sequence (Sequential progress) and got a higher point in learning point density than average (Think more detail, Detail Oriented), evidently expressed the tendency of Reflective and Sequential type; (b) User #6565569 learning more than one chapter in one day four times (Try something out, Overall picture, Non-sequential progress), thus tend to behave in a more Active and Global manner.

3.4. Intelligent Analysis Process

Convolutional Neural Network (CNN) is a class of deep learning, feed-forward neural networks that have successfully applied to analyzing visual imagery. The network can be trained to classify images in high recognizing efficiency with low error rate.

Recurrent Neural Network (RNN) is another popular NN model which has the strong capability of processing variable-length sequence data and is often applied to the field of Natural Language Processing. Some research employs a unified CNN-RNN framework learns a joint image-label embedding to characterize the semantic label dependency as well as the image-label relevance, and it can be trained end-to-end from scratch to integrate both information in one framework, the model achieves better performance than the state-of-the-art multi-label classification models [35]. The model for intelligent analysis in this paper is mainly composed of two parts, a CNN for learning styles detection and a RNN combined with gated recurrent unit for learning styles prediction. A hidden internal state was added in the conventional feed-forward deep models network [36]. Standard RNNs updates their hidden state $h_t$ using function as following

$$h_t=g\left(W{x}_t+U{h}_{t-1}\right)$$

(4)

where g is a smooth and bounded function such as the logistic sigmoid function, $x_t$ is the input of the unit at time t. The RNN outputs a probability distribution over the next element of the sequence, given its current state $h_t$.

Gated Recurrent Unit (GRU) aims at dealing with vanishing gradient problem for RNN, the unit gates learn when and by how much to update the hidden state of the unit [37]. The activation of the GRU is a linear interpolation between the previous activation and candidate activation

$$h_t=\left(1-z_t\right)h_{t-1}+z_t{\tilde{h}}_t$$

(5)

And the update gate $z_t$ is given by

$$z_t=\sigma \left(W_z·\left[h_{t-1},x_t\right]\right)$$

(6)

The candidate activation function is computed as

$${\tilde{h}}_t=\tan \mathrm{h}\left(\mathrm{W}·\left[r_t{h}_{t-1},x_t\right]\right)$$

(7)

The reset gate is given by

$$r_t=\sigma \left(W_r·\left[h_{t-1},x_t\right]\right)$$

(8)

The proposed hybrid architecture of two parts can be illustrated in Figure 5.

In the first part, a serial of log sequences are sliced into snapshot segments by pieces marked with t, t-1, and t-2 time frame, which are composed of different user learning styles. The data was processed by a CNN with a simple multi-layered network architecture with fully connected layers, named Observer. The input layer has 8 × 73 units, 8 channels, based on the snapshot vector, followed by two hidden layers of 128 units with the ReLU activation function. The output layer has four output neuron for different learning styles in four dimensions, $L_{i,t}$, which uses softmax activation function and outputs the probability of learning styles in each dimension for the given state.

In the second part, the output of the Observer is connected with a GRU-based RNN, named Inference Engine where the received learning style $L_{i,t}$ detected in the first part is used to predict the next style $L_{i,t+1}$. With the hidden state in multiple GRU layers, the input can be optionally connected deeper in network which may reinforce the memory effect, hence the performance of prediction can be improved.

4. Model Training and Test

4.1. Datasets

User behavior logs from xuetangX.com between 1 September 2016 and 31 January 2017 are adopted, which contains 162,650 independent users’ behavior actions in 1115 courses, including 7,530,648 videos seek records, 348,942 forum visits/post logs, and 36,604,330 navigation records. Activate sessions of length less than three are filtered out by preprocessing, and 153,426 users are left in the dataset. The dataset is split into a training set of 122,740 users (80%) and a test set of 30,686 users (20%), and the data are shuffled by course group. The training dataset is used for training the neural network, while the test dataset is used to evaluate the trained network. A lecture based activity analyzing model proposed by Graf was used to labeling the learning styles in terms of the dimensions of FSLSM [25], as indicated in Section 3.3, for the users in the training dataset. The distribution of learning styles of the learners in training dataset is visualized as Figure 6, indicates that most users are intended to be reflective style in processing dimension, which means they are preferred impersonal and fewer activities in community, same in Chinese university classrooms. In perceive dimensions the majority users are sensing styles, almost two times than intuitive styles, most users preferred existing ways. Visual content is primary receiving style by users. Sequential style got 20% more than global style users, not as much as expected for Chinese users, this could be come from the actions taken by many learners jump in online learning environment, which increases the number of global style users.

4.2. Training and Test

The model is implemented and run with Google Tensorflow 1.2.1 trained with the above training dataset, the learning rate is 0.0001 with the Adam optimization algorithm [38] and batch size for 10. In the first part of the intelligent analysis process, the Observer tries to identify learning style with the given behavior snapshot, acting as to solve a classification problem. In this case, identical data examples in our dataset can have a different dimension of learning styles. Then in the second part, the Inference Engine ranks possible Learning Style depends on the previous Learning styles in a learning session provided by Observer, learning to rank approaches generally better than other approaches [39]. Ranking can be pointwise, pairwise, or list wise. Pairwise ranking outperforms the other two, it compares the score or the rank of pairs of a positive and a negative item and the loss enforces that the rank of the positive item should be lower than that of the negative one.

Several architectures are also examined, and the single layer of GRU units is found to be the best performer while adding additional layers might result in the performance loss in training.

The effectiveness of the model is evaluated by processing the test dataset with trained Observer and Inference Engine. The best test result reaches an average error rate of 9.235% in four dimensions, which means that the model can properly detect the learning styles most of the time and the accuracy can meet the requirement of the system. Through interaction between users and LMS, the present and previous behavior patterns of users are dynamically obtained to predict their next learning styles. Evidence shows that 2869 times of the event for learning style change are detected on the testing dataset in the intelligent analysis process, with 1703 times (59.33%) in science courses, as listed in

Table 3. Statistics of the events for learning style change.

Event	Times	Percent
from Reflective to Active	1032	35.97%
from Active to Reflective	373	13.00%
from Intuitive to Sensing	673	23.46%
from Sensing to Intuitive	423	14.74%
from Verbal to Visual	86	3.00%
from Visual to Verbal	121	4.22%
from Global to Sequential	32	1.12%
from Sequential to Global	129	4.50%

.

4.3. Discussion

By contrast to high cost of determining student learning style in face to face classroom, the suggested model can detect learning style dynamically in an acceptable cost by taking the advantage of LMSs and intelligent analysis method, makes that is possible make interventions by individual learning style. The result reveals that learning style may change in the learning process, especially when learning tougher or more quantitative contents such as science courses. Changes in Visual/Verbal dimension are relatively less than in other dimensions. The possible reason is that most online contents are in video or graphic format and fewer verbal contents provided. It is noteworthy that quite some users change from Reflective to Active. It is found by further investigations on the logs that the change is usually caused by forum related behaviors, which implies that it is possible to enhance engagement of this kind of students by sending them messages about the latest discussions on course forums.

5. Results from Empirical Research on Sustainability Education

The proposed model for dynamic learning style identification has been applied on xuetangX.com, and the effectiveness is evaluated in this section based on the empirical research on the online study of sustainability-related courses.

Some sustainability-related MOOCs on xuetangX.com are listed in

Table 4. Sustainability-related MOOCs on xuetangX.

Course	University	Enrollments
Ecology in the humanistic perspective	Jilin University	12,000 1
Green buildings and Sustainable development	Tsinghua University	10,000 1
China’s perspective on climate change	Tsinghua University	6313
Air Pollution Control Engineering	Tsinghua University	5755
Hydraulic Structures	Tsinghua University	1993
Environmental pollution incidents and emergency response	Beijing Normal University	753
Corporate Social Responsibility Introduction	Tsinghua University	348
Cyclic economy and sustainable development enterprises	Beijing Normal University	215
Ecology and Environmental Sciences	HuBei University	143

¹ Numbers more than 10,000 are rounded to the nearest thousand. Enrollments do not include SPOC (Small Private Online Course) platform.

, offered by Tsinghua University, Jilin University, Beijing Normal University, and Hubei University, respectively. Some courses have been operated since 2015, and some are just online in 2017. The total enrollment of the courses is 37,520, with an average of 4168 for each. Two courses of running more than two rounds and enrollments more than 6000 are selected as the target courses to conduct the empirical research. The primary goal of the research is to examine whether the learning performance can be improved when the learning styles of the students are identified by the proposed model, and thus the corresponding pedagogical models suitable for the specific learners are provided.

A simple pedagogical model was designed, in which proper actions would be taken while the event for learning style change was detected, as listed in in

Table 5. Action list of learning style change event detected.

Event	Action 1	Trigger
Reflective to Active	Push latest threads from forums	>2
Active to Reflective	Send message with quiz at last stops	=1
Intuitive to Sensing	Send message with suggested reading materials	=1
Sensing to Intuitive	Send message with extra relative reading materials	=1
Verbal to Visual	Ignore	-
Visual to Verbal	Ignore	-
Global to Sequential	Send message about reminder information for next chapter	>2
Sequential to Global	Send message for outline of course	>2

¹ Actions triggered by event or learning style does not change.

. The investigation was limited to providing only different course contents at this time, therefore content-related events “Verbal to Visual” and “Visual to Verbal” were ignored.

The two selected MOOCs were “Green buildings and Sustainable development” and “China's perspective on climate change”. The traditional LMS was enhanced with the functions of learning style identification and recommendation message feedback (actions in Table 5). The message was sent by email, notifications, and hints on courseware. Learning behaviors were collected daily, and processed by the proposed intelligent analysis model on 2:00 AM every day. The messages driven by events were sent on 10:00 AM. During the study period of the courses, 918 messages were sent among which 210 were read by learners; the average read rate is 22.8% with details shown in

Table 6. Statistics for actions against learning style change.

Action	Times	Message Read Rate 1
Push latest threads from forums	103	25.24%
Send message with quiz at last stops	425	21.88%
Send message with suggested reading materials	135	17.77%
Send message with extra relative reading materials	83	13.25%
Send message about reminder information for next chapter	140	36.42%
Send message for outline of course	32	15.6%

¹ Message read rate is the rate of learners receiving the message and opening the link.

.

The enhanced LMS with the intelligent analysis model was implemented in early 2017. A model with five categories of direct effect and moderations: information quality, system quality, use, user satisfaction and net benefits were used to evaluate the success of LMSs from a student’s point of view [40]. Five indexes in two categories: Net Benefits and Use can be benchmarked to evaluate the learning performance and engagement of the users, and the results of two rounds of course learning are listed in

Table 7. Benchmark index of user interaction and course performance.

Index Categories	Index	Round-2016#2	Round-2017#1	Improvement (%)
Net Benefits	Completion Rate	6.56%	8.67%	2.11% (32.16%)
Net Benefits	Average quiz score	71	79	8 (11.2%)
Use	Discussion threads	136	155	19 (13.97%)
Use	Discussion Replies	306	354	48 (15.68%)
Use	Discussion Views	1124	1349	225 (20.02%)

, where the learning process for Round-2016#2 was completed with traditional LMS (Interventions triggered by Teaching Assistant manually) and that for Round-2017#1 was with enhanced LMS (Interventions triggered by learning style detecting model to specific learners).

From the results of above experiments on the learning of sustainability-related subjects with MOOCs are easy to find evidence about improvement on the learning performance due to the introduction of the intelligence analysis mechanism. The users can significantly benefit from the individualized pedagogical model in accord with their learning styles, especially for the study of complex and cross-disciplinary subjects. Another valuable finding is the remarkable improvement on the interactions among the users in the forums, this could positively help users learn from each other rather than only from courses or instructors which is very important for sustainability education. Of course, the more general conclusion still depend on the further investigation with more rigorous control on the experiments.

6. Summary

The learning style identification is an essential problem for individualized teaching or learning which can significantly improve the learning performance and participation, especially in an online or hybrid education environment. In this paper, a kind of intelligent analysis mechanism was proposed based on the hybrid neural network model to dynamically detect and predict the learning styles of learners through their behavior in an interactive learning system. A convolutional neural network was designed to observe and identify the state of user behavior, and then connected with a modified GRU-based recurrent neural network to predict the learning style of next state. The testing process showed that the trained model in this paper can identify the learning styles of users efficiently and effectively, and could be further extended for the intelligent analysis of an AI enforced adaptive learning management system. This is valuable for the next considerations. The future work could also include the exploration of new methods to further increase the accuracy of learning style detection and prediction, as well as more sophisticated pedagogical methods suitable for online learning.

Massive online learning is one of the most important ways to make full use of the limited global education resources and increase the equity of education, which are the primary concerns of education sustainability. With great potential to offer stronger support and better service to learners in order to increase their study performance in online learning environment by adopting the advanced technologies (such as the intelligent analysis mechanism proposed in this paper) can manage the learning process effectively. We are planning to enhance the adaptive learning system of xuetangX.com by providing more diverse contents and sophisticated learning paths for users according to much accurate identification of their profiles and learning styles, which will better serve over 9,000,000 learners across the world. We still have a long way to go.