Next Article in Journal
Organic and Conventional Herbs Quality Reflected by Their Antioxidant Compounds Concentration
Next Article in Special Issue
Smart Learning
Previous Article in Journal
Evaluation of the Effectiveness of N-Acetylcysteine (NAC) and N-acetylcysteine-cyclodextrins Multi-Composite in Pseudomonas aeruginosa Biofilm Formation
Previous Article in Special Issue
Representing Data Visualization Goals and Tasks through Meta-Modeling to Tailor Information Dashboards
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Monitoring Students at the University: Design and Application of a Moodle Plugin

by
María Consuelo Sáiz-Manzanares
1,*,
Raúl Marticorena-Sánchez
2 and
César Ignacio García-Osorio
2,*
1
Departamento de Ciencias de la Salud, Facultad de Ciencias de la Salud, Universidad de Burgos, Research Group DATAHES, Pº Comendadores s/n, 09001 Burgos, Spain
2
Departamento de Ingeniería Informática, Escuela Politécnica Superior, Universidad de Burgos, Research Group ADMIRABLE, Escuela Politécnica Superior, Avd. de Cantabria s/n, 09006 Burgos, Spain
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2020, 10(10), 3469; https://doi.org/10.3390/app10103469
Submission received: 25 April 2020 / Revised: 11 May 2020 / Accepted: 12 May 2020 / Published: 18 May 2020
(This article belongs to the Special Issue Smart Learning)

Abstract

:
Early detection of at-risk students is essential, especially in the university environment. Moreover, personalized learning has been shown to increase motivation and lower student dropout rates. At present, the average dropout rates among students following courses leading to the award of Spanish university degrees are around 18% and 42.8% for presential teaching and online courses, respectively. The objectives of this study are: (1) to design and to implement a Modular Object-Oriented Dynamic Learning Environment (Moodle) plugin, “eOrientation”, for the early detection of at-risk students; (2) to test the effectiveness of the “eOrientation” plugin on university students. We worked with 279 third-year students following health sciences degrees. A process for extracting information records was also implemented. In addition, a learning analytics module was developed, through which both supervised and unsupervised Machine Learning techniques can be applied. All these measures facilitated the personalized monitoring of the students and the easier detection of students at academic risk. The use of this tool could be of great importance to teachers and university governing teams, as it can assist the early detection of students at academic risk. Future studies will be aimed at testing the plugin using the Moodle environment on degree courses at other universities.

1. Introduction

The 2030 Agenda for Sustainable Development (The 2030 Agenda for Sustainable Development and the SDGs https://ec.europa.eu/environment/sustainable-development/SDGs/index_en.htm) lists a number of social challenges to which universities now have to respond. In a changing world, one challenge is the creation of inclusive, innovative, and reflective societies where Information and Communication Technology for Inclusion (ICTI) and New Information and Communication Technologies (NICT) will find a place, especially in university teaching. The effectiveness of these resources used in the teaching–learning process has recently been demonstrated in some studies [1,2,3]. However, other studies have shown that approximately 48% of teachers are not using those same resources efficiently [4,5]. This aspect is a challenge for research into technological innovation in university teaching. The ultimate objective is to improve the effectiveness of student learning—effectiveness that is based on the acquisition of skills related to the practical implementation of conceptual and procedural contents learned within the university environment, so that they can be efficiently applied following graduation. Another challenge is course abandonment among students and approaches to lowering dropout rates. Specifically, in Spanish universities drop-out rates from degree courses are approximately 18.72%, while in the rest of Europe dropout rates are between 8%—in Portugal, the Netherlands, Belgium, and Austria—and around 21% in the United Kingdom [5]. These figures equate to an economic expense of more than 1500 million Euros per year, or approximately 6000 Euros per student. The report on Spanish universities [4] prepared by Crue Universidades Españolas (Crue Universidades Españolas, a non-profit association formed of a total of 76 Spanish universities (50 public and 26 private), is the main interlocutor on behalf of the universities with the central government and plays a key role in all regulatory developments that affect Higher Education in Spain. http://www.crue.org/.) stressed that efforts should be made to reduce drop-out rates by increasing performance and academic success rates. Furthermore, it was also concluded in this report [4] that academic performance can be directly related to the efficiency of teaching performance.
Along these lines, the application of NICTs to teaching in blended learning environments is a challenge for university managers, although their application could in all probability increase the quality of educational systems [5]. Quality standards agencies (e.g., ENQUA (European Association for Quality Assurance in Higher Education) [6], ANECA (Agencia Nacional de Evaluación de la Calidad y Acreditación) [7], and ACSUCYL (Agencia para la Calidad del Sistema Universitario de Castilla y León) [8]) have proposed the implementation of different evaluation protocols that, among others, consider the quality of teaching methodologies, evaluation procedures, and teaching strategies within these environments.

1.1. Characteristics of Teaching in Blended Learning Environments

Currently, a high percentage of teaching, whether classroom-based or online, is done in blended learning mode using Learning Management Systems (LMSs). LMSs are modular and collaborative digital environments such as Moodle (Modular Object-Oriented Dynamic Learning Environment) [9]. The effectiveness of these environments at facilitating the development of metacognitive skills [10,11] among students and, therefore, learning acquisition, has been demonstrated through both conceptual and procedural content construction [12]. These spaces provide learners with the possibility of developing a mental scaffolding for task resolution which promotes both effective and practical learning [13]. In addition, if used in the LMSs, hypermedia resources will increase Self-Regulated Learning (SRL) because these resources promote planning, supervision, control, and personal reflection upon the learner’s own practice [14]. All these approaches encourage the participation of learners in their own learning processes—hence, the use of NICT for the adaptation of teaching to the specific characteristics and needs of each student. In addition, direct feedback in real time can be provided from the LMSs on the resolution of a task (process and product) [15]. These learning environments also enhance SRL [16]. Thus, all these aspects together will bring added motivation [17,18,19]. However, to obtain those benefits a good pedagogical design for curricular planning will be in need of the following [16,20]:
  • Monitoring actions through the development of dynamic activities that include hypermedia resources. These actions facilitate monitoring progress through the systematic checking of goals. This is a process that contains assessment, the proper identification of information, the evaluation of the content of responses to goals, and self-questioning. Machine Learning techniques are used to perform the analyses.
  • Discrimination between effective vs. ineffective learning strategies. Procedures for detecting task difficulties are analysed and tasks at varying degrees of difficulty are presented to learners.
  • The development of help-seeking behaviour through data that can be used to analyse time and effort in planning, the degree of task difficulty, and interest shown towards the scope of the task.
In addition, recent research in e-learning [21] supports the hypothesis that these environments are more effective at achieving deep and high-quality learning if they include hypermedia resources [12], which can be adapted to the learning pace of each student and increase the use of metacognitive skills. Additionally, these resources need to be supported by the use of educational Data Mining techniques [22].
The project-based learning (PBL) approach is a methodological tool (it refers to a constructive and collaborative environment in which it is believed students acquire a deeper knowledge through problem solving or practical assumptions related to each discipline when it is applied) with which a good design for this sort of work can be implemented [23,24,25]. In this methodology, a systematic method of teaching is applied that can help students build deep learning [25]. The method is based on the use of research questions that help students build their learning and increase their learning outcomes. In addition, this methodology uses process-oriented feedback [26] in a collaborative learning framework, a type of feedback that promotes effective learning outcomes. The reason for its effectiveness is that it helps students to use SRL [27], which increases their motivation towards the learning object [19]. Moreover, if applied through the LMSs, the effects of this methodology are enhanced due to student-student, student-teacher, and students-teacher interactions within these learning environments [28]. The analysis of the records or logs will allow a detailed and individualized follow up of learning processes and the personal development of each student within a short time [20,23,24], aspects which are essential to achieve personalized learning [29].
Along these lines, programmed tutoring has in recent research [30] been shown to increase student motivation towards the subject matter. The benefits of this type of program are focused on the personalization of learning as a tool for the prevention of academic dropout [31]. As previously mentioned, an LMS registers student and teacher interactions and behaviour on the platform. In addition, the application of automatic learning techniques facilitates the detection of both successful and at-risk behavioural patterns. These patterns explain up to 52% of the variance in learning outcomes. The successful behaviours detected on the platform were [16,32,33]:
  • The overall effort made to follow the subject.
  • The time the student spends on practical tasks.
  • The time spent by the student on tasks of a theoretical nature.
  • The effort invested in completing the questionnaires.
  • The time spent on forum discussions.
    (1)
    The quality of the discussions in the forums (type and length of messages).
    (2)
    The time spent analysing the feedback from the teacher.
    (3)
    The number of messages read in the forums.
  • The contribution to the creation of content.
  • How soon or how late the activities are completed and delivered.
  • The number of “to visit” links to reference information.
Closely related to these results are those from other studies that point to both improved learning outcomes and higher levels of interaction on the platform through the personalized monitoring of students [33]. Likewise, information collected from the monitoring processes of one study predicted student learning outcomes by 61.3% and their patterns of behaviour on the platform by 56.1% [2]. In another, the use of hypermedia smart tutoring systems explained 60.4% of the variance in learning outcomes [34]. Similarly, in various other studies the use of PBL methodology predicted learning outcomes on quizzes by 42.3%, and the use of effective learning patterns by 74.2% (these data refer to the multiple correlation coefficient or determination coefficient (R2), which represents the proportion of the variability explained by the independent variable with respect to the dependent ones), in addition to increasing student satisfaction with the teaching–learning process [3,30,31,32,33,34].

1.2. Characteristics of the Learning Management System in the Detection of the At-Risk Student

As with Moodle, LMS can incorporate different types of modules either to show simple resources, such as files, folders, books, pages, tags, etc., or more advanced modules, such as databases, chats, questionnaires, surveys, forums, glossaries, active areas, lessons, SCORM (Sharable Content Object Reference Model), packages, wikis, and workshops, etc. Various student interactions are collected from them all: the time at which the module is accessed, the viewing time, the time at which the access ends, and the modules that have and that have not been seen [35]. All this information can be downloaded in log files, which generate a Time Series Cross Section (TSCS). LMSs also include learning analytics systems, although most of these systems calculate only very simple metrics, which is why different investigations [35,36,37,38] have pointed out the need to develop plugins, designed to increase the power of measurement, interpretation, and visualization in order to monitor student progress on the platform. For example, Dobashi [35] used Excel for the analysis, visualization, and interpretation of the registered information. However, he concluded that Excel is not an ideal tool, as the number of rows and columns are a limitation for analysing student data. In turn, the studies by Luna et al. [36] proposed the use of a Moodle plugin compatible with specific Educational Data Mining (EDM) techniques: supervised and unsupervised Machine Learning techniques [38,39,40,41,42]. These authors concluded that LMSs must integrate tools and plugins; not only simple learning analytics, but also EDM tools that permit the use of supervised learning techniques (prediction) [36,37,38] and unsupervised learning techniques (classification) that provide a user-friendly interface for the teacher, which facilitates their use throughout the course of study. These plugins will help with the detection of at-risk students [41] and the implementation of actions to avoid academic failure and high dropout rates [42]. In line with these conclusions, the studies by Moreira Félix et al. [37] and Dimić et al. [42] placed special emphasis on the advantages of including a prediction module within the plugins in order to facilitate the academic monitoring of students. They also indicated that these plugins must be accompanied by visualization tools. The authors proposed their development through a Java SQL database server for processing the data with other interconnected tools, such as WEKA (Waikato Environment for Knowledge Analysis) [43], RapidMiner [44], Orange [45], and Knime [46], among others.
Storage capacity problems can affect such plugins due to high data volumes that can influence their productivity levels. These problems can be solved using compression methods for higher storage capacity, thereby reducing the data-exchange time. In a study by Reynaldo et al. [47], the authors proposed to use a Hypertext Preprocessor programming language (PHP) using an algorithm implemented in a Moodle-based e-learning server.
In summary, all these studies agreed that the extraction and the treatment of logs within LMSs are laborious and complex processes for the teacher. For this reason, LMSs must include plugins that offer teachers different functionalities for processing, interpreting, and visualizing the results. Within these plugins, the use of supervised and unsupervised learning techniques will be essential for monitoring students [48]. The plugins will facilitate an analysis of student learning patterns and their past grades, which will allow the teacher to offer a personalized educational response tailored to personal characteristics and needs throughout the learning process of each student on the degree course [48,49].
Based on the studies presented above, the objectives of this work will be:
(1)
To design a plugin for Moodle, known as “eOrientation”, for the early detection of at-risk students.
(2)
To test the effectiveness of the “eOrientation” plugin on university students.

2. Materials and Methods

2.1. Participants

The sample group consisted of 279 students—137 from a nursing degree (Group 1) and 142 from an occupational therapy degree (Group 2). In Table 1, the distribution of the sample can be seen in relation to the independent variables gender and age (see Table 1).

2.2. Instruments

(a)
UBUVirtual Platform. This platform is an LMS developed in the Moodle environment, version 3.7.
(b)
“eOrientation” Moodle Plugin. The “eOrientation” plugin was developed within an ongoing research project funded by the Junta de Castilla y León (Spain). Customized access can be set up with this plugin by the course (subject) modules that are available on each course. It provides a graph for each group, showing the total number of accesses for each user within a specific period previously selected by the teacher. The graph incorporates the average number of accesses to each platform module. Likewise, personalized notifications related to learning process monitoring can be sent to a student or a group of students using the plugin through an email sent to a platform-messaging system. In addition, a table with all or part of the information registered can be exported in different formats (.csv, .xlsx, HTML table, .json, .ods, .pdf). More detailed information on the “eOrientation” plugin is presented in the development of objective 1 (see point 6: Patents) in the results section.
(c)
Design of the subjects. The subjects “Quality Management Methodology” and “Early Stimulation” kept to the same design and applied the same PBL teaching methodology. The subjects contained five thematic units with the following structure: presentation of the unit, complementary information, and satisfaction survey in each unit. Additionally, in both subjects quizzes were used on the platform, providing automatic feedback to the answers given by the student. Likewise, flipped classroom experiences and glossaries were made available to students. The following types of accesses (to different resources) could be set up, depending on the design of activities and resources performed by the teacher on the platform. In this case, the types of accesses were: “feedback”, “quizzes”, “theoretical information”, “practice”, “complementary information”, and “information on PBL”.

2.3. Procedure

Authorization from the University of Burgos Bioethics Committee was obtained prior to the start of the investigation. Convenience sampling was applied to select the sample. No economic compensation was offered to any of the participants. Furthermore, the students were informed of the objectives of the research and they all gave their informed consent in writing [50]. Subsequently, a research project was drawn up within the research group of the University of Burgos DATAHES (Data Analysis Techniques Applied in Health Environments Sciences Research Group), which was submitted to the call for support for Recognised Research Groups of the Junta de Castilla y León and selected in the resolution of 5th July 2019.
Next, the technology designed in the project was developed for the implementation of the Moodle plugin so that it would include the functionalities indicated in Section 2.2 point b. Lastly, the tool was applied in a pilot study to a sample of 279 students on health sciences degrees, as described in Section 2.1, enrolled on two different third-year courses and taught by the same teacher with a similar structure on the platform.
In summary, the data collected in the logs and selected and exported through the plugin were processed in the statistical analysis software SPSS v.24 [51], where prediction and classification techniques were applied, and in the data visualization software Orange v.23.1 [45].

2.4. Statistical Analysis

A normality analysis of the distributions was performed using asymmetry and kurtosis statistics. The data distributions were also tested with Analysis of Variance (-ANOVA-) fixed-effect factor (type of degree) parametric statistical tests. In addition, both supervised (Multiple Regression Analysis (MRA)), Decision Tree (DT), and unsupervised (k-means clustering) Machine Learning techniques were used.

2.5. Previous Statistical Analyses

Before starting the study, the sample was tested to confirm that its distribution was within the parameters of normality. To do so, both the skewness and the kurtosis values of the selected indicators were established. The highest skewness value, |2.00|, indicates extreme skewness and the lowest value indicates that the sample follows a normal distribution. Kurtosis values between |8| and|20| suggest extreme kurtosis [52]. Acceptable values of asymmetry and kurtosis were found in the “access type” categories except for “supplemental material” and “information on PBL” which was eliminated from the study in order to apply parametric statistics to the analysis of the other categories (“feedback”, “quizzes”, “theoretical information” and “practices”) (see Table 2).

3. Results

3.1. Contrasting Objectives

3.1.1. Objective 1

Regarding the first objective (“To design a plugin for Moodle, known as ‘eOrientation’, for the early detection of at-risk students”), a plugin for Moodle, “eOrientation”, was developed using the PHP [53] programming language. The installation of the plugin on the platform is easily managed with administrator rights. Having logged onto the Moodle platform as an administrator and navigated from “Site Administration” to “Plugins” and then to “Install Plugins”, a zip file with the plugin installation files can then be unpacked to start the installation (see Figure 1).
Once the plugin is installed, the teacher can use it with ease. Within a subject, clicking on the cogwheel icon in the top right-hand corner (the “settings cog”) and selecting the option “more” (the last entry on the settings menu) and then selecting the “Report” tabs will provide access to the plugin (see Figure 2).
Once inside the plugin, the different elements that each teacher has included in the design of the subject on the platform may be accessed. Within these categorization criteria, the teacher may include all those different course modules. An example of the options that may be inserted in a category in the “eOrientation” plugin can be seen in Figure 3.
In this study, the design of the subjects included such activities as “feedback”, “quizzes”, “theoretical information”, “practices”, “supplementary material”, and “information on PBL”. The teacher can use the plugin to view, edit, and delete the classification of the elements on the platform (see Figure 4).
Clicking on the icon representing an eye, the teacher can access a table of frequencies. This table records student accesses to each of the activities. For each activity, the number of standard deviations to be applied to the average number of accesses are recorded so that the teacher can compare the record on student accesses. In addition, the assigned frequency will be displayed in red (see Figure 5) for those students whose accesses are under the pre-set standard deviation. All the data can be exported in .csv, .xlsx, HTML table, .json, .ods, and .pdf formats.
The teacher will also be able to obtain a graphic analysis of student frequency of access to each activity. This table includes the mean levels of student participation (see Figure 6).
Finally, the teacher can identify the at-risk students and send them a message, through the platform messaging system, either to arrange a tutorial or to provide guidance on improving the learning process (see Figure 7).

3.1.2. Objective 2

Regarding objective 2 (“To test the effectiveness of the ‘eOrientation’ plugin on university students”), the “eOrientation” Moodle plugin was available to 279 health science students following two degree courses: Nursing and Occupational Therapy. The logs were extracted from the type of accesses indicated in the section on instruments for the design of subjects (2.2) (“feedback”, “quizzes”, “theoretical information”, “practices”, “supplementary material”, and “information on PBL”) in order to check the effectiveness of the tool. Automatic learning techniques for prediction (MRA and DT) and clustering (k-means) were used for the analysis of these data, all of which were computed with the statistical package SPSS v.24.
Firstly, we studied whether the “degree type” variable predicted student learning behaviours when accessing the following aspects: “feedback”, “quizzes”, “theoretical information”, “practices”, “supplementary material”, and “information on PBL”. The “degree type” variable was found to predict 59.2%, R2 = 0.592, of student behaviour patterns on the platform for these activities. Specifically, the partial correlations showed significant differences for accesses to “feedback” (t = 4.11, p = 0.000), quizzes (t = −6.51, p = 0.000), and “practices” (t = 6.35, p = 0.000). The Tolerance (T) values were found in an interval between 0.753 and 0.942 (the accepted T value was less than 1, so these data were within the expected patterns). For the Variance Inflation Factor (VIF) values, the values found were in the range of 1.06 to 1.26 (the accepted value of VIF was within the range of 1 to 10). It was therefore concluded that the distribution lay within the parameters of normality and that parametric statistics could be applied.
Regarding the prediction of the variables, it was found that the variable with the greatest predictive value among the two groups of degree students was the “frequency of access to the quizzes”. Specifically, three groups could be identified, the third of which had a higher percentage of nursing students (85.7%) compared to occupational therapy students (14.3%), and that ratio was reversed in the first group (30.7%/69.3%), as shown in Figure 8.
The clustering technique was also applied, and three clusters were found for the distribution of “frequency of student access” to resources on the platform (see Table 3). Likewise, significant differences between clusters were found under “feedback”.
To sum up, a contingency table was calculated to check the student distributions within the two degrees within the three clusters (see Table 4). It was found that there was a higher percentage of Occupational Therapy degree students within Cluster 2 (60.56%) and a higher percentage of Nursing degree students within Cluster 1 and Cluster 3 (49.63% and 24.09%, respectively).
Finally, Orange software-based data visualization tools were used. Specifically, aspects of prediction and classification were displayed. In Figure 9, we can see the different frequency of student access (Nursing degree vs. Occupational Therapy degree) in the variables, in which significant differences were found for the three clusters. In the blue coloured spots, which represent nursing student groups, the results show greater evenness for all types of access, except for access to quizzes. The yellow coloured spots that represent the groups of occupational therapy students reflect a greater dispersion of access to “feedback”, “theoretical information”, and “practices”.
Similarly, in Figure 10, the distribution of students in the three clusters can be seen.
As an example, the visualization of students within three clusters is presented with respect to the “degree type” variable (see Figure 11) and “quizzes” variable (see Figure 12).
Both a heat map of frequency of student access for the variables for which significant predictions were detected and a hierarchical cluster analysis can be seen in Figure 13. “Accesses to quizzes” shows the highest differences.
It is important to point out that in both groups of students (nursing and occupational therapy), the use of PBL methodology, “feedback” in the LMS, and “quizzes” supported by technological resources produced good results, as both the performance rates (the percentage ratio between the number of credits passed and the total number of students enrolled on a given academic year with respect to the number of credits passed by those students in that same year) and the success rates (the ratio between the number of credits passed over the total number of students enrolled on the degree). Both the (performance and success) rates were 100% for the subjects “Quality Management Methodology” on the Nursing degree. These same rates on the “Early Stimulation” subject module of the Occupational Therapy degree were, respectively, 95.6% and 98.2%. No abandonment was recorded on either course.

4. Discussion

The work to achieve an inclusive society in the framework of the 2030 Horizon is directly related to the use of ICTI and NICTs. However, their efficient usage is partially related to the versatility and usability of the tools that are used. Reports on universities within Spain have indicated that 48% of the teaching staff have difficulty managing these resources [5]. The knowledge of society in the 21st century is based on a digital society in which learning is increasingly taking place in blended learning environments, in both face-to-face and online teaching environments [1]. The ultimate goal is to achieve more effective learning than before through more efficient use of personal and material resources [13,14,15]. These achievements are directly related to the use of LMS, hypermedia resources, and automatic and immediate feedback almost immediately after the completion of various activities [14,18]. An essential component for the effective use of these resources is the design of the LMS, as it is directly related to the promotion of student participation and interaction with subject-related materials and resources [3]. A well-designed LMS will increase the motivation towards the learning object [17,18,19]. The acquisition of deep learning is pursued [12], an objective that is directly related to the application of metacognitive skills, the use of process-oriented feedback [26], and SRL [25,27]. The ultimate achievement of this methodology, both pedagogical and technological, is personalized learning [29,30,32]. Likewise, these achievements, which are designed to motivate the personalization of learning, are related to the inclusion of hypermedia resources in blended learning environments [29]. However, this study has shown that the learning patterns of students on the platform are not similar, even though, in principle, both the pedagogical and technological aspects of the learning methodology might appear to be so. Specifically, groups that apparently share a very similar series of variables (age, gender, branch of knowledge, etc.) have shown different learning responses. One possible cause might have been that the resources were neither implemented in the same way nor generated the same educational response in different groups [32]. In this study, the “degree type” variable predicted 35.2% of the variance with respect to the behavioural patterns. In addition, the variable frequency of “access to the quizzes” was postulated as an essential element in the differentiation between behaviour patterns in the LMS among the students on the two degree courses. This aspect has also been observed in other studies [2,29]. In fact, if a contingency table is drawn up between the grouping found with the clustering (k-means) technique and the group of origin (degree), the coincidence is high. This seems to indicate that there are variables specific to each group of students that are particular and that contribute to an identity of interaction and learning. However, this hypothesis will have to be contrasted in subsequent studies, by analysing other characteristics of the groups that may help explain the difference in learning patterns.
The design and implementation of a PHP plugin within Moodle has resulted in a responsive and rapidly operated tool. The plugin has made it easier to give a personalized educational response to each student [47]. It has also facilitated the detection of students at academic risk and the implementation of specific resources for each one [35,36,37,38]. Despite the differences found between the two groups of students (nursing and occupational therapy), the academic results were highly acceptable in both groups, as their success and performance rates showed, ranging between 95% and 100%. Similarly, the data analyses with a learning analytics system that includes comparative averaged values and deviation from the average offer additional advantages, as well as the option of storing the results in databases that can be exported in different file formats (.csv, .xlsx, HTML table, .json, .ods, .pdf). This functionality facilitates the use of different statistical packages or libraries in which more complex data-analysis techniques can be applied, such as supervised and unsupervised learning techniques [36,47,48,49]. The use of these techniques establishes predictive and grouping factors that will facilitate the development of specific and personalized learning for each student at each moment of their learning process.

5. Conclusions

LMSs tools can be used to analyse the records of student and teacher interactions on the Moodle platform throughout the teaching–learning process; this facilitates the follow-up of students and the detection of students at risk of either failure or academic abandonment. However, the inclusion of these systems is not widespread and therefore represents a challenge for university management. The visualization of the monitoring through easy-to-use tools for the teacher is a commitment in the framework of a modern university for the 21st century that aims to respond to the challenges of the 2030 Horizon. Another essential aspect is the use of EDM techniques to detect and visualize student behaviour patterns on the platform and to present visual analyses. Such patterns are indicators of different ways of learning and reveal very valuable information on which resources are the most effective in relation to each learning pattern. Subject-customized designs may therefore be offered for each learning pattern that may be detected in the not-too-distant future, which is a possible development related to precision teaching. As already indicated in the introduction, the effectiveness of the use of LMS and blended learning techniques is directly related to the effective use and management of technological resources among teachers. The continuance of research in this field is therefore important, as is the design of tools and their implementation for data analysis and visualization that provide complete, simple, and clear information to teachers. In addition, training programs may be given to teachers on functional tools that facilitate the smooth use of data analysis and visualization resources.
In summary, although the results of this study are promising, they should be treated with caution, as the pilot study was performed with a specific sample and only some courses were supported through Moodle. Therefore, future studies are needed with larger samples to expand the type of degrees that are studied and include other Moodle-based courses that will facilitate the validation of plugin functionalities for the identification of new areas for improvement.

6. Patents

The Moodle plugin, “eOrientation”, was developed with resources from the Recognized Research Group of the University of Burgos DATAHES, Grant number BU032G19, under software registration BU-09-20 [53].

Author Contributions

Conceptualization, M.C.S.-M. and R.M.-S.; methodology, M.C.S.-M.; validation, M.C.S.-M.; formal analysis, M.C.S.-M and C.I.G.-O.; investigation, M.C.S.-M.; data curation, M.C.S.-M.; writing—original draft preparation, M.C.S.-M.; writing—review and editing, M.C-S.-M. and R.M.-S.; visualization, M.C.S.-M. and C.I.G.-O.; supervision, C.I.G.-O.; project administration, M.C.S.-M.; funding acquisition, M.C.S.-M.; Software, M.C.S.-M. and R.M.-S. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Consejería de Educación de la Junta de Castilla y León (Spain) (Department of Education of the Junta de Castilla y León), grant number BU032G19, and grants from the University of Burgos for the dissemination and the improvement of teaching innovation experiences of the Vice-Rectorate of Teaching and Research Staff, the Vice-Rectorate for Research and Knowledge Transfer, 2020, and the Departamento de Ciencias de la Salud the University of Burgos (Spain).

Acknowledgments

Our thanks to all the students who participated in this study and to the Committee of Bioethics of the University of Burgos (Spain).

Conflicts of Interest

The author declares no conflicts of interest.

Ethics Statement

The Ethics Committee of the University of Burgos approved this study, Nº IR 30/2019. Written informed consent was in each case requested from the students who participated in this research. They all gave their written informed consent in accordance with the Declaration of Helsinki.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

References

  1. Cerezo, R.; Sánchez-Santillán, M.; Paule-Ruiz, M.P.; Núñez, J.C. Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Comput. Educ. 2016, 96, 42–54. [Google Scholar] [CrossRef]
  2. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.M.; García Osorio, C.I.; Díez-Pastor, J.F. How do B-learning and learning patterns influence learning outcomes? Front. Psychol. 2017, 8, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Díez-Pastor, J.F.; García-Osorio, C.I. Does the use of learning management systems with hypermedia mean improved student learning outcomes? Front. Psychol. 2019, 10, 1–14. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. CRUE Universidades Españolas. La Universidad Española en Cifras 2017–2018. [Spanish University in Figures]; CRUE: Madrid, España, 2019; Available online: http://www.crue.org/Documentos%20compartidos/Publicaciones/Universidad%20Espa%C3%B1ola%20en%20cifras/UEC%201718_FINAL_DIGITAL.pdf (accessed on 11 May 2020).
  5. Ministerio de Ciencia Innovación y Universidades. Datos y Cifras del Sistema Universitario Español 2018–2019 [Facts and Figures of the Spanish University System 2018–2019]; Secretaría General Técnica del Ministerio de Ciencia, Innovación y Universidades: Madrid, España, 2019; Available online: http://www.educacionyfp.gob.es/dam/jcr:2af709c9-9532-414e-9bad-c390d32998d4/datos-y-cifras-sue-2018-19.pdf (accessed on 11 May 2020).
  6. European Association for Quality Assurance in Higher Education (ENQA). Available online: https://enqa.eu/ (accessed on 11 May 2020).
  7. Agencia Nacional de Evaluación de la Calidad y Acreditación (ANECA). Available online: http://www.aneca.es/ (accessed on 11 May 2020).
  8. Agencia para la Calidad del Sistema Universitario de Castilla y León (ACSUCYL). Available online: http://www.acsucyl.es/web/es/agencia-para-calidad-sistema.html (accessed on 11 May 2020).
  9. Saiz-Manzanares, M.C.; Queiruga-Dios, M.Á.; Marticorena-Sánchez, R.; Escolar Llamazares, M.C.; Arnaiz González, Á. Cuestionarios de e-autoevaluación y e-feedback: Una aplicación en Moodle [E-self-assessment and e-feedback questionnaires: An application in Moodle]. Eur. J. Health Res. 2018, 2, 135–148. [Google Scholar] [CrossRef]
  10. Azevedo, R. Using hypermedia as a metacognitive tool for enhancing student learning? The role of self- regulated learning. Educ. Psychol. 2005, 40, 199–209. [Google Scholar] [CrossRef]
  11. Järvelä, S.; Malmberg, J.; Koivuniemi, M. Recognizing socially shared regulation by using the temporal sequences of online chat and logs in CSCL. Learn Instr. 2016, 42, 1–11. [Google Scholar] [CrossRef]
  12. De Kock, W.D. Speech versus text supported hints in learning to solve word problems. Comput. Human Behav. 2016, 57, 300–311. [Google Scholar] [CrossRef]
  13. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Arnaiz-González, Á.; Escolar Llamazares, M.C.; Queiruga Dios, M.Á. Detección del alumno en riesgo en titulaciones de Ciencias de la Salud: Aplicación de técnicas de Learning Analytics [Detection of at-risk students with Learning Analytics Techniques]. Eur. J. Investig. Heal. Psychol. Educ. 2018, 8, 129–142. [Google Scholar] [CrossRef] [Green Version]
  14. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; García-osorio, C.I.; Queiruga-Dios, M.Á. Conductas de aprendizaje en LMS: SRL y feedback efectivo en B-Learning [Learning behaviors in LMS: SRL and effective feedback in B-Learning]. In Temas actuales de investigación en las áreas de la Salud y la Educación [Current research topics in the areas of Health and Education], 1st ed.; Núñez, J.C., Ed.; SCINFOPER: Almería, Spain, 2017; pp. 747–752. [Google Scholar]
  15. Sáiz-Manzanares, M.C.; Queiruga-Dios, M.Á. Evaluación de estrategias metacognitivas: Aplicación de métodos online [Metacognitive skills analysis: An aplication of online assessment method]. J. Psychol. Educ. 2018, 13, 33–45. [Google Scholar] [CrossRef] [Green Version]
  16. Cerezo, R.; Núñez, J.C.; Rosário, P.; Valle, A.; Rodríguez, S.; Bernardo, A.B. New media for the promotion of self-regulated learning in higher education. Psicothema 2010, 22, 306–315. Available online: http://www.psicothema.com/pdf/3731.pdf (accessed on 11 May 2020).
  17. Segedy, J.R.; Biswas, G. Towards using coherence analysis to scaffold students in open-ended learning environments Towards using coherence analysis to scaffold students in open-ended learning environments. In Proceedings of the 17th International Conference on Artificia Intelligence in Education, Proceedings of the of AIED 2015, Madrid, Spain, 22–26 June 2015; Boticario, J., Muldner, K., Eds.; AIED2015: Madrid, Spain, 2015; pp. 49–56. [Google Scholar]
  18. Saiz-Manzanares, M.C.; Queiruga-Dios, M.Á.; Marticorena-Sánchez, R.; Escolar-Llamazares, M.C.; Arnaiz-González, Á. Cuestionarios de e-autoevaluación y e-feedback: Una aplicación en Moodle. Eur. J. Heal. Res. 2018, 4, 135. [Google Scholar] [CrossRef]
  19. Zimmerman, B.J.; Moylan, A. Self-regulation: Where metacognition and motivation intersect. In Handbook Metacognition Educ; Hacker, D.J., Graesser, A.C., Eds.; Routledge: New York, NY, USA, 2009; pp. 299–315. [Google Scholar] [CrossRef]
  20. Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Marticorena-Sánchez, R.; Zaparín-Yáñez, M.J.; Cerezo-Menéndez, R. Lifelong Learning from Sustainable Education: An Analysis with Eye Tracking and Data Mining Techniques. Sustainability 2020, 12, 1970. [Google Scholar] [CrossRef] [Green Version]
  21. Norman, E.; Furnes, B. The relationship between metacognitive experiences and learning: Is there a difference between digital and non-digital study media? Comput. Human Behav. 2016, 54, 301–309. [Google Scholar] [CrossRef] [Green Version]
  22. Matwin, S.; Mielniczuk, J. Challenges in Computational Statistics and Data Mining; Springer: Cham, Switzerland, 2016. [Google Scholar] [CrossRef]
  23. Sáiz-Manzanares, M.C.; Escolar, M.C.; Marticorena Sánchez, R.; García-Osorio, C.I.; Queiruga-Dios, M.Á. Aprendizaje Basado en Proyectos utilizando LMS: Una experiencia en Ciencias de la Salud [Project Based Learning using LMS: An experience in Health Sciences]. In Temas actuales de investigación en las áreas de la Salud y la Educación [Current research topics in the areas of Health and Education], 1st ed.; Núñez, J.C., Ed.; SCINFOPER: Almería, Spain, 2017; pp. 739–746. [Google Scholar]
  24. Fidalgo-Blanco, Á.; Sein-Echaluce, M.L.; García_Peñalvo, F.J. Impact Indicators of Educational Innovations Based on Active Methodologies. In Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality, New York, NY, USA, 16–18 October 2019; TEEM’19: León, Spain, 2019; pp. 763–769. [Google Scholar] [CrossRef] [Green Version]
  25. Azevedo, R.; Johnson, A.M.; Graesser, A.C. Use of Hypermedia to Assess and Convey Self-Regulated Learning. In Handbook of Self Regulation of Learning and Performance.; Zimmerman, B.J., Schunk, D.H., Eds.; Routledge: Abingdon, UK, 2011; pp. 102–121. [Google Scholar] [CrossRef]
  26. Hattie, J. Calibration and confidence: Where to next? Learn Instr. 2013, 24, 62–66. [Google Scholar] [CrossRef]
  27. Harks, B.; Rakoczy, K.; Hattie, J.; Besser, M.; Klieme, E. The effects of feedback on achievement, interest and self-evaluation: The role of feedback’s perceived usefulness. Educ. Psychol. 2014, 34, 269–290. [Google Scholar] [CrossRef]
  28. Peramunugamage, A.; Usoof, H.; Hapuarachchi, J. Moodle mobile plugin for problem-based learning (PBL) in engineering education. In Proceedings of the Conference: 2019 IEEE Global Engineering Education Conference (EDUCON), Dubai, UAE, 9–11 April 2019; IEEE: Dubai, UAE, 2019; pp. 850–867. [Google Scholar] [CrossRef]
  29. Sáiz-Manzanares, M.C.; García-Osorio, C.I.; Díez-Pastor, J.F.; Martín Antón, L.J. Will personalized e-Learning increase deep learning in higher education? Inf. Discov. Deliv. 2019, 47, 53–63. [Google Scholar] [CrossRef]
  30. Wentzel, K.R.; Edelman, D.W. Instruction Based on Peer Interactions. In Handbook of Research on Learning and Instruction; Wentzel, K.R., Alexander, P.A., Eds.; Routledge: Abingdon, UK, 2016; pp. 102–121. [Google Scholar] [CrossRef]
  31. Saqr, M.; Fors, U.; Tedre, M. How learning analytics can early predict under-achieving students in a blended medical education course. Med. Teach. 2017, 39, 757–767. [Google Scholar] [CrossRef]
  32. Zacharis, N.Z. A multivariate approach to predicting student outcomes in web-enabled blended learning courses. Int. High Educ. 2015, 27, 44–53. [Google Scholar] [CrossRef]
  33. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Arnáiz-González, Á.; Díez-Pastor, J.F.; Rodríguez-Arribas, S. Blended Learning: An Experience With Infographics and Virtual Laboratories Using Self-Regulated Learning. In Proceedings of the 13th International Technology, Education and Development Conference, Valencia, España, 11–13 March 2019; INTED: Valencia, España, 2019; pp. 2966–2971. [Google Scholar] [CrossRef]
  34. Gueye, A.D.; Faye, P.M.D.; Gueye, B.; Lishou, C. Scheduling Synchronous Tutoring Sessions in Learning Activities. Adv. Intell. Syst. Comput. 2020, 916, 344–352. [Google Scholar] [CrossRef]
  35. Dobashi, K. Automatic data integration from Moodle course logs to pivot tables for time series cross section analysis. Procedia Comput. Sci. 2017, 112, 1835–1844. [Google Scholar] [CrossRef]
  36. Luna, J.M.; Castro, C.; Romero, C. MDM tool: A data mining framework integrated into Moodle. Comput Appl. Eng. Educ. 2017, 25, 90–102. [Google Scholar] [CrossRef]
  37. Félix, I.M.; Ambrósio, A.P.; Neves, P.S.; Siqueira, J.; Brancher, J.D. Moodle predicta: A data mining tool for student follow up. In Proceedings of the 9th International Conference on Computer Supported Education (CSEDU 2017), Porto, Portugal, 25–27 April 2017; SCITEPRESS: Porto, Portugal, 2017; pp. 339–346. [Google Scholar] [CrossRef]
  38. Felermino, M.D.A.; Ali, S.C.N. Moodle Data Retrieval for Educational Data Mining. Int. J. Sci. Eng. Technol. 2015, 4, 523–525. [Google Scholar] [CrossRef] [Green Version]
  39. Arnaiz-González, Á.; Díez-Pastor, J.F.; Rodríguez, J.J.; García-Osorio, C. Instance selection for regression: Adapting DROP. Neurocomputing 2016, 201, 66–81. [Google Scholar] [CrossRef]
  40. Brito, M.; Medeiros, F.; Bezerra, E. A report-type plugin to indicate dropout risk in the virtual learning environment moodle. In Proceedings of the 19th International Conference on Advanced Learning Technologies (ICALT), Maceió, Brazil, 15–18 July 2019; IEEE Xplore: Maceió, Brazil, 2019; pp. 127–128. [Google Scholar] [CrossRef]
  41. Balogh, Z.; Kuchárik, M. Predicting student grades based on their usage of LMS moodle using Petri nets. Appl. Sci. 2019, 9, 4211. [Google Scholar] [CrossRef] [Green Version]
  42. Dimić, G.; Predić, B.; Rančić, D.; Petrović, V.; Maček, N.; Spalević, P. Association analysis of moodle e-tests in blended learning educational environment. Comput. Appl. Eng. Educ. 2018, 26, 417–430. [Google Scholar] [CrossRef]
  43. Waikato Environment for Knowledge Analysis (Weka). Free Software; University of Waikato: New Zealand, Australian, 2020; Available online: https://www.cs.waikato.ac.nz/ml/weka/ (accessed on 11 May 2020).
  44. RapidMiner. Software of Data Mining; University of Dortmund: Rhineland-Westphalia, Germany, 2020; Available online: https://rapidminer.com/us/ (accessed on 11 May 2020).
  45. Orange. Software Package. Available online: https://orange.biolab.si/docs/ (accessed on 11 May 2020).
  46. KNIME. Software: Creating and Productionizing Data Science; GmbH: Zurich, Switzerland, 2020. [Google Scholar]
  47. Reynaldo, V.; Wicaksana, A.; Hansun, S. Brotli data compression on moodle-based E-learning server. ICIC Express Lett. Part B Appl. 2019, 10, 963–970. [Google Scholar] [CrossRef]
  48. Dobudko, T.V.; Ochepovsky, A.V.; Gorbatov, S.V.; Hashim, W.; Maseleno, A. Functional monitoring and control in electronic information and educational environment. Int. J. Recent. Technol. Eng. 2019, 8, 1383–1386. [Google Scholar] [CrossRef]
  49. Badea, G.; Popescu, E.; Sterbini, A.; Temperini, M. Exploring the Peer Assessment Process Supported by the Enhanced Moodle Workshop in a Computer Programming Course. Adv. Intell. Syst. Comput. 2020, 1008, 124–131. [Google Scholar] [CrossRef]
  50. Amo, D.; Alier, M.; García-Peñalvo, F.J.; Fonseca, D.; Casañ, M.J. Protected Users: A Moodle Plugin to Improve Confidentiality and Privacy Support through User Aliases. Sustainability 2020, 12, 2548. [Google Scholar] [CrossRef] [Green Version]
  51. IBM Corp. SPSS Statistical Package for the Social Sciences (SPSS); Version 24; IBM: Madrid, Spain, 2016. [Google Scholar]
  52. Bandalos, D.L.; Finney, S.J. Item parceling issues in structural equation modeling. In New Development and Techniques in Structural Equation Modeling; Marcoulides, G.A., Schumacker, R.E., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2001; pp. 269–296. [Google Scholar]
  53. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, M.C.; Escolar-Llamazares, M.C.; Rodriguez-Arribas, S. eOrientation Computer Software para Moodle. In Detección del Alumno en Riesgo Académico en la Universidad; BU-09-20; General Registry of Intellectual Property: Madrid, Spain, 16 January 2020. [Google Scholar]
Figure 1. Installing the “eOrientation” plugin.
Figure 1. Installing the “eOrientation” plugin.
Applsci 10 03469 g001
Figure 2. Accessing the “eOrientation” plugin.
Figure 2. Accessing the “eOrientation” plugin.
Applsci 10 03469 g002
Figure 3. Example of different course modules. In this case, select context, assignments: practice unit. Note: “Estimulación Temprana” = early stimulation; Grupo = group; “Presencial” = face to face.
Figure 3. Example of different course modules. In this case, select context, assignments: practice unit. Note: “Estimulación Temprana” = early stimulation; Grupo = group; “Presencial” = face to face.
Applsci 10 03469 g003
Figure 4. Classification functionality of the teaching elements in the “eOrientation” plugin.
Figure 4. Classification functionality of the teaching elements in the “eOrientation” plugin.
Applsci 10 03469 g004
Figure 5. Detection of frequencies below the pre-set level (standard deviation) applied by the teacher to each activity.
Figure 5. Detection of frequencies below the pre-set level (standard deviation) applied by the teacher to each activity.
Applsci 10 03469 g005
Figure 6. Bar graph of student participation in teacher feedback on different activities.
Figure 6. Bar graph of student participation in teacher feedback on different activities.
Applsci 10 03469 g006
Figure 7. Functionality of sending a message to facilitate the orientation of the at-risk student.
Figure 7. Functionality of sending a message to facilitate the orientation of the at-risk student.
Applsci 10 03469 g007
Figure 8. Decision tree. Note. 1 = nursing students; 2 = occupational therapy students.
Figure 8. Decision tree. Note. 1 = nursing students; 2 = occupational therapy students.
Applsci 10 03469 g008
Figure 9. Visualization of the behaviour of students in both grades (nursing and occupational therapy) for variables with significant predictions.
Figure 9. Visualization of the behaviour of students in both grades (nursing and occupational therapy) for variables with significant predictions.
Applsci 10 03469 g009
Figure 10. Visualization of the behaviour of the students of the three clusters (Cluster 1, Cluster 2, Cluster 3) for the variables with significant predictions.
Figure 10. Visualization of the behaviour of the students of the three clusters (Cluster 1, Cluster 2, Cluster 3) for the variables with significant predictions.
Applsci 10 03469 g010
Figure 11. Visualization of clusters showing degree student behaviour (1 = nursing degree, 2 = occupational therapy degree) for the variable quizzes.
Figure 11. Visualization of clusters showing degree student behaviour (1 = nursing degree, 2 = occupational therapy degree) for the variable quizzes.
Applsci 10 03469 g011
Figure 12. Visualization of student behaviour in the three clusters (1 = Cluster 1; 2 = Cluster 2; 3 = Cluster 3) in the variable quizzes.
Figure 12. Visualization of student behaviour in the three clusters (1 = Cluster 1; 2 = Cluster 2; 3 = Cluster 3) in the variable quizzes.
Applsci 10 03469 g012
Figure 13. Heat map and hierarchical clustering in the variables.
Figure 13. Heat map and hierarchical clustering in the variables.
Applsci 10 03469 g013
Table 1. Description of the sample regarding the variables gender and age.
Table 1. Description of the sample regarding the variables gender and age.
Participant Type Gender
NnMennWomen
MageSDageMageSDage
Nursing students (Group 1)1371921.421.0211822.182.56
Occupational Therapy students (Group 2)1421822.942.2412422.894.11
Total2793722.161.8624222.543.46
Note. Mage = Mean of age; SDage = Standard Deviation of age.
Table 2. Analysis of the parameters of asymmetry and kurtosis normality in the types of access to the platform.
Table 2. Analysis of the parameters of asymmetry and kurtosis normality in the types of access to the platform.
Types of AccessNRangeMinMaxMSDSkewnessKurtosis
SSESSE
Feedback2798008016.1615.411.330.151.780.29
Quizzes2793850385123.7561.010.930.151.540.29
Theoretical information2798418517.7911.701.630.154.340.29
Practices279113011322.2120.171.500.152.830.29
Supplementary material279183018317.6419.523.300.1520.920.29
Information on PBL279106010612.3313.803.120.1513.700.29
Note. M = Mean; SD = Standard Deviation; Min = Minimum; Max = Maximum; S = Statistic; SE = Standard Error; PBL = project-based learning.
Table 3. Centres of final clusters for the learning results variable in blended learning type 1 and type 2.
Table 3. Centres of final clusters for the learning results variable in blended learning type 1 and type 2.
ActivitiesMaximum Cluster 1 n = 121Cluster 2 n = 122Cluster 3 n = 36
Feedback19141913
Quizzes23914172239
Theoretical information21152114
Practices26202618
Note. Cluster 1 n = 121; Cluster 2 n = 122; Cluster 1 n = 36.
Table 4. Contingency table between students in the two grades and the distribution in the clusters.
Table 4. Contingency table between students in the two grades and the distribution in the clusters.
DegreeCluster 1%Cluster 2%Cluster 3 Total
Nursing students (Group 1)6849.633626.283324.09137
Occupational therapy students (Group 2)5337.328660.5632.11142
Total12143.3712243.733612.90279

Share and Cite

MDPI and ACS Style

Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; García-Osorio, C.I. Monitoring Students at the University: Design and Application of a Moodle Plugin. Appl. Sci. 2020, 10, 3469. https://doi.org/10.3390/app10103469

AMA Style

Sáiz-Manzanares MC, Marticorena-Sánchez R, García-Osorio CI. Monitoring Students at the University: Design and Application of a Moodle Plugin. Applied Sciences. 2020; 10(10):3469. https://doi.org/10.3390/app10103469

Chicago/Turabian Style

Sáiz-Manzanares, María Consuelo, Raúl Marticorena-Sánchez, and César Ignacio García-Osorio. 2020. "Monitoring Students at the University: Design and Application of a Moodle Plugin" Applied Sciences 10, no. 10: 3469. https://doi.org/10.3390/app10103469

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop