# Artificial Intelligence to Counterweight the Effect of COVID-19 on Learning in a Sustainable Environment

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Context

- The use of learning management platforms (learning management systems, or LMS) such as Google Classroom or Moodle, which contain assessment tools with the possibility of creating open-ended, multiple-choice or true-false questions, among others;
- The use of video call tools with Teams, Zoom or Meet, which have been used in evaluations and tutoring.

- the positive predisposition of users towards the tool;
- technographic and demographic data describing the use of technological solutions and their rates of use [24].

## 3. Materials and Methods

- (1)
- Pre-COVID-19. Both teaching and assessment were done face-to-face. This happened in academic years 2016/2017, 2017/2018 and 2018/2019;
- (2)
- COVID-19 confinement. Academic year 2019/2020;
- (3)
- Post-COVID-19 confinement. In academic year 2020/2021, students were not confined at their homes, so some restrictions could be softened. In the case of our subjects, teaching was online but final exams were face-to-face.

- A statement with an explicit indication of the modifiable parameter(s);
- Minimum and maximum values of each parameter;
- A programming code (Matlab in e-valUAM) that calculates the answer of the problem as a function of the values of the parameters

- Random Forest with parameters:

## 4. Results

#### 4.1. Previous Analysis

#### 4.2. Classification Using Data from Previous Years

#### 4.3. Classification including Data from All Academic Courses

- month1: 0.08483153402588516;
- month1count: 0.12197664175455633;
- month2: 0.14907869843738905;
- month2count: 0.21073154819260959;
- month3: 0.1779015601504041;
- month3count: 0.2554800174391559.

## 5. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- UNITED NATIONS. 17 Goals to Transform Our World. Available online: https://www.un.org/sustainabledevelopment/ (accessed on 22 September 2021).
- UNITED NATIONS. Goal 4: Quality Education. Available online: https://www.un.org/sustainabledevelopment/education/ (accessed on 22 September 2021).
- Mitra, S. Hole-in-the-Wall. Available online: http://www.hole-in-the-wall.com/Beginnings.html (accessed on 22 September 2021).
- Giroux, H.A. Democracia, Educación Superior y el espectro del Autoritarismo. Entramados. Educ. Y Soc.
**2015**, 2, 15–27. [Google Scholar] - Morin, E. Los Siete Saberes Necesarios para la Educación del Futuro, 1st ed.; 6th reimp. Original Title: Les Septs Savoir Nécessaires À Léducation du Futur. Publicado por la UNESCO en 1999 ed.; Ediciones Nueva Visión: Buenos Aires, Argentina, 2011; p. 122. [Google Scholar]
- Tonucci, F. Enseñar o Aprender; Editorial Losada S.A.: Buenos Aires, Argentina, 1996. [Google Scholar]
- Semushina, E.Y.; Ziyatdinova, J.N. On-line Testing of Engineering Students as a Form of Assessment when Studying English in Distant Form. In Teaching and Learning in a Digital World; Springer: Berlin/Heidelberg, Germany, 2018; Volume 716, pp. 475–480. [Google Scholar]
- Domínguez, C.; Jaime, A.; Sánchez, A.; Blanco, J.M.; Heras, J. A comparative analysis of the consistency and difference among online self-, peer-, external- and instructor-assessments: The competitive effect. Comput. Hum. Behav.
**2016**, 60, 112–120. [Google Scholar] [CrossRef] - Subirats, L.; Fort, S.; de Haro, N.; Sacha, G.M. Data Analysis for the prediction and correction of students’ wrong learning strategies. In Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality, León, España, 16 October 2019; pp. 260–264. [Google Scholar]
- Haro-García, N.D.; Comas-López, M.; Hincz, K.P.; Mazalu, M.; Sacha, G.M. Soft Skills assessment in Art and Globalization. In Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, Salamanca, España, 24 October 2018; pp. 199–204. [Google Scholar]
- Hellas, A.; Ihantola, P.; Petersen, A.; Ajanovski, V.V.; Gutica, M.; Hynninen, T.; Knutas, A.; Leinonen, J.; Messom, C.; Liao, S.N. Predecir El Rendimiento Académico: Una Revisión Sistemática de la Literature; Association for Computing Machinery: New York, NY, USA, 2018; pp. 175–199. [Google Scholar]
- Berka, P.; Marek, L. Bachelor’s degree student dropouts: Who tend to stay and who tend to leave? Stud. Educ. Eval.
**2021**, 70, 100999. [Google Scholar] [CrossRef] - Tsiakmaki, M.; Kostopoulos, G.; Koutsonikos, G.; Pierrakeas, C.; Kotsiantis, S.; Ragos, O. Predicting University Students’ Grades Based on Previous Academic Achievements. In Proceedings of the 2018 9th International Conference on Information, Intelligence, Systems and Applications (IISA), Zante, Grecia, 23 July 2018. [Google Scholar]
- Karlos, S.; Kostopoulos, G.; Kotsiantis, S. Predicting and Interpreting Students’ Grades in Distance Higher Education through a Semi-Regression Method. Appl. Sci.
**2020**, 10, 8413. [Google Scholar] [CrossRef] - Sravani, B.; Bala, M.M. Prediction of Student Performance Using Linear Regression. In Proceedings of the 2020 International Conference for Emerging Technology (INCET), Belgaum, India, 5 June 2020. [Google Scholar]
- Márquez-Vera, C.; Cano, A.; Romero, C.; Noaman, A.Y.M.; Fardoun, H.M.; Ventura, S. Early dropout prediction using data mining: A case study with high school students. Expert Syst.
**2016**, 33, 107–124. [Google Scholar] [CrossRef] - Asif, R.; Hina, S.; Haque, S.I. Predicting Student Academic Performance using Data Mining Methods. Int. J. Comput. Sci. Netw. Secur.
**2017**, 17, 187–191. [Google Scholar] - Pérez-Villalobos, C.; Ventura-Ventura, J.; Spormann-Romeri, C.; Melipillán, R.; Jara-Reyes, C.; Paredes-Villarroel, X.; Rojas-Pino, M.; Baquedano-Rodríguez, M.; Castillo-Rabanal, I.; Parra-Ponce, P.; et al. Satisfaction with remote teaching during the first semester of the COVID-19 crisis: Psychometric properties of a scale for health students. PLoS ONE
**2021**, 16, e0250739. [Google Scholar] [CrossRef] [PubMed] - Ho, I.M.K.; Cheong, K.Y.; Weldon, A. Predicting student satisfaction of emergency remote learning in higher education during COVID-19 using machine learning techniques. PLoS ONE
**2021**, 16, e0249423. [Google Scholar] [CrossRef] [PubMed] - Rutgers, School of Arts and Sciences. Thomas A, Edison Papers. Available online: http://edison.rutgers.edu/connect.htm (accessed on 23 September 2021).
- Christodoulou, D. Teachers vs Tech?: The Case for an Ed Tech Revolution; Oxford University Press: Oxford, UK, 2020. [Google Scholar]
- UNESCO. Convención Internacional de los Derechos del Niño’; Unicef Comité Español: Madrid, Spain, 2006. [Google Scholar]
- de la Serna, M.C.; Accino, J.A.; Rivas, M.R. Formative evaluation tools within European Space of Higher Education (ESHE): E-portfolio and e-rubric. In Proceedings of the Project R+D+i eRubrics on the Evaluation of Practicum Competences 2014–2017, Málaga, Spain, June 2007. [Google Scholar]
- Mitra, A.; Hullett, C. Toward Evaluating Computer Aided Instruction: Attitudes, Demographics, Context; Elsevier: Amsterdam, The Netherlands, 1997; Volume 20, pp. 379–391. [Google Scholar]
- United Nations Educational, Scientific and Cultural Organization (UNESCO). Quick Guide to Education Indicators for SDG 4; UNESCO Institute for Statistics: Montreal, QC, Canada, 2018; p. 14. [Google Scholar]
- Molins-Ruano, P.; González-Sacristán, C.; Díez, F.; Rodríguez, P.; Moñivas, S.G. An adaptive model for computer-assisted assessment in programming skills. Int. J. Eng. Educ.
**2015**, 31, 764–770. [Google Scholar] - Molins-Ruano, P.; Jurado, F.; Rodríguez, P.; Cerezo, S.A.; Moñivas, S.G. Influence of the students’ learning strategy on the evaluation scores. In Proceedings of the 3rd International Conference on Technological Ecosystems for Enhancing Multiculturality, Porto, Portugal, 7 October 2015. [Google Scholar]
- Molins-Ruano, P.; Rodríguez, P.; Cerezo, S.A.; Moñivas, S.G. Modelling experts’ behavior with e-valUAM to measure computer science skills. Comput. Hum. Behav.
**2016**, 61, 378–385. [Google Scholar] [CrossRef] - Available online: www.evaluam.eu (accessed on 22 October 2021).
- Cady, F. The Data Science Handbook; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2017. [Google Scholar]
- Gonzalez, T.; De La Rubia, M.A.; Hincz, K.P.; Comas-Lopez, M.; Subirats, L.; Fort, S.; Sacha, G.M. Influence of COVID-19 confinement on students’ performance in higher education. PLoS ONE
**2020**, 15, e0239490. [Google Scholar] [CrossRef] [PubMed]

**Figure 2.**e-valUAM interface for students, showing one question from a self-evaluating test in the subject Applied Computing. The question in English would read “Calculate the product of the following matrices and return the value of the determinant of the resulting matrix, where a = 4”.

**Figure 3.**Scatter plot of students. The two numeric attributes of month3 and month2 are represented, together with the categorical final score (grade).

**Figure 5.**Accuracy in the classification of students’ performance for years 2019 to 2021 using a dataset that includes information from students in 2017 and 2018.

**Figure 6.**Classification task focusing on the detection of students at risk of failure, excluding data from the last month.

**Table 1.**Exploratory data analysis of the dataset. In the 1st quartile, 25% of the data is below this point. In the 2nd quartile, 50% of the data lies below this point, and in the 3rd quartile, 75% of the data lies below this point.

Attribute | Description |
---|---|

month1 [0–10]: the average mark of the month before the penultimate month | Mean: 1.86, Standard deviation (Std): 2.17, Minimum (Min): 0, 1st quartile (Q1): 0, 2nd quartile (Q2): 1, 3rd quartile (Q3): 3.5, Maximum (Max): 7.5 |

month1count: number of attempts of the month before the penultimate month | Mean: 3.42, Std: 4.42, Min: 0, Q1: 1, Q2: 2, Q3: 5, Max: 54 |

month2 [0–10]: the average mark of the penultimate month | Mean: 2.65, Std: 2.98, Min: 0, Q1: 0, Q2: 1.35, Q3: 5.20, Max: 10 |

month2count: number of attempts of the penultimate month | Mean: 3.55, Std: 4.40, Min: 0, Q1: 1, Q2: 2, Q3: 5, Max: 30 |

month3 [0–10]: the average mark of the last month | Mean: 4.60, Std: 2.64, Min: 0, Q1: 2.72, Q2: 4.70, Q3: 6.60, Max: 10 |

month3count: number of attempts of the last month | Mean: 9.95, Std: 9.06, Min: 0, Q1: 3.75, Q2: 8, Q3: 14, Max: 58 |

grade [Fail, Remarkable and Excellent]: the final grade of the subject were Fail [0–5.5], Remarkable (5.5,9) and Excellent [9,10] | Fail: 23, Remarkable: 239, Excellent: 134 |

**Table 2.**Statistical data of the simulation, taking into account information from the whole semester.

Precision | Recall | F1-Score | Support | |
---|---|---|---|---|

Excellent | 0.75 | 0.67 | 0.71 | 45 |

Fail | 0.29 | 0.25 | 0.27 | 8 |

Remarkable | 0.72 | 0.79 | 0.75 | 66 |

Accuracy | 0.71 | 119 | ||

Macro avg | 0.59 | 0.57 | 0.58 | 119 |

Weighted avg | 0.70 | 0.71 | 0.70 | 119 |

Precision | Recall | F1-Score | Support | |
---|---|---|---|---|

Excellent | 0.55 | 0.69 | 0.61 | 45 |

Fail | 0.18 | 1.00 | 0.30 | 8 |

Remarkable | 0.56 | 0.15 | 0.24 | 66 |

Accuracy | 0.41 | 119 | ||

Macro avg | 0.43 | 0.61 | 0.38 | 119 |

Weighted avg | 0.53 | 0.41 | 0.38 | 119 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Subirats, L.; Fort, S.; Atrio, S.; Sacha, G.-M. Artificial Intelligence to Counterweight the Effect of COVID-19 on Learning in a Sustainable Environment. *Appl. Sci.* **2021**, *11*, 9923.
https://doi.org/10.3390/app11219923

**AMA Style**

Subirats L, Fort S, Atrio S, Sacha G-M. Artificial Intelligence to Counterweight the Effect of COVID-19 on Learning in a Sustainable Environment. *Applied Sciences*. 2021; 11(21):9923.
https://doi.org/10.3390/app11219923

**Chicago/Turabian Style**

Subirats, Laia, Santi Fort, Santiago Atrio, and Gomez-Monivas Sacha. 2021. "Artificial Intelligence to Counterweight the Effect of COVID-19 on Learning in a Sustainable Environment" *Applied Sciences* 11, no. 21: 9923.
https://doi.org/10.3390/app11219923