Next Article in Journal
Adaptive Generalized Synchronization between Circuit and Computer Implementations of the Rössler System
Next Article in Special Issue
A Pilot Experience with Software Programming Environments as a Service for Teaching Activities
Previous Article in Journal
Fibrinogen and Fibrin Differentially Regulate the Local Hydrodynamic Environment in Neutrophil–Tumor Cell–Endothelial Cell Adhesion System
Previous Article in Special Issue
Insights from Learning Analytics for Hands-On Cloud Computing Labs in AWS
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Atenea Exams, an IMS LTI Application to Solve Scalability Problems: A Study Case

1
Institute of Sciences of Education (ICE), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain
2
Service Engineering and Information Systems Department, Universitat Politècnica de Catalunya, 08034 Barcelona, Spain
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(1), 80; https://doi.org/10.3390/app11010080
Received: 25 November 2020 / Revised: 17 December 2020 / Accepted: 18 December 2020 / Published: 24 December 2020
(This article belongs to the Special Issue Innovations in the Field of Cloud Computing and Education)

Abstract

:

Featured Application

Study case for LMS Interoperability and Scalability, using Open Source Technologies in corporate information systems.

Abstract

In 2004, the Universitat Politècnica de Catalunya decided to change one of its critical and strategic information services: the online teaching support platform. An Open Source software, Moodle, was adopted and the previous proprietary software was abandoned. In 2020, the home confinement of the population due to the pandemic caused by COVID-19 has been a stress test for the entire university community, and very specifically by those responsible for providing the support of the online teaching support platform. The increase in activity and the prospect of the generalization of online assessment raised the concerns of possible scalability issues with the quiz functionality during the examination period. The solution applied involved using Moodle’s IMS LTI interoperability features to deploy a high-performance version of the Moodle Quizz Module As A Service (SaaS), to plug-and-play within the university’s LMS without drawing on its internal resources. This solution combined with a systems strategy, private cloud operations, internal communication and teacher training, solved this problem and successfully complete the assessment of the university course in a situation of confinement.

1. Introduction

Atenea is the Virtual Campus of the Universitat Politècnica de Catalunya (UPC) based on the free software Moodle. The Moodle-based version of Atenea has been in operation since 2004, when it replaced the previous Lotus Notes-based environment [1,2]. The current Atenea is fully integrated into the teaching, enrollment and academic management processes of the university, being the service most valued by the university’s teaching and research staff in internal surveys, with an average rating of 3.9 out of 5 obtained in the internal survey among the UPC’s faculty answered by 1545 participants with more than a 50% response rate.
The project to change the UPC virtual campus to Moodle was proposed and executed by the Institute of Education Sciences of the UPC (ICE) and UPCnet—the ICT service company that provides services and is owned by the university itself. The change meant making an unprecedented decision until 2004: to move a critical service for the university to Open Source technology with General Public License (GPL), without any guarantee or any external company support that was responsible for the system, its maintenance and its evolution.
This decision implied that UPC itself with its own resources had to develop the expert knowledge and ability to integrate Moodle into its processes and systems, leverage the platform, organize its processes around and support its users. Time has shown that the decision was right. As a result, UPCNet, the consulting and technology services company created by the UPC twenty years ago, opened a new line of business doing consulting and services related to Moodle; in 2019 UPCNet under the new brand IThinkUPC (https://www.ithinkupc.com/) became a Moodle Partner.
At that time, the decision to adopt Moodle as a technology to implement Atenea, was strongly influenced by the report on the selection process of the Learning Management System (here in after LMS) based in Open Source technologies for the Universitat Jaume I, carried out Center for Education and New Technologies (CENT) which was visited by an UPC delegation [3,4,5]. Other factors in favor of the decision include:
  • Moodle is a platform based 100% on web technologies that in 2004 had become a standard.
  • Moodle design is focused on the needs of teachers, is designed by teachers and developed by a teacher learning community and based on pedagogical principles (constructivism and social constructionism) [6].
  • Several UPC professors were already using Moodle for their teaching, with installations of Moodle hosted in PCs in their offices set up and operated by the professors themselves.
  • At the School of Informatics of the UPC, several final degree projects which aim was to contribute to the code base of the Moodle community, were being developed. Students doing these projects became internal talent of UPC and UPCNet, experts in the environment and active participants in the Moodle community.
  • Another important influence is the cultural change within the UPC with respect to the free software movement. UPC’s Board of Management launched the Chair of Free Software to serve as a vehicle for all the initiatives undertaken by the University in this area. The Chair of Free Software, which was created in September 2004, aimed to raise awareness of Open Source software at UPC and in society at large [7].
During the first four months of the 2005–2006 academic year, a pilot test was carried out with 9 schools, involving 132 subjects, 216 groups and 2908 students. In January 2006, once the testing phase was over, the UPC approved the full implementation of Moodle as the technological platform for the Atenea virtual campus for the next academic year. The adoption of Moodle as a teaching support tool at Atenea was a success, with very few problems considering the level of adoption and size of the university.
At the second MoodleMoot España conference organized in the Canary Islands in 2005, the case of the adoption of Moodle by the UPC was presented. The case of Atenea as an instance of Moodle for more than 30,000 users was a success story that validated the viability of Moodle at the technological level, as a teaching and organizational tool, and contributed to other Spanish universities and education departments adopted Moodle as an LMS to support their university, primary and secondary education.
The UPC has been active in its collaboration within the Moodle community, doing research, providing services, proposing improvements and adding developments to the codebase. Until 2020 the Atenea platform has been operating without problems serving the university community.
In 2010, UPCnet started changing its internal server architectures in its Data Processing Center (DPC). After an initial phase of virtualization of the physical servers, the DPC started to operate a private cloud. A private cloud is a model of hosting where cloud computing services and IT infrastructure are hosted privately and dedicated to a single organization with isolated access. It can be hosted on-premise or by a third-party cloud hosting provider. With this change in architecture UPCNet gained the flexibility of the cloud computing model—being able to create, start, stop and pause virtual computing assets for the projects and the operations, and having the option of using seamlessly assets in public clouds (like Amazon Web Services—if the necessity arises).
Operating a private cloud on its DPC allows UPCNet to ensure GDPR compliance and other EU regulations in internal UPC projects like the Atenea Platform, and external projects of UPCNet’s customers. For example, the UPCNet private cloud runs the Agora platform which serves independent Moodle instances for all the government primary and secondary education centers and some of the private owned ones—more than 2500 Moodle instances.
On March 14, 2020, the confinement of the population throughout Spain was decreed due to the emergency caused by the COVID-19 pandemic. For the UPC, as for the rest of the state’s universities, this means moving all teaching to online format overnight. The daily creation of new resources and activities immediately doubled as did most activity indicators.
There was also a significant increase in workload. Internal server resources saw an 800% increase in platform stress. UPCNet technicians needed to address the workload issue by identifying bottlenecks and increasing resources dedicated to virtual machines. Some of these actions needed to be done urgently in a few weeks.
Once it was clear that the confinement would last until the end of the course, Atenea officials saw that the assessments should be done online in all faculties of the UPC. There are many ways to do online assessment, ranging from the delivery of exercises, the continuous assessment of activities and practices, presentations and oral exams by video conference, etc. One of the forms of assessment is the use of test-type exams, implemented in Moodle with the Quiz module.
The Moodle Quiz module allows you to define questionnaires in a very agile way. It allows you to work with various types of questions, create question banks organized by categories, create personalized exams for each student with random questions from established categories, randomize the order of answers, take self-assessment tests, make more than one attempt, and much more. This is a very complex module, with more than 100,000 lines of source code and making intensive use of the database engine.
At that time, the main question was whether Atenea would be able to bear the overload once large or very large courses (≥200 students) started doing questionnaires simultaneously. In previous simulations the UPCNet technicians discovered performance issues while doing some of the tests based on the Quiz module, and experienced some glitches during partial evaluations. So the main question UPC was forced to address was how to solve these performance issues when turning to online learning and assessment.
The main aim of this paper is to explain how UPC solved the presented problem using the Atenea based-Moodle virtual campus.
In the case of Atenea, we found out that the fact that the UPC has cultivated its expertise, knowledge a collaborative relationship with the Moodle.org community over the prevlious 15 years, has been a key aspect to solve the problem of massive online evaluation using the Quiz module. The UPC has been able to respond to the challenges of the COVID-19 crisis in part because it has this internal talent. By outsourcing critical services to SaaS models, certain complexities cannot be addressed and innovated upon.

2. LMS Interoperability

2.1. Introduction to Moodle’s Interoperability

Learning and teaching is one of those application areas where there are available different learning platforms, repositories, tools, types of content, etc. These systems should interoperate among them to provide better and more useful learning services to students and teachers, and to do so web services and interoperability specifications are needed.
In 2008 most LMS were not service oriented, so any attempt to integrate external applications with the LMS should be done ad hoc. This approach has important disadvantages such as the difficulty in maintaining and extending the new integrated system or the limited impact of these solutions [8].
Moodle is not an exception, in 2008 an extension of the Moodle design that allows to offer web services, in various protocols and forms of authentication was proposed. This extension of webservices facilitated the tasks of integration with academic management systems, and allowed the development of mobile apps that act as a client of Moodle, as shown in Moodbile [9,10,11,12].
In 2010, the version 2.0 of Moodle was published, it incorporated the Webservices extension described in Figure 1. The webservices extension provides services like authentication, authorization, enrollments, access to rosters, etc. Gradually the number of APIs in this webservices architecture allowed enough control over Moodle as to develop a fully operational Mobile client.
Interoperability specifications define ways to exchange information and/or interaction between systems, they can be focused on the exchange of a specific type of information or address interoperability in a global way [13,14]. According to Wilson one possible scenario of interoperability is based on the integration of external tools into the LMS [15]. Some examples of experiences that integrate tools with LMS and learning platforms can be found in [16,17,18,19,20,21,22].

2.2. IMS Learning Tools Interoperability

In 2008 the IMS Global Learning Consortium was working on a draft for a specification of its Learning Tools Interoperability (LTI) standard [23]. The purpose of this standard was to enable the execution of educational applications as a service as an activity within an LMS course. At that time, the standard was only a draft, it was considered to be way too complex and there was no reference implementation.
A simplified specification called SimpleLTI was proposed, a kind of minimally viable LTI product [24]. A reference implementation of SimpleLTI consumer was developed for Moodle by researchers of UPC, under the direction of Severance and Alier. SimpleLTI evolved into a more complete specification called BasicLTI, which implemented the webservices communications between the Provider and the Consumer. Gradually all major LMS vendors (Moodle, Sakai, Blackboard, Canvas, Desire2Learn) started to support BasicLTI.
In 2012 when IMS finally published the IMS LTI 1.0 specification 2012 Moodle incorporated in its official distribution a LTI consumer extension under the name “External Tool”. Figure 2 shows the “External Tool” (in its Catalan translation “Eina externa”) in the palette of available learning activities in Atenea.
Years later, a Moodle extension (LTI Producer) that allows a Moodle course or activity to act as an LTI service provider was added to the main distribution.

2.3. The LTI Protocol

IMS LTI enables the model of SaaS for learning activities within the context of the LMS. LTI enables to plug an external application into the LMS within a determinate context.
In the LTI protocol there are two relevant actors: the LTI Provider and the LMS acting as a Consumer. A service in the form of a Web application is provided (the Learning Activity or Learning Tool) to the LMS users so they can access this tool once some setup is done.
In an LMS, a learning tool is contextualized in a virtual classroom that may be a Moodle Course, the group or subgroup of students and the activity itself. For instance, a Discussion Forum in the Main Group of the Introduction to Cloud Computing Course. These three elements: course, group and activity define a specific activity and its context. When a user accesses the activity, he does so with an identification (Id) and a set of specific permissions and privileges granted by his Role (teacher, student, visitor, etc.), and this is managed by the LMS with internal services of authentication and authorization. In addition to that, the activity reports its outcomes to the LMS: logging information, learning outcomes and grades.
When a system acts as an LTI Provider, offering an online learning activity as a service via the web, it will receive the activity context (course, group, activity id), and the user’s identification, authentication and role from the LTI Consumer (in our case the LMS) during the launch of the activity and via webservices communications between Provider and Consumer.
Let us suppose that we have a web app that implements a Code Dojo where students can do coding exercises and some kind of group interaction. The Code Dojo can act as LTI Provider, so it is set up to get the user’s Id, context, and role from the LTI Launch when a student or teacher clicks on the LTI Activity (External Tool) in the LMS as illustrated in Figure 3. The Code Dojo can report back to the LMS information about the student’s logs, learning outcomes and grades via webservices defined by the LTI Protocol.
Before the students can access the LTI learning activity a configuration of LTI activity in the LMS is required. An LMS activity—in a specific course and group—connects to a specific LTI service, this is set up with the External Tool in Figure 3, where the teacher can enter the proper configuration parameters (URL and Secret). Once the LMS LTI Consumer is set up, when the LMS users access this activity, they will be redirected to the LTI provider’s website with the proper context, identification and authorizations to enjoy it.
As shown in Figure 3, when the LMS user accesses an LTI Activity, in the LTI Launch the LMS communicates to the LTI Provider the execution context of the activity, an identifier for the user—which may or may not include the user’s personal data depending on how it is configured the system—and the permissions of this user in the context of the activity (teacher, student, etc.). The LTI provider can send student activity records and assessment data back to the LMS.
There are some advantages of using an external LTI tool compared to modular Moodle extensions:
  • It does not pose a security or integrity risk to a Moodle instance. If LTI Activity drops, it does not affect the operation of the LMS. There is simply one activity that stops working. A security or performance issue in a module can compromise the operation of the entire LMS.
  • There is no need to update LTI activities when the LMS is updated. While the activity complies with the LTI protocol, it acts independently.
  • LTI Activities allow new forms of collaboration, a company can offer an LTI Activity as a service to customers around the world and regardless of the LMS they use (Moodle, Canvas, Sakai, Blackboard, etc.), as most LMSs currently support the LTI standard. However it poses a risk to the privacy and confidentiality of students.
  • Because Moodle can act as an LTI Provider, unofficial module functionalities can be incorporated into a large instance such as Atenea. You simply need to install them on a separate Moodle server and offer them as LTI activities.
In fact, you can also offer features offered by default in the Moodle distribution to solve scalability and performance issues, as we will see in the next section.

3. Solution and Results

The confinement of the population throughout Spain caused by the COVID-19 pandemic forced the UPC, and the rest of the state’s universities, to move all teaching to online format overnight. The daily creation of new resources and activities in Atenea immediately doubled as did most activity indicators. There is also a significant increase in workload (see Figure 4 and Figure 5).
All of the indicators analyzed increased following the same patterns, and these increases compounded its impact on the demand of computing resources in the platform which increased more than 8-fold.

3.1. Training the Faculty

Given this situation, the Institute of Education Sciences (ICE) of the UPC was a key player, as the body responsible for faculty training. During this crisis period, the ICE provided a very fast and proactive response, working in forced marches, becoming a benchmark of expertise to make the abrupt transition to online learning. A number of specific actions were taken:
  • The creation of specific courses for the transition to online teaching. Some of which were designed, prepared and offered to faculty urgently in a few days.
  • The documentation produced by various units of the university was organized in a single point of documentary access.
  • Multimedia content (video podcasts) was created to answer the specific questions received.
  • Q&A sessions were conducted through videoconferencing, with massive participation (more than 200 faculty members simultaneously).
With regard to teacher training, it should be mentioned that from March to July 2020, 79 specific training courses are being developed for the transition to non-face-to-face teaching, with a total of 6190 applications for registration. This offer responds to a specific total of 4559 registrations, with an average of 57, 7 attendees per course. The academic and management part obtained a quality index valued at 4.53 out of 5, according to the surveys carried out after each course and on all participants (See Table 1).
The change in the way the Atenea platform is used is not limited to the faculty. Students are starting to use the platform more and in a different way. The maximum number of concurrent sessions goes from 4000 to 6000. Surprisingly, Figure 6 shows how the levels of access on weekends and during Easter holidays are equivalent to, or higher, than those of school days before confinement.
The next academic year Q1-2020-2021 started with some courses taught in the classrooms and some courses with a hybrid approach: teaching done part-time online and part-time in the classrooms. This situation changed in November 1—when we returned to online teaching due to the second wave of the COVID-19 pandemic. However, even if not all the teaching is done online in this semester we can observe that the usage of Atenea keeps on intensifying: the maximum number of concurrent sessions in Atenea are maintained and they even increase slightly (see Figure 7). Whether this suggests a permanent change in the way teachers and students use online tools is a question that will be answered after this pandemic end.
The way the faculty uses the LMS for the learning design in their courses also changed. A comparative analysis of the activity in the LMS between the course 2018–2019 Q2 and the course 2019–2020 Q2 reveals some interesting facts. The number of tasks (assignments) in the platform doubled, and half of the courses presented more than 40 tasks while in the last year the number was down to 7. The number of questionnaires per student and course quadrupled compared to the last course.

3.2. A solution to Cope with Scalability Issues in Atenea

As stated in the previous section, the Atenea platform is suddenly subject to a very significant increase in workload. Specifically, the UPC wants to assure that Atenea will be able to bear the overload when students in several large or very large courses, start the online questionnaires in order to do online assessment. This concern is caused by performance issues observed while doing some of the tests based on the Quiz module, in partial evaluations.
Before we explain the solution proposed to deal with this performance tests in Atenea, we need to discuss the current systems architecture of Atenea, why it was chosen and its evolution in the last 15 years.

3.2.1. Atenea: System Architecture

As mentioned above, with the adoption of Moodle as a teaching support platform, the UPC does an investment in the university’s internal talent. Recently, Moodle.com, the company responsible for the development and distribution of Moodle, established its second headquarters in Barcelona, and has hired several people from UPC and UPCNet [15].
As a result of the university’s involvement with Moodle, several research and development projects are being carried out which have as a relevant result for this article, the incorporation of two new interoperability technologies into the Moodle system itself.
The Moodle-based Atenea virtual campus must serve as a training support platform at the UPC for [25]:
  • 22,682 undergraduate students
  • 5269 master’s degree students
  • 2210 doctoral students
  • 2870 continuing education students
  • 64 undergraduate studies
  • 75 university master’s degree programs
  • 46 doctoral programs
  • 234 continuing education programs
The Atenea platform has considerable quality and performance requirements. It was estimated that the platform should handle at least 5000 simultaneous requests, and offer high availability. It is not acceptable for the platform to fall, for example, in the middle of a questionnaire-type exam. The Atenea platform needs to be available 24/7 throughout all year. The service only stops very occasionally for maintenance, usually on weekends and with notice to the faculty so it does not interfere with academic activity, like deadlines for task deliveries. Since its migration to the private cloud architecture these services interruptions are rare.
Therefore, a study had to be conducted to ascertain scalability and reliability parameters.
Moodle works in a simple LAMP environment (Linux, Apache, MySQL and PHP), which can be installed and run on a low-end PC from 2004. However, if one needs to support a large number of simultaneous connections, as many as 9000, there is a major scalability problem as there are several bottlenecks to be solved. Moodle, since its version 1.4, supported PostgreSQL, a more robust and scalable database than MySQL at the time. A first step in increasing Moodle performance was to move the database server to a dedicated, optimized machine. However, the adjustment was not enough for the scalability requirements, as 9000 possible concurrent transactions far exceed what a PHP application server can support. At that time two alternatives were presented [26]:
  • Multi-instance Moodle: implement Atenea with several instances of Moodle, separated by the organizational context (School, degree, campus). See Figure 8.
  • One single high-performance Moodle instance: work in a single instance, with code, database and file system shared by several servers working in parallel. See Figure 9.
The first option involves greater complexity in the software. The second option involves greater complexity in the configuration of systems. In the following paragraphs we are going to discuss the advantages and disadvantages of both options and the final selected architecture UPC adopted.
The first option (called multi-instance Moodle) is based on the creation of a first web application as a point of entry and user authentication. After logging in, users find a list of links to the courses where they are enrolled. When you click on a link, the user is directed to the server hosting the specific Moodle course. Instead of operating a single instance of high-performance Moodle, it works with multiple instances that receive a usage load within their performance parameters.
This option has the following advantages:
  • It works with Moodle instances with workloads that servers know they can take on.
  • At a time when the speed of Internet connection was limited in some of the UPC Campuses (the UPC has a Campus in 7 different locations in the province of Barcelona back in 2004), the machines that serve the Moodle instances can be located in each campus to provide a good service.
  • If a Moodle instance fails due to a server or software issue, this only affects the courses served by that instance, without affecting the rest of the university.
  • It allows to assemble machines with modules of educational application developed by third parties, for teaching innovation projects. Moodle has a modular architecture and allows you to extend its functionality with activity modules, course, topics and other extensions. Some of the projects developed internally at the UPC consist of activity modules for teaching innovation projects. These modules can be securely installed on a multi-instance system.
However, this option presents several drawbacks:
  • At that time (2004), the UPC had not yet trained enough developers to work with Moodle in such endeavor, and the project to create a multi-instance management software is considered risky.
  • The processes of integration with academic management (user registration, course registration and grades) become much more complex in a multi-instance system.
  • Day to day, administration of a multi instance installation can be more complex and demanding of effort.
  • The developed software must be maintained and updated to work with each new version of Moodle. It is important to update frequently to the latest version for security updates.
  • At that moment, the university management was implementing a policy of centralizing services of all kinds, and it was not seen as a positive thing to move servers to different campuses.
The second option (one single high performance Moodle instance) is based on the implementation of a single instance of Moodle to serve all schools and colleges in the university. This instance runs on 9 servers.
Two of these machines act as load balancers. These load balancers route the requests of the web client to all the servers able to satisfy these requests in a way that maximizes the speed, the utilization of the capacity of the systems and guarantees that there is no server with an excess of work, something that can degrade system performance. If a server fails, the load balancer redirects traffic to the remaining online servers.
The load balancer performs the following functions:
  • Distributes client requests or network load efficiently across multiple servers.
  • Ensures high availability and reliability by sending requests only to servers that are online.
  • Provides the flexibility to add or subtract servers as per demand.
One of the load balancers is active, balancing web traffic. While the other is in passive mode, to replace of the other balancer immediately in the event of a service outage.
Six machines act as front-end servers, executing Atenea’s PHP code according to the requests they receive from the load balancer. These six front-ends (or Application Servers) run the same source code and work on the same network-mounted file system via Networked File System (NFS). This networked, fiber-optic shared file system is mounted in a high-performance disk cabinet, with cache, backup-restore, and hot disk replacement systems.
Finally, all front-ends share the same PostgreSQL database management system, which runs on an optimized server and manages the transactions of the six servers.
It is important to remember that in 2004 when we talk about servers we are still talking about physical machines, not virtualized or cloud servers.
The advantages of this configuration were, at that time:
  • Load testing proves that this architecture worked and fully supports performance requirements.
  • The integration of the system with the academic management system (user registration, course registration and sending notes) is much simpler with a single instance of Moodle.
  • UPCnet already has the expert technical staff to configure and manage complex server architectures.
  • The policy of the university management is favorable to a single instance that integrates the service to the whole university.
  • The disadvantages of this option are:
  • Single point of failure. A drop in service implies a drop in service for the entire university.
  • Extensive load and performance testing is required each time you upgrade to a new version of Moodle, and make changes as needed.
  • Third-party modules cannot be installed—even if developed at the UPC—in the single instance of Moodle.
  • A malfunction affects the entire university.

3.2.2. The Selected Architecture

The UPC chose option 2: a single high-performance instance. This choice was copied by many other institutions, some such as the University of Barcelona which with the technical support of UPCNet deployed its virtual Campus for 80,000 users.
Independent Moodle servers called Atenea Labs were made available to teachers who requested them to accommodate teaching innovation activities with experimental Moodle modules. The servers at Atenea Labs were not initially connected to the UPC’s user authentication system, and user registrations for student courses must be made manually. However, one of the modules tested at Atenea Labs, the Wiki module developed by David Castro, Ferran Recio, Jordi Piguillem and other FIB students, became part of the official distribution of Moodle from version 2.0 in 2010.
The architecture chosen in a single instance turns out to be a good option and together with a good execution by UPCNet, the Moodle-based Atenea platform has been providing services for 16 years without major incidents.
In the autumn term of the 2019–2020 academic year, the Atenea system supports approximately:
  • 60,000 daily user sessions
  • Peaks of 6000 simultaneous sessions (with historical peaks of 9000 concurrent sessions)

3.2.3. Virtualization and Private Cloud

From 2005, the Data Processing Center (CPD) of the UPC began to incorporate virtualization systems. By 2009, the entire Atenea service was moved to virtualized servers in the UPC Private Cloud, in virtualization clusters. A virtualization Cluster is a group of hosts. When a host is added to a cluster, the host’s resources become part of the cluster’s resources. The cluster manages the resources of all hosts within it. Clusters enable high availability and distributed resource scheduling [27,28].
This move to a virtualized system does not mean a change of architecture. The scheme of 2 load balancers, 6 front-ends, fiber optic connected disk cabinet and a server for the database engine is still maintained. However, virtualization allows certain advantages over using physical servers:
  • Better server recovery capability in case of crash. The virtualization cluster manages the recovery of virtual servers in the event of a crash on the physical machine where the virtual machine is running.
  • Possibility of implementing high availability strategies, forcing the presence of redundant services in different clusters. For example, not putting all front-ends or load balancers in the same cluster.
  • Possibility to dynamically distribute physical resources (processors, memory, etc.) to one service or another depending on the workload. In this way, load peaks can be addressed, and resources redirected to other services when the system has less load. It turns out that the workload at Atenea varies greatly depending on the calendar (enrollment periods, course, homework assignments, exams, holidays), day of the week, time, or even time of day: when there is an exam all students start the quiz at the same time.

3.3. Atenea Exams

Once we have explained the architecture of Atenea we are going back to one of the problems we faced when turning to online learning: the possible overload because of a massive use of the Quiz Moodle module.
A possible solution would have been to create a duplicate of Atenea to conduct the quizzes, this would have involved creating duplicates of all courses, with all student registrations, which presents a lot of technical problems regarding data integrity. It would also generate different entry points to courses and places to take an exam, creating a worse user experience for the students. Another problem is the synchronization of student grades. If we were to have two Ateneas: where can the student find her or his grades?
The solution chosen was to deploy a high-performance version of the Moodle Quizz Module As A Service, to plug-and-play within the university’s LMS without drawing on its internal resources, solving the possible overload problem. This would be implemented by a new Moodle acting as an LTI Provider, so that the teachers create the questionnaires there and connect them with the Atenea courses where they teach their subjects as an External Activity.
This LTI service would be supported by a new high-performance standalone Moodle instance that could support a hypothetical maximum load of 9000 simultaneous connections. This new instance was called Atenea Exams, making clear its purpose.
Atenea Exams was mounted following the same system architecture as Atenea: 2 load balancers, 6 front-end, a dedicated server for the database. All virtualized in virtualization clusters in the private cloud. The only difference is that the drive bay used is a new high-performance drive with all the high-speed SSDs, which has already been purchased but not deployed into production.
Atenea Exams is not set as a copy of Atenea the LMS, is just a LTI Provider that offers the Moodle Quiz Activity and the Task Activity like a SaaS. The professor creates the Quiz in his course in Atenea Exams, publishes it as a LTI Activity using the Moodle LTI Provider feature and sets it up in Atenea using the Moodle External Tool (LTI Consumer) activity module.
In Figure 10 we present an overview of the interaction between Atenea Exams and Atenea. Atenea Exams is a platform set up only for the purpose of publishing quizzes as LTI Applications, Atenea is the LMS already in production. Let us analyze how it works:
(1)
The professor logs in Atenea Exams, there she finds a course with the same name and code as the one she is teaching in Atenea. This course is empty. There she can create a quiz, either from scratch or importing it from a backup file from her Atenea course.
(2)
Using the Moodle LTI Provider plugin the professor publishes the quiz as a LTI application.
(3)
The professor logs onto Atenea.
(4)
She goes to her course and creates an External Tool activity as shown in Fig 2. and binds the activity to the quiz she has created in step 2.
(5)
The students log in Atenea, access their course and click on the External Tool activity.
(6)
The activity redirects the student to Atenea Exams, where the student is automatically logged in by the LTI protocol, and the student can take her test.
With LTI external activities the students find the questionnaires transparently, as they would do with another quiz in the Atenea course. And the grades are set back in the Atenea gradebook.
However, the complication of creating questionnaires in Atenea Exams, publishing them as an LTI activity and incorporating them as an external activity in the Atenea course is completely passed on to teachers. This is not trivial, if we consider the situation: more than learning how to do online lectures for a course where the workload is already very high, the move to teleworking (often working with each teacher’s particular equipment and internet connections) and in confinement situation, competing for spaces, resources, and quiet moments to work with other family members, often also confined students.
This involved a significant effort of communication and faculty training. Training materials and courses had to be put together urgently, including web documentation and a training video explaining to teachers how to create questionnaires and integrate them into their courses [29]. Three training sessions were also organized by videoconference, in which approximately a hundred participants in each, and lasted for almost two hours.
The results of the experiment was satisfactory. With the combined use of the Atenea and Atenea Exams platforms, the evaluation process could be conducted throughout the university with minimal incidents.

4. Conclusions

A university’s Virtual Campus is a critical asset. This study case shows that using Open Source technology for critical information systems is a good practice. The organization cultivates the expertise at all levels necessary to face a challenge or a crisis. Having the operational, technical, internal training capacity, analysis and proposal of solutions to problems, and decision making.
In the case of Moodle, the UPC had cultivated this expertise over the previous 15 years plus a collaborative relationship with the Moodle.org community.
The massive usage of the Quiz Module in the Atenea platform entailed the risk of recurring failures of the single instance of the LMS for the university. The consequences of such failures would be dire in a critical moment for the institution. The solution provided was the use set of interoperability technologies, which were coincidentally contributed to the Moodle community by researchers from UPC, to create a SaaS plug and play version of the Moodle Quizz that spared the use of internal resources from the LMS platform and preserving its stability.
The UPC was been able to respond to the challenges of the COVID-19 crisis because it has this internal talent. When critical and strategic services are outsourced certain complexities cannot be addressed. The response must come from an external entity with other priorities and no obligation to respond beyond the agreed terms of quality of service (QoS). Since the UPC and UPCnet operate their own private cloud, these problems fall to the roof of the UPC, which has the ability to solve them on its own, maintaining the option of using Public Cloud infrastructures if necessary.
On the other hand, some of the services needed to make the transition to face-to-face teaching are the tools in the Google Suite that were already deployed at the UPC just before the emergency. This indicates that the line that separates the services that a university must self-procure and those that it must outsource depend on the criteria of each institution.
However, the authors believe that the essential services of Virtual Campus and academic management must be operated by the institution itself using Open Source technologies, and the capacity and talent to innovate in this space must be retained—leaving interoperability mechanisms such as IMS LTI to incorporate third-party services as appropriate.

Author Contributions

Conceptualization, M.A. and M.J.C.; methodology, M.J.C.; software, M.A. and M.J.C.; validation, A.L., J.A.; formal analysis, M.A.; investigation, M.A., M.J.C. and J.A.; resources, A.L.; data curation, J.d.P. and M.A.; writing—original draft preparation, M.A.; writing—review and editing, M.J.C.; visualization, M.J.C., M.A. and J.d.P.; supervision, M.J.C.; project administration, A.L.; funding acquisition, A.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study did not require ethical approval. No personal information was used in this study. All data was anonymized and aggregated.

Informed Consent Statement

According to the GDPR all the students using the Atenea platform have been informed and given consent to the treatment of their data.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to compliance with GDPR.

Acknowledgments

We want to pay respect to those who had some part in the story told in this paper and have helped us to fill in some blanks and better understand it. Jordi Adell and the CENT of the Universitat Jaume I, who were the first ones to bet on Moodle as a LMS for a big institution. Ramón Martí, Oriol Sanchez, Enric Ribot who led the migration to Moodle for the Atenea platform. Martin Dougiamas Moodle lead developer and founder who always listened to all of our suggestions regarding interoperability. Charles Severance Sakai developer and author of SimpleLTI and BasicLTI who worked with us in the early implementations of SimpleLTI and BasicLTI. And the researchers of the DFWikiteam David Castro, Ferran Recio, Jordi Piguillem and Nikolaos Galanis who implemented the first versions of Moodle Webservices, SimpleLTI and BasicLTI for Moodle. The authors want to thank the collaboration of UPCNet/IthinkUPC, specially Nacho Ruiz de Erechun, Miguel Lopez and Fran Rodriguez. Last but not least we are grateful to the Education Sciences Institute (ICE) of the Universitat Politècnica de Catalunya (UPC). Specifically, we owe our deepest gratitude to its scholarship holder Francesc Salvador and the ICE’s administrative team.

Conflicts of Interest

The authors have no conflicts of interest.

References

  1. Alcober, J.; Rincon, D. The Digital Campus: An experience of an asynchronous collaborative learning platform at the UPC. In Proceedings of the ISTEP2000—International Symposium on Telemedicine and Teleeducation in Practice, Kosice, Slovakia, 22–24 March 2000. [Google Scholar]
  2. Alcober, J. The UPC Digital Campus and the Project Based Learning: A compatibility effort. In Proceedings of the International Conference on Network Universities & E-Learning (mENU 2003), Valencia, Spain, 31 May 2003; ISBN 84-9705-369-9. [Google Scholar]
  3. Adell, J.; Castellet, J.; Gumbau, J. Selección de un entorno virtual de enseñanza/aprendizaje de código fuente abierto para la Universidad Jaume I. In Centre d’Educació i Noves Tecnologies de la UJI, Servei d’Informàtica i Gabinet Tècnic del Rectorat; Internal Report; 2004. Available online: http://cent.uji.es/doc/eveauji_es.pdf (accessed on 5 November 2020).
  4. Adell, J. Arqueología digital: Los primeros servidores web de España. 2002. Available online: https://elbonia.cent.uji.es/jordi/2008/03/15/arqueologia-digital-los-primeros-servidores-web-de-espana/ (accessed on 5 November 2020).
  5. ABERDOUR, M. Open Source Learning Management Systems. In EPIC White Paper. 2007. Available online: http://www.cedma-europe.org/newsletter%20articles/Kineo/Open%20Source%20Learning%20Management%20Systems%20(Oct%2007).pdf (accessed on 23 December 2020).
  6. Dougiamas, M.; Taylor, P. Moodle: Using learning communities to create an open source course management system. In Proceedings of the EdMedia+ Innovate Learning, Association for the Advancement of Computing in Education (AACE), Honolulu, HI, USA, 23 September 2003; pp. 171–178. [Google Scholar]
  7. UPCCommons. Open Knowledge UPC Portal. Free and Open Source Chair. Available online: https://upcommons.upc.edu/handle/2099.2/2152 (accessed on 5 November 2020).
  8. Alier, M.; Guerrero, M.J.C.; Gonzalez, M.A.C.; Penalvo, F.J.G.; Severance, C. Interoperability for LMS: The missing piece to become the common place for e-learning innovation. Int. J. Knowl. Learn. 2010, 6, 130. [Google Scholar] [CrossRef]
  9. Casany Guerrero, M.J.; Alier Forment, M.; Mayol Sarroca, E.; Piguillem Poch, J.; Galanis, N.; García Peñalvo, F.J.; Conde González, M.Á. Extending Moodle services to mobile devices: The Moodbile project. In Proceedings of the UBICOMM 2012, the Sixth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, Barcelona, Spain, 23–28 September 2012; pp. 24–28. [Google Scholar]
  10. Alier, M.; Casany, M.J. Moodlbile: Extending moodle to the mobile on/offline scenario. In Proceedings of the IADIS International Conference Mobile Learning, Amsterdam, The Netherlands, 22–25 July 2008; pp. 11–18. [Google Scholar]
  11. Alier, M.; Casañ, M.J.; Piguillem, J. Moodle 2.0: Shifting from a Learning Toolkit to an Open Learning Platform. In International Conference on Technology Enhanced Learning; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
  12. Al-Rayis, E.; Kurdi, H. Performance Analysis of load balancing Architectures in Cloud computing. In Proceedings of the 2013 European Modelling Symposium, Manchester, UK, 20–22 November 2013; pp. 520–524. [Google Scholar]
  13. Alario-Hoyos, C.; Wilson, S. Comparison of the main alternatives to the integ-ration of external tools in different platforms. In Proceedings of the International Conference of Education, Research and Innovation, ICERI 2010, Series Comparison of the Main Alternatives to the Integration of External Tools in Different Platforms, Madrid, Spain, 15–17 November 2010; pp. 3466–3476. [Google Scholar]
  14. Severance, C.; Hardin, J.; Whyte, A. The coming functionality mash-up in Personal Learning Environments. Interact. Learn. Environ. 2008, 16, 47–62. [Google Scholar] [CrossRef]
  15. Wilson, S.; Sharples, P.; Griffiths, D. Distributing Education Services to Personal and Institutional Systems Using Widgets. In Mash-Up Personal Learning Environments—F. Wild, M. Kalz, M. Palmér: Mash-Up Personal Learning Environments. Proceedings of the 1st Workshop MUPPLE’08, Maastricht, The Netherlands, 17 September 2008, CEUR Workshop Proceedings; ISSN 1613-0073. pp. 25–33. Available online: CEUR-WS.org/Vol-388/ (accessed on 23 December 2020).
  16. Alier, M.; Mayol, E.; Casañ, M.J.; Piguillem, J.; Merriman, J.W.; Conde, M.Á.; Severance, C. Clustering Projects for eLearning Interoperability. J. Univers. Comput. Sci. 2012, 18, 106–122. [Google Scholar] [CrossRef][Green Version]
  17. Conde, M.; García-Peñalvo, F.; Piguillem, J.; Casany, M.; Alier, M. Interoperability in eLearning Contexts. Interaction between LMS and PLE. In 1st Symposium on Languages, Applications and Technologies; Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik: Wadem, Germany, 2012. [Google Scholar]
  18. Booth, A.G.; Clark, B.P. A service-oriented virtual learning environment. Horizon 2009, 17, 232–244. [Google Scholar] [CrossRef]
  19. De-la Fuente-Valentín, L.; Leony, D.; Pardo, A.; Delgado-Kloos, C. Mashups in Learning Design: Pushing the Flexibility Envelope. In Mash-Up Personal Learning Environments—1st Workshop MUPPLE’08; Maastricht School of Management: Maastricht, Maastricht, The Netherlands, 2008; pp. 18–24. [Google Scholar]
  20. Verpoorten, D.; Glahn, C.; Kravcik, M.; Ternier, S.; Specht, M. Personalisation of learning in virtual learning environments. In Proceedings of the Learning in the Synergy of Multiple Disciplines of the 4th European Conference on Technology Enhanced Learning, Nize, France, 29 September–2 October 2009. [Google Scholar]
  21. Multiple Disciplines, Series Personalization of Learning in Virtual Learning Environments; Springer: Nice, France, 2009; pp. 52–66.
  22. Wilson, S.; Sharples, P.; Griffiths, D.; Popat, K. Moodle wave: Reinventing the vle using widget technologies. In Mash-Up Personal Learning Environments—2nd Workshop MUPPLE’09; Wild, F., Kalz, M., Palmér, M., Müller, D., Eds.; CEUR Proceedings: Nize, France, 2009; Volume 506, pp. 47–58. [Google Scholar]
  23. IMS LTI. Learning Tool Interoperability. IMS LTI 1.3 and LTI advantages. Available online: http://www.imsglobal.org/activity/learning-tools-interoperability (accessed on 5 November 2020).
  24. Severance, C.; Hanss, T.; Hardin, J. Ims learning tools interoperability: Enabling a mash-up approach to teaching and learning tools. Technol. Instr. Cogn. Learn. 2010, 7, 245–262. [Google Scholar]
  25. Universidad Politècnica de Catalunya, Hechos y Cifras. Available online: https://www.upc.edu/es/la-upc/la-institucion/hechos-y-cifras (accessed on 5 November 2020).
  26. Sánchez, O.; Martinez, N.; Ribot, E. Servei ATENEA. In Jornada Dia d’Atenea 2009; Universitat Politècnica de Catalunya, Institut de Ciències de l’Educació: Barcelona, Spain, 2009. [Google Scholar]
  27. Roush, E.T.; Zoram, T. Virtual Cluster Based Upon Operating System Virtualization. U.S. Patent No. 8,200,738, 12 June 2012. [Google Scholar]
  28. Gebhart, A.; Erol, B. Dynamic Cluster Expansion through Virtualization-Based Live Cloning. U.S. Patent No. 8,887,158, 11 November 2014. [Google Scholar]
  29. Alier, M. ICE Epicentre Podcast 15: Atenea Exams. Institut de Ciencies de l’Educació-UPC, 2020. Available online: https://www.youtube.com/watch?v=XV91RjcXXr8 (accessed on 5 November 2020).
Figure 1. Moodle webservices architecture.
Figure 1. Moodle webservices architecture.
Applsci 11 00080 g001
Figure 2. External Tool (LTI) in the Moodle Atenea instance.
Figure 2. External Tool (LTI) in the Moodle Atenea instance.
Applsci 11 00080 g002
Figure 3. Launch of a LTI Tool.
Figure 3. Launch of a LTI Tool.
Applsci 11 00080 g003
Figure 4. When switching to a confinement situation, the average length of the session and the average number of pages viewed per session go up significantly.
Figure 4. When switching to a confinement situation, the average length of the session and the average number of pages viewed per session go up significantly.
Applsci 11 00080 g004
Figure 5. There is a significant raise in the number of daily sessions on the Atenea platform when confinement begins.
Figure 5. There is a significant raise in the number of daily sessions on the Atenea platform when confinement begins.
Applsci 11 00080 g005
Figure 6. Concurrent sessions in Atenea (March and April 2020) compared to the same period 2019. The arrow indicates the beginning of the confinement. Notice the scale change.
Figure 6. Concurrent sessions in Atenea (March and April 2020) compared to the same period 2019. The arrow indicates the beginning of the confinement. Notice the scale change.
Applsci 11 00080 g006
Figure 7. Concurrent sessions in Atenea platform September–November 2020, compared to the same period in 2019. Notice the scale change.
Figure 7. Concurrent sessions in Atenea platform September–November 2020, compared to the same period in 2019. Notice the scale change.
Applsci 11 00080 g007
Figure 8. Multi-instance Moodle architecture schema.
Figure 8. Multi-instance Moodle architecture schema.
Applsci 11 00080 g008
Figure 9. One single high-performance Moodle instance architecture schema.
Figure 9. One single high-performance Moodle instance architecture schema.
Applsci 11 00080 g009
Figure 10. Overview of the Atenea Exams—Atenea interaction.
Figure 10. Overview of the Atenea Exams—Atenea interaction.
Applsci 11 00080 g010
Table 1. Data on faculty training activity during the confinement course 2019–2020 2ond semester. “Before Covid-19” column covers the period from 2 February 2020 to 17 February 2020, “During Covid 19” covers the period 18 February 20 to 31 July 2020. The assessment is the result of surveys presented to all participants in the ICE’s courses, with more than 60% of participation.
Table 1. Data on faculty training activity during the confinement course 2019–2020 2ond semester. “Before Covid-19” column covers the period from 2 February 2020 to 17 February 2020, “During Covid 19” covers the period 18 February 20 to 31 July 2020. The assessment is the result of surveys presented to all participants in the ICE’s courses, with more than 60% of participation.
ICE’S RESOURCESBEFORE CONFINEMENTDURING CONFINEMENT
# COURSES2379
# ENROLLMENTS5851514
# UNIQUE FACULTY ENROLLED5851514
# VIDEO PODCASTS PRODUCED016
# PODCAST’S REPRODUCTIONS8344
# WEBSITE VISITS >57,000
# COURSE EVALUATION (MEAN) 4.53/5
# ORGANIZATION EVALUATION (MEAN) 4.53/5
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Alier, M.; Casany, M.J.; Llorens, A.; Alcober, J.; Prat, J.d. Atenea Exams, an IMS LTI Application to Solve Scalability Problems: A Study Case. Appl. Sci. 2021, 11, 80. https://doi.org/10.3390/app11010080

AMA Style

Alier M, Casany MJ, Llorens A, Alcober J, Prat Jd. Atenea Exams, an IMS LTI Application to Solve Scalability Problems: A Study Case. Applied Sciences. 2021; 11(1):80. https://doi.org/10.3390/app11010080

Chicago/Turabian Style

Alier, Marc, María José Casany, Ariadna Llorens, Jesús Alcober, and Joana d’Arc Prat. 2021. "Atenea Exams, an IMS LTI Application to Solve Scalability Problems: A Study Case" Applied Sciences 11, no. 1: 80. https://doi.org/10.3390/app11010080

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop