Next Article in Journal
Identification of Heart Arrhythmias by Utilizing a Deep Learning Approach of the ECG Signals on Edge Devices
Next Article in Special Issue
A Framework for a Seamless Transformation to Online Education
Previous Article in Journal
Exploration of the Impact of Cybersecurity Awareness on Small and Medium Enterprises (SMEs) in Wales Using Intelligent Software to Combat Cybercrime
Previous Article in Special Issue
Literature Review on MOOCs on Sensory (Olfactory) Learning
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Heuristic Evaluation of Microsoft Teams as an Online Teaching Platform: An Educators’ Perspective

1
Faculty of Information Technology, Department of Software Engineering, Philadelphia University, Amman 19392, Jordan
2
School of Computer Science, University of Hull, Hull HU6 7RX, UK
*
Author to whom correspondence should be addressed.
Computers 2022, 11(12), 175; https://doi.org/10.3390/computers11120175
Submission received: 7 November 2022 / Revised: 28 November 2022 / Accepted: 2 December 2022 / Published: 4 December 2022
(This article belongs to the Special Issue Present and Future of E-Learning Technologies)

Abstract

:
The way that education is delivered changed significantly during the COVID-19 pandemic to be completely online in many countries for many institutions. Despite the fact that they are not online teaching platforms, virtual meeting platforms were utilized to deal with this transformation. One of the platforms Philadelphia University utilized for the unplanned shift to online teaching was Microsoft Teams. This paper examines how heuristic evaluation may be used to guide the evaluation of online meeting platforms for teaching and focuses on the use of heuristic evaluation to assess the level of usability of Microsoft Teams. The level of Zoom’s usability is also evaluated using heuristic evaluation in order to compare it to that of Microsoft Teams and to assess Microsoft Teams’ overall usability in comparison to other platforms being used for the same purpose. Microsoft Teams was identified as having a few issues that need to be addressed. Additionally, strengths, weaknesses, opportunities, and threats to Microsoft Teams’ usability were assessed.

1. Introduction

COVID-19, a new coronavirus disease, was identified as a ‘pandemic’ by the World Health Organization (WHO) on March 11, 2020, due to its quick global spread [1]. Following that, governments started to announce lockdowns, prompting educational institutions to suddenly take decisions on an unexpected and compulsory shift to online teaching. Academic institutions faced challenges as a result of this rapid academic revolution, as there was no prior preparation for both educators and learners, resulting in several challenges while practicing the online learning process. This complete transformation to online learning and online teaching requires a suitable evaluation that measures and reflects the quality of the entire learning and teaching experience [2]. An important aspect of this evaluation is evaluating the online meeting platforms considering how user-friendly these platforms are and how effective and efficient they are in achieving the specified goals for teaching.
Microsoft Teams was the primary platform used at Philadelphia University for the rapid shift to online teaching during COVID-19 and later. However, by all means, it was a big challenge to use this platform, not only within our university but also worldwide. This was due to the lack of knowledge in using these platforms for online teaching. As a teaching platform, it has been improving since then to match the needs of the users and to become easier to use. However, it has been reported by many lecturers that they are still facing lots of difficulties while using the platform.
Usability is a fundamental criterion for assessing e-learning technology and systems as it reflects the quality and prioritizes the users’ actual needs [3]. Therefore, evaluating the usability of the used platform and investigating its contribution to the learning process is crucial. This paper evaluates the usability of Microsoft Teams as an online teaching platform. It also examines how other platforms such as Zoom, which are used for online teaching, meet the requirements of educators. Moreover, the paper identifies several shortcomings and challenges with Microsoft Teams as an online teaching platform in comparison to Zoom as a platform that is used for the same purpose. The paper does not consider more specialized Virtual Learning Environment (VLE) platforms, which tend to focus on supporting learners generally. The problem addressed here is on some of the most adopted platforms—notably Microsoft Teams and Zoom—that were used to supplement established VLEs to provide a synchronous learning environment.

2. Background

2.1. Usability and Usability Evaluation

Usability, as explained by the International Standards Organization [4], is the extent to which a product can be utilized by specific users to accomplish specific goals in a given application context with efficiency, effectiveness, and satisfaction [5]. Moreover, [6] defined usability as a metric that measures how efficiently, effectively, and successfully a specific user can use a product or design to achieve a specified objective in a given context. Therefore, usability is a quality metric that assesses how easy it is to use a user interface, and during the design phase, “usability” refers to techniques for enhancing the ease of use [7]. In this paper, usability is defined as a quality metric that assesses how easily, effectively, and efficiently a platform can achieve a user’s goals.
Usability is important because it is one of the key factors in gaining users’ satisfaction and confidence, which is essential to the survival of platforms such as MS Teams. If users were not satisfied with the existing platform, they would look for a reasonable alternative that delivers all of the features offered by the current platform.
Thus, from the previous definitions and as explained by [8], we perceived that in studying the usability of Microsoft Teams we have three important aspects to focus on as shown in Table 1: specified users (lecturers in our case), goal (delivering an online lecture in our case), and context (teaching in our case).
Furthermore, studying the usability of Microsoft Teams should be directed to measure its effectiveness, efficiency, and the extent to which users are satisfied.
A review of the literature revealed that researchers can employ either usability testing (User Experience (UX) is another name for it) or Heuristic evaluation to examine and assess the usability of online learning platforms. Usability evaluation is the process of assessing a product or device’s usability on several levels [9]. The process, that concentrates on observing users while interacting with a product when carrying out genuine and meaningful tasks, is called usability testing [8]. Usability tests and inspection methodologies may be performed to measure and evaluate the usability of a product that has already been designed into it [10]. Moreover, in usability tests, the focus is on the potential end users and their experience while using the product. In the case of e-learning systems, individual interviews, questionnaires, online surveys, heuristic evaluations, expert reviews, remote testing, and other approaches can be utilized for evaluating their usability [11].
A well-known technique for quick evaluation of the effectiveness of new technologies and interface problems is heuristic evaluation [3,12,13]. Daniela and Rusu [10] identified heuristic evaluation as one of these approaches, which is a type of inspection that finds usability issues using usability heuristics or principles. Moreover, remote usability testing is possible when the user is at a different location (either unmoderated or moderated) [11]. Additionally, [13,14] defined heuristic evaluation as a method for usability evaluation that is informed by heuristics analysis in which several specialists in the field are required to apply their specialized knowledge to speculate on an interface solution. The ten basic concepts of interaction, designed by Jakob Nielsen [15] that were updated in November 2020, are general rules of thumb rather than particular usability requirements or guidelines, thus, they are known as “heuristics”. Therefore, using these ten heuristics while developing interfaces is regarded best practice. To determine whether a system adheres to usability standards, a system should be evaluated by three to five experts because using multiple evaluators has the potential to produce more accurate results.
Squires and Preece [16] originally proposed the use of heuristic evaluation to measure usability, quality, and potential for learning the applications with an educational focus. Albion and Benson et al. [17] also used Nielson’s heuristics and added extra heuristics to them that are related to e-learning. Moreover, [18] applied the heuristics that were developed by [16] for pedagogic applications. Even though heuristic evaluation is frequently employed in online learning and other domains, not everyone follows Nielsen’s suggested ten principles [9].
Thus, it can be seen that there are many methods for measuring usability in the literature, as without measurement, it is impossible to control usability requirements or to determine if a product has developed to meet its users’ requirements [19].
This study evaluates the usability of Microsoft Teams as a platform for online teaching using heuristic evaluation. In other words, it evaluates how well Microsoft Teams fits the usability requirements of educators in higher education in order to deliver a lecture. Additionally, it aims to evaluate how Microsoft Teams is employed and how efficient and effective it is. Considering Nielsen’s heuristics, the researchers built their technique.

2.2. Microsoft Teams

Microsoft Teams is a software that was developed by Microsoft in the Office 365 bundle. This communication platform offers file storing, chatting and video/voice conferencing, which has the potential to enable its users to perform group discussions as well as one-to-one meetings. Due to the pandemic, Microsoft Teams and some of its competitors such as Google Meet and Zoom gained much more interest and usage in the educational field.
The number of users of Microsoft Teams has increased significantly between 2019 and 2022. In 2019, the number of daily active users was 13 million [20], while in 2022 it reached more than 270 million monthly active users [21]. That increase was because of the improvement in the features that were provided by Microsoft Teams to its users as there are some features that help in enhancing the education process and virtual learning such as chatting, creating teams, conversations as groups, quizzes, assignments, and channels.

2.3. Evaluation of Microsoft Teams and Online Learning Platforms

The most popular technologies for lectures in higher education institutions recently have been Microsoft Teams and Zoom, and because these platforms were not created with education in mind at first, learning effectiveness was noticeably diminished [9]. Many studies such as [22,23] stated that during online lectures, both students and teachers reported numerous issues.
Throughout the COVID-19 pandemic and the sudden transfer to online learning, [24] conducted research in Jenin city to see how using technologies such as Microsoft Teams helped to enrich English education. The results of studying a sample of twenty-five (25) English language teachers showed that Microsoft Teams has features that enrich the interactive learning process by allowing users to share content and files, as well as screen sharing, which allows educators to present appropriate content while the class is online.
Sari and Nayir [25], on the other hand, looked at how teachers, administrators, and scholars felt about continuing online education. The data were analyzed by a working group of 65 teachers. The research revealed issues with students’ Internet access, as well as a lack of infrastructure and classroom management as a result.
Moreover, [26] assessed and contrasted the online learning tools’ usefulness using the System Usability Scale (SUS) questionnaire, which primarily focuses on efficiency, ease of use, and ease of learning. The findings of this research show that, compared to e-learning platforms and Microsoft Teams, Zoom performs better in terms of usability. Additionally, [27] combined the System Usability Scale (SUS), Human–Computer Interaction (HCI)-based technique, Technology Acceptance Model (TAM), and Information Systems (IS)-based approach to use them for the usability evaluation of Microsoft Teams. In their study, [27] assessed the efficiency of Microsoft Teams as an online learning platform in terms of how usable it is seen by students. However, educators (lecturers) are also on another side of this argument and knowing about their perspectives on the usability of online learning platforms is vital.
After reviewing the literature, we found that e-learning platforms’ usability was the subject of many studies. These studies used many heuristics for evaluating the overall experience of online learning; however, they were frequently not focused on the teaching experience while delivering an online lecture. Accordingly, there is a scarcity of studies that consider evaluating the educators’ experience while using online teaching platforms, especially the usability of Microsoft Teams as an online teaching platform. Thus, this paper utilizes Nielsen’s heuristics for the evaluation of Microsoft Teams as an online teaching platform. In order to validate our results, Zoom is also evaluated as it is a platform that is in use for the same purpose. The results of the evaluation of the two platforms are compared to obtain an overall evaluation of Microsoft Teams.

3. Evaluation Procedure and Results

The study was initially approved by the Philadelphia University Research Ethics Committee (Faculty of Information Technology). For the purpose of evaluating Microsoft Teams, the authors used Nielsen’s heuristics to evaluate both Microsoft Teams and Zoom, as indicated in Table 2. Zoom is assessed in order to compare Microsoft Teams with other platforms out there. Thus, a group of experts with proven expertise in e-learning and computer education research, as well as in teaching software engineering and computer science courses at the University level, evaluated the two platforms. Thus, the procedure served as an end-user evaluation to help in getting professional suggestions and recommendations for updating the platforms.
Any learning management system or any software that is in use as a learning management system should help in achieving key educational institutions’ goals such as delivering and tracking courses, which can be subdivided into the following sub-goals: creating a course, managing a course and delivering a course. The sub-goal “Delivering a course” can be achieved by delivering online lectures, interacting with students, and tracking students’ performance.
In order to deliver an online lecture, a lecturer is required to perform a variety of tasks. Thus, to determine the end tasks, the sub-goal of delivering a lecture is broken down into sub-goals and activities. The authors as experts developed a number of scenarios and embedded certain tasks to be carried out to evaluate the two platforms according to the same standards. Scenarios were prepared in a way that each scenario has a set of related tasks that are directed to achieve a sub-goal.
Then, scenarios that achieve the goal of “delivering an online lecture” are built, which enables us to reconsider the approach of delivering a traditional lecture. The scenarios required to deliver a virtual or online lecture are specified and created. The scenarios that are intended to achieve goals are used in order to achieve a consistent evaluation. The scenarios were given to the experts so they could complete the same tasks and assess them using the heuristics. Figure 1 below shows the process of specifying the tasks (actions) and functions that are required to achieve a goal.
The following is an example of one of the scenarios that achieves the goal: deliver an online lecture.
A 90-min online virtual lecture could be split up into different actions that are performed by the lecturer to start the lecture and to manage it. Moreover, there are a number of activities that have to be performed throughout the lecture and homework for the next lecture or that may even need to be submitted after one week.
To start an online lecture, the lecturer starts a meeting and makes sure that all his/her students are able to join before starting the online lecture. The lecturer then shares the material (such as lecture slides) and, if desired, begins to record the lecture. The lecturer may also need to share the whiteboard to demonstrate some concepts to the students. The lecturer will then give the students an assignment to complete as classwork, which must be turned in during the lecture. There is a chance that the lecturer will have to assign different tasks to various groups. The lecturer may also be required to engage with the class; for example, by encouraging a student, showing appreciation for what they said, or expressing surprise at something they observed. The lecturer may also need to record the lecture in addition to dividing the class into groups and interacting with each one separately. He or she should be able to finish the meeting after the lecture before leaving.
Then, the implementation phase started, where we began the evaluation procedure by having the experts perform tasks in each scenario and rating the platform according to the heuristics. Figure 2 demonstrates the phases of the evaluation procedure.
As shown in Table 2, in light of Nielsen’s heuristics, four experts were asked to rate the platforms (Microsoft Teams and Zoom) on a scale of 1 to 5, with 1 being the worst and 5 being the best. Where the platform does not apply, they were asked to enter N/A. The average expert evaluation is then calculated and the results are shown in Table 2 and Figure 3. For more details on the mapping of each heuristic in Table 2 to the assessment carried out by the expert evaluator, see Table A1 in the Appendix A.

4. Discussion of Results

The focus was on the activities (functions) necessary for delivering an online lecture that should be available and obvious for lecturers to finish the lecture efficiently and satisfy educators. Some of the core activities were creating a Team or initiating a meeting, scheduling a meeting, and adding members to the meeting, enabling/disabling video and audio, sharing content, recording meeting, uploading content and chatting. According to Nielsen’s heuristics, users should always be informed and provided with suitable feedback in a timely manner about what is going on. The average of experts’ evaluation for this heuristic is 3.25 out of 5 for Microsoft Teams and 4 out of 5 for Zoom. Experts stated that Zoom provides the user with appropriate information regarding the current status such as: sharing the screen, activating the whiteboard or ongoing recording. Moreover, while idle, the status of the actions conducted on Zoom is clear, for example, Zoom shows the following message: (‘The user does not have any upcoming meetings. To schedule a new meeting click Schedule a Meeting’). On the other hand, in Microsoft Teams, experts reported a few issues. For example, the status of the actions conducted on the platform is clear when interacting but while idle it is not clear. Additionally, when creating a team with the same name as an existing team, experts found that Microsoft Teams allows them to do so without alerting users that this is an existing team; currently, the user can search for the team’s name first and if it does not exist, they can create it. According to Nielsen’s heuristics, this is an action that the users should be informed about because it has consequences. Another example that is related to the same issue is that when the camera is open while the user’s screen is shared, it does not tell the user that his camera is turned on. As mentioned before, the visibility of the system status is crucial, no action that has consequences may be made without telling the user, and feedback to a user should always be given straight away, in accordance with Nielsen’s heuristics. When turning on the camera, providing the user with feedback such as a sound notification is recommended. In addition to that, when using Zoom for quizzes and polls, the host is informed of the results live, while in MS Teams, that is not the case.
The second heuristic states that the design should communicate with the user’s language, employ concepts that the user is familiar with, and present the information in a way that is both natural and logical for the lecturer to find comfortable. It should also use terms, expressions, and concepts that are well-known to educators. When searching across Microsoft Teams and Zoom, all the used words and sentences are familiar to novice and expert users, and the conventions used on the interface are understandable and can be easily recognized by the user. The recently added and updated reactions such as: smileys, raising a hand, out/break, etc., eased the communication and simulated real-world interactions. As a result, Zoom exceeded Microsoft Teams in this regard, achieving an average of 4.25 out of 5 compared to Microsoft Teams’ average of 4. For example, in this essence, MS Teams and Zoom could provide sign language interpretation features, which could be enabled by the user, so verbal and nonverbal communication are addressed on both platforms.
The third heuristic supposes that users can undo mistakes and stop unwanted actions through a clearly marked “emergency exit” available to them. On this, the experts rated Microsoft Teams with an average of 3.5 and Zoom with an average of 3.25. All experts suggested that several enhancements can still be achieved in this domain, such as modifying sent messages or undoing sending a file to a specific user or deleting a message sent to all. Therefore, Zoom and MS Teams would provide users with more than one option to be able to exit such as the cancelling, pausing or deletion of a team, member, meeting or many other items.
Regarding the consistency and matching standards heuristic, users should not have to guess whether actions, various phrases, or circumstances have the same meaning because the usual operating system standards are maintained. Similar to Zoom’s average score of 4 out of 5, Microsoft Teams here receives an average score of 4. The home page of Microsoft Teams, for instance, offers two options for the same function: “invite people to join you” AND “search participants and share invitation”, indicating a lack of consistency in this situation. Experts claim that there is a proper distinction between various actions in both platforms and no ambiguity in understanding the various phrasing. In this regard, Zoom and Microsoft Teams showed a reasonable level of consistency.
The error prevention heuristic assumes that good error messages are crucial for error prevention but better designs also deal with potential issues before they arise. Conditions that are prone to errors should be checked or eliminated and a confirmation option should be given before users agree to an action. Microsoft Teams achieved 3.25 in this regard while Zoom achieved 3.5. Experts specified that to some extent, Zoom has a good messaging system, such as: confirming exiting the meeting or confirming the acceptance of recording the meeting. However, further indications could be added such as: confirming sharing of a file with the participants. In addition, experts pointed out that in Microsoft Teams, some options with important frequently used functions were grouped next to each other in a way that could be confusing when using these objects. It was also found that the objects were placed in a way that is uncomfortable for the user; as an example of this, see Figure 4, which shows a screenshot of Microsoft Teams’ objects. For instance, the leave button is next to the share button, thus it is quite easy to leave by mistake. Additionally, there is potential for the user to unintentionally click the camera button, which may result in the camera opening without telling the user that their camera was turned on, as criticized by the first heuristic “Ensures visibility of system status”. On the other hand, in Zoom, it is less likely that the user would accidentally push the start video button without intending to do so.
The heuristic known as “supports recognition rather than recall” precepts that by making elements, actions, and alternatives visible, the design should reduce the user’s memory load. Moving from one user interface element to another should not need the user to recall or remember. Things such as menu items and field labels should be obviously visible or accessible. Zoom’s average score is not particularly impressive here. A case that justifies this relatively low score is when looking at the case of switching from the normal session to the breakout sessions, all options could be retrieved and utilized; however, after exiting the system and it shutting down, (for instance, as a result of an internet outage), all previous settings (such as chat) were lost and could not be retrieved and recovered.
Microsoft Teams on the other hand receives a relatively low average score of 2.5, which is justified by the experts through many examples. For example, when looking at Microsoft Team’s objects as in Figure 2, if the user presses the three dots, there are more than 15 options in the same dropdown menu to choose from including the meeting recording option, which may confuse the user. Returning to the “supports recognition rather than recall” heuristic, it can be observed that this is not fulfilled since Microsoft Teams’ architecture does not eliminate memory overhead. In Zoom, as in Figure 5, the user is informed that there are alternatives for the video and they may select what they want, such as choosing a background or a video filter, due to the arrow to the top right of the video button. However, with Microsoft Teams, the user must point at the camera option in order to see the options, which are, according to lecturers, insufficient and do support the current heuristic.
The seventh heuristic “Supports flexibility and efficiency”, provides that to serve both inexperienced and experienced users, employing shortcuts may accelerate interaction for expert users while maintaining novice users’ usability. Shortcuts also allow users to customize routine tasks. Microsoft Teams achieved an average score of 3.5 and Zoom achieved 3.25, which shows that there are some opportunities for both platforms to speed use.
According to the “Uses aesthetic and minimalist design” heuristic, interfaces should not contain unnecessary information that is infrequently used. Every extra piece of information that is added to an interface competes with the essential pieces and decreases their relative visibility. For this heuristic, Microsoft Teams and Zoom achieved an average score of 4 and 3.75, respectively, as both have little irrelevant information.
In terms of the “Helps users recognize and recover from errors” heuristic, which highlights the importance of error messages that are expressed in simple terms, clearly stating the issue, and offering a solution, Microsoft Teams achieved an average score of 3.5 and Zoom achieved 3.75.
The last heuristic is “Provides help and documentation” which states that in an ideal world, the system should be self-explanatory. However, it can be essential to provide documentation to ensure that users can complete their tasks. In this regard, Microsoft Teams receives an average of 4 and Zoom an average of 3.5.
Therefore, as shown in Figure 6, it is found that Microsoft Teams offers a number of strengths, including the simplicity of carrying out fundamental tasks necessary for delivering an online lecture. Furthermore, because there is help available, even tasks that are not obvious are simple to complete.
Additionally, there are many opportunities that should be taken to improve Microsoft Teams. For example, there is the opportunity to customize Microsoft Teams and modify its objects to allow the planning of a more usable, convenient and efficient interface. Moreover, it is important that Microsoft Teams provides feedback to the user for any action that is performed while delivering a lecture (during a meeting), for example, when turning the camera on. Moreover, for problems such as creating a team with a similar name to an existing team, the user should be notified. In other words, Microsoft Teams should allow more notifications which have the potential to make its use easier and more effective. Microsoft Teams can also be integrated with other applications which have the potential to automate some tasks and help in achieving ease of use such as adding students to a team.
In terms of threats to its survival and existence, satisfying users’ needs with competitors such as Zoom may lead to less satisfaction with MS Teams’ users and accordingly threaten its survival.
Finally, the lack of notifications, tasks that are not obvious to users, and the lack of support for many languages could be considered as the weaknesses that matter most.

5. Research Insights

Significant functional and pedagogical limitations were described by the experts as factors limiting their level of satisfaction with online teaching platforms as online educators. Considering experts’ suggestions for having extra features such as having more notifications could help stakeholders meet their goals smoothly and improve the interaction between educators and learners. Moreover, improving the online teaching platforms and updating them with additional features has the potential to improve the learning outcomes and the overall quality of the online educational experience.
Lecturers should relax while delivering the lectures. It is possible to discontinue using the platform if the platform does not carry out users’ actions correctly or carries out some actions without making users aware that such actions are being carried out. It is evident that platforms with all of these features can enhance the effectiveness of remote teaching and yet, incorporating more features to support them might make them even better.

6. Conclusions and Future Work

To conclude, for the purpose of reviewing and evaluating online teaching platforms, the heuristic evaluation is found to be a successful method and efficient tool that was simple to apply and relatively quick. On the other hand, heuristic evaluation was criticized for only testing the thoughts of the experts who are conducting the evaluation [3]. Such an evaluation might only indicate the preferences of the experts involved, not any actual interface flaws or concerns. In order to ensure that the participating experts were qualified to make well-informed decisions on online teaching and HCI, it was particularly important to select experienced software and computer scientists who would be capable of making reliable professional judgments about pedagogy and usability. The evaluation procedure did in fact take place based on the expertise of the experts in three universities (Philadelphia University in Jordan, the University of Hull in the United Kingdom, and RWTH Aachen University in Germany). All of the experts are already educators who are using Microsoft Teams and Zoom for teaching and accordingly they are able to reliably evaluate their efficiency. Furthermore, using pre-determined scenarios that are intended to achieve a specific goal helped in achieving a consistent evaluation and helped in directing the discussion toward the issue in question. It can be observed that Microsoft Teams is an efficient and effective tool and that it does not require many resources. Furthermore, there is a potential for educators (lecturers) to perform specific tasks easily.
We concluded that Microsoft Teams provides sufficient functionality for lecturers as an online teaching platform that can be utilized to deliver lectures within a collaborative and interactive environment. Microsoft Teams can be considered user-friendly according to our results, as it did not create any form of frustration during our experiments. However, better organization of the functionality of Microsoft teams and automation of some processes has the potential to save time and improve the teaching process. For future work, we will consider evaluating Microsoft Teams from the student’s perspective.

Author Contributions

Conceptualization, L.A.-Q. and O.A.S.S.; Methodology, L.A.-Q.; Analysis, L.A.-Q.; Validation, L.A.-Q., O.A.S.S. and N.G.; Writing—original draft preparation, L.A.-Q.; Data curation, L.A.-Q., O.A.S.S. and N.G.; Writing—original draft preparation, L.A.-Q.; Writing—review and editing, L.A.-Q.; Expert evaluation, L.A.-Q., O.A.S.S. and N.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Deanship of Scientific Research at Philadelphia University.

Institutional Review Board Statement

The study was approved by the Research Ethics Committee (Faculty of Information Technology-Philadelphia University).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

For data supporting reported results please contact [email protected].

Acknowledgments

We would also like to thank Lubna Ali, a research assistant from the RWTH Aachen University in Germany, for conducting an in-depth evaluation.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1 provides a mapping of the heuristics in the assessment carried out by the expert evaluator.
Table A1. Mapping of Nielsen’s heuristics to the actual evaluation.
Table A1. Mapping of Nielsen’s heuristics to the actual evaluation.
HeuristicsExplanation
Ensures visibility of system status Users should always be kept up to date on developments by the design, which should provide important and relevant feedback in a timely manner.
Maximizes match between the system and the real world The interface should be user-friendly. Instead of using internal jargon, utilize words, phrases, and ideas that the user is already familiar with. Present information in a natural and logical order, and observe real-world conventions.
Maximizes user control and freedom Users can undo mistakes and stop unwanted actions, while also having an “emergency exit” that is marked clearly and available to them.
Consistent and matches standardsUsers should not have to guess whether various expressions, circumstances, or actions mean the same thing. Operating system rules and standards are adhered to.
Prevents errors Since concise error messages are crucial, the best designs take care to predict problems before they occur. Before users take an action, error-prone scenarios should either be avoided, detected, or provided with a confirmation option.
Supports recognition rather than recallThe amount of memory required from the user should be reduced by making elements, options, and actions visible. When users navigate between different parts of the interface, they should not need to remember a lot of information. For example, menu options should be obvious and simple to find.
Supports flexibility and efficiency of useThe design serves both inexperienced and experienced users by using shortcuts that accelerate interactions for expert users whilst such shortcuts remain hidden from novice users. The design should allow users to customize routine actions.
Uses aesthetic and minimalist designInterface should not have unnecessary information or less frequently used functions because having such information added to an interface has the potential to reduce the visibility of core functions.
Helps users recognize, diagnose and recover from errorsExpressing error messages in simple terms where the problem is identified clearly and offering a recommendation for a fix.
Provides help and documentationThe system should be self-explanatory to enable users to carry out the tasks that they require, documentation might be needed.

References

  1. World Health Organization. WHO Timeline—COVID-19, March 2020. Available online: https://www.who.int/news-room/detail/27-04-2020-who-timeline-covid-19 (accessed on 4 September 2020).
  2. Robinson, C.C.; Hullinger, H. New benchmarks in higher education: Student engagement in online learning. J. Educ. Bus. 2008, 84, 101–109. [Google Scholar] [CrossRef]
  3. Nielsen, J.; Molich, R. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Seattle, WA, USA, 1–5 April 1990. [Google Scholar]
  4. International Organization for Standardization. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs)-Part 11: Guidance on Usability; International Organization for Standardization: Geneva, Switzerland, 1998. [Google Scholar]
  5. IEEE Brand Experience. Introduction to Web Usability and Accessibility. IEEE Brand Experience. Available online: https://brand-experience.ieee.org/resources/usability/ (accessed on 26 January 2022).
  6. Interaction Design Foundation. What Is Usability? The Interaction Design Foundation, UX Courses. 2014. Available online: https://www.interaction-design.org/literature/topics/usability (accessed on 26 January 2022).
  7. Nielsen, J. Usability 101: Introduction to Usability. Nielsen Norman Group, 3 January 2012. Available online: https://www.nngroup.com/articles/usability-101-introduction-to-usability/ (accessed on 26 January 2022).
  8. Barnum, C.M. Usability Testing Essentials: Ready, Set…Test! Morgan Kaufmann: Burlington, MA, USA, 2020. [Google Scholar]
  9. Ismail, H.; Khelifi, A.; Harous, S. A Cognitive Style Based Framework for Usability Evaluation of Online Lecturing Platforms-A Case Study on Zoom and Teams. Int. J. Eng. Pedagog. 2022, 12, 104–122. [Google Scholar] [CrossRef]
  10. Quiñones, D.; Rusu, C. How to develop usability heuristics: A systematic literature review. Comput. Stand. Interfaces 2017, 53, 89–122. [Google Scholar] [CrossRef]
  11. Usability Evaluation Methods|Usability.Gov. Usability.gov. 2019. Available online: https://www.usability.gov/how-to-and-tools/methods/usability-evaluation/index.html (accessed on 4 February 2022).
  12. Nielsen, J.; Robert, M. Usability Inspection Methods; John Wiley: New York, NY, USA, 1994; ISBN 0-471-01877-5-14. [Google Scholar]
  13. Nielsen, J. Usability inspection methods. In Proceedings of the Conference Companion on Human Factors in Computing Systems, Boston, MA, USA, 28 April 1994; pp. 413–414. [Google Scholar]
  14. Interaction Design Foundation. “What Is Heuristic Evaluation?” The Interaction Design Foundation, UX Courses. 2019. Available online: www.interaction-design.org/literature/topics/heuristic-evaluation (accessed on 27 October 2020).
  15. Nielsen, J. 10 Heuristics for User Interface Design. Nielsen Norman Group, Nielsen Norman Group, 24 April 1994. Available online: https://www.nngroup.com/articles/ten-usability-heuristics/ (accessed on 24 September 2022).
  16. Squires, D.; Preece, J. Predicting Quality in Educational Software: Evaluating for Learning, Usability and the Synergy between Them. Interact. Comput. 1999, 11, 467–483. [Google Scholar] [CrossRef]
  17. Albion, P. Heuristic evaluation of educational multimedia: From theory to practice. In Proceedings of the ASCILITE 1999: 16th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education: Responding to Diversity, Brisbane, Australia, 5–8 December 1999; pp. 9–15. [Google Scholar]
  18. Brayshaw, M.; Gordon, N.; Nganji, J.; Wen, L.; Butterfield, A. Investigating heuristic evaluation as a methodology for evaluating pedagogical software: An analysis employing three case studies. In International Conference on Learning and Collaboration Technologies; Springer: Cham, Switzerland, 2014. [Google Scholar]
  19. Jokela, T.; Koivumaa, J.; Pirkola, J.; Salminen, P.; Kantola, N. Methods for quantitative usability requirements: A case study on the development of the user interface of a mobile phone. Pers. Ubiquitous Comput. 2006, 10, 345–355. [Google Scholar] [CrossRef]
  20. Spataro, J. Microsoft Teams Reaches 13 Million Daily Active Users, Introduces 4 New Ways for Teams to Work Better Together. Microsoft 365 Blog. 2019. Available online: https://www.microsoft.com/en-us/microsoft-365/blog/2019/07/11/microsoft-teams-reaches-13-million-daily-active-users-introduces-4-new-ways-for-teams-to-work-better-together/ (accessed on 26 August 2022).
  21. Foley, M. Microsoft: Teams Now has More Than 270 Million Monthly Active Users. ZDNet. 2022. Available online: https://www.zdnet.com/article/microsoft-teams-now-has-more-than-270-million-monthly-active-users/ (accessed on 21 August 2022).
  22. Arora, A.K.; Srinivasan, R. Impact of pandemic COVID-19 on the teaching–learning process: A study of higher education teachers. Prabandhan Indian J. Manag. 2020, 13, 43–56. [Google Scholar] [CrossRef] [PubMed]
  23. Aboagye, E.; Yawson, J.A.; Appiah, K.N. COVID-19 and E-learning: The challenges of students in Tertiary Institutions. Soc. Educ. Res. 2021, 2, 1–8. [Google Scholar] [CrossRef]
  24. Bsharat, T.R.; Behak, F. The impact of Microsoft teams’ app in enhancing teaching-learning English during the Coronavirus (COVID-19) from the English teachers’ perspectives’ in Jenin city. Malays. J. Sci. Health Technol. 2020, 7. [Google Scholar] [CrossRef]
  25. Sari, T.; Nayır, F. Challenges in distance education during the (Covid-19) pandemic period. Qual. Res. Educ. 2020, 9, 328–360. [Google Scholar] [CrossRef]
  26. Abushamleh, H.; Jusoh, S. Usability Evaluation of Distance Education Tools Used in Jordanian Universities. In 2021 Innovation and New Trends in Engineering, Science and Technology Education Conference (IETSEC); IEEE: Amman, Jordan, 2021; pp. 1–5. [Google Scholar] [CrossRef]
  27. Pal, D.; Vanijja, V. Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India. Child. Youth Serv. Rev. 2020, 119, 105535. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The process of specifying the tasks (actions) and functions that are required to achieve a goal.
Figure 1. The process of specifying the tasks (actions) and functions that are required to achieve a goal.
Computers 11 00175 g001
Figure 2. Evaluation Procedure.
Figure 2. Evaluation Procedure.
Computers 11 00175 g002
Figure 3. Comparison between Microsoft Teams and Zoom according to the evaluation procedure.
Figure 3. Comparison between Microsoft Teams and Zoom according to the evaluation procedure.
Computers 11 00175 g003
Figure 4. A Screenshot of Microsoft Teams Objects.
Figure 4. A Screenshot of Microsoft Teams Objects.
Computers 11 00175 g004
Figure 5. A Screenshot of Zoom’s Objects.
Figure 5. A Screenshot of Zoom’s Objects.
Computers 11 00175 g005
Figure 6. SWOT Analysis.
Figure 6. SWOT Analysis.
Computers 11 00175 g006
Table 1. Usability Aspects.
Table 1. Usability Aspects.
Usability AttributeApplication in Online Teaching
Specified usersEducators
Effectively meeting goalDelivering online lecture
Context Online Teaching in Higher Education institutions
Table 2. Nielsen’s heuristics applied to evaluate Microsoft Teams on a scale of 1 to 5.
Table 2. Nielsen’s heuristics applied to evaluate Microsoft Teams on a scale of 1 to 5.
HeuristicsExperts’ Evaluation (Microsoft Teams)Experts’ Evaluation (Zoom)
Ensures visibility of system status45
44
23
34
Avg: 3.25Avg: 4
Maximizes match between the system and the real world 55
44
34
44
Avg: 4Avg: 4.25
Maximizes user control and freedom 44
32
44
33
Avg: 3.5Avg: 3.25
Consistent and matches standards4 4
44
44
44
Avg: 4Avg: 4
Prevents Errors44
43
23
34
Avg: 3.25Avg: 3.5
Supports recognition rather than recall34
23
34
24
Avg: 2.5Avg: 3.75
Supports flexibility and efficiency 43
33
33
44
Avg: 3.5Avg: 3.25
Uses aesthetic and minimalist design43
44
44
44
Avg: 4Avg: 3.75
Helps users recognize and recover from errors44
33
44
34
Avg: 3.5Avg: 3.75
Provides help and documentation44
44
42
44
Avg: 4Avg: 3.5
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Al-Qora’n, L.; Salem, O.A.S.; Gordon, N. Heuristic Evaluation of Microsoft Teams as an Online Teaching Platform: An Educators’ Perspective. Computers 2022, 11, 175. https://doi.org/10.3390/computers11120175

AMA Style

Al-Qora’n L, Salem OAS, Gordon N. Heuristic Evaluation of Microsoft Teams as an Online Teaching Platform: An Educators’ Perspective. Computers. 2022; 11(12):175. https://doi.org/10.3390/computers11120175

Chicago/Turabian Style

Al-Qora’n, Lamis, Omar Al Sheik Salem, and Neil Gordon. 2022. "Heuristic Evaluation of Microsoft Teams as an Online Teaching Platform: An Educators’ Perspective" Computers 11, no. 12: 175. https://doi.org/10.3390/computers11120175

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop