Next Article in Journal
Independent or Influential? Spatial-Temporal Features of Coordination Level between Urbanization Quality and Urbanization Scale in China and Its Driving Mechanism
Next Article in Special Issue
Knee Osteoarthritis Grade Does Not Correlate with Quadriceps Muscle Strength or Bone Properties of the Calcaneus in Men Aged 80 Years or More Who Can Walk Independently
Previous Article in Journal
Association between Occupational Characteristics and Overweight and Obesity among Working Korean Women: The 2010–2015 Korea National Health and Nutrition Examination Survey
Previous Article in Special Issue
External Validation of the ‘PHYT in Dementia’, a Theoretical Model Promoting Physical Activity in People with Dementia
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Heuristic Evaluation of an IoMT System for Remote Health Monitoring in Senior Care

atlanTTic Research Center, School of Telecommunications Engineering, University of Vigo, 36310 Vigo, Spain
School of Telematics, University of Colima, Colima 28040, Mexico
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(5), 1586;
Submission received: 10 January 2020 / Revised: 25 February 2020 / Accepted: 25 February 2020 / Published: 1 March 2020
(This article belongs to the Collection Aging and Public Health)


This paper presents the usability assessment of the design of an Internet of Medical Things (IoMT) system for older adults; the evaluation, using heuristics, was held early on the design process to assess potential problems with the system and was found to be an efficient method to find issues with the application design and led to significant usability improvements on the IoMT platform.

1. Introduction

The Internet of Things (IoT), defined as the interconnection of any daily use device (or among them) anywhere and anytime to the Internet [1], have penetrated several fields of healthcare [2], ranging from reminders of medication intakes [3] to the remote monitoring of patients’ vital signs [4]. Recent advances in sensors and mobile devices have led to the development of wearable devices that connect with smart phones to analyze the data obtained from people who monitor their health using them [5]. Through IoT medical technology, physicians or caregivers (responsible for the health of patients) can remotely supervise, in real time, the physical condition of patients [6], these advances have led the coining of the term Internet of Medical Things (IoMT), which is customized healthcare IoT [7].
Based on the needs of elderly people caregivers detected in a questionnaire we applied during the course of this research, we present a proposal for a real-time health monitoring IoMT system focused on supervising older adults living in geriatric residences, aimed to provide physicians and caregivers in nursing homes with a support tool.
Usability allows us to evaluate how easy it is to use a system, as well as how functionality is related to both the task and the person performing it [8]; for example, there are system’s elements perceived as very useful for a particular group of users, but for another, their use could be viewed as a catastrophe.
There are multiple tests to measure usability, one of them is the Heuristic Evaluation, which consists of an inspection evaluation that should be executed, preferably by expert evaluators based on previously established heuristic principles [9], used to catalog the findings according to the degree of the detected problem. An inspection usability evaluation consists of having evaluators inspecting the interface to find usability problems in a design [10].
The objective of this article is to present the procedure and the results of an IoMT-system early-stage heuristic evaluation of usability to determine its friendliness level for caregivers and family members.

2. Related Work

This section presents a comparison of different approaches, design context and products discussed within the context of providing health monitoring for elders using Internet of Medical Things technologies.
For example, Li et al. [11] introduces a wireless sensor network health monitoring and alarm system that monitors, stores in a database, and analyzes body health data to send warning messages when an abnormal behavior is detected.
Chandel et al. [12] propose the use of sensors embedded in commercial mobile devices for continuously monitoring healthcare or fitness on elderly, due to their attractive form factor and low power consumption.
The system proposed by Roman et al. [13] consists of an early wearable prototype for health monitoring that measures body temperature, heart rate, and fall detection to show these data on a Liquid Cristal Display (LCD) screen and send notifications to the caregiver using a Global System for Mobile communications (GSM) module.
In contrast with those systems, our proposal aimed to create a system that provides complete support for elders and their caregivers on nursing homes; Table 1 shows the advantages of our proposal in relation to the presented related works.

3. Methods

This research is using the User Centered Design (UCD) methodology [14], which is an iterative process with four stages: Specify the usage context, Specify requirements, Produce design solutions, and Evaluation.

3.1. Specify the Usage Context

To determine the users’ use context, we developed and applied a semi-structured interview (see Appendix A) to 14 caregivers in five elderly residences in the State of Colima in Mexico. We analyzed the recordings using thematic analysis to identify the characteristics that the IoMT system should include to support the healthcare monitoring of the elderly living in nursing homes. The interviews were analyzed through a comparative verification of the evidence using the MAXqda2 software (VERBI Software GmbH, Berlin, Germany) [15]. As a first step, the authors conducted a systematic analysis of the transcripts of the interviews to obtain the categories and subcategories that helped deepen the understanding of the information provided by the interviewees. Subsequently, the data was coded and analyzed through an interpretation of what the interviewees said.
Some of the most relevant findings were:
  • Remote Monitoring: Have real-time internet-based remote monitoring of elderly’s vital signs to create a digital history of these records.
  • Alert Notifications: Generate and set alerts for caregivers and family members when the IoMT system detects an unusual situation.
  • Medical Record: The system must be able to store the health records and offer a simple and secure access to them in real time.
  • Communication with Family: To increase the family members’ involvement in the elderly care, the IoMT system must have a function to send detailed reports of elders’ activity to the family.
Based on those findings, the next UCD phase started.

3.2. Specify Requirements

3.2.1. System Architecture

Figure 1 illustrates the IoMT system architecture.
The authors used the Hexiwear biometric bracelet as a wearable IoMT device for the remote monitoring and measure of heart rate, temperature, and calories of the patients wearing them [16].
While the Hexiwear bracelet includes its cloud platform called “WolkAbout IoT Platform”, which stores all the sensed information and makes it available to developers through its application programming interface (API), it is not the only available option.
Instead of using traditional computational equipment (such as a PC) to transfer the information from sensors and send the data to the cloud to process them, the use of microcontrollers such as Arduino or RaspberryPi is spreading rapidly. They have enough computing power and memory onboard to perform lightweight tasks, information processing, and direct transmission to the cloud (with an available internet connection). This paradigm is known as Edge Computing [17], since it allows analyzing relevant data almost in real-time near where the generation of the data occurs, at the edge of the internet. To read sensor data from the edge layer and send it to the smartphone, we utilized the Bluetooth Smart (BLE) through Javascript [18].

3.2.2. Software Requirements

The software functional requirements (which are considered the cornerstone in a software development project) are conditions or capabilities that must be included in the application, requested by a customer or obtained from the users’ context to solve a problem or achieve an objective [19].
Table 2 and Table 3 list the requirements that are directly linked to the IoMT app and describe its complete functionality, as well as the conditions that must be met, obtained from the interviews and the analysis of the project context.
Non-functional requirements are the requirements that express conditions that the software must meet or specific qualities of the software [19].
Table 3 presents the non-functional requirements for the IoMT app.

3.3. Produce Design Solutions

With the requirements and the architecture, we created a medium-fidelity prototype of the mobile application with a mockup software. Figure 2 shows the main wireframes of the prototype.

3.4. Evaluation

At this stage of the project, we performed a heuristic evaluation study through an expert review of the IoMT prototype to ensure that the requirements identified after the contextual study were included in an easy-to-use manner for caregivers and family members and before the development of a fully functional application.

3.4.1. Participants

A pilot test with the IoMT prototype was performed, first, by a single expert and later by five different experts. The six evaluators (five men and one woman) belong to the area of computer science and are considered experts in usability and interfaces [20]. The average age of the six experts was 36 years (min 28, max 55), and the general information of the evaluators is shown in Table 4.
The result of the heuristic evaluation is a list of the usability problems with their respective degrees of severity, from which the interface designers can prioritize and make the necessary corrections.

3.4.2. Process

Before each evaluation, the heuristics and the severity scale of the problems were discussed briefly with the evaluators, to make sure everything was clear. Each of the evaluators of the IoMT system performed the evaluation individually to avoid biases and ensure independent results.
The evaluated heuristics principles were: (1) Visibility of system status; (2) Match between system and the real world; (3) User control and freedom; (4) Consistency and standards; (5) Error prevention; (6) Recognition rather than recall; (7) Flexibility and efficiency of use; (8) Aesthetic and minimalist design; (9) Help users recognize, diagnose, and recover from errors; (10) Help and documentation. According to Nielsen [21], the values for the ratings were between 0 (Not a problem) and 5 (Catastrophe).
Each expert had an evaluation form that contained the 10 heuristics and the description of the possible problems to be found, each problem or item had a scale to evaluate its severity.

4. Results

The experts identified a total of 47 problems in the IoMT system of which 18 (38.29%) were unique problems.
Table 5 shows experts’ classification of the problem’s severity. About 80.85% of the problems were cataloged as “Can be improved”. Expert 6 identified a critical problem and no evaluator identified problems as catastrophe.
No problem was identified at the same time by all the evaluators, 18 items (38.29%) were identified as problems only by an expert. All the problems identified are distributed in all ten heuristics, that is, a particularly problematic heuristic was not found. It is important to point out that 17 items (36.17%) were not identified as problems.
Figure 3 illustrates the relationship between the heuristics and the severity of the problems found by the experts. The most frequent classification was “Can be improved” in which 38 (80.8%) of the problems were identified and “Serious problem” with 6 (12.7%) problems in that category. Two (4.2%) problems were classified as “Minor problem” and one (2.1%) as “Critical problem”.
Figure 4 shows the distribution of the problems in the heuristics. According to the evaluators, the heuristic “Recognition rather than recall” did not present problems, while the heuristics with more problems were “Match between system and the real world”, “User control and freedom”, and “Aesthetic and minimalist design” (17.02% each), followed by the heuristic “Help and documentation” with 12.76%.

5. Discussion

This study confirmed that experts validated a strong level of usability toward our IoMT system for elderly health monitoring. The literature discusses different design concepts and products to improve health monitoring for seniors, mostly living in their own homes. The main contribution of this paper is to validate the feasibility of the use of our IoMT application to support caregiving through continuous health monitoring in a nursing home environment.
The heuristic evaluation proved that it is a feasible method to evaluate the usability of a system based on sensors and mobile devices, so evaluating the IoMT helped in discovering usability problems to be solved before its official release to the elderly and caregivers in public nursing homes. Our results were in accordance with previous studies reported in the literature, such as the case studies presented by Alexandru et al. [22], which consisted of a heuristics evaluation of mobile applications suited for helping people to manage a healthy lifestyle. Furthermore, the work of Stellefson et al. [23] performed a heuristic evaluation of a system designed to help educate seniors with Chronic Obstructive Pulmonary Disease (COPD) to determine whether the prototype was successful in adhering to formal usability guidelines for this population.

6. Conclusions

This paper presented the heuristic evaluation of an IoMT platform with a total of 47 problems identified as potential difficulties when used by users.
Even though some usability problems were identified in the heuristic evaluation, this type of evaluation depends on the experience of the evaluators; thus, some problems could only be found by a real caregiver. Therefore, after addressing and resolving these findings, a user assessment with actual caregivers was performed and reported on [6].

Author Contributions

Conceptualization, P.C.S.-M.; data curation, J.C.-C., R.B.-M. and L.E.A.-R.; formal analysis, P.C.S.-M. and L.E.A.-R.; investigation, P.C.S.-M. and J.C.-C.; methodology, P.C.S.-M. and L.E.A.-R.; validation, J.C.-C., L.E.A.-R. and R.B.-M.; writing—original draft, P.C.S.-M. and J.C.-C.; writing—review and editing, R.B.-M. and L.E.A.-R. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.


The authors would like to thank the experts who participated in the heuristic evaluation and to the caregivers for the interviews. We thank Luis Durán for his help with the design of the wireframes and application of the interviews.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A


What is your name?
What is your age?
What is your occupation?
Do you have a specialty?
How long have you worked here?


What services does this nursing home offer?
With how many rooms does it count?
What is the number of residents staying in this place?
What are the requirements to receive the elderly?
What type of staff does the nursing home have?
  • Briefly describe the tasks and functions of each worker.

Contact with Relatives

What percentage of the elderly maintain contact with their relatives?
  • How is the interaction of the elderly with the family?
Are family members aware of the health of the elderly?
  • How do family members help supervise the health of the elderly?
How does the family intervene in case the elderly person needs medical attention?

Health Monitoring

How many people do you have in your care?
Do you have a care plan for the elderly?
  • What can the care plan contain?
  • Normally, what is the time interval in which elders undergo regular medical check-ups?
During a routine medical diagnosis, what are the physiological parameters that are obtained?
  • How do you measure these values?
  • Have difficulties been encountered in this process? List them.
Do you keep a record of the elder’s health status?
  • How do you do that?
  • Are there problems when carrying out the registration? List them.
What is the protocol to follow in case the elderly person needs medical attention?
What are the main challenges in the health care of an elderly person?
What are the needs of the elderly in terms of health?
What are the main diseases or conditions that the older adults treated for?


  1. Morgan, J. A Simple Explanation of “The Internet of Things”. Forbes. 2014. Available online: (accessed on 24 February 2020).
  2. Usak, M.; Kubiatko, M.; Shabbir, M.S.; Viktorovna Dudnik, O.; Jermsittiparsert, K.; Rajabion, L. Health care service delivery based on the Internet of things: A systematic and comprehensive study. Int. J. Commun. Syst. 2020, 33, e4179. [Google Scholar] [CrossRef]
  3. Santana-Mancilla, P.; Anido-Rifón, L. The technology acceptance of a TV platform for the elderly living alone or in public nursing homes. Int. J. Environ. Res. Publ. Health 2017, 14, 617. [Google Scholar] [CrossRef] [PubMed]
  4. Bansal, S.; Chowell, G.; Simonsen, L.; Vespignani, A.; Viboud, C. Big data for infectious disease surveillance and modeling. J. Infect. Dis. 2016, 214, 375–379. [Google Scholar] [CrossRef] [PubMed]
  5. Pasluosta, C.F.; Gassner, H.; Winkler, J.; Klucken, J.; Eskofier, B.M. An emerging era in the management of parkinson’s disease: Wearable technologies and the internet of things. IEEE J. Biomed. Health Inform. 2015, 19, 1873–1881. [Google Scholar] [CrossRef] [PubMed]
  6. Durán-Vega, L.A.; Santana-Mancilla, P.C.; Buenrostro-Mariscal, R.; Contreras-Castillo, J.; Anido-Rifón, L.E.; García-Ruiz, M.A.; Montesinos-López, O.A.; Estrada-González, F. An iot system for remote health monitoring in elderly adults through a wearable device and mobile application. Geriatrics 2019, 4, 34. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Manogaran, G.; Chilamkurti, N.; Hsu, C.-H. Emerging trends, issues, and challenges in Internet of Medical Things and wireless networks. Pers. Ubiquitous Comput. 2018, 22, 879–882. [Google Scholar] [CrossRef] [Green Version]
  8. Goodwin, N.C. Functionality and usability. Commun ACM 1987, 30, 229–233. [Google Scholar] [CrossRef]
  9. González, M.P.; Pascual, A.; Lorés, J. Evaluación heurística. In Introducción a la Interacción Persona-Ordenador; Asociación Interacción Persona-Ordenador: Lleida, España, 2001. [Google Scholar]
  10. Nielsen, J. Usability Inspection Methods; ACM Press: New York, NY, USA, 1994; pp. 413–414. [Google Scholar]
  11. Li, H.; Yang, C.; He, Z.; Imran, M.A.; Ahmad, W. Health Activities Monitoring and Warning System for Geriatric Daily Living in Extra Care Homes; IEEE: Piscataway, NJ, USA, 2019; pp. 386–391. [Google Scholar]
  12. Chandel, V.; Sinharay, A.; Ahmed, N.; Ghose, A. Exploiting IMU Sensors for IOT Enabled Health Monitoring; ACM Press: New York, NY, USA, 2016; pp. 21–22. [Google Scholar]
  13. Roman Richard, A.A.; Sadman, M.F.; Mim, U.H.; Rahman, I.; Rahman Zishan, M.S. Health Monitoring System for Elderly and Disabled People; IEEE: Piscataway, NJ, USA, 2019; pp. 677–681. [Google Scholar]
  14. Abras, C.; Maloney-Krichmar, D.; Preece, J. User-centered design. In Encyclopedia of Human-Computer Interaction; Bainbridge, W., Ed.; Sage Publications: Thousand Oaks, CA, USA, 2004. [Google Scholar]
  15. VERBI, G. Maxqda. Available online: (accessed on 24 February 2020).
  16. Hexiwear. IoT and Wearables Development Platform. Available online: (accessed on 24 February 2020).
  17. Shi, W.; Cao, J.; Zhang, Q.; Li, Y.; Xu, L. Edge computing: Vision and challenges. IEEE Internet Things J. 2016, 3, 637–646. [Google Scholar] [CrossRef]
  18. Jonsson, A.; Lundquist, A. Make Javascript Apps for Hexiwear 101. 2016. Available online: (accessed on 24 February 2020).
  19. Pressman, R.S.; Campos Olguín, V.; Enríquez Brito, J.; Villegas Quezada, C.; Ferro Castro, B.J. Ingeniería Del Software: Un Enfoque Práctico; Mc-GrawHill: New York, NY, USA, 2010; ISBN 978-607-15-0314-5. [Google Scholar]
  20. Nielsen, J. How to Conduct a Heuristic Evaluation. 1995. Available online: (accessed on 24 February 2020).
  21. Nielsen, J. Severity Ratings for Usability Problems. 1995. Available online: (accessed on 24 February 2020).
  22. Alexandru, A.; Ianculescu, M.; Coardos, D. Improved patient engagement in self-management of health, a key to sustainable preventative healthcare systems. In Wireless Mobile Communication and Healthcare; Perego, P., Rahmani, A.M., TaheriNejad, N., Eds.; Springer International Publishing: Cham, Switzerland, 2018; Volume 247, pp. 129–137. ISBN 978-3-319-98550-3. [Google Scholar]
  23. Stellefson, M.; Chaney, B.; Chaney, D. Heuristic evaluation of online copd respiratory therapy and education video resource center. Telemed. E Health 2014, 20, 972–976. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Internet of Medical Things (IoMT) system architecture.
Figure 1. Internet of Medical Things (IoMT) system architecture.
Ijerph 17 01586 g001
Figure 2. (a) Screen for login; (b) main screen with monitoring data and notifications; and (c) communication screen.
Figure 2. (a) Screen for login; (b) main screen with monitoring data and notifications; and (c) communication screen.
Ijerph 17 01586 g002
Figure 3. Problems identified in each heuristic classified according to their degree of severity.
Figure 3. Problems identified in each heuristic classified according to their degree of severity.
Ijerph 17 01586 g003
Figure 4. Distribution of problems in heuristics.
Figure 4. Distribution of problems in heuristics.
Ijerph 17 01586 g004
Table 1. Advantages of related works.
Table 1. Advantages of related works.
IoT Health MonitoringReal Time WarningsAvailability and Low CostNon-Invasive
Li et al. [11]XXX
Chandel et al. [12]XXX
Roman et al. [13]XXX
Our proposalXXXX
Table 2. IoMT app functional requirements.
Table 2. IoMT app functional requirements.
Functional RequirementDescription
FR-01The mobile application must have user authentication.
FR-02The mobile application should be able to show caregivers the latest data recorded by vital signs sensors.
FR-03The mobile application must allow caregivers to open a communication module to communicate with the elderly relative.
FR-04The mobile application should allow family members to open a communication module to communicate with caregivers of the elderly.
FR-05The mobile application should allow the caregiver and the relative to navigate through a calendar, in order to visualize the data generated on the selected days and with the possibility of adding reminders for those days.
FR-06The mobile application should allow caregivers to add, edit, or delete reports of diseases or illnesses.
FR-07The mobile application should allow the caregiver to access clinical information of the elderly person as required.
FR-08The mobile application should allow the relative to review information about the illnesses or diseases of the elderly.
Table 3. IoMT app non-functional requirements.
Table 3. IoMT app non-functional requirements.
Non-Functional RequirementDescription
NFR-01The device with the sensors for monitoring vital signs should have Bluetooth Smart (BLE) to send the generated data to the edge microcontroller.
NFR-02The smartphone must have a version of the Android operating system.
NFR-03The availability of the system must be permanent (100%), service level for users 24/7.
NFR-04The stored data can be consulted and updated permanently and simultaneously, without affecting the response time.
NFR-05The system must have an updateable documentation that allows carrying out maintenance operations with the least possible effort.
NFR-06The system must support concurrency of users according to the resources of the infrastructure.
Table 4. Expert evaluators characteristics.
Table 4. Expert evaluators characteristics.
1 (Pilot)Male38
Table 5. Number of problems and their classification by expert.
Table 5. Number of problems and their classification by expert.
Can be improved610875238
Minor problem0001012
Serious problem4010106
Critical problem0000011

Share and Cite

MDPI and ACS Style

Santana-Mancilla, P.C.; Anido-Rifón, L.E.; Contreras-Castillo, J.; Buenrostro-Mariscal, R. Heuristic Evaluation of an IoMT System for Remote Health Monitoring in Senior Care. Int. J. Environ. Res. Public Health 2020, 17, 1586.

AMA Style

Santana-Mancilla PC, Anido-Rifón LE, Contreras-Castillo J, Buenrostro-Mariscal R. Heuristic Evaluation of an IoMT System for Remote Health Monitoring in Senior Care. International Journal of Environmental Research and Public Health. 2020; 17(5):1586.

Chicago/Turabian Style

Santana-Mancilla, Pedro C., Luis E. Anido-Rifón, Juan Contreras-Castillo, and Raymundo Buenrostro-Mariscal. 2020. "Heuristic Evaluation of an IoMT System for Remote Health Monitoring in Senior Care" International Journal of Environmental Research and Public Health 17, no. 5: 1586.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop