The SHAPES Smart Mirror Approach for Independent Living, Healthy and Active Ageing
- Physical condition: It is common for older adults to suffer physical impairments, so in the design of the interaction, vision impairment, haptic deterioration and hearing loss have to be considered.
- Computer literacy: Older adults, in their majority, are unfamiliar with technology, so interaction with the interface and even with the use of the certain menus, buttons or other graphical user interface elements can be challenging. They have also a limited understanding of the processes and very frequently find that the interface controls are non-intuitive.
- Cognitive condition: Older adults also present different cognitive impairments, such as the reduction of the attentional control, visuospatial function and working memory. For this reason, the development of very intuitive interfaces is necessary to increase its usability for older adults.
- Interface and control design: Text size, colour, and font, use of intuitive control elements, having constant feedback on what is being done and having contextual help, constitute the basis for interface design.
- Input controls: Devices with touch screen are more intuitive for this age group; however, if there are physical limitations, voice commands might be more convenient to execute different actions, or even as text input. Another option is gaze-tracking devices, although this form of interaction is recommended primarily for users with severe motor impairment. Finally, using TV-like interfaces, such as a TV remote control, makes them comfortable when using applications that incorporate them.
- Natural language: Older users are often confused by long messages with many options. If we offer the user too many choices they may feel confused and overwhelmed in making the right choice. On the other hand, using simple descriptive language makes the interface more intuitive for them.
- Cognitive Evaluation: The employment of digital assistance and smart devices in older adults allows caregivers to monitor and support their cognitive state, facilitating the early detection of cognitive decline.
2. Previous Works
- It aims to provide a comprehensive ambient assisted living solution for older individuals seeking active and healthy ageing, and extending independent living.
- It has been developed in the context of the SHAPES project and the broader technological platform it provides. This has enabled the co-design of the solution with different stakeholders and partners with broad experience in the field.
- Special emphasis is made in the pilots and use cases described: care for older individuals with neurodegenerative diseases and physical rehabilitation at home.
- It incorporates a rich combination of digital solutions for the aforementioned purpose comprising: home sensors with wireless communications, smart gateway functionalities, videoconferencing calls, wearable devices and cameras for physical activity monitoring, sound system, voice assistant, notifications and calendar reminder.
- It provides the means for interaction between the end users and therapists and caregivers.
3. Materials and Methods
3.1. The SHAPES Digital Solutions Development Methodology
3.1.1. The Co-Creation Cycle
3.1.2. The Co-Design Cycle
3.1.3. The Co-Experimentation and Co-Deployment Cycle
3.1.4. The Co-Execution Cycle
3.1.5. The Co-Evaluation Cycle
3.2. The SHAPES Smart Mirror Hardware
3.3. The Smart Mirror Services
3.3.1. The Call Service
3.3.2. The Fall Detector
3.3.3. The Physical Activity Monitor
- Activities: The activities that the user has carried out in a 24-h period, among the following ones: asleep state, offline state, walking state and resting state.
- Calories: Calories burnt, based on the activity and intensity of the carried out activities.
- Number of steps: The number of steps taken by the user, automatically tracked.
- Heart beat: The system will automatically and repeatedly record the heart rate indicating the maximum value, the minimum value and the current value.
- Sleep quality: It considers two sleep states, namely: light sleep and deep sleep.
3.3.4. The Voice Assistant
- The audio input that is received is transcribed with the trained Spanish model and the speech recognition service. In this case, the audio input is the phrase quoted by the caller and the name of the contact with whom he/she wants to make the call.
- The intention is recognised with the intention recognition service and mapped against the intention to make a call.
- The call service is launched via the intention handling service, by triggering the command that perform a call to the contact stated in the audio input.
- The call service starts the call.
3.3.5. Calendar with Reminders
3.3.6. The Home Monitoring System
4. Results and Discussion
- Use case 2, training of orofacial musculature.
- Use case 3, video-based rehabilitation tool.
- Use case 4, wearable motion monitoring devices.
- Recruitment and retention:
- At least 80% of the target cohort (older adults) were successfully recruited into the pilot during the recruitment period.
- At least 80% of recruited participants within the target cohort remained enrolled in the pilot until the end of the study.
- Technical performance:
- No functional bug is reported by users in the last phase of the use-case pilot deployment.
- A total of 100% of functional bugs are fixed in the pre-trials phase.
- The overall technical solution assures 90% of availability.
- User engagement and acceptance:
- The overall user experience of the Phyx.io tool using the short version of the User Experience Questionnaire (UEQ-S) was classified as ‘Excellent’, ‘Good’ or ‘Above average’ based on published benchmark data.
- At least 80% of the older people under a rehabilitation routine program stick to it.
- At least one care provider/caregiver scored the Phyx.io functionalities above average rating (>68) in the System Usability Scale (SUS).
- Other indicators (examples, to be defined in a measurable manner):
- Older people: type of interaction with Phyx.io and duration.
- Caregiver/Health Care Professional: number of routines assigned/reviewed each day.
- Caregiver/Health Care Professional: number of videocalls scheduled per week.
4.1. The SHAPES Smart Mirror Platform Validation
4.2. Use Case 1: Digital Assistant for Older People with Mild Cognitive Impairment
- The temperature of the dwelling has not suffered any important variation, always in the range of 20 °C and 26 °C and humidity between the 47 %H and 52 %H.
- The movement sensors provides information about where, in the dwelling, is the person located and therefore, carrying out activities. Between 9:00 a.m. and 2:00 p.m., no movement is detected in the dwelling. The person leaves the house for work. Then, between 23:00 p.m. and 7 a.m. only the bedroom area detects movement. The person is in the bedroom either sleeping (when no movement is detected) or preparing for it. During the afternoon, the main activity is located in the living room, apart from the periods during which the person visits the kitchen for meal preparation and eating.
- Regarding the door and window sensors, the information they provide is that doors are, most of the day, open, apart from the cooking or sleep time.
4.3. Use Case 2: Training of Orofacial Musculature
4.4. Use Case 3: Video-Based Rehabilitation Tool
- Components of the application. In the first part we evaluate the components that comprise the digital solution.
- Detailed usability. This part takes into account the different interaction functionalities as well as the user interface.
- Overall system evaluation. This last part is composed of a general question about the contact with the system.
- Authenticate on the platform.
- Search something.
- Identify the patients assigned to a professional.
- Explore details of a patient.
- Explore exercises and routines existing in the system.
- Explore exercises and routines assigned to a patient.
- Sort exercises by name and module.
- Create exercises and routines.
- Assign exercises and routines to a patient.
- Delete exercises and routines.
- Delete exercises and routines assigned to a patient.
- Identify the location of the facility to which they belong.
- Edit his/her user details.
- Go to the system start.
- Close session.
4.5. Use Case 4: Wearable Motion Monitoring Devices
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- World Report on Ageing and Health; World Health Organization: Geneva, Switzerland, 2015.
- 2019 Revision of World Population Prospects; United Nations: New York, NY, USA, 2019.
- Ritchie, H.; Roser, M.; Age structure. Our World Data 2019. Available online: https://ourworldindata.org/age-structure (accessed on 30 June 2021).
- Foster, L.; Walker, A. Active and successful aging: A European policy perspective. Gerontology 2015, 55, 83–90. [Google Scholar] [CrossRef] [PubMed]
- Tkatch, R.; Musich, S.; MacLeod, S.; Kraemer, S.; Hawkins, K.; Wicker, E.R.; Armstrong, D.G. A qualitative study to examine older adults’ perceptions of health: Keys to aging successfully. Geriatr. Nurs. 2017, 38, 485–490. [Google Scholar] [CrossRef] [PubMed]
- Pérez Díaz, J.; Abellán García, A.; Aceituno Nieto, P.; Ramiro Fariñas, D. Un Perfil de las Personas Mayores en España; Informes de envejecimiento en red nº 25; Indicadores Estadísticos Básicos: Madrid, Spain, 2020; 39p. [Google Scholar]
- Chodzko-Zajko, W.J.; Proctor, D.N.; Singh, M.A.F.; Minson, C.T.; Nigg, C.R.; Salem, G.J.; Skinner, J.S. Exercise and physical activity for older adults. Med. Sci. Sport. Exerc. 2009, 41, 1510–1530. [Google Scholar] [CrossRef] [PubMed]
- Knight, E.L.; Avorn, J. Quality indicators for appropriate medication use in vulnerable elders. Ann. Intern. Med. 2001, 135, 703–710. [Google Scholar] [CrossRef] [PubMed]
- Dzierzewski, J.M.; Dautovich, N.D. Who Cares about Sleep in Older Adults? Clin. Gerontol. 2018, 41, 109–112. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Maswadi, K.; Ghani, N.B.A.; Hamid, S.B. Systematic literature review of smart home monitoring technologies based on IoT for the elderly. IEEE Access 2020, 8, 92244–92261. [Google Scholar] [CrossRef]
- Borelli, E.; Paolini, G.; Antoniazzi, F.; Barbiroli, M.; Benassi, F.; Chesani, F.; Chiari, L.; Fantini, M.; Fuschini, F.; Galassi, A.; et al. HABITAT: An IoT solution for independent elderly. Sensors 2019, 19, 1258. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Majumder, S.; Aghayi, E.; Noferesti, M.; Memarzadeh-Tehran, H.; Mondal, T.; Pang, Z.; Deen, M.J. Smart homes for elderly healthcare—Recent advances and research challenges. Sensors 2017, 17, 2496. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Sprint, G.; Cook, D.J.; Fritz, R.S.; Schmitter-Edgecombe, M. Using Smart Homes to Detect and Analyze Health Events. Computer 2016, 49, 29–37. [Google Scholar] [CrossRef]
- Madureira, P.; Cardoso, N.; Sousa, F.; Moreira, W.; Oliveira-Jr, A.; Bazzani, M.; Gouverneur, P. My-AHA: Software Platform to Promote Active and Healthy Ageing. Information 2020, 11, 438. [Google Scholar] [CrossRef]
- Majumder, S.; Mondal, T.; Deen, M.J. Wearable sensors for remote health monitoring. Sensors 2017, 17, 130. [Google Scholar] [CrossRef] [PubMed]
- Dodd, C.; Athauda, R.; Adam, M. Designing User Interfaces for the Elderly: A Systematic Literature Review. In Proceedings of the Australian Conference on Information Systems (ACIS 2017), Hobart, Australia, 4–6 December 2017. [Google Scholar]
- Wang, J.; Fu, Y.; Lou, V.; Tan, S.; Chui, E. A systematic review of factors influencing attitudes towards and intention to use the long-distance caregiving technologies for older adults. Int. J. Med. Inf. 2021, 153, 104536. [Google Scholar] [CrossRef] [PubMed]
- Di Plinio, S.; Perrucci, M.G.; Aleman, A.; Ebisch, S.J. I am Me: Brain systems integrate and segregate to establish a multidimensional sense of self. NeuroImage 2020, 205, 116284. [Google Scholar] [CrossRef] [PubMed]
- Park, H.S.; Lee, G.A.; Seo, B.K.; Billinghurst, M. User experience design for a smart-mirror-based personalized training system. Multimed. Tools Appl. 2021, 80, 31159–31181. [Google Scholar] [CrossRef]
- Njaka, A.C.; Li, N.; Li, L. Voice controlled smart mirror with multifactor authentication. In Proceedings of the 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MI, USA, 16–19 September 2018; pp. 1–8. [Google Scholar]
- Jin, K.; Deng, X.; Huang, Z.; Chen, S. Design of the smart mirror based on raspberry pi. In Proceedings of the 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Xi’an, China, 25–27 May 2018; pp. 1919–1923. [Google Scholar]
- Miotto, R.; Danieletto, M.; Scelza, J.R.; Kidd, B.A.; Dudley, J.T. Reflecting health: Smart mirrors for personalized medicine. NPJ Digit. Med. 2018, 1, 1–7. [Google Scholar] [CrossRef]
- Purohit, N.; Mane, S.; Soni, T.; Bhogle, Y.; Chauhan, G. A computer vision based smart mirror with virtual assistant. In Proceedings of the 2019 International Conference on Intelligent Computing and Control Systems (ICCS), Madurai, India, 15–17 May 2019; pp. 151–156. [Google Scholar]
- Nathan, S.S.; Sulaiman, A.; Kamarulzaman, A.A.; Tiera, F.; Berahim, M. “Brilliantreflect”: Smart mirror for smart life. Int. J. Electr. Comput. Eng. 2019, 9, 1663. [Google Scholar] [CrossRef][Green Version]
- Sun, Y.; Geng, L.; Dan, K. Design of smart mirror based on Raspberry Pi. In Proceedings of the 2018 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), Xiamen, China, 25–26 January 2018; pp. 77–80. [Google Scholar]
- Alzahrani, S.M. Development of IoT mining machine for Twitter sentiment analysis: Mining in the cloud and results on the mirror. In Proceedings of the 2018 15th Learning and Technology Conference (L&T), Jeddah, Saudi Arabia, 25–26 February 2018; pp. 86–95. [Google Scholar]
- Muneer, A.; Fati, S.M.; Fuddah, S. Smart health monitoring system using IoT based smart fitness mirror. Telkomnika 2020, 18, 317–331. [Google Scholar] [CrossRef]
- García, I.C.A.; Salmón, E.R.L.; Riega, R.V.; Padilla, A.B. Implementation and customization of a smart mirror through a facial recognition authentication and a personalized news recommendation algorithm. In Proceedings of the 2017 13th International Conference on Signal-Image Technology &Internet-Based Systems (SITIS), Jaipur, India, 4–7 December 2017; pp. 35–39. [Google Scholar]
- Dang, C.T.; Aslan, I.; Lingenfelser, F.; Baur, T.; André, E. Towards somaesthetic smarthome designs: Exploring potentials and limitations of an affective mirror. In Proceedings of the 9th International Conference on the Internet of Things, New York, NY, USA, 22–25 October 2019; pp. 1–8. [Google Scholar]
- Nadafa, R.A.; Hatturea, S.; Bonala, V.M.; Naikb, S.P. Home security against human intrusion using Raspberry Pi. Procedia Comput. Sci. 2020, 167, 1811–1820. [Google Scholar] [CrossRef]
- Mathivanan, P.; Anbarasan, G.; Sakthivel, A.; Selvam, G. Home automation using smart mirror. In Proceedings of the 2019 IEEE International Conference on System, Computation, Automation and Networking (ICSCAN), Pondicherry, India, 29–30 March 2019; pp. 1–4. [Google Scholar]
- Balaji, V.; Raja, S.K.S.; Aparna, S.; Haritha, J.; Kanya, M.L. Smart Assistance for Asperger Syndrome using Raspberry Pi. Int. J. Innov. Technol. Explor. Eng. 2019, 91, 2681–2683. [Google Scholar]
- Galčík, F.; Guniš, J.; Horvát, Š.; Krajči, S.; Krídlo, O.; Opiela, M.; Szabari, A.; Andrejková, G.; Šveda, D. Case Studies in Data Science and Internet of Things. In Proceedings of the 2019 17th International Conference on Emerging eLearning Technologies and Applications (ICETA), Starý Smokovec, Slovakia, 21–22 November 2019; pp. 23–28. [Google Scholar]
- Ikeuchi, N.; Sakai, E.; Suzuki, H. A Proposal of IoT Device Cooperation System Using Smart Mirror and Biological Information. In Proceedings of the 2020 IEEE 9th Global Conference on Consumer Electronics (GCCE), Kobe, Japan, 13–16 October 2020; pp. 246–247. [Google Scholar]
- Dongare, A.; Devale, I.; Dabadge, A.; Bachute, S.; Bhingarkar, S. A Study Based on Advancements in Smart Mirror Technology. Smart Innov. Syst. Technol. 2020, 196, 233–240. [Google Scholar] [CrossRef]
- Safa, M.; Meenakshi, K.; Saranya, G.; Geetha, G.; Karpagaselvi, S. Enhancing Smart Mirror with Human Health Care Perspective towards Intelligence in IoT Based Environment. In Proceedings of the 2020 International Conference on Power, Energy, Control and Transmission Systems (ICPECTS), Chennai, Tamil Nadu, 29–30 April 2020; pp. 1–8. [Google Scholar]
- Shreyansh, K.; Aditi Sathe, R.S. Smart mirror. Int. J. Recent Technol. Eng. (IJERT) 2019, 8, 925–929. [Google Scholar]
- Sarnin, S.S.; Akbar, A.; Mohamad, W.N.W.; Idris, A.; Fadzlina Naim, N.; Ya’acob, N. Maleficent mirror with alexa voice services as an internet of things implement using raspberry pi 3 model b. In Proceedings of the TENCON 2018–2018 IEEE Region 10 Conference, Jeju Island, Korea, 28–31 October 2018; pp. 1202–1207. [Google Scholar] [CrossRef]
- Silapasuphakornwong, P.; Uehira, K. Smart Mirror for Elderly Emotion Monitoring. In Proceedings of the 2021 IEEE 3rd Global Conference on Life Sciences and Technologies (LifeTech), Nara, Japan, 9–11 March 2021; pp. 356–359. [Google Scholar] [CrossRef]
- Jain, D.R.; Holla, K.C.S.; Rashmi, M.G.; Vanitha, P. Magic Mirror-An Intelligent Tool for Elderly People. Int. J. Recent Technol. Eng. (IJERT) 2018, 6, 15. [Google Scholar]
- Ennis, A.; Rafferty, J.; Synnott, J.; Cleland, I.; Nugent, C.; Selby, A.; McIlroy, S.; Berthelot, A.; Masci, G. A Smart Cabinet and Voice Assistant to Support Independence in Older Adults. UCAmI 2017 Ubiquitous Comput. Ambient Intell. 2017, 10586, 466–472. [Google Scholar]
- SHAPES Pan-European Pilot Campaign Plan. Available online: https://shapes2020.eu/wp-content/uploads/2020/11/D6.1-SHAPES-Pan-European-Pilot-Campaign-Plan.pdf. (accessed on 7 October 2021).
- Monitoring and Evaluating Digital Health Interventions. A Practical Guide to Conducting Research and Assessment. Available online: https://apps.who.int/iris/handle/10665/252183 (accessed on 23 September 2021).
- Bhattacharyya, O.; Mossman, K.; Gustafsson, L.; Schneider, E.C. Using Human-Centered design to build a digital health advisor for patients with complex needs: Persona and prototype development. J. Med. Internet Res. 2019, 21, e10318. [Google Scholar] [CrossRef] [PubMed][Green Version]
- SHAPES Personas and Use Cases. Available online: https://shapes2020.eu/wp-content/uploads/2020/11/D2.5-Personas-and-Use-Cases-V1.pdf (accessed on 22 October 2021).
- Srivastava, A.; Bhardwaj, S.; Saraswat, S. SCRUM model for agile methodology. In Proceedings of the 2017 International Conference on Computing, Communication and Automation (ICCCA), Noida, India, 5–6 May 2017; pp. 864–869. [Google Scholar] [CrossRef]
- World Health Organization. Action Plan for the Prevention and Control of Noncommunicable Diseases in the WHO European Region; Technical Report; World Health Organization; Regional Office for Europe: Copenhagen, Denmark, 2016. [Google Scholar]
- Jo, A.; Coronel, B.D.; Coakes, C.E.; Mainous, A.G., III. Is there a benefit to patients using wearable devices such as Fitbit or health apps on mobiles? A systematic review. Am. J. Med. 2019, 132, 1394–1400. [Google Scholar] [CrossRef] [PubMed]
- Dorado, J.; del Toro, X.; Santofimia, M.J.; Parreño, A.; Cantarero, R.; Rubio, A.; Lopez, J.C. A computer-vision-based system for at-home rheumatoid arthritis rehabilitation. Int. J. Distrib. Sens. Netw. 2019, 15, 1550147719875649. [Google Scholar] [CrossRef]
- Bazarevsky, V.; Kartynnik, Y.; Vakunov, A.; Raveendran, K.; Grundmann, M. BlazeFace: Sub-Millisecond Neural Face Detection on Mobile GPUs. arXiv 2019, arXiv:1907.05047. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Sandler, M.; Howard, A.G.; Zhu, M.; Zhmoginov, A.; Chen, L. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, (CVPR) 2018, IEEE Computer Society, Salt Lake City, UT, USA, 18–22 June 2018; pp. 4510–4520. [Google Scholar] [CrossRef][Green Version]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.E.; Fu, C.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Lecture Notes in Computer Science, Proceedings of the Computer Vision—ECCV 2016—14th European Conference, Part I, Amsterdam, The Netherlands, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer: New York, NY, USA, 2016; Volume 9905, pp. 21–37. [Google Scholar] [CrossRef][Green Version]
- Kartynnik, Y.; Ablavatski, A.; Grishchenko, I.; Grundmann, M. Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs. arXiv 2019, arXiv:1907.06724. [Google Scholar]
- Tan, M.; Le, Q.V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, CA, USA, 9–15 June 2019. [Google Scholar]
- Picorelli, A.M.A.; Pereira, L.S.M.; Pereira, D.S.; Felício, D.; Sherrington, C. Adherence to exercise programs for older people is influenced by program characteristics and personal factors: A systematic review. J. Physiother. 2014, 60, 151–156. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Zhong, B.; Yang, F. How We Watch TV Tomorrow? Viewers’ Perception Towards Interactivity on Smart TV. Int. J. Asian Bus. Inf. Manag. (IJABIM) 2018, 9, 48–63. [Google Scholar] [CrossRef]
- Vimalkumar, M.; Sharma, S.K.; Singh, J.B.; Dwivedi, Y.K. ‘Okay google, what about my privacy?’ User’s privacy perceptions and acceptance of voice based digital assistants. Comput. Hum. Behav. 2021, 120, 106763. [Google Scholar] [CrossRef]
- Fernandes, T.; Oliveira, E. Understanding consumers’ acceptance of automated technologies in service encounters: Drivers of digital voice assistants adoption. J. Bus. Res. 2021, 122, 180–191. [Google Scholar] [CrossRef]
- Valdez, A.C.; Ziefle, M. The users’ perspective on the privacy-utility trade-offs in health recommender systems. Int. J. Hum.-Comput. Stud. 2019, 121, 108–121. [Google Scholar] [CrossRef][Green Version]
- Forjaz, M.J.; Rodriguez-Blazquez, C.; Guerrero-Fernández de Alba, I.; Gimeno-Miguel, A.; Bliek-Bueno, K.; Prados-Torres, A. Application of the JA-CHRODIS Integrated Multimorbidity Care Model (IMCM) to a Case Study of Diabetes and Mental Health. Int. J. Environ. Res. Public Health 2019, 16, 5151. [Google Scholar] [CrossRef][Green Version]
- Bedell, S.E.; Jabbour, S.; Goldberg, R.; Glaser, H.; Gobble, S.; Young-Xu, Y.; Graboys, T.B.; Ravid, S. Discrepancies in the use of medications: Their extent and predictors in an outpatient practice. Arch. Intern. Med. 2000, 160, 2129–2134. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Johnson, M.J.; Williams, M.; Marshall, E.S. Adherent and nonadherent medication-taking in elderly hypertensive patients. Clin. Nurs. Res. 1999, 8, 318–335. [Google Scholar] [CrossRef] [PubMed]
- Morrissey, E.C.; Casey, M.; Glynn, L.G.; Walsh, J.C.; Molloy, G.J. Smartphone apps for improving medication adherence in hypertension: Patients’ perspectives. Patient Prefer. Adher. 2018, 12, 813. [Google Scholar] [CrossRef][Green Version]
- Huang, C.Y.; Nguyen, P.A.A.; Clinciu, D.L.; Hsu, C.K.; Lu, J.C.R.; Yang, H.C.; Wu, C.C.; Tsai, W.C.; Chou, Y.C.; Kuo, T.B.; et al. A personalized medication management platform (PMMP) to improve medication adherence: A randomized control trial. Comput. Methods Programs Biomed. 2017, 140, 275–281. [Google Scholar] [CrossRef] [PubMed]
- Orcioni, S.; Pellegrini, R.; Seepold, R.; Gaiduk, M.; Madrid, N.M.; Conti, M. Medication adherence supported by mHealth and NFC. Infor. Med. Unlocked 2021, 23, 100552. [Google Scholar] [CrossRef]
- Nicolson, P.J.; Bennell, K.L.; Dobson, F.L.; Van Ginckel, A.; Holden, M.A.; Hinman, R.S. Interventions to increase adherence to therapeutic exercise in older adults with low back pain and/or hip/knee osteoarthritis: A systematic review and meta-analysis. Br. J. Sport. Med. 2017, 51, 791–799. [Google Scholar] [CrossRef] [PubMed]
- Li, J.; Hodgson, N.; Lyons, M.M.; Chen, K.C.; Yu, F.; Gooneratne, N.S. A personalized behavioral intervention implementing mHealth technologies for older adults: A pilot feasibility study. Geriatr. Nurs. 2020, 41, 313–319. [Google Scholar] [CrossRef]
- Resnick, B.; Luisi, D.; Vogel, A. Testing the Senior Exercise Self-efficacy Project (SESEP) for use with urban dwelling minority older adults. Public Health Nurs. 2008, 25, 221–234. [Google Scholar] [CrossRef]
- Bandura, A.; Adams, N.E. Analysis of self-efficacy theory of behavioral change. Cogn. Ther. Res. 1977, 1, 287–310. [Google Scholar] [CrossRef]
- Climent-Pérez, P.; Spinsante, S.; Mihailidis, A.; Florez-Revuelta, F. A review on video-based active and assisted living technologies for automated lifelogging. Expert Syst. Appl. 2020, 139, 112847. [Google Scholar] [CrossRef]
- Nguyen, T.H.C.; Nebel, J.C.; Florez-Revuelta, F. Recognition of activities of daily living with egocentric vision: A review. Sensors 2016, 16, 72. [Google Scholar] [CrossRef] [PubMed]
- Arning, K.; Ziefle, M. “Get that camera out of my house!” conjoint measurement of preferences for video-based healthcare monitoring systems in private and public places. In Proceedings of the International Conference on Smart Homes and Health Telematics, Geneva, Switzerland, 10–12 June 2015; pp. 152–164. [Google Scholar]
- Leino-Kilpi, H.; Välimäki, M.; Dassen, T.; Gasull, M.; Lemonidou, C.; Scott, A.; Arndt, M. Privacy: A review of the literature. Int. J. Nurs. Stud. 2001, 38, 663–671. [Google Scholar] [CrossRef]
- Lewis, J.R. Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int. J. Hum.-Comput. Interact. 2002, 14, 463–488. [Google Scholar]
- Bangor, A.; Kortum, P.T.; Miller, J.T. An empirical evaluation of the system usability scale. Int. J. Hum.-Interact. 2008, 24, 574–594. [Google Scholar] [CrossRef]
- Martins, A.I.; Queirós, A.; Silva, A.G.; Rocha, N.P. ICF based Usability Scale: Evaluating usability according to the evaluators’ perspective about the users’ performance. In Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion, Vila Real, Portugal, 1–3 December 2016; pp. 378–383. [Google Scholar]
- Martins, A.I.; Queirós, A.; Cerqueira, M.; Rocha, N.; Teixeira, A. The international classification of functioning, disability and health as a conceptual model for the evaluation of environmental factors. Procedia Comput. Sci. 2012, 14, 293–300. [Google Scholar] [CrossRef][Green Version]
- Martins, A.I.; Queirós, A.; Rocha, N.P. Validation of a usability assessment instrument according to the evaluators’ perspective about the users’ performance. Univers. Access Inf. Soc. 2020, 19, 515–525. [Google Scholar] [CrossRef]
- Martins, A.I.; Rosa, A.F.; Queirós, A.; Silva, A.; Rocha, N.P. Definition and validation of the ICF–usability scale. Procedia Comput. Sci. 2015, 67, 132–139. [Google Scholar] [CrossRef][Green Version]
|||Smart mirror||Fitness and health||Kinect camera to capture the user’s movements. The system generates an avatar of the user’s body and a contour shape that indicates the correct exercise position. Interaction with the mirror is through gestures.|
|||Smart mirror||General||A Raspberry Pi device with a camera and a microphone provide multimedia services while ensuring high-level security throughout the system. Facial recognition system for authentication and voice recognition for interaction. For traffic, news or weather information, Amazon Alexa Voice Service is used.|
|||Smart mirror||General||A Raspberry Pi device with a camera, a microphone and a speaker offer a smart mirror solution for smart home. Voice recognition and facial recognition techniques for authentication and interaction. Touch interaction through an infrared frame. Automatic wake-up module through an infrared induction module.|
|||Smart mirror||Healthcare||A Rapsberry Pi device with a camera for facial recognition with a tool based on python, OpenCV and deep learning. Temperature, humidity, pressure, noise and light in the room is measured through a microcontroller with LoRa and Bluetooth wireless transceiver. Mood detection with Microsoft Azure Emotion API, Calendar with Python and CalendarLabs API, Weather with Yahoo Weather API and Location with Google Maps API.|
|||Smart mirror||General||Smart mirror with virtual assistant to interact with the lighting in the house and provide different information using Alexa. This system is composed of a Raspberry Pi with web cam with microphone and speakers. Motion detection, and face recognition are other characteristics.|
|||Smart mirror||General||Smart mirror as information panel with clock, date, weather and traffics, alarm clock and daily reminder using Todoist Application and holiday calendar. A Raspberry Pi is the basis of this system.|
|||Smart mirror||General||Comprises a Raspberry Pi, a display module, a wireless transceiver module, a clock module, a Bluetooth module, a speech synthesis module and auxiliary function module. Information about temperature, weather, date, time, news and other information.|
|||Smart mirror||Social network||Display with the results of the sentiment analysis using Twitter on Raspberry Pi.|
|||Smart mirror||Health||This health fitness system has different sensors are used such as DHT11 room temperature sensor, ultrasonic sensor, PIR motion sensor, IR temperature body, a weight foot scale to obtain user data. A USB camera connected to the Raspberry Pi of the smart mirror allows for facial recognition. BIA (Bioelectrical Impedance Analysis) history, BMI (Body Mass Index) analysis, weight history and body temperature are some of the data shown on the display and in the Android Application.|
|||Smart mirror||General||The functionalities offered are date and time, weather information, personalized news, user’s mail, user’s calendar, music, facial recognition, speech recognition and text-to-speech. An Android application is responsible for mirror access through facial recognition using Microsoft Azure. The information can be obtained by using services such as Dark Sky, RSS of the newspaper Perú 21, Google Calendar and Gmail.|
|||Smart mirror||Healthcare and psychology||User tracking with respect to the affective state recognition from facial expressions. Ambient light change that gives feedback on the user’s emotional state.|
|||Smart mirror||Home security||Smart Mirror using DHT 22 sensors and image processing techniques to detect human intrusion like Yolo and Haar cascade classifier of the OpenCV. A Raspberry Pi and a camera to provide the latest news, weather information with touch based control or mobile based control.|
|||Smart mirror||Home automation||The proposed system has a Raspberry Pi with camera and microphone. An ESP8266 is in charge of the home control. Facial Recognition and Biometric Identification are other functionalities. All the information collected by the mirror can be accessed via web.|
|||Smart mirror||Smart assistance and General||Through a Raspberry pi and a Camera, facial recognition is performed to load the daily activities that correspond to that identified person. Improvements such as voice control and other functionalities are proposed for the future.|
|||Smart mirror||General||Project with a Raspberry pi a camera and microcontroller with different sensors like fingerprint sensor, distance sensor, motion sensor, RFID reader and some other components. Protocols such as MQTT and Node-Red are used for the processing and visualisation of sensor data.|
|||Smart mirror||Health||The proposed smart mirror is composed of a Raspberry Pi, a camera and speakers and a wristband. User detection employs Actcast. The Fitbit APi and wristband are used to acquire the user’s biometrical information.|
|||Smart mirror||General||Raspberry Pi, 4K LCD Screen and RGB camera on top. The functionalities of the proposed system are portrait log, gesture and speech UI, life rhythm visualisation, touch control module, automatic wake-up module, user registration using Open CV and authentication and emotion detection.|
|||Smart mirror||Health||Raspberry Pi and web camera for a mirror with facial recognition, emotion recognition and healthcare functionalities. General information like weather, to-do list and clock are the other functionalities of this proposal system. BMI and health data in the mirror using Fitbit app.|
|||Smart mirror||General||A Rapsberry Pi, microphone, speakers and proximity sensors are the main elements of this design. Interaction through sensors and voice.The different modules it contains are notice, newsfeed, update notification, weather, schedule, status, MQTT events.|
|||Smart mirror||General||Raspberry Pi, microphone and speakers use ALEXA for voice interaction. The features of this design are date, time, weather, greetings and voice services.|
|||Smart mirror||Emotional and psychology||A mirror composed of a camera, LED lamps, speakers, microphone and an IoT board. The purposes of this mirrors are the communication with user through a chat bot, estimation of long-term depression through facial recognition, evaluation of conversation mood & tone through speech recognition and behavior identification through pose recognition.|
|||Smart mirror||General, home automation||A Raspberry Pi, a camera, a microphone and a speaker are included in this mirror. The functionalities offered are face recognition, home automation and voice activation and control subsystem using Amazon services and Alexa skills. It also offers the latest news, calendar, weather forecast, clock, calendar and updates.|
|||Smart cabinet||Ambient assisted living||This smart cabinet consists of a smart mirror and a medication sensing platform. This solution is defined as Smart Home in a Box (SHIB). Using amazon services, it is intended that this cabinet works as a medication store for the elderly, providing interaction through voice commands.|
|Services of the SHAPES Smart Mirror|
|✓ ||✓ [20,21,28,35]||✓ |
|-||-||✓ [27,34,36]||-||-||✓ ||✓ [22,27]|
|[31,40]||-||-||-||✓ ||-||✓ ||-|
|Home Monitoring||Presence sensors, window|
and door sensors, temperature
|Periodic activity recognition|
Early detection of behavioral changing
|Call Service||RFID, microphone, camera||VideoCalls||Easy contact with relatives and healthcare|
staff using a video-conference system
|Fall Detector||IMU sensor||Fall detection alarm||To detect and reduce time for being attended|
under fall events
|Smartband||Activity Report||To monitor long-term activity|
|Depth Camera||Physical Routine feedback||To assist older people in its rehabilitation|
|Depth Camera||Rehabilitation Report||To assist physiotherapist on patient|
|Depth Camera||Orofacial exercise guide||Orofacial rehabilitation|
|Voice Assistant||Microphone||Voice interactions||Easy management of smart mirror services|
|Calendar||Event entries||Reminders (Physical activity,|
medication and appointments)
|To improve adherence to medication|
and physical activity
|Login Service||User credentials, RFID||User sessions log||Grant access to the platform and user|
|Pilot Campaign Phases|
|Phase 1: Plan, Design|
|Scenarios to validate initial|
concepts and approaches
|Phase 2: Mock-up or|
to assess user acceptance and
|Phase 3: Hands-on|
|Hand-on Experiments to|
validate functional elements
and gather user feedback
|Phase 4: Deployment|
in controlled environment
|Experimenting with a single|
SHAPES digital solution up
to demonstrating (part of)
the platform in a controlled
|Phase 5: Deployment|
in real-life use cases
|Demonstrations in real-life|
conditions involving the
|UR-01||USER1: Main persona, older individuals who live alone and wants to|
|UR-02||USER2: Therapist or caregiver, the person who supervises the state of the|
|UR-03||AIM1: Gather information about the physical state of a person measured|
in terms of his/her activity (number of steps, burnt calories, sleep hours
and quality, etc.)
|UR-04||AIM2: Track the evolution of such parameters.|
|UR-05||AIM3: Provide users with feedback about their daily performance|
in terms of number of steps, burnt calories, slept hours, etc.
|UR-06||AIM4: Improve physical condition as result of having a more active|
|UR-07||AIM5: Have the tranquility of having the therapist or caregiver|
supervising the evolution of the different parameters.
|UR-08||HOW1: The system will use a wearable band to track such parameters.|
|UR-09||HOW2: The system will put all the collected data in a temporal|
|UR-10||HOW3: The system will visualize that information using graphics|
and statistics to help their interpretation.
|UR-11||MEASURE1: Is the user more aware about his/her physical activity?|
|UR-12||GOAL1: Improve the physical activity based on having a more active|
|UR-01||USER1: Main persona, older individuals who are experiencing loss|
of strength of orofacial or body musculature either due to the
degenerative process associated with age or due to an accident or
health event such as a stroke.
|UR-02||USER2: Therapist, the person that supervises the rehabilitation process.|
|UR-03||AIM1: Provide a set of scheduled routines, with prescribed|
exercises that have to be performed following the instructions
of an avatar which, when deviations from the baseline
exercise occur, will provide instructions so as to correct the postures.
|UR-04||AIM2: Track the realization of the routine in order to collect|
data about performance, time, number of corrections, etc.
|UR-05||AIM3: Provide users, therapist and caregivers feedback about|
the engagement to the rehabilitation plan as well as performance.
|UR-06||AIM4: Improve physical condition of the orofacial musculature.|
|UR-07||AIM5: Feel that the therapist is nearby, supporting the rehabilitation|
process, in the same way as though the user were at the clinic
where the therapist provides support.
|UR-08||HOW1: The system will guide the user through the realization of the|
different routines comprising the rehabilitation plan.
|UR-09||HOW2: The system provides easy to interpret graphics|
|UR-10||HOW3: The system provides a video-call system for a direct|
contact with the therapist when doubts or need for support arise.
|UR-11||MEASURE1: Does the user feel his/her physical state or|
orofacial musculature is improving or maintaining?
|UR-12||GOAL1: Improve the physical state or orofacial musculature|
of the user.
|Energy consumption||High||10 mW||1 mW||100 mW|
|Range||1000 m||10 m||30 m||100 m|
|Scalability (number of nodes)||32||20||<6000||6000|
|Interoperability||WiFi Comptabible devices||Bluethooth compatible devcies||Diferent Manufacturers||Same manufacturer|
|magic-mirror-2||Service in charge of managing the smart mirror interface||1.08%||8.38%|
|Miband-dc||Mi Band data collection service||1.34%||3.26%|
|Fall-detector||Service for data collection and fall detection by means of the MetaMotionR sensor||1.11%||8.79%|
|Components of the application|
|Top navigation tabs||60%||40%||-0.9|
|Bottom navigation tabs||0%||100%||1.4|
|Icons intrisic meaning||0%||100%||1.1|
|Overall system evaluation|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chaparro, J.D.; Ruiz, J.F.-B.; Romero, M.J.S.; Peño, C.B.; Irurtia, L.U.; Perea, M.G.; Garcia, X.d.T.; Molina, F.J.V.; Grigoleit, S.; Lopez, J.C. The SHAPES Smart Mirror Approach for Independent Living, Healthy and Active Ageing. Sensors 2021, 21, 7938. https://doi.org/10.3390/s21237938
Chaparro JD, Ruiz JF-B, Romero MJS, Peño CB, Irurtia LU, Perea MG, Garcia XdT, Molina FJV, Grigoleit S, Lopez JC. The SHAPES Smart Mirror Approach for Independent Living, Healthy and Active Ageing. Sensors. 2021; 21(23):7938. https://doi.org/10.3390/s21237938Chicago/Turabian Style
Chaparro, Javier Dorado, Jesus Fernandez-Bermejo Ruiz, Maria J. Santofimia Romero, Cristina Bolaños Peño, Luis Unzueta Irurtia, Meritxell Garcia Perea, Xavier del Toro Garcia, Felix J. Villanueva Molina, Sonja Grigoleit, and Juan C. Lopez. 2021. "The SHAPES Smart Mirror Approach for Independent Living, Healthy and Active Ageing" Sensors 21, no. 23: 7938. https://doi.org/10.3390/s21237938