Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System
Abstract
:Featured Application
Abstract
1. Introduction
- To design and develop a multisensory system suitable for the objective and automatic assessment of functional capacity and determination of dependency status.
- To validate the system in clinical settings, ensuring its reliability, sensitivity, and specificity in assessment of functionality during the performance of ADLs.
- To provide a practical guide for the development of a multisensory automatic assessment system and the process to clinically validate it.
2. Protocol Description
2.1. Development Team and Collaborative Framework
2.2. Selection of Activities of Daily Living (ADLs)
- In the first stage, the complete list of all available ADLs will be carefully analyzed, using as a reference the classification of activities from the Assessment Motor and Process Skills (AMPS) [12], as it is an observational assessment tool that evaluates the quality of performance in ADLs and has a wide range of activities classified according to their difficulty of execution. The research group of clinicians, particularly the occupational therapists, will make a selection based on the most complete activities. Consideration will be given to the number of performance components and functional requirements they contain, their feasibility based on environmental characteristics, their level of difficulty and physical effort, their efficiency and safety, and the need for assistance to ensure a correct and complete evaluation of functional capacities. The selection strategy will consider the following criteria:
- Allow for the most objective evaluation of the subject regardless of gender.
- Be capable of evaluating performance with the greatest possible independence from the subject’s pathology or physical or psychological impairment.
- Be a known activity for the largest possible population.
- Be complementary activities to each other in order to observe the most limiting components of performance.
The list of selected activities will be fully detailed, considering their movement characteristics, cognitive processing of the action, and performance requirements. - In the second stage, the selected ADLs will be examined by the team of medical specialists to determine if limitations and characteristic signs of dependence could be detected after the performance and observation of these activities.
- In the third stage, a group of clinical experts in occupational therapy, external to the research team will independently score the activities according to the following: (i) the suitability of the activity to measure the desired parameters; (ii) the appearance of relevant functionality parameters. This scoring will be based on observations made in a population with dependence when performing the selected activities.The three previous phases will conclude with the selection of the final ADLs, which must complement each other and serve as the basis for the development of the EYEFUL system.
- In the final fourth stage, the clinical research team, along with the engineering team, will evaluate the technical implications of the final selection of ADLs, in what respect to both suitable sensor configurations, and the requirements of parameters to be measured. The engineering team will provide information on the current and foreseeable capabilities of automatic systems to successfully analyze the available activities. From this coordinated effort, the engineering and clinical experts will also propose the required modifications needed to shape the final activities. This will allow for an early integration of the technical and clinical perspectives in the methodological design.
2.3. Definition of the Sensorization, Computing, and Communication Configuration for ADL Monitoring
2.4. Definition of Quantitative Metrics and Their Relationship with Measurable Functional Parameters
- Metrics related to physical aspects such as body position, balance, speed of movement, bimanual coordination, etc.
- Metrics related to interaction with objects such as fine coordination, object manipulation, hand–eye coordination, etc.
- Metrics related to cognitive aspects such as problem solving, sequencing, following instructions, etc.
2.5. Definition of Relevant Inferences
2.6. Design of the Technical and Clinical Validation Processes
- Technical validation: A fully detailed technical validation will be designed, in which the performance and correct functioning of the systems related to both the characterization of subjects’ activity and the estimation of evaluation values will be assessed. A population over 18 years old will be selected. For the initial evaluations, the subjects will not have any pathology or functional limitation, fulfilling the strict requirements of data privacy and informed consent. These subjects will perform the selected activities for the automatic systems training, verification, and analytical validation of EYEFUL [30].
- Clinical validation: A comprehensive clinical validation will be designed with a focus on determining if the final system meets sensitivity and specificity criteria to detect the necessary functional limitations to discriminate a situation of dependence [30,31,32]. As a guide to the clinical validation of a system such as EYEFUL, the same processes and clinical validation of traditional tests, tools, and measurement instruments will be followed [30]. The steps involved in the clinical validation process are reproducibility analysis, content validity analysis, concurrent validity analysis, and discriminant validity analysis [30,31,32].
2.7. Algorithmic Development and Technical Validation
- The first one by integrating them into the process of assessing functionality. So, from the catalog of defined elements, we will select the most discriminative subset in order to score the functional level. Machine learning techniques are to be used in the selection of those metrics in a data driven approach. The close coordination with the clinical team will again be necessary in this effort.
- The second one, once the extracted metrics form different sensors are validated, their fusion is to be considered to improve the overall estimations (related to the users’ location and pose, activity characterization and assessment, emotional state characterization, etc.).
2.8. Deployment and Clinical Validation
- Be over 18 years of age.
- Have no recognized degree of dependence.
- Accept and sign the informed consent form.
- To be over 18 years of age.
- To have a recognized degree of dependence (from degree I, level 1 to degree II, level 2).
- To be able to walk independently, with or without support products (canes, crutches, walkers, etc.)
- To accept and sign the informed consent form.
- Having a cognitive, physical, or sensory deficit that impedes the understanding of questionnaires or the performance of the tests.
- Wheelchair users.
- Reliability analysis will examine the degree to which EYEFUL measures without error, and internal consistency will evaluate whether the metrics measuring dependence are homogeneous with each other.
- Content validity will evaluate whether EYEFUL covers all the metrics it is intended to measure.
- Criterion validity will seek to establish the degree of correlation between the scores obtained with EYEFUL and those obtained from the assessment tools established as the gold standard. This will require a sample of 45 subjects without dependence.
- Finally, for the analysis of construct validity, the degree of EYEFUL’s ability to distinguish between the population without dependence and the population with dependence will be determined. This will require a sample of 90 subjects, 45 non-dependent and 45 dependent, and the resulting EYEFUL scores of both groups will be used. These subjects will follow the inclusion and exclusion criteria described above. To determine the appropriate sample size for the clinical validity phases, the G*Power software (version 3.1.7) was used [31,32].
2.9. Dissemination of Results
3. Materials
4. Organizational Details and Reproducibility
4.1. Criteria for Choosing ADLs
4.2. Flowcharts and Documentation
5. Discussion
5.1. Implications for Clinical Practice
5.2. Contributions to Research
5.3. Addressing Limitations and Future Directions
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Acknowledgments
Conflicts of Interest
References
- Kekäläinen, T.; Luchetti, M.; Sutin, A.; Terracciano, A. Functional Capacity and Difficulties in Activities of Daily Living from a Cross-National Perspective. J. Aging Health. 2023, 35, 356–369. [Google Scholar] [CrossRef] [PubMed]
- Dias, E.N.; da Silva, J.V.; Pais-Ribeiro, J.L.; Martins, T. Validation of the advanced activities of daily living scale. Geriatr. Nurs. 2019, 40, 7–12. [Google Scholar] [CrossRef] [PubMed]
- Dias, E.G.; Duarte, Y.A.D.O.; Almeida, M.H.M.D.; Lebrão, M.L. As Atividades avançadas de vida diária como componente da avaliação funcional do idoso. Rev. Ter. Ocup. Univ. São Paulo 2014, 25, 225–232. [Google Scholar] [CrossRef]
- Romero Ayuso, D.M. Actividades de la Vida Diaria. An. Psicol. 2007, 23, 264–271. [Google Scholar]
- Cook, D.J.; Schmitter-Edgecombe, M.; Jonsson, L.; Morant, A.V. Technology-Enabled Assessment of Functional Health. IEEE Rev. Biomed. Eng. 2019, 12, 319–332. [Google Scholar] [CrossRef] [PubMed]
- González-Rodríguez, R. El análisis de la actividad en el proceso de valoración de la situación de dependencia. Rev. Electrónica Ter. Ocup. Galicia TOG 2015, 7, 66–86. [Google Scholar]
- Ley 39/2006, de 14 de Diciembre, de Promoción de la Autonomía Personal y Atención a las Personas en Situación de Dependencia. Boletín Oficial del Estado, Número 299, 15 de Diciembre de 2006. Available online: https://www.boe.es/eli/es/l/2006/12/14/39/con (accessed on 15 April 2024).
- Real Decreto 174/2011 por el que se Aprueba el Baremo de Valoración de la Situación de Dependencia Establecido por la Ley 39/2006, de 14 de Diciembre, de Promoción de la Autonomía Personal y Atención a las Personas en Situación de Dependencia. (Boletín Oficial del Estado, Número 42, 18 de Febrero de 2011). Available online: https://www.boe.es/eli/es/rd/2011/02/11/174 (accessed on 15 April 2024).
- Jiménez, S.; Viola, A. Observatorio de la Dependencia: Cuarto Informe; Estudios Sobre la Economía Española no. 2022-16; Fedea: Madrid, Spain, 2022. [Google Scholar]
- Cohen, M.E.; Marino, R.J. The tools of disability outcomes research functional status measures. Arch. Phys. Med. Rehabil. 2000, 81 (Suppl. S2), S21–S29. [Google Scholar] [CrossRef] [PubMed]
- Giambelluca, E.; Panigazzi, M.; Saade, A.; Imbriani, M. Assessment of functional status and rehabilitative strategies in occupational therapy: Role of the Groningen Activity Restriction Questionnaire. G. Ital. Med. Lav. Ergon. 2019, 41, 52–57. [Google Scholar] [PubMed]
- Bray, K.; Fisher, A.G.; Duran, L. The Validity of Adding New Tasks to the Assessment of Motor and Process Skills. Am. J. Occup. Ther. 2001, 55, 409–415. [Google Scholar] [CrossRef]
- Mcalister, C.; Schmitter-Edgecombe, M.; Lamb, R. Examination of Variables That May Affect the Relationship Between Cognition and Functional Status in Individuals with Mild Cognitive Impairment: A Meta-Analysis. Arch. Clin. Neuropsychol. 2016, 31, 123–147. [Google Scholar] [CrossRef]
- Fisher, A.G.; Griswold, L.A.; Munkholm, M.; Kottorp, A. Evaluating domains of everyday functioning in people with developmental disabilities. Scand. J. Occup. Ther. 2017, 24, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Lussier, M.; Adam, S.; Chikhaoui, B.; Consel, C.; Gagnon, M.; Gilbert, B.; Giroux, S.; Guay, M.; Hudon, C.; Imbeault, H.; et al. Smart Home Technology: A New Approach for Performance Measurements of Activities of Daily Living and Prediction of Mild Cognitive Impairment in Older Adults. J. Alzheimers. Dis. 2019, 68, 85–96. [Google Scholar] [CrossRef]
- Nef, T.; Urwyler, P.; Büchler, M.; Tarnanas, I.; Stucki, R.; Cazzoli, D.; Müri, R.; Mosimann, U. Evaluation of Three State-of-the-Art Classifiers for Recognition of Activities of Daily Living from Smart Home Ambient Data. Sensors 2015, 15, 11725–11740. [Google Scholar] [CrossRef] [PubMed]
- Alberdi Aramendi, A.; Weakley, A.; Aztiria Goenaga, A.; Schmitter-Edgecombe, M.; Cook, D.J. Automatic assessment of functional health decline in older adults based on smart home data. J. Biomed. Inform. 2018, 81, 119–130. [Google Scholar] [CrossRef] [PubMed]
- Jekel, K.; Damian, M.; Storf, H.; Hausner, L.; Frölich, L. Development of a Proxy-Free Objective Assessment Tool of Instrumental Activities of Daily Living in Mild Cognitive Impairment Using Smart Home Technologies. J. Alzheimers Dis. 2016, 52, 509–517. [Google Scholar] [CrossRef] [PubMed]
- Suzuki, Y.; Teruya, K.; Mochizuki, H.; Nagasawa, A.; Kondo, T.; Shimoda, N. Evaluation of Activities of Daily Living/Instrumental Activities of Daily Living to Accurately Determine Severity of Moderate and Severe Alzheimer’s Disease: Comparison of Assessments by Receiver Operating Characteristic Curve and Discriminant Analyses. Dement. Geriatr. Cogn. Dis. Extra. 2019, 9, 227–235. [Google Scholar] [CrossRef] [PubMed]
- Nielsen, L.M.; Kirkegaard, H.; Østergaard, L.G.; Bovbjerg, K.; Breinholt, K.; Maribo, T. Comparison of self-reported and performance-based measures of functional ability in elderly patients in an emergency department: Implications for selection of clinical outcome measures. BMC Geriatr. 2016, 16, 199. [Google Scholar] [CrossRef] [PubMed]
- Lago, P.; Roncancio, C.; Jiménez-Guarín, C. Learning and Managing Context Enriched Behavior Patterns in Smart Homes. Future Gener. Comput. Syst. 2019, 91, 191–205. [Google Scholar] [CrossRef]
- Niknejad, N.; Ismail, W.B.; Mardani, A.; Liao, H.; Ghani, I. A comprehensive overview of smart wearables: The state of the art literature, recent advances, and future challenges. Eng. Appl. Artif. Intell. 2020, 90, 103529. [Google Scholar] [CrossRef]
- Noor, M.H.M.; Salcic, Z.; Wang, K.I. Ontology-based sensor fusion activity recognition. J. Ambient. Intell. Humaniz. Comput. 2020, 11, 3073–3087. [Google Scholar] [CrossRef]
- Dawadi, P.N.; Cook, D.J.; Schmitter-Edgecombe, M. Automated Cognitive Health Assessment from Smart Home-Based Behavior Data. IEEE J. Biomed. Health Inform. 2016, 20, 1188–1194. [Google Scholar] [CrossRef] [PubMed]
- Kour, N.; Sunanda Arora, S. Computer-Vision Based Diagnosis of Parkinson’s Disease via Gait: A Survey. IEEE Access 2019, 7, 156620–156645. [Google Scholar] [CrossRef]
- Munro, J.; Damen, D. Multi-modal domain adaptation for fine-grained action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 122–132. [Google Scholar]
- Nguyen, T.H.; Nebel, J.C.; Florez-Revuelta, F. Recognition of Activities of Daily Living with Egocentric Vision: A Review. Sensors 2016, 16, 72. [Google Scholar] [CrossRef] [PubMed]
- Morgan, C.; Craddock, I.; Tonkin, E.L.; Kinnunen, K.M.; McNaney, R.; Whitehouse, S.; Mirmehdi, M.; Heidarivincheh, F.; McConville, R.; Carey, J.; et al. Protocol for PD SENSORS: Parkinson’s Disease Symptom Evaluation in a Naturalistic Setting producing Outcome measuRes using SPHERE technology. An observational feasibility study of multi-modal multi-sensor technology to measure symptoms and activities of daily living in Parkinson’s disease. BMJ Open 2020, 10, e041303. [Google Scholar] [CrossRef] [PubMed]
- Park, S.; Fisher, A.G.; Velozo, C.A. Using the assessment of motor and process skills to compare occupational performance between clinic and home settings. Am. J. Occup. Ther. 1994, 48, 697–709. [Google Scholar] [CrossRef] [PubMed]
- Goldsack, J.C.; Coravos, A.; Bakker, J.P.; Bent, B.; Dowling, A.V.; Fitzer-Attas, C.; Dunn, J.; Godfrey, A.; Godino, J.G.; Gujar, N.; et al. Verification, analytical validation, and clinical validation (V3): The foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). NPJ Digit. Med. 2020, 3, 55. [Google Scholar] [CrossRef] [PubMed]
- Carvajal, A.; Centeno, C.; Watson, R.; Martínez, M.; Sanz Rubiales, Á. How is an instrument for measuring health to be validated? Anales Sis. San. Navarra 2011, 34, 63–72. [Google Scholar]
- Luján-Tangarife, J.A.; Cardona-Arias, J.A. Construcción y validación de escalas de medición en salud: Revisión de propiedades psicométricas. Arch. Med. 2015, 11, 1–10. [Google Scholar]
- Max Planck Institute for Psycholinguistics, The Language Archive, “Elan (Version 6.6),” Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen. 2023. Available online: https://archive.mpi.nl/tla/elan (accessed on 15 April 2024).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Obeso-Benítez, P.; Pérez-de-Heredia-Torres, M.; Huertas-Hoyas, E.; Sánchez-Herrera-Baeza, P.; Máximo-Bocanegra, N.; Serrada-Tejeda, S.; Marron-Romera, M.; Macias-Guarasa, J.; Losada-Gutierrez, C.; Palazuelos-Cagigas, S.E.; et al. Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System. Appl. Sci. 2024, 14, 3415. https://doi.org/10.3390/app14083415
Obeso-Benítez P, Pérez-de-Heredia-Torres M, Huertas-Hoyas E, Sánchez-Herrera-Baeza P, Máximo-Bocanegra N, Serrada-Tejeda S, Marron-Romera M, Macias-Guarasa J, Losada-Gutierrez C, Palazuelos-Cagigas SE, et al. Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System. Applied Sciences. 2024; 14(8):3415. https://doi.org/10.3390/app14083415
Chicago/Turabian StyleObeso-Benítez, Paula, Marta Pérez-de-Heredia-Torres, Elisabet Huertas-Hoyas, Patricia Sánchez-Herrera-Baeza, Nuria Máximo-Bocanegra, Sergio Serrada-Tejeda, Marta Marron-Romera, Javier Macias-Guarasa, Cristina Losada-Gutierrez, Sira E. Palazuelos-Cagigas, and et al. 2024. "Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System" Applied Sciences 14, no. 8: 3415. https://doi.org/10.3390/app14083415