A Checklist to Assess Technologies for the Diagnosis and Rehabilitation of Geriatric Syndromes: A Delphi Study

Abstract

Technologies for frail elderly individuals facilitate the integration of care services, support the post-discharge period, and enhance independence and quality of life while reducing isolation. However, the lack of methodological rigor in studies on technologies for diagnosing and treating geriatric syndromes limits applied research. This study aimed to develop and validate a checklist considering technical readiness, clinical needs, and context to support the use of technologies primary in clinical practice and also in research settings. To this aim, a Delphi procedure was conducted in four rounds, followed by a pilot test assessing the checklist’s practical effectiveness. Twenty-nine items were defined and discussed. Among them, no item was deleted, while a total of eight items were reformulated. At the end of the assessment steps, 73% of items showed a high median relevance rating. The pilot test showed the difficulty in finding relevant information to complete the checklist as the only critical issue. This tool represents the first validated checklist in the field of technology-based healthcare for elderly individuals and supports the application of technologies in the diagnosis and treatment of geriatric syndromes by clinicians and researchers.

1. Introduction

Aging is a complex and progressive biological process involving structural and functional changes at the cellular, tissue, and organ levels. These changes lead to a gradual decline in physical and cognitive abilities, a reduction in sensory, mental, and physical functions, and an increased vulnerability, resulting in higher morbidity, multimorbidity, disability, and mortality. This process is influenced by a combination of genetic, environmental, and behavioral factors, which determine how one ages [1,2,3]. As people age, the prevalence of specific age-related diseases increases significantly. Cardiovascular diseases such as hypertension, heart failure, and ischemic heart disease are among the leading causes of morbidity and mortality in older adults [4]. Similarly, metabolic disorders like type 2 diabetes and dyslipidemia become more frequent with age, contributing to additional health complications [5]. Cancer incidence also rises substantially in the elderly population, with aging recognized as one of the primary risk factors for several malignancies, including lung, breast, colorectal, and prostate cancers [6].

Globally, life expectancy is rising rapidly. According to forecasts, the percentage of people aged 60 and older will reach 21.1% by 2050, compared with 9.2% in 1990 and 11.7% in 2013 (World Health Organization; https://www.who.int/news-room/fact-sheets/detail/ageing-and-health (accessed on 18 December 2024)) [7]. This demographic shift, which involves an increase in the prevalence of chronic diseases and disabilities, represents a significant challenge for the sustainability of social and healthcare systems [8,9,10]. As aging progresses, the prevalence of frailty, comorbidity, and multimorbidity also increases, reducing life expectancy and compromising daily activities [11,12]. For instance, frailty has a prevalence of around 10% in individuals aged 65 or older [8].

Quality of life in older adults is often compromised by the progression of these age-related conditions [13]. Several factors influence the well-being of elderly individuals, including physical limitations, cognitive decline, and emotional health [14]. A key indicator of functional health in this population is performance status, which reflects an individual’s ability to carry out daily activities, manage personal care, and engage in social interactions [15,16,17]. Impaired performance status, commonly seen in frail older adults, is associated with an increased risk of institutionalization, dependence on caregivers, and reduced overall life satisfaction [18,19].

These conditions have a significant impact on quality of life, as well as on healthcare and social costs [8,9,10]. Additionally, they contribute to postoperative complications, an increased risk of mortality, and a substantial rise in healthcare costs, stemming from the frequency and duration of emergency department visits and hospital admissions [8,20,21,22]. In this scenario, managing frailty and aging effectively has become a public health priority [8].

The transition to an aging society has shifted towards more person-centered care models, with growing attention to frailty, comorbidity, and multimorbidity. These conditions have become a focus of increasing interest among clinicians and researchers. In this context, the use of assistive health technologies (AHTs), which are designed to maintain or improve functionality, autonomy, and well-being, as well as medical devices (MDs), which are used for prevention, diagnosis, and treatment, can have a significant positive impact on the elderly population [23]. However, a critical aspect is ensuring the accessibility and effective use of these technologies by older adults so that technologies can be effectively integrated into care models and contribute to improving patients’ quality of life [20].

The relevance of technology in the health field, and the importance of having a well-defined framework and procedures to assess technologies used in health services from several points of view (e.g., clinical effectiveness, safety, economic evaluation, and organizational aspects), are also clearly expressed by the emergence of specific regulation at the European level, such as in the case of the Health Technology Assessment (HTA) [24]. The HTA is the systematic evaluation of the properties, effects, or impact of a health technology in comparison with another technology, considering evidence about medical, economic, social, and ethical issues related to the use of the health technology [24]. The European Union is now in the implementation phase of this regulation (Regulation EU 2021/2282 on HTA) that will enter into application on 12 January 2025 [25].

Despite the intention of massively integrating technologies into various services and care environments, nowadays, studies evaluating the efficacy and effectiveness of technologies for diagnosing and treating geriatric syndromes often suffer from a lack of methodological rigor (e.g., imbalance toward a more represented number of observational studies compared with interventional ones). This not only limits the reliability of their findings but also prevents generalizing results across diverse patient populations. Consequently, the translation of research from the laboratory to clinical practice—what is known as translational research—is significantly hindered. Without a solid foundation of well-designed studies, it becomes difficult to apply technological advancements in a meaningful way to improve patient care in real-world settings [20]. In addition, in an increasingly complex world where new technologies are being proposed at an ever-increasing speed, the lack of structured tools such as checklists to help scan the technological landscape (e.g., to assess the proposals with the best level of technological readiness, clinical effectiveness, safety, and cost effectiveness) could possibly have a negative impact on the speed with which translational research selects and investigates the most promising technologies for the clinical field.

The objective of this study was therefore to develop and validate a checklist tool to assist the use of and research on technologies when applied in clinical samples and settings. The approach used is based on the Delphi method, a structured procedure for obtaining insights and perspectives from people with specific expertise on a topic or issue to inform decision-making in a particular area [26,27].

2. Materials and Methods

We followed a four-round Delphi method [27] for the development—the first two Delphi rounds—and validation—last two Delphi rounds—of the checklist. We also tested the checklist’s usefulness through a pilot test.

The Delphi method is a structured and iterative approach that utilizes a panel of experts to reach a consensus on a specific issue. The process involves multiple rounds of questionnaires or surveys, where experts anonymously provide their opinions and insights. After each round, the responses are collected, summarized, and fed back to the participants, allowing them to reconsider and revise their answers based on the group’s feedback. This iterative process helps refine ideas, address ambiguities, and progressively converge towards a consensus. The process continues until the responses stabilize or a clear agreement is reached to minimize bias and promote open discussion.

In our study, the Delphi method was employed to develop and validate a checklist tool designed to assist in the evaluation of technologies in clinical settings. The first two rounds focused on the development phase, where experts provided input on the tool’s structure and content, suggesting adjustments and refinements. The first Delphi round has been devoted to defining the general structure of the checklist. In the second Delphi round, the specific items were developed. The subsequent rounds concentrated on validating the checklist, ensuring that it accurately reflected the key factors necessary for assessing assistive technologies in elderly care. The third Delphi round assessed, using a group of experts responding to a questionnaire, the usefulness of the checklist and the adequacy of its structure. The fourth Delphi round, using the same approach of the third round, further refined the items of Section 2 and 3 of the checklist.

2.1. Development Phase

The development phase covered the first two Delphi rounds. During the development phase, the general structure of the checklist—the first round—and the relative items of each section—the second round—were defined.

In greater detail, regarding the checklist’s structure, a multidisciplinary subgroup of seven (A.G., P.D.T., C.T., M.SB., L.R., F.G., and A.GR.) clinicians and researchers (3 physicians, 3 psychologists, 1 engineer) of four institutions of the Italian Aging Network defined, by an open discussion interactive session where discrepancies were solved by consensus, the checklist general domains, which were: technologies’ readiness, clinical transferability, and research.

The technology readiness domain (from here on “Section one”) was described through the measurement system known as “Technology Readiness Level” (TRL). This method evaluates the maturity level of technology across nine readiness levels [28] that cover the development of technology’s basic principles (TRL-1) and concept (TRL-2), the experimental proof of concept (TRL-3), the lab (TRL-4) and environmental testing (TRL-5 and TRL-6), the prototype demonstration (TRL-7), the system qualification (TRL-8), and finally the application and commercial readiness (TRL-9) [29,30].

The clinical transferability domain (from here on “Section two”) and the research domain (from here on “Section three”) arose from the results of previous systematic reviews of our group [8,9,20], showing the limits of studies investigating the use of technologies for the diagnosis and treatment of frail, comorbid, or multimorbid old people. These limits first regard the clinical application of technologies. Our past reviews found that when technologies are applied in clinical settings involving old people, usually, patients’ inclusion/exclusion criteria, as well as conditions of frailty, comorbidity, and multimorbidity, are not clearly defined. Moreover, the aim(s) of technologies’ application (e.g., diagnosis, prognosis, prevalence, or intervention) is not always expressed, and usability, acceptability, and safety are not always assessed. Usually, studies do not define the technologies’ efficacy criteria; they do not conduct a cost-effectiveness analysis; and they do not assess whether technologies have an organizational impact on clinical settings. Section two of the checklist aimed to overcome these issues. Second, the limits regarding the research in this field usually includes pilot studies that are not methodologically rigorous and conducted on small samples of old people. Therefore, Section three referred to a step model proposed by our group, according to which technology research should comprise the advantages of conducting both “Efficacy” studies, which are highly controlled RCTs on selected groups of patients, and “Effectiveness” studies, that are pragmatic studies on large groups of patients in real-world settings [20].

In the Delphi second round, the main and sub-items of Section two and Section three were developed by A.G. and P.D.T. based on the results of our previous reviews [8,9,20]. Then, the items were checked for completeness and accuracy by all authors through two formal qualitative online meetings where each participant expressed, in an open discussion, his/her judgment and discrepancies were solved by consensus. Instead, as Section one consisted of the technology readiness level (TLR) scale [29,30], the 9 items of this section arose from the extensive literature on the TRL [28,29,30].

2.2. Validation Phase

The validation phase covered the last two Delphi rounds. During the validation phase, items from Section two and Section three of the checklist were assessed. The items in Section one were not included in the assessment because they refer to a well-validated method for estimating the maturity of technologies [29,30]. The two validation rounds involved a panel of 13 and 10 experts from six institutions of the Italian Aging Network, respectively (see

Table 1. Characteristics of the Delphi panel and pilot participants.

Study Phase	N° of Participants	Profession	Years of Professional Experience in ICT Field Median (Range)
First and second Delphi rounds	7	3 Physicians 3 Psychologists 1 Engineer	14 (6–35)
Third Delphi round	13	6 Engineers 3 Psychologists 1 Neuropsychologist 1 Physician 1 Biologist 1 Computer scientist	16 (3–25)
Fourth Delphi round	10	3 Engineers 2 Neuropsychologists 2 Psychologists 1 Physician 1 Biologist 1 Computer scientist	10.5 (2–26)
Pilot test	11	6 Engineers 4 Psychologists 1 Physician	17 (2–25)

). To obtain the final panel for each validation round, we contacted by email a large group of 30 clinicians and researchers within the Italian Aging Network. Clinicians and researchers were experts at using technologies for the diagnosis and treatment of frail old people in terms of past education and field(s)/years of the technologies’ application. When contacted, they were invited to join both validation rounds. We excluded experts belonging to the group involved in the development phase. Among the 30 people initially contacted, 17 did not reply to our invitation and 13 were finally included in the panel (see Table 1). Among these, 3 participated in the first but not in the second validation round.

Participation in Delphi surveys was considered as evidence of consent, with voluntary participation and the option to withdraw at any time without providing a reason.

In the first validation round, eligible experts received an email inviting them to assess, through a series of 5-point Likert scales (from 0 = not at all to 4 = very much), the usefulness of the checklist and the adequacy of the checklist structure, as well as the usefulness of items of Section two and Section three. At the end of the quantitative evaluation, experts were asked to qualitatively suggest modifications to the checklist structure and/or items’ content or phrasing. In the second validation round, experts were contacted a second time by email to assess, through a series of 5-point Likert scales (from 0 = not at all to 4 = very much), the usefulness of items that were modified or introduced right after the first validation round.

2.3. Pilot Test: Methods

In the pilot test, eleven participants used the checklist to evaluate an health technology of their choice and judged their experience, through a series of 5-point Likert scales (from 0 = not at all to 4 = very much), according to the following questions: simplicity of use of the checklist; simplicity of finding relevant information to complete the checklist; competence in information and communication technologies (ICTs) needed to use the checklist; and usefulness of the information obtained using the checklist.

2.4. Statistical Analysis

In the first validation round, the median usefulness rating, the median adequacy of the scale structure rating, and the median rating of the usefulness of each item of the checklist were evaluated. Moreover, for the items further assessed during the second validation round, the median usefulness ratings were compared with the evaluations of these items obtained in the first validation round by using the Brunner–Munzel test [31].

For the pilot study, the median rating for the 4 assessed areas (simplicity of use of the checklist; simplicity of finding relevant information to complete the checklist; impact of the level of ICT competence on the use of the checklist; and usefulness of the information obtained using the checklist) have been calculated.

Analyses were performed with Jamovy software (Version 2.3, 2022) http://www.jamovi.org. p values were two-tailed, with a significance level of 0.05, and a median value of 3/5 or more was judged as a positive result on the 5-point Likert scale.

3. Results

Twenty-nine items were defined, discussed, and formally assessed during the Delphi steps. No item was deleted, while a total of eight items were reformulated at the end of the evaluation steps. Section one of the checklist includes the nine items of the technology readiness level (TLR) scale [29,30], and they were integrated into the checklist as they were in the original version without any further assessment or modifications. The remaining items were finally divided into two sections: Section two, composed of 22 items, which refers to clinical transferability; and Section three, composed of 7 items, which refers to research data.

Table 1 shows the general characteristics of the participants in all the phases of the study.

3.1. Development Phase: First and Second Delphi Rounds

During the first two Delphi rounds, the general structure of the checklist and the specific items were developed.

The first development round was devoted to defining the general structure of the checklist through an open discussion. The discussion among participants (A.G., P.D.T., C.T., M.SB., L.R., F.G., and A.GR.) revealed that the checklist would be helpful to analyze three main areas (i.e., checklist sections) about the use of technology in geriatric health services: one related to the technology itself; one related to the implementation of technology in the health services; and one related to clinical research about the technology. After the open discussion, participants reached a general agreement about this checklist structure.

The second development round was devoted to defining items of the three sections of the checklist. The group of participants agreed that Section one of the checklist, assessing the readiness of the technology under consideration, could effectively be represented by one of the most used tools in the EU area, called the TRL scale [29,30]. This tool, being a well-known standard in the assessment of technologies in the framework of the EU research funding program, was chosen as the best tool to saturate the assessment process of Section one of the checklist. The development of items of the checklist in Section two, devoted to the implementation of technology in health services, requested two steps. Based on data that emerged in previous systematic reviews of our research group [8,9,20], the first step defined the seven areas to which the items of Section two belong. The seven areas were: 1. reference to a classification model for the technology at hand; 2. usability, acceptability, and safety issues; 3. type of population and setting (e.g., clinical or home settings) targeted by the use of the technology; 4. objectives for the use of the technology; 5. technology efficacy and effectiveness; 6. operability in healthcare services; and 7. need for cultural and technical change for the use of technology. The second step, after agreement on the seven areas, where discrepancies were solved by consensus, defined the items for each area. The items were firstly developed by A.G. and P.D.T. based on the results of the mentioned systematic reviews of our research group and then checked for completeness and accuracy by all authors through two formal qualitative online meetings, where each participant expressed, in an open discussion, their judgment; discrepancies were solved by consensus. All participants expressed their opinion on each item.

Following the same approach, the checklist items regarding the clinical research section were formulated starting from a previous work of our group [20], in which we proposed a step model to conceptualize a research approach for the validation of technologies to intervene in geriatric syndromes. The step model was substantially based on the typical stages of pharmacological research, from the preclinical to the post-market phase. Each step of our model was then converted into a checklist item.

3.2. Validation Phase: Third and Fourth Delphi Rounds

The validation phase started with the third Delphi round, in which two general questions about the usefulness of the checklist and the adequacy of its structure were assessed by the group of experts. Moreover, the usefulness of each of the 19 items of section two and the 7 items of Section three of the checklist were specifically judged. The results are presented in

Table 2. Median (interquartile range) judgment for the first and second Delphi rounds (NA = not applicable; ns = not significant).

Items	Third Delphi Round Median (IQR)	Fourth Delphi Round Median (IQR)	p Value
Usefulness of the checklist	4.00 (1)	NA
Adequacy of the structure	3.00 (1)	NA
Relevance items Section two
Item 1	3.00 (1)	3.00 (0)	ns
Item 2	3.00 (1)	NA
Item 3	4.00 (1)	3.50 (1)	ns
Item 3.1	NA	4.00 (1)	ns
Item 3.2	NA	3.00 (0.75)	ns
Item 4	4.00 (1)	4.00 (0.75)	ns
Item 5	4.00 (1)	NA
Item 6	3.00 (1)	4.00 (0)	ns
Item 6.1	NA	4.00 (0)
Item 7	4.00 (1)	4.00 (0.75)	ns
Item 8	4.00 (1)	NA
Item 9	4.00 (1)	NA
Item 10	4.00 (0)	NA
Item 11	4.00 (1)	NA
Item 12	4.00 (1)	NA
Item 13	4.00 (1)	4.00 (0.75)	ns
Item 14	4.00 (1)	NA
Item 15	3.00 (2)	3.00 (1)	ns
Item 16	4.00 (1)	NA
Item 17	3.00 (2)	NA
Item 18	3.00 (1)	NA
Item 19	4.00 (1)	NA
Relevance items Section three
Item 1	3.00 (2)	NA
Item 2	4.00 (1)	NA
Item 3	4.00 (1)	NA
Item 4.1	4.00 (1)	NA
Item 4.2	4.00 (1)	NA
Item 5	4.00 (1)	NA
Item 6	4.00 (1)	3.50 (1)	ns

. All items were assessed by experts, and no missing data were present. According to a 1–5 Likert scale, the median (IQR) ratings for the usefulness of the checklist and adequacy of its structure were four (one) and three (one), out of five, respectively. On the same 5-point Likert scale, a median relevance rating of four out of five for 73% of items assessed, and a median rating of three out of five for the remaining items emerged (see Table 2). From a qualitative point of view, experts suggested some lexical modifications (mainly rephrasing for items 1, 3, 4, 6, 7, 13, and 15 of Section two and item 6 for Section three due to the excessive length of the sentences used or vagueness of some of the terms used) and the introduction of sub-items for Section two (items 3 and 6) to proceed more in depth in the assessment of specific points. The checklist was then modified according to experts’ suggestions by adding three further sub-items (two sub-items for item 3 and one sub-item for item 6 of Section two; see Table 2), and a fourth Delphi round was implemented. During the second validation round (fourth Delphi round), experts were asked to assess, on a 1–5 Likert scale, the usefulness of each modified or introduced item and sub-items for Sections two and three of the checklist. A total number of 11 items and sub-items were assessed: 10 for Section two and 1 for Section three. The results of the second assessment are presented in Table 2. All items were evaluated by experts, and no missing data were present. A median rating of three or more emerged for each assessed item. No significant differences emerged in the median rates between modified items and their original version (see Table 2) at the Brunner–Munzel test level [31], suggesting that after the lexical modification, the relevance of the items did not significantly change.

At the end of both validation steps (third and fourth Delphi rounds), the checklist comprised a 9-item TRL scale in Section one, 22 items in Section two, and 7 items in Section three.

3.3. Pilot Test: Results

In the pilot test, eleven experts used the checklist (see

Table 3. Checklist.

Experience:
Checklist: Technology Readiness Level
				Evaluation
PHASE	TRL	STAGE	DESCRIPTION	YES	NO
Research	TRL1	Basic principles observed	Identification of the new concept; identification of the integration of the concept; identification of expected barriers; identification of applications; identification of materials and technologies based on theoretical fundamentals/literature data; preliminary evaluation of potential benefits of the concept over the existing ones
	TRL2	Technology concept formulated	Enhanced knowledge of technologies, materials and interfaces is acquired; new concept is investigated and refined; first evaluation about the feasibility is performed; initial numerical knowledge; qualitative description of interactions between technologies; definition of the prototyping approach and preliminary technical specifications for laboratory test
	TRL3	Experimental proof of concept	First laboratory scale prototype (proof of concept) or numerical model realized; testing at laboratory level of the innovative technological element (being material, sub-component, software tool, …), but not the whole integrated system; key parameters characterizing the technology (or the fuel) are identified; verification of experimental application through simulation tools and cross-validation with literature data (if applicable).
Development	TRL4	Technology validated in lab	(Reduced-scale) Prototype developed and integrated with complementing sub-systems at laboratory level; validation of the new technology through enhanced numerical analysis (if applicable); key performance indicators are measurable; the prototype shows repeatable/stable performance (either TRL4 or TRL5, depending on the technology)
	TRL5	Technology validated in relevant environment	Integration of components with supporting elements and auxiliaries in the (large-scale) prototype; robustness is proven in the (simulated) relevant working environment; the prototype shows repeatable/stable performance (either TRL4 or TRL5, depending on the technology); the process is reliable and the performances match the expectations (either TRL5 or TRL6, depending on the technology); other relevant parameters concerning scale-up, environmental, regulatory, and socio-economic issues are defined and qualitatively assessed
	TRL6	Technology pilot demonstrated in relevant environment	Demonstration in relevant environment of the technology fine-tuned to a variety of operating  conditions; the process is reliable and the performances match the expectations (either TRL5 or TRL6, depending on the technology); interoperability with other connected technologies is demonstrated; the manufacturing approach is defined (either TRL6 or TRL7, depending on the technology); environmental, regulatory, and socio-economic issues are addressed
Deployment	TRL7	System prototype demonstration in operational environment	(Full-scale) Pre-commercial system is demonstrated in operational environment; compliancy with relevant environment conditions, authorization issues, and local/national standards is guaranteed, at least for the demo site; the integration of upstream and downstream technologies has been verified and validated; the manufacturing approach is defined (either TRL6 or TRL7, depending on the technology)
	TRL8	Active CommiSystem completed and qualified	Technology experimented in deployment conditions (i.e., real world) and has proven its functioning in its final form; manufacturing process is stable enough for entering a low-rate production; training and maintenance documentation are completed; integration at system level is completed and mature; full compliance with obligations, certifications, and standards of the addressed markets
Operations	TRL9	Actual system proven in operational environment	Technology proven fully operational and ready for commercialization; full production chain is in place and all materials are available; system optimized for full rate production
Instructions for completion: the order of the items is sequential, so an X should be placed on the readiness level of the technology used (e.g., a technology rated at readiness level 7 means it has already reached the previous levels, TRL1–TRL6).
Checklist: Transferability in Practice
				Evaluation				Specification
CATEGORY	ITEM	DESCRIPTION		YES	NO	NR	NA
Technology	1	Has the technology being tested or in use been categorized in a formal technology classification model (e.g., [32] Carretero et al., 2015; TAALXONOMY)? If YES, please specify the name of the model in the “Specification” field.
	2	If the technology has been classified using a formal model, please specify the application area based on the taxonomy used in the “Specify” field (e.g., Telemedicine; Ambient Assisted Living; Smart Homes).
Usability/Acceptability/Security	3	Were specific tools used to evaluate the usability, acceptability, and safety (both in terms of usability and data protection) of the described technology? If YES, please specify which variable(s) and tools in the “Specification” field.
	3.1	Regarding question 3, are the assessment tools used validated tools (proven effectiveness in measuring the variables of interest)?
	3.2	Regarding question 3, are the assessment tools used specific to a particular technology?
	4	If available, have the results obtained from the usability, acceptability, and safety analysis (both in terms of usability and data protection) been described in formal, consultable documents (e.g., research articles, technical sheets, etc.)?
Population and setting	5	Have inclusion and exclusion criteria been defined for the populations targeted by the technology? If YES, please briefly state them in the “Specification” field.
	6	Are there indications regarding the setting in which the technology has been applied (e.g., inpatient, outpatient, home, semi-residential, and residential services)? If YES, please specify which ones in the “Specification” field.
	6.1	Regarding question 6, is the autonomous use of the technology by the patient expected?
Objectives	7	Have the usage objectives of the technology been clearly defined (e.g., Diagnosis/Prognosis/Prevalence/Intervention)? If YES, please specify the objective(s) in the “Specification” field.
	8	Has a clear procedure been defined for the use of the technology with the clinical populations identified in item 5?
Effectiveness	9	Have clear criteria been defined for evaluating the effectiveness of the technology based on its objectives (e.g., variables and outcome measures, diagnostic accuracy)?
	10	Have the results obtained from the efficacy studies been described in formal, consultable documents (e.g., research articles, technical sheets, etc.)?
	11	Are there formal documents in which the strengths and weaknesses highlighted by the conducted trials are described?
Operability	12	Have the infrastructures and equipment necessary for the use of the technology been clearly defined (e.g., shared platforms, interoperability between different platforms and/or databases)?
	13	Has a cost-effectiveness analysis been performed for the technology considered? If YES, please specify the variables considered in the “Specification” field.
	14	Is there a fee covering system from the National Health Service for services provided using the considered technology?
	15	Is the use of a specific care model (e.g., Chronic Care Model; Expanded Chronic Care Model; Palliative Care Model) planned for the application of the technology within the intervention setting? If YES, please specify which one in the “Specification” field.
	16	Have the criteria and procedures for ensuring privacy and secure data exchange for the use of the considered technology been clearly defined?
Change	17	Has an analysis been conducted on the need for industrial evolution of the considered technology?
	18	Has the potential need for a change in professional culture for the operators who will use the considered technology been analyzed and defined (e.g., modification of routines and procedures in clinical practice)?
	19	Has the potential need for an organizational change for the use of the considered technology been analyzed and defined?
Checklist: Research
				Evaluation				Specification
PHASE	PHASE	DESCRIPTION		YES	NO
Preclinical studies	1	Development of a new technology or adaptation of an existing technology
	2	Recognition of the technological tool based on its technical characteristics by a recognized institution (e.g., obtaining the CE mark).
	3	Analysis of the target population, usability, acceptability, and safety.
Diagnostic Studies/Clinical Research	4.1	Observational diagnostic studies on small groups.
		Observational diagnostic studies on large groups and in real-world settings.
	4.2	Intervention studies on small selected groups (e.g., RCT, pre-post studies, cross-over studies). If one or more of these studies have been conducted, please specify the type of study in the “Specification” field.
		Intervention studies on large groups and in real-world settings (e.g., RCT, pre-post studies, cross-over studies). If one or more of these studies have been conducted, please specify the type of study in the “Specification” field.
Revision	5	Review/approval of the applicability of the technology on large patient groups in clinical practice settings by a recognized institution.
Post-marketing	6	Post-marketing monitoring (e.g., adverse events, effectiveness in usual-care settings).
Instructions for completion: The order of the items is sequential (except for Phase 4, where sub-phases 4.1 and 4.2 are alternative to each other, and for Phase 2, as for research purposes, in some cases, the technology can be applied regardless of the recognition/approval of technical characteristics). Therefore, an X should be placed on the level of research that has been conducted (e.g., to perform an analysis of the target population, usability, acceptability, and safety—pre-clinical study Phase 3—it is assumed that there is already a technology to use, which may have been developed from scratch or adapted from an existing technology—pre-clinical study Phase 1—and that the technology has been recognized as usable based on its technical characteristics—pre-clinical study Phase 2).
Legend
NR	Not reported		This item refers to the absence of useful information to adequately answer the question
NA	Not applicable		This item refers to the case where the question is not applicable to the technology under consideration

) to evaluate a specific technology and judged their experience according to four questions: simplicity of using the checklist; simplicity of finding relevant information to complete the checklist; ICT competence needed to use the checklist; and usefulness of the information obtained using the checklist. The results of the assessment are presented in

Table 4. Median (IQR) judgment for the pilot study.

Items	Median (IQR)
Simplicity of use	3.00 (1)
Simplicity to found relevant information to complete the checklist	2.00 (0.50)
ICT competence needed to use the checklist	3.00 (1)
Usefulness of the information obtained using the checklist	3.00 (0)

. All items were assessed by experts, and no missing data were present. According to a 1–5 Likert scale, a median rating of three emerged for the first, third, and fourth questions, suggesting that the checklist was found to be moderately simple to use, a moderate level of ICT competence was found to be needed to use the tool, and information emerging from the use of the checklist was found to be moderately useful. A median rating of two emerged for the second question, suggesting that our experts judged it quite difficult to find relevant information useful to complete the checklist.

4. Discussion

Frailty, neurodegeneration, and geriatric syndromes have a significant impact at the clinical, social, and economic levels, mainly in the context of the aging world. Recently, information and communication technologies (ICTs), virtual reality tools, and machine learning models have been increasingly applied to the care of older patients to improve diagnosis and interventions. However, studies testing the efficacy and effectiveness of technologies for the diagnosis and treatment of geriatric syndromes still lack methodological rigor, thereby preventing translational research from bench to bedside. The objective of this study was to develop and validate through a Delphi approach a checklist to assess, from different points of view (technology readiness level, clinical transferability, and available research data), technologies for the diagnosis and treatment of geriatric syndromes.

The defined checklist was developed and validated through four Delphi rounds and one pilot test. The Delphi method is a robust method for reaching a consensus and is commonly used for validating checklists. Moreover, the introduction of a pilot study is highly suggested as a further step [33]. Our results showed that both the usefulness of the checklist and its adequacy in terms of global structure were positively evaluated by our experts, suggesting a good general quality of the tool. In greater depth, most items of Sections two and three (19/26, 73%, in the first validation round) received a median relevance rating of four out of five. It informed us about the perceived usefulness, from the experts involved in the study, of the checklist items. This result was confirmed after the second validation round, suggesting that the checklist retains highly relevant items for assessing technologies.

The pilot testing highlighted a critical issue with the use of the checklist. Our panel of experts rated the simplicity of finding relevant information to complete the checklist as two out of five. This rating aligned with their qualitative feedback after using the checklist to assess ICT technologies in their field. In particular, the reported difficulties mainly regarded finding information about clinical transferability. Because almost all items of the checklist obtained a good judgment regarding their relevance, in line with previous research of our group [8,20], we hypothesize that the difficulties are due to the lack of robust methodologies testing the efficacy and effectiveness of technologies for the diagnosis and treatment of geriatric syndromes and the lack of a clear procedure about what kind of information technologies should be proposed in clinical settings.

However, our panel of experts found it quite simple to use the checklist (judgment for simplicity of use was three out of five), and this suggests that the tool could be used without a great effort in terms of comprehension of the general structure and its specific parts, time needed to be completed, and types of information to search for, confirming the usability of the tool [34].

Finally, experts involved in the pilot testing judged the information obtained using the checklist as useful, with a median rating of three out of five. This result suggests that the developed tool could help address a gap in the current landscape of technologies for diagnosing and treating geriatric syndromes. Despite the availability of many technologies for clinical use, the critical challenge remains: accurately determining whether a technology is the best choice for a specific clinical need. The checklist developed in our study tries to cover this need, allowing a systematic judgment across several points of view (technological readiness, real-world implementation, and research results). This, in turn, helps clinicians and researchers to take advantage of robust information to reach informed decisions about the technology at hand and its adequacy to reply to a specific clinical need. This approach is in line with what the European Network for Health Technology Assessment recommends about the evaluation process of health technologies, which is: a clear assessment of previous studies’ results; the disclosure of the rationale for using technology; the clinical indication of the population; the kind of intervention and comparators; the evidence about safety and effectiveness; and the definition of study design (https://tools.eunethta.be/guidelines.html (accessed on 20 December 2024)).

To the best of our knowledge, this is the first checklist developed and validated to assess and deepen the adoption of technologies in the rehabilitation and evaluation of geriatric syndromes. Specifically, based on the results of our study, the checklist provides a structured tool to support clinicians and researchers in the evaluation of the robustness of data and information available about the application of technologies to diagnose and treat old people suffering from geriatric syndromes. The checklist allows us to use of technologies after a standardized evaluation through specific criteria, facilitating a systematic and repeatable approach to data collection and analysis [8,20]. This aspect improves the replicability of studies and clinical practices and allows for comparing the effectiveness of different technologies, creating a foundation for more solid and meaningful evidence [20]. Furthermore, using our checklist could help healthcare professionals by offering them clear guidelines on how to integrate technologies, meeting users’ needs, into patients’ daily practice, thereby improving acceptance and use [8]. Moreover, it provides a strong frame of reference describing all steps useful to obtain a technology of good quality to be used consciously for diagnostic or interventional purposes in a real-world setting. Finally, our checklist does not require a high level of ICT expertise to be completed. Indeed, based on our pilot test results, only a moderate level of expertise was judged as needed to use the checklist.

The main limitation that emerged from our study relates to the difficulty of finding relevant information to complete the checklist. It is possible that this checklist did not take into consideration other possible and important items covering a clearer and in-depth understanding of all aspects that are needed to reach a complete assessment regarding the use of technology in the rehabilitation and assessment of geriatric syndromes. In addition, the focus of the checklist on the older population only may in itself be a further potential limitation of this tool. Future studies could focus on these limitations by exploring other evaluation areas that allow us to develop and validate a more complete version of the checklist. Moreover, the Delphi procedures we followed in our study, mainly during the validation steps, involved different groups of clinicians and researchers. The fact that some experts did not oversee the procedure from beginning to end may have introduced some biases, thereby affecting the study results. Finally, the present checklist should be formally tested using a more detailed evaluation procedure than that used in our pilot testing, which was only devoted to recollecting the impression of participant about the experience to use the checklist. The assessment of specific case examples in further pilot testing with a more detailed quantitative approach would be an interesting future development of this preliminary study.

5. Conclusions

This is the first validated checklist, in the field of technology-based healthcare of old people, that integrates considerations about tools’ technical readiness, the clinical needs of targeted population and setting, and research implementation. The tool is intended to support clinicians and researchers in the application of technologies to diagnose and treat old people suffering from geriatric syndromes. While our results are promising, future research will be needed to further implement the quality of this checklist to use it as a possible standard tool to assess technologies in this field.