Experimenting Mobile and e-Health Services with Frail MCI Older People ^†

Abstract

The ageing population has become an increasing phenomenon world-wide, leading to a growing need for specialised help. Improving the quality of life of older people can lower the risk of depression and social isolation, but it requires a multi-dimensional approach through continuous monitoring and training of the main health domains (e.g., cognitive, motor, nutritional and behavioural). To this end, the use of mobile and e-health services tailored to the user’s needs can help stabilise their health conditions, in terms of physical, mental, and social capabilities. In this context, the INTESA project proposes a set of personalised monitoring and rehabilitation services for older people, based on mobile and wearable technologies ready to be used either at home or in residential long-term care facilities. We evaluated the proposed solution by deploying a suite of services in a nursing home and defining customised protocols to involve both guests (primary users) and nursing care personnel (secondary users). In this paper, we present the extended results obtained after the one-year period of experimentation in terms of technical reliability of the system, Quality of Experience, and user acceptance for both the user categories.

1. Introduction

The world is facing an increasing trend in the number and proportion of older persons with respect to the global population. Population ageing is a challenge, especially in terms of Long-Term Care (LTC) costs, which could increase greatly if inappropriate solutions are integrated in the care process. Even though life expectancy is increasing over time, quality of life decreases with ageing, due to multiple medical problems and deficits in different ability domains (i.e., cognitive, motor, social), which generally leads to frailty conditions. In fact, frailty is a geriatric syndrome that generally affects people over 65 years old, and represents a state of vulnerability with increased risk of poor health outcomes, including falls, incident disability, hospitalization and mortality [1]. This condition makes the general health status really complex and chronic [2].

Most older people would prefer to continue to live in their own home, to maintain their independence and autonomy. In 2011, the proportion of older persons in Europe, aged 65–84 years and living in nursing homes, was $1.7\%$. Considering people over 85 years, this percentage increases more than seven times, reaching $12.6\%$, with a substantial prevalence of women (http://ec.europa.eu/eurostat/web/population-demography-migration-projections/). These people need specialised help and continuous monitoring, training and rehabilitation to maintain their quality of life and discard the risk of depression, social isolation, and frailty decline. To this aim, the introduction of e-health services in nursing homes and, in general, in residential LTC facilities, can help stabilise the health conditions of the guests in terms of physical, mental, and social capabilities. In addition, the use of personalised monitoring and rehabilitation services can contribute to the reduction of LTC costs, both in nursing home institutions (with appropriate spaces and personnel support) and directly at home (for those who can continue living independently). Another fundamental requirement for LTC is the possibility to implement an individual integrated care program, providing a complete picture of the health status of each subject during time, and personalised feedback as well as coaching solutions to help reduce unhealthy behaviors.

The integrated care approach was recently defined by the World Health Organization (WHO) in the Integrated Care for Older People (ICOPE) programme [3] with a focus on re-orienting primary care providers and health systems to respond to the great diversity in physical and mental capacities of older people, and to provide person-centred and integrated services. This type of programs can also help physicians address acute conditions based on a complete clinical and behavioural picture. Integrated care fits in the wider context of health and well-being research, that has witnessed a growing interest both from the research communities and governments. Specifically, an EU commission funded several research projects both in FP7, as reported in [4], in the more recent H2020 Societal Challenge 1-Health, demographic change and well-being program (e.g., SUSTAIN [5], MARIO [6], SENSE-Cog [7], My-AHA [8], NESTORE [9]), and in the dedicate joint program Ambient Assisted Living (AAL) (e.g., Amica [10], eCAALYX [11]).

Most of these projects address specific problems, both in terms of technological solution (e.g., monitoring systems, tele-health, online social networks, etc.) and specific health conditions. However, they all present a major drawback: the lack of a systemic and integrated approach in both clinical and technological areas. Specifically, they do not consider the physiological, psychological and social aspects as main contributors of the whole health and well-being picture of a subject. Moreover, the monitoring systems developed and implemented in the projects, both for personal and environmental data collection, mainly address home-based scenarios only.

A recent review study was published with the aim of identifying and describing the key elements of integrated care models for older people reported in the literature [12]. It shows that the types of integrated care interventions are a combination of interventions that are clinically (micro level) or professionally (meso level) focused, with only one case reported on a combination of an organisational/service and system (macro level) integrated care intervention. As a result, it reported that evidence of elements of integrated care for older people focuses particularly on micro clinical care integration processes, while there is a relative lack of information regarding the meso and macro system-level care integration strategies. This aspect is also confirmed by a study that examines the specific actions required to implement integrated care at community level and identifies actions needed in systems (macro level) and services (meso level) globally [13]. It showed that, while the WHO has already undertaken research at the micro level [14], there are specific actions guide that still need to be addressed to provide integrated care for older people through harnessing community involvement, building skills in the paid and unpaid workforce, prioritising service delivery in community settings, and levering health information and digital systems.

The lack of a holistic solution that can provide more general services, customizable by users requirements, poses a big challenge in this research field. In this context, we presented in a previous work [15] INTESA project, aimed at designing and developing a set of personalised monitoring and rehabilitation services based on mobile, wearable, free contact and touch screen technologies developed to provide an integrated care and monitoring program for frail older people. The proposed services, as part of a general framework, were deployed in a LTC facility located in Italy, and they were experimented with by a group of Mild Cognitive Impairment (MCI) frail older persons for a one-year period. The INTESA research project (http://www.progetto-intesa.it/) [16] was co-funded by Tuscany region and it relies on the collaboration with the nursing home RSA Tabarracci in Viareggio (Italy), partner of ICARE srl (http://www.icareviareggio.it/). In this paper, we present the results of the study at the end of the one-year experimentation phase, both from a technological, acceptance level, and quality of experience of the system.

Contributions

The main contributions of the work rely on an in-depth analysis of the system reliability in terms of data availability and validity, collected over one-year experimentation in a LTC facility. The analysis was conducted both at system and component level, both for single and group of users. Furthermore, the obtained results were further expanded considering how the system reliability, user acceptance and quality of experience influenced each other. To this aim, we conducted a survey at the end of the experimentation, in order to evaluate the impact of the proposed solutions on the daily life of primary (i.e., frail older people) and secondary (i.e., nurses, care givers, and socio-health professionals) users. The survey covers the Quality of Experience (QoE) and the level of acceptance of the proposed solutions. Users’ answers provide useful follow up insights, especially if compared with those obtained by the mid-term evaluation (month 4). User acceptance and QoE are fundamental to analyse how human factors, related to primary and secondary users’ involvement, can influence the reliability of the system and vice versa.

The paper is organized as follows: Section 2 provides a general description of the INTESA technological solutions and their deployment in the LTC facility. Section 3 gives details on the recruitment phase and criteria in order to introduce the user categories. Section 4 presents the main results in terms of expected and collected data, reliability of the single services and the entire system, and the compliance of the primary users in executing the requested activities. Section 5 presents the Quality of Experience (QoE) and Acceptance results for both the user categories and their evolution during the project lifetime. Section 6 presents the lessons learnt from this real-world experience, while Section 7 draws the conclusions of the work.

2. Overview of the INTESA Project and Services

The main goal of the INTESA project is to provide a suite of personalised and highly innovative mobile and e-health services to improve the quality of life and well-being of older people, with particular attention paid to frailty conditions. The INTESA system, and the involved services, are developed by using modular programming techniques, in order to easily introduce additional services and tools. It also provides a complete combined care approach through integrated data analysis tools applied to medium and long-term monitoring of heterogeneous health conditions: body composition and nutrition, motor and cognitive abilities, sleep quality, stress conditions and social interactions.

From a technological point of view, each module is developed using the best state-of-the-art technologies in terms of e-health, e-inclusion, wireless sensors networks, data mining and artificial intelligence. As shown in Figure 1, the INTESA system is composed of five main monitoring services:

• Sleep Quality monitoring service (keyword ‘Sleep’ in the figure), based on the use of Force Sensor Resistors (FSR) placed under the mattress of the user’s bed [17,18,19], connected with a Raspberry Pi mini-PC installed under the bed.
• Nutrition and Body Composition monitoring service (keyword ‘Nutritional’ in the figure), based on the use of a mobile app that collects data about menu choices and consumptions, connected to a bioimpedance scale able to measure weight, body mass index (BMI), basal metabolic rate (BMR), percentages of body water, body muscle, body fat and body bone mass.
• Motor, cognitive and balance monitoring service (keyword ‘Stability, Cognitive, and Motor’ in the figure), consisting of a Microsoft Kinect V2 infrared-based sensor for gesture and motion recognition, a Nintendo Wii balance board for stabilometric assessment [20], and a portable electroencephalogram (EEG) headset for the collection of EEG traces. These devices represent the hardware of a subsystem aimed at supporting the user to execute physical and cognitive rehabilitation exercises, designed specifically for the target primary users.
• Stress monitoring service (keyword ‘Stress’ in the figure), aimed at monitoring physiological signals derived from wearable sensors during the execution of physical and cognitive rehabilitation. The target of the service is to analyse the stress level during the therapy as input for possible personalizations.
• Social interactions monitoring service (keyword ‘Social Interactions’ in the figure), built upon an indoor localization system, using Ultra Wide Band (UWB) anchors and tags.

These services were deployed in a LTC facility hosting up to 60 guests distributed over three floors. Each floor is equipped with bedrooms, a kitchen and living room, 2–3 dining rooms for social interactions and guided activities. In addition, there is a gym for rehabilitation and common spaces for interactions, both indoor and outdoor. The project team was allowed to instrument a dedicated ambulatory room of the structure to execute individual sessions of motor and cognitive rehabilitation and stress monitoring, and the bedrooms of voluntary guests for the installation of the sleep monitoring system. It is worth noting that, thanks to the modularity of the INTESA architecture, some of the services were deployed at different times during the experimentation phase, depending on the requested laboratory testing. In particular, the social interactions monitoring service (in the light grey box in Figure 1) was deployed some months after the beginning of the experimentation. The delay was mainly due to technical and structural issues arisen during the installation of UWB anchors in the ceiling and in specific rooms. Furthermore, during the experimentation phase all the services were constantly improved and personalised for the reference environment and the users’ characteristics. These aspects had an impact both on the overall data collection campaign and the user experience as it will be detailed in Section 4 and Section 5.

From the medical point of view, the reference LTC facility has a 24 h nursing service, while it relies on General Practitioners (GPs) of single guests as a medical support. It provides the possibility of psychological, nutritional and neurological consultation through the collaboration with medical specialists and a daily support from physiotherapists. The general offer of the nursing home towards each single guest includes a daily check on the pharmacological therapy adherence and the general health conditions, 1 h of cognitive and motor rehabilitation in group, and a session of socio-educational activities. Through the collaboration with INTESA, the nursing home has significantly increased its offer, both in terms of innovative health monitoring solutions and data collection on its guests, providing a more accurate and integrated care approach, and new tools for the personnel, aimed at simplifying their job in the daily assistance.

3. Primary and Secondary Users

The INTESA services were deployed in the LTC facility after the presentation of the project objectives both to the personnel and to the guests and familiars in February 2018. Then, we implemented a screening of the interested guests and, for each of them, we collected a signed informed consent on the privacy and security policies adopted in the project for personal data management, according to the recent EU General Data Protection Regulation (GDPR). The experimental phase started in mid March 2018 and lasted for one year. Even though there were different experimenting periods for different services, the overall experimentation maintained its mid- and long-term evaluation.

After the presentation of the project, we received the interests of the majority of the guests (primary users) with their approval and that of their families. Specifically, 57 guests participated to the nutritional monitoring through the mobile app compiled by the care givers, while only a subset was able to execute the body composition monitoring, since it requires the ability to maintain the standing position in autonomy for 2–3 min. Among those, 35 guests declared their interest in participating in all the services. To address the technical and user capability requirements of the proposed services, we implemented the following screening procedure to evaluate the ability of each guest. The participation of the same subjects to all the proposed services allowed us to evaluate the impact of the integrated care approach proposed by INTESA.

The general screening of each participant included: Mini Mental State Examination (MMSE), Berg Balance Scale assessment, Waterloo Test for limb dominance, and measurement of anthropometric parameters (BMI, weight, brachial and gastrocnemius muscle circumference), Mini Nutritional Assessment (MNA), Activity Daily Living (ADL) and Instrumental ADL (IADL), Apathy Evaluation Scale (AES), Hamilton Depression Scale (HDS), Cohen Perceived Stress (CPS) and Insomnia Severity Index (ISI). During the experimentation phase, as follow-up procedure, MMSE, ADL and IADL tests were executed after six months, while all the others were executed once per month.

Among the 35 subjects proposed, we recruited 10 persons able to execute all the requested activities, and five as control.

During the one-year period, four subjects dropped out due to health problems and complications. The goal of INTESA is to prevent the decline of the frailty condition while maintaining and improving the basic abilities of older people. However, in case external health factors generate a sudden decline with subsequent inabilities and impairments, INTESA is not able to provide additional support to the person. For these reasons, all the drop outs were substituted by new enrollments with similar recruitment parameters. Specifically, the subjects are $80.4$ years old, on average, six women and four men, with an education level of $7.9$ school years on average.

Table 1. Voluntary screening and recruitment.

Test	Thresholds	Avg. Score
MNA (21 items)	$<17$ malnutrition, $[17,23.5]$ risk of malnutrition, $[24,30]$ normal	25.4
MMSE (30 items)	$>=23$ absence of dementia	25.6
ADL (6 items)	$6/6$ normal	5.8
IADL (8 items)	$8/8$ normal	3.4
AES (18 items)	$[0,42]$ higher values, more severe apathies	33
HDS (21 items)	$<=7$ no depression, $[8,17]$ slight depression, $[18,24]$ moderate depression, $>=25$ severe depression	20.5
CPS (14 items rated on a 5 point scale)	$>=20$ high stress	16.2
ISI (7 items)	$[0,7]$ absence of insomnia, $[8,14]$ subthreshold insomnia, $[15,21]$ moderate insomnia, $[22,28]$ severe insomnia	16.1

presents the threshold values generally used to evaluate the output of the screening tests and the average score of our population. The selected subjects can be categorised with Mild Cognitive Impairment (MCI) since they are affected by an intermediate stage of cognitive decline, frequently associated with physiological aging, that often, but not always, represent a transitional phase from cognitive changes or deficits of normal aging to pathological features found in neurodegenerative and vascular dementia. MCI evaluation derives from the results of MMSE, ADL and IADL tests, with particular reference to attentive and executive functions [21].

All the recruited subjects (and related familiars) received a detailed description of the INTESA objectives and the proposed solutions, including the policies we adopt for storage, anonymization, analysis and use of the personal data. They participated in a dedicated informative event and everybody signed the informed consent, according to the new EU GDPR regulation.

As far as the secondary users are concerned, the nursing care personnel includes four different categories: licensed nurses, social health assistants, educational operators and physiotherapists, each one with specific role and responsibilities. In addition, a medical specialist was enrolled by the LTC facility specifically to support the project. The deployed services directly request the interaction with the two physiotherapists available in the institution for the cognitive, motor and stress monitoring sessions, while about 10 social health assistants were recruited as main users of the Nutrition and Body Composition monitoring service, with particular reference to the DoEatWell (DEW) mobile application. The nursing home personnel work independently at the different floors of the structure and cover all the daily routines. In December 2018, four secondary users (two physiotherapists and two social health assistants) left the nursing home and they were substituted by new personnel, without any communication to the project partners. Therefore, the new secondary users received instructions, as to how to manage the INTESA services and support the primary users, directly from their predecessors and through the user manuals provided by the project consortium.

4. Experimental Results: Services’ Reliability and Users’ Compliance

In a mid- and long-term experimentation of e-health and personalised monitoring services, the first analysis must be dedicated to the services’ reliability, in terms of amount of data collected with respect to what expected from the experimental design, how much of this data is unreliable, both due to technical and user issues and, eventually, how the reliable data provides innovative results on the health and well-being status of the monitored persons.

INTESA services’ reliability and users’ compliance were evaluated for each single service, both for individual users and the entire group. We defined two main metrics:

• Data Availability ($D_A$) represents the adherence rate of the users to the single sessions of each service, and it is calculated as the ratio between the number of collected measurements and the expected ones. The measurement unit changes for the different services (e.g., number of monitored days, number of active sessions, number of reported meals).
• Data Validity ($D_V$) represents the percentage of collected data that are also valid results of the monitoring sessions. It represents the number of measurements that have not been discarded due to technical problems, failures and malfunctions of each single service.

Cumulative results for the entire user group are reported in

Table 2. Aggregated service statistics within the respective trial periods. For the last two columns, we also show the reasons why the values are not 100% (PU, SU, or TI).

Service	# Involved Users	Trial Period	# Expected	# Collected	# Collected	# Valid
					# Expected	# Collected
					[reason]	[reason]
Sleep	17	05/03/2018 - 05/03/2019	4377	3143	71.8% [PU,SU,TI]	98.34% [TI]
Stability	8	07/03/2018 - 20/12/2018	48	36	75% [SU,TI]	100% [-]
Cognitive	11	07/03/2018 - 07/01/2019	110	26	23.6% [SU]	100% [-]
Motor	11	07/03/2018 - 07/01/2019	110	10	9.1% [SU]	100% [-]
Stress	10	05/03/2018 - 22/05/2018	20	19	95% [PU]	100% [-]
Nutritional (Preferences)	17	26/03/2018 - 08/03/2019	8368	7377	88.2% [PU,SU]	100% [-]
Nutritional (Questionnaires)	17	26/03/2018 - 08/03/2019	16736	12030	72% [SU]	100% [-]
Body Composition	8	26/03/2018 - 03/09/2019	498	324	65.2% [PU,SU]	92% [PU]
Social Interactions	6	10/12/2018 - 02/02/2019	300	67	22.3% [PU,SU]	54.1% [TI]

, divided by the different services. Specifically, the table contains the following items for each row:

• Involved users: the number of enrolled users for the specific service.
• Trial period: the time interval during which the service was active for monitoring. It started from the first day of the service deployment, and lasted until the last day of the service activity.
• Expected measurements: the number of expected measurements for each service in the trial period. For each service, the study protocol defined the ideal number of measurements in order to obtain a complete evaluation of the involved parameters.
• Collected/Expected (%): percentage of collected over expected measurements as a measure of Data Availability ($D_A$).
• Valid/Collected (%): percentage of valid over collected measurements as a measure of Data Validity ($D_V$).

In addition, in order to provide an immediate analysis of the reported results, we add some notes to explain the main motivations for each service that could be related to: (i) Primary User (PU), e.g., resistance to the involvement in the proposed activities, absences due to personal reasons, etc.; (ii) Secondary User (SU), e.g., difficulties in the interaction with the devices/softwares, too much training effort, etc.; (iii) Technical Issues (TI), e.g., unmanaged missing connectivity, hardware faults, software bugs, etc..

It is worth noting that it was possible to enrol the maximum number of primary users (i.e., 17 frail older voluntary subjects) for those services that require mainly a passive monitoring activity, without any specific effort by the users themselves. Specifically, the sleep quality and nutritional monitoring have the maximum participation and registered the minimum resistance by the primary users. For the other services, the main requirements to maintain autonomously the standing position for a few seconds, and the ability to move inside the nursing home, became discriminant factors for the primary users enrollment, reducing the number of involved users to 8 for stability and body composition. For the other services related to stress monitoring and cognitive-motor rehabilitation we enrolled exactly the number of users requested by the study protocol. Instead, as far as social interactions monitoring is concerned, we initially recruited eight users, but two of them, involved only in this service, stopped using the device a few days after deployment. For this reason, we reported only data relating to the remaining six users.

In the following paragraph, we detail the results obtained for the different services and the main motivations.

As far as the sleep monitoring service is concerned, a measurement represented a daily indication of the sleep quality, which was based on 24 hours of continuous monitoring. For this service, the expected data was represented by the overall number of days during which the service should have been active in the bedrooms of the involved users. The collected measurements were instead the number of days during which the service actually acquired data. The main reason for which the sleep monitoring service did not collect all the expected data is mainly attributed to the voluntary and involuntary disconnection of the recording devices from the mains. In fact, during the initial period of the trial, some of the primary users voluntarily switched off the devices since they were disturbed by the light, although faint, generated by the onboard LEDs. This issue was quickly fixed by modifying the devices as soon as the problem was raised. Another reason the users disconnected the devices was the need for a power plug to charge their personal devices; in this case, multiple power plugs were provided whenever this situation was detected. Regarding the secondary users, the common reasons for the device disconnections were due to the bed cleaning activities, during which the operators inadvertently unplugged the devices power supplies. The sleep monitoring devices are prototypes and still require further enhancements to their design in order to reduce the care asked of the user, during their usage. Eventually, a minor source of data loss is due to occasional hardware failures caused by the wearing out of the devices. Concerning the validity of the collected data, Table 2 shows a very high percentage of meaningful measures, proving the reliability of the service.

Regarding the stability monitoring service, this is a measurement consisting of a single execution of the BERG #10 exercise while standing on the Nintendo Wii-fit balance board. The board is connected to a custom software designed to collect, visualize, and send data to the INTESA back-end. The chosen exercise consists of stepping on the balance board, turning to look behind over left and right shoulders while standing, and then getting off the board. The whole sequence is recorded on a laptop by the operator attending the session by means of an application that receives the body weight distribution over the four angles of the balance board in real time. The collected data are then sent to the remote server for the stabilometric assessment process. In this context, the expected data represents the amount of exercises that should have been carried out during the trial based on the INTESA study protocol, while the collected data is the amount of exercises received by the platform. As shown in Table 2, the difference between the expected and the collected measures consists in 25% of data loss, and it is mainly due to the process involving the operator and the recording application. This leads to future improvements to the application, concerning both the graphical user interface and the automation of the data saving and transmission procedure.

As far as the cognitive and motor activity is concerned, the number of expected data represents the number of expected sessions for the entire group of primary users, while the collected data represents the number of executed sessions. These services were partially penalized by insufficient technical support to the secondary users, in terms of both timing of the sessions and required training effort. Specifically, the time required for the initial setup of the sessions was not compatible with the very tight deadlines of the nursing personnel’s daily agenda.

For the stress monitoring service we defined one measurement as the result of a monitoring session of physiological signals during cognitive training activities supported by physical exercise or rest. The main goal of this service is to evaluate the short-term improvement of physical exercise on the cognitive performance of frail elderly subjects, while simultaneously assessing the physiological stress response to the proposed training. We designed a randomized cross-over study in which each of the 10 involved users participated in two monitoring sessions, thus resulting in 20 expected sessions. Nine users successfully participated in both sessions, while one user refused to complete one session, thus resulting in 19 collected sessions ($D_A=95\%$). All the collected measurements are also valid ($D_V=100\%$).

The nutritional monitoring service included two types of measurements, both collected by using DEW mobile application:

• number of menu choices (user preferences);
• number of meal questionnaires reporting dish consumption.

The former refers to the number of main meals (i.e., lunch and dinner) chosen by the users according to the daily menu proposed by the internal canteen, while the latter refers to the number of questionnaires related to meal consumption for the four daily meals (i.e., breakfast, lunch, snack, and dinner). In the case of menu choices, each user should contribute two reports per day within the trial period, while four questionnaires should be filled in for each day. Sundays were excluded from the trial since the service was not active, resulting in a time interval of 299 days. The rationale of this decision is that on Sunday the canteen provides special menus and some of the users receives external visits, often bringing additional food or home specialties.

The DEW app is the service that received the highest compliance from both primary and secondary users. In the first case, probably because their interaction with the service is mediated by the care giver; the second because the app integrates also the communication with the canteen as far as the guests’ choices, simplifying the management of orders. Therefore, DEW registered 57 enrolled guests within the trial period, with different timing, due to drop-out, new entrances and temporary residences. To provide a comparable analysis with the other services, in this paper we present the results derived from the use of DEW by 17 users, the same enrolled for the other services. The percentage of collected data for both menu choices (88.2%) and meal questionnaires (72%) are satisfactory considering a one-year trial period. DEW mobile application usage for collecting menu choices outperforms filling the questionnaires, thus indicating that the proposed mobile solution is a valuable tool to automate and simplify the process of collecting meal choices and sending orders to the canteen. Conversely, secondary users show less compliance in filling in four daily meal questionnaires since several questionnaires have not been compiled, especially those related to breakfast and dinner. This may suggest that the questionnaires compilation represents an overhead for the secondary users, which could be alleviated by integrating all meals report in a unique questionnaire per day to be managed, probably, after dinner.

As far as the weight and body composition monitoring is concerned, each measurement represents the bioimpedance scale result of a single user. Data provided by the bioimpedance scale are transmitted to DEW app through a B-LE connection, and then transmitted to the remote server to be analysed. The number of expected measurements is established based on the study protocol, in which we considered 3 measurements per week for each user within the trial period. For this service we originally enrolled 10 users, while two dropped out during the trial period. Considering thus 498 expected measures for 8 users, we collected 65.2% of the expected data, which were then filtered to remove outliers. These can be due both to postural and balance deficits that do not allow the user to perform the measurement correctly, e.g. if the user is unable to maintain a stable standing position during the measurement and/or when she leans on other persons and/or surrounding objects. In addition, we observed also unreliable data in case the user refuses to step on the scale without socks (condition suggested also in the user manual). Therefore, after this analysis we removed 26 outliers, obtaining 92% data validity. In this case, the service reliability is thus mainly related to a difficult interaction between the primary users and the device, which generates a lower compliance with respect to the other services.

Finally, for the social interactions monitoring service we considered (for each subject) the number of days for which at least one measurement was collected through the indoor localization system. Since it is not possible to estimate neither the expected daily number of indoor localization measurements, nor the expected daily number of social interactions, we assumed that each user should have contributed with at least one indoor localization measurement per day within the trial period (50 days). Unfortunately, only 67 days presents the minimum measurements for the involved users, which represents 22.3% of the expected ones, resulting in a total of 55,741 collected indoor localization measurements. This data clearly points out a poor adherence of the six involved users in continuously wearing the proposed device during the day, despite the customised design both for wearing and recharging it. Secondary users reported that many users did not wear the wearable device for all day long, or they used it until it was out of charge and forgot to recharge it for several days. In addition, the system was not designed to provide an immediate feedback to the primary users, and some of them declared that they do not understand the real advantage of using it. These results suggest an improvement of the devices’ design and autonomy (avoiding daily charge), and the need for personalized and immediate feedback on the user behaviour to better understand the utility of this service.

Single User Data Availability and Validity

To provide a detailed view of data availability and validity,

Table 3. Data availability for each user.

User	Sleep	Stability	Cognitive	Motor	Stress	Nutritional		Body	Social
ID						Prefs	Qnrs	Comp.	Interactions
1	23.3%	25%	20%	10%	100%	99.1%	93.3%	126%	-
2	48.4%	-	-	-	-	88.1%	68.6%	46.8%	-
3	54.3%	-	-	-	-	89.3%	75.3%	-	-
4	80.8%	50%	30%	10%	100%	88.1%	59.8%	69.6%	22%
5	58.0%	75%	40%	10%	100%	88.3%	59.7%	53.6%	12%
6	30.7%	-	20%	20%	100%	81.5%	75.6%	-	-
7	69.6%	100%	20%	0%	-	78.1%	64.6%	-	-
8	69.5%	-	10%	10%	50%	88%	59.2%	-	24%
9	74.8%	-	-	-	-	88.3%	69.6%	-	-
10	72.2%	75%	40%	20%	100%	88.6%	58.7%	66.7%	-
11	100.0%	62.5%	20%	10%	100%	97.4%	87.9%	50.9%	-
12	45.0%	100%	20%	0%	100%	88.6%	59.4%	88.4%	10%
13	63.7%	-	-	-	-	90%	76%	-	-
14	43.2%	50%	30%	10%	100%	89.1%	75.8%	53.4%	30%
15	100.0%	-	-	-	-	100%	91.7%	-	-
16	44.8%	-	10%	0%	100%	89.1%	75%	-	36%
17	80.3%	-	-	-	-	87.6%	67.5%	-	-

and

Table 4. Data validity for each service and user.

User	Sleep	Stability	Cognitive	Motor	Stress	Nutritional		Body	Social
ID						Prefs	Qnrs	Comp.	Interactions
1	100%	100%	100%	100%	100%	100%	100%	97%	-
2	100%	-	-	-	-	100%	100%	71.9%	-
3	100%	-	-	-	-	100%	100%	-	-
4	100%	100%	100%	100%	100%	100%	100%	97.9%	66.6%
5	99.5%	100%	100%	100%	100%	100%	100%	100%	0%
6	100%	-	100%	100%	100%	100%	100%	-	-
7	98.6%	100%	100%	100%	-	100%	100%	-	-
8	99.6%	-	100%	100%	100%	100%	100%	-	58.2%
9	94.1%	-	-	-	-	100%	100%	-	-
10	97.9%	100%	100%	100%	100%	100%	100%	89.1%	-
11	94.4%	100%	100%	100%	100%	100%	100%	96.5%	-
12	98.9%	100%	100%	100%	100%	100%	100%	93.4%	75.7%
13	100%	-	-	-	-	100%	100%	-	-
14	97.2%	100%	100%	100%	100%	100%	100%	86.5%	46.8%
15	100%	-	-	-	-	100%	100%	-	-
16	100%	-	100%	100%	100%	100%	100%	-	34.8%
17	97.0%	-	-	-	-	100%	100%	-	-

present the percentages per each user and service. It is worth noting that if there is no value for a specific service, it means that the user has not been enrolled for that service.

Concerning the sleep monitoring service, we can see that for some users the percentage of collected data is much lower than others. This aspect is strictly related to how primary and secondary users interacted with the system, as commented in the overall case. Specifically, some users reported to be more annoyed than others by the light emitted during the night. Also the position of the bed/power outlet in some rooms could have represented an obstacle during the daily cleaning service, leading to unaware disconnections. Another cause of data loss was the occasional corruption of the memory cards installed inside the recording devices. This issue was caused by their intensive usage over time. In fact, these kinds of events, absent in the first few months of activity, happened with more frequency as the monitoring period increased.

Regarding Stability, Cognitive, and Motor services, they all show a significant decrease in data availability for almost all the users due to the study protocol itself. For an effective evaluation of the interaction among the different domains during the experimentation, at first, the protocol was designed to be as much comprehensive as possible, i.e., both motor and cognitive sessions at the same time, leading to a too big amount of time requested to complete each session with respect to the general daily activities of the subjects. After a modification of the protocol, each user performed a single typology (once cognitive, once motor) per session, together with the stability test. These protocol updates led to a loss of sessions, also because secondary users had to be trained again to properly use the provided devices and software. Furthermore, during the experimentation, there were some shift changes in secondary users that required additional time for training the new personnel. On the technical side, the application developed to record the stability exercises, experienced a few problems concerning both the instability of the laptop wireless connection with the balance board, and the usability of the graphical user interface, contributing to the loss of some sessions. All these aspects were taken into consideration for improving the user experience in the future versions of the application.

As far as the stress monitoring is concerned, all the involved users (except userID=8), successfully attended all the training sessions, so their service availability is 100%, while userID=8 decided to participate only in one session due to personal problems.

Nutritional data regarding meal choices exhibits a more stable pattern than meal questionnaires, with percentages ranging from 78.1% to 100%. Percentages of collected meal questionnaires show a wider range of values ranging from 58.7% to 93.3%. Moreover, for each user the data availability for meal questionnaires is less than that related to meal choices. These results are mainly due to the effort requested of the secondary users, which exploits the mobile application for the automatic canteen orders (resulting in a good adherence to meal choices), while neglect the questionnaires part.

Regarding body composition, we can notice that userID=1 contributed with a greater number of measurements than expected, thus resulting in a data availability of 126%. userID=12 reached a data availability greater than 88.4%, while the minimum value is obtained for userID=2 (46.8%). For all the other involved users the collected measurements range from 50.9% to 69.6% of those expected.

Finally, the social interaction monitoring service exhibits the worst performance in terms of data availability, with percentages ranging from 10% to 36%.

In terms of data validity for each user and service, as shown in Table 4, we note that all the services reaches the 100% data validity for most of the involved users, except for body composition and social interactions services. This is mainly due to a reduced compliance of the primary users with those services.

Specifically, for the sleep monitoring service, valid data represents the amount of days during which the system has been active and in which the collected data are meaningful, i.e. they do not contain any artifacts due to sensor or system malfunction. Presented results shows a satisfactory data validity among all the involved users, ranging from 94.1% to 100%. The few values under 100% are due to technical issues as reported in Table 2. In particular, some of the FSRs broke during the experimentation due to the fragile nature of the sensors themselves. The project team verified every week the sensors status, and when one of them was broken it was rapidly changed; however we lost accuracy in the data for the previous week. Instead, for the stability assessment, valid data represents the amount of fully completed BERG 10 exercises on the balance board. As it can be seen, all the involved users were able to successfully execute all the exercises included in the proposed training protocol. The same applies to cognitive and motor assessment services.

All the data gathered from the stress monitoring services are valid (100% validity). Specifically, we measured the data packet loss in the wireless communication between the wearable devices and the mobile node (smartphone), which resulted $\le 0.1\%$ for all the sessions. Furthermore, every signal time series (i.e., HR, HRV, and EDA) was preliminary evaluated in order to assess the percentage of reliable data samples. Each session resulted in $\ge 95\%$ reliable data, thus no session needed to be repeated.

As previously explained, nutritional data were collected directly by the secondary users by exploiting DEW mobile app. Meal choices were registered according to user preferences with respect to the proposed daily menu, while meal consumption was observed and reported by compiling DEW meal questionnaires. This service relied on a trusted ground truth based on caregiver annotations and observations, so data can be considered 100% valid for both meal choices and questionnaires.

Regarding the body composition service, userID = 2 exhibits the lowest reliability (71.9%), with 9 outliers on 32 collected measurements. For the others users the service reliability ranges from 86.5% to 100%, with several detected outliers between 1 and 5. This behavior is due to a reduced compliance of the primary users and some difficulties in completing the task in a stable standing position on the balance.

Lastly, for the social interactions monitoring service, valid data were computed by discarding invalid indoor positioning measures. An indoor positioning measure was considered invalid if it contains a negative value for X and/or Y coordinates, or if X and/or Y values are out of range by considering the dimension of each room. These localization measurements are not valid and they cannot be analyzed in order to infer social interactions among users. Again, this service exhibits the worst performance also concerning data validity. We can notice that userID = 5 do not have any valid data within the trial period, while for the other 5 users data validity ranges from 34.8% to 75.7%.

Experimental results obtained in terms of services reliability are further justified by the analysis of the user acceptance and quality of experience we present in the next section.

5. User Acceptance and Quality of Experience: Mid-Term and End-Term Evaluations

We evaluated the user acceptance and QoE of INTESA project and services in two different time periods: June 2018 (i.e., four months after the deployment), and one year later, to better understand the evolution in the user perception of the proposed solutions. In our previous paper [15], we presented the results obtained in June 2018 by 10 primary users and five secondary users. Unfortunately, during the experimentation period there were some drop-outs and not all the same guests were present in the LTC facility after one year, but they were substituted by new enrollments. For this reason, in order to present consistent data on primary users evaluation, we selected only data provided by the same users in the two reference dates (June 2018 and June 2019). In addition, considering INTESA objectives, we decided to use the questionnaire proposed by Mosteghel and Oghazi [22] to assess the user acceptance of gerontechnology [23]. To simplify and adapt the questionnaire to the primary users population, we created a sub-list of eight questions (

Table 5. User acceptance (primary users).

Item	Avg. Score June 2018	Avg. Score June 2019
UA1: I find INTESA useful for my daily activities	3.83	3.0
UA2: I find INTESA wearable technology easy to use	3.5	3.0
UA3: I find INTESA activities easy to perform	2.66	3.16
UA4: I feel calm and confident when using INTESA technology	2.33	1.66
UA5: I feel calm and confident when performing INTESA activities	1.33	1.33
UA6: I feel able to complete INTESA activities	2.5	3.16
UA7: I feel in a good health status	2.0	2.33
UA8: I feel satisfied participating in INTESA	3.6	2.83

) related to the seven areas of the gerontechnology acceptance questionnaire, namely Perceived Usefulness, Perceived Ease of Use, Gerontechnology Self-Efficacy, Gerontechnology Anxiety, Cognitive Ability, Physical Function, and Self-reported health conditions, plus a general question about the overall satisfaction of the user. All the items were measured on a 5-point Likert Scale (1 = strongly disagree, 5 = strongly agree).

Table 5 presents the average results for both evaluations so that we can discuss the initial and final perception of the primary users involved for the entire one-year period. It is worth remembering that in December 2018, the localization system was deployed as a support to the social interactions monitoring service, requiring thus the primary user to continuously wear (during the day) an additional device (i.e., an UWB tag). This device has not been positively accepted by the users, mainly due to its management issues (i.e., battery daily recharge, continuous wearing), which requested an active role of both primary and secondary users (as support to primary users). In addition, in the same period, four secondary users left the LTC facility and they were substituted by new personnel.

Looking at the primary users’ results, we can observe that the utility perception of INTESA services decreases from very useful to useful (from 3.83 to 3 for item UA1), according to the fact that also the ease of use of the wearable technology decreases (from 3.5 to 3 for UA2). However, the ease to perform the specific activities slightly increases (UA3). This evolution is mainly due to the introduction of the new devices and the related activities, as also reported by some of the primary users. In addition, even though the users’ self-confidence in completing INTESA activities lightly increases (from 2.5 to 3.16 for UA6), the users report an increased apprehension in the use of the technology (UA4) and in the activity performance. At the same time, the self-awareness of the frail health status lightly increases (from 2.0 to 2.33, UA7) and there is a short reduction of the users’ satisfaction (from 3.6 to 2.83), while maintaining a quite neutral evaluation. All these results reflect the negative evaluation of the new devices by the primary users.

In addition, in order to quantify the Quality of Experience (QoE) of the overall INTESA system, we interviewed the secondary users by exploiting the System Usability Scale (SUS) questionnaire [24] at the end of the experimentation phase. We used the same questionnaire presented in the previous work, so that we can compare the mid-term and end-term results. SUS provides a reliable tool for measuring the usability of a wide variety of products and services [25] and, for this reason, secondary users in charge of managing the structured activities of the project were asked to answer it. SUS is based on a 10 item questionnaire (

Table 6. Quality of experience (secondary users).

Item	Avg. Score June 2018	Avg. Score June 2019
QoE1: I think that I would like to use INTESA frequently	3.8	4.0
QoE2: I found INTESA unnecessarily complex	2.4	2.66
QoE3: I thought INTESA was easy to use	4.2	3.83
QoE4: I think that I would need the support of a technical person to be able to use INTESA	1.6	2.66
QoE5: I found the various functions of INTESA well integrated	4.2	3.33
QoE6: I thought there was too much inconsistency in the system	1.8	2.83
QoE7: I would imagine that most people would learn to use INTESA very quickly	4.2	4.0
QoE8: I found INTESA very cumbersome to use	3.0	3.66
QoE9: I felt very confident using INTESA	4.2	4.0
QoE10: I needed to learn a lot of things before I could get going with INTESA	2.8	3.5

) with each item presenting a five-option response using the same Likert Scale used in the gerontechnology acceptance questionnaire. It is worth noting that, in the two reference dates for SUS evaluation, the personnel has changed, specifically the two physiotherapists and two social health professionals, maintaining the same role and activities of the previous ones. Since the personnel change has not been communicated to the project partners, the new personnel did not receive a technical training in the user of INTESA services, while they followed the instructions of the previous operators, referring to the user manual delivered at the beginning of the experimentation. This resulted in a minor self-confidence of the new personnel in completing the requested activities.

Secondary users results highlight that INTESA services can be integrated in the daily routine of the LTC facility, even though they are considered quite complex and the evaluation of the ease of use is lightly reduced in the second period, probably due to the lack of training for the involved personnel. This motivation supports also the results of the subsequent questions, in which the user autonomy is reduced, the learning results to be more difficult, and there is an increase in the technical support requests, even though the technical issues result to be acceptable, referring to prototype services.

However, all the evaluations maintain neutral/positive scores, demonstrating the appreciation of the system by the secondary users and providing useful suggestions for further improvements.

6. Lessons Learnt

The INTESA experience provided us with important suggestions on how to improve the single services and technological solutions in terms of technical issues, usability and acceptability, as detailed in the description of the single services results. However, in general, the main feedback derived from the one-year experimentation is related to the integration of these novel and innovative solutions in a real LTC facility. It is fundamental to collaborate with the final users (both primary and secondary) since the beginning, integrating their requirements by design, considering both the personnel activities and time tables and the guests’ habits, which represent a fundamental part of the primary user acceptability. The service providers should provide a dedicated training program for the technical personnel of the LTC facility, so that they can train directly the nursing personnel and solve minor technical issues that can arise in the use of the single devices and services. In addition, all the services and the integrated system should provide periodical feedback to both primary and secondary users in terms of report on the health and well-being status of each guest, and motivational suggestions to improve habits and behaviour. We are currently working on this direction as further enhancement of INTESA system.

7. Conclusions

INTESA presents a suite of mobile and e-health services for personalised monitoring of heterogeneous health conditions and habits of frail older people. The main target of INTESA is to provide a complete integrated care approach by using innovative technologies and personalised solutions that can be used both at home, by older people able to live autonomously, and at LTC institutions. In the latter case, these facilities can improve their offer with additional monitoring and rehabilitation services to prevent risky situations, including cognitive and motor decline, depression and social isolation. To evaluate the efficacy of INTESA in a real scenario, we deployed its services in an Italian LTC facility hosting 60 frail subjects with about 30 people as nursing and social care workers. We experimented the system and most of the services for one year with a reference group of 10 active subjects and 5 controls. During the trial period, there were some drop-outs of primary users, due to medical and personal problems, which were substituted by new enrollments. In addition, there were also some changes in the LTC personnel directly involved in the experimentation. Moreover, a new localization service was introduced after several months from the beginning, to support the social interactions monitoring service. This required the primary users to wear an additional device (a UWB tag) and the secondary users to support the primary in the management of the device itself. All these conditions contributed to the final performance of the system, both in terms of technical reliability, user acceptance and quality of experience for both users’ categories. Specifically, this long-term experimentation demonstrated the reliability of the proposed solutions in terms of data availability and validity, and it provided important suggestions to enhance INTESA services to be effectively integrated in LTC care process. In fact, despite the encountered issues, the system was considered acceptable by primary and secondary users, maintaining a good perception of the guests about the utility and usability of the proposed services and a good satisfaction in their personal participation in the project. As far as the nursing care personnel is concerned, they found the system useful and a good support for the daily routine, even in absence of a technical training, which can further improve their engagement. Currently we are deeply analysing the data collected by the single services to evaluate their efficacy and impact on the health status of the involved subjects, and we are studying innovative solutions to automatically detect anomalous behaviour and provide personalised feedback to further empower frail older people.