HIV Patients’ Tracer for Clinical Assistance and Research during the COVID-19 Epidemic (INTERFACE): A Paradigm for Chronic Conditions

Abstract

The health emergency linked to the SARS-CoV-2 pandemic has highlighted problems in the health management of chronic patients due to their risk of infection, suggesting the need of new methods to monitor patients. People living with HIV/AIDS (PLWHA) represent a paradigm of chronic patients where an e-health-based remote monitoring could have a significant impact in maintaining an adequate standard of care. The key objective of the study is to provide both an efficient operating model to “follow” the patient, capture the evolution of their disease, and establish proximity and relief through a remote collaborative model. These dimensions are collected through a dedicated mobile application that triggers questionnaires on the basis of decision-making algorithms, tagging patients and sending alerts to staff in order to tailor interventions. All outcomes and alerts are monitored and processed through an innovative e-Clinical platform. The processing of the collected data aims into learning and evaluating predictive models for the possible upcoming alerts on the basis of past data, using machine learning algorithms. The models will be clinically validated as the study collects more data, and, if successful, the resulting multidimensional vector of past attributes will act as a digital composite biomarker capable of predicting HIV-related alerts. Design: All PLWH > 18 sears old and stable disease followed at the outpatient services of a university hospital (n = 1500) will be enrolled in the interventional study. The study is ongoing, and patients are currently being recruited. Preliminary results are yielding monthly data to facilitate learning of predictive models for the alerts of interest. Such models are learnt for one or two months of history of the questionnaire data. In this manuscript, the protocol—including the rationale, detailed technical aspects underlying the study, and some preliminary results—are described. Conclusions: The management of HIV-infected patients in the pandemic era represents a challenge for future patient management beyond the pandemic period. The application of artificial intelligence and machine learning systems as described in this study could enable remote patient management that takes into account the real needs of the patient and the monitoring of the most relevant aspects of PLWH management today.

1. Background

People living with HIV infection (PLWH), particularly those with immunodeficiency and immune dysregulation, may be at increased risk of morbidity and mortality during SARS-CoV-2 infection [1]. Although the literature data do not appear to be in complete agreement, several papers have documented an increased risk of severe disease and death associated with HIV cofactor during COVID-19 infection [2,3,4,5].

In Italy, the impact of the COVID-19 pandemic on National Health Service facilities has primarily involved infectious disease facilities, with potential consequences on HIV diagnosis, treatment, and prevention. The model for the management and control of HIV infection in Italy has been based, since the development of Law No. 135/90, on the central role of the infectious diseases’ structures, through an articulation of care services in acute inpatient wards, day hospital structures, dedicated outpatient clinics for the taking charge and treatment, and integrated home care structures.

The results obtained by the entire care system dedicated to the treatment of PLWH risk being compromised by the impact of COVID-19 on the National Health Service, and in particular on the infectious diseases structures, which are central to the strategy of intervention and control of the new pandemic. The negative impact may involve both the provision of care in acute wards, in particular for people with newly diagnosed HIV and late presentation (AIDS presenters), as well as outpatient facilities for the care and management of chronic patients with stable HIV infection, with possible losses in follow-up and reduced continuum of care.

This scenario could persist in the continuation of the COVID-19 pandemic, resulting in serious harm to people living with HIV, as reported by global health agencies (WHO, UNAIDS) [6,7], the European Parliament, and the European Commission, which already pointed out a step back from the WHO 90-90-90 target and fear a strong risk of failure to meet the 2030 SDG targets, reminding governments of the importance of ensuring HIV care and prevention services even in these times of COVID-19 emergencies.

The “Istituto Superiore di Sanità” has drawn up a plan that foresees and encourages the use of telemedicine to allow hospital structures to use tele-consultation and remote management of chronic patients [8]. Tele-consultation and telemedicine in general are applications that allow, in these moments when travel is by definition limited, to monitor patients suffering from chronic pathologies, who need regular and constant care and control. This concept has been stressed also by international guidelines on the management of PLWH [9,10,11,12].

People living with HIV/AIDS (PLWHA) represent a typology of patients for whom telemedicine could, particularly in pandemic times, represent an extremely useful tool of support and clinical management [13,14,15,16]. People living with HIV today are people who have a life expectancy very similar to that of the general population, due to the extraordinary effect that antiretroviral therapies have produced on survival and on the reduction of HIV-related morbidity [17]. Nowadays, PLWHA patients face a chronic condition with all the consequences that this situation entails (symptoms, side effects of the drugs, management of adherence and quality of life, periodic supply of drugs, blood samples to check the tolerability of the drugs, any new clinical events that may occur, needing advice from other specialists, etc.).

Since the end of February 2020, the Fondazione Policlinico Gemelli continues to represent a reference center for the care of patients affected by COVID-19. For this reason, during these months the outpatients’ clinics of the FPG that take care of patients with chronic pathologies have been able to provide services and visits only if mandatory, with inevitable repercussions on the management of chronicity in all its aspects.

With the persisting of pandemic restrictions, it is essential to find ways of managing these patients remotely to allow continuity of care that will inevitably not reflect what happened in pre-epidemic periods. Moreover, it seems to be crucial in this period, to quickly identify potential sources for spreading of SARS-CoV-2 diffusion and transmission among potentially high-risk population such as PLWHA. In addition, the pandemic experience has provided insight into how the pressure on the healthcare system from carefully selected patients affected by chronic diseases can be mitigated through the use of remote care systems without losing a proper doctor-patient interaction.

For these reasons, we designed a pilot study for PLWH with stable chronic disease, with the objective of creating an “integrated clinical assistance” through an efficient operating model to “follow” the patient, capture the evolution of his/her disease, provide assistance and care, and interact also to establish proximity and relief through a remote collaborative model. Here, we describe the entire protocol including rationale and design of the study, with particular regard to the technical description of the innovative aspects of integrated care inherent in the study itself. Moreover, some very preliminary results on the learning of models to predict alerts of interest are shown.

2. Methods

2.1. Study Design, Duration, and Setting

This is an interventional monocentric study. The first phase of the study has been based on the set up of a mobile application for both Android and iOS that can be downloaded on the patient smartphone. This app is set up with functionalities to support and achieve objectives such as health and quality of life monitoring, antiretroviral treatment adherence, and assessment of SARS-CoV-2 infection risk.

In a future phase, it is foreseen to connect data self-reported by using the application and the clinical data from the electronic reporting system of the hospital to provide the clinicians with an overview as complete as possible.

The different monitoring aspects of the system are shown in Figure 1 and include:

(1)

Self-reported prescreening on symptoms and signs compatible with SARS-CoV-2 infection, possible access to healthcare facilities, possible results of swabs/serologies, and evaluation of hospitalization risk due to COVID-19, on the basis of selected self-reported stress tests.

(2)

Specific self-reported aspects to monitor patients’ life (quality of life, anxiety, depression, HIV-related symptoms, adherence to treatments) through triggered standardized questionnaires, to be filled in by patients at specific timepoints. Additional questionnaires will be then triggered according to patients’ answers to the first generic questionnaire (EQ 5D-3L) identifying specific unmet needs, with a sequential approach based on a predefined algorithm. All reported outcomes, triggered or self-reported, will be always accessible on the app in a dedicated area, the “health diary”, for patients and on a dedicated dashboard for physicians. Patients’ self-reported data also generate tailored alerts on the basis of their actual needs and predefined scores in order to provide physicians with useful information for patient management.

(3)

Important parameters for drug management (BMI, MDRD). These data will be calculated periodically on the basis of individual lab results and they will be used to adjust dose of drugs.

(4)

Drug adherence is monitored through linkage of self-reported information on adherence with hospital pharmacy’s refill data and laboratory results (plasma HIV/RNA).

(5)

Periodic blood sample control through an alerting system of laboratory results that, in the case of abnormal values, will warn physicians via tailored alerts.

(6)

Any new onset of symptoms/side effects: patients can report any symptoms/side effects on a specific area of the app, and this report will appear as an alert to the physician.

(7)

Support tools for patients such as FAQs, User Guide, and disease-specific information.

(8)

Notifications containing different kind of information and/or recommendations are automatically sent to patients on the basis of their answers to questionnaires, lab results, or drugs refill.

(9)

An automated feedback system to the treating physician to highlight those situations that need attention. This includes self-reported symptoms (i.e., bothersome symptoms), any self-reported mental health issues, any changes in periodic laboratory tests, any problems with adherence to antiretroviral therapy, and any symptoms or signs consistent with a risk of SARS-CoV-2 infection.

2.2. Study Population

All PLWHA, on antiretroviral therapy treated in Fondazione Policlinico Universitario “A. Gemelli” IRCCS (FPUAG IRCCS) of Rome, Italy, who will be considered eligible, will be enrolled and asked to install the app on their smartphone.

2.2.1. Inclusion Criteria

• Patients’ age >18 years.
• Patients able to use apps and smartphone devices without any caregivers or, in the case of patient unable to use them, patients with caregiver able to use apps and smartphone devices.
• Patients able to provide informed consent.

2.2.2. Exclusion Criteria

• Patients not able to use smartphone devices and applications without the presence of caregivers.
• Recent diagnosis of HIV infection (<3 months).
• Any unstable clinical condition requiring hospitalization.

2.3. Data Collection

The study is based on standardized data collection procedures able to efficiently process large amounts of data and provide the physicians with a panel of useful information to evaluate patients’ health conditions, quality of life, mental health, how HIV impacts their QoL, and to monitor eventual worsening.

Five different data sources were used for data collection: 4 internal to the electronic data reporting system of the hospital and 1 external (Healthentia app).

From the electronic data reporting system of the hospital, we defined eight structured variables (laboratory exams, drugs refill from pharmacy, weight, height, gender, study degree, civil status).

A standard ETL procedure was developed for the extraction of such data using SAS Institute software analytics tool and SAS^® Vyia^® (Cary, NC 27513, USA), which refreshes on daily basis in order to continuously include both new enrolled patients and new data from patients already registered to the app (new laboratory exams, drugs refill for pharmacy, etc.).

Hospital data are integrated with real-world data coming from the Healthentia app (Innovation Sprint Sprl, Brussels, Belgium), through which will be collected symptoms potentially related to SARS-CoV-2 infection, symptoms related to HIV, adherence to treatments, quality of life, anxiety, and depression. The collected data are already outlined in Figure 1. They are collected via questionnaires (validated and custom ones) and during a functional test.

We employ validated questionnaires for HIV-related symptoms through the questionnaire “HIV Symptom Rating Questionnaire (HIVSRQ)”, health-related quality of life through the questionnaires “EQ-5D-3L” (comprising the following five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression), anxiety through “General Anxiety Disorder-7 (GAD-7)”, and depression through “Patient Health Questionnaire-9 (PHQ-9)”.

We employ custom (non-validated) questionnaires to collect adherence to antiretroviral therapy, SARS-CoV-2 infection assessment (tracking of symptoms related to SARS-CoV2 infection for its early detection), the result of a swab test, and the SARS-CoV-2 vaccine.

Finally, we employ the chair stand test (functional test), during which the oxygen saturation (one of the most important parameters to assess the risk of hospitalization for patients with SARS-CoV-2) is monitored.

Each questionnaire mentioned above has many questions, i.e., results to many attributes of the data. The question of which of the attributes is of high importance in predicting the outcomes of interest is addressed by analyzing the decisions of the learnt models in Section 4.3.

Data will be collected for a duration of 2 years starting from 1 January 2021. About 1500 patients currently followed up at the Infectious Diseases Clinic met the inclusion criteria. These patients represent the share of patients without urgent clinical needs or whose periodic medical examination can be carried out every 4–6 months.

On the basis of previous experiences on such a kind of studies, we found that up to 30% of the patients could be interested in being enrolled in such a study and using Healthentia. As such, with expected recruited population of 450 patients, we aimed at analyzing at least 220 patients who actively used the app in order to accurately describe our recruited population with a confidence interval of 95% and margin of error of 5%.

Unfortunately, the pandemic has only allowed us to recruit 61 patients at this point, since the beginning of the study in January 2021. Recruitment is still active though, with the latest patients being enrolled in November 2021 in an ongoing process. The enrolment process across time and some demographic information about the study population is given in Figure 2.

2.4. Workflow Structure

INTERFACE is an interventional study with a purpose of collecting, as previously described, real-world data (RWD) regarding patient outcomes and parameters. In order to be enrolled to the study, patients have to sign an informed consent form declaring their awareness about data sharing for research purpose or use the eConsent process available on the app. Patients are enrolled on the study by downloading the application for free from the AppStore or Google Play, installing it on their own smartphone and registering. Patients can retire themselves from the study by uninstalling the application without any formal communication, and they can obtain the definitive elimination of collected data at any moment.

Data flow from the Healthentia app to a protected Microsoft Azure Server cloud environment where Healthentia SaaS Solution is hosted by Innovation Sprint, and there is a point-to-point cryptographic encryption of the data that makes it impossible for third parties to read the information in transit in clear text. FPUAG IRCCS, as the data controller with the right of access, use, and management of data for research purposes, has access to Innovation Sprint’s servers.

The study is conducted in the framework of Gemelli Generator Real-World Data facility (G2 RWD), the innovative research center of FPUAG IRCSS. Within the Gemelli Generator Real-World Data (G2 RWD) architecture, INTERFACE data collected using Healthentia will be integrated with clinical data and stored in a dedicated research Data Mart (i.e., organized subsets of data on a specific area of knowledge) in FPUAG IRCCS servers. In coherence with G2 RWD roles, if necessary, selected supervisors will be authorized to access the pseudo-anonymized data and identify patients for eventual clinical needs according to the collected data.

2.5. Efficacy Indicators

In order to evaluate the effectiveness of this project, after 2 years, we will consider the following indicators:

• Proportion, characteristics, and cofactors of self-reported outcomes and changes in them according to the epidemic situation and during the time of the study.
• Proportion of clinical intervention provided following alert messages received.
• Proportion of unmet needs as generated by patients’ reported outcomes.
• Proportion of intervention on drug regimen modification according to parameters recorded.
• Number of PLWHA with self-reported symptoms/signs of COVID-19 who can be correctly and timely allocated to different in-hospital access.
• Prevalence of PLWHA regularly followed as outpatients with confirmed COVID-19.

3. Healthentia e-Clinical Environment

For the purposes of INTERFACE, a very comprehensive and adaptive e-Clinical environment, provided by the partner company Innovation Sprint, has been customized to meet the needs of INTERFACE protocol. The e-Clinical environment is based on Healthentia solution, which facilitates clinical trial optimization, accelerates trial processes, reduces failure rates, and validates drug/intervention efficacy and effectiveness with RWD insights. Healthentia is a Class I Medical Device with CE mark, a medical decision support software intended for monitoring of non-vital parameters to support decision making during clinical trials, according to RWD gathered from patients taking part in the clinical investigation.

Healthentia extends the use of traditional electronic patient-reported outcomes (ePRO)/electronic clinical outcomes assessment (eCOA) applications by adding lifestyle, behavioral, and health-related data collected from smartphones and internet of things (IoT) devices. Applying artificial intelligence (AI) and machine learning techniques on these data, one can discover behavioral biomarkers and cluster patients into behavioral phenotypes, which allows the activation of smart services for the prediction of clinical outcomes, the generation of prevention alarms, and the linking of phenotypes with intervention efficacy. Furthermore, on the basis of reported outcomes, the AI module is able to generate automatic alerts in the case of adverse events. These automatic and prevention alarms support decision making by the investigator during clinical trial for the benefit of the individual patient’s health. The main components that are utilized for the purposes of the INTERFACE study are:

ePRO/eCOA—The ePRO/eCOA component is responsible for the communication with the mobile application that runs in iOS and Android devices. The component has all functionalities and services that are consumed by the smartphone app via Healthentia API. The Healthentia app interacts with the ePRO/eCOA component to allow the patient from the comfort of their home to:

• fill in health-related and quality of life questionnaires from their device;
• log events for symptoms, school/work absence, treatment, or hospitalization, etc.;
• receive ad hoc messages from the system or the investigator of a study;
• receive automatic messages from the system or the investigator of a study;
• receive notification for filling in questionnaires or medication reminders;
• find protocol/treatment or device-related information;
• contact the investigator;
• use the chatbot functionality for questions and virtual coaching.

The ePRO/eCOA component also includes the questionnaire editor and scheduler. The questionnaire editor allows the investigator to create or edit questionnaires that will be delivered to the patients. The editor provides the functionality to create complicated questionnaires with advanced routing between questions, and even between questionnaires as well as multiple types of questions and user interfaces, e.g., single, multiple, image selection, location-based, etc. The questionnaire scheduler is the functionality that is used by the investigators to create rules for the automatic submission of questionnaires to specific recipients. These rules may include the registration date, or even the answers from the subject and the auto-tagging of a subject into a certain category.

Smart services—Further to the currently supported features of Healthentia, i.e., collecting data from patients, the wealth of information collected is used in real time by the Healthentia machine learning (ML) services to provide useful insights for clinical endpoints. On the basis of the patients’ data, models are learned both for predictive and generative purposes. Predictive models can estimate future clinical outcomes from current and past data. Once such predictors of outcomes are validated, they are considered biomarkers for the particular outcomes at hand. Unlike traditional biomarkers, these models form composite digital biomarkers by non-linearly combining various information attributes to predict the wanted outcome. Such predictive models are shown in Section 4.3, but the results there are considered preliminary, and the models are not yet validated into biomarkers. Generative models are built by modelling patient clusters and can be used to generate synthetic data. Different clustering algorithms can be used to derive the clusters, which can then be validated by verifying that all patients in a cluster exhibit some common outcome. When this is verified, the cluster models are validated into phenotypes. In Figure 3, the lifecycle of RWD in Healthentia is described.

The lifecycle of RWD in Healthentia can be described in six steps. In step 1, data are captured and visualized at real-time basis. In step 2, digital composite biomarkers are discovered, and patients are clustered using ML algorithms. For this purpose, physiological, psychological, and sociological data are used. In step 3, training of the selected patient model is performed by means of observational sequences from the trial in order to allow smart services to be applied, such as prediction of clinical outcomes and alarms (see step 4). At the end of a study, the investigator can further use the derived patients’ models to run in silico trials (see step 5), while the enriched models can be utilized for digital therapeutic (DTx) services, such as the orchestration of virtual coaching messages.

In the case of the INTERFACE study, patients will be grouped in phenotypes on the basis of ML processing of the RWD, and a digital composite biomarker will be derived, consisting of several dimensions, for each of the data points captured, e.g., questionnaire scores and scales. This digital composite biomarker will then be optimized by identifying which of the dimensions will have significant impact to the endpoints, and when an AI training process is followed, it will be used to predict future outcomes. The digital composite biomarker can then be used for future in silico studies and/or driving DTx decisions for HIV patients.

4. Data Analysis

The sample will be described using a descriptive analysis with the aim of analyzing how health conditions, mental health, and quality of life changes during the use of Healthentia.

Qualitative variables will be summarized with percentage frequency tables by analyzing questionnaires filled in by patients.

Quantitative variables will be described using the minimum, maximum, mean, median, standard deviation, and interquartile range; then, through histogram and box plot, we will see the shape of the data distribution. Normality of continuous variables will be checked using the Kolmogorov–Smirnov test.

Inferential techniques, defined after a data exploration phase, will be then used with the aim of implementing a Guardian Bot, a software able to automatically detect earlier predictors of a worsening trend of the main disease or of SARS-CoV-2 infection occurrence. In this manuscript, we only present preliminary results regarding the construction of a machine learning model for predicting the alerts shown in Figure 1.

4.1. Interface Integration with Generator Real World Data

INTERFACE will leverage on the overall architecture and capabilities of Gemelli GENERATOR Real-World Data facility (G2 RWD), the innovative research center of FPUAG IRCCS, with the principal aim to transform data and information into actionable knowledge while fully respecting the privacy, data integrity, and intellectual property for the benefit of all.

Indeed, if the main purpose of INTERFACE is totally in line with G2 RWD objectives: the exploitation of G2 RWD framework represents a quality guarantee for INTERFACE both in terms of data collection, quality, and analysis and of data privacy and protection. Actually, G2 RWD architecture has been approved by the Local Ethical Committee and widely used in FPUAG IRCCS with good results [23].

The architecture and methods that G2 RWD will contribute for INTERFACE are based on:

• the creation and update of a dedicated Data Mart (organized subsets of data on a specific area of knowledge) in which all data collected both for the app and from the electronic data reporting system of the hospital are correctly stored and organized, in order to be easily accessible and understandable for physicians;
• Mathematical and statistical tools available, ranging from the traditional qualitative-quantitative analysis techniques to the more advanced artificial intelligence algorithms, to provide new modeling hypotheses to be tested and validated with clinicians for research purposes;
• The data architecture, defined to provide the highest degree of protection, in accordance with all GDPR and security requirements.

4.2. Data Privacy and Data Protection

All privacy matters are analyzed with the Policlinico Gemelli Data Protection Officer (DPO) so that every G2 RWD study will be compliant with GDPR Italian and European directives and regulation (EU Directive 2016/679 and under Italian Laws: Decreto Legislativo 196/2003, Decreto Legislativo 101 2018, Autorizzazione Generale Garante 9/2016). The data architecture that supports RWD studies has been designed to provide the highest degree of protection, in accordance with all GDPR and security requirements. In this respect, RWD provides “protection by design” in each step of the process.

4.3. Learning Models to Predict Alerts

The questionnaire pipeline of Figure 1 is triggered once per month, and depending on the answers of the patients, the maximum number of questions answered is 255. When there is no need to answer some questions, they are assigned to the default “no problem” state. As a result of the answers given, four types of alerts might be issued:

• Frailty alert, binary (OK, alert);
• HIV symptoms alert, binary (OK, alert);
• Adherence alert, binary (OK, alert); and,
• Psychological/psychiatric support alert, tristate (OK, psychological, psychiatric).

Given the current state of the patient, i.e., the answers to the questionnaires of the current month, the alerts are fully determined. The machine learning (ML) goal of INTERFACE is to predict the alarms given the patient history. The history is determined as the questionnaire answers of part months, and it can span one month (predict the alarms in the current month given the questionnaire answers of the previous month) or more. The input vectors comprise the individual answers in a history window of n months, i.e., of $255·n$ attributes, $n=1,2,\dots$, while the output vectors comprise of the four attributes for the four alerts.

The patients answer the questionnaires on a monthly basis. Thus, a patient being n months in the INTERFACE study provides $N-n$ pairs of input and output vectors. Since there are not a lot of patients in the study, especially for long durations, large values of the history window drastically reduce the number of available vectors, albeit they increase the actual information each vector caries. The lack of adequate vectors to learn ML models is a typical situation where vectors are contributed on a monthly or even weekly basis.

Although the alerts are not really rare, raising them is not the common state. Hence, the state of the output attributes is far from balanced, with the majority of the vectors indicating “no problem” and only few of them resulting to alerts. Thus, the ML problem at hand is a heavily imbalanced one, and therefore the accuracy is not a good performance metric for the predictors. We employ balanced accuracy [24] instead.

The lack of data and the imbalance nature of the model learning task make it a particularly hard to tackle. The situation is depicted in

Table 1. Volume of data per history depth and outcome state.

Attribute	History (Months)	Counts per State
		#1	#2	#3
Frailty	1	186	13	N/A
	2	143	6	N/A
HIV symptoms	1	139	60	N/A
	2	100	49	N/A
Adherence	1	190	9	N/A
	2	141	8	N/A
Psychological, psychiatric	1	163	25	11
	2	124	18	7

N/A: not applicable.

. The number of available vectors (198 for $n=1$ or 149 for $n=2$) forbids the use of neural network [25] predictors of any usable depth. Training of random forests [26] though a few estimators is possible, and this is the selected ML algorithm training our INTERFACE predictive models.

For the same reason, the split of the available vectors in training, validation, and testing sets is not practical. Instead, we train and validate the models using k-fold cross-validation [27]. We split the data in k folds of size 5 each (39 for $n=1$ or 29 for $n=2$) and use $k-1$ of them for training and 1 for validation. We repeat the training 30 times for each fold selection and select the highest balanced accuracy of the 10 as the balanced accuracy of the fold. We average the balanced accuracies of the $k-1$ folds to obtain the balanced accuracy for one configuration of the random forest. We vary the number of estimators as different configurations and select the best of them all.

The number of estimators (trees) of the random forest are varied in the training. The tree creation parameters are as follows: bootstrap samples are used instead of the complete dataset when building trees, randomly selected at consecutive training sessions. The quality of tree node splits is measured using Gini impurity and best split is sought for considering the square root of the total number of features. Tree nodes are expanded until they contain a single sample, with the number of leaf nodes being unlimited. Even though the different class population is not balanced, no attempt has been made to assign different weights to the different classes.

The average training balanced accuracy for one month is 92%, while for two months it is the perfect 100%. Such a training gives us no generalization confidence however, and therefore we proceed with the k-fold cross-validation scheme discussed above. The resulting average balance accuracy across all four outcomes as a function of the number of random forest estimators is shown in Figure 4. Note that the maximum performance for two months of history generally increased for a larger number of estimators, while for one month, the algorithm found an optimum point at a small number of estimators. The reason that the cross-validation did not result in better results for two months of history is because of the fewer vectors available, with the problem at hand having double the features.

The decisions of the classifier are analyzed using Shapley Additive explanations (SHAP) analysis [28]. SHAP analysis is applied on every decision, yielding the effect of each attribute in the feature vector towards a positive or negative prediction. These attributes are questions in the different questionnaires, and thus the SHAP analysis actually yields the on average most important questions to ask for each outcome. This is shown in Figure 5.

Two notes are in order. Firstly, the ranking of the questions shown in Figure 5 is averaged across all the population. It is interesting to discover how this ranking varies when considering the different individuals. Unfortunately, there are too few different patients with many decisions (many months in the study) to conduct this analysis just yet. This brings us to the second note: Since the data we currently have at hand are limited, we cannot actually claim that the most important factors from a clinical viewpoint are found. Instead, at this point the analysis is more of a machine learning rather than of clinical importance. That being said, we observe that the overall health question of the ED-5D questionnaire is always high in importance. The frailty and adherence alerts seem to have a very large dependency on their most important attribute, the rest being quite lower, while the rest of the alarms have a more even dependency on the different attributes. Most of the important factors for the psychological/psychiatric alarm fall in the anxiety/pessimism categories.

Finally, the most important factors for the decisions based on one month history are compared to those based on two months in Figure 6. The attributes whose names finished with a “-1” were those of the oldest month. Note that many of the most important attributes were from the oldest month, with the current one not ranking high, such as the PHQ sleep, preference to die or lack of interest, or the GAD ability to control worries or to sit up. On the other hand, in some cases, the attributes of the two months of history appeared in pairs, such as the most important pair of EQ_5D health of the previous and this month or the EQ_5D pain.

5. Ethical Aspects

The protocol has been approved by Local Ethical Committee (Protocol n 3436).

6. Discussion

During the COVID-19 pandemic, medical care for PLWHA is at risk of being compromised, calling into question the excellent results achieved in this field. The main scientific societies have produced guidelines suggesting how to ensure an adequate standard of care during this phase, promoting different models of care involving tools such as telemedicine [29,30].

Moreover, during the COVID-19 pandemic, machine learning and artificial intelligence systems are being used increasingly more accurately to optimize the care of people, identify predictive factors for the diagnosis and prognosis of the disease itself, and for the monitoring of mental health [31,32,33,34].

In the protocol described here, we propose a study based on the integration of telemedicine and telemonitoring with a standardized approach translating the huge amount of data collected on the day-to-day experience, integrating with historical health data, into useful insight and medical practices based on patients’ experience and lessons learned by the artificial intelligence approach. In more detail, the system is expected to improve clinical decision making by providing physicians with additional data about patient needs and behaviors. For example, it may help clinicians to detect mood disturbances that do not impair patient daily activities, such as persistent depressive disorder. The system is indeed presenting to every recruited patient validated questionnaires for depression screening. In turn, in our system, these questionnaires are designed to trigger specific alerts that are then displayed directly to the attention of the clinician, enabling them, eventually, to refer the target patient to mental health services. We thus believe that our system will help clinicians to assess mental health issues in people living with HIV during the COVID-19 pandemic. Alternatively, when clinical data from the electronic reporting system of the hospital will be integrated, the system is designed to draw to the physician attention altered lab results directly on his/her smartphone, empowering the clinician, even in the outpatient setting, to act promptly in case of major disturbances. A final example of how we expect our system to improve clinical care is adherence to ARV therapy, which is, in our view, a substantial challenge during the present pandemic. This will be possible by selected questionnaires (able to trigger specific “low adherence” alerts to the doctor’s attention), along with drawing to the clinician’s attention the latest results of patient’s viral load. Moreover, what has been demonstrated, even if only preliminarily, in the predictive model on future alert detection, suggests that such a model may be of fundamental importance in optimizing the clinical management of PLWH. The possibility of detecting problems related to mental health or adherence to treatment, for example, may allow the prediction of a tailored management of the patient with consequent optimization of resources. For instance, we have already observed that several mental health issues have been identified among the enrolled patients and we believe that this evidence could be a useful tool to draw attention to the need to provide patients with a more effective support at this regard.

It has been reported that telemedicine is variably accepted by patients, precisely because of the risk of “depersonalization” that such a tool may carry [35]. Moreover, in recent years, the attention of research and the HIV patient community is increasingly focused on what is called the “fourth 90”, i.e., everything that concerns the global health status of patients, their quality of life, and their feeling [35]. For these reasons, developing an e-health tool that complements but does not replace care, and that focuses on what is defined as “patients reported outcomes”, could address multiple unmet needs that remote care of HIV patients has not yet solved.

It has been reported that the adoption of remote healthcare has exposed critical gaps in access, such as socioeconomic disparities that may prevent many vulnerable persons from benefiting from telehealth innovations, namely, defined as “digital divide” [36]. In order to mitigate this problem, the present study will make use of strategies both inherent in the product (easy access modes, user guide, explanatory videos, etc.) and strategies to involve the community of patients for the support of people with different social or educational frailties.

The difficulties potentially encountered by PLWHA during the pandemic are also shared with patients suffering from other chronic diseases. In our opinion, this study could represent a model of integrated management also for other chronic pathologies such as oncological pathologies for two fundamental reasons: Firstly, because oncology has long paid relevant attention to the aspects reported directly by patients in terms of quality of life, symptoms, experience with the care system, etc., leading to consider PROs as potential hard outcomes in the design of clinical trials [37]. Secondly, because the oncological patient presents many aspects of linkage to the care center that are very similar to what is reported for PLWHA in terms of continuity of care and “personalization” of care. Finally, the study is beginning to yield enough monthly data to facilitate learning of predictive models for the alerts of interest. Such models are learnt for one or two months of history of the questionnaire data. No attempt has yet been carried out to integrate with the hospital data.

7. Conclusions

The development of such an e-health system may respond to clinical care needs for a specific type of chronicity such as HIV infection. In particular, in this COVID-19 pandemic period, such a system can be an important tool for managing the quality of life of the HIV patient. The COVID-19 pandemic is having a deleterious impact on the physical and, above all, mental health of the chronic population and also on the HIV-infected population [38], and the identification of tools capable of identifying physical and mental health problems and producing interventions tailored to the specific needs of the patient represents a significant improvement in the standard of care for people with HIV, even in the absence of physical access to the reference healthcare facilities. Moreover, it can represent a simple and widely useful system to perform a prescreening for chronic patients during the COVID-19 epidemic in order to prevent potential transmission clusters within the hospital as much as possible.

Early evidence that the prediction of the alerts of interest can be made is given. As on the one side, the study yields more data points, facilitating more training and the inclusion of more months of history, and on the other side, the hospital data augment the data attributes at hand, we expect the prediction to become more accurate. Upon reaching this point, the SHAP analysis of the decisions will yield clinically significant results for the importance of the different attributes. Subsequent development of the study involves the effect of applying such a machine learning system on the health indicators shown in the methods, in order to assess how far such a system can be applied in increasingly advanced chronicity management.