Web-Based Patient-Reported Outcomes for ENT Patients—Evaluation of the Status Quo, Patients’ View, and Future Perspectives

Background: Patient-reported outcomes (PRO) assess disease burden and indicate unmet needs. Home-based electronic PRO measures (ePROMs) can support tumor aftercare (TAC). Creating an ePROM is the next step after implementing the software “OncoFunction” to assess PROs during TAC of head- and neck-cancer patients (HNC). Therefore, internet use and perception on ePROMs of ENT and TAC patients were evaluated. Methods: From May–July 2020, ENT patients at a high-volume outpatient department aged >18 without need for emergency treatment were invited to complete a questionnaire concerning internet use and access, hardware, and opinion on the chances, requirements, and designs of ePROMs. Results: 415 questionnaires were evaluated; 46.3% of the respondents visited the common consultation hour (CCH) and 44.3% TAC; 71.9% were internet users, being younger than non-internet users; and 36.4% of TAC patients were non-internet users and 16.3% of them were without a web-enabled device. Significant differences existed in age and assessment of future perspectives between internet-/non-internet users and TAC/CCH patients, respectively. Regarding the design of ePROMs, patients preferred quarterly and short surveys. Data safety and feedback were important. Conclusions: ePROMs are not suitable for everyone because of missing internet access and experience. A tailored approach to implement ePROMs in TAC is needed.


Introduction
In most cancer centers, tumor patients are assigned to tumor aftercare (TAC) after completion of tumor therapy according to the NCCN guidelines [1][2][3], aiming at achieving the longest survival and best quality of life (QoL). TAC is essential for detecting potentially recurrent diseases and monitoring early and late therapy-associated toxic side-effects at the earliest time possible to interfere by applying the indicated therapies. In patients with head and neck cancer (HNC), commonly reported side-effects after therapy are swallowing disorders, voice impairments, dry mouth (xerostomia), and pain, as well as psychological problems like depression, fatigue, and anxiety [4][5][6].
Clinicians may have a different view of the patient's impairments and underestimate or miss symptoms by only evaluating clinical, biochemical, or radiological abnormalities [7][8][9][10]. Nonspecific symptoms or symptoms that are not directly related to the physician's specialty, might not be detected and discussed if the patient does not report his complaints himself and the physician focuses on his field of expertise during the time-limited TAC visit, missing essential information about functional aspects [11,12].

Results
419 questionnaires were evaluated. Four questionnaires were excluded from the analysis as they did not fulfill the inclusion criteria because of age and might have been handed out by accident to a parent. In total, 192/415 (46.3%) of patients visited a doctor for common ENT issues during the CCH, 184/415 HNC patients (44.3%) visited the department for TAC, and 39/415 (9.4%) did not provide their reason for consultation. In the following, the readers need to be aware that using pairwise deletion to consider missing values results in different total numbers.
In total, 264/400 (66%) of the patients were male and 136/400 (34%) were female. The chronologic ages ranged from 20 to 90 years (mean 58.2 years; standard deviation (SD) 16 years). General information about patients is shown in Table 1. Table 2 introduces the status quo of internet access and use among CCH and TAC patients, as well as internet and non-internet users. Table 3 indicates the patients' perception of ePROMs and their design, evaluated in the subgroups of CCH and TAC patients, as well as internet and non-internet users.
According to the visits of TAC in the mentioned time interval, the collected questionnaires cover at least 76% of TAC patients. The response rate of CCH patients settles above.

Internet Access and Use of the Whole Cohort of ENT Patients
Most patients used the internet (297/413, 71.9%). Smartphones or computers were the main sources for gaining access to the internet. Patients often owned more than one web-enabled device (196/414, 47.3%), and 54/414 patients (13.0%) did not own any internetenabled device. Ownership of more than one device was associated with a higher education level, e.g., university degree and younger age (both p < 0.001). Non-internet users were significantly older than those using the internet (mean age 67.8 (SD 11.6) vs. 54.4 (SD 16.0) years; p < 0.001). The most common reasons for spending time on the internet were searching for information, communication via email, and online shopping; 59.6% of the patients used the internet for less than 2 h a day and 74% for less than 4 h a day for private concerns (n = 366); 185/352 (52.6%) patients did not need the internet for work at all, whereas 98/352 (27.8%) spent less than 2 h per day on the internet for work-related tasks.
Concerning health issues, many patients got advice and information exclusively from their doctors and did not spend time on the internet for health-related research (259/373, 69.4%). Most patients had never visited a medical website (237/349, 67.9%).   Table 3. Opinion on and attributed chances linked to using digital media in health care for patient-reported outcomes (PRO) and designing ePROM (electronic PRO-measurement) of ENT patients stratified according to reason for consultation (common consultation hour vs. tumor aftercare) and internet use (yes vs. no).

Internet Access and Use by HNC Patients vs. Other ENT Patients
There were significant differences between the groups of patients admitted to TAC and CCH for other ENT issues (Table 2). Compared to CCH patients, more TAC patients were male (73.9% vs. 55.2%) and of higher age (mean 64.3 (SD 10.6) vs. 51.4 (SD 17.65) years); 30/184 (16.3%) TAC patients vs. 13/192 (6.8%) CCH patients did not own a web-enabled device. After applying the Bonferroni correction, the level of education did not remain significant. 67/184 (36.4%) TAC patients and 29/192 (15.1%) CCH patients negated using the internet (all p < 0.001). In Figure 1, the distribution between either CCH or TAC and internet use is visualized.
private concerns (n = 366); 185/352 (52.6%) patients did not need the internet for work at all, whereas 98/352 (27.8%) spent less than 2 h per day on the internet for work-related tasks.
Concerning health issues, many patients got advice and information exclusively from their doctors and did not spend time on the internet for health-related research (259/373, 69.4%). Most patients had never visited a medical website (237/349, 67.9%).

Internet Access and Use by HNC Patients vs. Other ENT Patients
There were significant differences between the groups of patients admitted to TAC and CCH for other ENT issues (Table 2). Compared to CCH patients, more TAC patients were male (73.9% vs. 55.2%) and of higher age (mean 64.3 (SD 10.6) vs. 51.4 (SD 17.65) years); 30/184 (16.3%) TAC patients vs. 13/192 (6.8%) CCH patients did not own a webenabled device. After applying the Bonferroni correction, the level of education did not remain significant. 67/184 (36.4%) TAC patients and 29/192 (15.1%) CCH patients negated using the internet (all p < 0.001). In Figure 1, the distribution between either CCH or TAC and internet use is visualized. Searching for information, emailing, and shopping were the most common internet activities in both groups. Both groups differed significantly in the amount of time spent on the internet, but they do not spend a significantly different amount of time on seeking information concerning health-related issues. Searching for information, emailing, and shopping were the most common internet activities in both groups. Both groups differed significantly in the amount of time spent on the internet, but they do not spend a significantly different amount of time on seeking information concerning health-related issues.

Chances and Future Imagination of ePROMs from the View of the Whole Cohort of ENT Patients
Several items of the questionnaire evaluated the patient's attitude toward ePROMs, including the design of recurrent questionnaires and engagement in using electronic applications reporting patient data (Table 3). In total, 206/406 (50.7%) ENT patients would accept a device-based application (APP) as a communication tool. If suffering from a chronic disease, 52.8% of ENT patients prefer an interval of disease monitoring via questionnaire every three months or less. In the opinion of 71% of the respondents, the questionnaire should take less than 10 min, and 84.2% of the patients think that an ePROM should consist of fewer than 20 questions. In all, 211/364 (58.0%) are confident to collect health data (e.g., heart rate and blood pressure), possibly by APP support, and 178/362 (49.2%) assume an improvement in patient care by using an APP in addition to regular consultations with physicians. Replacing consultation hours with online consultations is refused by 238/381 (62.5%) patients. Data safety, keeping their own data under control, and receiving a response to the information sent via APP are desired and/or seen as requirements for agreeing to transfer personal data via APP. However, 162/339 (47.8%) would like to make appointments via email, especially those of younger age (p < 0.001) or with a university degree (p = 0.004).

Patients' View on Chances and Future Prospects of ePROM Use
The patient's view on reporting personal data via electronic application(s) differs when dividing ENT patients into subgroups of tumor patients and non-tumor patients. The majority of TAC patients cannot imagine communicating via APP with their hospital (103/181, 56.9%), whereas most CCH patients can (116/188, 61.7%) and only 38.3% cannot envisage digital communication (72/188; p = 0.001). Thus, more TAC patients than CCH patients do not want to replace physical doctor's appointments with an online consultation (127/175, 72.6%, vs. 87/170, 51.2%; p < 0.001). Moreover, the benefits of using an APP, the capability of collecting their own health data, and the willingness to transfer data via APP to a hospital are judged differently by TAC and CCH patients (all p < 0.004). Interestingly, CCH patients preferred shorter intervals between surveys in cases of suffering from a chronic disease compared to TAC patients (p < 0.001). There were no differences in the patients' views on the design of the questionnaire (p > 0.15) ( Table 3).

Discussion
In this cross-sectional, mono-centric study, internet access and activities of ENT patients visiting CCH and TAC were assessed and their perceptions of the design and future perspectives of ePROMs were evaluated. There were significant differences in chronologic age, and assessment of future perspectives according to internet use (yes vs. no) and the reason for consultation (TAC vs. CCH). As HNC affects more men, the frequency of male participants consulting their physician for TAC after the cure of their HNC was higher and correlated with the higher number of questionnaires completed by men. The majority of ENT patients in this study used the internet or owned at least one web-enabled device. Internet users were significantly younger than non-internet users, just as CCH patients compared to TAC patients. The internet was mostly used for seeking information not related to health information, which, as such, was mostly provided by the treating physician. About 50% of the ENT patients have a positive attitude toward using an APP for PRO. Data safety and receiving feedback are very important for the patients when completing a PROM. TAC patients seem to be more dismissive and skeptical in their perceptions concerning the use and benefit of ePROMs.
PROMs in general give insight into the patient's problems in daily life caused by functional and physical impairment. Provided unlimited and barrier-free access to devices and their connectivity, regular use of home-based ePROMs, reported in the everyday surrounding of the patient, might be best to capture problematic areas. The results of this study are helpful when designing an ePROM and implementing it in the clinical routine. As not every patient is able to access the internet and answer appropriately, or to handle the recording of relevant issues, caution is advised before trying to implement an ePROM. Nevertheless, several studies indicate the benefit of PROMs, showing that creating an ePROM is worth the effort. Besides improved symptom control and health-related quality of life [22], Basch et al. reported a significantly increased overall survival and prolonged treatment with chemotherapy of patients with metastatic breast, genitourinary, gynecologic, or lung cancers [16]. Their patients were assigned to either a computerexperienced or computer-inexperienced group and within these groups either to usual care or the intervention arm with self-reported PROs. Randomized to the intervention arm, computer-experienced patients had to complete weekly home-based ePROMs, in addition to the PROMs at each clinical visit, while the inexperienced completed PROMs only at the clinical visit. Nurses intervened when alerted via email in cases of severe symptom worsening. Health-related quality of life was less impaired in the intervention arm but without statistical significance in the subgroup of computer-inexperienced patients with balanced baseline characteristics between both subgroups. However, selection bias cannot be excluded whenever patients are stratified into experienced vs. inexperienced because of substantial confounding.
Remarkable findings were also reported in a study by Denis et al. [23,24]. As these authors recognized that some patients who experience an increase in symptom burden did not contact a doctor but waited until their regular TAC appointments, e.g., in 3 months, to discuss their impairment, they chose weekly intervals to monitor symptom burden in detail via ePROMs. In a randomized controlled trial with 133 patients with metastatic lung cancer, the intervention group completed ePROMs weekly. By alerting the oncologic staff whenever the symptom burden increased, they triggered earlier utilization of appropriate treatment intervention and achieved a significantly better overall survival (OS) than the control group (OS 22.5 months vs. 14.9 months, p = 0.03). In the intervention group, lessregular CT scans were made because of the results of earlier studies when an increase in symptom burden has indicated a relapse better than the regular CT scans (every 3-6 months in the control arm and every 6 months in the intervention arm) [25,26]. Additionally, a cost benefit was described when using web-based PRO surveillance in patients with lung cancer [27].
Until now, it is unclear if PROs have a positive effect on the overall survival of HNC patients and further studies are needed. Nevertheless, it is essential to identify patients at risk of having no benefits from implementing ePROMs as they are, for instance, visually impaired, functionally illiterate, or otherwise handicapped and, hence, not able to appropriately use them.
The majority of our patients had access to the internet (333/376, 88.7%) and used it regularly (297/413, 71.9%). Of the non-internet users, 42.2% (49/116) did not own a webenabled device and, thus, definitely did not have any access to the internet. Non-internet users being older than internet users on average in this study is in line with the findings of Wintner et al., who report a strong skepticism of older patients toward eHealth [28]. They assumed that older people are less familiar with online activities and, because of missing online skills, are skeptical and refuse eHealth. Exploring the needs of supportive care, Jansen et al. [29] also reported an association of lower age with a positive attitude toward self-management and eHealth.
By contrast, there is a growing (and aging) population using new media and even among the oldest there are many being well-educated in using the internet. Our data suggest that especially younger (cancer) patients are more familiar with online activities and might benefit from the implementation of an online tool, as they are more active and/or employed. To them, data transfer is conceivable if data safety and receiving feedback to their data are guaranteed.
Currently, one-third of the presented HNC patients do not use the internet, and thus collecting health data via app or using ePROMs will be difficult. Their motivation appears to be limited as they do not see any advantage of using an app in addition to regular visits; 56% of TAC patients cannot imagine communicating with their physician via APP. These circumstances call for a tailored approach concerning the implementation of new media in TAC or health care in general, with training and assistance offered to introduce and/or support the completion of the ePROM meeting the individual needs of particular patient groups. Support is especially needed when patients are in poor general condition, have speech problems, or are simply not familiar with the language, illiterate, visually impaired, or physically not able to handle an internet-enabled device. Technical problems demand special support.
A Dutch study on HNC patients after total laryngectomy reported similar findings concerning internet use [30]: 68% of the patients reported regular use of the internet, especially those of younger age and with better education, and those with shorter time after a laryngectomy. According to these authors, eHealth, besides recording PROs, might provide tools to inform about the disease and serve patients in the difficult phase directly after laryngectomy. A weakness of their study is the systematic selection of computerfriendly patients, omitting all others. According to the study design, questionnaires were sent to 944 patients with a history of total laryngectomy; 288 questionnaires were sent back without tracking or making inquiries of the non-respondents, leading to a systematic selection of compliant and motivated patients and a serious selection bias allowing only conclusions for a "best-case scenario". Our study covers a raw sample of ENT patients in general and TAC patients. There is no selection of computer-experienced and new-mediafriendly or compliant patients in our study because every patient entering the outpatient department was invited to participate via face-to-face contact with the study nurse to answer the questionnaire during the waiting time before entering the doctor's room.
Our cohort differs from the standard German population according to internet use. In an annual survey conducted by the most important German public-service television broadcasters ARD/ZDF via telephone interview [31], 94% of the Germans affirm using the internet. Thus, internet use is less common in our cohort of ENT patients (71.9%). In the group aged >70 years, 77% of the general German population are internet users. Of the elderly (≥70 years) in our cohort, internet users were less frequent (53.9%; 48/89), and according to the reason of visit, 58.8% were in CCH (20/34) and only 50.9% (28/55) in TAC. Comparing these frequencies with the general German population indicates ENT patients in general, and especially TAC patients, are a vulnerable group differing from the general population regarding internet use and willingness to use ePROMs. Thus, specialized and custom-tailored approaches are needed and ePROMs cannot be used to replace regular visits.
From our patients' point of view, a good ePROM may contain up to 20 questions that can be answered in less than 10 min every 3 months. Patients are willing to transfer their data if (1) data safety is guaranteed (CCH and TAC patients), (2) they receive feedback or even an answer to their query (CCH and TAC patients), and (3) they can access their prior data and incrementally alter it, or if (4) they have a personal advantage (CCH patients) or benefit by reducing the frequency of their doctor appointments (TAC patients). Simultaneously, 72.6% (127/175) of the TAC patients cannot imagine replacing the physical consultations with the treating physician, including medical examinations, with online consultations.
We decided to use pairwise deletion in statistical analysis to cover a raw sample of ENT patients. To prevent missing values, a notification for missing answers could be inserted into an online tool; in general, questionnaires should be kept short and easy to understand. Comparing paper-based and electronic questionnaires, Kongsved et al. [32] found a higher response rate to paper-based questionnaires sent via postal service to the patients compared to those who answered an online questionnaire. To achieve a sufficient response rate of the online questionnaires, the authors had to send at least one reminder to the patients. On the other hand, online questionnaires were complete in 97.8% of the cases without missing values, compared to 63.4% of the paper-based questionnaires. Additionally, web-based PROMs were more cost-effective than paper-based questionnaires. Because of the higher completion rate using electronic PRO, the regular PRO-assessment during TAC in our clinic using the software "OncoFunction" is tablet-based (and offers the advantage of easy recording and lossless storage of PRO without any form of interpretation and confirmation bias).
For a successful implementation and acceptance of ePROMs by patients, feedback given directly, or at least within the near time, by the physician is desired to keep the patients' willingness to further enter their data. Therefore, editing and replying to ePROMs must be integrated into the clinical routine to not be missed when the patient is not physically present. Designing an interactive response system that immediately provides recommendations, such as, for instance, to proceed by arranging an appointment with the treating ENT specialist, might be helpful to maintain adherence to TAC programs offering partial replacement of regular visits by ePROM assessments and, moreover, to transfer electronic alerts into actions when needed. In particular, disabled and immobile patients living in rural regions may benefit from recording their data and interactive response systems before seeking direct interaction and examination by their physician without prior notice. Trained (oncology) nurses could sift the alerts and assign the patient to a matching path as realized and described by Basch et al. [22]. The software "OncoFunction" already provides a traffic light alert system when PROs are worse compared to their last visit and might be suitable for ePROM in an adapted form alerting the involved ENT specialists.
Currently, a relevant number of HNC patients are not familiar with internet and web-based solutions. These patients need special care in daily clinical practice. Arranging regular telephone calls with patients who are not familiar with the internet and web-based solutions would be necessary to get PROM data instead of ePROM data. This solution might be more expensive than web-based solutions but may minimize the risk of losing those patients who require the TAC the most, for instance (functional) illiterate patients, from aftercare.
There are some limitations to our study. First, the used questionnaire is not validated. Within a quality measure, we were interested in the patients' opinion to gain experience and identify potential pitfalls concerning implementing new media into patient care. Because the questionnaire will not be used for other concerns outside of the quality measure, a detailed validation is time-and resource-consuming with little to no benefit. The included questions were short and were expressed in simple language. In fact, by using written questionnaires, a potential bias created by excluding illiterate patients may remain. This may result in an underestimation of proponents of ePROMs and, thus, underline our call for a tailored approach when implementing ePROMs or new media in TAC.
Many questionnaires were not filled in completely. Questions especially in the middle or at the end of the questionnaire have been skipped, potentially because patients were overloaded with information and, thus, lost concentration and interest in answering the questions. This goes in line with the preference of many patients for questionnaires requiring little time to complete. Furthermore, sometimes questions printed on the back side of the paper may have been overlooked.
Because the presented survey was part of a quality measure, we went without secondary information about other patients' characteristics such as level of income, profession, relationship status, smoking, and alcohol consumption, or detailed information on medical conditions. The questionnaire was designed only to obtain the essential minimum of personal data to assess the patients' views on ePROMs and the feasibility of their implementation. Nonetheless, we gained clear information on the subject of interest as described above, even if only core data were used. Obviously, for developing a validated tool to assess ePROs, further detailed information is needed but was beyond the investigators' point of interest at the time the study was conducted.

Conclusions
PROMs promise improved patient-centered health care and symptom monitoring. The majority of patients has access to the internet and is familiar with its use. Many patients are open-minded to completing ePROMs regularly in cases of chronic disease. Nonetheless, a significant number of HNC patients have neither internet access nor are familiar with using online questionnaires and, thus, are skeptical concerning the use of new media in health care, including ePROMs. Therefore, a tailored approach is needed to implement ePROMs in TAC.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the University Hospital Leipzig (file number 447/20-ek).

Informed Consent Statement:
Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The datasets used and analyzed during the current study are available from the corresponding authors upon reasonable request. The data are not publicly available because they contain information that could compromise research participant privacy.