Delivery of Virtual Care in Oncology: Province-Wide Interprofessional Consensus Statements Using a Modified Delphi Process

Virtual cancer care (i.e., teleoncology) was rapidly adopted during the COVID-19 pandemic to meet the needs of patients with cancer. However, there is a paucity of guidance for clinicians regarding virtual cancer care. We sought to develop consensus-based statements to guide the optimal provision of virtual care for clinicians caring for patients with cancer, using a modified Delphi consensus process with a 29-member panel consisting of an interprofessional group of clinicians caring for patients with cancer and patient representatives. The consensus process consisted of two rounds and one synchronous final consensus meeting. At the end of the modified Delphi process, 62 of 62 statements achieved consensus. Fifty-seven statements reached consensus in the first round of the process. Concerns regarding the ability to convey difficult news virtually and maintaining similar standards as in-person care without disproportionate strain on clinicians and patients were addressed in the consensus process. We achieved interprofessional consensus on virtual cancer care practices. Further research examining the impact of virtual cancer care on person-centred and clinical outcomes are needed to inform practices during the COVID-19 pandemic and beyond.


Introduction
The novel coronavirus, SARS-CoV-2, responsible for the coronavirus disease (COVID- 19), is the foremost public health priority globally and has dramatically altered the delivery of clinical care. Physical distancing, which was the quintessential public health intervention to limit the spread of the virus, greatly limited traditional in-person cancer care [1]. Persons diagnosed with cancer, as well as those undergoing active cancer treatment, are at higher risk for COVID-19 complications [2,3]. A recent systematic review and meta-analysis found that the case-fatality rate of COVID-19 in patients with cancer was 22% compared with 6% for those without cancer [4]. Despite COVID-19 vaccines becoming more widely available, emerging variants of concern combined with uncertainty about vaccine response require that patients with cancer to balance the competing risks of contracting COVID-19, and the morbidity and mortality associated with cancer [5]. Patients with cancer continue to face unique challenges during the pandemic related to their care, such as physical distancing efforts hindering in-person care; increased risk of infection when attending appointments and treatments; treatment delays or unavailability due to ramp downs of clinical care; management of adverse effects of treatments [6] and suboptimal responses to vaccination [7]. To meet pressing patient needs, virtual cancer care (i.e., teleoncology) has been rapidly adopted to prioritize the safety of this patient population [1,[8][9][10][11][12][13][14][15].
Virtual cancer care was originally developed to improve access to care in remote and rural settings, as well as to meet the growing demands by a limited oncology workforce [16]. Virtual care can include consultations, follow-up appointments, remote supervision of chemotherapy, and access to clinical trials [16]. There have been published examples of virtual cancer care for more than 20 years [17], including practices from North America and Australia [16,18,19]. Studies of teleoncology used to meet the needs of rural patients have demonstrated high patient and provider satisfaction [18,20]. A study in Australia found that virtual care allowed for safe remote administration of chemotherapy [21]. In addition, teleoncology initiatives are associated with decreased costs and improved access to care [16]. However, despite these promising studies, there is a dearth of high-quality evidence that demonstrates the impact of virtual cancer care on important person-centred and clinical outcomes or determines the appropriateness and ideal context in which to administer virtual care [22,23].
Although the COVID-19 pandemic has thrusted virtual cancer care to the fore, there are currently no widely established best practices nor evidence-based guidance for clinicians [23]. As an unprecedented number of patients with cancer receive their care remotely, identification of best practices for virtual cancer care is urgently needed to guide clinicians during the COVID-19 pandemic and beyond [24]. With the absence of high-quality evidence during a pandemic that precluded methodologically rigorous and pragmatic studies, the development of guidance for virtual cancer care practice using other feasible methodologies are warranted [23]. Consequently, Ontario Health (Cancer Care Ontario) identified an urgent need to develop consensus practice statements to provide guidance to clinicians. We developed expert consensus recommendations for virtual cancer care using a modified Delphi process with the intention to set the foundation for the future of cancer care in Ontario, a Canadian province with 14.5 million people.

Modified Delphi
We used a modified Delphi consensus process based on guideline development methodology established by the American Society Clinical Oncology [25]. A formal consensus process provides a transparent and reproducible method to develop recommendations when high-quality evidence is lacking, making it suitable in addressing questions about virtual cancer care. To limit in-person meetings during the pandemic, the process was conducted virtually using both an online survey platform (Survey Monkey Inc., San Mateo, CA, USA, www.surveymonkey.com, 5 April 2021) and teleconference software (Zoom Video Communications Inc., San Jose, CA, USA, www.zoom.us, 5 April 2021). This study was approved by the Sunnybrook Health Sciences Centre Research Ethics Board.

Literature Review
To create the first draft of consensus statements, we conducted a systematic literature review of Embase and MEDLINE for peer-reviewed articles in April 2020 (Supplementary Materials, Figure S1). The inclusion criteria were purposely broad, and we included any articles that described or investigated synchronous virtual cancer care delivered by oncology physicians. To ensure that we considered new practice-changing publications that may affect our consensus statements, we incorporated results from a systematic review of randomized control trials in virtual cancer care conducted by Ontario Health (Cancer Care Ontario) in March 2021 [23].

Steering Committee
The Steering Committee (SC) consisted of 10 members: a medical oncologist, hematologist oncologist, radiation oncologist, surgical oncologist, patient representative, oncology pharmacist, one research coordinator, and three medical trainees (a resident physician and two medical students). The SC identified key questions (Appendix A) regarding virtual cancer care and used literature as a guide to draft statements. Clinical questions were grouped into three sections: (A) demographics, logistics, and implementation, (B) diagnosis and prognosis, and (C) clinical characteristics, active management, and follow-up. The SC drafted the first set of consensus statements by using literature to answer key clinical questions (Supplementary Materials, Table S1).

Consensus Process
We assembled a diverse Consensus Group (CG) consisting of an interprofessional collection of clinicians and patients to capture a wide spectrum of perspectives and avoid potential biases inherent in the development of consensus recommendations [25]. Ontario is a province with a population of 14.5 million distributed over a large geographic area and thus we aimed to include individuals with opinions and experiences related to rurality, discipline, Indigenous status, and clinical specialty [26]. Therefore, we invited clinicians, patient representatives, and administrators (including virtual care stakeholders such as the Ontario Telehealth Network) across the province associated with Ontario Health (Cancer Care Ontario) to be part of the CG.
We conducted two rounds of anonymous modified Delphi surveys and a final synchronous virtual consensus meeting. Draft statements were inputted into an anonymous online survey and distributed via email to the CG (Round 1 of the consensus process). We used a ≥75% threshold, determined a priori, for consensus with agreement achieved by a "agree" or "strongly agree" on a 5-point Likert scale. Respondents were able to provide open-text comments for each draft recommendation. After Round 1, the SC used comments to revise statements that did not achieve 75% agreement. The revised statements were sent to the CG for a total of two consensus rounds. Two e-mail reminders were sent to CG members for each round. After Rounds 1 and 2, statements not reaching consensus and those that achieved consensus but had comments with clear suggestions for improvement were presented to the CG at the final consensus meeting, a synchronous virtual discussion. A simple majority via anonymous polling (using Zoom) was used to determine consensus at the final meeting.

Consensus Group
The CG had twenty-nine members and included medical cancer specialists (four medical oncologists, two hematologists, three radiation oncologists), seven surgical oncologists (including a neurosurgeon), three family physicians, one psychiatrist, two patient representatives, two nursing and allied health professionals, and four health care administrators. CG members included academic and community-based clinicians, as well as representatives from Indigenous, and rural/remote communities. The response rate in Round 1 was 97% (n = 28) and 93% (n = 27) in Round 2. There were 22 members of the CG present during the final consensus meeting.

Consensus Process
The SC drafted 62 statements ( Figure 1) from 90 articles identified in the systematic review (no new articles were identified from the more recent systematic review of randomized control trials). After Round 1, 57 of 62 (92%) statements achieved consensus.
Statements not reaching consensus were revised according to comments and sent to the CG. After Round 2, 61 of 62 (98%) statements reached consensus. During the final consensus meeting, the statement that did not reach consensus was discussed, revised, and re-rated. Qualitative feedback was summarized and provided to CG members during the final meeting. Seven statements that were revised according to comments related to clarification and wording were also voted upon during the final meeting. There was agreement on all the revised statements after the final consensus meeting.

Consensus Process
The SC drafted 62 statements ( Figure 1) from 90 articles identified in the systematic review (no new articles were identified from the more recent systematic review of randomized control trials). After Round 1, 57 of 62 (92%) statements achieved consensus. Statements not reaching consensus were revised according to comments and sent to the CG. After Round 2, 61 of 62 (98%) statements reached consensus. During the final consensus meeting, the statement that did not reach consensus was discussed, revised, and rerated. Qualitative feedback was summarized and provided to CG members during the final meeting. Seven statements that were revised according to comments related to clarification and wording were also voted upon during the final meeting. There was agreement on all the revised statements after the final consensus meeting.

Section A-Demographics, Logistics, and Implementation
The CG reached consensus regarding recommendations stating that all patients with cancer should be offered virtual care, with particular attention to overcoming accessibility issues (e.g., those with limited comfort with technology, language barriers) ( Table 1). Statements surrounding logistics reinforced that documentation should be consistent with in-person care in addition to informed consent for the virtual encounter. Statements about the integration of systems of cancer care, including electronic medical records, interdisciplinary collaboration, access to local health care resources, were addressed in several statements. Finally, the CG reached consensus regarding the need for access to virtual care training options for clinicians and patients.
After Round 1, consensus was achieved on 18 of 19 statements in Section A. in the statement that did not meet consensus (Statement A2f), it (Table 1) had 39% agreement. This statement originally recommended that clinicians use an onboarding process (gradually increasing familiarity with virtual cancer care through using strategies such as selecting engaged, technologically savvy, and familiar patients) when starting to use teleoncology [27]. The comments suggested that the CG perceived that a self-guided onboarding process was not necessary for clinicians. Rather, comments suggested that formal training options in virtual care for clinicians be developed and made available to facilitate the process. This statement was revised accordingly and all statements in Section A reached consensus in Round 2. Minor revisions for clarity were unanimously agreed upon during the

Section A-Demographics, Logistics, and Implementation
The CG reached consensus regarding recommendations stating that all patients with cancer should be offered virtual care, with particular attention to overcoming accessibility issues (e.g., those with limited comfort with technology, language barriers) ( Table 1). Statements surrounding logistics reinforced that documentation should be consistent with in-person care in addition to informed consent for the virtual encounter. Statements about the integration of systems of cancer care, including electronic medical records, interdisciplinary collaboration, access to local health care resources, were addressed in several statements. Finally, the CG reached consensus regarding the need for access to virtual care training options for clinicians and patients.

Demographics and Implementation
A1a. All patients should be considered and, if clinically feasible, offered the option of virtual cancer care regardless of demographics (e.g., gender, race/ethnicity, language spoken, income, education, rurality, physical and/or mental disabilities, indigenous identity). Special effort should be made towards patients without good access to technology, or those who are uncomfortable with using technology.

89%
--A1b. It is recommended that resources be created and disseminated to all health care providers and patients to overcome barriers to virtual cancer care. These can include written, video, and/or verbal guidance provided by a member of the oncology team (e.g., clinical administrator) in advance of the virtual visit or point of care resources.

96%
-- A4c. Access to primary care and emergent care must be included in the discussion of risks and benefits of virtually managed cancer care. Patients receiving virtual cancer care should be counselled on possible risks specific to their care (e.g., chemotherapy toxicity, lymphedema, post-surgical complications) and cancer (e.g., visceral crisis) and appropriate avenues to reach care. Therefore, we encourage that the patient's local health care provider is made aware of ongoing cancer care and that patients are aware of local resources in the event of complications.

93%
--* Percentages are included if the statement was presented to the Consensus Group.
After Round 1, consensus was achieved on 18 of 19 statements in Section A. in the statement that did not meet consensus (Statement A2f), it (Table 1) had 39% agreement. This statement originally recommended that clinicians use an onboarding process (gradually increasing familiarity with virtual cancer care through using strategies such as selecting engaged, technologically savvy, and familiar patients) when starting to use teleoncology [27]. The comments suggested that the CG perceived that a self-guided onboarding process was not necessary for clinicians. Rather, comments suggested that formal training options in virtual care for clinicians be developed and made available to facilitate the process. This statement was revised accordingly and all statements in Section A reached consensus in Round 2. Minor revisions for clarity were unanimously agreed upon during the final consensus meeting for three Section A statements (A2f, A2g, and A4b). All clarifying revisions were related to making statements more adaptable to different clinical scenarios.

Section B-Diagnosis and Prognosis
The CG reached consensus on 20 statements to guide the communication of diagnostic and prognostic information ( Table 2). The recommendations emphasized the importance of understanding patient preferences regarding method of communication before this information is conveyed. In addition, the CG achieved consensus specific to recommendations to optimize communication through virtual platforms. Finally, CG members highlighted the unique situation of conveying bad news such as new metastatic or palliative diagnoses. The need for clinicians to exercise discretion in the decision to convey this information virtually is emphasized in the consensus statements. Specifically, patient-centred factors including the level of support available to the patient, the symptom burden experienced by the patient, and the nature of the patient-provider relationship should be considered in decisions to use virtual cancer care during these encounters.  After Round 1, consensus was achieved on 16 of 20 statements in Section B except B1a (Table 2), which had 61% agreement. This section originally contained recommendations for the administration of a formal health literacy screen, such as the Brief Health Literacy Screen [28], in preparation for the virtual encounter. CG members indicated that difficulties in operationalizing a formal screening tool would outweigh its benefits and this recommendation was removed in the final statement.
Similarly, Statement B2a initially recommended the use of a structured and validated framework to communicate diagnosis and prognosis virtually and reached 57% agreement with many comments suggesting that a formal framework is not necessary. In Round 2, the revised statement, which included wording on the use of empathic communication and consideration of the patient-provider relationship, achieved 62% agreement. During the final consensus meeting, this statement was revised after the CG discussed that the decision to disclose a diagnosis or prognosis virtually should be influenced by whether virtual care can better meet a patient's needs relative to in-person care (e.g., if virtual care provides more timely access). Statement B3a, which originally contained wording about challenges in effectively communicating metastatic or palliative diagnoses, had 61% agreement after Round 1 ( Table 2). CG members suggested that this is highly dependent on clinical and patient factors and may be feasible for some patients; the statement was revised to reflect consideration based upon the patient-provider relationship, expected patient response to the diagnosis, and available support, and reached consensus in Round 2 with 96% agreement.

Section C-Clinical Characteristics, Active Management, and Follow-Up
All 23 Section C statements reached consensus in Round 1 (Table 3). Two statements (C1c and C4d) were discussed in the final consensus meeting to improve wording based on comments received. There was agreement with statements that acknowledged patient and health care provider preferences, in addition to clinical appropriateness, as important considerations when deciding to provide virtual cancer care. Furthermore, consensus statements addressed situations that may warrant in-person assessment, such as circum-stances when physical examination is necessary. Recommendations unique to surgical oncology, radiation, systemic therapy, and survivorship reached agreement. Finally, in addition to active management, there was consensus regarding the use of virtual cancer care other aspects of care (e.g., access to clinical trials, symptom and pain management, cancer prevention). Table 3. Section C-Clinical characteristics, active management, and follow-up.

General Clinical Considerations
C1a. If physical examinations and/or investigations (e.g., bloodwork, imaging, pathology) essential for diagnosis/prognosis, symptom management, and/or choice of treatment could not be obtained through a virtual consult, an in-person face-to-face consult is required.

100%
--C1b. Appropriateness of engaging individual patients in virtual care visits for active management depends on the health care provider, as well as patient preference when clinically appropriate.

89% --
C1c. Both curative and non-curative intent virtual management of patients with cancer is appropriate, unless in-person assessment is required by either the health care provider or the patient.

93%
-88% (n = 17) C1d. When available (depending on treatment centre infrastructure), allied health care (e.g., nursing, pharmacy, social work) support should be offered to ensure optimal patient care. C1h. Virtual care could be used for cancer prevention, symptom and pain management, and the assessment of nutrition, drug toxicity, and psychosocial factors (e.g., psychological counselling, activities of daily living, etc.).

100% --
C1i. Virtual follow-up visits that require discussion about recurrent or progressive disease and/or change of treatment due to treatment failure should prompt the health care provider to request that the patient be accompanied on the telephone or video conference by a caregiver.

Surgical Oncology
C2a. First consultations with potential surgical cancer patients should be held in-person if there is a requirement for formal physical examination of relevant organ system or other in-person investigations. Otherwise, virtual consult is appropriate.

96%
C2b. Surgical planning and post-operative follow up of patients with cancer could be conducted virtually, either when no additional physical examinations or investigations (e.g., bloodwork, imaging, pathology) are needed, or when these examinations and investigations could be completed locally for patients living in remote areas. In the latter case, such consultations are only appropriate if the surgeon is comfortable with the extent of physical examinations performed by local health care providers and their experiences and skills.
100% --C2c. Post-surgical patients can be assessed virtually unless the need for wound assessment and/or physical examination is required to provide optimal care. If virtual care is performed, we recommend engaging in homecare to patients with active wound issues (i.e., wound care).

Qualitative Feedback
We achieved consensus for 62 statements regarding the practice and delivery of virtual oncology care. A few overarching themes emerged from the comments and discussions during the modified Delphi process. First, CG members generally indicated that virtual cancer care should follow the same standards as in-person care, and thus recommendations should reflect current standards of care. Second, CG members felt that health care systems and infrastructure should strive to minimize undue burden on patients and families, and clinicians relative to in-person care. Finally, respondents generally perceived virtual cancer care as appropriate for most patients in most clinical scenarios when physical examination was not deemed necessary. However, recommendations require flexibility so that they can be adapted to different clinical and patient situations.

Discussion
A diverse CG, consisting of an interprofessional group of clinicians and patient representatives, reached consensus on 62 statements regarding the delivery of virtual cancer care. Almost all statements reached consensus after the first round. Comments provided by respondents and discussion during the final meeting highlighted that the consensus achieved was premised on the perception that virtual cancer care should reflect the same standards of practice as in-person care. Furthermore, most minor clarifications and wording revisions reflected the desire to adapt virtual cancer care according to clinical and patient needs. To our knowledge, this consensus document is the first to focus on the delivery of virtual cancer care.
Although our findings suggest that virtual care is suitable for most patients and clinical scenarios when physical examination was not required, there were areas of initial disagreement related to discussing bad news or difficult clinical scenarios with patients (e.g., a new diagnosis of cancer, recurrent and/or progressive disease, and poor prognosis). Early discussions revolved around whether these scenarios should always prompt an in-person visit, rather than a virtual one. This is consistent with a recent study from Ontario that found physicians' perceptions on the quality and safety of virtual cancer care improved since its implementation [13]. In addition, published reports on experiences with virtual cancer care prior to the COVID-19 pandemic have suggested that it is possible to successfully deliver bad news virtually after consideration of a patient's unique context (e.g., available local supports) [29]. In our study, these findings were reflected with the CG achieving consensus after the statements were revised to explicitly consider patients' unique context (such as the ability to understand, process, and follow-up on health information and the patient-provider relationship) as well as clinical appropriateness. Our understanding of clinical decision making surrounding the suitability of certain patients and difficult clinical scenarios will likely evolve as virtual cancer care continues to be further normalized as part of patients' cancer journeys.
Another important aspect of virtual cancer care elucidated during the consensus process is the relative lack of infrastructure for both patients and health care providers to provide optimal virtual care to cancer patients (e.g., digital platforms, communications, logistics, and specific administration supports). Moreover, infrastructure varies greatly across different institutions and regions of the province, although professional organizations, such as European Society for Medical Oncology (ESMO) [30], tout virtual cancer care, there is a paucity of guidance regarding systemic changes to integrate it into routine oncology practice [31]. For example, whereas clinician reimbursement for virtual care is necessary, it alone is likely insufficient in supporting virtual cancer care during the COVID-19 pandemic and beyond [1,13]. To this end, our consensus statements on virtual oncology practice, while also providing guidance to clinicians provincewide, can serve as a catalyst and initial guide for institutions, policymakers, and governments when developing system-wide supports for virtual cancer care.

Strengths and Limitations
Strengths of our consensus process include the diversity of the CG (clinicians, patients, rurality) and the high response rate. Furthermore, the anonymity between rounds in the modified Delphi process prevented overwhelming influence of any particular member [32]. The synchronous final consensus meeting also facilitated clarification on the sources of disagreement and were resolved accordingly. However, our recommendations should be considered in the context of our study's limitations. Although we used a comprehensive literature review to draft our statements and validated methods to achieve agreement, the statements should be interpreted as expert opinion based on consensus in the broader context of an evolving and rapidly growing field of virtual cancer care.

Conclusions
Our consensus statements provide clear guidance to clinicians delivering virtual cancer care. Virtual care should meet the same standards expected with in-person cancer care. Patient preferences and circumstances are important factors to consider to appropriately offer and tailor virtual cancer care. With the expansion of virtual cancer care during and after the COVID-19 pandemic, further research is needed to guide best practices. Prior to the pandemic, the impetus for virtual cancer care was to improve access to care (e.g., provide care for patients in rural areas) [23]. However, the paradigm surrounding virtual cancer care has shifted from a tool to improve access to care, to an essential modality in almost all aspects of cancer care. Therefore, research is needed to examine virtual cancer care's impact on important person-centred and clinical outcomes to inform best practices in the future.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/ 10.3390/curroncol28060445/s1, Figure S1: Overview of systematic review results. Table S1: Key citations that informed first draft consensus statements.