Next Article in Journal
Treatment Adherence Interventions for Burn Patients: What Works and What Role Can Motivational Interviewing Play?
Next Article in Special Issue
Aspects of Clinical Utility of the Distress Thermometer and Problem List after Burns
Previous Article in Journal
Damage Control Surgery after Burn Injury: A Narrative Review
Previous Article in Special Issue
Matching Qualitative Inquiry Design and Practice to Contemporary Burns Research Questions: Are We Getting It Right?
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
EBJ
Volume 3
Issue 2
10.3390/ebj3020025
ebj-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Electronic Patient-Reported Outcome Measures in Burn Scar Rehabilitation: A Guide to Implementation and Evaluation

by
Jill Meirte
1,2,* and
Zephanie Tyack
3,4,†
1
Department of Rehabilitation Sciences and Physiotherapy REVAKI-MOVANT, Faculty of Medicine and Health Sciences, University of Antwerp, 2610 Wilrijk, Belgium
2
Oscare, Organisation for Burns, Scar After-Care and Research, 2170 Antwerp, Belgium
3
UQ Child Health Research Centre, The University of Queensland, 62 Graham St, South Brisbane, QLD 4101, Australia
4
Australian Centre for Health Services Innovation (AusHSI), School of Public Health and Social Work and Centre for Healthcare Transformation, Queensland University of Technology, 60 Musk Avenue, Kelvin Grove, QLD 4059, Australia
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Eur. Burn J. 2022, 3(2), 290-308; https://doi.org/10.3390/ebj3020025
Submission received: 16 February 2022 / Revised: 25 March 2022 / Accepted: 31 March 2022 / Published: 5 April 2022
(This article belongs to the Special Issue Innovation in Burn Scar Prevention and Management)
Browse Figures
Review Reports Versions Notes

Abstract

:
In burn scar rehabilitation, electronic patient-reported outcome measures (ePROMs) are increasingly being used in research and clinical settings as part of patient- and family-centred care. These measures can identify patients’ needs and monitor the therapeutic progress of both adults and children. The feedback of information from ePROMs to clinicians treating patients with scarring and psychosocial issues may have therapeutic benefits. However, testing the effectiveness of ePROMs used in the routine clinical care of patients with burn scarring is in its infancy, and one of the greatest challenges remains the implementation of ePROMs in real-world clinical settings. The aim of this paper is to provide a guide for clinicians and researchers involved in burn scar rehabilitation to assist in implementing ePROMs in clinical settings. The guide outlines strategies, processes, and considerations for ePROM implementation and the accompanying resources. Two real-world case studies of ePROM implementation are presented in burn scar clinics in Belgium and Australia. Additionally, ten recommendations for the implementation of ePROMs are provided based on research evidence and the lessons learned by the authors. The information provided should pave the way forward for using and testing these ePROMs in research and practice.

1. Introduction

Increased attention is being given to patient- and family-centred care in which the voices of patients and families are heard and where they play an active role not only in treatment but also in research as research partners [1,2]. Patient-reported outcome measures (PROMs) are being used in research and clinical settings as part of patient-and family-centred care to identify patients’ needs and monitor the therapeutic progress of both adults [3] and children [4]. The COVID-19 pandemic has helped to highlight the value of PROMs as part of telehealth interventions [5]. In the words of Nick Black—a leader in PROM research, “Patient Reported Outcome Measures could help transform healthcare” [6].
PROMs are questionnaires completed by patients which measure their perceived health status, symptoms, functional status, or (health-related) quality of life [7]. These measures may be either generic (not burn-specific) or condition-specific (specific to the burn injury or scarring) and may be used for multiple purposes, including screening, diagnosis, prognosis, follow-up, and planning. Routine clinical assessment of patients’ treatment, scarring, and psychosocial functioning [8] can be achieved using PROMs, with potential therapeutic benefits [9,10]. To achieve this, PROMs are ideally chosen to capture areas that are most meaningful to the patients and their families.
Testing the effectiveness and implementation of the patient-reported outcome measures used in the routine clinical care of scars is in its infancy. However, rigorous studies across a range of areas in health care have identified some positive findings. Studies examining the acceptability and feasibility of these measures in clinical practice have indicated generally positive findings for patient–clinician communication [5] but mixed findings for integration within the existing workflows and an unclear impact on patient outcome and clinical parameters [11,12,13,14]. A recent Cochrane systematic review found that PROMs feedback probably improves quality of life and increases patient–physician communication, diagnosis and notation, and disease control with moderate certainty [15]. However, further rigorous research is needed to examine the effectiveness and implementation of using ePROMs in burn scar care. Using implementation science could assist in reducing the evidence-to-clinical-practice gap [16].
As in other fields of care, burn scar care is faced with a clear shift towards digital health and e-health. The development of the internet, mobile information-sharing technologies, and the widespread use of computer tablets and smartphones has led to the emergence of electronic PROMs (ePROMs). Generic, burn- and scar-specific PROMs that were originally developed using pen and paper have been transferred into ePROMs and electronic care pathways in clinical practice.
For centres where paper-based PROMs have already been implemented, potential benefits of moving to ePROMs have been reported. These include greater alignment with patient preferences and acceptability and lower costs, and similar or faster completion times for ePROMs. Higher data quality and response rates, and better symptoms’ management and patient–clinician communication, have also been reported [17]. However, one of the greatest challenges remains the implementation in real-world clinical settings.
The aim of this paper is to provide a guide for clinicians and researchers involved in burn scar rehabilitation in order to assist with the implementation of ePROMs in the clinical setting. This implementation may be conducted as part of a research, clinical, or quality assurance initiatives; and targets healthcare clinicians as well as policy makers and researchers working in acute hospital, subacute, or after-care settings delivering burn care. It is intended that the guide and evidence base presented will be refined and updated as it is tested and applied in practice.

2. Organisation of the Paper and Context behind Recommendations

Our paper is organised into three sections. Firstly, we present the processes and strategies to support our guide to implementing ePROMs by addressing issues related to technology platforms and systems, privacy and confidentiality, different modes of presentation (for example, paper and electronic), PROM psychometrics, and the acceptability and feasibility of ePROM implementation in practice. Secondly, we present two case studies illustrating these processes. Finally, we present our top 10 recommendations for ePROM implementation based on the lessons we have learned in delivering these measures as part of burn scar care.
Systematic reviews, best-practice guidelines, and original studies in burns and scars have been used to formulate the guide. Systematic reviews on the barriers to and facilitators of the implementation of ePROMs have informed the processes and strategies outlined [17,18]. Best-practice guidelines have informed guidance regarding the selection of outcome measures (e.g., ISOQOL [19,20] and COSMIN guides [21]), paediatric administration [22], and moving from paper-based to electronically administered PROMs [23]. These guidelines have been detailed in Appendix A.
The conceptual frameworks drawn upon include those developed in the field of burn scar management, quality-of-life and well-being research—the ultimate goal of burn scar care, and implementation science. These frameworks have informed the selection of the outcomes, study processes, and evaluation processes underlying the case studies. The experience of the authors in developing, validating, and implementing scar-related ePROMs and digital scar pathways have also contributed to the guide. For example, this experience has highlighted the importance of obtaining feedback from patients with burn scars and their families regarding the timing, length, content of PROMs, and IT barriers early in the implementation process. Questions commonly asked by clinicians and researchers seeking to commence the implementation of ePROMs in burn settings, as well as broader hospital settings, have also been covered in our guide. These questions have arisen as part of three local and international special interest groups that one of the authors has regularly attended over the last 2 years (in implementation science, quality of life research, and implementing PROMs and PREMs), and in the clinical settings where the authors conduct their research.

3. Body Section

3.1. Processes and Strategies for Implementing ePROMs

3.1.1. Preparing for Implementation

  • Population
Whether or not you work in the burn scar rehabilitation setting or beyond, considering the characteristics of the target population, including age and technology literacy, are important. There are unique challenges when implementing child and adolescent PROMs. It has been established that reliability and validity of self-report is questionable below the age of 8 years. For the ages of 8 to 11 years, the reliability and validity of the child report improves, and between 12 and 18 years, self-report is preferred [22]. Research has shown that for children ePROM administration using the internet can be feasible, reliable, and valid [24,25,26]. Screen-based modes of administration can help children stay focused and engaged and improve the quality of the self-reported data, while minimizing missing data [17]. Overall, younger patients prefer ePROMs over paper formats, as they may be more familiar with the use of the internet and have access to necessary technological resources [17,27,28,29].
It should be noted that ePROM research with children requires careful attention to all aspects of the data collection process. Researchers need to use age-appropriate language for explaining the study purpose and procedures to children. Before children begin completing ePROMs, interviewers or other study staff members should inform them of what will be required of them, the purpose of the questions that will be asked, the intended use of the data, the confidentiality procedures, and what to do if they become uncomfortable or want to stop participating. While some children may not entirely comprehend the study details, researchers should make an effort to give children a general understanding and ensure that they feel comfortable [22].
People who are computer illiterate, older, or have no access to infrastructure could potentially be disadvantaged when ePROMs are implemented [29,30,31,32,33]. Be aware of the digital divide: some patients may be less willing or even unable to complete ePROMs without assistance due to computer illiteracy or to having no access to the internet or technological devices [17]. Checking the functional ability, mental health, and cognitive capacity of the target population [23] is advised as challenges in these areas may interfere with timely completion. Our experience indicates that in some cases the burden of completion may be too high and alternate methods of eliciting information are needed, as outlined in the section addressing equity.
Knowledge regarding the technology literacy of the target population and patients is important (i.e., experience using digital technology and computers, access to a computer/tablet/smartphone, or internet access at home). Limited experience by patients in using technology may affect their satisfaction, preference, and willingness to use electronic formats [17]. An educational session may not be necessary for everyone; however, it may be indispensable when working with children, people working with tablets for the first time, and computer illiterate and older people to provide instructions and determine whether assistance is required [25,32,34,35].
  • Context and setting
The ability for ePROMs to be completed in the clinical setting, research area, or home setting should be checked prior to implementation. This includes checking internet connectivity in all the locations where ePROMs will be administered and being aware that medical equipment may interfere with the signal.
  • Resources
Appendix A provides resources for researchers and clinicians covering the relevant frameworks, guidelines, reports, and papers to consider for different aspects of ePROM implementation in burn scar rehabilitation.
Other resources to consider are the specific equipment that is necessary for starting ePROM implementation. Be aware of one-time investments and costs at the start. Although overall the implementation of ePROMs can be more economic than the usual care [27,36], an initial one-time large investment may be needed. This may involve the purchasing of tablets or wireless printers (if paper printing is necessary), the hiring of computer programs or infrastructure for the collection of data, and the costs for online and technology support. When using a web-based platform, access to the internet is needed and may require procuring cellular (3G/4G) internet.
  • Mode of administration
The implementation of ePROMs has been reported to facilitate improved data quality, completion times equal to or faster than paper-based PROMs, lower administration times, and better clinical decision making and symptom management [17]. However, the consideration of special populations is required when implementing web-based questionnaires with, for example, patients with low economic resources and patients unfamiliar with internet use [27].
For the transition of paper PROMs to ePROMs, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) offers a clear framework for decisions regarding the level of evidence needed to support the modifications that are made to PROMs when they are migrated from paper to electronic devices. Three levels of modification (minor, moderate, and substantial) of the original paper-based PROM to ePROM have been reported, and an effective strategy for testing measurement equivalence (reliability and validity) has been provided [20]. When altering the Mode of Administration (MOA) from paper to tablet, moderate modifications (such as splitting a single item across multiple screens, requiring the patient to use a scroll bar to see all the items or responses) generally require equivalence testing together with usability testing. More recent evidence suggests that previous usability evidence in a representative group is sufficient to assume equivalence, as opposed to per-study testing [37]. Depending on the level of change, additional testing may be required to establish the equivalent reliability and feasibility of ePROMs [34]. If equivalence has been established, then setting up the PROM to correspond to the way it was validated will maintain validity, such as, for example, placing the same questions on each page rather than being split across pages.
Bring your own device (BYOD), where patients use their own device for ePROM administration, seems likely to be the preferred method of administration in the future [37]. There are however technical and practical considerations to take into account. BYOD may reduce costs and allow patients to work on familiar equipment. Things to keep in mind are that patients may turn off in-app notifications, remove the study app, change devices, run out of data or device storage, and be interrupted by other activities on the device [37].
The feasibility and utility of implementing an ePROM for burns has been examined in the US, using the Young Adult Burn Outcome Questionnaire (YABOQ), with real-time benchmarking feedback in a burn outpatient practice [9]. That study examined the data of 12 patients, aged 19–30 years, 1–24 months from injury, who completed the PROM and demographic data on an iPad in the office before outpatient visits. The study found preliminary evidence of the feasibility and potential utility of the real-time use of the burn-specific ePROM. Qualitative results supported the hypotheses that ePROMs can facilitate communication between patient and provider in burn outpatient settings and help providers identify the clinical issues to address. [9]. These findings have been confirmed in our own work, which has indicated the feasibility of ePROM implementation in burn scar rehabilitation clinics if barriers can be addressed [10,38].

3.1.2. Selecting, Administering, and Scoring PROMs

  • Selecting
Clinicians may feel it is within their role to select PROMs for implementation but obtaining the input of researchers may be valuable to review the psychometric properties of the appropriate measures. Involving patients in the selection is vital to ensure the content, wording, and layout is appropriate. Choosing an appropriate PROM should take into account the population, the purpose for administration (diagnostic/screening, prognostic, and monitoring), validity, content, recall period, time required for administration, available languages, and ease of use [39]. Questions that clinicians and researchers can ask themselves to guide the selection of PROMs have been added to the resources (Appendix A).
We suggest that clinicians and researchers select burn scar outcomes based on qualitative work that exists in relation to these outcomes, in the absence of consensus regarding the outcomes of importance for people with burn scars. Some of this qualitative work has been included in Appendix A, although it may not be all-encompassing. Qualitative methods of identifying relevant outcomes have been recommended as a precursor to consensus approaches to ensure patient perspectives are incorporated in the final core outcome sets [39].
  • Administering
ePROMs can be built into readily available platforms and packages, including Qualtrics, Question Pro, REDcap, Survey Gizmo, and Survey Monkey. Some organisations will have system-wide site licences for these platforms and packages that can be accessed.
During the in-centre or hospital administration of ePROMs, technical and practical issues need to be thought through in advance. This should include determining whether it is possible for the patient to complete the ePROM alone or whether assistance is required and, if a tablet is used, having a secure location where it can be stored (or adding a security cable lock and security code). Making sure instructions are available for each specific ePROM, either verbally or as part of the electronic delivery, should be considered to provide context regarding what the information is being used for and how to complete the ePROMs.
Differences in response options between paper-based and electronic PROMs should be considered when selecting the mode of administration. For electronic measures, it has been suggested that respondents should be able to opt out of answering questions or be able to skip questions [40]. This needs to be considered when applying settings as survey platforms such as Qualtrics require participants to respond to each question before proceeding unless opting out is applied to the forced choice setting.
  • Scoring
It is important to consider that implementing electronic score calculations in a web-based platform is a one-time effort and human resources cost. Once calculations are implemented and tested, all future responses on the questionnaires are calculated automatically and are thus instantly available. Scoring questionnaires electronically provides the benefit of scalability and having the results available immediately.

3.1.3. ePROM Results, Feedback, Evaluation, and Training

Health professionals may need to consider monitoring ePROM results even when ePROMs are used in research as ePROMs are not neutral activities; they can change the way patients think about their condition [11] and may trigger emotional responses [41]. Visualisation of the results (for example, using graphical displays or visual aids) seems to be valued by patients [42], and using a historical timeline can give the patient and healthcare professional better insight into the evolution of their health status [31,43,44]. Something to be aware of is that exposure to the ePROM itself can increase the willingness to use it [30], and reviewing the results with a healthcare professional is associated with increased odds of perceiving ePROMs as beneficial [45].
There are many ways in which the ePROM results can be used clinically. For instance, automatic scoring of the PROMs in real time may highlight surprisingly low or high scores for a PROM domain or item and thus indicate the need for open communication and discussion of issues that otherwise may not have been picked up. Identifying patient needs early may contribute to earlier identification of the need for multidisciplinary input. Remote patient follow-up using ePROMs is also possible and may allow more time for care during in-person visits to a burn (after-care) centre. Finally, the ePROM results can be discussed alongside the findings of clinical or physical measurements and may reveal aspects of functioning and disability that give additional insight into the impact of the burn scarring on the patient or the burden of treatment.
To evaluate whether or not the implementation of ePROMs is successful, patient and clinical outcomes, implementation outcomes [46], and mediator/proximal variables (for example, perceived relative advantage of ePROMs) should be considered [47]. Measures that could be considered for evaluation include Patient Experience Measures (for example, satisfaction with training and discussion of ePROM results), and assessing organisational readiness to implement ePROMs [47].
Experts in the field of PROM implementation have recommended training as a critical strategy for PROM implementation [47]. The features of training that should be considered are included in the accompanying resources (Appendix A). However, a recent review of RCTs evaluating the use of PROMs as interventions, which identified training of clinicians regarding PROMs prior to trial commencement, resulted in no obvious impact on the results [48]. Thus, further evidence is needed regarding the effect of training.

3.1.4. Overarching Considerations

  • Ethical considerations
Health professionals having the capacity and desire to respond to issues that are identified on ePROMs is an important ethical consideration when implementing these measures routinely [11].
  • Security
Security requirements for ePROMs will vary depending on the legislation, policy culture, health authority, national policies, and contexts [35]. Early collaboration with Information Technology (IT) personnel is advised in order to overcome organisational IT barriers and obtain support. The storage of information overseas may not be legal in some jurisdictions unless participants are fully informed of the implications and provide written consent. Stored information may need to be non-identifiable in these cases, which may limit the information that can be captured electronically.
  • Policy and culture
Broader health service policy and internal culture may influence the implementation and sustainability of ePROM implementation and thus should be considered early. For example, in health services or departments where ePROM implementation is identified as a priority, the resources to support implementation (such as IT support and solutions to overcome privacy concerns) may be readily accessible, or teams may be able to work together to overcome barriers.
  • Changing behaviour and readiness for implementing ePROMs
Remarkably consistent barriers to implementing ePROMs have been identified across conditions, settings, and countries, including IT issues, competing demands from existing workloads, and the time and cost of implementation [45]. Our own experience and findings from our case studies support this evidence. The strategies to address these barriers should go a long way towards being ready for implementation, if facilitators of implementation in local settings are identified and considered alongside barriers [47].
  • Equity of access
Equity of access should be considered to ensure that models of care based on ePROMs do not widen already wide disparities in access to healthcare and health outcomes [48]. Making ePROMs available in more than one language is one way of addressing this. If this is not possible, based on lack of cross-cultural validation of PROMs or limited resources to allow purchasing licences in different languages, then the collection of data to inform future equitable access to ePROMs should be considered. This could start as simply as collecting information on the social determinants of health or looking at the acceptability of using ePROMs in people from diverse cultural and linguistic backgrounds [49]. Alternate person-centred methods of PROM administration to elicit patient perspectives and facilitate communication could also assist in capturing the voice of people with health and literacy challenges [50]. These alternate methods could include creating photos or selecting photos from archives [42,50], video-based methods [18,51], digital storytelling [52], and Ecological Momentary Assessments (EMAs) [53]. EMAs are repeated assessments of behaviours and experiences in real-time natural environments, usually administered using technology such as electronic diaries, telephones, and sensors [54].
  • Conceptual frameworks
The use of conceptual frameworks can inform the development of implementation questions and hypotheses in research, the selection of strategies to assist implementation, and the identification of barriers and facilitators to implementation [53]. Recent papers on ePROM implementation [47] and implementation science [53] provide guidance on applying frameworks that may be useful for ePROM implementation, including Normalisation Process Theory [55], the Consolidated Framework for Implementation Research (CFIR) [56], and the Integrated Framework for Promoting Action on Research Implementation in Health Services (I-PARIHS) [57]. In situations when information from ePROMs is fed back to clinicians for use in clinical consultations or to patients, Feedback Intervention Theory may also be applicable in understanding how feedback from ePROMs might work to elicit a therapeutic response (the mechanism of action) [15,58]. The essence of this theory is that feedback can draw a person’s attention to gaps between their current and ideal health state, resulting in positive but also sometimes negative effects [15,58].
  • Dissemination of information
To promote equity, dissemination methods that resonate with key stakeholders or people with burn scars should be considered; these could include images or narratives [50]. Involving the target group, for example people with burn scars, in the preparation of information that will be disseminated is a strategy that may assist in ensuring that the messages are appropriately pitched (for example, they should be readable, unambiguous, relevant, and easily comprehended) [59]. Other methods likely to improve the success of dissemination include the use of social media to broaden access to health information [59]. Figure 1 illustrates the processes and strategies that should be considered at different stages of ePROM implementation.

3.2. Case Studies

Our case studies are real-world examples of implementing ePROMs in clinical practice from Europe and Australia. These examples include: (1) a digital pathway based on ePROM data in a community-based scar after-care centre for clinical decision making and (2) implementing ePROMs and graphical displays of information from ePROMs as part of a research initiative to guide the treatment of children with burn scars attending a children’s hospital. In Figure 2 and Figure 3, the context, outcome selection, ePROM intervention, and implementation considerations for each case are outlined.
In case study 1 on ePROM implementation in Belgium, the implementation was for research purposes as well as clinical follow-up during post-burn rehabilitation. Prior to the implementation, PROMs were already commonly administered in the setting. A literature review on the advantages and disadvantages of ePROMs [17] and a validation study on the chosen ePROMs [38] were performed prior to implementation. All patients eligible to receive after-care were recorded in the dashboard of the platform used. For every patient, the results of the assessments, ePROM results, and treatments were recorded and grouped per patient and per research study or specific care pathway in the platform. With the onset of the COVID-19 pandemic, the possibility of using their own device or smartphone to complete the ePROMs remotely or on-site was an extra advantage. The digital care pathway, Scarpath [60], allowed the continuation of the research follow-up with ePROMs during periods of restrictions and lockdowns when physical attendance for in-centre visits was postponed or not possible. In the burn scar rehabilitation setting where ePROMs were implemented, our mission was to be patient-centred and holistic, taking into account the biopsychosocial functioning of patients, which the initiative aligned with. When implementing the ePROM pathway, the physical and psychosocial aspects (for example, scar quality and overall quality of life) were bundled into one dashboard per patient which helped to gain a holistic patient-centred view of functioning at a specific moment in time or over time.
In case study 2 on ePROM implementation in Australia, the implementation was for research purposes initially. Prior to the implementation, PROMs were commonly administered in the burn scar setting in paper form but were not scored in real time. The final results of the research are yet to be published. Preliminary findings identifying the barriers and benefits of implementation indicated that the barriers included: completing PROMs at initial consultations in clinical units, which could be a barrier to natural communication; a lack of capacity by clinicians to respond to some of the issues identified; technology issues; and competing demands [10]. The benefits included use of PROMs that targeted areas of importance to families and relatively safe topics and high value placed on being asked about some topics that would not typically be raised [10].

3.3. Lessons Learned

Based on the lessons learned from our experiences and a review of the evidence, we have 10 recommendations for implementing ePROMs, as illustrated in Figure 4. We suggest that many of these recommendations should be considered prior to full implementation. Cognitive interviews, in which 5 to 10 patients in the target group (patients, parents, or clinicians) complete the PROMs as part of a ‘think-aloud’ exercise, are ideally conducted to establish the content validity of ePROMs in the study setting. From our own experience, both in clinical settings and in burn scar rehabilitation research, involving the target group in cognitive interviews is highly recommended. Having a stakeholder or advisory committee involved from the start of the initiative can contribute to successful implementation.

4. Discussion

This paper reports on the development of a guide to implement ePROMs in burn scar clinical practice, drawing on evidence, implementation case studies, and the authors’ experience. The guide presented should assist researchers and clinicians to consider important processes and strategies when deciding to implement PROMs in clinical practice as part of research, clinical, or quality assurance initiatives. Future work should validate the guide with input from the target group of clinicians conducting scar management as well as by involving patients and families.
In the ideal world, synergy can develop across the stages of burn scar care by capturing the outcomes that matter to patients in each of these stages using ePROMs (and electronic treatment pathways). The first steps in this direction have been made by the Burn Centers Outcome Registry of the Netherlands (BORN), Scarpath Belgium, and the implementation of paediatric ePROMs in Australia (PEDS-ePROM study). For the implementation of ePROMs to be fully realised, understanding the costs of implementing these measures and the impact on long-term outcomes to patients will remain critical to deliver even better value in burn scar rehabilitation [65]. Using feedback from ePROMs implemented in routine burn scar clinical care there is an opportunity to reshape future burn scar rehabilitation.
In the future, the use of the guide and the implementation of ePROMs need to be evaluated for the impact on clinical, patient, health service, and implementation outcomes and tailored for specific contexts. Patient outcomes may include health-related quality of life, communication, adherence to treatment, and reconstructive and psychosocial outcomes. Further work will also be needed to determine how the guide can inform policy [66] and practice, for example in the provision of incentives and support for the use of ePROMs in health services.
Limitations in the development of our guide include a limited evidence base for implementing ePROMs in a burn scar rehabilitation context, involving two countries (Belgium and Australia); thus, broader evidence has been drawn upon. The remarkable consistency of barriers identified across the studies and settings means barriers may be generalisable across conditions and settings, but the facilitators of implementation are likely to be more context-specific [47]; thus, there is a need further examination in relation to implementing ePROMs locally in burn scar rehabilitation settings. This generalisability of the barriers may not extend to low-resource settings as most studies have been conducted in high-resource settings. Work is now needed in low-resource settings to determine the transferability of the processes, strategies, and recommendations in the guide. The potential benefits of ePROMs in these settings include rapid assessment and the use of data to inform assessment, service provision, and public health initiatives [67].

5. Conclusions

The guide presented in this paper and the accompanying resources should be viewed as a starting point for implementing ePROMs, with the ultimate goal of enhancing communication with clinical teams and the health-related quality of life of patients with burn scars. As well as addressing the barriers to implementing PROMs that come from the broader literature, researchers and clinicians implementing PROMs should consider factors that impact on implementation in their own setting to tailor the implementation. Refinement of the guide will be required in the future as the ePROM evidence advances in relation to implementing ePROMs in burn scar rehabilitation settings and as more original studies are conducted in multidisciplinary, cross-cultural, and international contexts.

Author Contributions

Both authors, Z.T. and J.M., had equal contribution to the conceptualization, methodology, resources, writing—original draft preparation, writing—review and editing, and visualization. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding except for the research underlying the Australian case study which was funded by the Children’s Hospital Foundation, Australia.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable except for the case studies of implementing ePROMs. Individual written consent was obtained for participating in research linked to the case studies but as the case studies are presented at the service level or by referring to previously published findings this is not detailed in the case studies.

Data Availability Statement

Not applicable.

Acknowledgments

We acknowledge the people and patients who contributed to the research and clinical implementation of ePROMs in the case studies, some of whom are listed in the references linked to the in-text citations in the case studies.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A. Resources to Support Choosing and Implementing PROMs in Routine Clinical Care

Broad DescriptionKey Information and Relevant Points of Attention
Clinical guidelines and processesISOQOL user’s guide implementing PROMs in clinical practice. Version 2. 2015This guide aims to help clinicians interested in using PROMs in their clinical practice to think through key aspects of implementation.
Questions addressed include:
1. What are your goals for collecting PROs in your clinical practice?
2. What resources are available?
3. Which key barriers require attention?
4. Which groups of patients will you assess?
5. How do you select which questionnaire to use?
6. How often should patients complete questionnaires?
7. How will the PROs be administered and scored?
8. What tools are available to assist in interpretating PROMs?
9. When, where, how, and to whom will results be presented?
10. What will be done to respond to issues identified through PROMs? [11]
ISOQOL companion guide and users guideThis guide builds on the user’s guide to assist clinicians to address considerations involved in implementing and using PROMs in clinical care using information from real-world case studies [20].
A PROM implementation cycle across four phases is presented with accompanying instruments, checklists, methods, handbooks, and standards. The four phases of implementation are:
1. Goal setting
2. Selecting PROs and PROMs
3. Developing and testing quality indicators
4. Implementing and evaluating the PROMs and indicators [68].
Selecting outcomes and evaluation in burn and burn scar populationsPatient-Reported Outcomes in routine care—a true innovation but only if used correctlyChoosing the right PROMs depends on the purpose of use, for example whether it is for clinical use or for research, for a single measure or for longitudinal follow-up.
This paper suggests three questions to prepare for implementation:
1. Is the goal to measure change at an individual or a group level?
2. Do the PROMs ask questions that are relevant to my patients, clinic, and/or research?
3. Is the PROM validated for my population of interest? [69]
Systematic review of PROMs used in adult burn research in articles from January 2001 to September 2016 *Thirteen generic PROMs were reported to have evidence of validation data with English speaking adults with burn injuries:
1. Perceived Stigmatization Questionnaire (PSQ)
2. Social Comfort Questionnaire (SCQ)
3. Satisfaction with Appearance Scale (SWAP)
4. Short Form 36-item Medical
5. Outcomes Survey (SF-36)
6. DASH and QuickDash
7. POSAS
8. LLFI-10
9. Community Integration Questionnaire
10. Brief Cope
11. McGill Pain Scale
12. Brief Fatigue Inventory
13. Davidson Trauma Scale
Four burn-specific PROMs were reported to have been validated in English with adults with a burn:
1. Burn-Specific Health Scale-Abbreviated (BSHS-A)
2. Burn-Specific Health Scale-Brief (BSHS-B)
3. Young Adults Burns Outcomes Questionnaire (YABOQ)
4. Burn-Specific Pain Anxiety Scale (BSPAS) [3]
Systematic review of PROMs used in child and adolescent burn research from January 2001 to March 2013 *Two generic PROMs were validated in English with children and adolescents with burns:
1. Perceived Stigmatisation Questionnaire
2. Social Comfort Scale
One burn-specific PROM was validated with adolescents with burns:
1. Children Burn Outcomes Questionnaire for children aged 5–18
Note: the search was limited to articles that had been written up for research purposes [4]
Outcomes important to patients with burns during scar management and comparison to burn-specific PROMs based on qualitative researchEight core outcome domains were identified as important to children and adults in burn scar management:
1. scar characteristics and appearance
2. movement and function
3. scar sensation
4. psychological distress, adjustments, and a sense of normality
5. body image and confidence
6. engagement in activities
7. impact on relationships
8. treatment burden [70]
Seeding the value-based healthcare and standardised measurement of quality of life after burn debate *Call to use the same quality of life measure at 4 weeks and 3 months post burn. Quality of life measures for the adult burn population established and emerging were: SF-36, EQ-5D, BSHS-B, VR-36 and LIBRE profile. [71]
Strong suggestion for teams to pilot a standardised schedule of administration at 4–6 weeks, 3 months, 6 months, 12 months, and 24 months after burn injury date.
Core Outcome Set for Burns (COSB) which may have relevance in choosing outcomes to measure preventing or treating patients with burn scars *Consensus methods used to identify the seven highest ranked outcomes for burns involving clinicians and patients across different stages of burn care:
1. death
2. specified complications
3. ability to do daily tasks
4. wound healing
5. neuropathic pain and itch
6. psychological well-being
7. time to return to work/school [72]
A systematic review of the quality of burn scar rating scales for clinical and research use and guide for choosing burn scar scales based on an updated systematic reviewQuality of burn rating scales:
  • The Patient and Observer Scar Assessment Scale (POSAS) - high quality rating for reliability of the total score and vascularity subscale.
  • The Vancouver Scar Scale (VSS) - indeterminate ratings for construct validity, reliability, and responsiveness.
  • The other 17 scales – indeterminant ratings due to methodological issues. [73]
Guide to selecting burn scar rating scales for clinical practice:
The content, purpose, test sample characteristics, and feasibility of eight scales is reviewed in depth, including POSAS, VSS, modified VSS, Matching Assessment using Photographs with Scars (MAPS), and Visual Analogue Scale [74].
Checklists for assessing study quality related to PROM development and validation by COSMIN Available checklists can be used to assess the methodological quality of PROMs and studies on PROM development or testing. Checklists have been developed by the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) group and include:
  • COSMIN Risk of Bias checklist for PROMs
  • COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments
  • COSMIN Study Design checklist [21]
Barriers to PROM implementationRemarkably consistent barriers to the implementation of PROMs have been reported across settings and studies and should be considered prior to implementationFour consistent barriers to PROM implementation have been identified:
1. Technology
2. Stakeholder uncertainty about the use of PROMs
3. Stakeholder concerns about negative impacts of PROM use
4. Competing demands from established clinical workflows [47]
Difficulty tailoring to individual patients has also been identified as a barrier to implementation in a review of systematic reviews. [18]
Measurement equivalence of paper and ePROMsRecommendations on evidence needed to support measurement equivalence between paper and ePROMsA general framework for decisions regarding the level of evidence needed to support modifications that are made to PROMs when they are migrated from paper to ePROM devices.
Key issues:
  • Determination of the extent of modification required to administer the PROM on an electronic device (minor, moderate, substantial)
  • The selection and implementation of an effective strategy for testing the measurement equivalence of the two modes of administration [40].
A recent review suggests that previous usability evidence in a representative group is sufficient to assume equivalence, as opposed to per-study testing: Evidence and Recommendations for Clinical Trials and Bring Your Own Device [37].
Consider the nature of each response scale in the instrument to evaluate the appropriateness for migration to the target mode. Response scales such as graduated circles may be difficult to transition from paper to electronic platforms without alteration [75].
Pediatrics considerationsReport of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and Patient Reported Outcomes (PRO) on good research practices for the assessment of children and adolescents task force Good research practices outlined are related to:
  • Developmental differences and age-based criteria for PROM administration;
  • Content validity of pediatric PROMs (include children as content experts where possible)
  • Whether informant-reported outcome instruments are needed
  • Appropriate design and format for the target age group
  • Cross-cultural issues
Recommend that a child’s social and developmental contexts be captured in selected paediatric PROMs (e.g., family, school, peer contexts) [17]
Include child self-report perspectivesChild self-report about what makes a good life should be prioritised and may differ to parent report [76,77,78]. Where it is difficult to obtain self-report, it has been suggested that proxy report be obtained alongside child self-report where possible [78].
Frequently used generic PROMsSix PROMs were evaluated to report health states:
1. Pediatric Quality-of-Life inventory 4.0 (PedsQL)
2. Child Health Questionnaire (CHQ)
3. KIDSCREEN
4. KINDL
5. DISABKIDS
6. Child Health and Illness Profile (CHIP).
All capture domains of physical, social, emotional health, and school activities. No measure appropriately captured all relevant age-appropriate domains in PROMs:
  • Parent relations domain is discussed as vital for child development and well-being. Covered by: CHQ, KIDSCREEN, KINDL (domain family), and CHIP (domain resilience). Not covered by PedsQL.
  • Financial resources domain measuring the child’s perception of their financial resources is covered by KIDSCREEN.
The earliest age self-reported PROMs are available is 4 years [79].PROMIS paediatric scales were not considered by Arsiwala et al. (2021) [79]. PROMIS paediatric scales have the advantage of being able to be administered using Computerised Adaptive Testing (CAT’s), allowing items to be tailored to individual patients [80]. Considerations to support use of PROMIS scales in paediatric ambulatory care settings have been outlined, including care transitions and privacy. [80]
Additional resourcesTraining clinicians to use PROMs in clinical practiceDrawing from case studies of training programs for PROM implementation across three countries and in diverse clinical areas (adult oncology, lung transplant, paediatrics) it was concluded that training sessions should be:
  • Brief
  • Timed to fit with existing organizational practices
  • Flexible (group or individual and e-training sessions).
The most successful element of training was identified as experiential problem-based learning using video/audio-clips and real patient cases. [81]
Best Practices for Migrating Existing Patient-Reported Outcome Instruments to a New Data Collection ModeCreated by an ePRO consortium. Addresses issues that should be considered when migrating existing patient-reported outcome (PRO) instruments to any available data collection mode (e.g., paper, interactive voice response [IVR] system, tablet, web, handheld) [23]
* The relevance of burn outcomes to those with burn scars is not clear; thus, we advocate prioritising research targeting burn scar populations to guide the selection of PROMs for people with burn scars until further evidence emerges regarding the relevance of burn outcomes to people with burn scars.

References

  1. Rivera, S.C.; Mercieca-Bebber, R.; Aiyegbusi, O.L.; Scott, J.; Hunn, A.; Fernandez, C.; Ives, J.; Ells, C.; Price, G.; Draper, H.; et al. The need for ethical guidance for the use of patient-reported outcomes in research and clinical practice. Nat. Med. 2021, 27, 572–573. [Google Scholar] [CrossRef] [PubMed]
  2. Tyack, Z.; Simons, M.; McPhail, S.M.; Harvey, G.; Zappala, T.; Ware, R.S.; Kimble, R.M. Improving the patient-centred care of children with life-altering skin conditions using feedback from electronic patient-reported outcome measures: Protocol for a hybrid effectiveness-implementation study (PEDS-ePROM). BMJ Open 2021, 11, e041861. [Google Scholar] [CrossRef] [PubMed]
  3. Griffiths, C.; Guest, E.; White, P.; Gaskin, E.; Rumsey, N.; Pleat, J.; Harcourt, D. A Systematic Review of Patient-Reported Outcome Measures Used in Adult Burn Research. J. Burn Care Res. 2017, 38, e521–e545. [Google Scholar] [CrossRef] [PubMed]
  4. Griffiths, C.; Armstrong-James, L.; White, P.; Rumsey, N.; Pleat, J.; Harcourt, D. A systematic review of patient reported outcome measures (PROMs) used in child and adolescent burn research. Burns 2014, 41, 212–224. [Google Scholar] [CrossRef] [PubMed]
  5. Aiyegbusi, O.L.; Nair, D.; Peipert, J.D.; Schick-Makaroff, K.; Mucsi, I. A narrative review of current evidence supporting the implementation of electronic patient-reported outcome measures in the management of chronic diseases. Ther. Adv. Chronic Dis. 2021, 12. [Google Scholar] [CrossRef] [PubMed]
  6. Black, N. Patient reported outcome measures could help transform healthcare. BMJ 2013, 346, f167. [Google Scholar] [CrossRef] [Green Version]
  7. Dawson, J.; Doll, H.; Fitzpatrick, R.; Jenkinson, C.; Carr, A.J. The routine use of patient reported outcome measures in healthcare settings. BMJ 2010, 340, c186. [Google Scholar] [CrossRef]
  8. National Burn Care Review Committee. A Review of Burn Care in the British Isles; National Burn Care Review Committee: London, UK, 2001; pp. 1–6. [Google Scholar]
  9. Ryan, C.M.; Lee, A.F.; Kazis, L.E.; Shapiro, G.D.; Schneider, J.C.; Goverman, J.; Fagan, S.P.; Wang, C.; Kim, J.; Sheridan, R.L.; et al. Is Real-Time Feedback of Burn-Specific Patient-Reported Outcome Measures in Clinical Settings Practical and Useful? A Pilot Study Implementing the Young Adult Burn Outcome Questionnaire. J. Burn Care Res. 2016, 37, 64–74. [Google Scholar] [CrossRef] [Green Version]
  10. Tyack, Z.; Simons, M.; Zappala, T.; Harvey, G.; McPhail, S.; Kimble, R. Barriers and benefits to the routine use of electronic patient-reported outcome measures in treating children with life altering skin conditions. In 27th Annual Conference of the International Society for Quality of Life Research. Qual Life Res. 2020, 29 (Suppl. S1), S46. [Google Scholar]
  11. Greenhalgh, J.; Gooding, K.; Gibbons, E.; Dalkin, S.; Wright, J.; Valderas, J.; Black, N. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J. Patient Rep. Outcomes 2018, 2, 1–28. [Google Scholar] [CrossRef]
  12. Velikova, G.; Booth, L.; Smith, A.B.; Brown, P.M.; Lynch, P.; Brown, J.M.; Selby, P.J. Measuring Quality of Life in Routine Oncology Practice Improves Communication and Patient Well-Being: A Randomized Controlled Trial. J. Clin. Oncol. 2004, 22, 714–724. [Google Scholar] [CrossRef]
  13. Krawczyk, M.; Sawatzky, R. Relational use of an electronic quality of life and practice support system in hospital palliative consult care: A pilot study. Palliat. Support. Care 2018, 17, 208–213. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Espallargues, M.; Valderas, J.M.; Alonso, J. Provision of Feedback on Perceived Health Status to Health Care Professionals. Med. Care 2000, 38, 175–186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Gibbons, C.; Porter, I.; Gonçalves-Bradley, D.C.; Stoilov, S.; Ricci-Cabello, I.; Tsangaris, E.; Gangannagaripalli, J.; Davey, A.; Gibbons, E.J.; Kotzeva, A.; et al. Routine provision of feedback from patient-reported outcome measurements to healthcare providers and patients in clinical practice. Cochrane Database Syst. Rev. 2021. Issue 10. Art. No. CD011589. [Google Scholar] [CrossRef]
  16. Bauer, M.S.; Kirchner, J. Implementation science: What is it and why should I care? Psychiatry Res. 2019, 283, 112376. [Google Scholar] [CrossRef] [PubMed]
  17. Meirte, J.; Hellemans, N.; Anthonissen, M.; Denteneer, L.; Maertens, K.; Moortgat, P.; Van Daele, U. Benefits and Disadvantages of Electronic Patient-reported Outcome Measures: Systematic Review. JMIR Perioper. Med. 2020, 3, e15588. [Google Scholar] [CrossRef] [PubMed]
  18. Foster, A.; Croot, L.; Brazier, J.; Harris, J.; O’Cathain, A. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: A systematic review of reviews. J. Patient Rep. Outcomes 2018, 2, 1–16. [Google Scholar] [CrossRef]
  19. Aaronson, N.; Choucair, A.; Elliott, T. User’s guide to implementing patient-reported outcomes assessment in clinical practice. Int. Soc. Qual. Life Res. 2015, Version 2, 1–47. Available online: https://www.isoqol.org/wp-content/uploads/2019/09/2015UsersGuide-Version2.pdf (accessed on 25 March 2022).
  20. Chan, E.K.H.; Edwards, T.C.; Haywood, K.; Mikles, S.; Newton, L. Implementing patient-reported outcome measures in clinical practice: A companion guide to the ISOQOL user’s guide. Qual. Life Res. 2018, 28, 621–627. [Google Scholar] [CrossRef]
  21. Cosmin Checklists for Assessing Methodological Study Quality. Available online: https://www.cosmin.nl/tools/checklists-assessing-methodological-study-qualities/ (accessed on 25 March 2022).
  22. Matza, L.S.; Patrick, D.L.; Riley, A.W.; Alexander, J.J.; Rajmil, L.; Pleil, A.M.; Bullinger, M. Pediatric Patient-Reported Outcome Instruments for Research to Support Medical Product Labeling: Report of the ISPOR PRO Good Research Practices for the Assessment of Children and Adolescents Task Force. Value Heal. 2013, 16, 461–479. [Google Scholar] [CrossRef] [Green Version]
  23. ePRO Consortium. Best Practices for Migrating Existing Patient-Reported Outcome Instruments to a New Data Collection Mode. 2014. Available online: https://c-path.org/wp-content/uploads/2014/05/BestPracticesForMigratingExistingPROInstrumentstoaNewDataCollectionMode.pdf (accessed on 25 March 2022).
  24. Mangunkusumo, R.T.; Moorman, P.W.; Ruiter, A.E.V.D.B.-D.; Van Der Lei, J.; De Koning, H.J.; Raat, H. Internet-administered adolescent health questionnaires compared with a paper version in a randomized study. J. Adolesc. Heal. 2005, 36, 70.e1–70.e6. [Google Scholar] [CrossRef]
  25. Raat, H.; Mangunkusumo, R.T.; Landgraf, J.M.; Kloek, G.; Brug, J. Feasibility, reliability, and validity of adolescent health status measurement by the Child Health Questionnaire Child Form (CHQ-CF): Internet administration compared with the standard paper version. Qual. Life Res. 2007, 16, 675–685. [Google Scholar] [CrossRef] [Green Version]
  26. Young, N.L.; Varni, J.W.; Snider, L.; McCormick, A.; Sawatzky, B.; Scott, M.; King, G.; Hetherington, R.; Sears, E.; Nicholas, D. The Internet is valid and reliable for child-report: An example using the Activities Scale for Kids (ASK) and the Pediatric Quality of Life Inventory (PedsQL). J. Clin. Epidemiol. 2009, 62, 314–320. [Google Scholar] [CrossRef]
  27. Engan, H.; Hilmarsen, C.; Sittlinger, S.; Sandmæl, J.A.; Skanke, F.; Oldervoll, L.M. Are web-based questionnaires accepted in patients attending rehabilitation? Disabil. Rehabil. 2016, 38, 2406–2412. [Google Scholar] [CrossRef] [PubMed]
  28. Keurentjes, J.C.; Fiocco, M.; So-Osman, C.; Onstenk, R.; Koopman-Van Gemert, A.W.M.M.; Poll, R.G.; Nelissen, R. Hip and knee replacement patients prefer pen-and-paper questionnaires. Bone Jt. Res. 2013, 2, 238–244. [Google Scholar] [CrossRef] [PubMed]
  29. Richter, J.G.; Becker, A.; Koch, T.; Nixdorf, M.; Willers, R.; Monser, R.; Schacher, B.; Alten, R.; Specker, C.; Schneider, M. Self-assessments of patients via Tablet PC in routine patient care: Comparison with standardised paper questionnaires. Ann. Rheum. Dis. 2008, 67, 1739–1741. [Google Scholar] [CrossRef] [PubMed]
  30. Hartkopf, A.D.; Graf, J.; Simoes, E.; Keilmann, L.; Sickenberger, N.; Gass, P.; Wallwiener, D.; Matthies, L.; Taran, F.-A.; Lux, M.P.; et al. Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients. JMIR Cancer 2017, 3, e11. [Google Scholar] [CrossRef] [PubMed]
  31. Jongen, P.J.; Sinnige, L.G.; van Geel, B.M.; Verheul, F.; Verhagen, W.I.; A van der Kruijk, R.; Haverkamp, R.; Schrijver, H.M.; Baart, J.C.; Visser, L.H.; et al. The interactive web-based program MSmonitor for self-management and multidisciplinary care in multiple sclerosis: Utilization and valuation by patients. Patient Prefer. Adherence 2016, 10, 243–250. [Google Scholar] [CrossRef] [Green Version]
  32. McCleary, N.J.; Wigler, D.; Berry, D.; Sato, K.; Abrams, T.; Chan, J.; Enzinger, P.; Ng, K.; Wolpin, B.; Schrag, D.; et al. Feasibility of Computer-Based Self-Administered Cancer-Specific Geriatric Assessment in Older Patients with Gastrointestinal Malignancy. Oncologist 2013, 18, 64–72. [Google Scholar] [CrossRef] [Green Version]
  33. Wintner, L.M.; Giesinger, J.M.; Zabernigg, A.; Rumpold, G.; Sztankay, M.; Oberguggenberger, A.S.; Gamper, E.M.; Holzner, B. Evaluation of electronic patient-reported outcome assessment with cancer patients in the hospital and at home. BMC Med. Informatics Decis. Mak. 2015, 15, 1–10. [Google Scholar] [CrossRef] [Green Version]
  34. Holzner, B.; Giesinger, J.M.; Pinggera, J.; Zugal, S.; Schöpf, F.; Oberguggenberger, A.S.; Gamper, E.M.; Zabernigg, A.; Weber, B.; Rumpold, G. The Computer-based Health Evaluation Software (CHES): A software for electronic patient-reported outcome monitoring. BMC Med. Informatics Decis. Mak. 2012, 12, 126. [Google Scholar] [CrossRef] [Green Version]
  35. Schick-Makaroff, K.; Molzahn, A. Strategies to use tablet computers for collection of electronic patient-reported outcomes. Heal. Qual. Life Outcomes 2015, 13, 1–6. [Google Scholar] [CrossRef] [Green Version]
  36. Touvier, M.; Méjean, C.; Kesse-Guyot, E.; Pollet, C.; Malon, A.; Castetbon, K.; Hercberg, S. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur. J. Epidemiol. 2010, 25, 287–296. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Byrom, B.; Gwaltney, C.; Slagle, A.; Gnanasakthy, A.; Muehlhausen, W. Measurement Equivalence of Patient-Reported Outcome Measures Migrated to Electronic Formats: A Review of Evidence and Recommendations for Clinical Trials and Bring Your Own Device. Ther. Innov. Regul. Sci. 2019, 53, 426–430. [Google Scholar] [CrossRef] [PubMed]
  38. Meirte, J.; Hellemans, N.; Anthonissen, M.; Maertens, K.; Moortgat, P.; Van Daele, U. Equivalence and feasibility of the paper based and electronic version of the POSAS, EQ-5D and DLQI in a Belgian population. In Proceedings of the 18th European Burns Association Congress, Helsinki, Finland, 4–7 September 2019. Conference report. [Google Scholar]
  39. Jones, J.; Jones, L.; Keeley, T.J.H.; Calvert, M.J.; Mathers, J. A review of patient and carer participation and the use of qualitative research in the development of core outcome sets. PLoS ONE 2017, 12, e0172937. [Google Scholar] [CrossRef]
  40. Coons, S.J.; Gwaltney, C.J.; Hays, R.D.; Lundy, J.J.; Sloan, J.A.; Revicki, D.A.; Lenderking, W.R.; Cella, D.; Basch, E. Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report. Value Heal. 2009, 12, 419–429. [Google Scholar] [CrossRef] [Green Version]
  41. Slater, A.; Freeman, E. Patients’ views of using an outcome measure in palliative day care: A focus group study. Int. J. Palliat. Nurs. 2004, 10, 343–351. [Google Scholar] [CrossRef] [PubMed]
  42. Gee, K. Why Indeed? Res. Pr. Pers. Sev. Disabil. 2020, 45, 18–22. [Google Scholar] [CrossRef]
  43. Hochstenbach, L.M.; Zwakhalen, S.M.; Courtens, A.M.; van Kleef, M.; de Witte, L.P. Feasibility of a mobile and web-based intervention to support self-management in outpatients with cancer pain. Eur. J. Oncol. Nurs. 2016, 23, 97–105. [Google Scholar] [CrossRef] [Green Version]
  44. Griffiths, D.; Guest, C.; Pickles, E.; Hollen, T.; Grzeda, L.; White, M.; Tollow, P.; Harcourt, D. The development and vali-dation of the CARe Burn Scale—Adult Form: A Patient Reported Outcome Measure (PROM) to assess quality of life for adults living with a burn injury. J. Burn. Care Res. 2017, 44, 3. [Google Scholar] [CrossRef]
  45. Recinos, P.; Dunphy, C.J.; Thompson, N.; Schuschu, J.; Urchek, J.L.; Katzan, I.L. Patient Satisfaction with Collection of Patient-Reported Outcome Measures in Routine Care. Adv. Ther. 2016, 34, 452–465. [Google Scholar] [CrossRef]
  46. Proctor, E.; Silmere, H.; Raghavan, R.; Hovmand, P.; Aarons, G.; Bunger, A.; Griffey, R.; Hensley, M. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm. Ment. Heal. Adm. Policy Ment. Health Ment. Health Serv. Res. 2011, 38, 65–76. [Google Scholar] [CrossRef] [Green Version]
  47. Stover, A.M.; Haverman, L.; Van Oers, H.A.; Greenhalgh, J.; Potter, C.M.; On behalf of the ISOQOL PROMs/PREMs in Clinical Practice Implementation Science Work Group. Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings. Qual. Life Res. 2020, 30, 3015–3033. [Google Scholar] [CrossRef] [PubMed]
  48. Ishaque, S.; Karnon, J.; Chen, G.; Nair, R.; Salter, A.B. A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual. Life Res. 2018, 28, 567–592. [Google Scholar] [CrossRef] [PubMed]
  49. Brownson, R.C.; Kumanyika, S.K.; Kreuter, M.W.; Haire-Joshu, D. Implementation science should give higher priority to health equity. Implement. Sci. 2021, 16, 28. [Google Scholar] [CrossRef] [PubMed]
  50. Prior, S.J.; Mather, C.; Ford, K.; Bywaters, D.; Campbell, S. Person-centred data collection methods to embed the authentic voice of people who experience health challenges. BMJ Open Qual. 2020, 9, e000912. [Google Scholar] [CrossRef]
  51. Mesman, J.; Walsh, K.; Kinsman, L.; Ford, K.; Bywaters, D. Blending Video-Reflexive Ethnography With Solution-Focused Approach: A Strengths-Based Approach to Practice Improvement in Health Care. Int. J. Qual. Methods 2019, 18, 1–10. [Google Scholar] [CrossRef]
  52. Li, J.; Brar, A.; Roihan, N. The use of digital technology to enhance language and literacy skills for Indigenous people: A systematic literature review. Comput. Educ. Open 2021, 2, 100035. [Google Scholar] [CrossRef]
  53. Moullin, J.C.; Dickson, K.S.; Stadnick, N.A.; Albers, B.; Nilsen, P.; Broder-Fingert, S.; Mukasa, B.; Aarons, G.A. Ten recommendations for using implementation frameworks in research and practice. Implement. Sci. Commun. 2020, 1, 1–12. [Google Scholar] [CrossRef]
  54. Shiffman, S.; Stone, A.A.; Hufford, M.R. Ecological momentary assessment. Annu. Rev. Clin. Psychol. 2008, 4, 1–32. [Google Scholar] [CrossRef]
  55. Normalization Process Theory. Available online: http://www.normalizationprocess.org (accessed on 25 March 2022).
  56. Consolidation framework for Implementation Research. Available online: https://cfirguide.org/ (accessed on 25 March 2022).
  57. Harvey, G.; Kitson, A. PARIHS revisited: From heuristic to integrated framework for the successful implementation of knowledge into practice. Implement. Sci. 2015, 11, 33. [Google Scholar] [CrossRef] [Green Version]
  58. Kluger, A.N.; DeNisi, A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 1996, 119, 254–284. [Google Scholar] [CrossRef]
  59. Chapman, E.; Haby, M.M.; Toma, T.S.; De Bortoli, M.C.; Illanes, E.; Oliveros, M.J.; Barreto, J.O.M. Knowledge translation strategies for dissemination with a focus on healthcare recipients: An overview of systematic reviews. Implement. Sci. 2020, 15, 14. [Google Scholar] [CrossRef] [PubMed]
  60. Awell Health. Available online: http://awellhealth.com/ (accessed on 25 March 2022).
  61. The International Classification of Functioning Disability and Health. Available online: www.who.int/classifications/ICF/en (accessed on 25 March 2022).
  62. Meirte, J.; van Loey, N.; Maertens, K.; Moortgat, P.; Hubens, G.; Van Daele, U. Classification of quality of life subscales within the ICF framework in burn research: Identifying overlaps and gaps. Burns 2014, 40, 1353–1359. [Google Scholar] [CrossRef] [PubMed]
  63. Meirte, J. The ICF as a Framework for Post Burn Dysfunctioning: Evaluation, Quality of Life and Vacuum Massage in Patients with Hypertrophic Burn Scars. Ph.D. Thesis, University of Antwerp, Antwerp, Belgium, 2016. [Google Scholar]
  64. Digital Health Europe. Available online: https://digitalhealtheurope.eu/twinnings/dhe-twinning-results/scarpath/ (accessed on 25 March 2022).
  65. Friedstat, J.S.; Ryan, C.M.; Gibran, N. Outcome Metrics After Burn Injury. Clin. Plast. Surg. 2017, 44, 911–915. [Google Scholar] [CrossRef] [PubMed]
  66. Sheridan, S.L.; Donahue, K.E.; Brenner, A.T. Beginning with high value care in mind: A scoping review and toolkit to support the content, delivery, measurement, and sustainment of high value care. Patient Educ. Couns. 2019, 102, 238–252. [Google Scholar] [CrossRef]
  67. Kaspar, A.; Pifeleti, S.; Driscoll, C. The need for translation and cultural adaptation of audiology questionnaires to enable the development of hearing healthcare policies in the Pacific Islands: A Samoan perspective. Arch. Public Heal. 2021, 79, 1–4. [Google Scholar] [CrossRef]
  68. Van Der Wees, P.J.; Verkerk, E.W.; Verbiest, M.E.A.; Zuidgeest, M.; Bakker, C.; Braspenning, J.; De Boer, D.; Terwee, C.B.; Vajda, I.; Beurskens, A.; et al. Development of a framework with tools to support the selection and implementation of patient-reported outcome measures. J. Patient Rep. Outcomes 2019, 3, 1–10. [Google Scholar] [CrossRef] [Green Version]
  69. Young-Afat, D.A. Patient-Reported Outcomes in Routine Care—A True Innovation but Only If Used Correctly. JAMA Oncol. 2019, 5, 1258. [Google Scholar] [CrossRef]
  70. Jones, L.L.; Calvert, M.; Moiemen, N.; Deeks, J.J.; Bishop, J.; Kinghorn, P.; Mathers, J. Outcomes important to burns patients during scar management and how they compare to the concepts captured in burn-specific patient reported outcome measures. Burns 2017, 43, 1682–1692. [Google Scholar] [CrossRef] [Green Version]
  71. Edgar, D.; Van Daele, U.; Spronk, I.; van Baar, M.; van Loey, N.; Wood, F.; Kazis, L.; Meirte, J. Seeding the value based health care and standardised measurement of quality of life after burn debate. Burns 2020, 46, 1721–1723. [Google Scholar] [CrossRef]
  72. Young, A. A Global Core Outcome Set to Optimise the Evidence Base for Burn Care (COSB-i); University of Bristol: Bristol, UK, 2020. [Google Scholar]
  73. Tyack, Z.; Simons, M.; Spinks, A.; Wasiak, J. A systematic review of the quality of burn scar rating scales for clinical and research use. Burns 2011, 38, 6–18. [Google Scholar] [CrossRef]
  74. Tyack, Z.; Wasiak, J.; Spinks, A.; Kimble, R.; Simons, M. A guide to choosing a burn scar rating scale for clinical or research use. Burns 2013, 39, 1341–1350. [Google Scholar] [CrossRef] [PubMed]
  75. Hjermstad, M.J.; Fayers, P.M.; Haugen, D.F.; Caraceni, A.; Hanks, G.W.; Loge, J.H.; Fainsinger, R.; Aass, N.; Kaasa, S. Studies Comparing Numerical Rating Scales, Verbal Rating Scales, and Visual Analogue Scales for Assessment of Pain Intensity in Adults: A Systematic Literature Review. J. Pain Symptom Manag. 2011, 41, 1073–1093. [Google Scholar] [CrossRef]
  76. Hemmingsson, H.; Ólafsdóttir, L.B.; Egilson, S.T. Agreements and disagreements between children and their parents in health-related assessments. Disabil. Rehabil. 2016, 39, 1059–1072. [Google Scholar] [CrossRef]
  77. Ólafsdóttir, L.B.; Egilson, S.T.; Árnadóttir, U.; Hardonk, S.C. Child and parent perspectives of life quality of children with physical impairments compared with non-disabled peers. Scand. J. Occup. Ther. 2018, 26, 496–504. [Google Scholar] [CrossRef] [PubMed]
  78. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research. U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims: Draft guidance. Heal. Qual. Life Outcomes 2006, 4, 1–79. [Google Scholar] [CrossRef] [Green Version]
  79. Arsiwala, T.; Afroz, N.; Kordy, K.; Naujoks, C.; Patalano, F. Measuring What Matters for Children: A Systematic Review of Frequently Used Pediatric Generic PRO Instruments. Ther. Innov. Regul. Sci. 2021, 55, 1082–1095. [Google Scholar] [CrossRef]
  80. Cox, E.D.; Dobrozsi, S.K.; Forrest, C.B.; Gerhardt, W.E.; Kliems, H.; Reeve, B.B.; Rothrock, N.E.; Lai, J.-S.; Svenson, J.M.; Thompson, L.A.; et al. Considerations to Support Use of Patient-Reported Outcomes Measurement Information System Pediatric Measures in Ambulatory Clinics. J. Pediatr. 2020, 230, 198–206.e2. [Google Scholar] [CrossRef]
  81. Santana, M.J.; Haverman, L.; Absolom, K.; Takeuchi, E.; Feeny, D.; Grootenhuis, M.; Velikova, G. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual. Life Res. 2015, 24, 1707–1718. [Google Scholar] [CrossRef]
Ebj 03 00025 g001 550
Figure 1. Implementation strategy guide across all stages of ePROM implementation.
Figure 1. Implementation strategy guide across all stages of ePROM implementation.
Ebj 03 00025 g001
Ebj 03 00025 g002 550
Figure 2. Case study of a digital care pathway in adults with scars in Belgium [38,61,62,63,64].
Figure 2. Case study of a digital care pathway in adults with scars in Belgium [38,61,62,63,64].
Ebj 03 00025 g002
Ebj 03 00025 g003 550
Figure 3. Case study of implementing ePROMs in paediatric scar clinics in Australia (PEDS-ePROM study) [2,10].
Figure 3. Case study of implementing ePROMs in paediatric scar clinics in Australia (PEDS-ePROM study) [2,10].
Ebj 03 00025 g003
Ebj 03 00025 g004 550
Figure 4. Top 10 recommendations for ePROM implementation.
Figure 4. Top 10 recommendations for ePROM implementation.
Ebj 03 00025 g004
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Meirte, J.; Tyack, Z. Electronic Patient-Reported Outcome Measures in Burn Scar Rehabilitation: A Guide to Implementation and Evaluation. Eur. Burn J. 2022, 3, 290-308. https://doi.org/10.3390/ebj3020025

AMA Style

Meirte J, Tyack Z. Electronic Patient-Reported Outcome Measures in Burn Scar Rehabilitation: A Guide to Implementation and Evaluation. European Burn Journal. 2022; 3(2):290-308. https://doi.org/10.3390/ebj3020025

Chicago/Turabian Style

Meirte, Jill, and Zephanie Tyack. 2022. "Electronic Patient-Reported Outcome Measures in Burn Scar Rehabilitation: A Guide to Implementation and Evaluation" European Burn Journal 3, no. 2: 290-308. https://doi.org/10.3390/ebj3020025

APA Style

Meirte, J., & Tyack, Z. (2022). Electronic Patient-Reported Outcome Measures in Burn Scar Rehabilitation: A Guide to Implementation and Evaluation. European Burn Journal, 3(2), 290-308. https://doi.org/10.3390/ebj3020025

Article Metrics

Back to TopTop