Toward successful migration to computerized physician order entry for chemotherapy

Computerized physician order entry (cpoe) has been shown to significantly improve chemotherapy safety by reducing the number of prescribing errors1–7. Since the early 2000s, research has established a knowledge base for successful cpoe implementation and ongoing use. Some of the critical success factors include the design and usability of the cpoe system8–12, standardization of the ordering process13,14, seamless integration with other health information systems and user workflows12,15–17, effective training and support10,14,16,18, support from leadership10,14,16, collaborative project management14,16, and effective ongoing maintenance and support16,19. ABSTRACT


Background
Computerized physician order entry (cpoe) systems allow for medical order management in a clinical setting.Use of a cpoe has been shown to significantly improve chemotherapy safety by reducing the number of prescribing errors.Usability of these systems has been identified as a critical factor in their successful adoption.However, there is a paucity of literature investigating the usability of cpoe for chemotherapy and describing the experiences of cancer care providers in implementing and using a cpoe system.

Methods
A mixed-methods study, including a national survey and a workshop, was conducted to determine the current status of cpoe adoption in Canadian oncology institutions, to identify and prioritize knowledge gaps in cpoe usability and adoption, and to establish a research agenda to bridge those gaps.Survey respondents were representatives of cancer care providers from each Canadian province.The workshop participants were oncology clinicians, human factors engineers, patient safety researchers, policymakers, and hospital administrators from across Canada, with participation from the United States.

Results
A variety of issues related to implementing and using a cpoe for chemotherapy were identified.The major issues concerned the need for better understanding of current practices of chemotherapy ordering, preparation, and administration; a lack of system selection and procurement guidance; a lack of implementation and maintenance guidance; poor cpoe usability and workflow support; and other cpoe system design issues.An additional three research themes for addressing the existing challenges and Although the usability of cpoe systems has been identified as a critical success factor, the usability of existing cpoe systems seems to be far from optimal.Usability can be defined as the extent to which a product can be used by specific users to achieve specific goals, with effectiveness, efficiency, and satisfaction in a specific context of use 20 .Some of the commonly reported cpoe usability issues include an excessively complex and unintuitive user interface [21][22][23][24][25] , a cluttered or poorly organized display, lack of safeguards 21,24 , and inflexible or inefficient user interaction 21,[23][24][25] .
Furthermore, there is a paucity of literature investigating the usability of cpoe systems for chemotherapy, despite studies that illustrate its importance.Corrao et al. 26 found that, for most users, a cpoe system implemented without an initial usability evaluation raised issues of efficiency and satisfaction that could have been identified and resolved through usability testing before implementation.Khajouei et al. 27 compared the ordering efficiency of oncologists using a cpoe system in a laboratory setting with and without the use of standard order sets.Those authors found that, although ordering efficiency can be improved by integrating standard order sets into a cpoe system, efficiency can be significantly impaired by usability problems.
Knowledge about the experiences of cancer care providers in implementing and using a cpoe system for chemotherapy is relatively limited.A handful of studies describe the methods used for implementing a cpoe system for chemotherapy 6,14,19,28 .Harshberger et al. 29 reported improved user satisfaction and better completeness of chemotherapy orders and documentation after a cpoe system was implemented to replace paper-based charts at a large multi-site teaching hospital, and Brockstein et al. 30 described the impact of the cpoe implementation at the same institution on documentation, communication, operations, quality improvement, and research.At a large Dutch teaching hospital, Pirnejad et al. 31 found that a user requirement-driven and process-oriented cpoe system development process and proximity of the development site to the implementation site resulted in a preference by the chemotherapy and hematology clinicians for a home-grown cpoe system over a commercially available hospital-wide cpoe system.
To address the foregoing knowledge deficits, we conducted a mixed-methods qualitative study involving a workshop and a survey.The survey aimed to understand the adoption and use of cpoe by Canadian cancer care providers.The findings provided the context for discussions during the workshop.The workshop aimed to identify and prioritize knowledge gaps in chemotherapy cpoe adoption and usability literature and to establish a research agenda for bridging the knowledge gaps so as to improve clinical practice.

Workshop Design
A purposive sample of 30 experts and representatives of Canadian cancer care providers was established using the literature, online searches, and provincial liaisons from the Systemic Therapy Safety Committee of the Canadian Association of Provincial Cancer Agencies.Participants included medical oncologists, oncology pharmacists, oncology nurses, informaticians, human factors engineers, patient safety researchers, policymakers, and hospital administrators from across Canada, with participation from the United States.
The workshop took place November 28, 2012, at Toronto General Hospital, Toronto, Ontario.The first half of the workshop was designed to provide participants with context for the topic of concern.Two invited experts presented their knowledge about the use of cpoe in medical oncology and their recent research activities in the field.Their talks were followed by a presentation about the survey of Canadian cancer care providers.
The second half of the workshop consisted of two breakout sessions designed to identify the knowledge and practice gaps related to cpoe adoption and usability.During the first breakout session, participants whose organizations were currently using a cpoe system for ordering chemotherapy ("current cpoe users") were asked to identify the challenges they experienced associated with adopting and using a cpoe system.Participants who were not using a cpoe system ("future cpoe users") were asked to discuss the challenges associated with planning for cpoe system implementation, because many of the future cpoe users came from organizations that had started planning for implementation of a cpoe system.The researcher group was asked to discuss the knowledge gaps in chemotherapy cpoe usability.
Immediately after the first breakout session, two study investigators reviewed the verbatim transcripts of the issues presented to identify emergent categories.An open coding exercise was conducted for each of the issues, and then higher-level themes were derived based on the relationships between the categories.Themes were member-checked by presenting them to the workshop participants and revising them based on the resulting feedback.
The participants were asked to further refine and prioritize the themes to derive priority research questions.After the group discussions, a representative from each group reported the findings to the rest of the participants.Based on the research topics presented, the study investigators recorded the key research themes.

Survey Design
The survey consisted of 33 questions for current cpoe users and 7 questions for future cpoe users.survey for current cpoe users aimed to understand the status of cpoe usage, the type of cpoe system used, the procurement and implementation process used, the challenges experienced during cpoe implementation and use, and plans related to cpoe.The survey for future cpoe users focused on understanding the methods currently used by their organizations for communicating chemotherapy orders and on eliciting their plans related to cpoe.The questions were developed based on a review of the literature on cpoe usability and adoption and on consultations with a medical oncologist, human factors engineers, and a patient safety researcher.The final survey was approved by the research ethics board at the University Health Network (uhn reb 12-0488-AE).The survey and literature review served to inform the activities and discussions of the workshop.
The survey was distributed by provincial liaisons from the Canadian Association of Provincial Cancer Agencies Systemic Therapy Safety Committee.An online survey tool was used to administer the survey (http://www.surveymonkey.com:Survey Monkey, Palo Alto, CA, U.S.A.).Because the degree of oversight and regulation of each cancer agency varies from province to province in Canada, the liaisons were encouraged to forward the survey request to appropriate personnel at individual cancer care facilities within the organization, while ensuring that the survey would be completed by a single individual in each facility.Responses to the survey were collected from October 11, 2012, to November 9, 2012.

Workshop Findings
The first breakout session and the discussions that followed provided insights into the challenges and unmet needs that current cpoe users experienced in implementing and using their cpoe system and that future cpoe users were experiencing in planning for their cpoe adoption (summarized in Table i).The major themes were: • Need for a better understanding of current practices • System selection and procurement issues • Implementation and maintenance issues • System usability issues • Other system design-related issues The second breakout session led to the establishment of 3 key research themes (Table ii) that address the knowledge gaps and sources of the existing challenges.

Survey Findings
Twenty-four organizations from ten provinces completed the survey, including 7 representatives of provincial cancer agencies and 17 representatives of individual cancer care facilities (including community clinics, community hospitals, and academic hospitals).Thirteen organizations from six provinces were current cpoe users, and eleven organizations from six provinces were future cpoe users.
The current cpoe users reported having experienced various implementation challenges, including these: • Users learning to use the system (11 organizations) • Integration of the cpoe system with typical user workflows (9 organizations) • Persistence of paper-based tools (9 organizations) • Physician resistance to adoption of the cpoe (6 organizations) More specifically, current cpoe users reported that they had experienced issues related to cpoe system usability (Table iii) and workflow integration.Furthermore, 11 current cpoe users reported that their organization had experienced a patient safety incident related to cpoe use.Table iv presents the factors that were reported to have contributed to those incidents.

DISCUSSION
Our workshop and survey results show that the Canadian cancer care providers that have adopted a cpoe system for chemotherapy have experienced a variety of issues ranging from product selection to maintenance.In particular, poor cpoe system usability seems to be a major problem experienced by current cpoe users.The survey responses indicated that a patient safety incident related to cpoe use occurred in most organizations.Considering that some of our respondents might have been hospital administrators, who are aware mainly of major incidents, the results probably underrepresent the frequency of the problems that occurred.The workshop participants also reported various usability issues, including excessively complex and unintuitive user interfaces, lack of safeguards to minimize the potential for and impact of errors, lack of appropriate feedback, cluttered and unorganized information displays, and lack of support for interprofessional communication and task coordination.
Despite the rather broad range of usability issues experienced by the current cpoe users, little guidance is available to ensure that cancer care providers can select a cpoe system with good usability.The American Society of Clinical Oncology has published a number of articles to help cancer care providers select and implement an electronic health record (ehr) system 13,15,[32][33][34]  Commission for Healthcare Information Technology as its oncology ehr certification criteria 35 .Although the foregoing documents are necessary and useful, they are not focused on the usability of cpoe systems.
With regard to generic ehr systems, a few guidelines for evaluating their usability have been developed by notable medical technology organizations such as the Healthcare Information and Management Systems Society and the U.S. National Institute of Standards and Technology [36][37][38][39][40] .However, the foregoing documents do not address the unique needs of health care providers in a specific setting.

table i
Themes of challenges and unmet needs related to adoption of computerized physician order entry (cpoe), as reported by workshop participants

Need for better understanding of current practices and standardization
Large variations seem to exist in workflows and practices around chemotherapy ordering and administration.These variations are not currently well understood.Successful adoption of cpoe requires sources of workflow variations and eliminating those variations and practices that could be standardized or that are unnecessary.Overall, there needs to be guidance for better standardization of practices in systemic therapy (for example, standardizing body).

System selection and procurement issues
End users of chemotherapy cpoe systems are often not involved in the cpoe system product selection process.As a result, end users often have to adapt to a system, chosen by management, that does not meet their patient care and workflow needs.There needs to be a strategy (for example, organizational governance structure) ensuring that all levels of stakeholders, including end users, are involved in the system selection and procurement process.
It is often challenging to perform a comprehensive evaluation of a cpoe product to ensure that it meets all functional criteria in a meaningful manner.

Implementation and maintenance issues
There is a lack of guidance, specific to chemotherapy providers, for cpoe implementation planning and ongoing usage.Such guidance should include recommendations for necessary resources, workflow modifications, training needs, practice changes, and so on.This could ensure that providers reap the maximum benefits.
Transitioning from a paper-based system to a computerized system requires buy-in from end users, which might take time.A hybrid of paper-based tools and a cpoe system could be in use for a certain period of time.Risks associated with the utilization of a hybrid system need to be understood and managed.
cpoe system usability issues Existing cpoe systems tend to require too many steps to accomplish clinical tasks and are too complex to use.Such systems make it challenging to train health care professionals who are not necessarily inclined to technology; training relies on memory, which is one of the least effective risk mitigations.
Existing cpoe systems do not always provide appropriate feedback about the system state to users (for example, no feedback about the status of an order submitted).
Existing systems tend to overload screens with information, such that key information is buried in large amounts of irrelevant information.
The iterative and dynamic nature of chemotherapy ordering is not well supported by existing cpoe systems.cpoe systems should be equipped with more safeguards to minimize the chance that errors will occur and to prevent errors from propagating downstream all the way to the patient.cpoe systems should provide better support for interprofessional communication and task coordination.
Other issues related to cpoe system design Existing cpoe systems do not interface with many other health information technology systems in place (for example, pharmacy information system, patient scheduling system, radiation oncology systems).This lack of interoperability results in the use of error-prone workarounds and processes (for example, manual transcription).
New needs concernign oral chemotherapies are not well understood, nor appropriately supported by existing cpoe systems.

Current OnCOlOgy-VOlume 21, number 2, April 2014
Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).Some efforts have been made to establish ehr usability guidance that is more specific to primary care 41,42 and to pediatric patient care 43 , but such guidance does not seem to exist for chemotherapy.To the best of our knowledge, the only guidance document in medical oncology that discusses at least some aspects of cpoe system usability is the best practice guideline for systemic-therapy cpoe recently published by Cancer Care Ontario 44 .
Closely related to the usability of cpoe systems are the existing workflows for ordering, preparing, and administering chemotherapy.An important aspect of system usability is whether the workflows modelled by the cpoe system match user practice needs 40 .The workshop participants acknowledged that there are wide variations in workflows for chemotherapy, making integration of a cpoe system into user workflows difficult.The negative effects of unnecessary variances in workflows are only exacerbated when a cpoe system is implemented.Hence, the American Society of Clinical Oncology EHR Workgroup emphasizes the need for standardization of workflows and chemotherapy regimens before implementation of an ehr 13,15 .Nevertheless, insufficient understanding has been developed about the workflow variations that currently exist in chemotherapy ordering, preparation, and administration processes; about the reasons for those variations; and about which variations are required and should be supported by a cpoe system, and which are unnecessary and pose risks to patient safety.To help cancer care providers streamline their workflows, the American Society of Clinical Oncology EHR Workgroup suggests some best principles to follow 13,15 .However, cancer care providers are demanding more detailed and evidence-based guidance that they can readily apply.
Cancer care providers are also seeking step-bystep guidance (a roadmap) that would lead them to successful adoption of a cpoe system and allow them to reap its maximum benefits.A number of tools are available for implementing a general ehr, including guidelines from the U.S. Department of Health and Human Services, the American College of Physicians, and the American Medical Association [45][46][47] .However, compared with other domains, chemotherapy is unique in that chemotherapy treatments involve complex multidrug regimens that often encompass drugs with narrow therapeutic indices, complex dose calculations and adjustments, and unique documentation requirements 14 .Guidelines specifically designed for cancer care providers to address those unique challenges are therefore necessary.

CONCLUSIONS
Computerized physician order entry technology offers the potential to significantly improve patient safety and the efficiency and quality of chemotherapy care.However, cpoe usability factors have contributed to patient safety incidents in cancer care organization across Canada.Those incidents involved

Index Research theme description
Theme A There is a need to understand variances in workflows and practices involved in chemotherapy ordering and administration, including the extent and drivers of the variances.
Theme B There is a need for a computerized physician order entry (cpoe) procurement and implementation tool specifically for chemotherapy.
Theme C There is a need for better measures of the effects of cpoe implementation in medical oncology (for example, provider efficiency) to establish evidence for advancing adoption of cpoe by cancer care providers.wrong dose selections, drug deliveries at the wrong time, cpoe order communication breakdowns, and other issues.Many of the difficulties experienced by Canadian cancer care providers were also related to learning how to use the cpoe system, integrating the system with typical user workflows, and dealing with consequent user resistance to adoption and with persistence of paper-based tools.To start addressing usability problems with cpoe systems, the oncology community should take the actions necessary to fill the knowledge gaps concerning the prevalence of workflow variations in chemotherapy practice across organizations.Developing that knowledge will help to inform best practices and improve the fit between clinical practice and cpoe usability design.
The improved synergy between practice and cpoe design will facilitate successful adoption of cpoe systems, inform comprehensive procurement and implementation guidance, and lead to the realization of the intended patient safety benefits.

The Current OnCOlOgy-VOlume 21, number 2, April 2014
Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

Certification e224 Current OnCOlOgy-VOlume 21, number 2, April 2014
. In particular, Clinical Oncology Requirements for the EHR sets out the functional requirements, clinical data elements, and interoperability requirements that should be considered.Those requirements were adapted by the U.S.
Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).

table ii
Key research themes

table iii
Types of usability issues experienced by current users of computerized physician order entry (cpoe) systems table iv Factors contributing to patient safety incidents reported by current users of computerized physician order entry (cpoe) systems Wrong dose selection error in cpoe 7 Drug delivery scheduled for the wrong time (that is, date) in cpoe 7 Change to an order in cpoe did not get communicated 5 Failure to process an order 5 User ignoring an important alert or warning in cpoe 5 Duplicate medical order error in cpoe 4 Manual data entry error 4 Wrong drug selection error in cpoe 3 Wrong route of administration selection error in cpoe 2 Order made to a wrong patient in cpoe 2 Other (wrong regimen selected, wrong body weight) 2 Current OnCOlOgy-VOlume 21, number 2, April 2014 Copyright © 2014 Multimed Inc.Following publication in Current Oncology, the full text of each article is available immediately and archived in PubMed Central (PMC).