Next Article in Journal
COVID-19 Hurricane: Recovering the Worldwide Health System with the RE.RE.RE. (REsponse–REstoration–REengineering) Approach—Who Will Get There First?
Previous Article in Journal
COVID-19 Delta Wave Caused Early Overburden of Hospital Capacity in the Bulgarian Healthcare System in 2021
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 10
Issue 4
10.3390/healthcare10040601
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis

by
Shuo-Chen Chien
1,2,
Ya-Lin Chen
1,2,
Chia-Hui Chien
1,2,3,
Yen-Po Chin
1,2,4,
Chang Ho Yoon
5,6,
Chun-You Chen
1,2,7,8,
Hsuan-Chia Yang
1,2 and
Yu-Chuan (Jack) Li
1,2,9,*
1
Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan
2
International Center for Health Information and Technology, College of Medical science and Technology, Taipei Medical University, Taipei 110, Taiwan
3
Office of Public Affairs, Taipei Medical University, Taipei 110, Taiwan
4
Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA 02115, USA
5
Big Data Institute, University of Oxford, Oxford OX3 7LF, UK
6
Nuffield Department of Population Health, University of Oxford, Oxford OX3 7LF, UK
7
Department of Radiation Oncology, Taipei Municipal Wan Fang Hospital, Taipei 110, Taiwan
8
Information Technology Office in Taipei Municipal Wan Fang Hospital, Taipei Medical University, Taipei 110, Taiwan
9
Department of Dermatology, Taipei Municipal Wan Fang Hospital, Taipei 110, Taiwan
*
Author to whom correspondence should be addressed.
Healthcare 2022, 10(4), 601; https://doi.org/10.3390/healthcare10040601
Submission received: 6 March 2022 / Revised: 16 March 2022 / Accepted: 18 March 2022 / Published: 23 March 2022
Browse Figures
Versions Notes

Abstract

:
A clinical decision support system (CDSS) informs or generates medical recommendations for healthcare practitioners. An alert is the most common way for a CDSS to interact with practitioners. Research about alerts in CDSS has proliferated over the past ten years. The research trend is ongoing with new emerging terms and focus. Bibliometric analysis is ideal for researchers to understand the research trend and future directions. Influential articles, institutes, countries, authors, and commonly used keywords were analyzed to grasp a comprehensive view on our topic, alerts in CDSS. Articles published between 2011 and 2021 were extracted from the Web of Science database. There were 728 articles included for bibliometric analysis, among which 24 papers were selected for content analysis. Our analysis shows that the research direction has shifted from patient safety to system utility, implying the importance of alert usability to be clinically impactful. Finally, we conclude with future research directions such as the optimization of alert mechanisms and comprehensiveness to enhance alert appropriateness and to reduce alert fatigue.

1. Introduction

Making mistakes is human; even the experts are not exempt [1]. In order to prevent mistakes, alerts, defined as notifications or warnings that highlight the risk of danger, have been widely utilized in many fields [2,3,4]. This includes healthcare, where clinical decision support systems (CDSS) commonly use alerts to notify clinicians of actual or potential errors [5,6]. Previous studies have confirmed that alerts are an efficient way to prevent medication errors and streamline clinical workflow [7,8,9].
A CDSS is a computer program based on evidence-based clinical guidelines with or without artificial intelligence (AI), designed to support healthcare providers in identifying problems, resolving them, and reducing errors [10,11,12,13]. Mathematical models of AI are the main component of a CDSS and have been extensively studied [14,15,16]. Furthermore, the AI-based CDSS, used to help against the COVID-19 pandemic, has been widely used in many previous studies [17,18,19]. An example of a CDSS is the computerized physician order entry system (CPOE), which provides physicians with functions such as medication prescription, and laboratory and radiology orders [20,21,22,23,24]. There are different ways for alerts to appear in a CPOE, e.g., pop-up windows, interruptive vs. non-interruptive, active vs. passive alerts [25]. Moreover, alerts are not necessarily clinically relevant; administrative alerts too may frequently feature [26]
Bibliometric analysis is one method of synthesizing a literature review through citation statistics, which could help depict trending concepts in the field of interest and has been used in CDSS applications [27]. For example, Olufisayo Olusegun (2021) systematically reviewed the publications related to the appropriateness of CDSS alerts from 1997 to 2018 [28]. Cemal Aktürk (2021) used the bibliometric method to explore the concept of CDSS conducted between 2016 and 2021 [29]. However, the study of bibliometric analysis to uncover CDSS-alert-related articles is lacking. Therefore, different perspectives provided to this field are needed.
In this study, we aimed to use bibliometric and content analysis methods to explore the trend in using CDSS alerts in the last ten years. Especially, we included the implementation, evaluation, and optimization of CDSS. Along with the knowledge structure derived from bibliometric analysis, we conclude with a broad assessment of current CDSS research as well as suggestions for future research directions.

2. Materials and Methods

Literature review methods include systematic literature review, meta-analysis, bibliometric analysis, and content analysis [30]. We first extracted the bibliographic data published between 2011 and 2021 in the Web of Science (WoS) database. Subsequently, bibliometric and content analysis (both quantitative and qualitative types) were combined to explore our research question.

2.1. Bibliometric Analysis

Bibliometric analysis is a quantitative method using the bibliographic information of publications to analyze their impacts and relationships [27,31,32]. The well-known bibliometric data include authorships, citations, references, and keywords. The following four approaches using different bibliometric data (e.g., keywords, citations, and authorships) were used in this study to capture the conceptual, intellectual, and social networks [27].

2.1.1. Performance Analysis Using Citation Numbers

To determine the contribution of an article, we used the citation number as how many times an article is cited by others to represent its influence on science [27]. There are other ways to express the impact of a scientific article; however, a citation number is an objective and repeatable index that reflects an article’s relationship and the degree of relevance with others [31]. It is hypothesized that researchers refer to an article when it is regarded as relevant and of good quality. Specifically, total global citations (TGC) denotes the number of citations in the entire WoS database, whereas total local citations (TLC) denotes the number of citations among the included articles based on our search criteria.

2.1.2. Citation Analysis

Citation analysis is the most commonly used approach in bibliometrics as it depicts the intellectual structure of the research field [32]. One method in citation analysis is bibliometric coupling. This method relates articles by comparing the citations of two papers and determining the similarity. Precisely, bibliographic coupling requires at least two publications to cite the same article in their references [33,34].

2.1.3. Trending Research Concepts Using Keywords

Core keywords represent the conceptual structure of the specific field [35]. We used “KeyWords Plus” (provided by the WoS databases) to reveal the emerging keywords of our research question [36].

2.1.4. Country Collaboration Map

A social structure of a scientific research field can be well represented by delineating the networks between countries [27]. Country collaboration is established when two countries appear in the same article as the origins of the affiliations of authors. A country collaboration map was therefore generated to visualize the geographical relationships of our research question.

2.2. Content Analysis

Content analysis is a kind of qualitative study that is defined as “a research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes and patterns” [37]. A valid process of theme coding is often required to evaluate the content of included articles [38]. With the completion of coding, researchers summarize and interpret the coding concepts.

2.3. Data Extraction Process

On the basis of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, there were two stages in the data extraction process [39]. First, the bibliographic data were collected from Clarivate Analytics’ Web of Science (WoS), which encompasses 12,000 ISI-indexed journals [40]. The keyword search was conducted in August 2021. We considered a combination of keywords from (1) (“alert*” OR “alarm*” OR “warning*” OR “reminder*” OR “notification”) and (2) (“CPOE” OR “CDS” OR “computerized physician order entry” OR “computerized provider order entry” OR “clinical decision support”). Proceedings papers and early access papers were included in our study. Papers published before 2011 were excluded. Additionally, we excluded studies published as reviews, editorials, letters, books, corrections, and items. Non-English articles were also excluded.
In the second stage, a detailed examination of the articles was applied through reading titles and abstracts. We removed articles with the terms “cds” and “cpoe” related to other meanings; for example, cds was found to be related to Calgary Depression Scale, a clinical scale for patient-reported outcomes. Studies that investigated the alarms of medical devices, such as electroencephalography (EEG), electrocardiography (ECG), infusion pumps, and ventilators, were also excluded. Furthermore, publications that focused on the innovation and design of algorithms that alter the functions of CDSS were not considered as the scope of this study is to evaluate the application of CDSS alerts.
Lastly, we selected a list of crucial articles based on TGC and TLC for content analysis. We included articles with more than or equal to 40 TGC and 10 TLC. The coding process was conducted through the thorough reading of these articles, and thus a concept matrix was developed with a list of primary information, such as study design, alert types, and alert topics [41]. Our coding categories followed the suggestions from Gaur and Kumar (2018) [38].
We used the R package Bibliometrix and Matplotlib package v3.3.4 using Python 3.8.8 for bibliographic analysis and visualization [42,43].

3. Results

3.1. Initial Paper Selection Result

The WoS database returned 1385 records based on the keywords search (Figure 1). In the first stage, we removed papers published before 2011 (N = 227), review articles (N = 132), editorial materials (N = 23), meeting abstracts (N = 23), letters (N = 6), books (N = 4), corrections (N = 2), and news items (N = 1); we additionally removed one paper for being non-English. Subsequently, in the second stage, we screened the titles and abstracts of the remaining 966 articles and excluded 238 papers due to unsuitable content. Finally, a cohort of 728 papers was included for bibliometric analysis. The distribution of yearly publications is presented in Figure S1.

3.2. Most Relevant Journals, Authors, Institutions, and Articles

Performance analysis using citation numbers produced lists of the most relevant journals, articles, institutions, and authors. A total of 728 articles were published by 211 journals. Most of the top-ranked journals belonged to the categories of medical informatics, clinical informatics, and bioinformatics (Table 1). Journal of the American Medical Informatics Association (N = 93, 22.7%), Applied Clinical Informatics (N = 74, 18.1%), and International Journal of Medical Informatics (N = 47, 11.5%) were the top three productive journals, all of which are categorized into the field of medical informatics by the Science Citation Index Expanded (SCIE). PLoS One, a multidisciplinary journal, was also listed in the top 10. Notably, journals categorized in fields other than medical informatics by the SCIE, such as American Journal of Health-System Pharmacy (SCIE category: Pharmacology and Pharmacy) and the Journal of General Internal Medicine (SCIE category: Health Care Sciences & Services), were also listed, implying the emerging importance of informatics application in subfields of medicine.
Bates D.W., Wright A., Seger D.L., and Slight S.P. were the authors with the greatest productivity with high impact over the past ten years (Figure 2). These four researchers focus on the fields of CDSS and patient/medication safety. Notably, all four authors were affiliated with Brigham and Women’s Hospital in the included studies. In terms of the overall author’s impact in the previous ten years, Bates D.W. was the most influential researcher (TGC = 1031), followed by Wright A. (TGC = 583), Seger D.L. (TGC = 542), and Slight S.P. (TGC = 462).
The top three most influential institutions were the University of Washington (N = 86), Brigham and Women’s Hospital (N = 79), and Harvard Medical School (N = 70) (Table 2). Taipei Medical University (N = 47) was the only institution not located in the United States and ranked eighth in the list.
Regarding the impact of individual studies, the top 20 most relevant articles had TLCs greater than 15 and TGCs greater than 31 (Table S1). The most impactful paper in our research is Nanji et al. (2014) [44]. Eight out of the top 20 most relevant articles studied the effect of medication-related alerts in CPOE, particularly drug–drug interaction reminders.

3.3. Bibliographic Coupling

Bibliographic coupling portrays the scientific network between publications. The nodes and edges represent the coupled articles and the associations, respectively (Figure S2). Most articles could be classified into either Class A or Class B, where Class A was related to the implementation of medication-related alerts, and Class B focused on alert optimization to deliver the best practice.

3.4. Trending Research Concepts Using Keywords

In an attempt to grasp the gradual change of research concepts, first, the medium number of occurrences of each keyword (KeyWords Plus) was calculated. These numbers were then compared, and the top 4 frequent words of each year were represented in dots (Figure 3). Overall, the most frequent terms were “clinical decision-support” (N = 152), “systems” (N = 147), “care” (N = 136), and “impact” (N = 130), followed by “physician order entry” (N = 122), “alerts” (N = 98), “errors” (N = 53), and “adverse drug events” (N = 51). The emerging terms over the last three years included “usability” (N = 11) and “overrides” (N = 19). This finding corresponds to our research question in which the trend of CDSS alert research has transferred from system design and the reduction of medical errors with the aid of CDSS to the usability and override issues.

3.5. Country Collaboration Map

Figure 4 shows the country relationship of the included publications. The United States was the most active country in CDSS alert research and demonstrated the strongest association with the United Kingdom (N = 19), followed by South Korea (N = 15) and Canada (N = 12). In addition, unlike countries from other continents, Asian countries tended to work with countries from other continents rather than collaborate with each other.

3.6. Content Analysis

The twenty-four most impactful studies (TGC ≥ 40 and TLC ≥ 10) were extracted from the study cohort of 728 articles (Table S2) to conduct the content analysis. The majority of these studies were observational (83.3%) or used qualitative methods (87.5%) such as focus groups and interviews. Only four studies were interventional, namely, involving the modification of CPOE alert systems. Most studies were conducted in hospitals (70.8%), and only one study was done in community settings. The study populations were most commonly composed of physicians, followed by nurses and other personnel. Regarding the issues of interest in CDSS research, most studies concentrated on medication problems, such as adverse drug events (ADE) or drug–drug interactions (DDI), using alerts of interruptive or hard-stop design. This result specifically answers our research question about the research mainstreams of CDSS alerts. Finally, based on our results, we summarized the research gaps of current studies in the CDSS alert system and provided corresponding suggestions, as in Table 3.

4. Discussion

In this study, we used bibliometric and content analysis methods to explore the concepts of alerts in CDSS. Given the fast-growing body of research regarding alerts in CPOE and CDSS, a bibliometric analysis served as a timely summary of this technology’s recent historical trends and current focuses [29]. Indeed, the importance of a CDSS has led to a growing trend of research in the field of biomedical informatics [45]. In addition, it has recently shown its robust ability to assist people during the coronavirus pandemic period [46,47,48,49].
Keyword analysis helped to provide an insight into the evolution of hot topics over the past ten years in the field of CDSS alerts. In the earlier years, most of this research focused on the design and development of the CDSS (and its alerts), patient safety, and the reduction of adverse drug events [50,51]; recently, the focus has changed to the evaluation of the CDSS alert efficiency and usability [52]. Since the number of alerts used in CDSS has increased, studies regarding alert performance have become more popular. Common metrics to estimate the alert performance and describe the alert fatigue phenomenon include alert override rate and alert dwell time [53,54]. How to reduce the total number of alerts, increase the alert acceptance, and trigger the alert precisely have become hot research topics in this theme.
Content analysis was used to obtain a comprehensive understanding of influential publications in our study cohort, of which 24 items met the criteria (TGC ≥ 40 and TLC ≥ 10). Most healthcare providers used the hard-stop or interruptive alerts as displayed in the process [55,56]. Compared to the soft-stop or passive alerts, they have been proven to be a sufficient way to prevent medical errors [25]. However, the majority of alerting system designs are rule-based/silo nowadays, which means the alerts cannot be triggered depending on the different situations [57]. Thus, alert implementation will not increase the clinical decision value equivalently [58,59]. A context-aware solution based on machine learning should be conducted in future studies [60,61].
We also labeled the study location, population, and alert topics for the content analysis. The result shows that the receivers of CDSS alerts are not restricted to physicians but also comprise other medical personnel from various clinical settings [50,62]. These professional groups execute different clinical processes, influencing their importance judgment on the same alerts. Therefore, it is essential to explore the perspective discrepancy among these characteristics. Most of the studies focused on the clinical alerts (e.g., drug–drug interaction, adverse drug events, and allergy), which reached varying degrees of success in improving patient safety [63,64]. However, both administrative (low or no clinical relevance) and clinical alerts exist in the CDSS. Only focusing on specific ones may limit the improvements to the alert system [26]. We suggest that a comprehensive analysis for all types of CDSS alerts should be considered when designing a similar study in the future.
Here, we used the bibliometric methods to demonstrate the research landscape, including studies about implementation, evaluation, and optimization of CDSS alerts in the past ten years. Therefore, based on the results, we have the following suggestions while re-designing or conducting the study related to CDS alert systems. First, researchers should adopt multiple metrics to comprehensively collect a breadth of perspectives when assessing the effectiveness of alerting systems. Second, CDS system designers may consider implementing the AI-based precision alert system in the next generation of CDSS. Lastly, all types of CDSS alerts should be included in the study to grasp a holistic view of alert use in the settings. By following these guidelines, the clinical workflows may be improved, increasing alert efficiency with conceivable benefits to patient safety [65].

Limitations

Our study has several limitations. First, we only adopted the WoS as the source of bibliographic data. Not including publications in other databases such as Scopus and PubMed may have limited our findings. Thus, a comprehensive analysis will be conducted in our future study. Second, we only analyzed the studies in the last ten years, while the theme has been developed for over 30 years. However, it is more convincing to depict the trend topics based on the latest information. Lastly, we only included English-written publications in our analysis.

5. Conclusions

Our study depicted a comprehensive overview of the field of CDSS alerts. We found over 700 publications in the last ten years. The results demonstrated the trend of CDSS alert research with several aspects, including the contribution of journals, authors, institutions, and countries, keyword analysis, and content analysis. The findings of this study showed that research mainly focused on improving the quality of the CDSS alert system and increasing alert efficiency. We also provided some future directions for the research in this topic, encouraging researchers to design or validate an alert system towards the goal of decreasing alert fatigue and improving patient safety.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare10040601/s1, Figure S1. Distribution of yearly publications and the yearly averages of TGC per article, Figure S2. Bibliographic coupling (A = medication-related cluster and B = best practice cluster), Table S1. Ranking of top 20 articles (sorted by TLC), Table S2. The content analysis for 24 most impactful articles (TGC ≥ 40 and TLC ≥ 10). References [44,50,51,55,56,58,62,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85] are cited in the supplementary materials.

Author Contributions

Conceptualization, H.-C.Y., Y.-C.L.; methodology, S.-C.C., Y.-L.C. and C.-H.C.; formal analysis, S.-C.C. and Y.-L.C.; investigation, S.-C.C. and Y.-L.C.; resources, S.-C.C. and Y.-L.C.; data curation, C.-Y.C., C.H.Y. and Y.-P.C.; writing—original draft preparation, S.-C.C. and Y.-L.C.; writing—review and editing, C.-H.C., C.-Y.C., C.H.Y. and Y.-P.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available in this manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Laski, H.J.J.S. The limitations of the expert. Society 2020, 57, 371–377. [Google Scholar] [CrossRef] [PubMed]
  2. GY, D.D.; Sowmya, S.; Darshini, P.; Nanda, P. ACCIDENT ALERT SYSTEM. Int. J. Adv. Res. Eng. Technol. 2020, 11, 560–567. [Google Scholar]
  3. Jerry, D.J.; Thomas, L.; Panicker, S.T.; Shalu, R.; Mathew, J.T.; VS, B.V.J. Safety alert systems using dedicated short range communication for on road vehicles. In Proceedings of the 2018 International CET Conference on Control, Communication, and Computing (IC4), Thiruvananthapuram, India, 5–7 July 2018. [Google Scholar]
  4. Singh, N.R.; Rothe, P.; Rathkanthiwar, A. Implementation of safety alert system for elderly people using multi-sensors. In Proceedings of the 2017 International Conference of Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India, 20–22 April 2017. [Google Scholar]
  5. Kawamoto, K.; Lobach, D.F. Clinical decision support provided within physician order entry systems: A systematic review of features effective for changing clinician behavior. AMIA Annu. Symp. Proc. 2003, 2003, 361–365. [Google Scholar]
  6. Chazard, E.; Boudry, A.; Beeler, P.E.; Dalleur, O.; Hubert, H.; Tréhou, E.; Beuscart, J.-B.; Bates, D.W. Towards The Automated, Empirical Filtering of Drug-Drug Interaction Alerts in Clinical Decision Support Systems: Historical Cohort Study of Vitamin K Antagonists. JMIR Med. Inform. 2021, 9, e20862. [Google Scholar] [CrossRef] [PubMed]
  7. Kinlay, M.; Ho, L.M.R.; Zheng, W.Y.; Burke, R.; Juraskova, I.; Moles, R.; Baysari, M.J.A.C.I. Electronic Medication Management Systems: Analysis of Enhancements to Reduce Errors and Improve Workflow. Appl. Clin. Inform. 2021, 12, 1049–1060. [Google Scholar] [CrossRef] [PubMed]
  8. Sessions, L.C.; Nemeth, L.S.; Catchpole, K.; Kelechi, T.J. Nurses’ perceptions of high-alert medication administration safety: A qualitative descriptive study. J. Adv. Nurs. 2019, 75, 3654–3667. [Google Scholar] [CrossRef]
  9. Islam, M. An efficient human computer interaction through hand gesture using deep convolutional neural network. SN Comput. Sci. 2020, 1, 211. [Google Scholar] [CrossRef]
  10. Islam, M.M.; Iqbal, H.; Haque, M.R.; Hasan, M.K. Prediction of breast cancer using support vector machine and K-Nearest neighbors. In Proceedings of the 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), Dhaka, Bangladesh, 21–23 December 2017. [Google Scholar]
  11. Ayon, S.I.; Islam, M.M. Diabetes prediction: A deep learning approach. Int. J. Inf. Eng. Electron. Bus. 2019, 10, 21. [Google Scholar]
  12. Akter, L.; Islam, M.; Al-Rakhami, M.S.; Haque, M.J.S.C.S. Prediction of cervical cancer from behavior risk using machine learning techniques. SN Comput. Sci. 2021, 2, 177. [Google Scholar] [CrossRef]
  13. Akter, L.; Islam, M.M. Hepatocellular carcinoma patient’s survival prediction using oversampling and machine learning techniques. In Proceedings of the 2021 2nd International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), Dhaka, Bangladesh, 5–7 January 2021. [Google Scholar]
  14. Nasr, M.; Islam, M.M.; Shehata, S.; Karray, F.; Quintana, Y. Smart Healthcare in the Age of AI: Recent Advances, Challenges, and Future Prospects. IEEE Access 2021, 9, 145248–145270. [Google Scholar] [CrossRef]
  15. Islam, M.M.; Karray, F.; Alhajj, R.; Zeng, J. A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19). IEEE Access 2021, 9, 30551–30572. [Google Scholar] [CrossRef] [PubMed]
  16. Islam, M.M.; Tayan, O.; Islam, M.R.; Islam, M.S.; Nooruddin, S.; Kabir, M.N.; Islam, M.R.J.I.A. Deep learning based systems developed for fall detection: A review. IEEE Access 2020, 8, 166117–166137. [Google Scholar] [CrossRef]
  17. Islam, M.Z.; Islam, M.M.; Asraf, A. A combined deep CNN-LSTM network for the detection of novel coronavirus (COVID-19) using X-ray images. Inform. Med. Unlocked 2020, 20, 100412. [Google Scholar] [CrossRef] [PubMed]
  18. Muhammad, L.; Islam, M.; Usman, S.S.; Ayon, S.I.J.S.C.S. Predictive data mining models for novel coronavirus (COVID-19) infected patients’ recovery. SN Comput. Sci. 2020, 1, 206. [Google Scholar] [CrossRef]
  19. Rahman, M.M.; Manik, M.M.H.; Islam, M.M.; Mahmud, S.; Kim, J.-H. An automated system to limit COVID-19 using facial mask detection in smart city network. In Proceedings of the 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), Vancouver, BC, Canada, 9–12 September 2020. [Google Scholar]
  20. Ayon, S.I.; Islam, M.M.; Hossain, M.R. Coronary artery heart disease prediction: A comparative study of computational intelligence techniques. IETE J. Res. 2020, 1–20. [Google Scholar] [CrossRef]
  21. Haque, M.R.; Islam, M.M.; Iqbal, H.; Reza, M.S.; Hasan, M.K. Performance evaluation of random forests and artificial neural networks for the classification of liver disorder. In Proceedings of the 2018 International Conference on Computer, Communication, Chemical, Material and Electronic Engineering (IC4ME2), Rajshahi, Bangladesh, 8–9 February 2018. [Google Scholar]
  22. Islam, M.; Haque, M.; Iqbal, H.; Hasan, M.; Hasan, M.; Kabir, M.N. Breast cancer prediction: A comparative study using machine learning techniques. SN Comput. Sci. 2020, 1, 290. [Google Scholar] [CrossRef]
  23. Islam, M.R.; Moni, M.A.; Islam, M.M.; Rashed-Al-Mahfuz, M.; Islam, M.S.; Hasan, M.K.; Hossain, M.S.; Ahmad, M.; Uddin, S.; Azad, A. Emotion recognition from EEG signal focusing on deep learning and shallow learning techniques. IEEE Access 2021, 9, 94601–94624. [Google Scholar] [CrossRef]
  24. Islam, M.R.; Islam, M.M.; Rahman, M.M.; Mondal, C.; Singha, S.K.; Ahmad, M.; Awal, A.; Islam, M.S.; Moni, M.A. EEG channel correlation based model for emotion recognition. Comput. Biol. Med. 2021, 136, 104757. [Google Scholar] [CrossRef]
  25. Powers, E.M.; Shiffman, R.N.; Melnick, E.R.; Hickner, A.; Sharifi, M. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: A systematic review. J. Am. Med. Inform. Assoc. 2018, 25, 1556–1566. [Google Scholar] [CrossRef] [Green Version]
  26. Chien, S.-C.; Chin, Y.-P.; Yoon, C.H.; Islam, M.M.; Jian, W.-S.; Hsu, C.-K.; Chen, C.-Y.; Chien, P.-H.; Li, Y.-C. A novel method to retrieve alerts from a homegrown Computerized Physician Order Entry (CPOE) system of an academic medical center: Comprehensive alert characteristic analysis. PLoS ONE 2021, 16, e0246597. [Google Scholar] [CrossRef]
  27. Donthu, N.; Kumar, S.; Mukherjee, D.; Pandey, N.; Lim, W.M. How to conduct a bibliometric analysis: An overview and guidelines. J. Bus. Res. 2021, 133, 285–296. [Google Scholar] [CrossRef]
  28. Olakotan, O.O.; Mohd Yusof, M. The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review. Health Inform. J. 2021, 27, 14604582211007536. [Google Scholar] [CrossRef] [PubMed]
  29. Aktürk, C. Bibliometric Analysis of Clinical Decision Support Systems. Acta Inform. Pragensia 2021, 10, 61–74. [Google Scholar] [CrossRef]
  30. Green, B.F.; Hall, J.A. Quantitative methods for literature reviews. Annu. Rev. Psychol. 1984, 35, 37–54. [Google Scholar] [CrossRef]
  31. Stremersch, S.; Verniers, I.; Verhoef, P. The Quest for Citations: Drivers of Article Impact. J. Mark. 2006, 71, 171–193. [Google Scholar] [CrossRef] [Green Version]
  32. Boyack, K.; Klavans, R. Co-Citation Analysis, Bibliographic Coupling, and Direct Citation: Which Citation Approach Represents the Research Front Most Accurately? J. Am. Soc. Inf. Sci. Technol. 2010, 61, 2389–2404. [Google Scholar] [CrossRef]
  33. Martyn, J. Bibliographic coupling. J. Doc. 1964, 20, 236. [Google Scholar] [CrossRef]
  34. Kessler, M.M.J.A.d. Bibliographic coupling between scientific papers. Am. Doc. 1963, 14, 10–25. [Google Scholar] [CrossRef]
  35. Bornmann, L.; Haunschild, R.; Hug, S.E. Visualizing the context of citations referencing papers published by Eugene Garfield: A new type of keyword co-occurrence analysis. Scientometrics 2018, 114, 427–437. [Google Scholar] [CrossRef] [Green Version]
  36. Zhang, J.; Yu, Q.; Zheng, F.; Long, C.; Lu, Z.; Duan, Z. Comparing keywords plus of WOS and author keywords: A case study of patient adherence research. J. Assoc. Inf. Sci. Technol. 2016, 67, 967–972. [Google Scholar] [CrossRef]
  37. Hsieh, H.-F.; Shannon, S.E. Three approaches to qualitative content analysis. Qual. Health Res. 2005, 15, 1277–1288. [Google Scholar] [CrossRef] [PubMed]
  38. Gaur, A.; Kumar, M. A systematic approach to conducting review studies: An assessment of content analysis in 25 years of IB research. J. World Bus. 2018, 53, 280–289. [Google Scholar] [CrossRef]
  39. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef] [PubMed]
  40. Mongeon, P.; Paul-Hus, A. The journal coverage of Web of Science and Scopus: A comparative analysis. Scientometrics 2016, 106, 213–228. [Google Scholar] [CrossRef]
  41. Alon, I.; Anderson, J.; Munim, Z.H.; Ho, A. A review of the internationalization of Chinese enterprises. Asia Pac. J. Manag. 2018, 35, 573–605. [Google Scholar] [CrossRef]
  42. Aria, M.; Cuccurullo, C. bibliometrix: An R-tool for comprehensive science mapping analysis. J. Informetr. 2017, 11, 959–975. [Google Scholar] [CrossRef]
  43. Hunter, J.D. Matplotlib: A 2D Graphics Environment. Comput. Sci. Eng. 2007, 9, 90–95. [Google Scholar] [CrossRef]
  44. Nanji, K.C.; Slight, S.P.; Seger, D.L.; Cho, I.; Fiskio, J.M.; Redden, L.M.; Volk, L.A.; Bates, D.W. Overrides of medication-related clinical decision support alerts in outpatients. J. Am. Med. Inform. Assoc. 2014, 21, 487–491. [Google Scholar] [CrossRef] [Green Version]
  45. Bohmer, R.M.J. The Four Habits of High-Value Health Care Organizations. N. Engl. J. Med. 2011, 365, 2045–2047. [Google Scholar] [CrossRef]
  46. Saha, P.; Sadi, M.S.; Islam, M.M. EMCNet: Automated COVID-19 diagnosis from X-ray images using convolutional neural network and ensemble of machine learning classifiers. Inform. Med. Unlocked 2021, 22, 100505. [Google Scholar] [CrossRef]
  47. Rahman, M.M.; Islam, M.; Manik, M.; Hossen, M.; Al-Rakhami, M.S. Machine learning approaches for tackling novel coronavirus (COVID-19) pandemic. SN Comput. Sci. 2021, 2, 1–10. [Google Scholar] [CrossRef] [PubMed]
  48. Asraf, A.; Islam, M.; Haque, M. Deep learning applications to combat novel coronavirus (COVID-19) pandemic. SN Comput. Sci. 2020, 1, 1–7. [Google Scholar] [CrossRef] [PubMed]
  49. Al-Rakhami, M.S.; Islam, M.M.; Islam, M.Z.; Asraf, A.; Sodhro, A.H.; Ding, W.J.M. Diagnosis of COVID-19 from X-rays using combined CNN-RNN architecture with transfer learning. MedRxiv 2021. [CrossRef]
  50. Baysari, M.T.; Westbrook, J.I.; Richardson, K.L.; Day, R.O. The influence of computerized decision support on prescribing during ward-rounds: Are the decision-makers targeted? J. Am. Med. Inform. Assoc. 2011, 18, 754–759. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Riedmann, D.; Jung, M.; Hackl, W.O.; Stühlinger, W.; van der Sijs, H.; Ammenwerth, E. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med. Inform. Decis. Mak. 2011, 11, 35. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Olakotan, O.O.; Yusof, M.M. Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow. J. Biomed. Inform. 2020, 106, 103453. [Google Scholar] [CrossRef]
  53. Chien, S.-C.; Chin, Y.-P.; Yoon, C.-H.; Chen, C.-Y.; Hsu, C.-K.; Chien, C.-H.; Li, Y.-C. A Tool to Retrieve Alert Dwell Time from a Homegrown Computerized Physician Order Entry (CPOE) System of an Academic Medical Center: An Exploratory Analysis. Appl. Sci. 2021, 11, 12004. [Google Scholar] [CrossRef]
  54. McDaniel, R.B.; Burlison, J.D.; Baker, D.K.; Hasan, M.; Robertson, J.; Hartford, C.; Howard, S.C.; Sablauer, A.; Hoffman, J.M. Alert dwell time: Introduction of a measure to evaluate interruptive clinical decision support alerts. J. Am. Med. Inform. Assoc. 2016, 23, e138–e141. [Google Scholar] [CrossRef] [Green Version]
  55. Nanji, K.C.; Seger, D.L.; Slight, S.P.; Amato, M.G.; Beeler, P.E.; Her, Q.L.; Dalleur, O.; Eguale, T.; Wong, A.; Silvers, E.R.; et al. Medication-related clinical decision support alert overrides in inpatients. J. Am. Med. Inform. Assoc. 2018, 25, 476–481. [Google Scholar] [CrossRef]
  56. Ancker, J.S.; Edwards, A.; Nosal, S.; Hauser, D.; Mauer, E.; Kaushal, R. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med. Inform. Decis. Mak. 2017, 17, 36. [Google Scholar] [CrossRef] [Green Version]
  57. Légat, L.; Van Laere, S.; Nyssen, M.; Steurbaut, S.; Dupont, A.G.; Cornu, P. Clinical decision support systems for drug allergy checking: Systematic review. J. Med. Internet Res. 2018, 20, e258. [Google Scholar] [CrossRef] [PubMed]
  58. Phansalkar, S.; Van der Sijs, H.; Tucker, A.D.; Desai, A.A.; Bell, D.S.; Teich, J.M.; Middleton, B.; Bates, D.W. Drug—drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J. Am. Med. Inform. Assoc. 2013, 20, 489–493. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  59. Blecker, S.; Austrian, J.S.; Horwitz, L.I.; Kuperman, G.; Shelley, D.; Ferrauiola, M.; Katz, S.D. Interrupting providers with clinical decision support to improve care for heart failure. Int. J. Med. Inform. 2019, 131, 103956. [Google Scholar] [CrossRef] [PubMed]
  60. Khreis, N.; Lau, A.; Al-Jedai, A.; Al-Khani, S.M.; Alruwaili, E.H. An Evaluation of Clinical Decision Support and Use of Machine Learning to Reduce Alert Fatigue. Int. J. Comput. Commun. Eng. 2019, 8, 32–39. [Google Scholar] [CrossRef] [Green Version]
  61. Baron, J.M.; Huang, R.; McEvoy, D.; Dighe, A.S. Use of machine learning to predict clinical decision support compliance, reduce alert burden, and evaluate duplicate laboratory test ordering alerts. JAMIA Open 2021, 4, ooab006. [Google Scholar] [CrossRef]
  62. Payne, T.H.; Hines, L.E.; Chan, R.C.; Hartman, S.; Kapusnik-Uner, J.; Russ, A.L.; Chaffee, B.W.; Hartman, C.; Tamis, V.; Galbreth, B.; et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J. Am. Med. Inform. Assoc. 2015, 22, 1243–1250. [Google Scholar] [CrossRef] [Green Version]
  63. Bright, T.J.; Wong, A.; Dhurjati, R.; Bristow, E.; Bastian, L.; Coeytaux, R.R.; Samsa, G.; Hasselblad, V.; Williams, J.W.; Musty, M.D.; et al. Effect of Clinical Decision-Support Systems. Ann. Intern. Med. 2012, 157, 29–43. [Google Scholar] [CrossRef]
  64. Roumeliotis, N.; Sniderman, J.; Adams-Webber, T.; Addo, N.; Anand, V.; Rochon, P.; Taddio, A.; Parshuram, C. Effect of Electronic Prescribing Strategies on Medication Error and Harm in Hospital: A Systematic Review and Meta-analysis. J. Gen. Intern. Med. 2021, 34, 2210–2223. [Google Scholar] [CrossRef]
  65. Steitz, B.D.; Li, G.; Wright, A.; Dunworth, B.; Freundlich, R.E.; Wanderer, J.P. Non-Interruptive Clinical Decision Support to Improve Perioperative Electronic Positive Patient Identification. J. Med. Syst. 2022, 46, 15. [Google Scholar] [CrossRef]
  66. Seidling, H.M.; Phansalkar, S.; Seger, D.L.; Paterno, M.D.; Shaykevich, S.; Haefeli, W.E.; Bates, D.W. Factors influencing alert acceptance: A novel approach for predicting the success of clinical decision support. J. Am. Med. Inform. Assoc. 2011, 18, 479–484. [Google Scholar] [CrossRef]
  67. McCoy, A.B.; Waitman, L.R.; Lewis, J.B.; Wright, J.A.; Choma, D.P.; Miller, R.A.; Peterson, J.F. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J. Am. Med. Inform. Assoc. 2012, 19, 346–352. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Bryant, A.D.; Fletcher, G.S.; Payne, T.H. Drug interaction alert override rates in the Meaningful Use era. Appl. Clin. Inform. 2014, 5, 802–813. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Seidling, H.M.; Klein, U.; Schaier, M.; Czock, D.; Theile, D.; Pruszydlo, M.G.; Kaltschmidt, J.; Mikus, G.; Haefeli, W.E. What, if all alerts were specific–estimating the potential impact on drug interaction alert burden. Int. J. Med. Inform. 2014, 83, 285–291. [Google Scholar] [CrossRef] [PubMed]
  70. Coleman, J.J.; van der Sijs, H.; Haefeli, W.E.; Slight, S.P.; McDowell, S.E.; Seidling, H.M.; Eiermann, B.; Aarts, J.; Ammenwerth, E.; Ferner, R.E.; et al. On the alert: Future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med. Inform. Decis. Mak. 2013, 13, 1–8. [Google Scholar]
  71. Slight, S.P.; Seger, D.L.; Nanji, K.C.; Cho, I.; Maniam, N.; Dykes, P.C.; Bates, D.W. Are we heeding the warning signs? Examining providers’ overrides of computerized drug-drug interaction alerts in primary care. PLoS ONE 2013, 8, e85071. [Google Scholar] [CrossRef] [Green Version]
  72. Saverno, K.R.; Hines, L.E.; Warholak, T.L.; Grizzle, A.J.; Babits, L.; Clark, C.; Taylor, A.M.; Malone, D.C. Ability of pharmacy clinical decision-support software to alert users about clinically important drug—drug interactions. J. Am. Med. Inform. Assoc. 2011, 18, 32–37. [Google Scholar] [CrossRef] [Green Version]
  73. Carspecken, C.W.; Sharek, P.J.; Longhurst, C.; Pageler, N.M. A clinical case of electronic health record drug alert fatigue: Consequences for patient outcome. Pediatrics 2013, 131, e1970–e1973. [Google Scholar] [CrossRef] [Green Version]
  74. Bell, G.C.; Crews, K.R.; Wilkinson, M.R.; Haidar, C.E.; Hicks, J.K.; Baker, D.K.; Kornegay, N.M.; Yang, W.; Cross, S.J.; Howard, S.C.; et al. Development and use of active clinical decision support for preemptive pharmacogenomics. J. Am. Med. Inform. Assoc. 2014, 21, e93–e99. [Google Scholar] [CrossRef]
  75. Wright, A.; Hickman, T.T.T.; McEvoy, D.; Aaron, S.; Ai, A.; Andersen, J.M.; Hussain, S.; Ramoni, R.; Fiskio, J.; Sittig, D.F.; et al. Analysis of clinical decision support system malfunctions: A case series and survey. J. Am. Med. Inform. Assoc. 2016, 23, 1068–1076. [Google Scholar] [CrossRef] [Green Version]
  76. Russ, A.L.; Zillich, A.J.; McManus, M.S.; Doebbeling, B.N.; Saleem, J.J. Prescribers’ interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human–computer interaction. Int. J. Med. Inform. 2012, 81, 232–243. [Google Scholar] [CrossRef]
  77. Eppenga, W.L.; Derijks, H.J.; Conemans, J.M.; Hermens, W.A.; Wensing, M.; De Smet, P.A. Comparison of a basic and an advanced pharmacotherapy-related clinical decision support system in a hospital care setting in the Netherlands. J. Am. Med. Inform. Assoc. 2012, 19, 66–71. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  78. Scott, G.P.; Shah, P.; Wyatt, J.C.; Makubate, B.; Cross, F.W. Making electronic prescribing alerts more effective: Scenario-based experimental study in junior doctors. J. Am. Med. Inform. Assoc. 2011, 18, 789–798. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  79. Duke, J.D.; Li, X.; Dexter, P. Adherence to drug—drug interaction alerts in high-risk patients: A trial of context-enhanced alerting. J. Am. Med. Inform. Assoc. 2013, 20, 494–498. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  80. Scheife, R.T.; Hines, L.E.; Boyce, R.D.; Chung, S.P.; Momper, J.D.; Sommer, C.D.; Abernethy, D.R.; Horn, J.R.; Sklar, S.J.; Wong, S.K.; et al. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf. 2015, 38, 197–206. [Google Scholar] [CrossRef] [PubMed]
  81. Russ, A.L.; Zillich, A.J.; Melton, B.L.; Russell, S.A.; Chen, S.; Spina, J.R.; Weiner, M.; Johnson, E.G.; Daggy, J.K.; McManus, M.S.; et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J. Am. Med. Inform. Assoc. 2014, 21, e287–e296. [Google Scholar] [CrossRef] [Green Version]
  82. Topaz, M.; Seger, D.L.; Slight, S.P.; Goss, F.; Lai, K.; Wickner, P.G.; Blumenthal, K.; Dhopeshwarkar, N.; Chang, F.; Bates, D.W.; et al. Rising drug allergy alert overrides in electronic health records: An observational retrospective study of a decade of experience. J. Am. Med. Inform. Assoc. 2016, 23, 601–608. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  83. Romano, M.J.; Stafford, R.S. Electronic health records and clinical decision support systems: Impact on national ambulatory care quality. Arch. Intern. Med. 2011, 171, 897–903. [Google Scholar] [CrossRef]
  84. Wright, A.; Sittig, D.F.; Ash, J.S.; Bates, D.W.; Feblowitz, J.; Fraser, G.; Maviglia, S.M.; McMullen, C.; Nichol, W.P.; Pang, J.E.; et al. Governance for clinical decision support: Case studies and recommended practices from leading institutions. J. Am. Med. Inform. Assoc. 2011, 18, 187–194. [Google Scholar] [CrossRef] [Green Version]
  85. Hoffman, J.M.; Haidar, C.E.; Wilkinson, M.R.; Crews, K.R.; Baker, D.K.; Kornegay, N.M.; Yang, W.; Pui, C.H.; Reiss, U.M.; Gaur, A.H.; et al. PG4KDS: A model for the clinical implementation of pre-emptive pharmacogenetics. Am. J. Med. Genetics. Part C Semin. Med. Genet. 2014, 166c, 45–55. [Google Scholar] [CrossRef] [Green Version]
Healthcare 10 00601 g001 550
Figure 1. Paper selection process.
Figure 1. Paper selection process.
Healthcare 10 00601 g001
Healthcare 10 00601 g002 550
Figure 2. Top 20 authors’ publication productivity time.
Figure 2. Top 20 authors’ publication productivity time.
Healthcare 10 00601 g002
Healthcare 10 00601 g003 550
Figure 3. Core keywords analysis for trending topics.
Figure 3. Core keywords analysis for trending topics.
Healthcare 10 00601 g003
Healthcare 10 00601 g004 550
Figure 4. The country collaboration map for CDSS alert studies.
Figure 4. The country collaboration map for CDSS alert studies.
Healthcare 10 00601 g004
Table
Table 1. Most influential journals (sorted by the number of publications).
Table 1. Most influential journals (sorted by the number of publications).
#JournalsItemTGCTGC per ItemIF (2020)
N%
1Journal of the American Medical Informatics Association9322.7279830.14.50
2Applied Clinical Informatics7418.12323.12.34
3International Journal of Medical Informatics4711.555811.94.05
4BMC Medical Informatics and Decision Making348.32587.62.80
5American Journal of Health-system Pharmacy163.928918.12.64
6JMIR Medical Informatics163.9181.12.96
7PLoS ONE153.715710.53.24
8International Journal of Clinical Pharmacy143.4574.12.05
9Journal of Clinical Pharmacy and Therapeutics112.7898.12.51
10Drug Safety92.220122.35.61
11BMJ Quality & Safety92.2889.87.04
12Artificial Intelligence in Medicine92.2637.05.33
13CIN-COMPUTERS INFORMATICS NURSING92.2323.61.99
14Journal of General Internal Medicine82.031138.95.13
15Journal of Biomedical Informatics82.026533.16.32
16Pharmacoepidemiology and Drug Safety82.013516.92.89
17Journal of Medical Systems82.0405.04.46
18Medical Care71.714120.12.98
19American Journal of Medical Quality71.7365.11.85
20American Journal of Clinical Pathology71.7344.92.49
Abbreviations: N = Number of publications, TGC = Total global citations, IF = Impact factor.
Table
Table 2. Most influential institutions (sorted by the number of publications).
Table 2. Most influential institutions (sorted by the number of publications).
#InstitutionsNLocation
1University of Washington86Seattle, WA, USA
2Brigham and Women’s Hospital79Boston, MA, USA
3Harvard Medical School70Boston, MA, USA
4University of Pittsburgh68Pittsburgh, PA, USA
5Harvard University66Boston, MA, USA
6Vanderbilt University65Nashville, TN, USA
7Stanford University49Stanford, CA, USA
8Taipei Medical University47Taipei, TW
9Mayo Clinic43Scottsdale, AZ, USA
10University of Pennsylvania43Philadelphia, PA, USA
11Columbia University36New York, NY, USA
12Partners HealthCare International33Boston, MA, USA
13Indiana University School of Medicine32Indianapolis, IN, USA
14Cincinnati Children’s Hospital Medical Center31Cincinnati, OH, USA
15University of Michigan31Ann Arbor, MI, USA
16Case Western Reserve University28Cleveland, OH, USA
17University of California, Los Angeles28Los Angeles, CA, USA
18Icahn School of Medicine at Mount Sinai27New York, NY, USA
19Indiana University School of Medicine26Indianapolis, IN, USA
20Oregon Health & Science University26Portland, OR, USA
Abbreviations: N = Number of publications.
Table
Table 3. The summary of current research gaps and suggestions.
Table 3. The summary of current research gaps and suggestions.
#Current Research GapSuggestion
1Usually used only a single metric to evaluate the alert system’s efficiency.Adopting multiple metrics to comprehensively collect perspectives.
2Most of the studies focused on specific types of CDSS alerts.Consider including all types of CDSS alerts to grasp a holistic view of alert usage.
3The majority of alerting system designs are rule-based/silo.An AI-based precision alert system should be considered to implement in the next generation of CDSS.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chien, S.-C.; Chen, Y.-L.; Chien, C.-H.; Chin, Y.-P.; Yoon, C.H.; Chen, C.-Y.; Yang, H.-C.; Li, Y.-C. Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis. Healthcare 2022, 10, 601. https://doi.org/10.3390/healthcare10040601

AMA Style

Chien S-C, Chen Y-L, Chien C-H, Chin Y-P, Yoon CH, Chen C-Y, Yang H-C, Li Y-C. Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis. Healthcare. 2022; 10(4):601. https://doi.org/10.3390/healthcare10040601

Chicago/Turabian Style

Chien, Shuo-Chen, Ya-Lin Chen, Chia-Hui Chien, Yen-Po Chin, Chang Ho Yoon, Chun-You Chen, Hsuan-Chia Yang, and Yu-Chuan (Jack) Li. 2022. "Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis" Healthcare 10, no. 4: 601. https://doi.org/10.3390/healthcare10040601

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop