Next Article in Journal
Analysis and Evaluation of COVID-19 Web Applications for Health Professionals: Challenges and Opportunities
Previous Article in Journal
The Perspectives of Programme Staff and Recipients on the Acceptability and Benefits of the Ward-Based Outreach Teams in a South African Province
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews

Ivan Herrera-Peco
Azucena Santillán-García
José María Morán
Jessica Marian Goodman-Casanova
4 and
Daniel Cuesta-Lozano
Nursing Department, Faculty of Health Sciences, Alfonso X El Sabio University, Villanueva de la Cañada, 28691 Madrid, Spain
University Hospital of Burgos, 09006 Burgos, Spain
Nursing Department, Nursing and Occupational Therapy College, Universidad de Extremadura, 10003 Cáceres, Spain
Department of Mental Health, Regional University Hospital of Málaga, Biomedical Research Institute of Malaga (IBIMA), 29010 Málaga, Spain
Nurse Faculty of Medicine and Health Sciences, Nursing and Physiotherapy Department, University of Alcalá, 28805 Madrid, Spain
Author to whom correspondence should be addressed.
These authors contribute equally to this work.
Healthcare 2020, 8(4), 465;
Submission received: 11 October 2020 / Revised: 28 October 2020 / Accepted: 4 November 2020 / Published: 6 November 2020
(This article belongs to the Section Nursing)


Today, evidence-based nursing practice strives to improve health care, ensure adherence to treatment, improve health outcomes, and guarantee patient safety. The main scientific documents that nurses should consult, to obtain the best possible evidence, are systematic reviews and meta-analyses. However, this type of scientific document has a major issue if it uses retracted articles that could directly affect the consistency of the results shown in the reviews. The aim of this commentary is to present the current issue represented by the use of retracted articles in meta-analyses of systematic reviews and how researchers could detect them, through the use of different instruments, avoiding them, and providing a reliable SR or meta-analysis that could be useful for day-to-day clinical and research activities.

1. Introduction

The practice of nursing requires a continuous process of decision-making and problem-solving regarding health and care procedures provided to individuals, groups, and communities in healthcare facilities. Responsible decision making and problem solving requires knowing which solution is the most appropriate, most effective, and safest for the patient. Therefore, to ensure the best possible care, the knowledge of nursing staff must always be up to date. Often, we find the argument “it has always been done this way” in some healthcare facilities, assuming that it is not necessary to change procedures that have operated reasonably well over a long time. This statement has greatly impaired the need for updating the effectiveness and safety of care provided in hospitals or other healthcare facilities.
The evidence-based practice (EBP) in general, and evidence-based nursing (EBN) particularly, is a core competency for nurses these days because it guides healthcare decisions that ensure patient safety [1,2]. Understanding the results of recent research is essential for the delivery of the best care, and this is achieved only by the appropriate application of EBN.
The EBP is supported by three main cornerstones: (i) the best available scientific evidence, (ii) awareness of clinical experience (in the organizational context), and (iii) the experience of patients (personal characteristics, health status, needs, and preferences) [3]. Therefore, we can consider that EBO is an essential competence for nurses because it guides health care decisions that guarantee patient safety [1,2].
To implement and conduct EBP, the five-step model defined in the Declaration of Sicily, describing the essential steps for conducting adequate clinical practice based on the best scientific evidence, needs to be considered [4]. This five-step model includes: (i) knowing how to ask a clinical question, (ii) locating and collecting the most relevant scientific evidence, (iii) reading the scientific evidence critically, (iv) integrating the scientific evidence into nursing practice, the patient’s needs and preferences and personal values to make a clinical decision, and (v) evaluating the results [5].
The EBP, as we know, is the integration of the best research trials with clinical experience, but the integration of these research trials into practice takes time and it is necessary to provide an easy way for nurses and other clinical researchers (who are mainly affected by a large workload) to access the trials. Indeed, systematic reviews are the best way to identify, analyze, and summarize the findings of all relevant individual studies on a health or social issue, making the available evidence more accessible to nurses and other health researchers and facilitating the search for the most appropriate solution to a clinical or even social problem [6].
However, developing an SR involves covering the full range of available scientific methods for synthesizing information, and knowing how to apply them carefully and rigorously to produce higher quality reviews that are assessed and used by decision makers related to healthcare procedures.
There is an issue that seriously affects the quality of SR, making them poorly performed, by even conducting a misleading meta-analysis, and therefore seriously affecting the quality of one of the main decision-making tools useful for nurses and other clinical researchers. These concerns refer to the use of retracted articles, articles that have been removed from the scientific literature for a number of reasons, such as (i) compromised peer review process, (ii) duplication of publication, (iii) duplication of images, (iv) lack of ethical approval, (v) plagiarism, (vi) undisclosed conflict of interest, among others [7].
The use of retracted articles that had been previously retracted to develop an SR or meta-analysis, significantly affects the quality and consistency of the results shown in the reviews [8] and, therefore, seriously impacts on their validity as decision-making resources related to clinical or social issues.
Today, thousands of newly published life science studies are being released continuously. PubMed, for example, currently handles more than 30 million records of biomedical literature alone from MEDLINE, life science journals, and online books [9]. Therefore, it is easy to understand that nurses are faced with a large amount of new information, and as a result, reading and evaluating the comprehensive data available in the health science field is becoming almost impossible [10]. In addition to this situation, nurses face another barrier, such as the reproduction of study results in a consistent manner, and even, occasionally, insufficient data results in individual studies compromising confidence in the answers to clinical questions [11].
Therefore, nurses and other health science researchers need tools to access the best scientific evidence in the life science literature (medical, nursing, etc.). Systematic reviews and meta-analyses are the reference standard for synthesizing evidence in healthcare given their methodological rigorousness. Systematic reviews are developed on the basis of a protocol that describes the foundations, hypotheses, and planned methods of the review [12] and meta-analysis represents the use of statistical techniques to compile the results, or effects, of multiple studies, included in a systematic review [13].
Nevertheless, health professionals and health science researchers should be aware of a potential risk when searching for the best possible scientific evidence in the scientific literature: select and even use retracted articles in their systematic reviews and meta-analyses.
This commentary’s major objective is to support researchers by providing some instruments focused on the detection of retracted articles to develop, or even recognize, a reliable SR or meta-analysis that may be useful for day-to-day clinical and research activities.

2. Strategies to Detect Retracted Articles

For authors, there is no clear guidance on how to prevent the selection of a retracted article in their systematic review process, which could later be included in a meta-analysis [14]. Therefore, the most common options used to avoid the inclusion of this type of article were: (i) use alert services that warn researchers when a possible article to be included in the systematic review process has been retracted [15], (ii) consult a database of retracted articles that have been tracked and exposed [15], and finally, (iii) use a reference manager.
Some tools that could be used when researchers decide to examine the references used in the examinations (Table 1) are:
Alert services: nurses and other health researchers can use PubChase, which is linked to RetractionWatch, the largest online database of retracted articles and reasons for retraction.
Database: an example of this option is PubMed, which offers biomedical researchers an important tool for detecting retracted articles in the scientific literature.
In relation to the use of a reference manager, Zotero© is a clear example of a reference manager that warns researchers when a retracted article has been selected. Zotero© is also linked to RetractionWatch [16].
Furthermore, recently a new method to detect retracted papers that combine two possibilities described above, using a database to track retracted articles combined with a reference manager, has been published. This new way is called SCRUTATIOm [17] and it combines the use of a database for academic literature such as SCOPUS© with more than 40,000 journals titles, and the capability of the reference manager Zotero© to show if an article has been retracted.
SCRUTATIOm is a fast, reproducible, and reliable five-step solution for detecting retracted articles, which could be an essential screening tool for nurses who seek to check the consistency of results and conclusions arising from both meta-analyses and systematic reviews. A good and accurate literature review is a key element of systematic reviews, meta-analyses and research studies and always needs to be based on high quality original studies and logically never retracted. SCRUTATIOm is fast, reproducible, and enables the possibility to communicate the potential presence of flaws to the scientific community through a post-publication or post-peer review process.

3. Conclusions

We consider it essential to enhance EBN education, as well as training in the procedures for locating and selecting the most relevant scientific manuscripts. This requires filtering and identifying the articles retracted to avoid compromising the nursing body of knowledge. Use and access to the best possible scientific evidence is the safest way to improve decision-making [18], being updated and prepared to translate this knowledge into clinical nursing practice [19,20].

Author Contributions

Conceptualization, I.H.-P., A.S.-G., J.M.M., J.M.G.-C. and D.C.-L.; writing—original draft preparation, I.H.-P., A.S.-G., J.M.M. and J.M.G.-C.; writing—review and editing, I.H.-P., J.M.M. and D.C.-L.; supervision, I.H.-P. and J.M.M. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Horntvedt, M.T.; Nordsteien, A.; Fermann, T.; Severinsson, E. Strategies for teaching evidence-based practice in nursing education: A thematic literature review. BMC Med. Educ. 2018, 18. [Google Scholar] [CrossRef] [PubMed]
  2. Leufer, T. Teaching core EBP skills to postgraduate nursing students. Worldviews Evid. Based Nurs. 2020, 17, 404–405. [Google Scholar] [CrossRef] [PubMed]
  3. Bianchi, M.; Bagnasco, A.; Bressan, V.; Barisone, M.; Timmins, F.; Rossi, S.; Pellegrini, R.; Aleo, G.; Sasso, L. A review of the role of nurse leadership in promoting and sustaining evidence-based practice. J. Nurs. Manag. 2018, 26, 918–932. [Google Scholar] [CrossRef] [PubMed]
  4. Burns, H.K.; Foley, S.M. Building a foundation for an evidence-based approach to practice: Teaching basic concepts to undergraduate freshman students. J. Prof. Nurs. 2005, 21, 351–357. [Google Scholar] [CrossRef] [PubMed]
  5. Larsen, C.M.; Terkelsen, A.S.; Carlsen, A.F.; Kristensen, H.K. Methods for teaching evidence-based practice: A scoping review. BMC Med. Educ. 2019, 19. [Google Scholar] [CrossRef] [PubMed]
  6. Prasanna, M.D.; Agarwal, V. Systematic reviews: Challenges for their justification, related comprehensive searches, and implications. J. Korean Med. Sci. 2018, 33. [Google Scholar] [CrossRef] [Green Version]
  7. Moyan, E.C.; Kowalczuk, M.K. Why articles are retracted: A retrospective cross-sectional study f retraction notices at BioMed Central. BMJ Open 2016, 6. [Google Scholar] [CrossRef] [Green Version]
  8. Noyes, J. The Value of Synthesizing Evidence to Inform Cancer Nursing. Cancer Nurs. 2020, 43, 173–176. [Google Scholar] [CrossRef] [PubMed]
  9. García-Puente, M.; Pastor-Ramon, E.; Agirre, O.; Morán, J.M.; Herrera-Peco, I. Research note. Open letter to the users of the new PubMed: A critical appraisal. Prof. Inf. 2020, 29, 1699–2407. [Google Scholar] [CrossRef]
  10. Murad, M.H.; Montori, V.M.; Ioannidis, J.P.; Jaeschke, R.; Deveraux, P.J.; Prasad, K.; Neumann, I.; Carasco-Labra, A.; Agoritsas, T.; Hatala, R.; et al. How to read a systematic review and meta-analysis and apply the results to patient care: Users’ guides to the medical literature. JAMA 2014, 312, 171–179. [Google Scholar] [CrossRef] [PubMed]
  11. Lee, Y.H. An overview of meta-analysis for clinicians. Korean J. Intern. Med. 2018, 33, 277–283. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Moher, D.; Shamseer, L.; Clarke, M.; Ghersi, D.; Liberati, A.; Petticrew, M.; Shekelle, P.; Stewart, L.A.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015, 4. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Paul, M.; Leibovici, L. Systematic review or meta-analysis? Their place in the evidence hierarchy. Clin. Microbiol. Infect. 2014, 20, 97–100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Faggion, C.M. More detailed guidance on the inclusion/exclusion of retracted articles in systematic reviews is needed. J. Clin. Epidemiol. 2019, 116, 133–134. [Google Scholar] [CrossRef] [PubMed]
  15. Higgins, J.P.T.; Thomas, J.; Chandler, J.; Cumpston, M.; Li, T.; Page, M.J. Cochrane Handbook for Systematic Reviews of Interventions Version 6.1., 2nd ed.; Welch, V.A., Ed.; John Wiley & Sons Ltd.: Chichester, UK, 2019. [Google Scholar]
  16. Retracted Item Notifications with Retraction Watch Integration. Available online: (accessed on 22 September 2020).
  17. Morán, J.M.; Santillán-García, A.; Herrera-Peco, I. SCRUTATIOm: How to detect retracted literature included in systematics reviews and meta-analysis using SCOPUS© and ZOTERO©. Gac. Sanit. 2020. [Google Scholar] [CrossRef] [PubMed]
  18. Boisvert, I.; Clemesha, J.; Lundmark, E.; Crome, E.; Barr, C.; McMahon, C.M. Decision-Making in Audiology: Balancing Evidence-Based Practice and Patient-Centered Care. Trends Hear. 2017, 21. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Straus, S.E.; Tetroe, J.M.; Graham, I.D. Knowledge translation is the use of knowledge in health care decision making. J. Clin. Epidemiol. 2011, 64, 6–10. [Google Scholar] [CrossRef] [PubMed]
  20. Spence, J.D. The need for clinical judgement in the application of evidence-based medicine. BMJ Evid. Based Med. 2019, 25, 172–177. [Google Scholar] [CrossRef] [PubMed]
Table 1. Characteristics of some elements included in four options described as useful to detect retracted literature.
Table 1. Characteristics of some elements included in four options described as useful to detect retracted literature.
Option to Detect Retracted ArticlesSpecific Tool to UseCharacteristicsComments
Alert ServicesPubChaseA tool to find biomedical literature that show you when an article has been retracted.This tool can warm about retracted articles thanks to their connection with RetractionWatch database.
DatabasesPubMed©As the most widely used search tool for biomedical and life sciences literature, PubMed offer the option to show to researches a warn about if the article showed has been retracted and the reasons why it has occurred.
Reference ManagerZotero©Free reference manager.This tool can warm about retracted articles thanks to their connection with RetractionWatch database.
Mixed MethodSCRUTATIOm5 steps methods that works using A Database like SCOPUS© with a reference manager like Zotero©.The Zotero© capability to detect retracted articles thanks to their connection with RetractionWatch database is essential to this method.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Herrera-Peco, I.; Santillán-García, A.; Morán, J.M.; Goodman-Casanova, J.M.; Cuesta-Lozano, D. The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews. Healthcare 2020, 8, 465.

AMA Style

Herrera-Peco I, Santillán-García A, Morán JM, Goodman-Casanova JM, Cuesta-Lozano D. The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews. Healthcare. 2020; 8(4):465.

Chicago/Turabian Style

Herrera-Peco, Ivan, Azucena Santillán-García, José María Morán, Jessica Marian Goodman-Casanova, and Daniel Cuesta-Lozano. 2020. "The Evidence-Based Practice Silent Enemy: Retracted Articles and Their Use in Systematic Reviews" Healthcare 8, no. 4: 465.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop