On the Dearth of Retractions in Social Work: A Cross-Sectional Study of Ten Leading Journals

Abstract

In recent decades, there has been an increase in the number of retractions across the biomedical and social sciences. A high rate of retractions undermines the integrity of scholarly journals and threatens the credibility of scientific disciplines. It is unknown how common retractions are within the field of social work. The aim of this study was to determine the prevalence of retractions among ten leading social work journals. This cross-sectional study employed three search strategies. First, each journal’s website was searched using the keywords “retracted” and “retraction”. The same procedure was employed, for each journal, using Google Scholar’s advanced search function. Finally, the Retraction Watch Database was queried using the name of each journal. None of the 196 results produced from these search strategies resulted in the identification of a single retracted article. Reasons for this absence are explored and recommendations to enhance the integrity of social work research and journals are discussed.

1. Introduction

Scholarly journals play a key role in the development and dissemination of scholarly knowledge [1]. They are part of an iterative process: Journals assess the merits of novel research and scholarship—ostensibly selecting for work deemed most rigorous and/or impactful—and critique and purportedly improve it between submission and acceptance. These published works help to inform, justify, and contextualize future scholarship. This is evident, for example, by the way the literature reviews inform and support the background sections within manuscripts and grant proposals, or by the synthesis of results from a body of research via systematic reviews, meta-analyses [2,3] and, at an even higher order, umbrella reviews (i.e., the collection, assessment, and synthesis of multiple systematic reviews and meta-analyses) [4]. Newly published scholarship advances, supports, and sometimes challenges the prevailing set(s) of policies, interventions, and frameworks employed across the biomedical and social sciences.

Traditionally, editors and peer reviewers support journals by serving as a form of quality control or “gatekeeping” [5,6]. While recent developments in scholarly publishing have enabled the partial or even complete decoupling of peer review from this gatekeeping role (sometimes referred to as “journal-independent” peer review) [7,8,9], journals still largely adhere to a traditional model of single- or double-blind, pre-publication peer review. This is especially true for the discipline under study in the current paper—social work [10,11]. Although the published literature is in some ways validated by this model, the underlying processes are largely unstandardized and opaque, and its overall functioning is poorly understood [12,13]. Blatant errors (e.g., misreported or incorrect statistical results; inaccurate or misleading citations), omissions (e.g., selective reporting of results), misrepresentations (inflation or deflation of findings), and even cases of fraud (e.g., fabrication of data, falsification) are inevitably published [14,15,16,17].

To some extent this is to be expected. Both research and peer review are imperfect, human processes [18,19], which rely on the trust of authors, reviewers, and editors [20,21]. Therefore, it should not come as a surprise that, at times, those who might be considered bad faith actors (actively or passively) abuse and manipulate the publishing system. The following examples are illustrative.

In 2011, the award-winning social psychologist and former dean Diederik Stapel was found to have created and used fraudulent data (i.e., altering existing and/or constructing fictitious datasets) or manipulated existing data in more than 50 papers [22,23]. Some of these papers were published in the discipline’s top journals (e.g., Psychological Science, Journal of Personality and Social Psychology, Personality & Social Psychology Bulletin). Though no other persons were found to be complicit in the committed fraud, Stapel’s actions had a wide-ranging, negative impact on colleagues, co-authors, students, and the profession of psychology as a whole [23].

More recently, Lindsay and colleagues, in what has become known as the “Grievance Studies Affair” [24], wrote and submitted 20 questionable—and sometimes fraudulent—manuscripts to journals in the humanities and social sciences. This case, which received considerable media attention [25], caused something of a minor stir within the field of social work [18], since one well-known social work journal Affilia: Feminist Inquiry in Social Work was a target of the hoax—ultimately accepting one of the manuscripts for publication.

There are numerous other (varied) examples—both historical and contemporary—which cut across the biomedical and social sciences, and include gross misconduct, plagiarism, breaches in research ethics, and manipulated or faked peer review, among other things [26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48]. These actions contaminate the professional literature—including literature used and published by social work researchers and scholars—by certifying and canonizing false claims [49,50,51]. This undermines the validity of evidence-synthesis and the process of evidence-based practice. Xu et al. [52], for example, investigated the impact that 1330 retracted randomized controlled trials (RCT) had on subsequent systematic reviews and meta-analyses. Of the 3902 meta-analyses that could be analyzed, the removal of the retracted RCTs resulted in a change to the direction of the pooled effect in 8.4% of cases and a change in determination of statistical significance in 16% of cases. Kulkarni [53] discuss how image duplication and manipulation affected the synthesis of evidence gathered, and subsequent publication, of a review on preclinical studies of depression.

Additionally, research misconduct wastes resources, can lead to real-world harms to individuals and communities, and otherwise undermines trust in the overall scientific process [10,44,54,55,56,57]. This is particularly concerning for the discipline of social work, where there is a professional duty to develop and provide effective services to clients and communities [54]. Research misconduct and fraud risk exposing those served to potential harm and may damage the credibility of the profession [58].

1.1. The Prevalence of Research Misconduct and Fraud

Many of the above cases involve fraud. However, fraud is but one of several unsavory research practices under the broader umbrella of research misconduct. While there is no universally agreed upon definition, research misconduct is generally understood to include practices such as data fabrication, falsification, and plagiarism (commonly referred to as FFP) [59,60]. Some definitions also include insufficient ethics approval, gift authorship, and selective reporting of results, among other things [61,62]. “Honest mistakes” are typically [60], though not always, excluded from discussions of research misconduct.

How common are research misconduct and outright fraud? The data on this is uncertain, in part because most institutional investigations are not made public [63] and suspected cases are almost certainly underreported—to say nothing about the clandestine nature of such actions. Indirect evidence gives a sense of the upper and lower bounds. Marshall [64] summarizes research on the topic, with estimates ranging from one instance of fraud per 100,000 scientists down to one in 100. Others estimate somewhere near one in 10,000 [65]. In a 2021 meta-analysis of studies on research misconduct by Xie and colleagues [66] estimates that the self-reported prevalence of FFP is 2.9% (95% CI: 2.1–3.8%). The estimated prevalence of FFP, observed by others, is 15.5% (95% CI 12.4–19.2%). These figures parallel previous work on the topic [67], which estimated that about 2% of researchers have admitted to fraud, falsification, or other serious data manipulation. More recently, Heathers [68] criticizes this figure as too conservative—a point readily acknowledged in Fanelli’s 2009 paper [67]—and gives a rough estimate of around 1 in 7 papers, containing a fabricated or falsified element.

Estimated rates of plagiarism are similarly crude. Pupovac and Fanelli [69] synthesized the results of self-reported and non-observed plagiarism (i.e., observed or suspected plagiarism). The estimate that the self-reported prevalence of plagiarism is 1.7% (95% CI: 1.2–2.4%). The estimated prevalence of non-observed plagiarism is 29.6% (95% CI 17.4–45.5%). Synthesizing studies of textual similarity, as identified using plagiarism software and human verification, Pupovac [70] calculated a pooled estimate of 18% (95% CI: 12–25%) of articles with instances of plagiarism.

1.2. Editorial Responses to Misconduct and Fraud

When things go wrong, publishers and journal editors have a series of tools at their disposal to respond. These include errata, corrigenda, and expressions of concern. Errata are corrections made to a published article, typically in response to a minor error (e.g., a production error caused by the journal or publisher) [71]—one which does not affect the overall findings or messages. Corrigenda are fundamentally the same, referring to corrections of small errors caused by the article’s authors [71]. Expressions of concern are more serious in nature. Issued by the publisher or journal editor(s), expressions of concern are used to raise awareness that an article’s findings may be—but are not confirmed to be—unreliable (e.g., in an alleged, but still unproven case of research misconduct) [71]. In extreme cases—when an article is found to be seriously flawed and is thus determined to be unreliable—a paper may be retracted and removed from the published record altogether [71]. Organizations, such as the Committee on Publication Ethics (COPE, https://web.archive.org/web/20240109070233/, https://publicationethics.org/retraction-guidelines; accessed on 20 August 2024), the International Council of Medical Journal Editors (ICMJE; https://web.archive.org/web/20231108144736/, http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/corrections-and-version-control.html; accessed on 20 August 2024) and the Council of Science Editors (CSE, https://web.archive.org/web/20240110174932/, https://www.councilscienceeditors.org/3-5-correcting-the-literature; accessed on 20 August 2024), all have developed guidelines, standards, and resources for when and how to initiate a retraction.

1.3. How Common Are Retractions?

On the whole, retractions are relatively uncommon. Estimates range somewhere between two and eight retractions per 10,000 papers [72,73], though as noted by Cokol et al. [74] this may underestimate the number of papers which should correctly be retracted. Other research has emphasized the increase in retractions over the course of the last several decades [75]. Oransky and colleagues [73] have cataloged more than 4600 retractions in 2022—though the rate at which papers are retracted may be slowing slightly [72,76]. An analysis by Nature found that there were more than 10,000 retractions in 2023—pushing the ratio of papers retracted above 0.2% [77].

Even the most esteemed academic journals are susceptible [76,78,79]; dispelling the erroneous assumption that poor quality or otherwise suboptimal scholarship is published only in low-ranking journals [11,80].

This overarching question posed here is further obscured when taking into account disciplinary differences. Although retractions have increased over time, some studies suggest that rates are still higher in the biomedical and life sciences, compared to the social sciences and humanities [78]. A recent study [81] of retractions made on the basis academic misconduct (25,710 retractions) found wide variation in retraction rates across disciplines. Across the ten disciplinary categories, rates ranged from 1.7 retractions per 100,000 articles (physics) to 17.4 per 100,000 (electrical engineering, electronics, and computer science). The social sciences ranked fourth highest at 5.3 per 100,000, though this is still notably lower than the 8.9 retractions per 100,000 in the clinical and life sciences.

The presence of retractions is not necessarily problematic—in moderation, they may indicate a robust ability to self-correct. Fanelli [76] hypothesizes that these figures, rather than signifying an increase in the rate of fraud and misconduct reflect that researchers, editors, and institutions are more adept at identifying (and increasingly likely to report), papers that cause concern. It is unclear from the available data if this is true. Fang et al. [82] note that among a large sample of retracted articles in the biomedical and social sciences, the most common reason for retraction was “fraud or suspected fraud” (43.4%)—more than twice the rate of those retracted due to error (21.3%).

Nevertheless, these corrective actions may be a marker of a progressive journal or publication system [83], one that prioritizes knowledge generation over non-epistemic factors, such as novelty or perceived impact. If retractions are scarce or altogether absent, it may signify suboptimal mechanisms for self-correction within journals or the broader scientific community [44,84,85].

1.4. Purpose of the Study

How common are retractions within the field of social work? It is unclear from the studies and literature surveyed above. Given the notable increase in retractions over the past several decades, one would expect to find some obvious examples—even if the overall base rate is low. However, the minimal discussion about retractions within the social work literature yields little information about their disciplinary prevalence. Gibelman and Gelman [31], writing in the Journal of Social Work Education, note that “[P]ublic disclosure of cases of scientific misconduct within the social work research community have been absent” (p. 244). In 2017, in an editorial from the same journal, (then) Editor-in-Chief Joanne Yaffe [86] remarked, “…I am not aware of articles retracted from social work journals specifically…” (p. 369). In 2018, Ferguson and Clark [87] commented that, “The social work profession, at this point, holds limited knowledge about the state of misconduct in the field because social work researchers are not empirically investigating the conduct of research”.

The study is intended as a first step towards answering this question by determining the prevalence of retractions within ten leading social work journals and describing their characteristics.

2. Materials and Methods

This study described here is a preliminary and exploratory cross-sectional analysis of the frequency of retractions in social work. As such, it was not preregistered and there were no formal hypotheses—beyond the strong belief held by the author that there were likely some, but relatively few, retractions in the social work literature. Retractions are highlighted because their use is typically reserved as a response to the most egregious and pernicious scholarly practices. Accordingly, they represent the most forceful and concrete example of self-policing and correction. This study does not address the prevalence of corrections (i.e., errata and corrigenda), author and publisher withdrawal of manuscripts, or expressions of concern. Corrections are reserved for minor errors and issues which do not affect the overall findings or message. Withdrawn manuscripts, as described by COPE [88], typically occur prior to publication (e.g., during the review phase or while in-press) or when there is an administrative or production error [89]. As noted above, expressions of concern are reserved for instances where findings have not been confirmed to be unreliable or where investigation is still pending. This study aims to explore concrete instances where retractions have been made.

2.1. Sampling Frame and Analytic Strategy

A convenience sample comprising ten leading social work journals was used for this analysis. Journals were chosen as the unit of analysis because they provide data for understanding historical events and trends within the professional literature [90]. The journals, presented in alphabetical order, were as follows: (1) British Journal of Social Work, (2) Families in Society, (3) Journal of Social Work, (4) Journal of Social Work Education, (5) Journal of Sociology & Social Welfare, (6) Journal of the Society for Social Work and Research, (7) Research on Social Work Practice, (8) Social Service Review, (9) Social Work, (10) Social Work Research (formerly Social Work Research and Abstracts [SWRA]). These journals were chosen, not on the basis of impact factor, which has numerous limitations [83,91,92], but because they are representative of the field [93,94], are well-regarded by social work scholars [95], and generally have a substantial publication history.

Hodge and Lacasse [94] have nine of the ten journals listed above ranked among the top 20 by h-index; the exception being the then nascent Journal of the Society for Social Work and Research (JSSWR). All journals, on average, have a high-level of prestige among senior social work faculty, with all journals ranking among the top 15, and nine of ten ranking within the top 10 [95].

Three search strategies were employed. First, each journal website was reviewed. During January of 2023, two sets of key words (i.e., “retracted” and “retraction”) were used to search for retracted articles. Using these two terms permitted the broadest possible search for retracted articles, permitting the identification of articles labeled simply as “retracted” or “retraction”, but also being inclusive of articles and notices presented in other ways (e.g., “retraction notice”, “retraction statement”, “statement of retraction”). In cases where an article was not clearly labeled (e.g., in the title or via watermark) as being retracted, abstracts were inspected by the author. Then for each journal, using Google Scholar’s advanced search (https://scholar.google.com/#d=gs_asd&t=1679609281252; accessed on 20 August 2024) function, the journal’s title was entered (i.e., “Return articles published in”) with at least one of the terms “retracted” or “retraction” being required (i.e., “With at least one of the words”). Finally, these searches were supplemented using the Retraction Watch Database (version 1.0.6.0; see http://retractiondatabase.org/; accessed on 20 August 2024), the world’s most comprehensive resource on the topic [72], with more than 58,000 retractions listed as of May 2025. This database was queried using the names of each journal. All results were examined and where necessary, reviewed by following the posted DOI. The full Retraction Watch Database was queried a second time on 7 May 2025, but did not yield any additional results. The search strategies used spanned the entire publication histories of the chosen journal titles.

To help verify results from these three search strategies, editors-in-chief for all ten journals were emailed to inquire whether, to their knowledge, there had ever been any retractions made, historically, at their respective journals. All results obtained using the search strategies described above were reviewed by the manuscript’s sole author (DJD). There were no additional attempts to cross-validate the study’s results.

2.2. Data Availability

All data and materials can be freely accessed via the project’s Open Science Framework repository (https://osf.io/y5bdf/?view_only=6071353334724014a66d3ba1fb2fb8aa; (accessed on 20 August 2024)). Search results are saved as PDFs and organized by search strategy. Where possible, URLs for websites (and references) have been archived using the Internet Archive’s “Wayback Machine”. Websites/pages which prevent crawling have been saved as pdfs and are made available at the link above.

3. Results

3.1. Journal Characteristics

There is a fair amount of diversity among publishers of the ten journals. Oxford University Press and Sage Publishing each serve as the publisher for three journals in the sample. Two journals are published by the University of Chicago Press. One journal is published by Taylor & Francis. Finally, the Journal of Sociology & Social Welfare is jointly published by Western Michigan University, its College of Health and Human Services, and its School of Social Work (henceforth WMU/CHHS/SSW). Who publishes a journal is important as journal policies are often influenced by decisions and standards made by publishers.

All four major publishers (Oxford University Press, Sage Publishing, Taylor & Francis, University of Chicago Press) explicitly stated that they follow COPE guidelines on retractions [96,97,98,99]. All but four journals (i.e., Journal of Social Work Education, Journal of Sociology & Social Welfare, Journal of the Society for Social Work and Research, Social Service Review) were listed (https://publicationethics.org/members; accessed on 20 August 2024) as COPE members—though the Journal of Social Work Education pointed to their publisher’s membership to the organization, on its website (https://web.archive.org/web/20230920233817/, https://www.tandfonline.com/action/journalInformation?show=aimsScope&journalCode=uswe20; accessed on 20 August 2024). Only two journals, British Journal of Social Work (https://web.archive.org/web/20230824022839/, https://academic.oup.com/bjsw/pages/Code_Of_Practice; (accessed on 20 August 2024)) and the Journal of Social Work (https://web.archive.org/web/20231215224707/, https://journals.sagepub.com/author-instructions/JSW; (accessed on 20 August 2024)) had specific, if limited, discussion about the retraction process, on their websites. These and other characteristics are described in

Table 1. Characteristics of ten leading social work journals.

Journal Title (Year Established)	Publisher	Editor-in-Chief 1	Website
British Journal of Social Work (1971)	Oxford University Press	Maglajlic & Ioakimidis	https://academic.oup.com/bjsw/ (accessed on 20 August 2024)
Families in Society (1920)	Sage Publishing	Mogro-Wilson	https://journals.sagepub.com/home/fis
Journal of Social Work (2001)	Sage Publishing	Shardlow	https://journals.sagepub.com/home/jsw (accessed on 20 August 2024)
Journal of Social Work Education (1965)	Taylor & Francis	Parrish	https://www.tandfonline.com/journals/uswe20 (accessed on 20 August 2024)
Journal of Sociology & Social Welfare (1973)	WMU/CHHS/ SSW	McCormick	https://scholarworks.wmich.edu/jssw/ (accessed on 20 August 2024)
Journal of the Society for Social Work and Research (2010)	University of Chicago Press	Herrenkohl	https://www.journals.uchicago.edu/toc/jsswr/current (accessed on 20 August 2024)
Research on Social Work Practice (1991)	Sage Publishing	Thyer	https://journals.sagepub.com/home/rsw (accessed on 20 August 2024)
Social Service Review (1927)	University of Chicago Press	Mosley	https://www.journals.uchicago.edu/toc/ssr/current (accessed on 20 August 2024)
Social Work (1956)	Oxford University Press	Scheyett	https://academic.oup.com/sw (accessed on 20 August 2024)
Social Work Research (1994) Formerly Social Work Research & Abstracts, 1977–1993	Oxford University Press	Hawkins	https://academic.oup.com/swr (accessed on 20 August 2024) https://academic.oup.com/swra (accessed on 20 August 2024)

¹ Editors-in-Chief are as of January 2023.

.

3.2. Main Findings

None of the 136 results produced from journal websites using the search terms “retracted” and “retraction” was identified as a retracted article. In Social Service Review, a letter to the editor by Kendall [100] calls for the retraction of an article by James Brown IV—for comments made in their prior article on Marshall Field. However, no retraction appears to have been made (see the response by Brown IV) [101] in the same issue). Nor were any of the 59 results produced from subsequent searches of Google Scholar identified as a retraction. The Retraction Watch Database produced one result labeled as a retraction in Research on Social Work Practice, which was classified as a “duplicate publication through error by journal/publisher”. However, examination of the publisher’s posted notice [102] shows that the article was “withdrawn” due to an administrative error, in which the “article was published twice Online First with two different DOIs”. Therefore, this instance is not classified as a retraction in the current study. Full search results are described in

Table 2. Results of each search strategy, by journal title.

Journal Title (Years Covered)	Website Results (Search Term)	Google Scholar Results	Retraction Watch Results (by Title)
British Journal of Social Work (1971–2022)	20 results (retracted) 20 results (retraction)	13 results	0 results
Families in Society (1920–2022)	23 results (retracted) 23 results (retraction)	6 results	0 results
Journal of Social Work (2001–2022)	4 results (retracted) 4 results (retraction)	2 results	0 results
Journal of Social Work Education (1965–2022)	8 results (retracted) 8 results (retraction)	5 results	0 results
Journal of Sociology & Social Welfare (1973–2022)	10 results (retracted) 10 results (retraction)	1 result	0 results
Journal of the Society for Social Work and Research (2010–2022)	2 results (retracted) 2 results (retraction)	1 result	0 results
Research on Social Work Practice (1991–2022)	16 results (retracted) 16 results (retraction)	9 results	1 result
Social Service Review (1927–2022)	21 results (retracted) 21 results (retraction)	11 results	0 results
Social Work (1956–2022)	16 results (retracted) 16 results (retraction)	8 results	0 results
Social Work Research (1994–2022) Formerly Social Work Research & Abstracts (1977–1993)	3 results (retracted) 3 results (retraction) 13 results (retracted) 13 results (retraction)	1 result 2 results	0 results 0 results
Total search results	136 (retracted) 136 (retraction)	59 results	1 result
Total retractions identified	0 retractions	0 retractions	0 retractions

.

Of the eleven editors-in-chief queried (the British Journal of Social Work currently utilizes two editors-in-chief), only six responded. None reported retractions under their leadership or were aware of any under previous editorships. In a few cases (i.e., Families in Society, Journal of Social Work Education, Social Service Review), support staff (e.g., production editors and publishing personnel) were consulted by their respective editors—none of whom could identify any retractions at their affiliated journals.

All relevant materials, including complete search results for both website and Google Scholar searches, for each journal, are accessible as pdfs on the Open Science Framework (OSF) repository linked above. This is done, not only in keeping with the ethos of open science, but to address a limitation identified by Haddaway and Gusenbauer [103]—that search strategies relying on Google Scholar often lack transparency and reproducibility.

4. Discussion

The analysis presented here is not meant to support the strong claim that social work journals and editors do not make retractions. This claim is immediately falsified by a quick, non-systematic search of the social work literature [104]. Rather, it lends support, to the more modest (and perhaps interesting) claim, that retractions are not being made in the field, at a rate one would expect relative to the rest of the biomedical and social sciences. Of course, this conclusion is tentative and will need to be confirmed by a more comprehensive study—ideally a multi-analyst, preregistered review of all major social work journals. The opinions, beliefs, and editorial practices of journal editors could also be queried to provide greater context and may help partially explain this finding.

4.1. Why Are There So Few Retractions in Social Work?

A number of factors impede the identification and investigation of research misconduct and other issues that undermine the validity of published scholarship (e.g., questionable research practice) [66,105]. First, the incomplete reporting of statistical results makes evaluating a study’s findings difficult. It has long been known that many published articles lack sufficient detail in both study methods and results—rendering many unusable and/or uninterpretable [106]. Though this has not been systematically investigated, this is likely the case for social work research as well [105]. Second, even when results are sufficiently described, a lack of open data precludes any possibility for meaningfully assessing computational reproducibility in quantitative studies or determining validity of findings in qualitative studies.

Even after accounting for the numerous challenges and general complexities of sharing data, especially from qualitative studies [107,108,109,110], these issues will increasingly need to be confronted. In the U.S., for federally funded research (whether qualitative or quantitative), all scientific data is being required to be made as open and accessible as possible [111].

Finally, there is a general disinterest (by social workers) in reproducibility and replicability [14], which seems to have led the discipline to a place where every scholar wants to publish, but nobody (readers, peer reviewers, editors) wants to meaningfully validate this work. This final point is lent support by a 2018 systematic review by Ferguson and Clark [87]. In their sample of published articles on the topic of research ethics (n = 1409), only 16 (1%) came from within the social work literature.

4.2. Improving Pre- and Post-Publication Peer Review: Roles for Editors and Reviewers

Gibelman and Gelman [58] rightly note that allegations and investigations of research misconduct (and the retraction process more generally) are “after-the-fact” activities, emphasizing the importance of prevention (e.g., training and mentorship of burgeoning scholars). Others point toward the role of institutions and funders in preventing, deterring, and identifying misconduct and other questionable research practices [112]. Since these important efforts will take time, and are impacted by institutional and professional inertia, this discussion focuses on simple tools and methods that could actively be used now (before or after peer review), by reviewers, editors, and readers, to probe the results presented in quantitative and qualitative studies and bolster the integrity of the field and literature. First, a set of tools is described, which can be used to detect statistical inconsistencies and errors by relying on the results (e.g., sample size, mean, standard deviation) of commonly reported statistical tests. Second, the case if made that qualitative research (common within social work scholarship) could be enhanced through the use of preregistration and a commitment to open science practices. However, in making these recommendations, it is acknowledged that the current publishing dynamics and broader institutional factors unduly place the burden of research integrity on journals and publishers [113] and may also involve a disproportionate risk at the individual level [114]. The interested reader may consult Heathers [115] for an impassioned discussion about the necessity of this broad domain of “forensic metascience”, the essential technological, institutional, and social conditions and incentives required for its facilitation, and how individuals engaged in its conduct may be better supported [112,114].

4.2.1. Probing Quantitative Studies: Re-Analysis of Study Results

A number of tools exist to detect statistical inconsistencies or errors, which may be indicative of questionable or fraudulent research practices in quantitative studies [116]. The Granularity Related Inconsistency of Means (GRIM) test is used to check the consistency of reported means, when reporting results for integer data (i.e., assessing whether the reported results are possible, given the reported sample size and number of items). This is particularly useful when sample sizes are small. Similarly, the Descriptive BInary Test (DEBIT) assesses the accuracy of descriptive statistics for binary data, when the mean and standard deviation are reported. In these circumstances, a given sample size and reported mean will always yield a specific standard deviation. The Sample Parameter Reconstruction via Iterative Techniques (SPRITE) comprises a series of methods used to generate plausible distributions of data, based on the reported sample summary statistics (i.e., sample size, mean, standard deviation, range). This is a useful method for probing for data anomalies or irregularities, especially when the underlying data is unavailable or incomplete.

Statcheck, which is available as a web app (https://michelenuijten.shinyapps.io/statcheck-web/; accessed on 20 August 2024) and R package, extracts the results of significance tests from manuscripts, recomputes reported p-values based on the associated test statistics and degrees of freedom (where reported) [117]. Statcheck then compares the recomputed p-values with the reported p-values to see whether they are consistent. Besides reporting accuracy, this tool can aid in assessing whether a result is correctly interpreted as being statistically significant or not.

Preliminary work, which compared consistency in statistical reporting between two prominent psychology journals (Psychological Science, Journal of Experimental and Social Psychology) that implemented statcheck during peer review, with two control journals (Journal of Experimental Psychology: General, Social Psychology), lends support for its (judicious) use [117]. See Francis and Thunell [118] and Heathers [116,119] for an overview of these and other tools and techniques. For a discussion of how social work research may be made more reproducible/replicable, see Dunleavy and Lacasse [14].

4.2.2. Accountability and Verifiability in Qualitative Studies

While qualitative research presents its own unique challenges (e.g., particular epistemological assumptions, commitments, and goals; protection of participant anonymity and confidentiality), similar concerns remain around the trustworthiness, rigor, warrant, and validity of findings from these studies [120]. Although qualitative research is not susceptible to the same types of misconduct as its quantitative counterparts, opportunities still arise for the commitment of fraud, fabrication, and other idiosyncratic or questionable practices [24,109].

At least in some circumstances, the trustworthiness of qualitative research may be increased by completing a study preregistration. In this context, preregistration would include description of the study aims, design, hypotheses (if applicable), the approach or tradition(s) in which the study is conducted, and detailed information about how data is collected, coded (if applicable), weighed, and synthesized. This could also include detailed information about how data and analyses are validated (e.g., triangulation, use of multiple coders, member-checking, auditing by a researcher outside of the study). A justification for the use of preregistration in qualitative research and an example template is provided by Haven et al. [121]. Haven and Van Gootel [122] describe some common objections and limitations for the preregistration of qualitative studies.

Regardless of whether or not qualitative research is preregistered, its credibility may also be strengthened by making data and materials open for review (e.g., during the review stage, with publication). Following Dubois et al. [109,110], qualitative research should, as a default, be shared with readers and peer reviewers to the greatest extent possible. Understandably this may not be possible in all circumstances. But to the extent to which it is possible, even simply sharing materials (e.g., interview questions, coding information) can improve the trustworthiness and integrity of a study. The OSF (https://osf.io/) and Qualitative Data Repository (https://qdr.syr.edu/qdr-resources) offer some means and guidance for doing so.

5. Conclusions

Social work continues to grow as an academic, research-based profession. It is necessary that we ensure that our policies and practices are both ethical and evidence-based. Not only do social workers have a duty to generate research and scholarship in a way that facilitates the provision of effective service to clients and communities [54], but the field has a responsibility to continuously evaluate how that research and knowledge is vetted and disseminated, both before and after publication. Given that no retractions were able to be identified in this analysis, it is suggested that this has and is not being done effectively.

Rather than merely acting as if the published literature is the final word, social workers should continuously seek to appraise and critique the literature, to employ and reward pre- and post-publication peer review practices, and where necessary cleanse and clarify the scholarly corpus [43]. This may mean, going forward, that the field of social work sees a greater number of retractions. Of course, this need not necessarily be the case. Advances in technology and scholarly publishing are beginning to foster an environment that supports the version control of manuscripts, which may make retractions, if not obsolete, then at least less common [123].

One step towards these goals of increased oversight and accountability would be for social work editors to encourage peer reviewers to scrutinize the results of quantitative studies using the methods described in Section 4.2.1, before a manuscript is accepted for publication. This could be performed as a pilot test, using senior reviewers, before being scaled up across the editorial pipeline. Editors could also consider working with meta-researchers to publish the results of such efforts in the aggregate. Similarly, social work scholars could also systematically sample articles from leading social work journals to determine the rate at which findings are inconsistent or erroneous.

Another step would be the formal systematic review of the social work literature—which would entail sampling a more representative sample of social work journals. For example, this could entail picking the top 25 journals based on impact factor, those with the highest publication output, the highest readership, or the largest impact [124]. This would help to shore up the limitations of the convenience sample used here and give a clearer answer as to how common retractions are within the discipline. Subsequent research in this area may also benefit from consultation with a research librarian or information specialist, who can assist in developing a reproducible search strategy—including the specification of search terms, journals, and databases—and assist with methods for data extraction.

Finally, social work scholars may consider interviewing or surveying reviewers, editors, and publishers within the discipline, who may better be able to share their expertise as to what accounts for the purported lack of retractions within social work.