Previous Article in Journal
The Cover Letter as a Publication Support Gap: Structural Disadvantage and AI-Assisted Preparation for Non-Native English Speakers
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Publications
Volume 14
Issue 4
10.3390/publications14040063
publications-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework

by
Eungi Kim
* and
Vaishali Singh
Department of Library and Information Science, Keimyung University, Daegu 42601, Republic of Korea
*
Author to whom correspondence should be addressed.
Publications 2026, 14(4), 63; https://doi.org/10.3390/publications14040063
Submission received: 26 July 2026 / Revised: 18 September 2026 / Accepted: 21 September 2026 / Published: 24 September 2026
Browse Figures
Versions Notes

Abstract

The rapid adoption of large language models (LLMs) has prompted extensive debate about their appropriate role in peer review, scholarly publishing’s primary quality-control mechanism. However, AI has not yet been formally approved as a peer-review tool by most academic journals. This study reviews the emerging AI-in-peer-review literature to identify research trends, synthesize empirical evidence across review tasks, and develop a conceptual framework for AI-assisted review. Using a PRISMA-guided Scopus search (176 records identified, 162 included), we combined three-layer content analysis (theme, editorial stance, and AI autonomy) with a synthesis of 18 empirical studies. The literature expanded from 6 records before 2023 to 45 records in the first half of 2026 and remains dominated by commentary and opinion (57%), with the remaining 43% comprising research studies, technical work, and reviews. Editorial perspectives are generally balanced, and authors overwhelmingly favor assistive, human-in-the-loop AI over human-only or full automation. Empirical evidence shows a task-contingent pattern: AI performs well on narrowly defined evaluative tasks (Pearson r > 0.9 in some settings) but less reliably when predicting editorial decisions (accuracy 40–67%; correlations as low as ρ = 0.00). AI legitimacy may depend more on task type than on any governance position, a pattern we formalize in a Task-Contingent Legitimacy framework offering a task-tiered policy approach and testable propositions.

1. Introduction

Peer review is the central quality-control mechanism of scholarly publishing, but it has long been under strain from rising submission volumes, reviewer fatigue, and uneven review quality (Perlis et al., 2025), even before the advent of generative artificial intelligence (AI). Early exploratory work on computational support for editorial decisions predates this period by several years (Mrowinski et al., 2017). The release of ChatGPT in late 2022 added a further complication, introducing large language models (LLMs) capable of producing fluent, structurally plausible review reports, manuscript feedback, and editorial correspondence. Since the release of ChatGPT, the number of scholarly publications addressing AI’s role in peer review has grown from a handful annually to dozens per year, and early estimates suggest that a non-trivial share of reviews at major computer science venues are now partly or fully AI-generated (Singh Chawla, 2024).
Despite this rapid growth, the literature remains fragmented. Editorials, letters, and technical benchmarking studies often address similar questions using different conceptual vocabularies and without a common framework for comparing findings across task types, disciplines, or governance contexts. Existing reviews typically focus on particular disciplines, tools, or ethical issues (Price & Flach, 2017; Nabavi et al., 2026; Bauchner & Rivara, 2024). As a result, they provide valuable insights into individual aspects of AI-assisted peer review but do not offer an integrated synthesis that connects publication trends, conceptual perspectives, and empirical evidence across the broader literature.
This review aims to provide a comprehensive synthesis of the emerging literature on AI in peer review and to develop a conceptual framework for evaluating when AI use is appropriate across different peer review tasks. To achieve this, the paper combines a PRISMA-guided mapping of the literature, a content analysis examining three dimensions (thematic focus, editorial stance, and AI autonomy), and a synthesis of empirical studies evaluating AI performance in review-related activities. Building on these findings, the paper proposes a Task-Contingent Legitimacy framework that specifies when, and under what governance conditions, the use of AI in peer review is legitimate. Legitimacy is understood here, following Suchman’s (1995) foundational account, as a generalized perception that an actor’s or practice’s actions are appropriate within a socially constructed system of norms and values, a lens well suited to a technology whose acceptability is still being negotiated rather than settled. The model proposes that the legitimacy of AI is not determined by the technology itself but varies systematically according to the nature of the review task and the strength of the governance mechanisms that accompany its use.

2. Methods

2.1. Search Strategy

Records were identified through a title-field search of Scopus on 12 July 2026, combining artificial intelligence terms (“artificial intelligence”, “generative AI”, ChatGPT, “large language model*”, LLM*) with peer review terms (“peer review”, reviewer*, referee*) using Boolean AND. The verbatim query, executable in Scopus Advanced Search, was:
TITLE ((“artificial intelligence” OR “generative AI” OR ChatGPT OR “large language model*” OR LLM*) AND (“peer review” OR reviewer* OR referee*))
A title-only search was chosen deliberately over a title–abstract–keyword search to maximize topical precision at the cost of recall: because the target concept is narrow relative to the broader literature on “AI in scholarly publishing”, a title search yields a smaller but substantially more relevant corpus, at the expense of missing papers that address AI-assisted peer review without stating so in the title. This trade-off is discussed further in the Limitations section.

2.2. Screening and Eligibility

The search identified 176 records. One duplicate (a companion editorial republished verbatim in two affiliated journals) was removed prior to screening. Records were included if they addressed the use of artificial intelligence, including—but not limited to—large language models, specifically within the peer-review process of scholarly manuscripts, in any discipline, document type, or study design; records were excluded if they used the term “peer review” in an unrelated sense (for example, clinical or accreditation review), addressed AI in scholarly publishing broadly without a specific peer-review focus, or were non-substantive notices such as errata or corrections. The remaining 175 records were screened by title and abstract; 13 were excluded as either off-topic (n = 12, using “peer review” in a different sense, such as clinical radiotherapy quality checks, or addressing AI in scholarly publishing broadly without a specific peer-review focus) or as a non-substantive erratum (n = 1). All 162 remaining records were deemed eligible, yielding a final corpus of 162 included studies. Figure 1 presents the complete PRISMA flow through identification, screening, and eligibility. Only 13 of 175 screened records (7%) were excluded, reflecting the tight fit between the title-only query and the topic rather than lenient screening.
Because the included corpus spans document types as heterogeneous as single-paragraph editorials and full empirical studies, a single formal quality-appraisal instrument designed for one study design (for example, a risk-of-bias tool) was not applicable across the full set. Instead, Scopus indexing served as a baseline relevance filter, and confidence flags distinguishing classifications verified against the full text from classifications based on the title only served as indicators of coding confidence. The 18 studies retained for the quantitative evidence synthesis were selected specifically because they reported an original, quantitative, peer-review-relevant finding.

2.3. Coding Procedure

The included corpus was coded along three dimensions: thematic focus, editorial stance, and AI autonomy. Each dimension combined automated first-pass classification with manual verification. Thematic classification assigned each record a primary theme (for example, performance validation, ethics and integrity, policy and governance) based on the title and, where available, abstract content. An initial keyword-based pass classified 114 of 162 records on the basis of an unambiguous match in the title or abstract to a single predefined theme, and the remaining 48 records—predominantly editorials and letters that lacked a Scopus abstract—were resolved through a combination of targeted web searches to recover full-text content and close reading of the title in context, leaving 6 records genuinely unclassifiable from the available metadata. Named AI models were identified by pattern matching titles and abstracts against a list of major LLM product names (ChatGPT or GPT-family, Gemini, Claude, DeepSeek, Qwen, LLaMA, Copilot, Mistral, Gemma) in order to track which specific systems the literature discusses and how this has changed over time.
Editorial stance was coded for the subset of records that were commentary or opinion pieces (editorials, letters, notes, short surveys; n = 92 codable) as optimistic/advocacy, cautionary/critical, balanced/nuanced, or neutral/prescriptive. Empirical, technical, synthesis, and correspondence records (n = 70) were excluded from stance coding on the grounds that reporting a finding or restating a policy is not equivalent to taking an evaluative position. Autonomy position, coded independently of stance, captured the level of AI autonomy that each record advocates or empirically tests (full automation or replacement, assistive or human-in-the-loop, human-only or restricted, or not stated). This dimension reflects the substantive content of a position rather than its emotional valence, since a cautionary-toned piece can still endorse assistive use and a neutral empirical study can still test full automation. Throughout, coding decisions distinguish between classifications drawn from verified full text and those inferred from the title alone; the latter are flagged as lower confidence in the accompanying data files and should be treated as indicative rather than definitive.

2.4. Analytical Approach

Analysis proceeds in three stages, corresponding to the Results sections below: descriptive bibliometrics (publication trends, citation landscape, and named-model trajectories), content analysis (distributions of themes, editorial stances, and autonomy positions), and evidence synthesis (a structured summary of quantitative findings from the 18 empirical studies, organized by task type). These three layers are then brought together in the proposed theoretical framework. “Bibliometric-guided” refers here to the descriptive publication metrics reported above, growth trajectory, citation landscape, venue dispersion, and named-model trajectories, not a citation-network or co-citation method; these metrics guide the review in two concrete ways. The corpus’s extreme venue dispersion (129 journals, 109 with a single record) supports the case for an integrative, cross-disciplinary framework over a discipline-specific one, and the growth trajectory (from 6 pre-2023 records to 45 in the first half of 2026 alone) adds urgency to the framework’s policy aims. Both bibliometric findings are referenced explicitly where they inform the proposed model below.

3. Results

3.1. Bibliometric Results

Publication activity is overwhelmingly concentrated in the post-ChatGPT period. Of the 162 included studies, only 6 (3.7%) appeared before 2023; Figure 2 shows this trajectory year by year: output then rose from 20 records in 2023 to 36 in 2024, 55 in 2025, and 45 in just the first six and a half months of 2026, on pace to substantially exceed 2025 once the full year is observed.
Table 1 lists the 15 most-cited included studies in full. Citation counts unsurprisingly favor earlier publications, which have had more time to accumulate citations. The most-cited included record is a 2017 perspective piece on computational support for peer review (Price & Flach, 2017), followed by a 2024 review on AI’s role in supporting publishing and peer review (Kousha & Thelwall, 2024); both predate most of the corpus, so this ranking says as much about time to accumulate citations as it does about influence. A 2020 paper on AI and drug repurposing (Levin et al., 2020) ranks third despite addressing peer review only as a secondary concern within a broader argument about computational screening during the COVID-19 infodemic, a further illustration that citation rank in this corpus reflects age and adjacent relevance as much as centrality to the topic.
The corpus is also highly fragmented across venues. Across the 162 included records, 129 distinct journals and conference proceedings are represented, and 109 of these 129 venues contribute exactly one record each. No single journal contributes more than five records. Such dispersion indicates that the discourse on AI in peer review is unfolding largely field by field, within clinical, technical, and disciplinary outlets, rather than being concentrated in dedicated scholarly communication or library and information science venues.
Figure 3 shows named-model mentions by publication period. Named-model analysis shows that early studies focused almost exclusively on ChatGPT before expanding to multiple LLMs after 2025. Among records that name a specific AI system in the title or abstract, ChatGPT or GPT-family models account for the entire signal through 2023 and 2024 (3 of 3, then 14 of 14 tagged mentions). Greater model diversity, including systems such as Gemini, DeepSeek, Claude, and Qwen, appears only from 2025 onward; by the first half of 2026, ChatGPT or GPT still leads but no longer accounts for the near totality of tagged mentions it once did. This pattern should be read as a directional trend rather than a precise ratio, since only 37 of 162 records name a specific model (yielding 57 tagged mentions in Figure 3, as several records name more than one system) and thus the cell counts are small, but it nonetheless suggests a shift from asking whether ChatGPT works to asking which models perform best for which tasks.

3.2. Content Analysis

The three content-analytic dimensions coded here answer different questions about the same corpus: theme captures what the field discusses, stance captures how it evaluates what it discusses, and autonomy position captures what it actually recommends. Treated separately, each yields a useful but partial picture; read together, as the synthesis at the end of this section shows, they converge on a more specific claim than any single dimension could support on its own. This separation matters substantively, not only analytically: a cautionary or critical tone does not by itself imply opposition to the technology, since a critically toned piece can still advocate regulated, human-in-the-loop assistive use, just as a neutrally toned piece can advocate full automation.

3.2.1. Thematic Distribution

Figure 4 shows the distribution of the 162 included studies across content themes. The largest identifiable themes are human–AI collaboration and the future division of labor (27 records); ethics, integrity, and misconduct risk (24); policy and governance (23); and empirical performance and validation evidence (23). Smaller but distinct clusters address transparency and disclosure norms (13); detecting and policing undisclosed AI use (10); and bias, fairness, and equity (4). A further 11 records are prior reviews or scoping syntheses of this same topic and 9 are correspondence threads best read alongside the article they address. The remaining 18 records fall into five smaller categories shown individually in Figure 4: records still unclassified pending full-text review (6); efficiency, workload, and incentive structures (6); AI-assisted editorial and reviewer-matching tools (3); stakeholder attitudes and perceptions (2); and one outlier applying peer-review logic to AI systems themselves rather than the reverse (1). Policy/governance and empirical-validation themes are near equal in volume (23 records each), a pattern worth flagging even though it rests on a snapshot count rather than a time series and cannot by itself establish whether the two literatures actually grew in step. Bias and fairness remain notably underrepresented: at only 4 records, against one empirical study in this corpus that already finds measurable bias, the shortfall looks like under-investigation rather than a genuinely minor concern.

3.2.2. Editorial Stance

Editorial stance, plotted in Figure 5 across the 92 codeable records, splits four ways: balanced/nuanced positions (39) form the single largest category, ahead of cautionary/critical (26), neutral/prescriptive policy statements (18), and optimistic/advocacy (9). The literature is not polarized into optimistic and critical camps; most authors already treat the topic as multidimensional, weighing benefits against risks within the same argument (e.g., Perlis et al., 2025; Huang et al., 2025), though they rarely name the framework they are implicitly using. Unqualified enthusiasm is rare (9 records), which cuts against reading the growth in publication volume as uncritical hype.

3.2.3. Stated Position on AI Autonomy

Autonomy positions, shown in Figure 6, differ from editorial stance by capturing the substantive role assigned to AI rather than the tone of the discussion. Among the 120 records expressing a discernible position, assistive or human-in-the-loop use is by far the most common (79 records), compared with human-only or restricted use (16) and full automation or replacement (7). Assistive positions outnumber restrictive positions by nearly five to one and full automation by more than ten to one, indicating a clear preference for human-supervised AI rather than autonomous review. The remaining 18 records represent correspondence threads, mixed positions, or methodological discussions (Figure 6).
Reading the three dimensions together sharpens the picture: balanced stance, assistive use, and human–AI collaboration are each the modal category in independent codings of the same corpus. Together, they converge on one position from three angles, providing stronger evidence than any single measure alone, although this remains a pattern in independently stated positions rather than a demonstrated consensus. No record, however, articulates why assistive use is the right default rather than simply asserting it—a gap the next section is intended to fill.

3.3. Empirical Evidence Synthesis

Table 2 groups these findings by task type. Of the 23 records classified under performance and validation evidence, 18 report original quantitative findings; one further record is a non-empirical position piece, and the remaining four do not report findings in a form comparable across the task-type categories used below, so all five are excluded from the synthesis that follows. Because these studies use heterogeneous tasks, datasets, and metrics (accuracy, AUC, Spearman’s ρ, Pearson r, Cohen’s kappa), they cannot be pooled into a single effect size; grouping them this way instead reveals a consistent directional pattern that a flat list would obscure. Across task categories, AI performance varies systematically rather than being uniform: it is weakest on predicting final editorial outcomes and strongest on narrow evaluative judgments, with content generation in between. This pattern, developed further below, suggests that policy questions about AI in peer review are better framed around specific tasks than around AI use in the abstract.
Studies predicting final editorial outcomes (accept/reject, numerical score, journal-quartile placement) show the weakest and most consistent performance pattern in the corpus, regardless of model or fine-tuning: raw accuracy ranges from 40% to 67% (Mohamed et al., 2025a), correlations with true outcomes range from ρ = 0.00 to 0.46 depending on platform (Thelwall & Yaghi, 2025), and performance on predicting acceptance from reviewer comments alone varies substantially with fine-tuning, with AUC reaching 0.91 for a fine-tuned GPT-4-mini model but falling to 0.67–0.75 for untrained models (Hopkins et al., 2026).
Studies asking AI to generate review content occupy a middle ground: AI-written discussion sections can pass blinded review for a high-impact journal after revision (Sheridan et al., 2025), and a large-scale randomized deployment at a major AI conference found that 27% of reviewers revised their reviews after receiving AI-generated feedback, incorporating over 12,000 suggestions across more than 20,000 reviews (Thakkar et al., 2026). Yet head-to-head comparisons consistently find AI-generated reviews shallower and more structurally routine than human reviews, which are more thematically diverse by direct measurement (Rajakumar et al., 2026).
Studies asking AI to evaluate or judge quality, rather than generate content or predict outcomes, generally show the strongest performance: an unsupervised peer-review-style evaluation framework for multimodal models achieved Pearson correlations of 0.944 and 0.814 with human judgment in two benchmark settings (Zhang et al., 2025), though a related study assessing peer review quality directly found much weaker agreement (Cohen’s kappa in the poor-to-moderate range) on the more holistic task of judging an entire review’s quality (Tang et al., 2026).
A cluster of diagnostic studies explains why performance varies. LLMs used as automated reviewers are demonstrably vulnerable to adversarial text manipulation (Lin et al., 2025), systematically misclassify methodological flaws, and let strong rejection recommendations dominate the final decision out of proportion to their justification (J. Li et al., 2025); furthermore, reviewer-side biases account for over a third of decision variation in simulated review environments (Jin et al., 2024). These failure modes are systematic rather than random, which is itself evidence against treating “AI in peer review” as a single undifferentiated capability. Several further studies in the corpus, not itemized in the table above because they report qualitative or correlational rather than directly comparable quantitative findings, are consistent with the same pattern: a direct comparison of four LLMs against human reviewers found systematic differences in emphasis rather than equivalence (Joachim et al., 2025); reviewer-side text features predicted downstream citation impact independently of an LLM-assessed quality score (Sun, 2026); and a peer-review-style consistency-optimization framework for evaluating LLMs performed best when validated against narrow, well-specified criteria rather than holistic judgments (Ning et al., 2025)—the same narrow-task advantage documented in the evaluating-quality category above.

3.4. Thematic Literature Review

Building on the thematic mapping above, the qualitative literature provides greater insight into the arguments underlying each theme. Concerns about the integrity of peer review emerge as one of the most prominent themes. The literature identifies risks ranging from reviewers submitting AI-generated reports without disclosure (Cheng et al., 2024; Singh Chawla, 2024) to concerns that undisclosed AI assistance could allow low-quality or fabricated submissions to pass review undetected (Donker, 2023). Von Wedel et al. (2024) provide direct empirical evidence supporting these concerns, finding that an LLM used to review abstracts exhibited measurable affiliation-related bias, complicating the common assumption that AI assistance is inherently more objective than human review.
Closely linked to these concerns is the question of how AI should be governed in peer review. Formal editorial policies range from outright prohibition to conditional permission with disclosure requirements (Perlis et al., 2025; Munafò, 2024). Empirical audits consistently reveal substantial heterogeneity in journal practice. Across a sample of top medical journals, the majority that provide guidance explicitly prohibit AI use in peer review, while a sizeable minority permit it subject to confidentiality and disclosure requirements (Z.-Q. Li et al., 2024). A discipline-specific audit of neurosurgical journals reports a similarly uneven picture, with published policies on AI use in manuscript preparation and peer review varying widely in scope and specificity even within a single field (Mohamed et al., 2025b). A broader cross-sectional audit of medical-journal guidelines reaches the same conclusion, finding wide variation in whether, and how, generative AI is addressed at all (Yin et al., 2025). Similarly, an editorial in Nature Nanotechnology frames the central concern as over-reliance on a system whose reasoning cannot be fully inspected (Peer review in the time of artificial intelligence, 2026). Collectively, these findings suggest that the field has not yet converged on a single governance model.
Closely related to governance is the question of whether, and how, AI use should be declared. Some authors argue that mandatory declaration is becoming an outdated framing as AI assistance becomes normalized (Rozencwajg & Benhamou, 2026), while others argue the opposite, that disclosure remains essential precisely because undisclosed use is already occurring at scale and undermining trust in the review process (Zou, 2024; Liang et al., 2024).
A technical sub-literature has emerged specifically around detecting AI-generated or AI-modified review text. Liang et al. (2024) present a large-scale monitoring approach for estimating the fraction of AI-modified content in conference peer reviews, part of a broader finding—reported across several records in this theme—that the practice of undisclosed AI use in review is measurable and non-trivial in scale, rather than a hypothetical risk.
The largest single theme in the corpus addresses how AI and human reviewers should divide labor going forward. This theme is characterized by explicitly balanced framing: Perlis et al. (2025) frame their approach around efficiency gains while insisting editors and reviewers keep, in their words, “our hands on the wheel and our eyes on the road,” and Crawford et al. (2024) argue for keeping human flourishing at the center of any AI-augmented publishing model. A similar balance is struck by authors framing the goal explicitly as efficiency gains without sacrificing integrity (Doskaliuk et al., 2025), and the same broadly assistive framing recurs across a wider set of commentaries addressing responsible use in specific research communities and disciplines (Gatrell et al., 2024; Garcia, 2024; Felländer-Tsai & Overgaard, 2023; Kankanhalli, 2024). Combined with the autonomy-position finding above, this theme’s dominance is the strongest evidence of an increasingly consistent tendency toward assistive integration as the field’s dominant approach, without specifying a clear rationale.
Read together, these themes trace a consistent chain from evidence to framework. The integrity and governance themes converge on undisclosed AI use being measurable and already occurring at scale: a gap in enforcement, not in awareness, addressed by the framework’s governance axis (disclosure, human oversight, confidentiality, auditability). The human–AI collaboration theme establishes assistive, human-in-the-loop use as the dominant stated position in the corpus, without a stated rationale; the framework’s task axis is intended to fill that gap by specifying which tasks warrant that consensus and which may tolerate more or less autonomy. Where the literature is more fragmented, for example across disclosure policy, detection methods, and disciplinary audits, the framework organizes these positions along a common task-by-governance grid instead of treating them as independent debates.

4. A Task-Contingent Legitimacy Framework of AI in Peer Review

The model developed below is offered as a conceptual and normative proposition, inferred from and consistent with the patterns identified in this review, rather than as an empirical finding established by it; its claims are stated explicitly as testable propositions later in this section. The findings synthesized in this review point to a common underlying mechanism: the legitimacy of AI in peer review depends not on the technology itself but on the interaction between two factors—the task being performed and the governance surrounding its use. Task type determines the epistemic stakes of AI involvement, whereas governance determines the procedural safeguards, including disclosure, human oversight, confidentiality, and auditability, that make AI use acceptable. Together, these dimensions explain why AI may be appropriate for some peer review tasks but not others. This distinction is consistent with Suchman’s (1995) concepts of cognitive and moral legitimacy. It is also consistent with sociotechnical perspectives that argue technologies cannot be evaluated independently of the institutional contexts in which they operate (Selbst et al., 2019). Rather than treating risk, accountability, and transparency as separate dimensions, the present framework conceptualizes them as components of governance strength because they represent mechanisms through which AI use is regulated. Consequently, the central question is not whether AI should be used in peer review, but which peer review tasks can legitimately incorporate AI under appropriate governance conditions.
The proposed framework differs from adjacent concepts in the literature in a specific way: it does not simply reorganize them. Human-oversight and human-in-the-loop principles specify a governance mechanism but not when that mechanism is sufficient; responsible-AI and trustworthy-AI frameworks articulate broad ethical commitments, fairness, transparency, and accountability, applied uniformly across contexts regardless of task; and risk-based AI governance instruments tier obligations by the severity of potential harm to third parties, not by the epistemic nature of the task being performed. Task-Contingent Legitimacy instead treats task type and governance strength as two conceptually distinct but interacting axes empirically derived from this corpus’s performance data and autonomy positions; these are not principles asserted a priori. Its contribution is therefore a decision structure predicting where, along a task gradient, governance investment yields diminishing versus substantial returns—a relationship the existing frameworks do not specify.
The proposed model (Figure 7) holds that acceptability is jointly determined by both variables rather than by either one alone. Along the task dimension, it declines across a four-tier gradient, from administrative and triage work, through analytical support and evaluative judgment, to fully generative review writing. At every tier, acceptability rises as governance strengthens through disclosure requirements, human-in-the-loop mandates, confidentiality safeguards, and meaningful detection capacity. The two axes interact rather than simply add: robust governance can raise a high-stakes generative task to a moderate level of acceptability, but it cannot make that task as acceptable as a weakly governed administrative use, which is what makes this a genuinely two-dimensional claim rather than two separate one-dimensional ones. Figure 7 presents this model in full.
Two complementary strands of evidence support this framework. The empirical synthesis shows a performance pattern broadly consistent with the task gradient: evaluative-judgment tasks substantially outperform generative and predictive tasks, matching the ordering the model predicts for acceptability, although the empirical task categories (predicting outcomes, generating content, evaluating quality, diagnosing weaknesses) do not map one-to-one onto the model’s four tiers. The corpus contains no direct empirical evidence on analytical support or administrative tasks, an important gap for future research. The autonomy-position analysis independently converges on assistive, human-in-the-loop use as the dominant position, occurring more than ten times as often as full automation and nearly five times as often as outright prohibition. The shift from ChatGPT-focused studies in 2023–2024 to multi-model discussions in 2025–2026 further indicates a broader framing of AI, although it does not directly demonstrate task-contingent reasoning.
The framework has clear policy implications. Journal policies should be tiered rather than binary, scaling disclosure and validation requirements according to task risk instead of adopting blanket permission or prohibition. Detection and enforcement should focus primarily on generative uses, consistent with where the corpus’s detection literature concentrates its attention. Likewise, future validation research should move beyond asking whether AI can review manuscripts in general and instead evaluate specific review tasks, particularly administrative and triage functions that have gone largely unstudied.
The model’s two axes emerge directly from the coded corpus rather than an a priori taxonomy. Task type explains most observed variation in empirical performance, whereas governance strength reflects the corpus’s recurring emphasis on transparency, policy, and detection. Other moderators, including reviewer expertise and manuscript complexity, are plausible but lack sufficient empirical evidence. The proposed framework therefore represents a model that is consistent with the current evidence rather than a claim that these are the only factors influencing AI legitimacy.
The framework also generates three testable propositions. First, perceived legitimacy of AI assistance should decline monotonically from administrative to fully generative tasks, holding governance strength constant. Second, within a given task tier, mandatory disclosure should increase perceived legitimacy more than it reduces AI use, functioning as a legitimacy-repair mechanism rather than a deterrent. Third, reviewer and editor trust in AI-assisted evaluative judgments should be more sensitive to human-in-the-loop requirements than to improvements in model capability.

5. Discussion

This review extends previous studies of AI in peer review in three important ways. First, the bibliometric analysis establishes the structure and rapid growth of the literature, providing the context for a deeper synthesis rather than serving as an end in itself. Second, integrating three independent content-analytic dimensions—theme, editorial stance, and autonomy position—distinguishes what the literature discusses, how it evaluates those issues, and what role it assigns to AI, distinctions that thematic classification alone cannot capture. Third, organizing empirical evidence by peer review task reveals a systematic pattern in AI performance that provides the foundation for the proposed Task-Contingent Legitimacy framework, a task-based alternative to the binary permit versus prohibit policies that currently dominate the debate.
Placing the content-analytic and empirical findings side by side reveals a mismatch between where the literature expresses the greatest acceptance of AI use and where AI performs best. The strongest empirical results come from narrowly defined evaluative tasks, yet much of the literature frames AI acceptability primarily in terms of how generative or high-stakes a task appears rather than demonstrated performance (Perlis et al., 2025; Crawford et al., 2024). Conversely, outcome prediction, the task that could most directly reduce editorial workload, consistently shows the weakest performance in the empirical evidence (accuracy 40 to 67%; correlations as low as ρ = 0.00). The field, in other words, is most cautious about a task AI performs relatively well and comparatively more willing to automate one for which evidence remains weakest.
The boundary between assistive and generative AI is also less stable than the commentary literature often suggests. Although most authors advocate assistive AI under human oversight (Perlis et al., 2025; Gatrell et al., 2024), the empirical evidence already includes examples that extend beyond this boundary. One included study reported that an AI-written discussion section passed blinded peer review at a journal, with five of six reviewers recommending acceptance after revision (Sheridan et al., 2025). Stated positions on AI autonomy therefore do not fully align with demonstrated capability, suggesting that empirical advances may outpace existing governance assumptions.
A second tension concerns the relationship between concern and evidence. Ethics, integrity, and undisclosed AI use account for a substantial proportion of the literature, yet direct empirical evidence of concrete harm remains limited. This imbalance does not imply that the concerns are misplaced, only that discussion has expanded more rapidly than empirical validation. A similar inconsistency appears in the governance literature. Most proposed policies rely on disclosure as the primary safeguard (Munafò, 2024; Perlis et al., 2025), whereas a parallel technical literature has emerged precisely because undisclosed AI use is difficult to detect and appears to be occurring at scale (Liang et al., 2024; Singh Chawla, 2024). The literature therefore promotes disclosure as a governance mechanism while simultaneously documenting its practical limitations. The proposed Task-Contingent Legitimacy framework provides one way to reconcile these tensions by shifting attention from the binary question of whether AI should be used in peer review to identifying which review tasks are appropriate for AI and the governance conditions required for their legitimate use.

6. Limitations and Future Research Directions

6.1. Limitations

Several limitations qualify these conclusions. First, both the corpus and its metadata carry coverage constraints. The search was restricted to Scopus and to a title-field query, which maximizes precision at the cost of recall and likely misses relevant records, particularly outside biomedicine and computer science. Compounding this, a substantial share of the corpus—primarily editorials, letters, and notes—lacks an abstract in Scopus, which constrained the confidence of thematic, stance, and autonomy coding for those records; lower-confidence, title-only classifications are flagged in the accompanying data.
Second, the coding and evidence-synthesis procedures have inherent limits. The coding scheme was implemented through a single combined automated and verified pipeline rather than by multiple independent human coders, so no formal inter-rater reliability statistic (for example, Cohen’s kappa) is reported. Relatedly, the 18 empirical studies synthesized in Table 2 use heterogeneous, non-poolable metrics, so the task-type pattern should be read as directional and consistent rather than as a formal meta-analytic effect. The outcome-prediction studies underpinning the weakest-performance finding also skew toward earlier GPT-3.5/4-era models.
Lastly, the Task-Contingent Legitimacy framework itself is provisional: it is inferred from, and consistent with, the patterns in this corpus, but it has not yet been independently tested against primary data such as surveys or experiments involving reviewers and editors.

6.2. Future Research Directions

These limitations also suggest several directions for future research. Broader title–abstract–keyword searches and multi-database strategies, including Web of Science, could assess the extent to which the present Scopus-only, title-field approach affected corpus composition and could identify additional relevant evidence. Moreover, future coding studies could involve an independent second coder applying the same scheme to a subsample and reporting a formal inter-rater agreement statistic, providing an independent assessment of coding reliability beyond the confidence flags used in the present study. Replicating the outcome-prediction analyses with current-generation models would further clarify whether weaker prediction reflects a persistent limitation of LLMs or is partly attributable to the earlier models used in the available evidence. Finally, the Task-Contingent Legitimacy framework should be subjected to direct empirical testing, for example through surveys or experiments involving reviewers and editors, before it is treated as an empirically established model rather than a theoretically grounded synthesis.

7. Conclusions

Research on AI in peer review has expanded rapidly, from only a handful of publications before 2023 to dozens each year by 2025 and 2026. Despite this rapid growth, empirical evidence remains limited, and the literature has developed in parallel across a large number of journals and disciplines, making an integrated synthesis necessary. The combined evidence indicates that AI is most appropriate as an assistive tool under human oversight, performs best on narrowly defined evaluative tasks, and remains less reliable for predicting publication outcomes. At the same time, recommendations for disclosure coexist with a growing body of research suggesting that undisclosed AI use is already occurring at scale (Singh Chawla, 2024; Zou, 2024; Liang et al., 2024). Future work could extend this synthesis into adjacent areas that fall outside the core corpus, such as AI-assisted reviewer matching (Farber, 2024) and the detection of gender and geographic bias in review outcomes (Sebo, 2024; Verharen, 2023). The proposed Task-Contingent Legitimacy framework, offered here as a conceptual synthesis awaiting empirical validation rather than an established theory, formalizes these findings into a task-based policy framework and advances three testable propositions, shifting the debate from whether AI should be used in peer review to identifying the tasks and governance conditions under which its use is legitimate.

Author Contributions

Conceptualization, E.K.; methodology, E.K.; formal analysis, E.K. and V.S.; investigation, E.K.; data curation, E.K.; validation, E.K. and V.S.; writing—original draft preparation, E.K.; writing—review and editing, E.K. and V.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

All data generated or analyzed during this study are included in this published article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Arabzadeh, N., Ebrahimi, S., Sadeghian, S., Hosseini, S. M., Daqiq, A., Le, H. S., Bashari, M., & Bagheri, E. (2026). Can LLMs uphold research integrity? Evaluating the role of LLMs in peer review quality. In WSDM 2026—Proceedings of the 19th ACM international conference on web search and data mining (pp. 1341–1342). ACM Digital Library. [Google Scholar] [CrossRef] [Scilit]
  2. Bauchner, H., & Rivara, F. P. (2024). Use of artificial intelligence and the future of peer review. Health Affairs Scholar, 2(5), qxae058. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  3. Chang, Y., Li, Z., Zhang, H., Kong, Y., Wu, Y., So, H. K.-H., Guo, Z., Zhu, L., & Wong, N. (2025). TreeReview: A dynamic tree of questions framework for deep and efficient LLM-based scientific peer review. In EMNLP 2025—2025 conference on empirical methods in natural language processing, proceedings of the conference (pp. 15651–15682). Association for Computational Linguistics. [Google Scholar] [CrossRef] [Scilit]
  4. Cheng, K., Sun, Z., Liu, X., Wu, H., & Li, C. (2024). Generative artificial intelligence is infiltrating peer review process. Critical Care, 28(1), 149. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  5. Chu, Z., Ai, Q., Tu, Y., Li, H., & Liu, Y. (2024). Automatic large language model evaluation via peer review. In International conference on information and knowledge management, proceedings (pp. 384–393). Association for Computing Machinery. [Google Scholar] [CrossRef] [Scilit]
  6. Crawford, J., Allen, K.-A., & Lodge, J. (2024). Humanising peer review with artificial intelligence: Paradox or panacea? Journal of University Teaching and Learning Practice, 21(1), 9–16. [Google Scholar] [CrossRef] [Scilit]
  7. Donker, T. (2023). The dangers of using large language models for peer review. The Lancet Infectious Diseases, 23(7), 781. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  8. Doskaliuk, B., Zimba, O., Yessirkepov, M., Klishch, I., & Yatsyshyn, R. (2025). Artificial intelligence in peer review: Enhancing efficiency while preserving integrity. Journal of Korean Medical Science, 40(7), e92. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  9. Farber, S. (2024). Enhancing peer review efficiency: A mixed-methods analysis of artificial intelligence-assisted reviewer selection across academic disciplines. Learned Publishing, 37(4), e1638, (Erratum in 2025, Learned Publishing, 38, e1663). [Google Scholar] [CrossRef] [Scilit]
  10. Felländer-Tsai, L., & Overgaard, S. (2023). Adapting to the rapidly moving target artificial intelligence (AI) in scholarly publishing. Acta Orthopaedica, 94, 625. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  11. Garcia, M. B. (2024). Using AI tools in writing peer review reports: Should academic journals embrace the use of ChatGPT? Annals of Biomedical Engineering, 52(2), 139–140. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  12. Gatrell, C., Muzio, D., Post, C., & Wickert, C. (2024). Here, there and everywhere: On the responsible use of artificial intelligence (AI) in management research and the peer-review process. Journal of Management Studies, 61(3), 739–751. [Google Scholar] [CrossRef] [Scilit]
  13. Hopkins, B. S., Shah, I., Dallas, J., Borja, A. J., Gomez, D., Briggs, R. G., Cote, D. J., Chung, L., Shasby, G., Sisti, J., Rutka, J. T., & Zada, G. (2026). Application of large language and artificial intelligence modeling in the prediction of peer-review outcomes. Journal of Neurosurgery, 144(3), 720–728. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  14. Huang, X., Dou, S., & Yin, Z. (2025). The dual-edged sword: Artificial intelligence’s evolving role in academic peer review. Science China Information Sciences, 68(11), 216101:1–216101:3. [Google Scholar] [CrossRef] [Scilit]
  15. Jin, Y., Zhao, Q., Wang, Y., Chen, H., Zhu, K., Xiao, Y., & Wang, J. (2024). AgentReview: Exploring peer review dynamics with LLM agents. In EMNLP 2024—2024 conference on empirical methods in natural language processing, proceedings of the conference (pp. 1208–1226). Association for Computational Linguistics. [Google Scholar] [CrossRef] [Scilit]
  16. Joachim, M. V., Dodson, T. B., & Laviv, A. (2025). How artificial intelligence differs from humans in peer review. Journal of Oral and Maxillofacial Surgery, 83(8), 1040–1050. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  17. Kadi, G., & Aslaner, M. A. (2024). Exploring ChatGPT’s abilities in medical article writing and peer review. Croatian Medical Journal, 65(2), 93–100. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  18. Kankanhalli, A. (2024). Peer review in the age of generative AI. Journal of the Association for Information Systems, 25(1), 76–84. [Google Scholar] [CrossRef] [Scilit]
  19. Kousha, K., & Thelwall, M. (2024). Artificial intelligence to support publishing and peer review: A summary and review. Learned Publishing, 37(1), 4–12. [Google Scholar] [CrossRef] [Scilit]
  20. Levin, J. M., Oprea, T. I., Davidovich, S., Clozel, T., Overington, J. P., Vanhaelen, Q., Cantor, C. R., Bischof, E., & Zhavoronkov, A. (2020). Artificial intelligence, drug repurposing and peer review. Nature Biotechnology, 38(10), 1127–1131. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  21. Li, J., Li, Y., Hu, X., Gao, M., & Wan, X. (2025). Where do LLMs go wrong? Diagnosing automated peer review via aspect-guided multi-level perturbation. In CIKM 2025—Proceedings of the 34th ACM international conference on information and knowledge management (pp. 1572–1581). Association for Computing Machinery. [Google Scholar] [CrossRef] [Scilit]
  22. Li, Z.-Q., Xu, H.-L., Cao, H.-J., Liu, Z.-L., Fei, Y.-T., & Liu, J.-P. (2024). Use of artificial intelligence in peer review among top 100 medical journals. JAMA Network Open, 7(12), e2448609. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  23. Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., McFarland, D., & Zou, J. Y. (2024). Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. In Proceedings of the 41st international conference on machine learning (ICML 2024), PMLR 235 (pp. 29575–29620). PMLR. [Google Scholar]
  24. Lin, T.-L., Chen, W.-C., Hsiao, T.-F., Liu, H.-I., Yeh, Y.-H., Chan, Y.-K., Lien, W.-S., Kuo, P.-Y., Yu, P. S., & Shuai, H.-H. (2025). Breaking the reviewer: Assessing the vulnerability of large language models in automated peer review under textual adversarial attacks. In EMNLP 2025—2025 conference on empirical methods in natural language processing, findings of EMNLP 2025 (pp. 4819–4839). Association for Computational Linguistics. [Google Scholar] [CrossRef] [Scilit]
  25. Mohamed, A. A., Colome, D., Yang, J., Sargent, E. C., Flores-Milan, G., Sorrentino, Z., Sharma, A., Adogwa, O., Pirris, S., & Lucke-Wold, B. (2025a). Leveraging artificial intelligence in the peer review of neurosurgical research articles. Neurosurgical Review, 48(1), 631. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  26. Mohamed, A. A., Rajendran, S., Colome, D., Sargent, E. C., Schirmer, C. M., Vessell, M., Lucke-Wold, B., Sharma, A., Adogwa, O., & Pirris, S. (2025b). Neurosurgical journals’ policies on artificial intelligence use in manuscript preparation and peer review. Neurosurgical Review, 48(1), 670. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  27. Mrowinski, M. J., Fronczak, P., Fronczak, A., Ausloos, M., & Nedic, O. (2017). Artificial intelligence in peer review: How can evolutionary computation support journal editors? PLoS ONE, 12(9), e0184711. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  28. Munafò, M. (2024). A policy on the use of artificial intelligence and large language models in peer review. Nicotine and Tobacco Research, 26(5), 519. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  29. Nabavi, A., Safari, F., Shmoury, A. H., Tabet, S., Perdomo-Luna, C., & Celi, L. A. (2026). Artificial intelligence in scholarly peer review: A scoping review of applications, risks, and governance challenges. International Journal of Medical Informatics, 214, 106418. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  30. Ning, K.-P., Yang, S., Liu, Y.-Y., Yao, J.-Y., Liu, Z.-H., Tian, Y.-H., Song, Y., & Yuan, L. (2025). PiCO: Peer review in LLMs based on consistency optimization. In Proceedings of the thirteenth international conference on learning representations (ICLR 2025). ICLR. [Google Scholar]
  31. (2026). Peer review in the time of artificial intelligence. Nature Nanotechnology, 21(4), 479. [CrossRef] [Scilit] [PubMed]
  32. Perlis, R. H., Christakis, D. A., Bressler, N. M., Öngür, D., Kendall-Taylor, J., Flanagin, A., & Bibbins-Domingo, K. (2025). Artificial intelligence in peer review. JAMA, 334(17), 1520–1522. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  33. Price, S., & Flach, P. A. (2017). Computational support for academic peer review: A perspective from artificial intelligence. Communications of the ACM, 60(3), 70–79. [Google Scholar] [CrossRef] [Scilit]
  34. Rajakumar, H. K., Sankaran, K. A., Ashok, M. P., & Rachoori, S. (2026). Peer review in the age of artificial intelligence: A comparative study of human and AI-generated review reports. Postgraduate Medical Journal, 102(1208), 551–559. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  35. Rozencwajg, S., & Benhamou, D. (2026). Declare the use or non-use of artificial intelligence in peer-review: A relic of the past? Anaesthesia Critical Care and Pain Medicine, 45(4), 101812. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  36. Saad, A., Jenko, N., Ariyaratne, S., Birch, N., Iyengar, K. P., Davies, A. M., Vaishya, R., & Botchu, R. (2024). Exploring the potential of ChatGPT in the peer review process: An observational study. Diabetes and Metabolic Syndrome: Clinical Research and Reviews, 18(2), 102946. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  37. Sebo, P. (2024). Use of ChatGPT to explore gender and geographic disparities in scientific peer review. Journal of Medical Internet Research, 26, e57667. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  38. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. In FAT* 2019—Proceedings of the 2019 conference on fairness, accountability, and transparency (pp. 59–68). Association for Computing Machinery. [Google Scholar] [CrossRef] [Scilit]
  39. Shen, S. M., Wang, Z., Paul, K., Li, M.-H., Huang, X., & Koizumi, N. (2026). Evaluation of large language models for peer review in transplantation research: Algorithm validation study. JMIR AI, 5, e84322. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  40. Sheridan, G. A., Howard, L. C., Neufeld, M. E., Doyle, T. R., Hughes, A. J., Sculco, P. K., Beverland, D. E., Garbuz, D. S., & Masri, B. A. (2025). Can artificial intelligence generate scientific discussion that passes peer review for publication in a high-impact orthopaedic journal? Irish Journal of Medical Science, 194(4), 1191–1198. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  41. Singh Chawla, D. (2024). Is ChatGPT corrupting peer review? Telltale words hint at AI use. Nature, 628(8008), 483–484. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  42. Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. Academy of Management Review, 20(3), 571–610. [Google Scholar] [CrossRef] [Scilit]
  43. Sun, Z. (2026). Relationship between peer review quality and scientific impact: Insights from LLMs-assessed reviews. Journal of Informetrics, 20(2), 101801. [Google Scholar] [CrossRef] [Scilit]
  44. Tang, Y., Kang, Y., Wu, S., Zhang, R., & Sun, Z. (2026). Can large language models assess the quality of peer review? An empirical study. Scientometrics, 131(4), 2237–2259. [Google Scholar] [CrossRef] [Scilit]
  45. Thakkar, N., Yuksekgonul, M., Silberg, J., Garg, A., Peng, N., Sha, F., Yu, R., Vondrick, C., & Zou, J. (2026). A large-scale randomized study of large language model feedback in peer review. Nature Machine Intelligence, 8(3), 326–336. [Google Scholar] [CrossRef] [Scilit]
  46. Thelwall, M., & Yaghi, A. (2025). Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms. Scientometrics, 130(10), 5285–5307. [Google Scholar] [CrossRef] [Scilit]
  47. Verharen, J. P. H. (2023). ChatGPT identifies gender disparities in scientific peer review. eLife, 12, RP90230. [Google Scholar] [CrossRef] [PubMed]
  48. Vincent-Lamarre, P., & Larivière, V. (2021). Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome. Quantitative Science Studies, 2(2), 662–677. [Google Scholar] [CrossRef] [Scilit]
  49. Von Wedel, D., Schmitt, R. A., Thiele, M., Leuner, R., Shay, D., Redaelli, S., & Schaefer, M. S. (2024). Affiliation bias in peer review of abstracts by a large language model. JAMA, 331(3), 252–253. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  50. Yin, S., Huang, S., Xue, P., Xu, Z., Lian, Z., Ye, C., Ma, S., Liu, M., Hu, Y., Lu, P., & Li, C. (2025). Generative artificial intelligence (GAI) usage guidelines for scholarly publishing: A cross-sectional study of medical journals. BMC Medicine, 23(1), 77. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  51. Zhang, Q., Ning, M., Liu, Z., Huang, Y., Yang, S., Wang, Y., Ye, J., Chen, X., Song, Y., & Yuan, L. (2025). UPME: An unsupervised peer review framework for multimodal large language model evaluation. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 9165–9174). IEEE. [Google Scholar] [CrossRef] [Scilit]
  52. Zou, J. (2024). ChatGPT is transforming peer review—How can we use it responsibly? Nature, 635(8037), 10. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Publications 14 00063 g001
Figure 1. PRISMA flow diagram for study selection. Note. Database: Scopus; 176 records identified, 1 duplicate removed, 175 screened, 13 excluded, 162 included.
Figure 1. PRISMA flow diagram for study selection. Note. Database: Scopus; 176 records identified, 1 duplicate removed, 175 screened, 13 excluded, 162 included.
Publications 14 00063 g001
Publications 14 00063 g002
Figure 2. Included studies by publication year (N = 162). 2026 covers through July 12 only.
Figure 2. Included studies by publication year (N = 162). 2026 covers through July 12 only.
Publications 14 00063 g002
Publications 14 00063 g003
Figure 3. Named AI models by publication period.
Figure 3. Named AI models by publication period.
Publications 14 00063 g003
Publications 14 00063 g004
Figure 4. Thematic distribution of included studies (n = 162).
Figure 4. Thematic distribution of included studies (n = 162).
Publications 14 00063 g004
Publications 14 00063 g005
Figure 5. Editorial stance toward AI in peer review. Note. n = 92 codeable commentary/opinion records, out of 162 included studies; the remaining 70 were not stance-coded.
Figure 5. Editorial stance toward AI in peer review. Note. n = 92 codeable commentary/opinion records, out of 162 included studies; the remaining 70 were not stance-coded.
Publications 14 00063 g005
Publications 14 00063 g006
Figure 6. Stated position on AI autonomy in peer review. Note. n = 120 records with a discernible autonomy position, out of 162 included studies; the remaining 42 lack a discernible position.
Figure 6. Stated position on AI autonomy in peer review. Note. n = 120 records with a discernible autonomy position, out of 162 included studies; the remaining 42 lack a discernible position.
Publications 14 00063 g006
Publications 14 00063 g007
Figure 7. A Task-Contingent Legitimacy framework of AI in peer review: relative acceptability of AI involvement as a function of task tier and governance strength.
Figure 7. A Task-Contingent Legitimacy framework of AI in peer review: relative acceptability of AI involvement as a function of task tier and governance strength.
Publications 14 00063 g007
Table 1. Top 15 most-cited included studies.
Table 1. Top 15 most-cited included studies.
RankTitleYearSourceCitations
1Computational support for academic peer review: A perspective from artificial intelligence (Price & Flach, 2017)2017Communications of the ACM151
2Artificial intelligence to support publishing and peer review: A summary and review (Kousha & Thelwall, 2024)2024Learned Publishing121
3Artificial intelligence, drug repurposing and peer review (Levin et al., 2020)2020Nature Biotechnology64
4Here, there and everywhere: On the responsible use of artificial intelligence (AI) in management research and the peer-review process (Gatrell et al., 2024)2024Journal of Management Studies61
5The dangers of using large language models for peer review (Donker, 2023)2023The Lancet Infectious Diseases60
6Exploring the potential of ChatGPT in the peer review process: An observational study (Saad et al., 2024)2024Diabetes and Metabolic Syndrome: Clinical Research and Reviews56
7Use of artificial intelligence and the future of peer review (Bauchner & Rivara, 2024)2024Health Affairs Scholar52
8Use of artificial intelligence in peer review among top 100 medical journals (Z.-Q. Li et al., 2024)2024JAMA Network Open51
9Artificial intelligence in peer review: How can evolutionary computation support journal editors? (Mrowinski et al., 2017)2017PLoS ONE51
10Using AI tools in writing peer review reports: Should academic journals embrace the use of ChatGPT? (Garcia, 2024)2024Annals of Biomedical Engineering47
11Is ChatGPT corrupting peer review? Telltale words hint at AI use (Singh Chawla, 2024)2024Nature45
12Peer review in the age of generative AI (Kankanhalli, 2024)2024Journal of the Association for Information Systems42
13Artificial intelligence in peer review: Enhancing efficiency while preserving integrity (Doskaliuk et al., 2025)2025Journal of Korean Medical Science41
14Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews (Liang et al., 2024)2024Proceedings of Machine Learning Research35
15Generative artificial intelligence is infiltrating peer review process (Cheng et al., 2024)2024Critical Care35
Note. Citation counts reflect Scopus data as of 12 July 2026, and favor older publications.
Table 2. Empirical findings by task type (n = 18 studies).
Table 2. Empirical findings by task type (n = 18 studies).
CategoryStudyModel(s)SampleFinding
Predicting outcomesAI in neurosurgical manuscript predictionChatGPT-4o, Gemini, Copilot51 preprintsAccuracy 40–66.67%
(p < 0.001)
Predicting outcomesPredicting outcomes from reviewer textFine-tuned GPT-4-mini, GPT-3, BERT, GPT-2, GRUJNS journals, 2021–2023Best AUC 0.91 (fine-tuned); untrained 0.67–0.75
Predicting outcomesChatGPT predictive capacity across platformsChatGPT (30-run avg.)F1000Research, SciPost, ICLRρ = 0.00 to 0.46, platform-dependent
Predicting outcomesLLM evaluation, transplantation research (Shen et al., 2026)5 open-source LLMs200 papersBest: exact-match 0.35, loose-match 0.78
Generating contentAI-generated discussion passes peer reviewChatGPT-41 article, 6 reviewers83% recommended acceptance after revision
Generating contentChatGPT’s abilities in writing and review (Kadi & Aslaner, 2024)ChatGPT 3.0/4.015 reports + 15 articlesMerit score 4.9/10; review weaker than generation
Generating contentChatGPT in peer review, observational (Saad et al., 2024)ChatGPT 3.5/4.0 vs. 2 humans21 articlesAgreement 3.6–3.76/5
Generating contentHuman vs. AI-generated review reportsChatGPT398 reports, 119 articlesAI reviews more routine; human reviews more diverse
Generating contentReview Feedback Agent (randomized)Multi-LLM agent20,000+ reviews, ICLR 202527% of reviewers revised after AI feedback
Generating contentTreeReview, question-tree generation (Chang et al., 2025)LLM-basedICLR/NeurIPS benchmarkOutperforms baselines;
up to 80% reduction in token usage
Evaluating qualityCan LLMs assess peer review quality?GPT-4o, Claude 3.5, Gemini180 eLife papers~70% agreement; Kappa poor-to-moderate
Evaluating qualityUPME (unsupervised MLLM evaluation)Multimodal LLMsMMstar, ScienceQAPearson r = 0.944 and 0.814
Evaluating qualityPRE, LLM evaluation via peer review (Chu et al., 2024)11 LLMs incl. GPT-4Summarization, QA tasksOutperforms baselines; confirms single-LLM bias
Evaluating qualityCan LLMs uphold research integrity? (Arabzadeh et al., 2026)Zero-shot/fine-tuned LLMs vs. MLExpert-annotated benchmarksAdequate for structure; hybrid needed for facts
Diagnosing weaknessesWhere Do LLMs Go Wrong? (perturbation)GPT-4o, Gemini 2.0, LLaMA 3Multi-conference dataMisclassifies flaws; overweights rejection tone
Diagnosing weaknessesBreaking the Reviewer (adversarial)Multiple LLMsAdversarial test setText manipulation distorts assessments
Diagnosing weaknessesAgentReview (bias simulation)LLM agentsSimulated review process37.1% of decision variance from reviewer bias
Diagnosing weaknessesTextual analysis of AI-conference manuscripts (Vincent-Lamarre & Larivière, 2021)N/A (linguistic)AI conference submissionsWord choice predicts acceptance independent of quality
Note. Heterogeneous metrics across studies (accuracy, AUC, ρ, Pearson r); the category-level pattern is directional and consistent, not a pooled effect size. AUC = area under the curve; GRU = gated recurrent unit; ICLR = International Conference on Learning Representations; JNS = Journal of Neurosurgery; MLLM = multimodal large language model; NeurIPS = Conference on Neural Information Processing Systems; QA = question answering; UPME = unsupervised multimodal large language model evaluation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kim, E.; Singh, V. Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework. Publications 2026, 14, 63. https://doi.org/10.3390/publications14040063

AMA Style

Kim E, Singh V. Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework. Publications. 2026; 14(4):63. https://doi.org/10.3390/publications14040063

Chicago/Turabian Style

Kim, Eungi, and Vaishali Singh. 2026. "Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework" Publications 14, no. 4: 63. https://doi.org/10.3390/publications14040063

APA Style

Kim, E., & Singh, V. (2026). Artificial Intelligence in Peer Review: A Bibliometric-Guided Thematic Review and a Task-Contingent Legitimacy Framework. Publications, 14(4), 63. https://doi.org/10.3390/publications14040063

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop