Why Radiomics Rarely Reaches the Clinic: Reproducibility, Validation, and Evidence Gap—A Critical Narrative Review

Abstract

Radiomics has produced tens of thousands of publications yet almost no handcrafted radiomic signatures in routine clinical use, and the reasons are increasingly understood to be problems of reproducibility and clinical translation rather than of algorithms. This critical narrative review argues that the field systematically generates paper-grade evidence—findings sufficient to publish—far faster than decision-grade evidence—findings sufficient to change clinical practice. Drawing on meta-scientific research, we describe seven fragility mechanisms (publication bias, analytical flexibility, underpowering, HARKing [hypothesizing after the results are known], citation distortion, cognitive bias, and misaligned incentives) and show why radiomics is structurally exposed to all of them simultaneously: high-dimensional feature spaces, acquisition-dependent measurement instability, segmentation variability, retrospective single-centre data, small samples, and leakage-prone validation. We then summarise empirical evidence on the radiomics literature, which remains pervaded by suboptimal methodological quality, near-absent negative results, limited external validation, sparse calibration and clinical-utility assessment, low data and code sharing, and a measurable retraction signal. We interpret these patterns as the output of a self-reinforcing system rather than isolated errors, and argue that better algorithms alone cannot resolve them. Finally, we argue that closing this gap requires not better models but evidentiary discipline: the consistent, enforceable application of standards the field already has, and the calibration of published claims to the strength of the underlying evidence.

1. Introduction

Radiomics arose in the early 2010s from quantitative imaging and texture analysis research and was formalised in foundational accounts that reframed medical images as mineable, high-dimensional data [1,2,3]. A PubMed query for “radiomics” [All Fields] (PubMed/MEDLINE, searched May 2026) returned approximately 19,000 indexed records, of which roughly two-thirds were published in 2023 or later [4]. The premise is compelling: extracting hundreds to thousands of quantitative features from routine medical images to capture tumour biology non-invasively [1,2,5]. Each such feature or signature is, in effect, a candidate imaging biomarker, and its route to clinical use runs through the same qualification requirements that govern imaging biomarkers [6].However, this literature has produced virtually no tools in routine clinical use. Deep-learning detection, triage, and quantification systems have entered routine practice, with FDA authorisations of AI-enabled radiological devices exceeding 1000 by the end of 2025 [7]. However, no handcrafted radiomic signature has matched this deployment [8,9,10]. These deployments differ in kind: cleared deep-learning devices cluster in detection, triage, and quantification tasks, whereas handcrafted radiomic signatures are typically prognostic or predictive biomarker models, whose regulatory and deployment pathways diverge accordingly. The question is not whether radiomics works in principle but why a field generating thousands of papers per year has failed to produce decision-grade evidence—predictions robust enough to change clinical behaviour. We argue that this failure is not primarily algorithmic but evidentiary. We trace it to seven interacting fragility mechanisms (Section 3), show why radiomics is structurally exposed to them (Section 4), summarise their empirical signatures (Section 5), and set out the standards required to close the gap (Section 7 and Section 8).

This disconnect is not unique to radiomics; similar patterns appear across high-dimensional biomedical disciplines [11,12,13,14,15]. Meta-scientific research has identified structural mechanisms (publication bias, analytical flexibility, underpowered designs, and weak validation norms) that cause the literature to become populated with individually plausible but collectively unreliable findings [16,17,18,19]. Chalmers and Glasziou [20] famously estimated that a large share—on the order of 85%—of biomedical research investment may be avoidable waste; although the precise figure is debated, it conveys the scale of the concern. Smaldino and McElreath [21] formalised the key dynamic: when career success depends on publication productivity and publication depends on significant results, methods maximising publishability spread through the scientific population even if they degrade reliability. This could lead to literature in which paper-grade evidence can accumulate much faster than decision-grade evidence. We use paper-grade evidence to denote findings sufficient to support the publication of a radiomic association or model but insufficient to support clinical decision-making. Such evidence is typically retrospective, internally evaluated, discrimination-centred, and vulnerable to analytical flexibility. In contrast, decision-grade evidence denotes evidence sufficient to justify a change in clinical behaviour. Operationally, the model or signature must satisfy, at minimum: (i) independent external validation on data not used in development; (ii) explicit assessment and reporting of calibration; (iii) a pre-specified, analytically locked pipeline; (iv) benchmarking against clinically available or standard-of-care predictors; (v) demonstration of incremental value through decision-analytic methods such as net-benefit or decision-curve analysis; and, at implementation, (vi) prospective clinical evaluation [9,22,23,24]. This is a different axis from certainty grading. Schemas such as GRADE [25] assess how certain a body of evidence is about a defined clinical question; the paper-grade/decision-grade distinction asks whether a given model has been subjected to the tests above. The two can diverge: an association may rest on high-certainty evidence and still be paper-grade if its calibration, incremental value, and clinical utility have never been examined. This is not, however, a uniform characterisation. Programmes with standardised multicentric protocols, IBSI-compliant extraction, and independent external validation produce evidence of a substantially higher order than the typical retrospective single-centre study. The mechanisms we describe do not implicate individual researchers: they are properties of the research ecosystem, and they can produce fragile evidence even in the absence of misconduct [21,26].

2. Materials and Methods: Scope, Evidence Selection, and Narrative Synthesis

This review synthesises meta-scientific evidence on the conditions under which radiomics research is generated, validated, and translated into clinical practice. It is a critical narrative synthesis, reported in line with SANRA [27]; it interprets evidentiary mechanisms and their empirical signatures rather than pooling a pre-specified question, and its source selection is purposive rather than exhaustive.

Sources were identified through structured searches of PubMed/MEDLINE and Google Scholar, run through May 2026, combining the core term with methodological and translational qualifiers, and extended by backward and forward citation tracing of key methodological papers, meta-research studies, consensus documents, and reporting guidelines; the full search strategy and representative search strings are given in Supplementary Materials S1. We prioritised five categories of evidence: (i) radiomics-specific meta-research studies; (ii) systematic reviews and methodological audits assessing study quality, validation, reporting, reproducibility, or clinical translation; (iii) empirical studies quantifying specific failure modes such as leakage, acquisition dependence, feature instability, and insufficient sample size; (iv) consensus statements and reporting or appraisal frameworks relevant to radiomics, imaging AI, and clinical prediction modelling; and (v) foundational meta-scientific literature on publication bias, analytical flexibility, underpowered research, citation distortion, cognitive bias, and scientific incentives. Sources were included when they contributed directly to one of three analytic functions: defining a mechanism of evidentiary fragility, documenting an empirical signature of that mechanism in radiomics or imaging AI, or supporting a proposed governance response. Sources representing stronger or more successful translation—standardised multicentric programmes, IBSI-compliant extraction, externally validated signatures, and biomarker-qualification efforts—were deliberately sought and included alongside the critical literature.

3. Meta-Scientific Mechanisms of Evidentiary Fragility

Several mechanisms known from meta-scientific research can make biomedical literatures fragile, describing conditions under which plausible, statistically significant, or technically sophisticated findings may accumulate faster than robust, transportable, clinically useful evidence. They include publication bias [28,29,30,31], analytical flexibility and the garden of forking paths [32,33], underpowering and the winner’s curse [34,35,36], HARKing [37,38,39], citation distortion [40], and cognitive biases including motivated reasoning and confirmation bias [41,42].

These mechanisms are interrelated and can act as a self-reinforcing system (Figure 1). Publication bias rewards positive findings [28,31], which increases the payoff from analytical flexibility [32,33]. High-dimensional pipelines may amplify this: the larger the space of defensible analytic choices, the more paths can lead to a publishable result. Underpowering ensures that any significant finding overestimates the true effect [34,35]. Cognitive mechanisms—motivated reasoning [41], confirmation bias [42], and, in AI-assisted settings, automation bias [43]—guide researchers through the garden of forking paths toward configurations that yield significance, without deliberate deception. The resulting paper reports inflated performance. Citation distortion [40] can propagate these inflated claims; high-performing models are cited without reference to validation status, progressively hardening provisional findings into accepted facts. Finally, incentive structures determine which research behaviours are rewarded. When career advancement, publication success, and institutional prestige depend more on producing positive and novel outputs than on generating negative results, independent replications, or externally validated tools, methods that maximise publishability can spread even if they do not maximise reliability [21,26].

Figure 1. The self-reinforcing engine of fragile evidence. A weak exploratory signal (left) is progressively amplified as it passes through four interacting mechanisms, each shown with the general meta-scientific mechanism (upper label) and its radiomics-specific manifestation (lower label): analytical flexibility and the garden of forking paths, operating on a high-dimensional pipeline space with scanner- and segmentation-dependent measurement; the winner’s curse and data leakage, operating on small retrospective samples (p ≫ n) under leakage-prone internal validation; publication bias, which selects positive results; and citation lock-in, which amplifies positive signatures. The output is paper-grade evidence, which only rarely passes the gate to decision-grade evidence. Misaligned incentives close the loop (bottom), returning the system to further exploratory studies rather than to confirmatory ones. Notation: p ≫ n, indicating that the number of candidate predictors far exceeds the sample size.

4. Why Radiomics Is Structurally Exposed

The general mechanisms described above find an extreme case in radiomics, a field in which every structural vulnerability is simultaneously present. A high-dimensional feature space, physically unstable measurements, predominantly small retrospective monocentric samples, and leakage-prone validation converge to maximise analytical flexibility and structurally weaken evidentiary discipline.

4.1. Combinatorial Analytical Flexibility

A standard radiomic pipeline extracts hundreds to thousands of features from segmented image regions [44,45,46]. Each subsequent step—feature selection, normalisation, classifier choice, and hyperparameter tuning—is individually defensible; in combination, however, these choices generate a vast combinatorial space of plausible workflows. In radiomics, this multiplicity can produce a “vibration of effects,” whereby materially different results emerge from the same data depending on how the analysis is specified. Such flexibility makes unstable or spurious associations increasingly easy to obtain unless the pipeline is rigorously constrained and analytical multiplicity is explicitly controlled [47]. Methodological variants further expand this space: delta-radiomics, for example, multiplies the configuration count by the number of time-points and delta-metric choices [48].

4.2. Acquisition-Dependent Measurement Instability

Radiomic features are computational derivatives of images that are themselves products of complex acquisition and reconstruction chains, and their values inherently depend on the technical conditions under which images are generated, including scanner platform, acquisition protocol, and voxel geometry [49,50,51,52]. Controlled CT experiments have shown that tube current, noise index, and iterative reconstruction levels substantially alter feature reproducibility, especially for texture features that depend on spatial intensity distributions [53]. This technical dependence persists in patient-level data: in a same-patient study of liver metastases reconstructed across multiple dose levels, section thicknesses, kernels, and reconstruction settings, only 11% of tested radiomic features remained reproducible across technical variations, with reconstructed section thickness producing the largest single-parameter effect [54]. Zhu et al. [55] tested 93 features across five CT systems: scan–rescan (test–retest) repeatability was excellent (97.1% repeatable), and intra-system reproducibility across dose levels was high (mean ICC = 0.945), but inter-system reproducibility was near zero (mean ICC = 0.157; 0% of features with ICC > 0.90). Zhang et al. [56] showed that on photon-counting detector CT, 0% of features were robust to slice-thickness changes. A systematic review of 481 studies confirmed that acquisition introduces more feature variability than segmentation, particularly for MRI [57]. Repeatability concerns extend beyond CT and MRI: a test–retest analysis of deep-learning-based PSMA-PET segmentation—cited here as a comparator—documented non-trivial inter-scan variability even when the segmentation algorithm was deterministic [58]. The Image Biomarker Standardisation Initiative (IBSI) Phase 1 [46] and Phase 2 [59] have standardised computational definitions but cannot eliminate the physical dependence on acquisition conditions. Statistical harmonisation methods such as ComBat can reduce inter-site variability [60,61], but their corrective capacity remains incomplete and context-dependent [61,62].

4.3. Segmentation Variability

Radiomic feature extraction often relies on manually delineated regions of interest; inter-reader variability can propagate directly into feature values and their stability, although standardisation recommendations now exist [63,64,65]. A meta-analysis of CT-based machine-learning studies in renal tumours further found that pooled diagnostic performance differed according to phase-selection and manual segmentation strategy, with contour-focused single-phase approaches showing the highest pooled AUC [66]. Automated and semi-automated segmentation, including deep-learning methods, can reduce inter-reader variability but introduces its own reproducibility considerations; even deterministic algorithms can show non-trivial inter-scan variability [58], and feature stability varies across segmentation methods [67].

4.4. Retrospective, Monocentric Data Dominance

In the 2023 NEVER study [68], 95% of 149 sampled publications were retrospective, 75% were single-centre, and 91% used private data; in the 2024 self-reporting meta-research [69], 94% were retrospective and 68% single-centre. Such designs are especially exposed to all preceding vulnerabilities: when development and evaluation data originate from the same institutional environment, they often share scanners, protocols, and local clinical workflows, creating a weak test of generalisability and increasing the risk that models capture centre-dependent regularities rather than transportable biological signal [70,71]. Overlapping or reused patient cohorts have also been documented across radiomics publications, further complicating the independence and cumulative interpretation of the evidence base [72,73].

4.5. Small Samples Relative to Analytic Complexity

Zhong et al. [74] examined 116 radiomics studies from seven leading journals published in 2023: only 9.5% justified their sample size, the median events per predictor parameter (EPP) was 7.5, and, under the Riley et al. criteria [36], only 10.3% had a sufficient training sample size, with a median deficit of 268 patients. Horvat, Papanikolaou, and Koh [75] further noted that fewer than 20 published radiomics studies had incorporated clinical trial data, and that none had prospectively implemented radiomics as a clinical decision-support tool.

4.6. Leakage-Prone Validation

Leakage is not a single error but a family of train–test contamination mechanisms. In radiomics, recurrent forms include: (1) feature selection performed on the full dataset before cross-validation; (2) oversampling procedures such as SMOTE applied before data splitting; (3) harmonisation fitted on the entire dataset rather than within the training loop; (4) preprocessing statistics, including normalisation parameters, estimated using all available observations; and (5) hyperparameter optimisation evaluated through non-nested cross-validation [76,77,78,79,80]. Kapoor and Narayanan [77] documented leakage in at least 294 papers across 17 scientific fields, showing how easily it can generate overoptimistic claims. Within imaging pipelines, Marzi et al. [79] demonstrated that harmonisation before data splitting itself creates leakage and inflates apparent performance, while Gidwani et al. [80] showed that inconsistent partitioning across normalisation, feature selection, hyperparameter selection, and model assessment can markedly idealise radiomic models. The magnitude of the bias is substantial: oversampling leakage inflated AUC by up to +0.343 and produced AUCs as high as 0.90 on random data [78], whereas feature-selection leakage increased AUC-ROC by up to +0.15 across ten public radiomics datasets [76]. Even transparent external validation can expose the fragility of internally optimised signatures: a PET-radiomics model for recurrence-site prediction after head-and-neck re-irradiation declined from a reported balanced accuracy of 84.5% to 70% in an independent cohort, recovering only partially to 78% after cut-off recalibration [81].

4.7. Weak Linkage to Clinical Decision-Making

Calibration assessment and clinical utility evaluation remain consistently underreported in RQS-based appraisals [82,83]. In HPV-status prediction for oropharyngeal cancer, only 5% of studies reported calibration statistics [84]; in MRI-radiomics studies for MGMT promoter methylation prediction in glioma, only 8% did so [72]. In contrast, discrimination-focused reporting was far more common: 89% of HPV studies reported discrimination statistics [84], and all MGMT studies reported AUC or accuracy [72]. However, even a model achieving a high internal AUC offers limited clinically actionable information if its calibration is unreported, its net benefit has not been assessed [85,86], and its performance has not been benchmarked against simpler or clinically available predictors [44].

5. Empirical Signatures of Evidentiary Fragility

Across the literature, each fragility mechanism described above leaves a quantifiable trace. Table 1 collects these signatures alongside the mechanism each one reflects, and Figure 2 pairs them with the pipeline stage at which they arise and the safeguards that address them.

Table 1. Fragility mechanisms and their empirical signatures in radiomics. Rows are organised by fragility mechanism.

Mechanism of Evidentiary Fragility	Expected Ecosystem-Level Signature	Representative Measured Evidence in Radiomics
1. Persistently low and uneven methodological maturity	Formal standards, reporting frameworks, and quality tools expand, yet the average methodological quality of the published literature remains low and varies across radiomics subdomains.	Large-scale RQS evidence remains consistently poor: mean RQS 26.1% across 3258 assessments, with only 7.2% reaching ≥ 50% of the maximum score [83]; median RQS 31% across 1574 publications [82]; delta-radiomics median RQS 25%, with 51.2% of studies scoring < 25% [48]; endometrial MRI radiomics mean RQS 13.77 [87]
2. Positive result selection and publication bias	Null, negative, and non-superior findings are selectively underrepresented, while positive claims dominate the published record and are often insufficiently benchmarked against simpler alternatives.	Positive findings dominate the literature: NEVER found only 1/149 negative studies (0.7%) and no non-radiomic comparator in 44% [68]; glioma radiomics reported positive findings in 26/27 studies (96%) [88]; cardiovascular radiomics showed funnel-plot asymmetry by Egger’s test (z = −2.39, p = 0.017) [89].
3. Leakage-prone analytical flexibility	Researcher degrees of freedom and workflow errors that compromise separation between model development and evaluation inflate apparent performance.	Empirical leakage studies show substantial optimism: feature selection outside cross-validation inflated AUC by up to 0.15 [76]; oversampling before cross-validation biased AUC by up to 0.34, produced AUCs up to 0.90 with random outcomes, and was likely present in 5/34 radiomics papers from 2023 [78]
4. Underpowered model development and winner’s curse	Sample sizes remain too small relative to model complexity, favouring unstable estimates, optimistic effect sizes, and poor transportability.	In 116 binary-outcome radiomics prediction studies, only 11/116 (9.5%) justified sample size, and only 6 included an a priori calculation; median training size was 150, median EPP 7.5, median Riley-based shortfall 268 patients, and only 12/116 (10.3%) met all adequacy criteria [74].
5. External-validation deficit and fragile generalisability	Models are rarely tested on genuinely independent populations; when transported beyond development data, apparent performance commonly attenuates.	External validation was absent in 121/149 NEVER studies (81%) [68]; across 1574 radiomics publications, only 14% reported independent external validation, and 32% lacked any separate validation set [82]. In radiologic deep learning, external validation reduced median AUC by −0.046, with decline in 70/86 algorithms (81%) [90].
6. Acquisition-driven measurement instability	Apparent radiomic signal fails to transport across scanners, platforms, or acquisition settings, indicating vulnerability to non-biological measurement variation.	Acquisition effects markedly impair reproducibility: across five CT systems, 97.1% of features were repeatable on test–retest (scan–rescan), but inter-system reproducibility was poor (mean ICC/CCC 0.157 ± 0.174), with no feature reaching ICC/CCC > 0.90 across systems [55]; on photon-counting detector CT, no features were robust to high-pitch acquisition or slice-thickness changes [56]; and acquisition effects exceeded segmentation effects in a 481-study reliability review [57].
7. Reporting opacity, self-assessment inflation, and weak open-science practice	Methodological errors remain difficult to detect, formal self-audit is rare, and independent verification is constrained by incomplete reporting and limited sharing of code and data.	Transparency remains limited: only 7/117 studies (6%) included a self-reported checklist/quality score [69]; CLEAR adoption was 2%, with self-reported adherence exceeding expert-confirmed adherence by 21 percentage points [91]; among 257 studies, 6% shared data/open datasets, 7% shared code, and 3% shared both [92]; 0/195 empirical radiology articles shared analysis scripts [93].
8. Quality-sensitive heterogeneity in evidence synthesis	Methodological quality is not merely a descriptive deficit: it becomes a measurable source of variation in pooled performance estimates.	Study quality measurably affects pooled evidence: in endometrial MRI radiomics, higher RQS was associated with lower QUADAS-2 risk, more recent publication year, and higher reported performance [87]; and in CT hematoma-expansion deep learning, subgroup analyses showed significant performance differences by segmentation technique and study quality [94].
9. Weak cumulative evidence and stalled clinical translation	Local methodological fragilities accumulate into low certainty evidence at synthesis level and limited progression toward clinically embedded use.	Meta-evidence remains low-certainty: among 53 re-performed radiomics meta-analyses, only 3/53 associations (5.7%) were convincing, and 43/53 (81%) were weak [95]. Fewer than 20 oncologic radiomics studies used clinical-trial data, and no published model had been prospectively implemented as routine clinical decision support [75]; this is consistent with recent analyses describing a widening publication–translation gap [8].

Abbreviations: AUC, area under the receiver operating characteristic curve; CCC, concordance correlation coefficient; EPP, events per predictor parameter; ICC, intraclass correlation coefficient; RQS, Radiomics Quality Score (further abbreviations are defined in the Abbreviations list).

Table 1. Fragility mechanisms and their empirical signatures in radiomics. Rows are organised by fragility mechanism.

Mechanism of Evidentiary Fragility	Expected Ecosystem-Level Signature	Representative Measured Evidence in Radiomics
1. Persistently low and uneven methodological maturity	Formal standards, reporting frameworks, and quality tools expand, yet the average methodological quality of the published literature remains low and varies across radiomics subdomains.	Large-scale RQS evidence remains consistently poor: mean RQS 26.1% across 3258 assessments, with only 7.2% reaching ≥ 50% of the maximum score [83]; median RQS 31% across 1574 publications [82]; delta-radiomics median RQS 25%, with 51.2% of studies scoring < 25% [48]; endometrial MRI radiomics mean RQS 13.77 [87]
2. Positive result selection and publication bias	Null, negative, and non-superior findings are selectively underrepresented, while positive claims dominate the published record and are often insufficiently benchmarked against simpler alternatives.	Positive findings dominate the literature: NEVER found only 1/149 negative studies (0.7%) and no non-radiomic comparator in 44% [68]; glioma radiomics reported positive findings in 26/27 studies (96%) [88]; cardiovascular radiomics showed funnel-plot asymmetry by Egger’s test (z = −2.39, p = 0.017) [89].
3. Leakage-prone analytical flexibility	Researcher degrees of freedom and workflow errors that compromise separation between model development and evaluation inflate apparent performance.	Empirical leakage studies show substantial optimism: feature selection outside cross-validation inflated AUC by up to 0.15 [76]; oversampling before cross-validation biased AUC by up to 0.34, produced AUCs up to 0.90 with random outcomes, and was likely present in 5/34 radiomics papers from 2023 [78]
4. Underpowered model development and winner’s curse	Sample sizes remain too small relative to model complexity, favouring unstable estimates, optimistic effect sizes, and poor transportability.	In 116 binary-outcome radiomics prediction studies, only 11/116 (9.5%) justified sample size, and only 6 included an a priori calculation; median training size was 150, median EPP 7.5, median Riley-based shortfall 268 patients, and only 12/116 (10.3%) met all adequacy criteria [74].
5. External-validation deficit and fragile generalisability	Models are rarely tested on genuinely independent populations; when transported beyond development data, apparent performance commonly attenuates.	External validation was absent in 121/149 NEVER studies (81%) [68]; across 1574 radiomics publications, only 14% reported independent external validation, and 32% lacked any separate validation set [82]. In radiologic deep learning, external validation reduced median AUC by −0.046, with decline in 70/86 algorithms (81%) [90].
6. Acquisition-driven measurement instability	Apparent radiomic signal fails to transport across scanners, platforms, or acquisition settings, indicating vulnerability to non-biological measurement variation.	Acquisition effects markedly impair reproducibility: across five CT systems, 97.1% of features were repeatable on test–retest (scan–rescan), but inter-system reproducibility was poor (mean ICC/CCC 0.157 ± 0.174), with no feature reaching ICC/CCC > 0.90 across systems [55]; on photon-counting detector CT, no features were robust to high-pitch acquisition or slice-thickness changes [56]; and acquisition effects exceeded segmentation effects in a 481-study reliability review [57].
7. Reporting opacity, self-assessment inflation, and weak open-science practice	Methodological errors remain difficult to detect, formal self-audit is rare, and independent verification is constrained by incomplete reporting and limited sharing of code and data.	Transparency remains limited: only 7/117 studies (6%) included a self-reported checklist/quality score [69]; CLEAR adoption was 2%, with self-reported adherence exceeding expert-confirmed adherence by 21 percentage points [91]; among 257 studies, 6% shared data/open datasets, 7% shared code, and 3% shared both [92]; 0/195 empirical radiology articles shared analysis scripts [93].
8. Quality-sensitive heterogeneity in evidence synthesis	Methodological quality is not merely a descriptive deficit: it becomes a measurable source of variation in pooled performance estimates.	Study quality measurably affects pooled evidence: in endometrial MRI radiomics, higher RQS was associated with lower QUADAS-2 risk, more recent publication year, and higher reported performance [87]; and in CT hematoma-expansion deep learning, subgroup analyses showed significant performance differences by segmentation technique and study quality [94].
9. Weak cumulative evidence and stalled clinical translation	Local methodological fragilities accumulate into low certainty evidence at synthesis level and limited progression toward clinically embedded use.	Meta-evidence remains low-certainty: among 53 re-performed radiomics meta-analyses, only 3/53 associations (5.7%) were convincing, and 43/53 (81%) were weak [95]. Fewer than 20 oncologic radiomics studies used clinical-trial data, and no published model had been prospectively implemented as routine clinical decision support [75]; this is consistent with recent analyses describing a widening publication–translation gap [8].

Abbreviations: AUC, area under the receiver operating characteristic curve; CCC, concordance correlation coefficient; EPP, events per predictor parameter; ICC, intraclass correlation coefficient; RQS, Radiomics Quality Score (further abbreviations are defined in the Abbreviations list).

Figure 2. Fragility signatures, safeguards, and standards across the radiomics pipeline. Each of five sequential pipeline stages (centre) is paired with the empirical signature of paper-grade practice observed in the literature (above) and with the safeguards and standards that would address it (below): measurement and reporting—low, uneven methodological quality against IBSI Phases 1–2, CLEAR/CLEAR-E3, and CT-protocol harmonisation; design and power—underpowered, retrospective, single-centre designs against formal sample size calculation for prediction models, with study-level appraisal by RQS 2.0 and METRICS; model development—leakage-inflated internal performance against nested cross-validation, PROBAST + AI, and TRIPOD + AI; validation and utility—discrimination-centred evaluation that is rarely externally tested against independent external validation, calibration, decision-curve analysis, and DECIDE-AI; and open science and robustness—opaque and analytically fragile practice against FAIR code and data, preregistration and Registered Reports, TOP, and multiverse analysis. Values are indicative of the signatures reported in Table 1; the corresponding standards are set out in full in Table 2.

Figure 2. Fragility signatures, safeguards, and standards across the radiomics pipeline. Each of five sequential pipeline stages (centre) is paired with the empirical signature of paper-grade practice observed in the literature (above) and with the safeguards and standards that would address it (below): measurement and reporting—low, uneven methodological quality against IBSI Phases 1–2, CLEAR/CLEAR-E3, and CT-protocol harmonisation; design and power—underpowered, retrospective, single-centre designs against formal sample size calculation for prediction models, with study-level appraisal by RQS 2.0 and METRICS; model development—leakage-inflated internal performance against nested cross-validation, PROBAST + AI, and TRIPOD + AI; validation and utility—discrimination-centred evaluation that is rarely externally tested against independent external validation, calibration, decision-curve analysis, and DECIDE-AI; and open science and robustness—opaque and analytically fragile practice against FAIR code and data, preregistration and Registered Reports, TOP, and multiverse analysis. Values are indicative of the signatures reported in Table 1; the corresponding standards are set out in full in Table 2.

Table 2. Translational pipeline stages: vulnerabilities, minimum requirements, and standards. Rows follow the translational pipeline sequence.

Translational Stage	Principal Vulnerability/Failure Mode	Minimum Operational Requirement for a Non-Exploratory Clinical Claim	Primary Standard(s), Guideline(s), or Framework(s)	Empirical Evidence that the Requirement Remains Insufficiently Met
1. Study conception	Analytical flexibility, post hoc hypothesis shaping, selective reporting	Pre-specify the clinical question, population, endpoint, candidate predictors, analysis plan, validation strategy, and primary performance metrics	Preregistration [96]; Registered Reports [97]; TOP guidelines [98]	In broader meta-research (predominantly psychology), positive findings were reported in 44% of Registered Reports versus 96% of standard publication models [97,99]
2. Imaging measurement, feature extraction, and software traceability	Acquisition-, preprocessing-, filter-, and software-dependent feature instability	Use IBSI-compliant definitions; report acquisition, reconstruction, preprocessing, interpolation, discretisation, filters, extraction software, and software version	IBSI Phase 1, 2 [46,59]; documented open implementation: PyRadiomics [100]	Across photon-counting and dual-energy CT systems, mean inter-system ICC was 0.157, and no feature reached ICC > 0.90 at matched dose [55]. Feature values differed across radiomics software implementations [101]. Vendor-dependent quantitative CT differences were also reported outside radiomics [102].
3. Sample size adequacy and study positioning	Underpowered model development; unstable estimates; inflated apparent performance	Provide a formal sample size justification using prediction-model criteria	Riley et al. [36]	Sample size justification was absent in 90.5% of studies; only 10.3% met strict Riley-based criteria; median shortfall: 268 patients [74].
4. Model development and leakage-safe internal validation	Data leakage; optimistic bias; non-nested feature selection; misuse of resampling	Nest preprocessing, feature selection, resampling, hyperparameter tuning, and model selection within training folds; use leakage-safe internal validation.	Radiomic-signature safeguards [103]; nested cross-validation principles [104]; PROBAST + AI [105].	AUC inflation reached +0.34 when oversampling preceded cross-validation [78] and +0.15 when feature selection was applied before cross-validation [76].
5. Reporting transparency of radiomics and AI methods	Incomplete or non-verifiable pipeline reporting	Report item-by-item against the framework appropriate to the study scope: CLEAR, CLEAR-E3, TRIPOD + AI, and CLAIM where applicable	CLEAR [44]; CLEAR-E3 [106]; TRIPOD + AI [23]; CLAIM 2024 [107]	Only 7/117 radiomics papers (6%) included a self-reported reporting checklist or quality-scoring document [69]. CLEAR adoption reached 2%; self-reported versus expert-confirmed adherence was 91% vs. 66% (mean gap, 21 percentage points) [91].
6. Methodological appraisal	Conflation of reporting quality, methodological quality, and translational maturity	Use structured tools for methodological appraisal and translational readiness, rather than relying on discrimination metrics or narrative claims alone	METRICS [45]; RQS 2.0 and Radiomics Readiness Levels [22]	Median RQS was 31% of the maximum across 1574 publications [82]. In a 2025 diagnostic-accuracy synthesis, study quality assessed with METRICS emerged as a significant source of between-study differences in subgroup analyses [94].
7. Open science and computational reproducibility	Unavailable code; inaccessible datasets; non-reproducible computational workflows	Share code and data where feasible; otherwise, state access restrictions and provide sufficient computational detail for independent re-analysis	FAIR principles [108]; TOP guidelines [98]	In 257 radiomics papers published in leading journals, only 6% shared data, and 7% shared code [92]. Private data were used in 91% of papers in the NEVER study [68] and 89% in a separate meta-research sample [69]. In broader radiology and nuclear medicine AI, only 1/161 private-data studies shared the dataset [109].
8. External validation and transportability	Internal-only evidence; poor transportability across institutions, scanners, and protocols	Perform external validation on independent data; distinguish internal, internal–external, and external validation	Validation hierarchy and clinical prediction-model guidance [110,111]	External validation was absent in 81% [68] and 79% [69] of radiomic studies; only 14% of 1574 publications included external validation [82].
9. Calibration, clinical benchmarking, and incremental value	Discrimination-only evaluation; absent calibration; untested incremental clinical value	Report calibration alongside discrimination; compare against clinical, non-radiomic, or standard-of-care baselines; quantify added value	Calibration principles [85,112]; CLEAR comparison requirements [44]	Calibration was reported in 1/19 HPV-prediction studies [84] and 2/26 MGMT-prediction studies [72]. 44% of radiomics studies included no non-radiomic comparator [68]. In prostate radiotherapy, adding MRI radiomics increased the C-index from 0.69 to 0.70 over a clinical model [113].
10. Early clinical evaluation and trial-level evidence	Retrospective performance claims substituted for early clinical evaluation; incomplete protocol and trial reporting	Evaluate live clinical performance, safety, workflow effects, and human-factor consequences before trial-level claims; use AI-specific reporting extensions for interventional protocols and trial reports	DECIDE-AI [24]; SPIRIT-AI [114]; CONSORT-AI [115]	In oncology AI, median SPIRIT-AI concordance was 78.2% across 12 RCT protocols [116]; median combined CONSORT 2010/CONSORT-AI concordance was 82% across 57 RCT reports [117]
11. Deployment governance, regulatory readiness, and real-world translation	Clinical-readiness claims without deployability, lifecycle governance, or regulatory fitness	Address trustworthiness, robustness, fairness, explainability, traceability, post-deployment monitoring, and applicable regulatory requirements before claiming clinical readiness	FUTURE-AI [118]; GMLP/IMDRF [119]; applicable medical-device regulation, including EU MDR where relevant [120]	The research–clinical translation gap has been described as widening [8]. Routine oncologic implementation remains limited [9,73]. Progress in cost-effectiveness analysis is minimal or insignificant across radiomics studies [83].

Abbreviations: AUC, area under the receiver operating characteristic curve; CCC, concordance correlation coefficient; EPP, events per predictor parameter; ICC, intraclass correlation coefficient; RQS, Radiomics Quality Score (further abbreviations are defined in the Abbreviations list).

Table 2. Translational pipeline stages: vulnerabilities, minimum requirements, and standards. Rows follow the translational pipeline sequence.

Translational Stage	Principal Vulnerability/Failure Mode	Minimum Operational Requirement for a Non-Exploratory Clinical Claim	Primary Standard(s), Guideline(s), or Framework(s)	Empirical Evidence that the Requirement Remains Insufficiently Met
1. Study conception	Analytical flexibility, post hoc hypothesis shaping, selective reporting	Pre-specify the clinical question, population, endpoint, candidate predictors, analysis plan, validation strategy, and primary performance metrics	Preregistration [96]; Registered Reports [97]; TOP guidelines [98]	In broader meta-research (predominantly psychology), positive findings were reported in 44% of Registered Reports versus 96% of standard publication models [97,99]
2. Imaging measurement, feature extraction, and software traceability	Acquisition-, preprocessing-, filter-, and software-dependent feature instability	Use IBSI-compliant definitions; report acquisition, reconstruction, preprocessing, interpolation, discretisation, filters, extraction software, and software version	IBSI Phase 1, 2 [46,59]; documented open implementation: PyRadiomics [100]	Across photon-counting and dual-energy CT systems, mean inter-system ICC was 0.157, and no feature reached ICC > 0.90 at matched dose [55]. Feature values differed across radiomics software implementations [101]. Vendor-dependent quantitative CT differences were also reported outside radiomics [102].
3. Sample size adequacy and study positioning	Underpowered model development; unstable estimates; inflated apparent performance	Provide a formal sample size justification using prediction-model criteria	Riley et al. [36]	Sample size justification was absent in 90.5% of studies; only 10.3% met strict Riley-based criteria; median shortfall: 268 patients [74].
4. Model development and leakage-safe internal validation	Data leakage; optimistic bias; non-nested feature selection; misuse of resampling	Nest preprocessing, feature selection, resampling, hyperparameter tuning, and model selection within training folds; use leakage-safe internal validation.	Radiomic-signature safeguards [103]; nested cross-validation principles [104]; PROBAST + AI [105].	AUC inflation reached +0.34 when oversampling preceded cross-validation [78] and +0.15 when feature selection was applied before cross-validation [76].
5. Reporting transparency of radiomics and AI methods	Incomplete or non-verifiable pipeline reporting	Report item-by-item against the framework appropriate to the study scope: CLEAR, CLEAR-E3, TRIPOD + AI, and CLAIM where applicable	CLEAR [44]; CLEAR-E3 [106]; TRIPOD + AI [23]; CLAIM 2024 [107]	Only 7/117 radiomics papers (6%) included a self-reported reporting checklist or quality-scoring document [69]. CLEAR adoption reached 2%; self-reported versus expert-confirmed adherence was 91% vs. 66% (mean gap, 21 percentage points) [91].
6. Methodological appraisal	Conflation of reporting quality, methodological quality, and translational maturity	Use structured tools for methodological appraisal and translational readiness, rather than relying on discrimination metrics or narrative claims alone	METRICS [45]; RQS 2.0 and Radiomics Readiness Levels [22]	Median RQS was 31% of the maximum across 1574 publications [82]. In a 2025 diagnostic-accuracy synthesis, study quality assessed with METRICS emerged as a significant source of between-study differences in subgroup analyses [94].
7. Open science and computational reproducibility	Unavailable code; inaccessible datasets; non-reproducible computational workflows	Share code and data where feasible; otherwise, state access restrictions and provide sufficient computational detail for independent re-analysis	FAIR principles [108]; TOP guidelines [98]	In 257 radiomics papers published in leading journals, only 6% shared data, and 7% shared code [92]. Private data were used in 91% of papers in the NEVER study [68] and 89% in a separate meta-research sample [69]. In broader radiology and nuclear medicine AI, only 1/161 private-data studies shared the dataset [109].
8. External validation and transportability	Internal-only evidence; poor transportability across institutions, scanners, and protocols	Perform external validation on independent data; distinguish internal, internal–external, and external validation	Validation hierarchy and clinical prediction-model guidance [110,111]	External validation was absent in 81% [68] and 79% [69] of radiomic studies; only 14% of 1574 publications included external validation [82].
9. Calibration, clinical benchmarking, and incremental value	Discrimination-only evaluation; absent calibration; untested incremental clinical value	Report calibration alongside discrimination; compare against clinical, non-radiomic, or standard-of-care baselines; quantify added value	Calibration principles [85,112]; CLEAR comparison requirements [44]	Calibration was reported in 1/19 HPV-prediction studies [84] and 2/26 MGMT-prediction studies [72]. 44% of radiomics studies included no non-radiomic comparator [68]. In prostate radiotherapy, adding MRI radiomics increased the C-index from 0.69 to 0.70 over a clinical model [113].
10. Early clinical evaluation and trial-level evidence	Retrospective performance claims substituted for early clinical evaluation; incomplete protocol and trial reporting	Evaluate live clinical performance, safety, workflow effects, and human-factor consequences before trial-level claims; use AI-specific reporting extensions for interventional protocols and trial reports	DECIDE-AI [24]; SPIRIT-AI [114]; CONSORT-AI [115]	In oncology AI, median SPIRIT-AI concordance was 78.2% across 12 RCT protocols [116]; median combined CONSORT 2010/CONSORT-AI concordance was 82% across 57 RCT reports [117]
11. Deployment governance, regulatory readiness, and real-world translation	Clinical-readiness claims without deployability, lifecycle governance, or regulatory fitness	Address trustworthiness, robustness, fairness, explainability, traceability, post-deployment monitoring, and applicable regulatory requirements before claiming clinical readiness	FUTURE-AI [118]; GMLP/IMDRF [119]; applicable medical-device regulation, including EU MDR where relevant [120]	The research–clinical translation gap has been described as widening [8]. Routine oncologic implementation remains limited [9,73]. Progress in cost-effectiveness analysis is minimal or insignificant across radiomics studies [83].

Abbreviations: AUC, area under the receiver operating characteristic curve; CCC, concordance correlation coefficient; EPP, events per predictor parameter; ICC, intraclass correlation coefficient; RQS, Radiomics Quality Score (further abbreviations are defined in the Abbreviations list).

5.1. Quality, Publication Bias, and Validation

Two large 2024–2025 evidence syntheses converge on the finding that radiomics study quality has improved over time but remains persistently low overall [82,83]. These concerns had already been documented by Park et al. in 2020 [121] and were subsequently reinforced by Spadarella et al. [122] in 2023. Using the Radiomics Quality Score (RQS), Kocak et al. [82] found a median RQS of 31% across 1574 unique publications, with a strong positive temporal trend (Kendall’s tau = 0.908, p < 0.001); and Barry et al. [83] reported a mean RQS of 26.1% ± 17.8% across 3258 RQS assessments (with only 7.2% reaching ≥ 50% of the maximum score). Complementary METRICS (METhodological RadiomICs Score [45], with an explanation-and-elaboration companion [123]) audits show that methodological quality remains, at best, moderate across several subfields: prostate MRI, 52% [124]; cardiac CT/MRI, 54.5% [89]; and glioma radiomics, 57% [88]. The cardiovascular METRICS audit also illustrates why reported performance should not be equated with evidentiary robustness. Despite a pooled AUC of 0.81, Cavallo et al. reported high heterogeneity, statistically significant funnel-plot asymmetry, and only moderate average methodological quality, with 9 papers eligible for the meta-analysis [89]. Direct evidence of positive-result bias comes from the NEVER meta-research study; only 1 of 149 radiomics articles published in Q1 clinical radiology journals reported negative results (0.7%; rounded to 1% by the authors) [68]. Consistently, a glioma radiomics synthesis found that 26 of 27 studies (96%) reported positive effects, which the authors interpreted as evidence of high non-statistical publication bias [88]. Across two independent radiomics meta-research samples, external validation was absent in 81% (121/149) [68] and 79% (93/117) [69] of studies. In the broader radiologic deep-learning literature (invoked here as a comparator to handcrafted radiomics), externally validated algorithms also frequently lose performance. Yu et al. [90] found at least some external performance decrease in 70 of 86 algorithms (81%). Zhong et al. [95] re-performed 53 meta-analyses; only 3 of 53 (5.7%) reached a convincing level of evidence, whereas 43 of 53 (81%) were rated as weak. A recent deep-learning diagnostic synthesis likewise paired QUADAS-2 with METRICS in its quality appraisal; subgroup analyses showed significant performance differences according to study quality, indicating that methodological rigor can materially influence pooled estimates [94].

5.2. Reporting Transparency

Transparency remains exceptional rather than routine in radiomics research. Among 117 radiomics papers, only 7 (6%) reported the use of any checklist or quality-scoring instrument [69], and adoption of the CLEAR (CheckList for EvaluAtion of Radiomics research) guideline across eligible literature has reached only 2% [91]. In 257 radiomics studies published in leading radiology journals, 16 (6%) shared data or used publicly available datasets, 20 (7%) shared code, and only 7 (3%) did both [92]; in the broader radiology and nuclear medicine AI literature, just 1 of 161 private datasets was made available [109]. Reproducibility is further constrained by software dependence; nominally equivalent radiomic features can differ materially across extraction packages because of discrepancies in preprocessing and implementation [101], a broader problem in quantitative imaging, as illustrated for instance by reported cross-platform differences in cardiac CT measurements [102]. The IBSI framework [46,59] provides the technical basis for comparability through consensus definitions, reference values, and benchmarking; however, reporting and transparency practices have not kept pace [91].

5.3. Benchmarks and Translation

The translational deficit is also reflected in how radiomics claims are benchmarked. In a meta-research analysis of radiomics studies published in leading radiology journals, 44% made no comparison with non-radiomic approaches [68]. In treatment-response radiomics for non-small-cell lung cancer, comparison with the current gold standard was absent in all but two studies [125]. Even when direct comparisons are performed, incremental value is often limited or absent: in prostate radiotherapy, adding MRI radiomics increased the C-index only from 0.69 to 0.70 relative to a clinical-only model [113]; in advanced melanoma, CT radiomics failed to improve on a simpler clinical model for predicting benefit from checkpoint inhibitors [126]; and in locally advanced rectal cancer, MRI-based radiomic models showed no definite added value over clinical models for predicting pathological complete response after neoadjuvant chemoradiotherapy [127]. The problem, therefore, is not simply that radiomics has been slow to reach clinical practice, but that much of the literature still stops short of establishing why practice should change. This is the translational gap described by Kocak, Pinto dos Santos, and Dietzel [8], and it persists despite measurable improvements in formal study quality: Barry et al. showed that the dimensions most consequential for clinical adoption remain among the least developed [83]. RQS 2.0 and its radiomics readiness levels were introduced precisely to make that gap visible, shifting appraisal from isolated methodological adequacy to staged translational maturity [22,128].

5.4. Retractions as an Ecosystem Signal

Retractions provide a complementary ecosystem-level indicator of evidentiary fragility in radiomics. Demircioğlu identified 93 retracted radiomics publications across six databases, corresponding to an estimated mean retraction rate of 6.7 per 10,000 publications—a rate that, in absolute terms, is comparable to or below several biomedical baselines and is therefore best read as a weak, indirect signal. Among the 20 cases examined in detail, 11 retraction notices (55%) did not clearly distinguish misconduct from error or assign responsibility, and no major radiological or oncological journal appeared to have retracted a radiomics publication [129]. Against a background of persistently low code- and data-sharing rates, which hinder independent scrutiny of published work [92], this pattern suggests that formal retractions may underestimate the broader burden of problematic radiomics studies [129].

6. From Recurrent Patterns to Systemic Interpretation

Section 3, Section 4 and Section 5 described fragility mechanisms, their structural amplification in radiomics, and their empirical signatures. We interpret these not as independent issues, but as the coupled components of a self-reinforcing system (Figure 1). Where Ioannidis [17] distinguishes useful from merely publishable research, Huang et al. [9] specify the criteria for clinical translation, and RQS 2.0 [22] grades study-level readiness. The paper-grade/decision-grade distinction locates the gap in the interaction between fragility mechanisms and the publication ecosystem rather than in any single study.

Paper-Grade Evidence as the Product of a Self-Reinforcing System

Radiomics is structurally exposed because the vulnerabilities described above do not merely coexist; high dimensionality and analytical flexibility, acquisition- and segmentation-dependent measurement, retrospective single-centre designs, small samples, leakage-prone workflows, and limited sharing of code and data tend to co-occur and may jointly lower the threshold for exploratory findings to become publishable before they become clinically reliable [3,59,67,74,76,78,83,92,130,131,132,133,134,135].

Radiomic pipelines create many defensible analytical routes; preprocessing, discretisation, harmonisation, feature selection, classifier choice, hyperparameter tuning, and validation design can each be varied in ways that may appear individually reasonable. In such a setting, cognitive mechanisms such as motivated reasoning and confirmation bias may influence which routes are explored, retained, or interpreted as most compelling, without requiring deliberate misconduct [41,42]. When positive or apparently high-performing results are more likely to enter the visible literature, this flexibility can favour the accumulation of attractive findings over the accumulation of robust evidence, as illustrated in radiomics by the scarcity of negative studies [68].

Once published, these findings can acquire cumulative force, as provisional radiomic claims are transmitted through a literature already skewed toward positive findings rather than through a balanced evidentiary record. Broader meta-research shows that positive or statistically significant findings are preferentially cited, while citation networks can further transform qualified claims into apparent authority [40,136,137]. In the radiomics literature already enriched for positive results [68], this may create a secondary amplification layer; internally promising models can become part of the cumulative narrative before their external validity, calibration, and clinical utility have been adequately established. The same process is reinforced by institutional and publication incentives. Systems that reward novel, positive, technically sophisticated, and publishable outputs more strongly than correction, replication, external validation, or negative evidence can select for research behaviours that maximise publication success even when they do not maximise reliability [21,26].

The result is not literature of false claims, but literature in which paper-grade evidence can become visible, citable, and cumulatively influential before the harder tests required for decision-grade evidence have been satisfied [21,40,41,42,68]. Paper-grade evidence—typically retrospective, internally validated, and discrimination-centred—may justify publication but not clinical action. Decision-grade evidence requires independent external validation [9,75], adequate calibration [85], comparison with clinical alternatives, and incremental value demonstrated through decision-analytic evaluation [86,138]. Claim–evidence alignment should discipline the language of radiomics translation: exploratory studies should remain explicitly exploratory; internally validated models should be framed as candidates for external testing; generalisability should not be claimed without external validation; and clinical usefulness should not be claimed without calibration, benchmarking against clinically available alternatives, incremental value, and decision-analytic assessment of clinical utility—otherwise limited evidence is converted into overstatement, as described in the literature on biomedical spin, diagnostic test overinterpretation, and unsupported AI performance claims [139,140,141].

7. Standards and Safeguards Across the Radiomics Pipeline

Because the fragility described above is a property of the ecosystem rather than of any algorithm, the response is methodological discipline applied sequentially across the radiomics pipeline—from measurement and model development to validation and open science (Figure 2; Table 2).

7.1. Measurement, Reporting, and Statistical Planning

Stable measurement is the first requirement for credible radiomics. IBSI’s original feature-standardisation effort established consensus definitions and reference values for 169 radiomic features [46], while the subsequent filter-standardisation initiative provided reference implementations and verification resources for commonly used convolutional filters [59]. Measurement stability also depends on upstream image acquisition. Although not radiomics-specific, professional society efforts to harmonise CT protocols, such as the SIRM position papers [142,143], address an acquisition layer that is directly relevant to radiomic feature robustness, given the well-documented sensitivity of CT radiomics to scanner, reconstruction, slice-thickness, and discretisation choices [50,51,52]. Reporting standards are equally essential. The CLEAR guideline requires authors to report the software, version, feature definitions, segmentation and preprocessing details, and extraction parameters needed to assess reproducibility [44]; CLEAR-E3 provides worked explanations and examples for applying these requirements consistently [106]. Statistical planning must then match the effective complexity of the model development process. Riley et al. [36] provide practical methods for sample size calculation in prediction model development, and radiomics analyses should accordingly align candidate model complexity with the information content of the available data.

Radiomics ultimately produces candidate imaging biomarkers, whose clinical acceptance is judged within biomarker qualification frameworks. The FDA-NIH BEST resource defines biomarker categories and the goal of a qualified biomarker for a defined context of use [6], while the RSNA Quantitative Imaging Biomarkers Alliance (QIBA) issues Profiles that specify technical performance claims and conformance requirements for quantitative imaging biomarkers—for example, the DSC-MRI Profile for gliomas [144]. IBSI standardises upstream feature definitions [46,59], whereas these frameworks address the metrological and regulatory layer, which handcrafted radiomics has rarely engaged. Table 3 summarises the principal appraisal, reporting, and standardisation instruments and their differing scopes.

Table 3. Appraisal, reporting, and standardisation instruments relevant to radiomics.

Instrument	Function	Scope	Body	Ref.
IBSI (Phase 1–2)	Measurement standardisation	Radiomic features	IBSI	[46,59]
CLEAR (+CLEAR-E3)	Reporting checklist	Radiomics studies	ESR/EuSoMII	[44,106]
METRICS (+METRICS-E3)	Methodological-quality score	Radiomics studies	EuSoMII	[45,123]
RQS 2.0 + RRL	Quality + translational readiness	Radiomics studies	RQS group	[22]
TRIPOD + AI	Reporting guideline	Prediction models (regression/ML)	TRIPOD/EQUATOR	[23]
PROBAST + AI	Risk-of-bias & applicability	Prediction models (regression/ML)	PROBAST group	[105]
CLAIM (2024)	Reporting checklist	AI in medical imaging	Radiology: AI	[107]
QIBA Profiles	Metrological performance claims & conformance	Quantitative imaging biomarkers	RSNA QIBA	[145]
BEST	Biomarker taxonomy & qualification	Imaging/non-imaging biomarkers	FDA–NIH	[6]

7.2. Leakage-Free Model Development

Data-dependent operations must be confined to the training process and re-estimated independently within each resampling fold or validation split. Radiomics-specific studies have quantified the inflation caused by feature selection leakage [76] and by oversampling before cross-validation [78], while broader machine-learning methodology has documented leakage as a pervasive source of irreproducible scientific claims [77]. Leakage can arise at multiple stages of the pipeline, including preprocessing, feature selection, resampling, harmonisation, and model or hyperparameter selection. The radiomics literature already provides direct empirical evidence for some of these mechanisms [76]; separate work in medical imaging has shown that harmonisation performed before data splitting can also induce leakage and inflate downstream performance estimates [79]. Nested cross-validation helps separate model tuning from model evaluation [104]. PROBAST + AI [105] and TRIPOD + AI [23] should then be used to assess risk of bias and to ensure complete reporting of the resulting prediction model study.

7.3. Validation Hierarchy, Calibration, and Clinical Utility

Independent external validation is the minimum requirement for claims of generalisability beyond the source data, and prospective clinical evaluation becomes increasingly important as tools approach implementation [110,111]. For systems entering early live clinical evaluation, DECIDE-AI provides stage-specific reporting guidance tailored to AI-based decision-support studies [24]. Discrimination alone is insufficient for clinical credibility. For studies making clinical applicability claims, calibration should be assessed and reported explicitly [85,112], and decision-analytic utility should be evaluated through net-benefit approaches such as decision-curve analysis [86,146]. These model-centred assessments should be complemented by clinically meaningful and patient-centred outcome measures, which remain less frequently foregrounded in the evaluation literature on AI in radiology [147].

7.4. Open Science and Preregistration

Reproducibility requires preservation and, where feasible, sharing of the analytic materials needed to interrogate a radiomics claim: code, extraction configurations, model specifications, and reusable data resources [92,98,108]. The current literature remains far from this standard. In two independent radiomics meta-research samples, 91% and 89% of studies relied on private data [68,69], while a dedicated audit of sharing practices identified limited availability of models, code, and datasets as a major obstacle to clinical translation [92]. Preregistration addresses a complementary vulnerability: the blurring of confirmatory and exploratory analysis. By fixing hypotheses and analysis plans before outcomes are inspected, it helps distinguish prediction from postdiction and limits the scope for selective analytical adaptation [96,148].

7.5. Testing Robustness to Analytic Flexibility

Reporting standards document the analytic pipeline that was chosen, but they cannot establish whether a result reflects a stable signal or only one path through the space of defensible choices. Multiverse and specification curve analyses make that dependence measurable; the model is re-estimated across the full set of reasonable specifications (discretisation, harmonisation, feature selection, and classifier), and the distribution of results is reported in place of a single pipeline [149], while the specification curve orders these fits to show how far the signature depends on any individual choice [150]. This converts the vibration of effects from an unmeasured threat into a reported quantity; a result stable across this space is credible, while one that emerges only under a narrow configuration might be an artefact of analytic flexibility.

7.6. The Shifting Technical Frontier: Foundation Models and Generative AI

Self-supervised imaging foundation models are the most credible technical answer to handcrafted radiomics’ small samples and weak transportability [151], yet they relocate rather than remove the problem, concentrating capability and raising validation and governance challenges that, by their developers’ own account, strain current medical AI evaluation [152]. Generative methods are similarly double-edged; synthetic-data augmentation and harmonisation can enlarge scarce datasets and attenuate scanner effects [153], but in controlled phantom data image-level generative harmonisation improved appearance while reducing radiomic feature stability [154], so such steps must be validated on the downstream radiomic endpoint, not assumed beneficial.

8. Implications for the Field: From Available Standards to Routine Adoption

Although a substantial methodological infrastructure now exists [155,156,157] and formal quality has improved, its uptake remains marginal (Section 5.2). This reflects not the absence of standards but an incentive asymmetry. Methodological safeguards impose immediate costs on individual investigators (time, expertise, larger samples, and reduced analytical flexibility), while their benefits accrue collectively, through more reliable cumulative knowledge. The result is a collective action problem, mapping onto the distinction between the rewards of getting work published and those of getting it right [158]; as long as publication and advancement track productivity and positive findings, the privately rational choice diverges from the collectively useful one. Compliance will not become routine until methodological rigour is made consequential at the point of publication and evaluation [96,158].

Building on this incentive logic [158], reform should follow four design principles: interventions should be structural rather than exhortatory, composable rather than monolithic, enforceable rather than voluntary, and incentive-compatible rather than virtue-demanding. These principles are proposals, not validated remedies. Their effect on radiomics translation, as distinct from reporting quality, has not been tested, and each can fail in predictable ways. Entry requirements may favour groups already equipped to meet them, and mandates may produce formal rather than substantive compliance—a pattern already suggested by the divergence between self-reported and expert-confirmed CLEAR adherence [91]. They are accordingly best assessed against the outcome indicators set out below.

The principles translate into distinct responsibilities. Authors should pre-specify analytic plans, report against CLEAR, share code and data where feasible, and calibrate conclusions to the evidentiary tier actually reached [96,98,106,108,148]. Reviewers should apply PROBAST + AI and METRICS and assess whether a study’s claims match that tier [45,105]. At the editorial level, screening submissions against minimum methodological requirements before peer review, and weighing whether a study’s claims are commensurate with the evidence it attains, would serve the same end [22]. Registered Reports are the most promising structural lever for confirmatory work. In psychology, the only field where the format has been evaluated at scale, they reduced positive first hypothesis results from 96% of standard reports to 44% [97,99]—an effect not yet tested in radiomics, but one that shows how decoupling publication from result direction reshapes the visible evidence base. Funders should require data-sharing and, where appropriate, preregistration as conditions of award [96,98,108], and institutions should reward rigour and transparency over venue prestige, in line with DORA [159]. Progress should be tracked against concrete indicators: a rising share of published negative results, higher median quality scores, routine external validation, and greater use of Registered Reports for confirmatory studies.

9. Limitations and Boundary Conditions

Several limitations bound the claims of this review. By design it is a critical narrative synthesis with purposive, question-driven source selection rather than a systematic, bias-controlled survey; it is therefore itself exposed to the selection effects it describes, and its conclusions are interpretive rather than quantitative. The supporting evidence is also of uneven provenance: some is direct, such as the controlled leakage-inflation experiments; some is indirect, such as the retraction signal; and some is imported by analogy from deep-learning and other high-dimensional fields. The evidence base is also concentrated, as much of the appraisal infrastructure underlying the quality critique is derived from a small number of investigators and groups; common provenance may therefore carry common assumptions, and the literature’s apparent convergence rests on a comparatively narrow foundation. In addition, most of the empirical signatures are snapshots of the 2020–2025 period, whereas most of the standards against which they are read (CLEAR, METRICS, TRIPOD + AI, RQS 2.0) postdate much of the literature they appraise. Part of the observed under-adoption is therefore temporal, and this review necessarily describes the field with a delay, understating improvements already under way.

Moreover, robust work at the frontier of the field is not in dispute. Where formal certainty grading has been applied, it finds real signal. An overview that re-performed 53 radiomics meta-analyses graded three associations as convincing and seven as highly suggestive [95], and an umbrella review of artificial intelligence in cancer imaging rated roughly one-third of pooled estimates as moderate certainty [160]. Methodological quality is also rising over time [82,83], and a recent field review reaches the same diagnosis as this one; reproducibility and external validation, not algorithmic novelty, are the rate-limiting steps for clinical adoption, though standardisation efforts are beginning to address them [161]. Decision-grade radiomics therefore exists—produced by standardised, externally validated programmes that sit well above the typical retrospective single-centre study.

Finally, the entire translation gap is not attributable to evidentiary fragility. Imaging biomarker translation is intrinsically slow and capital-intensive, and its regulatory frameworks are still being consolidated. The European Union Artificial Intelligence Act phases in obligations for high-risk medical AI through 2026–2027 [162], and the United States Food and Drug Administration finalised its predetermined change control guidance only in late 2024 [163]. Limited data-sharing is often driven by privacy and governance constraints rather than reluctance, and for some clinical endpoints, the incremental value of radiomics over established predictors is genuinely marginal, itself a substantive finding and not a failure of effort.

10. Conclusions

Radiomics does not suffer primarily from a shortage of algorithms or features; it suffers from an evidentiary architecture in which paper-grade findings accumulate faster than the decision-grade evidence required to change clinical practice. The mechanisms responsible—analytical flexibility, measurement instability, underpowered and single-centre designs, leakage-prone validation, selective publication, and incentive misalignment—are mutually reinforcing, which is why purely technical fixes have not closed the translational gap. The constructive implication is that the field already possesses most of the standards it needs; what is missing are the composable, enforceable, and incentive-compatible mechanisms that make their sequential application routine and consequential, together with the editorial norm that claims be matched to the evidentiary tier actually achieved. Realigning publication with reliability is the path from paper-grade to decision-grade radiomics.