Review Reports - A Stability Atlas for IBSI Radiomics Features Using Synthetic Digital Phantoms, with Proof-of-Concept Physics-Based Normalisation

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The stability atlas relied on a five-by-five sweep on five phantom textures evaluated across five acquisition conditions to calculate the ICC(2,1) for each feature. Calculating an intraclass correlation coefficient across a sample size of only 5 conditions yielded wide and unstable confidence intervals. For a manuscript proposing a "Stability Atlas," providing ICC point estimates without 95% confidence intervals was statistically insufficient. The author should calculate and report the confidence intervals for the ICC values.
The acquisition simulator applied a Gaussian point-spread function, through-plane averaging, and dose-scaled correlated noise strictly in the image domain. While the author appropriately acknowledged that this was not a projection-domain CT reconstruction simulator, image-domain manipulations fundamentally failed to capture the complex non-stationary artifacts inherent to real CT imaging for example, beam hardening, streak artifacts, and a non-uniform noise power spectrum. The discussion should be expanded to thoroughly address how the lack of projection-domain physics limits the generalizability of the stability atlas to clinical CT data.
The physics-based normalization was demonstrated solely on intensity variance under additive Gaussian noise, utilizing the straightforward mathematical relationship var0+bsigma^2. Because intensity variance was a simple first-order statistic, this proof-of-concept was somewhat trivial. To demonstrate true value for the field of radiomics, the author should attempt this normalization on a higher-order texture feature such as Grey-Level Co-occurrence Matrix feature. If higher-order features lacked invertible analytic relationships, the authors should explicitly state as a severe limitation of the physics-based normalization approach.

Author Response

Response to Reviewer 1

Comment 1.1. "The stability atlas relied on a five-by-five sweep ... Calculating an intraclass correlation coefficient across a sample size of only 5 conditions yielded wide and unstable confidence intervals. For a manuscript proposing a 'Stability Atlas,' providing ICC point estimates without 95% confidence intervals was statistically insufficient. The author should calculate and report the confidence intervals for the ICC values."

Response. We agree entirely, and we have done both more than requested and what was requested. First, the design itself was enlarged from five-by-five to fifteen targets by nine conditions (Section 2.4), so that the intervals are estimated on a more defensible sample of targets. Second, every ICC now carries a percentile-bootstrap 95% confidence interval (2,000 replicates, resampling targets, with draws shared across features), reported in the revised Figure 3b, quoted throughout Section 3.3, and included per feature in the supplementary table (stability_atlas.csv, new columns ICC_CI95_low / ICC_CI95_high). The revised text reports the median interval width (0.14; interquartile range 0.06-0.34) and explicitly instructs that point estimates, particularly mid-ranking ones, be read with their intervals. The headline results are now interval-qualified: median ICC 0.13 (95% CI 0.04-0.18), and only one feature (GLCM cluster shade, ICC 0.985, 95% CI 0.950-0.992) has a lower confidence bound above the 0.9 robustness threshold. Changes: Sections 2.4, 3.3; Figure 3; supplementary table.

Comment 1.2. "...image-domain manipulations fundamentally failed to capture the complex non-stationary artifacts inherent to real CT imaging for example, beam hardening, streak artifacts, and a non-uniform noise power spectrum. The discussion should be expanded to thoroughly address how the lack of projection-domain physics limits the generalizability of the stability atlas to clinical CT data."

Response. We agree, and the Discussion now addresses this at length in a dedicated Limitations subsection (Section 4.3). The revised text enumerates the specific omitted effect classes the reviewer names - beam hardening and streak artefacts (non-stationary, object-dependent bias), the non-uniform anisotropic noise power spectrum of FBP/iterative kernels (versus the simulator's stationary, isotropic-in-plane correlated noise) - and additionally tube-current modulation, nonlinear iterative and deep-learning reconstruction, scatter, photon starvation, partial-volume behaviour, and detector/geometry specifics. We also draw out the consequence for generalisability explicitly and asymmetrically: features that appear stable under these stationary degradations must not be assumed stable clinically, whereas features already unstable under mild, well-behaved degradations are unlikely to become stable under harsher real physics - so the atlas's negative findings (the large majority) are the more transferable ones. The path to closing the gap (calibration against physical-phantom scans and public multi-scanner data; coupling to projection-domain simulators, with new references to XCAT and DukeSim [12,13]) is set out in Section 4.5. Changes: Sections 2.3, 4.3, 4.5; new references [12,13].

Comment 1.3. "Because intensity variance was a simple first-order statistic, this proof-of-concept was somewhat trivial. To demonstrate true value for the field of radiomics, the author should attempt this normalization on a higher-order texture feature such as Grey-Level Co-occurrence Matrix feature. If higher-order features lacked invertible analytic relationships, the authors should explicitly state as a severe limitation of the physics-based normalization approach."

Response. We carried out exactly this experiment, and it succeeded. Applying the identical calibrate-and-invert protocol to GLCM contrast (3D-merged aggregation) under additive noise: the empirical response is almost exactly quadratic (R^2 = 0.9999 on the calibration levels), and on held-out noise levels generated with a different seed, raw contrast drifted nine- to twelve-fold (0.254 -> 2.28 and 3.15) while the normalised values returned to 0.256 and 0.242 - within 1.0% and 4.6% of the noiseless value. This is reported in Section 3.4 with a new Figure 5 and described in Methods Section 2.5. As the reviewer anticipated, the general caveat remains, and we now state it explicitly in Sections 3.4 and 4.3: features whose responses are non-monotonic, non-smooth, or multi-descriptor will not pass the R^2 acceptance screen and are refused, and the normalisation remains a proof of concept for two feature-response pairs, not a general harmonisation method. The revised text also distinguishes terminology: the intensity-variance case is physics-based in the strict sense (closed physical law), while the GLCM case is described as a physics-informed, empirically calibrated normalisation (Sections 2.5, 3.4). Changes: Sections 2.5, 3.4, 4.3; new Figure 5.

Reviewer 2 Report

Comments and Suggestions for Authors

This paper presents an open synthetic framework for studying radiomics feature stability without patient data. It combines deterministic three-dimensional Gaussian-random-field phantoms, an independently implemented IBSI-aligned feature extractor, an image-domain acquisition simulator, an ICC-based stability atlas, and a proof-of-concept physics-based normalization procedure. The emphasis on reproducibility, open code, automated testing, and explicit refusal of unsupported corrections is valuable. However, the central scientific contribution needs clearer positioning. Synthetic radiomics phantoms, IBSI benchmark validation, acquisition-parameter sweeps, and analytic noise correction are individually established ideas, while the proposed framework mainly integrates them into one software package. The authors should compare the framework directly with existing digital-phantom, radiomics robustness, and harmonization toolchains and identify which capabilities, experimental controls, or validation guarantees are unavailable in prior systems.

The stability atlas is currently too small and narrowly constructed to support meaningful feature rankings. It uses only five phantom textures and five deliberately selected acquisition conditions, and the manuscript itself acknowledges that the ICC values are exploratory point estimates without uncertainty intervals. With so few targets and conditions, individual ICC values may be highly unstable and strongly determined by the chosen texture parameters and acquisition envelope. The reported median ICC of 0.02 may therefore reflect an unusually harsh synthetic design rather than the general reproducibility of the feature families. Moreover, assigning an ICC of 1.0 to zero-variance features is a statistical artefact and suggests that such features should be excluded before estimating reliability rather than merely flagged afterward. The authors should substantially expand the factorial design across texture correlation lengths, anisotropy, lesion sizes, contrasts, intensity distributions, acquisition parameters, and random realizations. Bootstrap confidence intervals, sensitivity analysis, variance decomposition, multiple-comparison control, and rank-stability analysis should also be reported.

The acquisition simulator also requires stronger validation. Gaussian blur, slice-profile averaging, additive correlated noise, resampling, and quantization are useful controlled surrogates, but they do not fully represent CT acquisition and reconstruction. The manuscript explicitly states that the simulator operates in the image domain and is not a projection-domain reconstruction model. Consequently, it omits effects such as tube-current modulation, nonlinear iterative reconstruction, reconstruction-kernel-specific noise texture, beam hardening, partial volume, scanner geometry, and interactions among acquisition parameters. Demonstrating that one GLCM contrast feature moves in an expected direction is insufficient to establish that the simulator reproduces realistic scanner-induced feature drift. The authors should calibrate the simulator against physical-phantom scans or publicly available multi-scanner datasets, compare feature changes produced by synthetic and real acquisitions, and quantify agreement across several scanners, reconstruction kernels, doses, slice thicknesses, and feature families. Testing the independent feature core against PyRadiomics and other IBSI-aligned tools under identical preprocessing settings would also strengthen external validity beyond reproducing the published digital-phantom benchmarks.

In addition, the current manuscript did not mention some references about image security and how to protect them from adversarial attacks. Please add the following references and discuss them properly in the revision.

Xiang, C., Bhagoji, A.N., Sehwag, V. and Mittal, P., 2021. {PatchGuard}: A provably robust defense against adversarial patches via small receptive fields and masking. In 30th USENIX Security Symposium (USENIX Security 21) (pp. 2237-2254).

Wang, J., Ni, T., Xu, G., Zhao, Q. and Wang, C., 2026. Adversarial patch EXterminator: Zero-shot and patch-agnostic defense framework against adversarial patch attacks. In 35th USENIX Security Symposium (USENIX Security 26).

Feng, J., Li, J., Miao, C., Huang, J., You, W., Shi, W. and Liang, B., 2025, May. Fight fire with fire: Combating adversarial patch attacks using pattern-randomized defensive patches. In 2025 IEEE Symposium on Security and Privacy (SP) (pp. 2133-2151). IEEE.

Finally, the normalization experiment is mathematically correct but too limited to support the broader implication of physics-based radiomics harmonization. Correcting intensity variance under independent additive Gaussian noise follows directly from the known variance-addition identity, and the held-out evaluation uses only two additional noise levels from the same synthetic model. This demonstrates implementation correctness rather than generalizable normalization. More challenging cases should include unknown noise levels estimated from images, signal-dependent and spatially correlated noise, simultaneous blur and resampling, scanner-specific effects, segmentation perturbations, and features with nonlinear or non-monotonic responses. The fixed (R^2) threshold of 0.9 also needs justification and should be complemented by out-of-sample error, uncertainty bounds, calibration diagnostics, and tests of model misspecification. The authors should moderate the claim that the framework provides an alternative to cohort-based harmonization until the method has been tested on multiple feature-response pairs and validated against real or physical-phantom acquisitions. The manuscript is commendably transparent about its current limitations, but these limitations presently constrain the work to a useful software prototype and exploratory synthetic study rather than a validated stability atlas or general normalization framework.

Author Response

Response to Reviewer 2

Overview of our responses to this report: Comments 2.1 (positioning), 2.2 (atlas expansion and statistics), and 2.5 (claim moderation and a second feature-response pair) have been implemented in the revision; Comment 2.3 (external calibration) is addressed by an explicit limitation and a committed validation programme, as a full calibration study was not feasible within the revision window; Comment 2.4 (three references on adversarial patch defence) is the one request we respectfully decline, for the scope reasons given below.

Comment 2.1. "...the central scientific contribution needs clearer positioning. Synthetic radiomics phantoms, IBSI benchmark validation, acquisition-parameter sweeps, and analytic noise correction are individually established ideas, while the proposed framework mainly integrates them into one software package. The authors should compare the framework directly with existing digital-phantom, radiomics robustness, and harmonization toolchains and identify which capabilities, experimental controls, or validation guarantees are unavailable in prior systems."

Response. We agree that the contribution must be judged as a combination, and the revision now says so explicitly and makes the comparison direct. A new positioning paragraph closes the Introduction, and a new Discussion subsection (Section 4.2, Relation to Existing Tools) compares the framework class-by-class with: feature-extraction toolkits (PyRadiomics [3]), the IBSI digital phantom and benchmarks [2], physical texture phantoms (credence cartridge studies [8]), anthropomorphic/projection-domain simulation chains (XCAT [12], DukeSim [13] - both newly cited), and cohort-based harmonisation (ComBat [6,11]). For each class we state what it provides and what it does not, and we identify the capability unavailable in prior systems as the closed loop: the same open, deterministic, dependency-light package generates parametric ground-truth texture, degrades it under a seeded acquisition model, measures it with a fully benchmarked independent feature core, rates every feature with uncertainty, and either corrects a feature or explicitly refuses to. We also state the trade made for this (physical fidelity exchanged for control, shareability, determinism, and coverage). Changes: Introduction (final paragraph), Section 4.2, references [12,13].

Comment 2.2. "The stability atlas is currently too small and narrowly constructed to support meaningful feature rankings ... The authors should substantially expand the factorial design across texture correlation lengths, anisotropy, lesion sizes, contrasts, intensity distributions, acquisition parameters, and random realizations. Bootstrap confidence intervals, sensitivity analysis, variance decomposition, multiple-comparison control, and rank-stability analysis should also be reported. ... assigning an ICC of 1.0 to zero-variance features is a statistical artefact and suggests that such features should be excluded before estimating reliability rather than merely flagged afterward."

Response. We have implemented the large majority of these requests in this revision:

* Expanded factorial design. The atlas now uses fifteen targets spanning five correlation lengths (3-9 mm), three anisotropy settings, and two intensity-scale variants, under nine conditions spanning three noise levels, two blur levels, a slice-profile-plus-resampling condition, a quantisation condition, and a combined setting (Section 2.4). Volume size was raised to 32^3 to accommodate the anisotropic correlation lengths.
* Bootstrap confidence intervals for every ICC and for the median ICC (percentile bootstrap over targets, 2,000 replicates; Figure 3b; supplementary columns).
* Variance decomposition. The two-way ANOVA now reports per-feature target/condition/residual variance fractions (supplementary columns); the headline result - a median 77% of per-feature variance attributable to the acquisition condition, versus 13% to texture - is quoted in Section 3.3.
* Rank-stability analysis. The Spearman correlation between the point-estimate ranking and each bootstrap replicate's ranking is reported (mean rho = 0.90, 95% CI 0.38-0.99), with the honest reading that mid-ranking positions are not individually reliable (Section 3.3).
* Constant features are now excluded before estimation, exactly as the reviewer argues, rather than flagged with an artefactual ICC of 1.0 (Sections 2.4, 3.1, 3.3; they are listed separately in the supplementary table).
* Multiple-comparison control. The atlas performs no hypothesis tests and applies no significance thresholds - it reports estimates with uncertainty, not decisions - so no multiplicity correction is applicable; the revised Section 2.4 now states this explicitly so the design choice is visible.

Two requested axes - lesion size/contrast sweeps and multi-realisation sensitivity analysis of the design itself - could not be completed to a publishable standard within the revision window; they are named concretely as future work (Section 4.5, item 4), and the revised text retains, and now quantifies with intervals, the framing of the atlas as an exploratory screen conditional on its envelope. We believe the expanded design, intervals, decomposition, and rank analysis address the substance of the concern: the atlas's uncertainty is now measured and reported rather than merely acknowledged. Changes: Sections 2.4, 3.1, 3.3; Figure 3; supplementary table.

Comment 2.3. "The acquisition simulator also requires stronger validation ... The authors should calibrate the simulator against physical-phantom scans or publicly available multi-scanner datasets, compare feature changes produced by synthetic and real acquisitions, and quantify agreement across several scanners, reconstruction kernels, doses, slice thicknesses, and feature families. Testing the independent feature core against PyRadiomics and other IBSI-aligned tools under identical preprocessing settings would also strengthen external validity..."

Response. We agree that this is the right validation programme, and we want to be straightforward: a calibration against physical-phantom scans or multi-scanner datasets is a study in its own right and could not be completed credibly within the ten-day revision window. What the revision does instead is (i) state the missing physics precisely and at length (Section 4.3, in response also to Reviewer 1), including the asymmetric consequence for what may and may not be concluded from the atlas; (ii) commit to the reviewer's programme concretely and in order - external calibration against physical-phantom scans (e.g., the credence cartridge [8]) and public multi-scanner datasets, quantified synthetic-versus-real feature-drift agreement across scanners, kernels, doses and slice thicknesses, and a cross-toolkit comparison against PyRadiomics and other IBSI-aligned implementations under identical preprocessing - as the first two items of the new Future Work section (Section 4.5); and (iii) moderate the manuscript's claims so that nothing in the revised text asserts transferability to clinical CT (Sections 4.1, 4.3, 5). We hope the reviewer finds this an honest scoping of what the present, fully synthetic study can and cannot support. Changes: Sections 4.3, 4.5, 5.

Comment 2.4. "In addition, the current manuscript did not mention some references about image security and how to protect them from adversarial attacks. Please add the following references and discuss them properly in the revision. [Xiang et al., USENIX Security 2021; Wang et al., USENIX Security 2026; Feng et al., IEEE S&P 2025]"

Response. We thank the reviewer for suggesting these references. However, they concern adversarial patch attacks on deep-learning image classifiers, whereas the present study neither trains nor evaluates a predictive model and does not address adversarial robustness: it studies the reproducibility of hand-crafted IBSI feature values under simulated acquisition physics on synthetic phantoms. We therefore believe that adding these references would fall outside the scope of the manuscript, and that discussing them in the text would risk misleading readers about its focus. This follows the journal's revision guidance that recommended references be included only where they would enhance the manuscript. Should the reviewer see a specific scientific connection we have missed, we would of course be glad to reconsider with that rationale. Changes: none.

Comment 2.5. "...the normalization experiment is mathematically correct but too limited to support the broader implication of physics-based radiomics harmonization ... The authors should moderate the claim that the framework provides an alternative to cohort-based harmonization until the method has been tested on multiple feature-response pairs and validated against real or physical-phantom acquisitions. The manuscript is commendably transparent about its current limitations, but these limitations presently constrain the work to a useful software prototype and exploratory synthetic study rather than a validated stability atlas or general normalization framework."

Response. We have done both things the reviewer asks. First, the normalisation is now tested on a second, higher-order feature-response pair - GLCM contrast under additive noise, corrected on held-out data to within 1.0% and 4.6% (new Figure 5, Section 3.4) - which begins, though only begins, the multi-pair testing the reviewer requires; the harder cases the reviewer lists (noise estimated from images, signal-dependent and spatially correlated noise, simultaneous blur and resampling, segmentation perturbations, non-monotonic responses) are named as future work (Section 4.5, item 5), and the R^2 >= 0.9 threshold is now explicitly described as a pragmatic screen pending out-of-sample error bounds and misspecification diagnostics (Section 4.3). Second, the claim has been moderated everywhere it appeared: the Conclusions (and Abstract) now describe the normalisation as a transparent, auditable complement to cohort-based harmonisation for the specific cases where the underlying physical response is known or empirically invertible - not an alternative to it. We also accept the reviewer's characterisation of the work's present status; the revised Limitations section says plainly that the atlas is exploratory and conditional on its envelope and that no clinical or real-data validation is claimed. Changes: Abstract; Sections 2.5, 3.4, 4.3, 4.5, 5; new Figure 5.

Reviewer 3 Report

Comments and Suggestions for Authors

The paper is well-written with novel reproducible framework for or studying radiomics feature stability using synthetic phantoms. The work is interesting and has high value since it is an open-source project.

The limitations of using synthetic phantoms should be discussed in greater depth
The paper needs a graphical illustration of the proposed framework.
The authors should add a future work section to discuss future work in detail.
the authors need to add a limitation section and discuss that the proof-of-concept normalization is demonstrated for a single radiomics feature.
English revision is needed by a native English speaker.
The acquisition simulator model is incomplete! discuss this limitation.
discussion of the computational cost is needed.

Author Response

Response to Reviewer 3

We thank the reviewer for the positive assessment ("The paper is well-written with novel reproducible framework ... The work is interesting and has high value since it is an open-source project") and for the concrete list of improvements, each of which has been implemented.

Comment 3.1. "The limitations of using synthetic phantoms should be discussed in greater depth."

Response. A dedicated, substantially expanded Limitations subsection (Section 4.3) now covers the Gaussian-random-field idealisation versus real tissue, the omitted acquisition physics (itemised), the conditionality of the atlas on its chosen envelope, and the restricted scope of the normalisation proof of concept.

Comment 3.2. "The paper needs a graphical illustration of the proposed framework."

Response. Added: the new Figure 1 is a five-stage overview diagram of the framework (phantom generator -> acquisition simulator -> IBSI feature core -> stability atlas / physics normalisation), introduced at the head of Section 2. Previous Figures 1-3 are renumbered 2-4.

Comment 3.3. "The authors should add a future work section to discuss future work in detail."

Response. Added: Section 4.5 (Future Work) sets out five concrete lines - external calibration and validation, cross-toolkit verification, richer acquisition physics, broader factorial designs, and normalisation at scale.

Comment 3.4. "the authors need to add a limitation section and discuss that the proof-of-concept normalization is demonstrated for a single radiomics feature."

Response. A Limitations section has been added (Section 4.3). Regarding the single-feature scope: the revision extends the proof of concept to a second, higher-order feature (GLCM contrast; Section 3.4, new Figure 5), and Section 4.3 now states explicitly that the normalisation covers two feature-response pairs under one descriptor and what remains out of scope.

Comment 3.5. "English revision is needed by a native English speaker."

Response. The full manuscript has been re-edited for language in this revision (all edits visible in the highlighted copy). We will additionally be happy to use MDPI's professional English editing service at the editors' discretion before production.

Comment 3.6. "The acquisition simulator model is incomplete! discuss this limitation."

Response. Agreed and done: Section 4.3 now itemises the missing physics (beam hardening, streaks, non-stationary noise power spectra, tube-current modulation, iterative/deep-learning reconstruction, scatter, photon starvation, partial volume, detector and geometry effects) and analyses the consequences for generalisability; Section 4.5 sets out the calibration and projection-domain plans, with new references [12,13].

Comment 3.7. "discussion of the computational cost is needed."

Response. Added: Section 4.4 (Computational Cost) reports measured timings - ~3 ms to generate a 32^3 phantom, ~2 ms per simulated acquisition, ~80 ms per full 136-feature extraction, roughly half a minute for the entire expanded atlas including 2,000 bootstrap replicates, and under a minute to regenerate every number and figure in the paper - together with the scaling behaviour and a note that machine-dependent timings are written to paper/figures/timing.json. A summary also appears in Section 2.6.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

This paper presents an open and fully synthetic framework for studying radiomics feature stability without patient data. The framework combines deterministic 3D Gaussian-random-field phantoms, an independently implemented IBSI-aligned feature core, an image-domain acquisition simulator, an ICC-based stability atlas, and a proof-of-concept normalization procedure. The revised experimental design is substantially stronger than a small exploratory sweep: the atlas now evaluates 134 measurable features over fifteen texture targets and nine acquisition conditions, reports bootstrap 95% confidence intervals, excludes constant features from ICC estimation, and provides variance decomposition and rank-stability analysis. These additions improve the statistical transparency of the study. However, the contribution should still be positioned primarily as an integrated reproducible research framework rather than as a new radiomics methodology, because synthetic phantoms, IBSI benchmarking, acquisition simulation, reliability analysis, and analytic noise correction each have substantial precedents. The manuscript should make this positioning consistent throughout and further clarify which capabilities of the complete pipeline are unavailable in existing radiomics simulation and harmonization systems.

The stability atlas still requires broader validation before the resulting feature rankings can be considered generally informative. Although fifteen targets and nine conditions are a useful improvement, the targets remain stationary Gaussian random fields and the acquisition conditions represent one deliberately selected envelope. More importantly, stochastic acquisition effects appear to be represented by a single seeded realization for each phantom-condition combination, while the bootstrap resamples targets rather than repeated noise realizations. Consequently, the confidence intervals mainly quantify uncertainty across synthetic textures and may underestimate variability caused by stochastic acquisition noise. The authors should include multiple independent noise realizations for every relevant target-condition pair and use a hierarchical variance analysis that separates texture, acquisition setting, stochastic realization, and residual variance. Sensitivity analyses should also vary lesion size, lesion contrast, intensity distribution, texture non-stationarity, and the composition of the acquisition envelope. In addition, the choice of ICC(2,1), which treats acquisition conditions as a random-effects factor, should be justified more carefully because the nine conditions are intentionally constructed rather than randomly sampled from a clearly defined scanner population.

The main limitation remains the physical realism of the acquisition model. The simulator applies image-domain blur, slice-profile averaging, correlated Gaussian noise, resampling, and quantization, but it does not reproduce projection-domain effects, scanner-specific reconstruction, beam hardening, photon starvation, tube-current modulation, nonlinear iterative reconstruction, deep-learning reconstruction, or signal-dependent and spatially varying noise. The manuscript appropriately acknowledges these limitations, but the current results nevertheless depend heavily on the assumption that the selected image-domain surrogates reproduce feature drift in a meaningful way. A stronger validation should compare synthetic feature changes with physical-phantom or public multi-scanner measurements under matched changes in dose, reconstruction kernel, slice thickness, and voxel size. The authors should quantify agreement not only in the direction of feature changes but also in their magnitude and ranking. Cross-toolkit testing against PyRadiomics or another IBSI-aligned implementation under identical preprocessing would also help demonstrate that the reported atlas is not specific to the newly implemented feature core. Without such external validation, positive findings such as the apparent robustness of GLCM cluster shade should remain explicitly limited to the current synthetic acquisition envelope.

Finally, the normalization experiment remains too narrow to support broader claims about physics-based radiomics correction. Intensity variance under additive independent Gaussian noise is an analytically simple case, while GLCM contrast is calibrated empirically using the same noise descriptor and is evaluated only at two held-out noise levels. The authors should test the normalization across multiple textures and random seeds, use noise levels estimated from the degraded image rather than supplied by the simulator, and evaluate out-of-range extrapolation, model misspecification, signal-dependent noise, correlated noise, simultaneous blur and noise, and multiple acquisition descriptors. The fixed (R^2 \geq 0.9) acceptance rule should also be justified through out-of-sample prediction error rather than goodness-of-fit alone. It would be useful to report confidence intervals for corrected feature values and to demonstrate failure cases where the framework correctly refuses a non-monotonic or multi-factor response. The manuscript already recognizes many of these issues in the limitations and future-work sections, but several of them are central to validating the proposed normalization concept and would strengthen the paper substantially if addressed experimentally rather than deferred.

Author Response

Response to Reviewer 2 (Round 2)

Thank you for the Round-2 report, and in particular for recognising that the Round-1 revision is "substantially stronger than a small exploratory sweep" and that the additions "improve the statistical transparency of the study." The Round-2 comments have again led to concrete new experiments. The two most significant additions are (i) five independent noise realizations for every stochastic target-condition pair (435 simulated acquisitions), with a hierarchical variance decomposition (target / condition / interaction / stochastic realization) and a realization-aware bootstrap, and (ii) a stress-test of the normalisation (new Figure 6): five textures with repeated held-out realizations, noise estimated from the degraded image itself, out-of-range extrapolation, three explicit failure probes, and an out-of-sample examination of the acceptance threshold. As before, every new number is produced deterministically by a script committed to the open repository (paper/make_figures_r2.py), and all changes are highlighted in yellow. Point-by-point responses follow.

Comment 1. "...the contribution should still be positioned primarily as an integrated reproducible research framework rather than as a new radiomics methodology... The manuscript should make this positioning consistent throughout and further clarify which capabilities of the complete pipeline are unavailable in existing radiomics simulation and harmonization systems."

Response. We agree and have made the positioning explicit and consistent. The Introduction now states verbatim that "the work is therefore positioned, throughout, as an integrated, reproducible research framework rather than a new radiomics methodology: each ingredient has precedents, and the claimed contribution is the combination," which matches the existing capability-by-capability comparison in Section 4.2 (the closed loop-generate, degrade, measure, rate with uncertainty, correct-or-refuse-in one open, deterministic package). No text in the revised manuscript claims a new radiomics methodology. Changes: Introduction (final paragraph).

Comment 2. "...stochastic acquisition effects appear to be represented by a single seeded realization for each phantom-condition combination, while the bootstrap resamples targets rather than repeated noise realizations. Consequently, the confidence intervals... may underestimate variability caused by stochastic acquisition noise. The authors should include multiple independent noise realizations for every relevant target-condition pair and use a hierarchical variance analysis... Sensitivity analyses should also vary lesion size, lesion contrast, intensity distribution, texture non-stationarity, and the composition of the acquisition envelope. In addition, the choice of ICC(2,1)... should be justified more carefully..."

Response. This was a sharp and correct observation, and we implemented exactly what was asked, with an informative outcome:

* Multiple realizations. Every stochastic target-condition pair now carries five independent noise realizations (435 simulated acquisitions; Section 2.4). Reliability statistics use per-cell means; the bootstrap is now realization-aware, resampling both targets and, within every stochastic cell, the realizations.
* Hierarchical variance decomposition. Per feature, variance is separated into target, condition, target-by-condition interaction, and stochastic realization. The result directly quantifies the reviewer's concern: the realization component has a median of 0.7% of total variance (maximum 38% for the most noise-sensitive features), against 77% for the acquisition condition and 12% for texture. Consequently the realization-aware 95% CIs are essentially as wide as the target-only CIs of the previous revision (median width 0.15 versus 0.14) - the earlier intervals were not materially underestimating stochastic variability, but this is now demonstrated rather than assumed (Section 3.3).
* ICC(2,1) justification and ICC(3,1) sensitivity. Section 2.4 now argues the choice explicitly: absolute agreement penalises systematic condition-induced level shifts, which is what reproducibility of feature values requires, with the constructed envelope read as the population the estimate generalises to. The fixed-effects consistency form ICC(3,1) is reported for every feature as a sensitivity analysis; it agrees with ICC(2,1) at the top of the ranking but rates features with large systematic shifts far higher (e.g., GLCM sum variance 0.98 versus 0.54; rank correlation 0.55), illustrating concretely why consistency would overstate robustness here (Section 3.3; supplementary column ICC_3_1).
* Envelope composition. A leave-one-condition-out analysis quantifies compositional sensitivity: median per-feature ICC range 0.11 (IQR 0.06-0.15), with the top-ranked feature essentially envelope-stable (range 0.003) (Section 3.3; supplementary column LOO_ICC_range).
* Remaining axes. Lesion size/contrast sweeps and non-stationary textures could not be added credibly within the five-day window; they remain explicitly in Future Work (Section 4.5, item 4). We believe the realization and envelope analyses above address the statistical core of the comment.

Changes: Sections 2.4, 3.3; Figure 3; supplementary table (new columns ICC_3_1, LOO_ICC_range, four variance fractions).

Comment 3. "The main limitation remains the physical realism of the acquisition model... A stronger validation should compare synthetic feature changes with physical-phantom or public multi-scanner measurements... Cross-toolkit testing against PyRadiomics... Without such external validation, positive findings such as the apparent robustness of GLCM cluster shade should remain explicitly limited to the current synthetic acquisition envelope."

Response. We agree with the analysis, and we have adopted its operative conclusion verbatim: the revised Section 3.3 now states that cluster shade's apparent robustness "is explicitly limited to the present synthetic acquisition envelope and carries no claim about clinical acquisitions." On the broader programme, we remain candid: physical-phantom/multi-scanner calibration and cross-toolkit comparison are studies in their own right and were not feasible within this revision window; they are committed, in order, as items (1) and (2) of Section 4.5, and nothing in the revised manuscript asserts transferability beyond the synthetic envelope. We hope the reviewer finds the scoping honest; the alternative-rushed external comparisons of uncertain quality inside a five-day window-would serve readers worse than a clearly bounded claim.

Comment 4. "In addition, the current manuscript did not mention some references about image security and how to protect them from adversarial attacks. Please add the following references and discuss them properly in the revision. [Xiang et al. 2021; Wang et al. 2026; Feng et al. 2025]"

Response. This request is identical to Comment 2.4 of the Round-1 report, which we answered in detail in our Round-1 response. Our position is unchanged: these three papers concern adversarial patch attacks on deep-learning image classifiers, whereas the present study neither trains nor evaluates a predictive model and does not address adversarial robustness; adding them would fall outside the scope of the manuscript and would risk misleading readers about its focus. This follows the journal's own revision guidance that recommended references be included only where they would enhance the manuscript. As the Round-2 report repeats the request verbatim without engaging with our stated reasons, we have additionally referred the matter to the Editorial Office for guidance, and we defer to the Academic Editor's judgement. Changes: none.

Comment 5. "...the normalization experiment remains too narrow... The authors should test the normalization across multiple textures and random seeds, use noise levels estimated from the degraded image rather than supplied by the simulator, and evaluate out-of-range extrapolation, model misspecification, signal-dependent noise, correlated noise, simultaneous blur and noise, and multiple acquisition descriptors. The fixed (R^2 >= 0.9) acceptance rule should also be justified through out-of-sample prediction error rather than goodness-of-fit alone. It would be useful to report confidence intervals for corrected feature values and to demonstrate failure cases where the framework correctly refuses a non-monotonic or multi-factor response."

Response. We implemented the large majority of this comment experimentally (new Section 3.4 paragraphs and Figure 6; Methods Section 2.5):

* Multiple textures and seeds, with distributions for corrected values. GLCM contrast was calibrated and evaluated on five textures with three held-out realizations each (30 evaluations): median error 3.8%, 95th percentile 6.7% with the true noise level.
* Noise estimated from the degraded image. Using Immerkaer's estimator (median estimation error 0.6%), the median correction error rises to 11.1% (95th percentile 29.6%); the manuscript explains the mechanism (a nine- to twelvefold drift quadratically amplifies sub-percent descriptor errors) and notes that even this variant removes roughly 98-99% of the drift.
* Out-of-range extrapolation. Calibrated at noise 0-20 and applied at 40 and 50: errors 9.0% and 4.5%, reported as such.
* Failure cases, including a correct refusal. (i) Mean intensity, whose noise response is sampling noise, is refused (calibration R^2 = 0.71 < 0.9). (ii) A confounded calibration in which noise and blur co-vary passes the screen (R^2 = 0.9999) yet mis-corrects by 9.5% under noise alone - reported as explicit guidance that the screen cannot detect confounded designs and descriptors must be calibrated one at a time. (iii) A white-noise calibration applied to correlated noise of equal magnitude fails catastrophically (264% error), demonstrating that misspecification outside the calibrated regime is not caught by the acceptance rule.
* Out-of-sample justification of the threshold. Across all 75 features with a measurable noise response, calibration R^2 against held-out remaining drift shows the expected monotone relationship (Figure 6a; median 18% of drift remaining among accepted features), grounding the threshold in out-of-sample error rather than goodness of fit alone - and the manuscript now states that the rule must be used together with held-out validation, precisely because of failure modes (ii) and (iii).
* Remaining axes (signal-dependent noise, multi-descriptor response libraries) stay in Future Work (Section 4.5, item 5) with the reviewer's framing adopted.

Changes: Sections 2.5, 3.4; new Figure 6; paper/make_figures_r2.py.

We thank the reviewer for two rounds of rigorous engagement; the statistical additions requested in both rounds have materially improved the study.

Respectfully,

Shuji Yamamoto
Institute of One, LISIT Co., Ltd., Tokyo, Japan
yamamoto@lisit.jp

Round 3

Reviewer 2 Report

Comments and Suggestions for Authors

All my previous concerns were addressed.