Clinical Image Quality and Reader Variability in 3D Synthetic Brain MRI Compared with Conventional MRI

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript provides a comparison of 3D synthetic MRI and conventional MRI. However, limitations related to reader variability, small sample size, and lack of diagnostic validation constrain its immediate clinical applicability. I suggest authors highlight the limitations, be cautious in conclusions, and emphasize training and validation.

Comments

• The study includes 31 patients and three readers. This might limit statistical power and generalizability, particularly for inter reader variability analyses, as it was consistently poor to moderate for several key parameters, particularly for 3D synthetic imaging.
• The systematic differences between experienced neuroradiologists and the trainee suggest that 3D SI interpretation is highly experienced dependent. The manuscript would benefit from a clearer discussion on the need for structured reader training or learning curves before routine clinical implementation.
• Contrast administration was inconsistent across patients and modalities. You should discuss it more critically as a potential source of bias.
• The study focuses on subjective image quality and confidence rather than diagnostic accuracy. Without correlation to clinical outcomes, pathology, or reference standards, it remains unclear how observed differences in contrast and SNR translate into real-world diagnostic performance. Add it to limitations.
• All imaging was performed on scanners from a single vendor using reconstruction software. This limits generalizability and raises concerns about reproducibility across vendors and software versions. Add it to limitations.

Author Response

See Word Doc attached

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents a systematic comparison of 3D SI and cMRI in routine neuroradiology. Using matched datasets and three readers, the authors evaluated image quality, diagnostic confidence, and the detection of key brain imaging findings. Overall, the study is clinically relevant; however, several issues related to methodological details, interpretation of agreement metrics, and result interpretation require further clarification to improve the rigor and readability of the manuscript.

1. The authors’ institutional affiliations are not fully listed and should be completed.
2. Figure 11 is presented without a corresponding explanation or discussion of the imaging findings shown. The authors are encouraged to add an explanation in the main text.
3. It is recommended that all tables adopt a consistent formatting and layout style to improve overall consistency and readability.
4. In the final paragraph of the Introduction, the authors state that the evaluation of 3D SI image quality remains limited and cite four related studies. However, these references are merely listed without discussion. The authors are encouraged to more specifically summarize the main limitations and shortcomings identified in prior studies regarding 3D SI image quality assessment, thereby more clearly motivating the research rationale and innovation of the present work.
5. In Figure 2, cMRI and 3D SI should be explicitly labeled. In addition, a cursor-like marker appears to remain in the center of the right-hand image and should be removed.
6. The overall clarity of Figure 3 is suboptimal, and an increase in image resolution is recommended. In addition, the specific meaning of y-axis labels such as x16 and x20 should be clearly defined so that readers can understand the corresponding scores or metrics.
7. The qualitative interpretation of Krippendorff’s alpha does not consistently follow commonly accepted agreement thresholds. For example, α = 0.588 in Figure 3 is described as “poor,” whereas values above 0.4 are generally considered to indicate at least moderate agreement. The authors are encouraged to clarify their agreement classification criteria or to distinguish between statistical agreement and clinical acceptability to avoid potential misunderstanding.
8. In Figure 4, Krippendorff’s alpha is reported to quantify interrater agreement; however, the accompanying analysis focuses primarily on medians and score distributions, with limited discussion of the agreement metrics themselves. Given that some alpha values are negative or close to zero, these findings are not fully consistent with descriptors such as “consistent” or “reliable” used in the text. A more comprehensive interpretation of the agreement results is recommended.
9. Figure 5 shows significantly lower ratings for 3D SI compared with cMRI in assessing FLAIR sulcal CSF–cortex contrast. However, interrater agreement for this metric is low for both modalities and particularly poor for 3D SI, with Krippendorff’s alpha approaching zero (α = 0.064). This suggests a strong dependence on individual reader experience. The authors are encouraged to further emphasize this limitation and its potential impact on clinical reproducibility in the discussion.
10. Figure 6 explores a potential association between GCA and sulcal CSF signal on 3D SI FLAIR images, but this trend is observed in only a single reader. The authors are encouraged to further clarify whether this finding is reader-dependent.
11. Figure 7 suggests lower interrater agreement for WMH detection on 3D SI, along with the presence of potential false-positive findings. The authors are encouraged to further discuss whether these observations reflect increased sensitivity or reduced specificity.
12. In line 343, the phrase “mostly excellent impressions” is highly qualitative. The authors are encouraged to avoid overly subjective wording and to support such statements with score distributions or statistical results.
13. In Figure 9, the upper and lower rows of images overlap and should be rearranged. Additionally, it may be helpful to directly annotate the corresponding scores within the figure to enhance visual clarity.
14. Axis labels such as x11 and x02 are not recommended. The authors are encouraged to explicitly label axes with the specific metrics or parameters they represent to improve readability.

Author Response

Please find the reply as Word Doc attached.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

This manuscript presents a careful reader-based comparison between 3D synthetic MRI (3D SI) and conventional MRI (cMRI) in routine neuroradiological practice. The authors systematically evaluate tissue contrast, signal-to-noise ratio, motion robustness, and reader variability using blinded assessments. A key strength is the focus on clinically relevant interpretive challenges rather than purely technical metrics. The study demonstrates that 3D SI provides diagnostically usable images with superior motion robustness, while also highlighting important limitations in cortical FLAIR contrast and vascular signal interpretation. Overall, the work provides balanced and practical guidance for the clinical integration of 3D synthetic imaging.

The results show substantial reader-dependent variability, particularly for subtle findings and FLAIR-based lesion detection. Could the authors clarify whether additional structured training or calibration specific to 3D SI (beyond the initial session) might reduce false positives and improve inter-reader agreement? A brief discussion on learning curves for synthetic imaging would strengthen the manuscript.

 

The broader and altered vascular signal characteristics observed in 3D SI may reduce sensitivity to vascular abnormalities. Can the authors elaborate on how this limitation might affect real-world diagnostic workflows, particularly in emergency or stroke-related imaging where conventional flow-void assessment is still commonly relied upon?

 

Experienced readers reported false-positive white matter lesions more frequently on 3D SI. Could the authors discuss whether these false positives tend to localize to specific anatomical regions (e.g., cortical–CSF interfaces) and how radiologists might systematically differentiate these artifacts from true pathology in daily practice?

 

Many large-scale and long-span longitudinal neuroimaging studies have traditionally relied on highly standardized sequences such as MPRAGE to ensure consistency across time. It would be valuable if the authors could expand the Discussion to address whether 3D synthetic MRI could be a suitable tool for such longitudinal studies. Specifically, how do the authors view the stability, reproducibility, and cross-time comparability of 3D SI-derived contrasts relative to conventional sequences in long-term follow-up settings?

10.1001/jamanetworkopen.2024.41505

10.1002/alz.70544

Author Response

Please find the reply as Word Doc attached.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Authors have addressed all issues.

Reviewer 2 Report

Comments and Suggestions for Authors

All of my comments have been addressed. I have no further comments.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors addressed my concerns properly.