Improved Productivity Using Deep Learning-Assisted Major Coronal Curve Measurement on Scoliosis Radiographs

Abstract

Background: Deep learning models have the potential to enable fast and consistent interpretations of scoliosis radiographs. This study aims to assess the impact of deep learning assistance on the speed and accuracy of clinicians in measuring major coronal curves on scoliosis radiographs. Methods: We utilized a deep learning model (Context Axial Reverse Attention Network, or CaraNet) to assist in measuring Cobb’s angles on scoliosis radiographs in a simulated clinical setting. Four trainee radiologists with no prior experience and four trainee orthopedists with four to six months of prior experience analyzed the radiographs retrospectively, both with and without deep learning assistance, using a six-week washout period. We recorded the interpretation time and mean angle differences, with a consultant spine surgeon providing the reference standard. The dataset consisted of 640 radiographs from 640 scoliosis patients, aged 10–18 years; we divided the dataset into 75% for training, 16% for validation, and 9% for testing. Results: Deep learning assistance achieved non-statistically significant improvements in mean accuracy of 0.32 for trainee orthopedists (95% CI −1.4 to 0.8, p > 0.05) and 0.43 degrees (95% CI −1.6 to 0.8, p > 0.05) for trainee radiologists (non-inferior across all readers). Mean interpretation time decreased by 13.25 s for trainee radiologists, but increased by 3.85 s for trainee orthopedists (p = 0.005). Conclusions: Deep learning assistance for measuring Cobb’s angles was as accurate as unaided interpretation and slightly improved measurement accuracy. It increased the interpretation speeds of trainee radiologists but slightly slowed trainee orthopedists, suggesting that its effect on speed depended on prior experience.

1. Introduction

Adolescent idiopathic scoliosis (AIS) has a global prevalence of 0.5–5.2% [1] and can lead to significant morbidity if left untreated [2]. Several countries have implemented national scoliosis screening programs to detect moderate AIS early in children and adolescents. Early detection of scoliosis enables the use of conservative treatments, such as bracing, to prevent curve progression and potentially reduce the need for surgical intervention [3]. In Singapore, more than 140,000 children are screened for AIS annually, of which approximately 8000 are identified for further radiographic evaluation [4].

Radiographic evaluation of suspected scoliosis is crucial for diagnosis and further management [5]. On a standing posteroanterior spinal radiograph, scoliosis is defined as a coronal curvature with a Cobb’s angle of 10 degrees or greater [6]. Figure 1 illustrates an example of the measurement of Cobb’s angle, which refers to the greatest angle between two lines passing through the superior and inferior endplates of two appropriately selected vertebrae. The severity of AIS is classified according to the angle of the major coronal curve, with curves of approximately 10–24°, 25–40° and >40° representing mild, moderate and severe scoliosis, respectively [7,8]. This classification is essential for prognostication and treatment planning [9,10].

Methods of measurement of the major coronal curve include manual (i.e., using printed radiographic films and Cobbometers) or computer-assisted techniques. Manual measurements performed by an experienced reader is the most widely used technique, but it requires significant technical expertise and may be vulnerable to inaccuracies, with varying intra- and inter-reader agreement [11]. In addition, manual measurement methods are time-consuming and labor-intensive, requiring dedicated reading sessions and potentially reducing the amount of time available for physician-patient interaction. Computer-assisted techniques are able to facilitate semi-automated measurements of Cobb’s angle, but require accurate input data, in terms of vertebral endplate selection. The correct selection of vertebral endplates remains challenging, especially for inexperienced readers performing the initial screening [12].

Deep learning techniques have found applications in the management of various spinal diseases, such as the automated detection of spinal stenosis on magnetic resonance imaging (MRI) scans [13,14], prediction of adverse events after spinal surgery [15], and prognostication of walking ability after spinal cord injury [16,17]. Deep learning solutions, utilizing convolutional neural networks (CNNs), have also been developed for the assessment of scoliotic curves on radiographic images [18,19,20,21,22]. These solutions involve manual segmentation of the vertebra for CNN training and incorporate various methods for deriving the major coronal curve, such as spline techniques [23]. Overall, these deep learning solutions show significant promise, and a recent study by Ha et al. (2022) [22] demonstrated a mean difference of 7.3° between major coronal curve angles derived by deep learning methods, and those obtained by the interpretations of expert readers. This is within the reported range of inter-observer discrepancies (three to five degrees) in Cobb’s angle measurements performed by expert readers [22,24,25,26].

These studies appear to demonstrate the accuracy of deep learning techniques in the measurement of Cobb’s angle; however, the translational impact of these deep learning solutions, when incorporated into actual clinical practice workflows to augment physician capabilities, remains unclear. There is a relative scarcity of studies examining the actual clinical impact of deep learning-augmented diagnostics; recent deep learning scoliosis studies have largely focused on advancing the technical performance of existing analytical tools. In a recent systematic review by Goldman et al. (2024) [27], studies investigating the use of deep learning techniques for imaging analysis of scoliosis primarily focused on the prediction of Cobb angles, or the progression of adolescent idiopathic scoliosis. The review found that only a small proportion of studies investigated the use of artificial intelligence in clinical decision support such as treatment management or diagnosis of AIS, and that most (up to 62.5%) studies did not report on practical strategies or guidelines on clinical implementation of their deep learning techniques. In another systematic review, Chan et al. (2024) [28] also reported that most existing machine learning models are not clinically deployable and require further external validation.

In this study, we utilized a deep learning model in a simulated clinical setting to assist physicians in the assessment and grading of scoliotic curves. This proof-of-concept study aims to investigate the impact of deep learning assistance on the interpretation times and accuracies of clinicians in the assessment of scoliosis radiographs. We hypothesize that deep learning assistance would have a beneficial impact in reducing the interpretation times of scoliosis radiographs, especially for less experienced readers.

2. Materials and Methods

The study was conducted in accordance with the Declaration of Helsinki, and the local Institutional Review Board (IRB) of the National Healthcare Group (NHG), Singapore approved the study protocol (IRB approval number—2021/01084, date of approval: 17 February 2022). The IRB granted the study a waiver of consent due to the study’s retrospective nature and minimal risk involved.

2.1. Scoliosis Dataset

This study included all consecutive pediatric patients (10–18 years old) diagnosed with AIS, who presented at the National University Hospital, Singapore during the period of January 2018–January 2019 and who had available standing, full-length, posteroanterior spinal radiographs. The study excluded adult patients (aged 18 years or older), as well as patients with spinal instrumentation and other skeletal or neuromuscular disorders.

We retrospectively extracted the radiographs from the platforms of various institutions—e.g., EOS^® Imaging System (Paris, France), AGFA (Mortsel, Belgium) and Fujifilm (Tokyo, Japan), etc.—and anonymized them for the purpose of this study. The selection of a wide range of radiographic images allowed the deep learning model to be trained to process images with varying levels of noise, brightness, and contrast, across a range of scoliotic curves.

The division of the dataset of scoliosis radiographs into 75% for training, 16% for validation and 9% for testing (480, 100 and 60 radiographs, respectively) was a split that was consistent with previous studies [29,30].

2.2. Context Axial Reverse Attention Network (CaraNet) Object Detection Model

In this study, we used the Context Axial Reverse Attention Network (CaraNet) to perform vertebral segmentation. CaraNet has been utilized in many applications and is found to be accurate in segmenting objects from images, utilizing axial reserve attention and channel-wise feature pyramid modules to detect global and local feature information [31].

2.3. Deep Learning Model Development

Four experienced radiologists, including two musculoskeletal radiologists (J.T.P.D.H., with 12 years of experience, and D.S.W.L., with two years of experience) and two neuroradiologists (A.M., with seven years of experience, and X.Z.L., with four years of experience) manually labeled the radiographs in the training and validation sets.

Pre-processing ensured that the images were consistent in terms of size and shape before model training. The division of the image dataset into the training, validation and test sets (480, 100 and 60 radiographs, respectively) followed the pre-processing phase. The use of standard image augmentation techniques (e.g., flip, rotation, etc.) allowed the dataset size to be increased from 580 to more than 7000 images.

Each radiologist labeled at least 50 whole-spine posteroanterior radiographs and used an annotation tool, Darwin (available at: https://darwin.v7labs.com/login, accessed on 1 May 2022) (V7, London, UK), to perform manual segmentation of all corners of the vertebral bodies from C7 to L5 (or the last unfused vertebral body). Manual labeling generated approximately 36,550 object annotations, which trained the CaraNet model to identify the corners of each vertebra, with a Dice coefficient of 0.93 for the training data and 0.88 for the validation set. We used the Adam optimizer and a batch size of 32 to train the CaraNet model. The model learning rate was 0.001; the number of training epochs was 500. The high volume of medical imaging data necessitated the use of an institutional supercomputer to train the deep learning model. The total model training time was 130 h.

The process above allowed deep learning model masks of the whole spine (showing individual vertebrae) to be obtained and compared to the manual annotations. The model achieved a Dice similarity coefficient of 0.88 and Intersection over Union (IOU) of 84% on validation; it achieved a Dice coefficient of 0.619 and IOU of 72% on testing. Using the masks, the deep learning model determined the center of each vertebra, and extrapolated a polynomial curve along the centers. The spline technique, or the exhaustive assessment of the maximum angles between the vertebrae center line pairs to determine the largest angle, was the method of choice for the measurement of Cobb’s angle. Once the model has identified the correct points on the line, the reader could view the model prediction as extended lines and text in the top right hand corner of the image. For physicians’ review and augmentation, the model (available at the following web server: https://radweb.sha.endeavour-poc.ai/, accessed on 1 May 2022) annotated the largest possible angle in the output images, as shown in Figure 2 and Figure 3. Appendix A provides more details on the deep learning model.

2.4. Reference Standard

For the held-out test set, a consultant orthopedic surgeon experienced in scoliosis assessment and surgery (J.H.J.T., with six years of experience) provided the reference standard in determining the major coronal curve angle and curve location for each radiograph, without access to the deep learning model.

2.5. Study Design

Eight residents performed the assessment of the major coronal curve with and without the assistance of the deep learning model on the held-out test set.

The readers comprised four trainee radiologists (R1–R4), and four trainee orthopedic surgeons (O1–O4). Each orthopedist possessed at least six months of experience working in a specialist scoliosis clinic and interpreting scoliosis radiographs, including the determination of major coronal curve angles. Therefore, the trainee orthopedists had an intermediate level of experience for major coronal curve assessment (intermediate readers), as compared to the trainee radiologists, who had little to no experience (novice readers).

The relationship between reader experience and performance on deep learning-assisted reads remains a relatively under-explored area in artificial intelligence research. Evidence from prior research is inconclusive, and the effects of deep learning augmentation on the performance of readers with varying levels of experience are inconsistent and heterogeneous [32]. Hence, another aim of this study is to quantify the impact of the experience level of observers on deep learning-assisted reads, through the comparison of orthopedic and radiology trainees.

Prior to the study, each reader utilized a training set of 10 radiographs for illustration and instruction on major coronal curve measurement. The readers performed manual measurements first, after which they accessed the deep learning model and the reference standard measurements for comparison.

The study divided the readers equally into two groups, with each group consisting of two trainee radiologists and two trainee orthopedists. Firstly, Group A (R1, R2, O1 and O2) interpreted 60 scoliosis radiographs without deep learning assistance, while Group B (R3, R4, O3 and O4) performed the radiographic interpretations with assistance from the deep learning model. After a washout period of six weeks [33], we reshuffled the test set at random to prevent familiarization, and reversed the utilization of the deep learning model—Group A interpreted the radiographs with deep learning assistance, and Group B performed the interpretations without. Figure 4 shows the study design, which aimed to minimize carryover effect [14]. The readers independently assessed the radiographs in a dedicated reading room [with a high-resolution monitor (BARCO MDCC-6430, 6MP, Belgium)]. This study recorded both the major coronal curve angle and time taken for assessment of each radiograph. Interpretation time was the duration from the opening of the radiograph to the transcription of Cobb’s angle by the reader. The system automatically logged the start and end of the interpretation (as defined) using time stamps, and calculated the elapsed time. We anonymized all images, and blinded the readers to the clinical histories, patient characteristics and prior studies.

2.6. Radiographic Assessment

The readers performed manual measurements of the major coronal curve angle without assistance from the deep learning model using the digital angle tool on the Picture Archiving and Communication System (PACS) (Centricity, GE Healthcare, Chicago, IL, USA), with the results overlaid on the images. When interpreting the radiographs with assistance from the deep learning model, the readers accessed the automated major coronal curve calculations and annotations overlaid on the images (as illustrated in Figure 2 and Figure 3). The readers could accept the model predictions or perform manual calculations using the same digital tool as per their clinical judgment, especially if the center curve alignment did not match the image.

2.7. Statistical Analysis

A biostatistician (Y.H.C.) performed all statistical analysis using Stata (Version 16, StataCorp, College Station, TX, USA). The main variable of interest was the time required to measure the primary coronal curve in each radiograph, comparing readings with and without support from the deep learning model.

Assuming a moderate effect size of 0.5 as per Cohen’s criteria for time difference (measured in seconds) between readings of enhanced and non-enhanced images, a priori analysis showed that at least 40 scoliosis radiographs were needed to achieve a statistical power of 80%, at a 5% two-tailed significance level.

The assessment of differences in angle measurements and time required for the primary coronal curve measurements between deep learning-assisted and unassisted reads involved comparing paired mean angle differences. This study utilized linear mixed model analysis to evaluate the time differences, treating readers as a random effect to adjust for any sequence effects, and treating the use of deep learning assistance (i.e., unassisted vs. assisted reads) as a fixed effect. Normalization of the timing data with the Fisher-Yates shuffle confirmed the validity of the linear mixed model p-values.

Standard statistical methods of analysis included the t-test, Mann–Whitney U test, and analysis of variance (ANOVA) for continuous variables. This study determined the means and standard deviations of continuous variables and presented their 95% confidence intervals. The definition of statistical significance was two-tailed p < 0.05.

3. Results

3.1. Patient Characteristics

This study excluded 160 patients out of 800 consecutive patients—40 adult patients, 100 patients with spinal instrumentation, and 20 patients with other skeletal or neuromuscular disorders. The study included a total of 640 patients with 640 posteroanterior scoliosis radiographs in the Digital Imaging and Communications in Medicine (DICOM) format.

The study divided the data randomly into the training, validation and test sets—580 patients with 580 posteroanterior spinal radiographs in the training and validation set, and 60 patients with 60 radiographs in the internal test set. Mean age of the 60 patients in the test set was 12.6 ± 2.0 (10–18) years. There were 43 (71.7%) female patients with a mean age of 11.8 ± 1.6 years (range 10–18 years) and 17 (28.3%) male patients with a mean age of 14.4 ± 1.8 years (range 12–18 years) in the internal test set. Figure 5 shows a flow chart of the study design.

Table 1. Patient demographics and scoliotic curve characteristics.

Characteristics	Test Set (n = 60)
Age (years), mean ± standard deviation (range)	12.6 ± 2.0 (10–18)
Sex, n (%)
Female	43 (71.7)
Male	17 (28.3)
Reference standard scoliosis grading (Cobb’s angle), n (%)
Mild (10–24°)	30 (50.0)
Moderate (25–40°)	25 (41.6)
Severe (>40°)	5 (8.3)

shows the characteristics of the patients included in this study.

3.2. Scoliotic Curve Characteristics

Table 1 shows the reference standard major coronal curve angles for the 60 radiographs in the test set. Overall, major coronal curves of <25 degrees (mild scoliosis) accounted for 50.0% (30/60) of the cases. Moderate scoliosis (Cobb’s angle in the range of 25–40°) and severe scoliosis (Cobb’s angle >40°) accounted for 41.6% (25/60) and 8.3% (5/60) of cases, respectively.

3.3. Major Coronal Curve Accuracy

On the 60 test set radiographs, the deep learning model demonstrated a mean angle difference of 3.9 ± 4.9° (95% CI: −5.9 to 8.8°), when compared to the reference standard.

Table 2. Mean Cobb’s angle differences for each reader with and without assistance from the deep learning model.

				ANOVA (DL-Assisted V. Unassisted)
Reader	DL Assistance	Mean Angle Difference (°)	95% CI	F-Statistic	p-Value
DL	-	3.9	−5.9 to 8.8	-	-
R1	No	−2.0	−3.1 to −1.0	21.65	<0.001 *
	Yes	1.4	0.4 to 2.4
R2	No	0.1	−0.9 to 1.1	0.23	0.630
	Yes	0.5	−0.5 to 1.4
R3	No	−0.1	−1.0 to 0.8	0.17	0.900
	Yes	−0.2	−1.1 to 0.7
R4	No	−3.2	−4.2 to −2.1	20.46	<0.001 *
	Yes	1.0	−0.5 to 2.6
O1	No	−0.5	−1.6 to 0.6	1.03	0.310
	Yes	0.5	−1.1 to 2.0
O2	No	−0.9	−1.9 to 0.2	0.01	0.940
	Yes	−0.8	−1.7 to 0.1
O3	No	−2.4	−4.0 to −0.8	4.37	0.039 *
	Yes	−0.5	−1.4 to 0.3
O4	No	1.2	0.3 to 2.1	0.98	0.330
	Yes	0.5	−0.5 to 1.6

Footnote: paired differences; mixed model with reader as the random factor. Abbreviations: 95% CI—95% confidence interval, ANOVA—analysis of variance, DL—deep learning, O—trainee orthopedist, R—trainee radiologist. * Statistically significant.

shows the mean angle differences of the individual readers, with or without deep learning assistance. Mean angle differences of readers R1, R4 and O3 showed significant improvements with the use of deep learning assistance, as compared to without. R1 had a mean angle difference of −2.0 ± 4.2° (95% CI −3.1 to −1.0°) without deep learning assistance, compared to an improved mean angle difference of 1.4 ± 3.8° (95% CI 0.4 to 2.4°) after utilizing deep learning assistance (p < 0.001). R4 had a mean angle difference of −3.2 ± 4.1° (95% CI −4.2 to −2.1°) without deep learning assistance, compared to an improved mean angle difference of 1.0 ± 5.9° (95% CI −0.5 to 2.6°) after utilizing deep learning assistance (p < 0.001). Mean angle difference of O3 also improved from −2.4 ± 6.1° (95% CI −4.0 to −0.8°), without deep learning assistance, to −0.5 ± 3.3° (95% CI −1.4 to 0.3°), with deep learning assistance (p = 0.039). The remaining five readers (R2, R3, O1, O2 and O4) showed no significant improvements in mean angle differences after utilizing deep learning model assistance, when compared against the reference standard (p-values ranging from 0.31 to 0.94).

Overall, when comparing the trainee radiologists and the trainee orthopedists, there were no significant differences in mean angle differences with or without assistance from the deep learning model (

Table 3. Differences in interpretation times and accuracies of trainee radiologists (novice readers) and trainee orthopedists (intermediate readers), between deep learning-assisted reads and unassisted reads.

	Reader	Mean Difference	95% CI	p-Value
Timing (s) 1	Ortho	3.9	−2.9 to 10.6	0.005
	Radio	−13.3	−19.6 to −6.9
Accuracy (°) 2	Ortho	−0.3	−1.4 to 0.8	>0.05
	Radio	−0.4	−1.6 to 0.8

Footnote: ¹ Difference in mean individual timings (in seconds) between deep learning-assisted reads and unassisted reads. ² Angle accuracy: differences in mean individual angles (in degrees) between deep learning-assisted reads and unassisted reads. Abbreviations: 95% CI—95% confidence interval, Ortho—trainee orthopedists, Radio—trainee radiologists.

). For unassisted reads, the mean angle differences ranged from −0.1 to −3.2° for the trainee radiologists, as compared to −0.5 to −2.4° for the trainee orthopedists. When assisted by the deep learning model, the mean angle differences ranged from −0.2 to −1.4° for the trainee radiologists, as compared to 0.5 to −0.8° for the trainee orthopedists.

3.4. Interpretation Time

Overall, the use of deep learning assistance improved the mean interpretation time per radiograph. There was a mean improvement of 4.7 s across all readers. There were significant differences across the two groups of inexperienced vs. experienced readers (

Table 4. Individual mean Cobb angle interpretation time (in seconds) per radiograph with and without assistance from the deep learning model.

Reader	DL Assistance	Interpretation Time (s), Mean ± SD	Mean Difference (s)	95% CI	p-Value
R1	No	19.3 ± 10.1	−10.4	−13.2 to −7.6	<0.001 *
	Yes	8.9 ± 6.4
R2	No	29.0 ± 6.9	−16.0	−18.7 to −13.3	<0.001 *
	Yes	13.0 ± 9.5
R3	No	39.1 ± 12.9	−11.5	−16.3 to −6.8	<0.001 *
	Yes	27.6 ± 14.8
R4	No	30.4 ± 13.3	−15.1	−21.2 to −9.0	<0.001 *
	Yes	15.3 ± 18.3
O1	No	9.7 ± 2.8	−3.9	−4.9 to −2.9	<0.001 *
	Yes	5.8 ± 2.6
O2	No	37.9 ± 13.4	4.9	−2.4 to 12.2	0.186
	Yes	42.8 ± 25.9
O3	No	50.8 ± 18.8	11.4	0.1 to 22.6	0.047 *
	Yes	62.2 ± 35.9
O4	No	26.9 ± 11.8	−2.1	−5.4 to 1.3	0.220
	Yes	24.9 ± 6.9

Footnote—paired differences. Abbreviations: 95% CI—95% confidence interval, DL—deep learning, O—trainee orthopedist, R—trainee radiologist, SD—standard deviation. * Statistically significant.

).

Deep learning assistance significantly reduced the interpretation times of all trainee radiologists, R1 to R4 (p < 0.001). The trainee radiologists achieved an average reduction of 13.3 s per radiograph. R2 showed the greatest improvement in mean interpretation time and achieved a reduction of 16.0 ± 10.5 s (95% CI −18.7 to −13.3) with the utilization of deep learning assistance (deep learning-assisted interpretation time of 13.0 ± 9.5 s, compared to an unassisted interpretation time of 29.0 ± 6.9 s) (Table 4); this represented a 55.2% saving in reading time (p < 0.001). R1 showed the smallest improvement in mean interpretation time, and achieved a reduction of 10.4 ± 11.0 s (95% CI −13.2 to −7.6) with the use of deep learning assistance (deep learning -assisted interpretation of time of 8.9 ± 6.4 s, compared to an unassisted interpretation time of 19.3 ± 10.1 s) (Table 4); this represented a 53.9% saving in reading time (p < 0.001). Figure 6 illustrates the impact of deep learning assistance on the mean interpretation times of individual readers.

As for the orthopedists in-training, only O1 had a small absolute time saving for deep learning-assisted reads, as compared to unassisted reads (Table 4). O1 showed a reduction in mean interpretation time of 3.9 ± 3.8 s (95% CI −4.9 to −2.9) (deep learning -assisted interpretation time of 5.8 ± 2.6 s, as compared to an unassisted interpretation time of 9.7 ± 2.8 s), which represented a 40.2% saving in reading time (p < 0.001). On the other hand, O3 showed an increase in mean interpretation time of 11.4 ± 43.4 s (95% CI 0.1 to 22.6) (deep learning-assisted interpretation time of 62.2 ± 35.9 s, as compared to an unassisted interpretation time of 50.8 ± 18.8 s, which represented a 22.4% mean time increase (p = 0.047). O2 and O3 had no significant differences in mean interpretation times between deep learning-assisted reads and unassisted reads.

4. Discussion

Radiographic evaluation of scoliosis and accurate measurement of the major coronal curve angle are essential for diagnosis and treatment planning [5]. In the clinic, manual measurement of Cobb’s angle is time-consuming, repetitive, and requires significant technical expertise (for accurate selection of the most angulated vertebral endplates). Deep learning techniques could potentially improve the efficiency and accuracy of major coronal curve angle measurements. This multi-reader study compared the accuracy and interpretation times of four trainee radiologists, without prior experience in the radiographic assessment of scoliosis (i.e., novice readers), to those of four trainee orthopedists with prior experience (i.e., intermediate readers), before and after the use of the deep learning model. Three of eight readers showed improvements in the accuracy of their major coronal curve angle measurements, as compared to a reference standard, after utilizing the deep learning model. This was greatest for R4—mean angle difference improved from -3.2° (95% CI −4.2 to −2.1°) to 1.0° (95% CI −0.5 to 2.6°) after using the deep learning model (p < 0.001). The remaining five readers showed no significant changes in mean angle differences after using the deep learning model (p-values ranging from 0.31 to 0.94). Overall, when comparing deep learning-assisted and unassisted major coronal curve assessments between the groups, the trainee radiologists had a mean time saving of 13.3 s (95% CI −19.6 to −6.9), compared to a slight time increase of 3.9 s (95% CI −2.9 to 10.6) for the trainee orthopedists (p = 0.005).

We believe that our study, serving as a proof of concept, has demonstrated the feasibility of the utilization of deep learning assistance by clinicians to augment their interpretations of scoliosis radiographs. Deep learning assistance has yielded non-inferior results in terms of accuracy, and has also resulted in statistically significant reductions in interpretation times for the group of less experienced readers. There is room for future studies to externally validate our study findings, and truly quantify the clinical utility of the deep learning model when it is applied in a variety of real-world, clinical practice settings.

4.1. Time Savings and Improved Diagnostic Efficiency

Prior studies have also demonstrated potential time savings by using deep learning models to perform repetitive tasks. Eng et al. [34] utilized a deep learning model for bone age assessment and achieved a reduced interpretation time of 102 s with deep learning assistance, as compared to an interpretation time of 140 s without deep learning assistance. Similarly, Lim et al. [14] were also able to achieve a reduction in mean interpretation time for detecting lumbar spinal stenosis on MRI scans by using a deep learning model, with up to 74% time savings for trainee radiologists (71 s with deep learning assistance, vs. 274 s without) and equivalent or superior inter-observer agreement for all stenosis gradings, as compared to radiologists who were not assisted by deep learning. Ahn et al. [35] also evaluated the impact of deep learning assistance on the interpretation of chest radiographs and the detection of four target findings (pneumonia, nodule, pneumothorax, and pleural effusion); the use of deep learning assistance allowed a 10% reduction in mean reporting time to be achieved (36.9 s with deep learning assistance, compared to 40.8 s without). There was also an improvement in reader sensitivities, but no negative impact on specificities. In our study, the trainee radiologists had reduced interpretation times with deep learning assistance, with a mean time saved of up to 16 s, or 55.2% (13.0 s with deep learning assistance, compared to 29.0 s without). Overall, three readers (R1, R4 and O3) showed increased interpretation accuracy with deep learning assistance, with R4 having the greatest reduction in mean angle difference, from −3.2 to 1.0°. Although the mean angle difference of −3.2° (without deep learning assistance) was the highest of all readers, it was still within the reported range of differences between expert readers (up to 10°); it also improved relative to our developed deep learning model (3.9°) and a prior published deep learning model by Ha et al. (7.3°) [22].

When comparing deep learning-assisted and unassisted major coronal curve measurements between the groups, the trainee radiologists had a mean time saving of 13.3 s (95% CI −19.6 to −6.9), compared to a slight time increase of 3.9 s (95% CI −2.9 to 10.6) for the trainee orthopedists (p = 0.005) (Table 3). We acknowledge that the effects are modest; however, they can accumulate over many cases, potentially translating into substantial savings in time and labor, when these deep learning techniques are formally incorporated into actual clinical workflows on an institutional or national level. In Singapore, more than 140,000 children are screened for adolescent idiopathic scoliosis annually, of which approximately 8000 are identified for further radiographic evaluation [4]. The scale of the national scoliosis screening program illustrates the necessity for enhanced productivity in the interpretation and reporting of scoliosis radiographs, a time-consuming and labor-intensive process. Increased automation of the radiographic interpretation process through the incorporation of deep learning techniques can potentially reduce the high workload and significant technical demands of radiographic analysis. Importantly, the readers also performed the measurements in an ideal environment with specialized PACS algorithms; we would expect greater time savings with the rudimentary manual measurements at the Health Promotion Board (the government agency of Singapore that conducts the national scoliosis screening program). A further prospective study will be required to validate this hypothesis.

Furthermore, there is room for further refinement of our deep learning model—with the inclusion of larger datasets and better model fitting, we can further improve and optimize the diagnostic accuracy of the model.

4.2. Clinical Impact and Cost-Effectiveness Thresholds

The thresholds of clinical significance of deep learning diagnostic techniques, and the cost-effectiveness of these diagnostics, remain an area of ongoing research. Future work should include cost-effectiveness analysis to fully elucidate the potential impact of integrating these deep learning techniques into actual clinical practice. In order to determine the thresholds of clinical significance or importance, future studies should balance the benefits of utilizing the deep learning model to augment existing diagnostic processes against the potential administrative, logistical and financial costs of implementation. Preliminary evidence from cost-effectiveness studies published by other authors on the utilization of artificial intelligence in other fields of radiology is encouraging. Prior research has shown that deep learning solutions may lead to substantial reductions in total operating costs, as compared to other capacity-enhancing methods [36]; the utilization of artificial intelligence to augment diagnostic tests can potentially result in significant cost savings and improve patient outcomes. Garg et al. (2025) [37] conducted a retrospective cost analysis on the use of deep learning-augmented chest radiography to detect active tuberculosis cases in Nigeria, and found that the use of chest radiographs and artificial intelligence in combination with symptoms screening, resulted in more cases detected at a lower cost per case, even after accounting for the implementation costs of artificial intelligence. Another study by Reginster et al. (2025) [38] examining the use of deep learning models for opportunistic screening of osteoporosis on chest radiographs found that the cost per quality-adjusted life year gained from screening was far below the threshold of cost-effectiveness.

Both of these studies demonstrated that the beneficial impact on patient outcomes, such as increased accuracy and efficiency of disease detection, earlier treatment initiation, and reduction in future complications, far outweigh the potential logistical and administrative costs of implementing these novel technologies.

Improved diagnostic processes will likely have a positive impact on the outcomes of patients with adolescent idiopathic scoliosis, by facilitating early treatment with conservative measures, such as bracing, to prevent curve progression, and reducing the need for costly surgical intervention. Studies of clinical utility should thus also take these factors into consideration, and, in this regard, longitudinal studies examining the long-term clinical impact of these deep learning-enhanced diagnostic processes would be of benefit, to allow for the accurate characterization of the thresholds of clinical importance and significance. Given the scale of the national scoliosis screening program in Singapore, where more than 140,000 children are screened for adolescent idiopathic scoliosis annually, we expect that the implementation of these deep learning techniques will result in significant time and cost savings. The public health impact of improved scoliosis diagnostics is also likely to be significant, given the prevalence and incidence of adolescent idiopathic scoliosis in Singapore. Furthermore, with the inclusion of larger datasets and better model fitting, we can further improve the accuracy and efficiency of our deep learning model, leading to greater time and cost savings.

In addition, the preliminary results of our proof-of-concept study demonstrated that the use of machine learning techniques may have a beneficial impact in the interpretations of scoliosis radiographs, by allowing clinicians to determine Cobb’s angle more efficiently and consistently. Beyond screening contexts, this also has direct implications on treatment planning, by allowing the accurate identification of patients who are at greatest risk of curve progression, and hence may require surgical intervention in future. A recent systematic review by Wong et al. (2022) has found strong and consistent evidence for Cobb’s angle to be one of the most significant predictive factors for curve progression [39].

The work of other researchers also demonstrated that artificial intelligence has an increasingly important role in other aspects of surgical planning, being utilized for various purposes such as the prognostication of surgical outcomes [40], prediction of compensatory postoperative changes in spinopelvic parameters [41], simulation of deformity correction [42] and the intraoperative insertion of spinal instrumentation (e.g., pedicle screws) [43].

Future work could build upon the preliminary results of this study, and investigate the use of the deep learning model and artificial intelligence techniques in clinical practice to guide treatment decisions. For example, Yahara et al. (2022) [44] developed a diagnostic tool using a deep convolutional neural network to predict the risk of curve progression and thus guide decision-making for therapeutic interventions in patients with adolescent idiopathic scoliosis.

We believe that the results from this study demonstrate the promise of deep learning-assisted diagnostics, which should be externally validated further in future studies.

4.3. Impact of Experience Level of Readers and Clinical Implications

In this study, the differences in interpretation times between the trainee radiologists and the trainee orthopedists could possibly be attributed to differences in experience and judgment [45]. The trainee radiologists were novice readers with no prior experience in the radiographic evaluation of scoliosis and may have been more likely to accept the deep learning model interpretations without confirmatory manual measurements. In comparison, the trainee orthopedists, possessing prior experience in the assessment of scoliosis radiographs, may have been more likely to rely on their manual measurements, rather than accept the deep learning model interpretations. Overall, despite the differences in prior experience and interpretation times, there were no significant differences in the accuracies of major coronal curve assessments between the trainee radiologists and trainee orthopedists, on both unassisted and deep learning-assisted reads (p > 0.05).

Previous studies have also explored the effects of the experience level of readers on the utilization of artificial intelligence. Current available evidence is relatively heterogeneous, but appears to weakly suggest an inverse relationship between the experience level of readers and the likelihood of acceptance of deep learning model predictions. Deep learning augmentation is also less likely to have a measurable impact on the performance of more experienced readers. In a study by Sung et al. (2021) [46] investigating the use of deep learning-based detection systems in the interpretation of chest radiographs, the non-radiology resident showed the greatest improvement in metrics such as area-under-the-curve analysis, as compared to experienced specialist thoracic radiologists, who showed the smallest increase, without significance, in per-lesion sensitivity. Similarly, in a study by Jang et al. (2020) [47], less experienced radiologists also contributed a higher-than-average incidence of report changes, when utilizing deep learning assistance to detect lung nodules on chest radiographs. Murata et al. (2019) [48] found that the use of computer-assisted diagnostics for the evaluation of maxillary sinusitis on panoramic radiographs was able to augment the performance of inexperienced readers to resemble that of experienced readers; however, experienced readers demonstrated high performance even without the deep learning system, and the use of computer-assisted diagnostics did not improve their performance.

This relationship (between reader experience and performance on deep learning-assisted reads) remains a relatively under-explored area in artificial intelligence research, but it has clinically relevant implications on the implementation of artificial intelligence in real-world practice. Based on current evidence from existing published studies, deep learning assistance may benefit inexperienced readers the most, and the implementation of deep learning-enhanced diagnostics in clinical settings that are lacking in resources [49], such as advanced imaging equipment or specialist physician expertise, would maximize their effectiveness. In these resource-scarce settings, artificial intelligence may have an integral role to play in enhancing the accuracy and efficiency of existing diagnostic tools, thus reducing the cognitive burden of physicians, and directly impacting patient outcomes. Future work should include more external validation studies to investigate the clinical utility of deep learning diagnostics, when deployed across a wide range of clinical settings and healthcare facilities.

4.4. Study Limitations

Our study has a few limitations. Firstly, the study focused on the evaluation of the major coronal curve and did not include the assessment of other features, such as secondary minor curves, or the Risser score for grading skeletal maturity.

Secondly, we used the spline technique of fitting a polynomial curve through the vertebral centers to develop the deep learning model. We chose the spline technique as it was superior to end-plate detection in the deep learning algorithm, especially on low-dose EOS radiographs such as those utilized at the Health Promotion Board of Singapore (which conducts Singapore’s national scoliosis screening program for children and adolescents). We postulate that the resolution for end-plate detection may be insufficient on low-dose radiographs, and that the spline technique, adopting the best fit curve, may reduce local precision, but increase the overall accuracy of major coronal curve measurement. This differs from human readers, who calculate the major coronal curve using the vertebral endplates, and may have led to reduced trust of the more experienced readers in the model interpretations.

Thirdly, as this was a retrospective study simulating a screening cohort of scoliosis radiographs, the study sample may have under-represented severe scoliosis cases, limiting the generalizability of the results. The performance of our deep learning model in the radiological assessment of severe scoliosis cases is uncertain, as it was largely trained on and fitted to mild or moderate cases—mirroring the cases encountered at a true screening center. Data from Singapore’s Health Promotion board (HPB) confirmed the proportion of cases in the current study sample. In other clinical contexts, such as an outpatient scoliosis clinic, the proportion of severe scoliosis cases may be greater. With the inclusion of more severe cases, re-fitting of our model may be necessary. The potential for greater inter- or intra-reader variability may also have an impact on the reads of less experienced observers. Our current test set fulfilled the sample size requirement determined by a priori power analysis; nevertheless, a larger test set would be beneficial in further increasing statistical power. Future studies could include larger test sets to determine more accurately the impact of deep learning assistance.

Fourthly, the readers accessed only a small training set of 10 cases prior to the formal interpretations, and this may have reduced some of the readers’ trust in the deep learning model’s interpretations. The study primarily involved trainees, and the lack of a more diverse group of readers may limit the generalizability of the findings. Less experienced readers, who often encounter difficulties in selecting the correct vertebral endplates for Cobb angle measurement, would likely benefit most from integrating deep learning into clinical workflows, through improved interpretation time and accuracy. Future studies should include larger and more heterogeneous reader groups to allow subgroup analysis, including assessing how deep learning affects accuracy among experienced readers across mild, moderate, and severe scoliosis. Based on preliminary data and existing literature, we hypothesize that experienced observers may achieve greater accuracy gains for severe, complex cases compared to mild or moderate ones, a hypothesis that future studies should test.

Finally, only a single expert orthopedic surgeon provided the reference standard for major coronal curve angle measurements. As most cases were mild or moderate, the use of a single reader as the reference standard may be acceptable (and is often the case in a true clinical scenario), but an expert panel is needed for complex cases to address inter- and intra-reader variability in Cobb angle measurement. Seah et al. (2021) [50] employed seven thoracic radiologists to establish ground truth for chest radiographs, showing that the optimal number of readers depends on data complexity; nevertheless, consensus among expert panels remains essential to mitigate inter- and intra-reader variability [51,52,53]. The inclusion of experts from multiple specialties (e.g., radiology and orthopedic surgery) may reduce bias. Future validation of this deep learning model in severe scoliosis should therefore involve multidisciplinary expert panels and larger test sets to assess inter- and intra-observer reliability.

In summary, future studies are required to address the generalizability gap, externally validate the results of our study, and confirm the utility of the deep learning model, when applied in a variety of real-world contexts, settings and populations. Further work could include additional features in the deep learning model; more extensive training sessions could also be conducted for the readers to enhance their understanding of the deep learning model’s interpretations [54]. Future studies should also prospectively evaluate the use of this deep learning model across multiple scoliosis screening centers where unassisted digital and physical measurements of the Cobb’s angle are both performed. The complete automation of the interpretation process, with the removal of human involvement, represents the ultimate goal of the development of these deep learning techniques, which will require successive, incremental steps to be achieved.

5. Conclusions

This study has demonstrated potential time savings for trainee radiologists utilizing a deep learning model (CaraNet) for measuring major coronal curve angles on posteroanterior scoliosis radiographs. Overall, trainee orthopedists with prior experience in the interpretation of scoliosis radiographs showed no time savings using the CaraNet model, and did not demonstrate improved reading accuracy, as compared to trainee radiologists. This study also demonstrated an interesting feature in behavior and implementation science, highlighting differences in potential bias between expert and non-expert readers.