Predicting the Risk of Malignancy of Lung Nodules Diagnosed as Indeterminate on Radial Endobronchial Ultrasound-Guided Biopsy

The next diagnostic step in cases of indeterminate radial probe endobronchial ultrasound (radial EBUS)-guided biopsy results remains uncertain. This study aimed to identify risk factors for malignancy based on clinical findings, chest computed tomography (CT), and radial EBUS images, and to estimate the risk of malignancy in lung nodules that showed indeterminate radial EBUS-guided biopsy results by constructing a nomogram. This retrospective study included 157 patients with indeterminate results on an initial radial EBUS biopsy performed at the Samsung Medical Center from January 2017 to December 2018, but with a definitive final diagnosis. Medical records, chest CT, radial EBUS images, and the final diagnoses were reviewed. Patients were randomly divided into training and validation sets. Factors related to malignancy were identified through logistic regression analysis, and a nomogram was constructed using the training set and subsequently applied to the validation set. Six factors in univariable and multivariable analyses, including upper lobe location, spiculation, satellite nodules, echogenicity, presence of dots or linear arcs, and patency of vessels and bronchi predicted malignancy. A nomogram was constructed based on these predictors. The area under the curve (AUC) value of the nomogram was 0.858 using the chest CT factors, which improved to 0.952 when radial EBUS factors were added. The calibration curve showed good agreement between the actual and nomogram-predicted malignancy outcomes. The utility of radial EBUS images for revealing risk factors of malignancy was confirmed. Furthermore, our nomogram was able to predict the probability of malignancy in lung nodules with indeterminate radial EBUS-guided biopsy results.


Introduction
Diagnosing peripheral pulmonary nodules remains a significant challenge for pulmonologists. Several quantitative models have been developed to estimate the pretest probability of malignancy of pulmonary nodules, to expedite treatment of malignant nodules and minimize procedures for benign ones [1]. Classical models for determining the probability of malignancy constructed by the Mayo Clinic [2], the Veterans Association (VA) [3], and Brock University [4] include clinical factors, such as age, smoking history, cancer history, and radiological characteristics on chest computed tomography (CT), such as the diameter and location of the nodule. Many attempts have been made to identify risk factors for malignancy of pulmonary nodules, which may inform the next diagnostic step.
With the detection of pulmonary nodules rising, this has led to improve diagnostic sensitivity of advanced bronchoscopic techniques, including radial probe endobronchial ultrasound (radial EBUS) [5]. In a meta-analysis performed in 2017, the overall diagnostic yield for radial EBUS was 70.6% [6].
Apart from its role in improving the bronchoscopic biopsy yield for peripheral lung nodules, the image findings of radial EBUS provide clues regarding the nature of the lesions [7]. Kurimoto et al. [8] analyzed radial EBUS images by focusing on internal echoes, vascular and bronchial patency, and the morphology of hyperechoic areas, and correlated the results with the histopathologic findings of surgical resection. Their research suggested that radial EBUS images can provide additional information, thus enhancing predictions of the malignancy potential of lesions relative to previously identified clinical and chest CT findings.
However, there are limitations when interpreting radial EBUS-guided biopsy results. In clinical practice, when the biopsy result is "positive for malignancy", clinicians can manage the nodules as cancer. On the other hand, remaining "indeterminate" results, which may involve atypical cells, fibrosis, and inflammation, do not completely exclude the possibility of cancer. There is no consensus regarding whether invasive procedures, such as biopsy or surgery, or clinical surveillance should be applied in such cases. The best diagnostic approach in cases with indeterminate results has not been determined.
In this study, we hypothesized that radial EBUS image findings may provide useful information for predicting the risk of malignancy in lung nodules and focused on the malignant potential of indeterminate radial EBUS-guided biopsy results. First, along with identifying risk factors for malignancy from clinical findings and chest CT, we also examined radial EBUS images. Second, based on the identified risk factors, we developed a nomogram to estimate the risk of malignancy in indeterminate radial EBUS-guided biopsy results, which enables us to identify patients who may receive greater benefit from additional procedures.

Study Patients
We retrospectively reviewed the medical records of 494 patients who underwent radial EBUS biopsy at the Samsung Medical Center, Seoul, Korea from January 2017 to December 2018. Among these patients, 212 initial radial EBUS-guided biopsy results were indeterminate. Eight patients were excluded due to missing radial EBUS images or poor image quality. An additional 47 patients without a definitive diagnosis were excluded, leaving a total of 157 patients included in this study. Definitive diagnoses were those (1) pathologically confirmed as cancerous by surgical specimen analysis, endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA), a core needle biopsy, or additional radial EBUS, or (2) with culture results of sputum or bronchial washing that revealed tuberculosis or atypical mycobacteria with adequate antibiotics response, or (3) with follow-up chest CT suggesting resolved or improved lesions with reassessment of radiologist confirming the possibility of benign. In these cases, the patients were observed for at least 6 months.
The Institutional Review Board of Samsung Medical Center approved this study and permitted the review and publication of patient records (IRB No. 2019-03-106). The requirement for informed consent was waived given the retrospective nature of the study.

Variables
Demographics, clinical characteristics and final diagnoses were obtained through electronic chart review. The clinical factors included age, sex, cancer history, and smoking history. All chest CT images were reviewed by a pulmonologist and a radiologist to determine the maximum diameter, location (upper lobe), type (solid or sub-solid), and shape (round or complex) of nodules, presence of emphysema with nodules, spiculation, bronchus sign, and presence of satellite nodules. Satellite nodules were defined as small discrete shadows near the main lesion [9]. All radial EBUS image findings were evaluated for the following features: echogenicity (homogeneous or heterogeneous); attenuation (maintenance of echogenicity considering midline as a fiducial line); margin (regular or irregular); shape (round or complex); presence of dots or linear arcs; and patency of vessels and bronchi.

Statistical Analysis
To develop a nomogram for predicting the malignant potential of lung nodules that showed indeterminate biopsy results on radial EBUS, a model was built using a training set and a validation set. For this, independent cohorts were devised by randomly dividing the patients into training (70%) and validation sets (30%).
The nonparametric Mann-Whitney U test was used to compare continuous variables, and the chi-square test or Fisher's exact test was used to compare categorical variables between the benign group and malignant group in the total data and between training set and validation set.
Univariable logistic regression analysis of the training set was used to identify potential predictors of malignancy. Using the variables with the result of p < 0.1 from univariable logistic regression, backward selection multivariable logistic regression analysis was used to analyze the associations among potential predictors. Selected factors with statistical significance were used to develop the nomogram.
The ability of the nomogram to discriminate (i.e., distinguish among patients with different outcomes) and calibrate (i.e., the degree of correspondence between predictions and actual outcomes) was quantified [10][11][12]. Discrimination performance was quantified based on the area under the curve (AUC). Calibration performance was measured by plotting the predicted malignancy against the actual outcomes, as tertiles, with 1000 bootstrap resamples [1,13].
In addition, the AUC of the nomogram constructed based only on chest CT factors was compared with that constructed using chest CT and radial EBUS factors, to verify the radial EBUS features as risk factors for malignancy.
All reported p values were two-sided and p < 0.05 was considered statistically significant. All analyses were performed using R software (version 3.2.2; R Foundation for Statistical Computing, Vienna, Austria).

Results
Among the 494 patients who underwent radial EBUS biopsy, 157 patients with indeterminate radial EBUS-guided biopsy results were included in this study ( Figure 1). The indeterminate results included fibrosis, inflammation, atypical cells, and negative for malignancy. Among the 157 patients, 90 were additionally diagnosed as malignant using other modalities, such as repeat radial EBUS, EBUS-TBNA, a core needle biopsy or surgical specimen analysis. The remaining 67 were diagnosed as benign based on culture results or radiological follow-up for at least 6 months.
The clinical characteristics, chest CT and radial EBUS image features of 157 patients with indeterminate radial EBUS-guided biopsy results are described in Table 1. Overall, 91 (58.0%) males and 66 (42.0%) females, with a mean age of 66.0 years, were included in the study. The prevalence of malignant lung nodules was 57.3%. In total, 108 (70%) of 157 lung nodules were randomly included in the training set. No significant difference was observed in the clinical, chest CT, or radial EBUS factors between the training and validation sets ( Table 2). The clinical characteristics, chest CT, and radial EBUS features of the training set are described in Table 3. Table 4 summarizes the results of the univariable and multivariable logistic regression analyses of the clinical, chest CT and radial EBUS factors in the training set. The multivariable analysis revealed that chest CT factors, including upper lobe location, spiculation, and absence of satellite nodules were associated with malignancy. Furthermore, malignant nodules tended to be more heterogeneous in terms of echogenicity, had more dots or linear arcs, and had vessels and bronchi that were not patent on radial EBUS images. However, no significant differences were observed in clinical factors between malignant and non-malignant cases.     The clinical characteristics, chest CT and radial EBUS image features of 157 patients with indeterminate radial EBUS-guided biopsy results are described in Table 1. Overall, 91 (58.0%) males and 66 (42.0%) females, with a mean age of 66.0 years, were included in the study. The prevalence of malignant lung nodules was 57.3%.  A nomogram was constructed using the six factors related to malignancy identified by multivariable regression (Figure 2). Through this nomogram, it became possible to calculate the probability of malignancy of lung nodules diagnosed as indeterminate.
J. Clin. Med. 2020, 9, x FOR PEER REVIEW 9 of 13 A nomogram was constructed using the six factors related to malignancy identified by multivariable regression (Figure 2). Through this nomogram, it became possible to calculate the probability of malignancy of lung nodules diagnosed as indeterminate. Nomogram for predicting the probability of malignancy. Nomogram for predicting the probability of malignancy of pulmonary nodules diagnosed as indeterminate on initial radial EBUSguided biopsy, based on clinical, chest CT, and radial EBUS factors. To obtain the nomogrampredicted probability, chest CT and radial EBUS factors of the patient must be placed on a nomogram scale. Then by drawing a vertical line up to the "Points" axis, corresponding prognostic points are obtained. Using the sum of the point acquired from each factor, another vertical line can be drawn from the "Total points" axis down to the "Probability of malignancy" axis, providing the probability of lung cancer.
Based on analysis of the training set, the AUC for the nomogram was 0.952 (95% confidence interval [CI] 0.914-0.990) (Figure 3). Based on analysis of the validation set, discrimination performance of the nomogram was good (AUC = 0.836; 95% CI 0.711-0.960). Calibration plots of the training and validation sets indicated good agreement between the predicted and observed outcomes, with close correspondence between the predicted and actual malignancy outcomes. Nomogram for predicting the probability of malignancy. Nomogram for predicting the probability of malignancy of pulmonary nodules diagnosed as indeterminate on initial radial EBUS-guided biopsy, based on clinical, chest CT, and radial EBUS factors. To obtain the nomogram-predicted probability, chest CT and radial EBUS factors of the patient must be placed on a nomogram scale. Then by drawing a vertical line up to the "Points" axis, corresponding prognostic points are obtained. Using the sum of the point acquired from each factor, another vertical line can be drawn from the "Total points" axis down to the "Probability of malignancy" axis, providing the probability of lung cancer.
Based on analysis of the training set, the AUC for the nomogram was 0.952 (95% confidence interval [CI] 0.914-0.990) (Figure 3). Based on analysis of the validation set, discrimination performance of the nomogram was good (AUC = 0.836; 95% CI 0.711-0.960). Calibration plots of the training and validation sets indicated good agreement between the predicted and observed outcomes, with close correspondence between the predicted and actual malignancy outcomes.
scale. Then by drawing a vertical line up to the "Points" axis, corresponding prognostic points are obtained. Using the sum of the point acquired from each factor, another vertical line can be drawn from the "Total points" axis down to the "Probability of malignancy" axis, providing the probability of lung cancer.
Based on analysis of the training set, the AUC for the nomogram was 0.952 (95% confidence interval [CI] 0.914-0.990) (Figure 3). Based on analysis of the validation set, discrimination performance of the nomogram was good (AUC = 0.836; 95% CI 0.711-0.960). Calibration plots of the training and validation sets indicated good agreement between the predicted and observed outcomes, with close correspondence between the predicted and actual malignancy outcomes.  (C) Calibration curve for the relationship between the predicted and actual malignancy outcomes. The x-axis represents the nomogram-predicted malignancy probability, and the y-axis indicates the actual outcome. The "calibration curve" (dotted line) closely approximates the "logistic calibration curve" (solid line), demonstrating good agreement between the nomogram-predicted and actual malignancy outcomes for indeterminate lung nodules. AUC = area under the curve; CI = confidence interval.
In addition, as shown in Figure 4, the AUC improved from 0.858 to 0.952 after adding the radial EBUS factors to the nomogram, verifying the role of radial EBUS features as risk factors for malignancy. In addition, as shown in Figure 4, the AUC improved from 0.858 to 0.952 after adding the radial EBUS factors to the nomogram, verifying the role of radial EBUS features as risk factors for malignancy.

Discussion
Our study was motivated by the lack of definitive guidelines for proceeding in cases with "indeterminate" radial EBUS-guided biopsy results, which do not exclude malignancy. We focused on the clinical utility of radial EBUS images for distinguishing between malignant and benign lesions, which has also been discussed in previous studies [7, 8,14].
First, we evaluated the radial EBUS images of pulmonary nodules, as well as clinical findings and chest CT images, to determine diverse risk factors for malignancy. Based on previous radial EBUS

Discussion
Our study was motivated by the lack of definitive guidelines for proceeding in cases with "indeterminate" radial EBUS-guided biopsy results, which do not exclude malignancy. We focused on the clinical utility of radial EBUS images for distinguishing between malignant and benign lesions, which has also been discussed in previous studies [7,8,14].
First, we evaluated the radial EBUS images of pulmonary nodules, as well as clinical findings and chest CT images, to determine diverse risk factors for malignancy. Based on previous radial EBUS image studies [7,8,14], echogenicity, attenuation, margin, nodule shape, the presence of dots or linear arcs, and the patency of vessels and bronchi were evaluated on radial EBUS images. Logistic regression analysis revealed that echogenicity, the presence of dots or linear arcs, and the patency of vessels and bronchi were significantly associated with malignancy.
Echogenicity on radial EBUS images is closely associated with the arrangement of cells and the quantity of fibrous stroma [7]. Several studies [15][16][17][18] on gastrointestinal tract lesions have revealed that heterogeneous echogenicity is more likely to indicate malignancy than homogeneous echogenicity. It is believed that loss of normal tissue and chaotic tumor cell growth with central fibrosis, necrosis, and hemorrhage leads to heterogeneity in the internal structure of lung masses on EBUS images [8]. Dots and linear arcs denote an irregular hyperechoic pattern within lesions, primarily around the radial probe [8]. Kurimoto et al. [8] analyzed radial EBUS images and surgical specimens and found that dots and linear arcs correspond to residual air in the alveoli, which is characteristic of a well-differentiated adenocarcinoma, indicating malignancy. The patency of vessels and bronchi is believed to reflect the degree to which anatomic structures are preserved in cases of lung parenchyma. Tumor cells grow to form a solid mass invading the lung parenchyma in cases of peripheral lung cancer, and the patency of the bronchi seldom remains intact as the tumor increases in volume [18]. Therefore, a lesion with patent vessels and bronchi based on radial EBUS probe is likely to be benign, as in our results.
According to our results, an upper lobe location, spiculation, and an absence of satellite nodules were, withal, the chest CT factors associated with malignancy. An upper lobe location and spiculation of pulmonary nodules are well known independent predictors of malignancy [2,4,19]. The National Lung Screening Trial data set showed 65.7% of all lung cancers were located in the upper lobe [20]. Furthermore, the result of NELSON trial [21], the largest European lung cancer CT screening trial, observed that 64.1% of all lung cancers were localized in the upper lobe. Especially, 45.0% were localized in the right upper lobe. This is a known phenomenon explained by the fact that the airflow at the beginning of the breath is the largest toward the right upper lobe bronchus that eventually causes more deposition of particles in tobacco smoke and their carcinogenic effects in the right upper lobe [22,23]. Spiculation is caused by an obstruction of pulmonary vessels, or by the filling of lymphatic channels with tumor cells; in turn, this results in interlobular septal thickening and fibrosis, and is highly predictive of malignancy (positive predictive value of up to 90%) [24,25]. In contrast, the satellite lesions frequently observed in patients with tuberculoma, or accompanied by organizing pneumonia, tended to be benign in a tuberculosis (TB)-endemic area [26]. In that study, in cases with satellite lesions around a solitary pulmonary nodule, the rate of benign status was 90.9%.
Based on the significant predictors of malignancy in univariable and multivariable analyses, we constructed a nomogram to calculate the probability of malignancy of pulmonary nodules with indeterminate radial EBUS-guided biopsy results. Our results are expected to be useful for patients and clinicians with respect to the decision between an invasive diagnostic procedure or observational follow-up.
The average predictive accuracy regarding the malignancy of pulmonary nodules was reported as 80% using current models based on clinical findings and chest CT factors [2,3]. In our study, the AUC of the nomogram using only chest CT factors was 0.858, but this improved to 0.952 after adding the radial EBUS factors. This confirmed the potential of radial EBUS factors for predicting malignancy.
Our study had several limitations. First, since our cohort comprised only patients with indeterminate radial EBUS-guided biopsy results, it showed a relatively homogeneous nature. For this reason, although several studies [2][3][4] have demonstrated the importance of clinical factors such as age, smoking history, and cancer history for predicting malignancy, these factors were not significantly associated with malignancy in our study. Furthermore, our nomogram could only be applied to patients with indeterminate radial EBUS-guided biopsy results and could not be used in calculating the malignant potential of general patients with pulmonary nodules. Second, even though our nomogram calculated the probability of malignancy of indeterminate radial EBUS-guided biopsy results, precise cut-off values indicating the need for an additional invasive procedure are still lacking; further studies are, therefore, required. British Thoracic Society guidelines [19] suggest patients with a low risk (<10%) of malignancy should undergo CT surveillance. When the risk is moderate-to-high (>10%), additional testing should include fine-needle aspiration, endobronchial ultrasonography, and positron emission tomography. Surgery is recommended for high risk (>70%) patients. Since our study included not only clinical and chest CT factors, but also radial EBUS factors, larger population-based studies should be performed to determine cut-offs specifically for radial EBUS. Third, this study was performed at a single center; further studies including multiple centers and larger populations and controlling of operator factors are needed to validate our results. Fourth, although presence of the satellite nodules was found to be highly predictive of benign pulmonary nodules, this finding was restricted to TB-endemic areas and may not apply elsewhere. Finally, this is a retrospective study that requires prospective validation, and a constructed nomogram must be confirmed through other group of patients with indeterminate radial EBUS-guided biopsy results. For this reason, a further study involving patients that had undergone radial EBUS from 2019 to 2020 in our center, is in progress to apply and validate our nomogram.
In conclusion, we identified six risk factors for malignancy based on analysis of radial EBUS images, as well as clinical findings and chest CT. Thus, radial EBUS may provide additional information to complement other clinical and radiological data. Furthermore, we developed a nomogram to estimate the probability of malignancy in patients with indeterminate initial radial EBUS findings, which may facilitate identification of patients at higher risk for malignancy.