Evaluation of Computer-Aided Detection (CAD) in Screening Automated Breast Ultrasound Based on Characteristics of CAD Marks and False-Positive Marks

The present study evaluated the effectiveness of computer-aided detection (CAD) system in screening automated breast ultrasound (ABUS) and analyzed the characteristics of CAD marks and the causes of false-positive marks. A total of 846 women who underwent ABUS for screening from January 2017 to December 2017 were included. Commercial CAD was used in all ABUS examinations, and its diagnostic performance and efficacy in shortening the reading time (RT) were evaluated. In addition, we analyzed the characteristics of CAD marks and the causes of false-positive marks. A total of 1032 CAD marks were displayed based on the patient and 534 CAD marks on the lesion. Five cases of breast cancer were diagnosed. The sensitivity, specificity, PPV, and NPV of CAD were 60.0%, 59.0%, 0.9%, and 99.6% for 846 patients. In the case of a negative study, it was less time-consuming and easier to make a decision. Among 530 false-positive marks, 459 were identified clearly for pseudo-lesions; the most common cause was marginal shadowing, followed by Cooper’s ligament shadowing, peri-areolar shadowing, rib, and skin lesions. Even though CAD does not improve the performance of ABUS and a large number of false-positive marks were detected, the addition of CAD reduces RT, especially in the case of negative screening ultrasound.


Introduction
Mammographic screening has reduced the rate of breast cancer mortality [1]. Recent guidelines for screening of breast cancer recommend mammography starting at age 45 or 50 years [2,3]. Although the incidence of breast cancer in Asian women is still lower than in Western countries, morbidity and mortality continue to increase in Asian countries [4]. The peak age of breast cancer in Asian countries is 40-49 years, whereas in Western countries the peak is around 60 to 70 years [5]. Asian women tend to have breasts with higher density compared with Western women [6]. Further, dense breast is an independent risk factor for developing breast cancer [7].
Real-time B-mode ultrasonography has emerged as an alternative imaging technique for breast cancer screening [8]. Ultrasound elastography can quantify stiffness distribution of tissue lesions and complements conventional B-mode ultrasonography. The development of computer-aided diagnosis has improved the reliability of the system, whilst the inception of machine learning, such as deep learning, has further extended its power by facilitating automated segmentation and tumor classification [9].
Automated breast ultrasonography (ABUS) was proposed as a supplementary screening modality recently, for increased cancer detection combined with digital mammography (DM), especially in dense breasts [10][11][12]. In addition, ABUS has been proposed in the diagnostic setting in a few recent studies [13].
However, due to the large number of images in a single scan, the reading time (RT) of a full ABUS examination can be prolonged and cancers may be easily overlooked [14]. For this reason, computer-aided detection (CAD) software for ABUS has been developed to facilitate the radiological interpretation of ABUS examinations [15]. Few studies investigated the effect of commercially available CAD systems for ABUS on the RT and screening performance of breast radiologists [16]. However, before using the CAD system clinically, it is necessary to analyze the characteristics of CAD marks. It could be useful for radiologists to have knowledge about the characteristics of CAD marks and the causes of false-positive marks.
In this study, we evaluated the effectiveness of computer-aided detection (CAD) system in screening automated breast ultrasound (ABUS) through diagnostic performance and reading time (RT). We also investigated and analyzed the characteristics of CAD marks and the causes of false-positive marks, to distinguish between true and false marks.

Materials and Methods
This retrospective study was approved by the institutional review board (IRB) of our institution. The need for informed consent was waived by the ethics committee due to the retrospective design. All procedures involving human participants were in accordance with the ethical standards of IRB issued by our institution, and assessments were carried out in accordance with the tenets of the Declaration of Helsinki of 1975, and its revision in 2013.

ABUS Acquisitions
The ABUS examinations were performed with the ACUSON S2000 Automated Breast Volume Scanner system (Siemens, Erlangen, Germany). This ABUS system acquires 3D Bmode ultrasound volumes over an area of 15.4 × 16.8 × 6 cm 3 volume data sets of the breast in one sweep using a mechanically driven linear array transducer (14L5). Adequate depth and focus can be obtained using predefined settings for different breast cup sizes. All ABUS examinations were performed by a single trained radiographer. To ensure coverage of the entire breast, three overlapping acquisitions including antero-posterior, medial, and lateral views were performed. The scan thickness was displayed at 1 mm intervals without overlap. A dedicated ABUS workstation was used to reconstruct the transverse slices into a 3D volume that can be read in a multiplanar hanging, with sagittal and coronal reconstructions.

CAD System
A prototype workstation was designed and developed specifically for high-throughput ABUS screening in this observer study (MeVis Medical Solutions, Bremen, Germany). In this prototype, each user action was logged with timestamps, which were subsequently used to estimate the time spent per case. The workstation was integrated with a commercially developed CAD software (QVCAD, Qview Medical Inc., Los Altos, CA, USA), which is designed to detect suspicious candidate regions in an ABUS volume highlighted with the so-called CAD marks ( Figure 1).
In addition, the QVCAD software provides an "intelligent" minimum intensity projection (MinIP) of the breast tissue in a 3D ABUS volume that can be used for rapid navigation through ABUS scans for enhancement of the possible suspicious regions. The CAD-based MinIP integrated with a multiplanar hanging protocol for ABUS displays the conventional ABUS planes. By clicking on the dark spot, the 3D multiplanar hanging automatically snaps to the corresponding 3D location. The crosshair is focused on a breast lesion that is marked by the CAD software with a green circular marker. The same lesion is also enhanced and visualized as a dark spot in the MinIP. A screenshot of the CAD-aided reading environment is presented in Figure 1. In addition, the QVCAD software provides an "intelligent" minimum intensity projection (MinIP) of the breast tissue in a 3D ABUS volume that can be used for rapid navigation through ABUS scans for enhancement of the possible suspicious regions. The CADbased MinIP integrated with a multiplanar hanging protocol for ABUS displays the conventional ABUS planes. By clicking on the dark spot, the 3D multiplanar hanging automatically snaps to the corresponding 3D location. The crosshair is focused on a breast lesion that is marked by the CAD software with a green circular marker. The same lesion is also enhanced and visualized as a dark spot in the MinIP. A screenshot of the CADaided reading environment is presented in Figure 1.
The number of CAD markers displayed per ABUS volume could be adjusted by changing the values of the false-positive rate (FPR) in the configuration setting of the CAD software. According to the manual from the manufacturer, FPR was defined as the total number of false-positive CAD markers in non-cancer volumes divided by the total number of non-cancer volumes. In this study, we set the FPR to 0.2 (i.e., 1 false-positive CAD marker in non-cancer volume per 5 non-cancer volumes), which was its default setting as in previous studies [16][17][18].

Study Design
The study included a total of 846 women aged 40-49 years who underwent ABUS screening from January 2017 to December 2017. The CAD (QVCADTM) system was used in all ABUS examinations and its diagnostic performance was evaluated retrospectively.
We analyzed whether CAD addition shortened the RT. The RT was determined by the expert breast radiologists based on their subjective perception in each of the following cases: (1) CAD with ABUS = ABUS only, (2) CAD with ABUS > ABUS only, (3) CAD with ABUS < ABUS only. We defined there is a difference when RT was shortened by more than 1 min. Furthermore, we analyzed the characteristics of CAD marks including the size of the marked lesion, lesion type (mass or non-mass), tissue composition under ultrasound, and The number of CAD markers displayed per ABUS volume could be adjusted by changing the values of the false-positive rate (FPR) in the configuration setting of the CAD software. According to the manual from the manufacturer, FPR was defined as the total number of false-positive CAD markers in non-cancer volumes divided by the total number of non-cancer volumes. In this study, we set the FPR to 0.2 (i.e., 1 false-positive CAD marker in non-cancer volume per 5 non-cancer volumes), which was its default setting as in previous studies [16][17][18].

Study Design
The study included a total of 846 women aged 40-49 years who underwent ABUS screening from January 2017 to December 2017. The CAD (QVCADTM) system was used in all ABUS examinations and its diagnostic performance was evaluated retrospectively.
We analyzed whether CAD addition shortened the RT. The RT was determined by the expert breast radiologists based on their subjective perception in each of the following cases: (1) CAD with ABUS = ABUS only, (2) CAD with ABUS > ABUS only, (3) CAD with ABUS < ABUS only. We defined there is a difference when RT was shortened by more than 1 min. Furthermore, we analyzed the characteristics of CAD marks including the size of the marked lesion, lesion type (mass or non-mass), tissue composition under ultrasound, and the causes of false-positive marks. The false-positive mark was defined as the mark located on the typical benign lesion or pseudo-lesions that require no additional studies following ABUS. The number of marks per patient and per lesion and the frequency of false-positive marks were also evaluated.
Two board-certified expert breast radiologists determined the characteristics of CAD marks based on consensus. In addition, the pseudo-lesions were also evaluated by two expert breast radiologists with consensus. The characteristics of pseudo-lesions were analyzed including the number, size, and location (right or left; antero-posterior, medial or lateral; upper, mid, or lower; inner, mid, or outer).
All women with suspicious lesions were recalled and US-guided 14G core-needle biopsy was performed. Patients who were not disease-positive were followed up in 2 years with radiologic examination using mammography or ultrasonography.
Based on the lesion type detected, the large mass lesions were more than the non-mass lesions (60 vs. 11). Based on tissue composition under ultrasound, the number of minimalto-mild cases in GTC was higher than moderate-to-marked cases (668 vs. 178). The rate of CAD positivity in moderate-to-marked lesions was higher than in minimal-to-mild. Table 1 summarizes the screening performance of CAD for ABUS per patient and per lesion. In the absence of the CAD mark, the readers determined that the reading time for CAD with ABUS was less than for ABUS only and easier to make a decision ( Table 2). Table 2 summarizes the number and characteristics of CAD marks per patient. Of 846 patients, 1032 CAD marks were marked in 534 lesions of 348 patients with a mean CAD mark per person of 0.8 (SD ± 1) (range 0-6) ( Table 3). No CAD mark was detected in 498 patients (48.3%).

Characteristics of All CAD Mark (n = 1032) Mean and median No. of CAD marks per patient
Among 530 false-positive marks, 459 marks were marked on the clear pseudolesions (Figures 2-4); the most common cause was marginal shadowing (209, 39.1%), followed by Cooper's ligament shadowing (143, 26.8%), peri-areolar shadowing (64, 12%), rib (37, 6.9 %), and skin lesions (6, 1.1%).      The false-positive marks on pseudo-lesions were frequently detected in the upper portion than in the mid-to-lower portion, and in the outer portion than in the mid-to-inner portion of breast (Table 4). There were more marks in the lateral view than in AP or medial views (Table 4).

Discussion
In this study, we evaluated the effectiveness of computer-aided detection (CAD) system in screening automated breast ultrasound (ABUS) through diagnostic performance and reading time (RT). A total of 846 patients displayed 1032 CAD marks and 534 CAD marks based on lesions. The sensitivity, specificity, PPV, NPV, and accuracy of CAD were  The false-positive marks on pseudo-lesions were frequently detected in the upper portion than in the mid-to-lower portion, and in the outer portion than in the mid-to-inner portion of breast (Table 4). There were more marks in the lateral view than in AP or medial views (Table 4).

Discussion
In this study, we evaluated the effectiveness of computer-aided detection (CAD) system in screening automated breast ultrasound (ABUS) through diagnostic performance and reading time (RT). A total of 846 patients displayed 1032 CAD marks and 534 CAD marks based on lesions. The sensitivity, specificity, PPV, NPV, and accuracy of CAD were 60.0%, 59.0%, 0.9%, 99.6% and 59.0% for 846 patients, respectively, while those of 1032 CAD marks were 60.0%, 48.3%, 0.6%, 99.6%, and 48.4%, respectively. The relatively higher NPV compared with other parameters indicates that the exam can be concluded with a negative study if no CAD mark is detected on ABUS. The presence of marks in multiple views did not suggest malignancy in this study. In the absence of the CAD mark, the readers determined that the reading time for CAD with ABUS was less than for ABUS only and easier to make a decision.
Several studies have reported that the performance of ABUS was comparable to that of hand-held ultrasound [20][21][22]. In addition, four prospective studies using ABUS demonstrated an increased cancer detection of 1.9-7.7 per 1000 examinations similar to hand-held ultrasound [10,11,14,23].
However, while the ABUS can yield standardized and structured images regardless of the experience of the operator, it takes much more time and effort to interpret the exams [24]. For this reason, the CAD system has been suggested as a supplementary method for interpreting ABUS results. However, the CAD system showed a high negative predictive value, and there were many false-positive CAD marks, which implied typical benign or pseudo-lesions that do not require further investigation. Usually, the falsepositive imaging results can affect the recall rate of the screening modality. The recall rate varied from 8.8% in the J-START study to 10.7% in the American College of Radiology Imaging Network (ACRIN) study [25,26]. However, few studies reported the characteristics of the causes of false-positive marks.
In addition to the diagnostic performance of CAD on ABUS, the previous studies evaluated the RT of CAD on ABUS [27][28][29]. Yang et al. reported that using CAD in the concurrent-reading mode, all readers saved 32% (16 s per 50 s per volume) in RT with a higher area under the receiver operating characteristic curve values compared with non-CAD mode [28]. Jiang et al. reported that although not all studies were interpreted faster with the CAD system, on average the savings were approximately 1 min per case [29]. In our study, it was less time-consuming and easier to make a clinical decision, especially in the case of a negative study.
In this study, we investigated and analyzed the characteristics of CAD marks and the causes of false-positive marks, to distinguish between true and false marks. Among 530 false-positive marks, 459 were identified clearly for pseudo-lesions; the most common cause was marginal shadowing, followed by Cooper's ligament shadowing, peri-areolar shadowing, rib, and skin lesions, all of which were easily distinguishable radiologically. The false marks for pseudo-lesions were detected more frequently in the upper rather than in the mid-to-lower portion and in the outer rather than in the mid-to-inner portion, probably because of bulkiness and flexibility of the upper and outer portion of the breast.
ABUS is a standardized examination with multiple advantages in both screening and diagnostic settings, including increased detection of breast cancer, improved workflow, and reduced examination time. However, ABUS has disadvantages and even some limitations. Disadvantages regarding image acquisition are the inability to assess the axilla, vascularization, and lesion elasticity. The limitations of interpretation include motion-or lesion-related artifacts due to poor positioning and the lack of contact [30]. In the review article about the pros and cons of ABUS by Ioana Boca et al., marginal shadowing and Cooper's ligament shadowing were defined as artifacts due to insufficient compression [30]. Peri-areolar shadowing is defined as a nipple artifact [30]. Despite the promising detection rate with CAD software in breast cancer, radiologists should determine whether a CAD software-marked lesion is a true-or false-positive lesion, given its positive predictive value and high false-positive rate [17]. The knowledge of these artifacts improves the diagnostic performance of radiologists.
There are several limitations to this study. First, we used only image data obtained with equipment from a single vendor, with a small number of participants. In addition, this study was performed only in academic institutions by a limited number of users, boardcertified expert breast radiologists, and does not represent varying clinical environments. Second, the absence of the numerical result of RT is the limitation of this study. The RT was determined by the expert breast radiologists based on their subjective perception. Finally, in our study, the expert radiologists' decision was a gold standard for suspicious lesions or pseudo-lesions. However, a large number of marks await the radiologist's rational judgment. Therefore, CAD users should be familiar with marks in various situations before using them, and the review summarizes the characteristics of CAD marks only without radiological evaluation. The knowledge of the characteristics of CAD marks and the causes of false-positive marks could improve the diagnostic performance of radiologists.

Conclusions
In conclusion, even though CAD addition does not improve the performance of screening ABUS and is associated with a large number of false-positive marks, CAD addition improves the negative predictive value and reduces RT, especially for negative screening ultrasound.