The Effectiveness of Semi-Supervised Learning Techniques in Identifying Calcifications in X-ray Mammography and the Impact of Different Classification Probabilities

Abstract

Identifying calcifications in mammograms is crucial for early breast cancer detection, and semi-supervised learning, which utilizes a small dataset for supervised learning combined with deep learning, is anticipated to be an effective approach for automating this identification process. This study explored the impact of semi-supervised learning on identifying mammographic calcifications by including 712 mammographic images from 252 patients in public datasets. Initially, 212 mammogram images were segmented into patches and classified visually for calcification presence. A subset of these patches, derived from 169 mammogram images, was used to train a ResNet50-based classifier. The classifier was evaluated using patches generated from 43 mammograms as a test data set. Additionally, 500 more mammogram images were processed into patches and analyzed using the trained ResNet50 model, with semi-supervised learning applied to patches exceeding certain classification probabilities. This process aimed to enhance the classifier’s accuracy and achieve improvements over the initial model. The findings indicated that semi-supervised learning significantly benefits the accuracy of calcification detection in mammography, underscoring its utility in enhancing diagnostic methodologies.

1. Introduction

Cancer is one of the leading causes of death worldwide. Breast cancer is the most common cancer in women and the leading cause of cancer-related death among women [1]. Early diagnosis is very important because breast cancer has a high chance of cure if it is diagnosed early and before it has metastasized [2]. A variety of imaging modalities are used to diagnose breast cancer, including mammography, magnetic resonance imaging (MRI), ultrasonography, histopathology, computed tomography (CT), positron emission tomography (PET), and thermography [3].

Mammography is one of the modalities used for breast cancer imaging and is routinely used. Mammography has been reported to detect approximately 80~90% of asymptomatic breast cancer cases [4], significantly reducing patient mortality [5]. When radiologists read screening mammograms, they look primarily for soft tissue findings and calcification clusters. Soft tissue findings can vary from masses (having various shapes and edges, such as smooth, spiny, indistinct, or irregular), asymmetry (dense tissue in one breast with no counterpart in the opposite breast), and disorganized architecture (abnormal composition of fibroglandular tissue) [6]. Calcification is caused by the deposition of calcium oxalate and calcium phosphate in the breast tissue and appears as bright white spots on mammographic images [7]. Calcifications may indicate breast cancer or precancerous changes in breast tissue, and the morphology and distribution of calcifications are important biomarkers of malignancy [6]. Benign calcifications are generally large, coarse, round, and smooth at the edges, whereas malignant calcifications are generally small and often referred to as microcalcifications [8]. Clustered microcalcifications are found in 30–50% of the cancer cases diagnosed by mammography [4]. However, they are difficult to detect and may be overlooked in breast cancer screening due to their size (approximately 0.1–1 mm) and dense surrounding mammary tissue [4,9]. Typical benign calcifications include cutaneous calcifications, vascular calcifications, coarse or popcorn-shaped calcifications, thick rod-shaped calcifications, round calcifications, oval or rim-shaped calcifications, milk of calcium calcifications, and suture calcifications. Calcifications of moderate concern include amorphous or indistinct, coarse, and heterogeneous calcifications. Calcifications of high malignant potential include those that are grouped or clustered, fine, linear, branched, pleomorphic, or numerous, and they are amenable to biopsy [8]. The distribution of the calcification indicates how the calcification is distributed throughout the breast, including diffuse/scattered, regional, clustered, linear, and segmental. In general, diffuse/scattered and regional calcifications are benign, while clustered calcifications suggest intermediate malignant potential, and linear and segmental calcifications are more likely to be malignant. Therefore, clustered, linear, and zonal distributions of calcifications are important [8,10]. Thus, the morphology and distribution of calcifications are useful in differentiating benign from malignant disease [10], and it is very important for radiologists to detect and evaluate calcifications [8]. In a study on mammography imaging, researchers determined that Rh filtration and a W anode significantly improved image quality and dose efficiency compared to a Mo anode and Mo filtration [11]. In this way, various studies are being conducted in mammography, from imaging to diagnosis.

In recent years, deep learning techniques have been used in a variety of fields [12,13,14,15]. There have been various studies using deep learning techniques for breast cancer, especially in the field of mammography [16]. Deep learning has been used in mammography not only for the detection and classification of microcalcifications and masses [17,18], but also for mammography-based risk prediction and stratification [19], the detection and classification of abnormal findings [20], breast density classification [21], and many other applications. However, CNNs (Convolution Neural Networks), used in deep learning, have a limit to the input matrix size. Therefore, when mammography images with a large matrix size are compressed to the CNN input size, information on small lesions such as small masses and microcalcifications may be lost. By segmenting the mammography image into patches, it is possible to input the image with the original resolution, and each patch generated by the segmentation can be classified according to the presence or absence of calcification, enabling the detection of calcification [22]. However, training with deep learning requires a large amount of data, and labeling the large number of patches generated by patch segmentation is time-consuming and labor-intensive. Therefore, we thought that semi-supervised learning would make it possible to learn efficiently from a small amount of labeled data.

Machine learning is generally divided into supervised learning, unsupervised learning, and semi-supervised learning, with the main difference being the presence or absence of labels on the training data [23]. Semi-supervised learning is a method that falls between supervised learning, which uses only labeled data, and unsupervised learning, which does not use labeled data [24]. Most research using deep learning in medical imaging focuses on supervised learning, which requires the large-scale labeling of data by experts. However, labeled data are difficult to obtain, whereas unlabeled data are increasing every day and can be prepared in large quantities and easily [25]. Semi-supervised learning combines unlabeled data with a limited amount of labeled data to reduce the effort of labeling, which is a time-consuming task, and to improve model performance by making use of unlabeled data [24]. This technique is often used, especially when limited labeled data and large unlabeled data are available [26]. The utility of semi-supervised learning has been tested in a variety of medical imaging applications [25,27].

Due to the limitations of the input size of the CNN, compressing mammographic images may result in the loss of small lesions. However, by dividing the images into patches, they can be input at their original resolution, allowing for the detection of calcifications by classifying each patch for the presence or absence of calcifications. Labeling a large number of generated patches is time-consuming and labor-intensive, so we considered that semi-supervised learning, which leverages a small amount of labeled data, would be effective. There have been several studies using semi-supervised learning for mammography [28,29]. Additionally, this study examined the differences in accuracy due to varying classification probabilities (threshold differences) in semi-supervised learning. No previous studies have been found that investigated the differences in classification accuracy for the presence or absence of calcifications in mammograms based on varying classification probabilities in semi-supervised learning. In this study, we investigate the effect of semi-supervised learning on mammographic calcification detection.

2. Related Work

The development and application of deep learning techniques for breast cancer diagnosis, especially in the field of mammography, have seen significant advancements in recent years. Initially, traditional machine learning techniques were widely employed, utilizing algorithms such as Multi-class Support Vector Machines (SVMs) and K-Means clustering. These methods focused on the statistical analysis of features extracted manually from images, providing a foundational approach for early classification efforts [30]. As the field progressed, the advent of convolutional neural networks (CNNs) marked a significant shift, with architectures such as MobileNetv2 and ResNet offering more sophisticated image analysis capabilities through deep learning, enabling automatic feature extraction directly from raw images [31,32].

Further advancements led to the integration of segmentation techniques alongside CNNs, enhancing the precision of classifications. Methods such as U-Net began to be employed for isolating lesions from surrounding tissue before classification, improving the accuracy by focusing the CNNs on relevant image segments only [33]. The culmination of these developments has been the adoption of multi-input CNN models that integrate segmented image data with patient metadata, using diverse architectures to capture a broad spectrum of mammographic features effectively. Researchers have explored various strategies to optimize the accuracy and efficiency of these models, particularly through the use of multi-input systems that combine different data types and processing techniques [34].

The incorporation of multi-input CNN models marks a pivotal advancement in mammographic imaging. These models integrate multiple data streams, such as high-resolution images, extracted regions of interest, and additional contextual information, to enhance diagnostic precision [35]. Some studies propose end-to-end models that maximize the use of high-resolution images through a multi-input strategy, including cropping, original image downsampling, and region-of-interest extraction using CAM, supplemented by attention mechanisms to enhance classification accuracy [36].

The adoption of semi-supervised learning techniques in mammographic imaging represents another significant advancement. Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data, which helps to improve model performance while reducing the effort of data labeling. This approach is particularly useful in medical imaging, where obtaining labeled data is often expensive and time-consuming [37,38]. In recent years, semi-supervised learning has been increasingly applied to mammographic image analysis, further enhancing the effectiveness and efficiency of deep learning models in this domain [39].

Therefore, in addition to the related work mentioned above and the studies on mammography discussed in the Introduction, the novelty of this study, compared to previous research, can be demonstrated (

Table 1. Novelty for this study compared to previous methods.

Previous Methods for Mammography	Novelty for This Study
-Detection of soft tissue findings	Utilization of semi-supervised learning for mammography
-Detection rate of 80–90% [4]	-Reduction of labeling effort
-Detection of calcification clusters [8,10]	-Improvement in classification accuracy
-Difficulty in detecting microcalcifications [4,9]	-Investigation of classification probability thresholds
-Information loss due to compression	-New approach to classification accuracy
-High labeling effort	-Combination of patch division and semi-supervised learning

) by the following aspects: the utilization of semi-supervised learning for mammography, the reduction of labeling effort, the improvement in classification accuracy, the investigation of classification probability thresholds, a new approach to classification accuracy, and the combination of patch division and semi-supervised learning.

3. Materials and Methods

3.1. Subjects

This study used image data from the Categorized Digital Database for Low Energy and Subtracted Contrast Enhanced Spectral Mammography images (CCD-CESM) [40] published in The Cancer Imaging Archive (https://www.cancerimagingarchive.net/ accessed on 21 May 2024). Of the 326 mammography image datasets included in this database, 712 low energy images of 252 CDD-CESM images were used. A breakdown of the image data used is shown in

Table 2. Number of mammography images in this study.

	Right		Left
	CC a Image	MLO b Image	CC Image	MLO Image
Number of images	173	174	183	182

^a Craniocaudal; ^b mediolateral oblique.

.

Table 3. Software and equipment used in this study.

Environment	Contents
Software	MATLAB 2023a (MathWorks)
OS	Windows 11
CPU	Intel Core i9-10920X 3.50 GHz
GPU	NVIDIA Quadro P5000 16 GB × 4
Memory	DIMM 2666 MHz 64.0 GB

shows the specifications of the PCs used in this study. MATLAB (MATLAB 2023a; The MathWorks, Inc., Natick, MA, USA) was used as the numerical analysis software.

3.2. Data Preprocessing

The 212 mammogram images (average matrix size: 2330 × 1310 pixels) were divided into 224 × 224 pixel patches, which is the input size of ResNet. The right breast image was divided into 224 pixel patches starting from the lower right side, the left breast image was divided into 224 pixel patches starting from the lower left side, and the missing areas were filled with zeros. The 15,049 patches were then visually classified according to the presence or absence of calcification, as shown in

Table 4. Number of classifications of the presence or absence as visual calcification.

Calcification	Number of Images
Yes	1029
No	14,020

. The calcification in the patches was visually classified and confirmed by four radiological technologists, each with varying levels of experience: one with more than 20 years, one with 10 years, one with 5 years, and one with less than 5 years of technician licensure. The criteria used for the presence or absence of calcification are identical to the data published in a previously submitted paper [22].

Among the classified patches, the patches generated from 169 mammographic images were used as training data, and the patches generated from the remaining 43 mammographic images were used as test data.

Table 5. Number of images with and without calcification in the training and test data.

	Training Dataset		Test Dataset
	without Calcification	with Calcification	without Calcification	with Calcification
Number of images	10,668	835	3352	194

shows a breakdown of the number of patches in the training and test data.

A classifier trained on unbalanced training data is likely to be biased toward the class with the largest number of images in the training data. Therefore, a model created with training data that have more patches without calcification than patches with calcification is more likely to classify patches as without calcification, which may lead to missing calcifications. Therefore, data augmentation was performed. The patches without calcification were inverted and doubled. The patches with calcification were rotated by 90°, 180°, and 270° respectively. The original patch was inverted, and the inverted patch was rotated by 90°, 180°, and 270°, respectively. The pixel luminance of each patch was then multiplied by 0.5 and 0.75 to expand the number of patches with calcification by a factor of 24. Figure 1 shows a schematic of data augmentation, and

Table 6. Number of original and augmented images with and without calcification in training data.

	Training (without Calcification)		Training (with Calcification)
	Original Data	Augmented Data	Original Data	Augmented Data
Number of images	10,668	21,336	835	20,040

below shows the changes in image data resulting from data augmentation.

In this study, we used a pretrained ResNet 50 [32] for transfer learning in ImageNet. The post-expansion training data were trained on ResNet50 to create a classifier that performs a two-class classification of the presence or absence of calcification.

Table 7. CNN and parameters used to create the classifiers.

	Parameters
CNN a	ResNet50
Mini batch size	128
Max epochs	10
optimizer	SGDM b
Initial learning rate	0.001

^a Convolutional neural network; ^b stochastic gradient descent with momentum.

shows the parameters used to create the classifiers.

3.3. Semi-Supervised Learning

The 500 new mammographic images were divided into 224 × 224 pixel patches, and the 38,211 unlabeled patches generated were used to infer the presence or absence of calcification with ResNet50, which had been trained on the labeled data. The number of patches was then determined by extracting the patches when the classification probabilities exceeded 0.80, 0.85, 0.90, 0.95, and 1.00. The patches that satisfied each classification probability underwent the following data augmentation processes. Initially, the brightness of the pixels in the original patch was reduced to 0.5 and 0.75 times their original value. Additionally, the original patch was rotated by 90°, 180°, and 270°, respectively. Then, after flipping the original patch, the flipped patch was also rotated by 90°, 180°, and 270°. Through these image augmentation processes, the number of patches was increased tenfold. The extended patches with high classification probabilities were then used for transfer learning to update existing classification models through semi-supervised learning. This was intended to incorporate the feature from the confidently classified unlabeled patches, aiming to improve the model’s performance and generalization capabilities without requiring additional labeled data. The number of transfer-learned patches was calculated for each classifier. The patches without calcification were extracted to match the number of patches with calcification that satisfied the classification probability so that the number of patches with calcification and without calcification used for transfer learning would be equal. The patches without breast tissues were excluded. Figure 2 shows a schematic of the transfer learning process used in this study.

3.4. Evaluation of Created Models

3.4.1. Accuracy Evaluation of Each Classifier

To evaluate the accuracy of the classifiers created by transfer learning, the initial classifier and the patches satisfying the respective classification probabilities (0.80, 0.85, 0.90, 0.95, and 1.00), the recall, precision, overall accuracy, area under the curve (AUC), and F1 score were calculated using the test data in Table 5. The test data in Table 5 were used to evaluate the accuracy of the classifiers. Each index was calculated as follows: true positive (TP), which predicts a positive class for those in the positive class; false positive (FP), which predicts a positive class for those in the negative class; true negative (TN), which predicts a negative class for those in the negative class; and true negative (TN), which predicts a negative class for those in the positive class; and false negative (FN), which predicts a negative class for those in the positive class. The definitions and formulas for each are given below.

(1)

Recall

Recall is an index that indicates the ratio of data that can be judged as positive by artificial intelligence to the total number of actual positive cases (TP + FN). The higher the recall value, the better the performance, as it indicates fewer missed classifications.

$$Recall=\frac{TP}{TP+FN}$$

(2)

Precision

Precision is an index that indicates the percentage of data that are actually positive among the data (TP + FP) judged positive by artificial intelligence. The higher the value, the better the performance of the model, as it indicates fewer misclassifications.

$$Precision=\frac{TP}{TP+FP}$$

(3)

Overall Accuracy

This index indicates the percentage of correctly classified TPs and TNs in the data classified by artificial intelligence. The higher the value, the better the performance is.

$$OverallAccuracy=\frac{TP+TN}{TP+FP+TN+FN}$$

(4)

Area Under the Curve (AUC)

The AUC is the area under the ROC curve, which is a curve drawn with the true positive rate (TPR) on the vertical axis and the false positive rate (FPR) on the horizontal axis. The AUC takes values from 0 to 1. The closer the value is to 1, the higher the discriminative power and the more accurate the model is.

(5)

F1 score

The F1 score is the harmonic mean of the two values, with values between 0 and 1.

$$F1score=\frac{2\times Recall\times Precision}{Recall+Precision}$$

3.4.2. Improvement Relative to the Initial Classifier

The relative improvement from the initial classifier of the classifier created by transfer learning using patches that satisfy the respective classification probabilities (0.80, 0.85, 0.90, 0.95, 1) is evaluated using the accuracy index of Section 3.4.1.

$$Improvementratio=\frac{TransferLearningClassifierAccuracyMetrics}{InitialClassifierAccuracyMetrics}\times 100[\%]$$

4. Results and Discussion

4.1. Patches Used for Transfer Learning

The number of patches satisfying each classification probability is shown in

Table 8. Number of patch images satisfying each classification probability.

Classification Probability	without Calcification	with Calcification
0.80	27,526	2563
0.85	27,245	2444
0.90	26,804	2298
0.95	25,960	2112
1.00	23,476	1754

.

Next,

Table 9. Number of patch images trained for transfer learning when creating each classifier.

Classification Probability	without Calcification	with Calcification	Total Number of Images for Transfer Learning
0.80	25,630	25,630	51,260
0.85	24,440	24,440	48,880
0.90	22,980	22,980	45,960
0.95	21,120	21,120	42,240
1.00	17,540	17,540	35,080

shows the number of patches learned during the transition when creating each classifier.

4.2. Evaluation of the Accuracy of Each Classifier and Improvement Relative to Initial Classifier

The accuracy of each classifier and the relative improvement from the initial classifier are shown in

Table 10. Accuracy of each classifier and relative improvement from the initial classifier.

Classification Probability	Recall (Improvement Ratio)	Precision (Improvement Ratio)	Overall Accuracy (Improvement Ratio)	AUC a (Improvement Ratio)	F1 Score (Improvement Ratio)
(original ResNet50)	0.778	0.751	0.974	0.969	0.765
0.80	0.866 (111.26%)	0.636 (84.71%)	0.966 (99.16%)	0.974 (100.53%)	0.734 (95.95%)
0.85	0.845 (108.61%)	0.643 (85.61%)	0.966 (99.19%)	0.970 (100.18%)	0.731 (95.55%)
0.90	0.856 (109.93%)	0.678 (90.19%)	0.970 (99.59%)	0.971 (100.28%)	0.756 (98.92%)
0.95	0.856 (109.93%)	0.675 (89.82%)	0.970 (99.57%)	0.976 (100.77%)	0.755 (98.69%)
1.00	0.830 (106.62%)	0.682 (90.81%)	0.970 (99.57%)	0.971 (100.28%)	0.749 (97.94%)

^a Area under the curve.

. The recall was highest when the classification probability was 0.80, the AUC was highest for the transfer-trained model for patches with a classification probability exceeding 0.95, and the initial classifier performed best for the other indices.

4.3. Discussion

Table 8 shows that the lower the classification probability, the more patches satisfy the classification probability. When the classification probability is 0.80 and 1.00, the number of patches without calcification and with calcification are significantly different: more than 4000 patches without calcification and more than 800 patches with calcification. Table 9 shows that the total number of transference patches learned differs greatly, depending on the classification probability. When the classification probability is 0.80 and 1.00, the total number of patches learned by transfer learning differs by more than 15,000 patches, showing a large difference. Visual inspection of the patches used for transfer learning showed that the patches with low classification probability tended to be incorrectly transferred from the patches with no calcification to those with calcification.

Section 4.2 show that transfer learning of new data by semi-supervised learning improved the recall and AUC compared to the initial classifier. Here, we discuss the recall and precision, which showed particularly large changes from the initial classifier. The transfer learning of new data by semi-supervised learning increased the recall compared to the initial classifier. This means that the false negative rate became lower, resulting in a reduction in missed patches with calcifications. The reason for this is thought to be that the transfer learning of new patches increased the diversity of calcifications learned by the classifier. Although the morphology and distribution of calcifications in the breast are diverse, we believe that the classifier was able to learn diverse features of calcification by transfer learning of new patches using semi-supervised learning. On the other hand, the transfer learning of new data by semi-supervised learning resulted in a lower precision compared to the initial classifier. This means that the false positive rate increased, and the patches without calcification were more likely to be incorrectly judged as having calcification. The reason for this is that the trained CNN incorrectly inferred unlabeled patches without calcification as having calcification and learned to transfer them, which increased the false positive rate. There is a trade-off between recall and precision. Mammography is used for screening and is said to be one of the most effective methods for the early detection of breast cancer [41]. The purpose of screening is to improve prognosis through earlier diagnosis and early intervention [42]. Therefore, for early diagnosis, it is important not to miss patches with calcifications, i.e., to lower the false negative rate, making the increase in recall very important. Additionally, the application of semi-supervised learning tended to correctly classify patches with calcifications in areas with dense breast tissue, where the images appear white, indicating that the patches with a small contrast between calcifications and breast tissue were more accurately identified. This suggests that calcifications in dense breasts can be more accurately classified as present. The breast is composed of glandular tissue and fatty tissue, and based on the proportion of these components, breast density is classified into four categories: “fatty”, “scattered fibroglandular”, “heterogeneously dense”, and “extremely dense”. The latter two categories, “heterogeneously dense” and “extremely dense,” are referred to as dense breasts, where there is little fat and a higher proportion of glandular tissue, causing the images to appear white. This reduces the sensitivity of mammography and lowers the detection rate of lesions [43,44]. Additionally, there is a deep relationship between breast density and breast cancer risk, with women who have dense breasts being at a higher risk of breast cancer compared to women with fatty breast tissue [45,46]. Given these factors, detecting calcifications in dense breasts is important. Thus, the increase in recall and the reduction in the false negative rate, along with the improved accuracy in classifying patches with calcifications in dense breasts, demonstrate the usefulness of semi-supervised learning in calcification detection.

Next, we discuss the differences in TP, TN, FP, and FN patches between the initial classifier and the classifier after transfer learning. First, compared to the initial classifier, the transfer learning of new data by semi-supervised learning tended to correctly classify the TP patches as having calcifications in areas with much mammary tissue, where the image was rendered white, that is, patches with little contrast between the calcification and the mammary gland. However, for the FP patches, compared to the initial classifier, after transfer learning, the classifier tended to incorrectly classify patches with more mammary tissue and no calcification where the image was rendered in white as having calcification. Thus, it is thought that transfer learning made it easier to classify patches with more mammary tissue and white images as having calcification.

Next, we discuss the difference in recall and precision by classification probability. From Table 10, recall tended to be higher when the classification probability was low and lower when the classification probability was high. The reason for this is that more patches satisfy the classification probability when the classification probability is low, more patches with calcification are learned by transfer, more various calcification features are learned, and the diversity of calcification learned by the classifier is expanded. Table 9 shows that the number of patches with calcification learned by the classifier differs by about 8000 when the classification probability is 0.80 and when the classification probability is 1.00. Therefore, it can be considered that more calcification features were learned when the classification probability was 0.80. On the other hand, precision tended to be low when the classification probability was low and high when the classification probability was high. The reason for this is thought to be that when the classification probability is high, only patches with greater certainty are learned to be transferred as having calcification, and patches without calcification are less likely to be incorrectly transferred as having calcification. As mentioned earlier, patches with lower classification probability tended to be incorrectly transfer-learned as having calcifications, even when they did not contain calcifications. Therefore, the lower the classification probability, the lower the precision will be. In this study, the patches without calcification were extracted only for the number of calcification patches that satisfied the classification probability, ensuring that the number of calcification patches with calcification and without calcification that were learned by transfer was equal. However, we believe that directly extracting the same number of patches without calcifications as the patches with calcifications used for transfer learning would increase the diversity of patches without calcifications and decrease false positives. (For example, consider the case where a classifier is created by transfer learning patches that satisfy the classification probability of 0.80. In this study, 2563 patches without calcification, matching the number of patches with calcifications that satisfied the classification probability, were extracted from 27,526 patches without calcifications. The number of patches with and without calcifications was then augmented by a factor of 10 for transfer learning. However, what is mentioned here is that 25,630 patches were directly extracted from the 27,526 patches without calcifications that satisfied the classification probability).

Next, we discuss the differences among TP, TN, FP, and FN patches by classification probability. For the FP patches, the patches with dense breast tissue tended to be classified as having calcifications, and the images with dense breast tissue and no calcifications were also incorrectly classified as having calcifications. For the TP patches, the lower the classification probability, the more likely it was to correctly classify patches with dense breast tissue and calcifications as having calcifications, leading to an improvement in recall and a reduction in missed calcifications within dense breasts. Therefore, while conventional supervised learning tends to miss calcifications within dense breasts, the semi-supervised approach in this study can detect calcifications with high accuracy, regardless of the breast density of the input images, contributing to the improvement of recall.

As with all classifiers, these classifiers tended to incorrectly judge a patch as without calcification when the area of calcification in the patch was small. If a patch contained many calcifications, slightly larger calcifications, or clusters of small calcifications, it could be classified as calcified. However, if a patch contained only one or two small calcifications, it tended to be incorrectly judged as without calcification. Therefore, this study divided the images into 224 × 224 pixel patches to fit the ResNet50 input size. However, if the patches were divided into smaller matrices and then enlarged to match the CNN input size by resizing, we believe that the area of calcification in the patches would increase, potentially reducing the number of false negative patches.

Next, we compare our results with those of similar studies. Since no study used semi-supervised learning to classify the presence or absence of calcification in mammograms, we compare our results with similar studies [29] that use semi-supervised learning for pixel-based classification for tumor segmentation in mammogram images. Comparing “Bayes” as a classifier for supervised learning and “Co-training” as a classifier for semi-supervised learning in similar studies, semi-supervised learning has higher accuracy and sensitivity, or recall, and lower positive predictive accuracy, or precision. As in similar studies, semi-supervised learning increased the recall and decreased the precision in this study. On the other hand, the overall accuracy of this study was comparable to that of the initial classifier, which was different from similar studies. The difference can be attributed to the fact that this study used a self-training method in semi-supervised learning, whereas similar studies used a co-training method in semi-supervised learning. In self-training, a classifier is created by learning labeled data, classifying unlabeled data with the created classifier, and updating the classifier by transferring the classified data with high classification probability to the original classifier as labeled data. However, this method incorrectly transfers learning when unlabeled data are classified differently from the actual data. In order to reduce such errors as much as possible, there is a method called co-training. In this method, the labeled data are first divided into two parts, each of which is trained to create two classifiers. Next, unlabeled data are classified using each classifier, and the classified data with a high classification probability are transferred to a different classifier as labeled data, aiming for better learning by creating two different classifiers and alternately interpolating the information held by the two classifiers [47]. We believe that the difference in the methods used can be cited as the reason for the difference in whether the accuracy was the same or increased.

A limitation of this study is that Asians tend to have more dense breasts than Westerners [48], which may affect the accuracy when used for mammographic images of Asian women. Therefore, in the future, it will be necessary to use a dataset of mammographic images of Asian women or to mix the data used in this study with mammographic images of Asian women. In addition, although this study only classified the presence or absence of calcification, calcification can be either benign or malignant [8,10], and benign or malignant classification is also necessary for the diagnosis of breast cancer.

In this study, ResNet, a common network model in the field of medical image classification [16], was used for classification, but the accuracy may change by changing the network model used. Additionally, one of the state-of-the-art (SOTA) models in image classification is OmniVec [49], but no studies have been found that use it for medical imaging. In a study published in 2024 [50] that classified mammographic images as normal, benign, or malignant, the images were input into the CNN without patch division. The approach of dividing mammographic images into patches and inputting them into the CNN, as adopted in this study, is a novel perspective.

This study applied semi-supervised learning to classify the presence or absence of calcifications in mammograms. Although semi-supervised learning has been validated in various fields of medical imaging, supervised learning remains predominant in mammography, and there are few studies applying semi-supervised learning to this field. No studies have been found that apply semi-supervised learning to classify the presence or absence of calcifications in mammograms. Furthermore, while this study examined the differences in accuracy due to classification probability thresholds in semi-supervised learning, no prior studies have verified the differences in accuracy for classifying the presence or absence of calcifications based on varying classification probabilities. This study provided new insights through the application of semi-supervised learning.

This study used a small amount of labeled data and a large amount of unlabeled data for learning. Supervised learning, which is the mainstream approach in medical research using deep learning, requires a large amount of labeled data. However, labeling medical data requires specialized knowledge and is time-consuming, making labeled data limited and difficult to obtain. On the other hand, unlabeled data continue to increase and can be easily and abundantly prepared. Our research method can efficiently utilize limited labeled data and a large amount of unlabeled data. Furthermore, it can utilize features obtained from labeled data while also leveraging features from a large amount of unlabeled data, enabling learning from more features.

For future consideration, as mentioned earlier, to improve accuracy, it is suggested to divide the mammography image into smaller patches, resize them, input them into the CNN, and use a co-training method. In addition, we believe that by devising a method for extracting patches that satisfied the classification probability of no calcification during transition learning, more features without calcification can be learned, and the number of false positives can be reduced, leading to improved accuracy. Although this study only classified the presence or absence of calcification, we believe that a classifier that learns the morphology and distribution of calcification in patches classified as having calcifications can be used to develop tools for classifying the distribution, distinguishing between benign and malignant calcifications, and determining the BI-RADS category.

5. Conclusions

In this study, we investigated the effect of semi-supervised learning on mammography calcification detection. The results suggest that semi-supervised learning is useful for improving the accuracy of calcification identification in mammography. Additionally, this study examined the impact of different classification probabilities (threshold differences) on the accuracy of calcification detection. Traditionally, supervised learning has been predominant in the field of mammography, and there have been few applications of semi-supervised learning. No prior studies have examined the differences in classification accuracy for calcification detection based on varying classification probabilities. Therefore, this study provides new insights through the application of semi-supervised learning.