Extraction of Stone Positions from a Sheet Image for Curling Match Database Construction
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe proposed YOLO11n-based model with pseudo-label semi-supervised learning shows promise for generalizable stone detection. However, there are still some concerns that the authors should be carefully considered before it can be accepted by this journal. Some of my comments are as follows.
(1) The choice of YOLO11n over higher-precision variants (e.g., YOLO11s/m) needs justification. Please explain whether this decision was driven by real-time processing requirements, hardware constraints, or specific trade-offs between speed and accuracy .
(2) The 0.75 confidence threshold for pseudo-label selection is empirically determined but lacks systematic validation. A comparative analysis with other thresholds (e.g., 0.70, 0.80) should be added to demonstrate its optimality .
(3) While the training set includes 108,114 images from 20 tournaments, the distribution of visual variations (e.g., stone colors, house designs) across tournaments is not specified. Please provide statistical breakdowns (e.g., image counts per tournament, color distribution) to validate the model’s generalizability .
(4) The Active Testing method estimates overall accuracy at 99.70% using uncertainty scores, but it is unclear how images with no detected stones (e.g., blank sheets) are handled. Addressing this edge case is critical for robust error estimation .
(5) In addition, I am wondering whether some video object detection methods can be helpful for the task, such as "Improving Surveillance Object Detection with Adaptive Omni-Attention over Both Inter-frame and Intra-frame Context", "Illuminating Vehicles With Motion Priors For Surveillance Vehicle Detection", and etc. Some discussions about these works should be included in the manuscript.
(6) Figure 5 lacks labeled pixel positions for critical reference lines (e.g., Tee line, center line). Adding these annotations will improve interpretability .
(7) Table 3 should include standard deviations for precision/recall metrics to demonstrate model stability across training seeds .
Author Response
Comments 1: The choice of YOLO11n over higher-precision variants (e.g., YOLO11s/m) needs justification. Please explain whether this decision was driven by real-time processing requirements, hardware constraints, or specific trade-offs between speed and accuracy .
Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have explained that YOLO11n was selected to ensure low-latency inference and high frame-rate processing for future real-time video applications. Since it already achieves an exceptionally high estimated accuracy of 99.37% (as detailed in Section 3.2), using larger models (YOLO11s/m) would require more computational resources without significantly improving the saturated accuracy. This change can be found in Section 2.1, Page 5, lines 131-136.
-------------------------------------------------------------------------------------------------------------
Comments 2: The 0.75 confidence threshold for pseudo-label selection is empirically determined but lacks systematic validation. A comparative analysis with other thresholds (e.g., 0.70, 0.80) should be added to demonstrate its optimality.
Response 2: We agree with this comment. We have, accordingly, revised the manuscript to clarify the systematic selection criteria for this parameter. We have added a sensitivity analysis (Table 4) evaluating confidence thresholds of 0.70, 0.75, and 0.80. The results demonstrated that our baseline model achieves a perfect Precision of 1.0000 across all three thresholds, completely eliminating the concern that the model might relearn its own errors. Since pseudo-label quality was guaranteed, the threshold of 0.75 was systematically chosen as the optimal sweet spot: it safely maintains 92.0% of the usable pseudo-labeled images, whereas increasing the threshold to 0.80 causes the usable data to drop sharply to 49.6%. Furthermore, while the 0.70 threshold also showed perfect precision in this test set, the 0.75 threshold was preferred to provide a conservative safety margin against visually extreme edge cases that might exist in the broader, unlabeled dataset. These updates can be found in Section 2.2, Pages 6-7, lines 182-200.
-------------------------------------------------------------------------------------------------------------
Comments 3: While the training set includes 108,114 images from 20 tournaments, the distribution of visual variations (e.g., stone colors, house designs) across tournaments is not specified. Please provide statistical breakdowns (e.g., image counts per tournament, color distribution) to validate the model’s generalizability .
Response 3: We agree with this comment. We have, accordingly, added a new table to the manuscript to address this point. We provided a statistical breakdown of the dataset including the number of images extracted per tournament to explicitly validate the model's generalizability across diverse data sources. This change can be found in Section 2.1, Pages 4-5, lines 104-105, and Table 1.
-------------------------------------------------------------------------------------------------------------
Comments 4: The Active Testing method estimates overall accuracy at 99.70% using uncertainty scores, but it is unclear how images with no detected stones (e.g., blank sheets) are handled. Addressing this edge case is critical for robust error estimation .
Response 4: We agree with this comment. We have, accordingly, revised the manuscript to explicitly clarify how images with no stones (blank sheets) are handled and evaluated within our Active Testing framework. We fully agree that addressing this edge case is critical. In our framework, blank sheets are properly handled through two mechanisms: First, empirically, non-target objects like hollow circles on blank sheets occasionally cause false positives among the surrogate models, naturally leading to high uncertainty scores. Second, and more importantly, our newly introduced clipping mechanism (α=0.2) mathematically guarantees a non-zero selection probability for every image, even if all models perfectly agree that no stones are present (i.e., the uncertainty score is exactly zero). Thus, blank sheets are never excluded from the probabilistic sampling candidates, ensuring robust error estimation. We have updated Section 2.3.2, Page 9, lines 263-267 to clarify this behavior.
-------------------------------------------------------------------------------------------------------------
Comments 5: In addition, I am wondering whether some video object detection methods can be helpful for the task, such as "Improving Surveillance Object Detection with Adaptive Omni-Attention over Both Inter-frame and Intra-frame Context", "Illuminating Vehicles With Motion Priors For Surveillance Vehicle Detection", and etc. Some discussions about these works should be included in the manuscript.
Response 5: We agree with this comment. We have, accordingly, revised the Conclusion section to discuss these advanced video object detection methods, alongside other dynamic tracking studies, as a highly relevant future direction. We fully agree that the recommended methodologies (adaptive omni-attention and motion priors) are highly effective for dynamic tracking. To contextualize these within our domain, we added a brief discussion on how integrating these techniques will be essential for our future goal of real-time curling video analysis. This change can be found in Section 4 (Conclusion), Page 17, lines 450-454.
-------------------------------------------------------------------------------------------------------------
Comments 6: Figure 5 lacks labeled pixel positions for critical reference lines (e.g., Tee line, center line). Adding these annotations will improve interpretability .
Response 6: We agree with this comment. We have, accordingly, modified to emphasize this point. To improve ease of interpretation, we have updated the figure to include pixel coordinates and revised the caption to mention the "center line" (x=149) and "tee line" (y=159). This change can be found in Section 2.5.2, Page 11, Figure 6.
-------------------------------------------------------------------------------------------------------------
Comments 7: Table 3 should include standard deviations for precision/recall metrics to demonstrate model stability across training seeds .
Response 7: We agree with this comment. We have, accordingly, revised to emphasize this point. While the standard deviations were already included in the table in the format of "mean ± std," we updated the table caption to state this clearly and ensure it is easily understood by all readers. This change can be found in Section 3.1, Page 13, Table 5.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe manuscript entitled “Extraction of Stone Positions from a Sheet Image for Curling Match Database Construction”, submitted to Applied Sciences under ID applsci-4187312, presents a computer vision–based method for automatically detecting the positions of curling stones from images of the playing sheet. The main goal of the study is to support the construction of a structured match database that can later be used for tactical analysis and performance evaluation in curling.
The authors describe an image-processing pipeline for stone detection and position extraction and evaluate the proposed method using images obtained from curling matches. While the topic is relevant for the development of data-driven analysis in sports, several aspects of the manuscript require clarification and improvement before it can be properly evaluated for publication.
Major Comments
The paragraphs between lines 68 and 85 appear to be misplaced. It would be more natural for the information presented there to be part of the motivational section, which should precede the “Contributions” of the authors. Immediately after the “Contributions,” one would normally expect a short subsection describing the organization of the paper.
At line 136, the authors state that 50 images were randomly extracted from the entire dataset and annotated, and that these images were used for training (80%) and validation (20%). Two paragraphs later, the authors state that 400 images were randomly extracted from the entire dataset, ensuring that there was no overlap with the training, validation, or test sets. Up to that point in the manuscript, however, no test set had been introduced, which creates some confusion, since the extraction of the test set is only discussed later in Section 2.3.1. Moreover, if 50 images had already been extracted from the “entire dataset,” then the dataset was no longer complete when the additional 400 images were selected. It is therefore unclear how the authors ensured that none of the 400 images overlapped with the original 50 images or with the 110 synthetic images generated through inversion.
When the authors state that the training data were augmented by adding horizontally flipped images, do they mean that the images were flipped with respect to the symmetry line of the house? Such an inversion would move all red stones from the left side to the right side of Figure 4(b). If so, with respect to which axis would a vertical inversion be defined? It might be useful to include a figure illustrating an original image together with the possible horizontal, vertical, and combined horizontal–vertical inversions. In the final dataset, is the target origin always fixed at coordinates (149, 159) pixels?
It is not entirely clear how the so-called “pre-trained” model was constructed. Did it consider the original set of 50 annotated images together with the 110 images generated through inversion? Is the hyperparameter configuration of this “pre-trained” model different from that of the model trained with 537 images? Was the 8:2 ratio used consistently in all training stages? Were 128 images used for training and 32 for validation in the final model? Finally, what criteria were used to determine the hyperparameters?
From the perspective of reproducibility, it would be very useful if the authors discussed the threshold value of 0.75 (line 155), which appears to have been determined empirically based on a qualitative evaluation of detection results. This is particularly important because the validation of pseudo-labels prevents the model from relearning its own errors. Images are selected for pseudo-labeling only when all detected objects have a confidence score ≥ 0.75. In practice, this may lead to an augmented training dataset that contains mainly less complex examples for the initial model. Do the authors consider the possibility that the expanded training set might under-represent more complex scenarios? In addition, the manuscript does not explicitly state which metric is used to compute the confidence score. Is this the standard confidence score produced by the YOLO detector?
The non-maximum suppression (NMS) post-processing technique is only briefly mentioned. In addition to providing references, it would be important to describe in more detail how this technique operates in the context of the present study.
The manuscript mentions a threshold value of 0.5 in Table 2, but it is not entirely clear whether this value refers to the confidence threshold used to filter detections or to the IoU threshold used for evaluation. The authors are encouraged to clarify the role of this parameter in the detection pipeline.
In Section 2.3.1, the authors state that the reported results were obtained from 10 models trained using the same training data but different random seeds. Should this be interpreted as an ensemble approach? If so, it would be useful to clarify what differences are introduced by the different seeds and whether they are sufficient to produce the intended variability among the models.
The Active Testing framework relies on probabilistic sampling based on an uncertainty score, followed by a bias-corrected estimator (e.g., the LURE estimator). However, the manuscript suggests that the 500 samples were selected deterministically in descending order of uncertainty. If this is the case, the theoretical assumptions of the estimator may not hold, and the resulting accuracy estimate may be biased. The authors are encouraged to clarify whether probabilistic sampling was used and how the sampling probabilities were incorporated into the estimator.
The manuscript states that accurate ground-truth stone positions are difficult to obtain from real images, which motivates the use of synthetically generated images for evaluating positional error. However, the detection model itself must have been trained using manually annotated bounding boxes in real images. It would therefore be helpful if the authors clarified how the ground-truth bounding boxes used for training were defined and what level of positional uncertainty may be associated with those annotations. In addition, the authors should discuss how representative the synthetic images used for error evaluation are with respect to real match conditions. Furthermore, the reported average positional error of 0.472 pixels appears extremely small and requires additional clarification so that the reader can properly interpret its practical meaning. First, the manuscript does not clearly define the error metric used to compute this value. It is not specified whether this error corresponds to the signed difference between predicted and reference coordinates, the absolute error, or another distance-based metric. Without a precise definition, it is difficult to assess the significance of the reported value. In addition, reporting only the mean error is insufficient to characterize the variability of the results. As a statistical quantity, the mean value alone does not describe the dispersion of the data. It would therefore be important to provide additional information such as the standard deviation or other measures of variability.
Since one of the stated goals of the work is to support the development of AI-based tools for real curling competitions, it would also be useful to express the positional error in physical units (e.g., meters) rather than only in pixel coordinates. This would allow readers to better assess the practical implications of the reported accuracy. If such a conversion is not considered necessary, the authors should clarify the reasons.
Finally, from a practical perspective, it would be helpful to discuss what level of positioning error can be considered acceptable in the context of curling analysis and decision support. Because the estimation of stone positions plays a central role in tactical evaluation, the manuscript should clarify the relationship between pixel coordinates and real-world spatial coordinates, and discuss how the reported accuracy relates to practical requirements.
The purpose of Figure 6 is also not entirely clear. The authors state that the three images on the left are generated images and that the three images on the right are real images, and then claim that the generated images are so similar to the real ones that they are difficult to distinguish at first glance. However, it is not obvious in which aspects the images are similar. In fact, the stones appear to be positioned differently between the generated and real images, which makes the intended comparison unclear.
The reference list is also relatively limited. Two of the cited works, [17] and [18], are included mainly to discuss future research directions involving trajectory tracking of curling stones using YOLO-based approaches. Since these works do not directly address the present problem, the remaining references correspond to a rather small number of sources. Among them, only a few correspond to peer-reviewed journal publications. The extremely small number of references from journal articles may raise concerns among readers of Applied Sciences regarding the scientific positioning of the work. The authors are encouraged to conduct a more comprehensive review of the literature in order to demonstrate more clearly how the proposed study relates to existing scientific contributions and how it advances the state of the art.
Overall, while the topic addressed in the manuscript is relevant and the proposed system may have practical value for curling analytics, the issues discussed above should be clarified before the work can be properly evaluated.
Minor Comments
Equations are part of the surrounding sentences and should therefore be punctuated accordingly. If an equation concludes a sentence, it should be followed by a period. If the sentence continues after the equation, it should be followed by a comma or semicolon. None of the seven equations in the manuscript appear to be properly punctuated.
The line immediately following an equation should normally begin without left indentation and start with a lowercase word such as “where,” followed by the definitions of the symbols used. In lines 186, 189, and 214, there are unnecessary left indentations. Left indentation is typically used to indicate a new paragraph, which does not seem to be the intended case in these instances.
Author Response
Comments 1: The paragraphs between lines 68 and 85 appear to be misplaced. It would be more natural for the information presented there to be part of the motivational section, which should precede the “Contributions” of the authors. Immediately after the “Contributions,” one would normally expect a short subsection describing the organization of the paper.
Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have modified the structure of the Introduction section to emphasize this point. We moved the paragraphs discussing the background of curling AI and the motivation for this study to precede the "Contribution" subsection. Consequently, the "Paper organization" subsection has been relocated immediately following the "Contribution" subsection. This restructuring significantly improves the logical flow from the background motivation to our proposed contributions. These changes can be found in Section 1, Pages 3-4, lines 67-100.
-------------------------------------------------------------------------------------------------------------
Comments 2: At line 136, the authors state that 50 images were randomly extracted from the entire dataset and annotated, and that these images were used for training (80%) and validation (20%). Two paragraphs later, the authors state that 400 images were randomly extracted from the entire dataset, ensuring that there was no overlap with the training, validation, or test sets. Up to that point in the manuscript, however, no test set had been introduced, which creates some confusion, since the extraction of the test set is only discussed later in Section 2.3.1. Moreover, if 50 images had already been extracted from the “entire dataset,” then the dataset was no longer complete when the additional 400 images were selected. It is therefore unclear how the authors ensured that none of the 400 images overlapped with the original 50 images or with the 110 synthetic images generated through inversion.
Response 2: We agree with this comment. We sincerely apologize for the confusion caused by our drafting error. We have, accordingly, corrected the manuscript to accurately reflect the sequential dataset sampling process. We clarified that the mention of ensuring no overlap with the "test sets" in that paragraph was indeed a mistake. Chronologically, the 500 images for the test set had not yet been created at that stage. The correct process was sequential sampling without replacement. First, 50 images were extracted from the entire pool of 108,114 images. Second, the 400 images for semi-supervised learning were randomly extracted from the remaining dataset. This purely sequential extraction guarantees absolutely no overlap with the initial 50 images (the training and validation sets). We have corrected the text to reflect this by removing the phrase "or test sets". Regarding your concern about the 110 synthetic images, these were generated internally through data augmentation (flipping) of the 40 training images. Therefore, they never existed in the original dataset pool, making any overlap impossible. These corrections and clarifications can be found in Section 2.2, Page 6, lines 152-156, 180-182.
-------------------------------------------------------------------------------------------------------------
Comments 3: When the authors state that the training data were augmented by adding horizontally flipped images, do they mean that the images were flipped with respect to the symmetry line of the house? Such an inversion would move all red stones from the left side to the right side of Figure 4(b). If so, with respect to which axis would a vertical inversion be defined? It might be useful to include a figure illustrating an original image together with the possible horizontal, vertical, and combined horizontal–vertical inversions. In the final dataset, is the target origin always fixed at coordinates (149, 159) pixels?
Response 3: We agree with this comment. We have, accordingly, revised the manuscript to clarify the details of the data augmentation process and added a new figure to illustrate the inversions as suggested. We added an explanation that the horizontal and vertical flips were simply applied directly to the pixel coordinate system of the entire image. To make this visually clear, we have included a new figure that illustrates an original image alongside its horizontal, vertical, and combined horizontal-vertical inversions. Furthermore, we clarified that the coordinate origin is consistently fixed at the top-left corner (0, 0) of the image across all data, rather than at the center of the house. Finally, we explicitly noted that when stones change positions due to these image flips, the bounding box coordinates in the corresponding label data are automatically recalculated and mathematically adjusted to match the new positions. These changes and the new figure can be found in Section 2.2, Page 6, lines 157-164, and Figure 4.
-------------------------------------------------------------------------------------------------------------
Comments 4: It is not entirely clear how the so-called “pre-trained” model was constructed. Did it consider the original set of 50 annotated images together with the 110 images generated through inversion? Is the hyperparameter configuration of this “pre-trained” model different from that of the model trained with 537 images? Was the 8:2 ratio used consistently in all training stages? Were 128 images used for training and 32 for validation in the final model? Finally, what criteria were used to determine the hyperparameters?
Response 4: We agree with this comment. We have, accordingly, revised the manuscript to clarify the construction of the pre-trained model, the dataset splitting strategy, and the hyperparameter configurations. We explicitly addressed the following three points in the text to answer your questions: 1) Definition of the pre-trained model: We clarified that the term "pre-trained model" in our study refers specifically to the baseline model trained exclusively on the initial 160 augmented images (derived from the 40 training images). 2) Consistent use of validation data: Regarding the 8:2 split, 40 of the initial 50 annotated images were allocated for training, while the remaining 10 images were strictly reserved for validation. Crucially, we clarified that this exact same set of 10 validation images was consistently used across both the pre-training phase and the subsequent semi-supervised learning phase. We did not use 128 images for training and 32 for validation in the final model; the validation set was strictly fixed to the initial 10 images to ensure a fair evaluation and prevent data leakage. 3) Hyperparameter consistency: We explicitly stated that the hyperparameter settings detailed in Table 2 were consistently applied across all training stages, including both the pre-training and the retraining phases, without any modifications. These clarifications have been added to Section 2.2, Pages 6-7, lines 154-156, 165-167, and 202-203.
-------------------------------------------------------------------------------------------------------------
Comments 5: From the perspective of reproducibility, it would be very useful if the authors discussed the threshold value of 0.75 (line 155), which appears to have been determined empirically based on a qualitative evaluation of detection results. This is particularly important because the validation of pseudo-labels prevents the model from relearning its own errors. Images are selected for pseudo-labeling only when all detected objects have a confidence score ≥ 0.75. In practice, this may lead to an augmented training dataset that contains mainly less complex examples for the initial model. Do the authors consider the possibility that the expanded training set might under-represent more complex scenarios? In addition, the manuscript does not explicitly state which metric is used to compute the confidence score. Is this the standard confidence score produced by the YOLO detector?
Response 5: We agree with this comment. We have, accordingly, revised the manuscript to clarify the systematic selection criteria for this parameter. We have added a sensitivity analysis (Table 4) evaluating confidence thresholds of 0.70, 0.75, and 0.80. The results demonstrated that our baseline model achieves a perfect Precision of 1.0000 across all three thresholds, completely eliminating the concern that the model might relearn its own errors. Since pseudo-label quality was guaranteed, the threshold of 0.75 was systematically chosen as the optimal sweet spot: it safely maintains 92.0% of the usable pseudo-labeled images, whereas increasing the threshold to 0.80 causes the usable data to drop sharply to 49.6%. Furthermore, while the 0.70 threshold also showed perfect precision in this test set, the 0.75 threshold was preferred to provide a conservative safety margin against visually extreme edge cases that might exist in the broader, unlabeled dataset. In addition, we added a brief explanation in Section 2.2 to explicitly state that the confidence score used in our study is indeed the standard score generated by YOLO, reflecting both objectness and class probability. These updates can be found in Section 2.2, Pages 6-7, lines 183-199, and Table 4.
-------------------------------------------------------------------------------------------------------------
Comments 6: The non-maximum suppression (NMS) post-processing technique is only briefly mentioned. In addition to providing references, it would be important to describe in more detail how this technique operates in the context of the present study.
Response 6: We agree with this comment. We have, accordingly, revised the manuscript to briefly clarify the role of NMS and the parameters in Table 3, along with adding an appropriate reference. We expanded the explanation in Section 2.2 to clarify how NMS operates in our specific context: if multiple bounding boxes for the same stone class overlap with an IoU greater than 0.3, only the one with the highest confidence score is retained. Furthermore, we explicitly clarified in the text that the "0.5" in Table 3 is the minimum confidence threshold used to filter out low-certainty predictions before applying NMS, distinguishing it from the IoU threshold. Finally, we added a foundational reference for NMS to properly credit the technique. These additions can be found in Section 2.2, Page 6, lines 168-174.
-------------------------------------------------------------------------------------------------------------
Comments 7: The manuscript mentions a threshold value of 0.5 in Table 2, but it is not entirely clear whether this value refers to the confidence threshold used to filter detections or to the IoU threshold used for evaluation. The authors are encouraged to clarify the role of this parameter in the detection pipeline.
Response 7: We agree with this comment. We have, accordingly, modified the manuscript to explicitly explain the NMS operation and clarify the parameters. We updated Table 3 to explicitly state that "0.5" is the "Confidence score threshold" used to filter out low-certainty detections, distinguishing it from the IoU threshold. This change can be found in Section 2.2, Page 7, Table 3.
-------------------------------------------------------------------------------------------------------------
Comments 8: In Section 2.3.1, the authors state that the reported results were obtained from 10 models trained using the same training data but different random seeds. Should this be interpreted as an ensemble approach? If so, it would be useful to clarify what differences are introduced by the different seeds and whether they are sufficient to produce the intended variability among the models.
Response 8: We agree with this comment. We have, accordingly, revised the manuscript to clarify that this is not an ensemble approach and explicitly described the differences introduced by the random seeds and their intended purpose. We first clarified in the text that the creation of the 10 models should not be interpreted as an ensemble approach for inference. Our final detection pipeline utilizes only the single best-performing model. The 10 models were generated for three specific purposes: (1) to ensure statistical reliability by evaluating the average performance, (2) to select the single best model for actual use, and (3) to serve as surrogate models for calculating prediction variance (uncertainty) in the Active Testing framework (Section 2.3.2). Regarding the differences introduced by the seeds, we explicitly stated that varying the random seeds alters the initial network weight distributions and the batch sampling orders during training. These random factors cause each model to converge to slightly different local optima, thereby producing slightly different decision boundaries. Finally, regarding whether this variability was sufficient, we confirmed that these differences successfully produced the intended variability among the models’ predictions to calculate the standard deviation of confidence scores. These clarifications have been added to Section 2.3.1, Page 8, lines 214-221.
-------------------------------------------------------------------------------------------------------------
Comments 9: The Active Testing framework relies on probabilistic sampling based on an uncertainty score, followed by a bias-corrected estimator (e.g., the LURE estimator). However, the manuscript suggests that the 500 samples were selected deterministically in descending order of uncertainty. If this is the case, the theoretical assumptions of the estimator may not hold, and the resulting accuracy estimate may be biased. The authors are encouraged to clarify whether probabilistic sampling was used and how the sampling probabilities were incorporated into the estimator.
Response 9: We agree with this comment. We deeply appreciate this critical observation. We have, accordingly, corrected the sampling method from deterministic top-K selection to true probabilistic sampling and updated the accuracy estimation. We fully agree with your concern that selecting samples deterministically in descending order of uncertainty violates the theoretical assumptions of the LURE estimator and introduces bias. To address this, we have thoroughly revised the methodology to use probabilistic sampling without replacement, where the selection probability is strictly proportional to the uncertainty score. Furthermore, to ensure the unbiasedness of the estimator, we incorporated a clipping mechanism (α=0.2) following Kossen et al. to guarantee non-zero probabilities for all candidates. We also corrected Equation (4) to properly normalize the probabilities over all remaining unlabeled images. Consequently, we re-evaluated the dataset using this corrected probabilistic approach. The resulting R_LURE was updated to 0.0063, and the estimated overall accuracy was updated to 99.37%. These updates can be found in Section 2.3.2, Page 9, lines 251 and 263-268, and Section 3.2, Page 14, lines 373-375.
-------------------------------------------------------------------------------------------------------------
Comments 10: The manuscript states that accurate ground-truth stone positions are difficult to obtain from real images, which motivates the use of synthetically generated images for evaluating positional error. However, the detection model itself must have been trained using manually annotated bounding boxes in real images. It would therefore be helpful if the authors clarified how the ground-truth bounding boxes used for training were defined and what level of positional uncertainty may be associated with those annotations. In addition, the authors should discuss how representative the synthetic images used for error evaluation are with respect to real match conditions. Furthermore, the reported average positional error of 0.472 pixels appears extremely small and requires additional clarification so that the reader can properly interpret its practical meaning. First, the manuscript does not clearly define the error metric used to compute this value. It is not specified whether this error corresponds to the signed difference between predicted and reference coordinates, the absolute error, or another distance-based metric. Without a precise definition, it is difficult to assess the significance of the reported value. In addition, reporting only the mean error is insufficient to characterize the variability of the results. As a statistical quantity, the mean value alone does not describe the dispersion of the data. It would therefore be important to provide additional information such as the standard deviation or other measures of variability.
Response 10: We agree with this comment. We have, accordingly, revised the manuscript to clarify the inherent positional uncertainty in manual annotations, the representativeness of the synthetic images, and added the standard deviation of the detection error. We first added an explanation that because training bounding boxes were manually annotated based on human visual observation, they inevitably contain human-induced positional uncertainty. This is exactly why real images are unsuitable for evaluating microscopic, sub-pixel level errors, motivating our use of synthetic images where exact mathematical centers are strictly known. Regarding the representativeness of these synthetic images, we replicated real-world scenarios by randomly combining 6 patterns of house colors, 6 patterns of yellow stones, and randomly placing hollow circles. We also explicitly reiterated that the error metric is defined as the Euclidean distance (L2 norm). Finally, as requested, we calculated and added the standard deviation of the detection error (0.228 px) to properly characterize the variability of the results. These additions and clarifications can be found in Section 2.5.3, Pages 11-12, lines 305-315 and 347-348, and Section 3.4, Page 16, lines 392-396.
-------------------------------------------------------------------------------------------------------------
Comments 11: Since one of the stated goals of the work is to support the development of AI-based tools for real curling competitions, it would also be useful to express the positional error in physical units (e.g., meters) rather than only in pixel coordinates. This would allow readers to better assess the practical implications of the reported accuracy. If such a conversion is not considered necessary, the authors should clarify the reasons.
Response 11: We agree with this comment. We have, accordingly, supplemented the manuscript to express the positional error in physical units to clarify the relationship between pixel coordinates and real-world spatial coordinates. We added a conversion calculation to physical units based on the standard size of a curling stone. Given that the standard radius of an actual curling stone is approximately 14.5 cm and its radius in the image is about 9 px, we defined that 1 pixel corresponds to roughly 1.61 cm. Consequently, we explicitly stated that our average detection error of 0.472 px translates to a real-world positional error of approximately 0.76 cm. These additions can be found in Section 3.4, Page 16, lines 397-401.
-------------------------------------------------------------------------------------------------------------
Comments 12: Finally, from a practical perspective, it would be helpful to discuss what level of positioning error can be considered acceptable in the context of curling analysis and decision support. Because the estimation of stone positions plays a central role in tactical evaluation, the manuscript should clarify the relationship between pixel coordinates and real-world spatial coordinates and discuss how the reported accuracy relates to practical requirements.
Response 12: We agree with this comment. We have, accordingly, supplemented the manuscript with a nuanced discussion on what level of positioning error can be considered acceptable from a practical perspective. We added a discussion acknowledging that while a sub-centimeter difference can alter the outcome in an actual game, constructing a database from Results Books has inherent limitations. The ground truth itself is bounded by the official recorder's manual input based on visual observation and the 0.5 px quantization limit of the image format. Because our model's detection error (approx. 0.76 cm) is strictly smaller than these inherent limitations, achieving higher accuracy from this specific PDF medium is fundamentally impossible. Therefore, we concluded that for the practical purpose of training curling AI for macroscopic tactical evaluation (such as expected score distributions), this sub-centimeter error is overwhelmingly overshadowed by the original recording errors, making it practically sufficient and highly acceptable. These discussions have been added to Section 3.4, Page 16, lines 416-425.
-------------------------------------------------------------------------------------------------------------
Comments 13: The purpose of Figure 6 is also not entirely clear. The authors state that the three images on the left are generated images and that the three images on the right are real images, and then claim that the generated images are so similar to the real ones that they are difficult to distinguish at first glance. However, it is not obvious in which aspects the images are similar. In fact, the stones appear to be positioned differently between the generated and real images, which makes the intended comparison unclear.
Response 13: We agree with this comment. We have, accordingly, replaced Figure 6 with a strictly matched image pair and added a quantitative comparison. We agree that comparing randomly generated configurations with real images made the visual similarities unclear. To explicitly prove the visual fidelity of our generation method, we picked a specific real image, extracted its exact color information, and generated a synthetic image with identical stone coordinates. We replaced Figure 7 with this matched pair. Furthermore, we calculated the Structural Similarity Index Measure (SSIM) between them, achieving a high score of 0.855. This quantitatively demonstrates that our synthetic images successfully replicate real visual features. These updates can be found in Section 2.5.3, Pages 12, lines 339-346.
-------------------------------------------------------------------------------------------------------------
Comments 14: The reference list is also relatively limited. Two of the cited works, [17] and [18], are included mainly to discuss future research directions involving trajectory tracking of curling stones using YOLO-based approaches. Since these works do not directly address the present problem, the remaining references correspond to a rather small number of sources. Among them, only a few correspond to peer-reviewed journal publications. The extremely small number of references from journal articles may raise concerns among readers of Applied Sciences regarding the scientific positioning of the work. The authors are encouraged to conduct a more comprehensive review of the literature in order to demonstrate more clearly how the proposed study relates to existing scientific contributions and how it advances the state of the art.
Response 14: We agree with this comment. We have, accordingly, expanded our references by adding a total of 5 recent peer-reviewed publications (including 3 journal articles and 2 conference papers) in both the Introduction and Conclusion sections to clarify the scientific positioning of our work. In the Introduction (Section 1), we cited recent comprehensive review articles on object detection in sports image analysis ([10,11]) to justify our methodology and clarify the current state of the field. In the Conclusion (Section 4), we incorporated advanced video object detection methodologies ([28–30]) to address dynamic tracking for future applications. These additions substantially increase our peer-reviewed journal references and clarify how our work advances the state of the art. These changes can be found in Section 1, Page 3, lines 60-63, and Section 4, Page 17, lines 450-454.
-------------------------------------------------------------------------------------------------------------
Comments 15: Equations are part of the surrounding sentences and should therefore be punctuated accordingly. If an equation concludes a sentence, it should be followed by a period. If the sentence continues after the equation, it should be followed by a comma or semicolon. None of the seven equations in the manuscript appear to be properly punctuated.
The line immediately following an equation should normally begin without left indentation and start with a lowercase word such as “where,” followed by the definitions of the symbols used. In lines 186, 189, and 214, there are unnecessary left indentations. Left indentation is typically used to indicate a new paragraph, which does not seem to be the intended case in these instances.
Response 15: We agree with this comment. We have, accordingly, corrected the punctuation and formatting throughout the manuscript. We added the appropriate punctuation (periods or commas) to the end of all seven equations depending on the sentence structure. Furthermore, we removed the unnecessary left indentations on the lines immediately following the equations and started them with lowercase words (e.g., "where") as per standard academic formatting guidelines. These changes have been applied throughout the manuscript (e.g., Section 2.3.2, Pages 8-9).
Reviewer 3 Report
Comments and Suggestions for Authors1.The abstract lacks specific presentation of key results，and it is unclear whether the 99.70% accuracy refers to image-level or object-level evaluation. It is recommended that supplementary explanations be added to the Abstract.
2.It is recommended that more specific descriptions of the current research status of curling AI be added in the introduction section, especially the specific bottlenecks faced by existing studies in terms of data acquisition, so as to better highlight the necessity of this study. Meanwhile, relevant studies on the use of YOLO or other object detection models in the field of sports image analysis in recent years should be added to demonstrate the timeliness of the method selection.
3.Several key parameter selections in the article, such as the confidence threshold of 0.75 for pseudo-label filtering and the 20-pixel masking range at the top and bottom, are justified only as "empirically based on qualitative evaluation." It is recommended to provide sensitivity analyses or more systematic selection criteria for these thresholds to enhance the credibility and reproducibility of the method.
4.The positional error verification uses generated images as a substitute for real images for evaluation. It is recommended to supplement quantitative comparisons of visual features between generated and real images and discuss the limitations of transferring evaluation results from generated images to real-world scenarios.
5.The conclusion section does not explicitly state the limitations of the study, such as the model's ability to handle extreme cases, the constraints of relying on Results Books as a data source, and the potential suboptimality of parameter selections. It is recommended to supplement the limitations discussion and specify future research directions, such as technical challenges in video detection (motion blur, viewpoint changes, etc.) and model optimization directions under real-time requirements.
Author Response
Comments 1: The abstract lacks specific presentation of key results，and it is unclear whether the 99.70% accuracy refers to image-level or object-level evaluation. It is recommended that supplementary explanations be added to the Abstract.
Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have revised the Abstract to clarify this point. We added a specific explanation that the 99.37% accuracy refers to an "image-level" evaluation, which is a highly strict criterion where all stones in a single image must be perfectly detected without any omissions or false positives. This change can be found in the Abstract, Page 1, lines 10-11.
-------------------------------------------------------------------------------------------------------------
Comments 2: It is recommended that more specific descriptions of the current research status of curling AI be added in the introduction section, especially the specific bottlenecks faced by existing studies in terms of data acquisition, so as to better highlight the necessity of this study. Meanwhile, relevant studies on the use of YOLO or other object detection models in the field of sports image analysis in recent years should be added to demonstrate the timeliness of the method selection.
Response 2: We agree with this comment. We have, accordingly, expanded the Introduction to emphasize this point. We explicitly described the current bottleneck in curling AI research, which is the heavy reliance on manual data extraction or hard-coded rules that fail to generalize across different tournament formats. Furthermore, to demonstrate the timeliness and necessity of our method, we added references to recent studies successfully applying deep learning-based object detection models in the field of sports image analysis. This change can be found in Section 1, Page 3, lines 50-54 and lines 60-63.
-------------------------------------------------------------------------------------------------------------
Comments 3: Several key parameter selections in the article, such as the confidence threshold of 0.75 for pseudo-label filtering and the 20-pixel masking range at the top and bottom, are justified only as "empirically based on qualitative evaluation." It is recommended to provide sensitivity analyses or more systematic selection criteria for these thresholds to enhance the credibility and reproducibility of the method.
Response 3: We agree with this comment. We have, accordingly, revised the manuscript to clarify the systematic selection criteria for these parameters. We have added a sensitivity analysis (Table 4) evaluating confidence thresholds of 0.70, 0.75, and 0.80. The results demonstrated that our baseline model achieves a perfect Precision of 1.0000 across all three thresholds, completely eliminating the concern that the model might relearn its own errors. Since pseudo-label quality was guaranteed, the threshold of 0.75 was systematically chosen as the optimal sweet spot: it safely maintains 92.0% of the usable pseudo-labeled images, whereas increasing the threshold to 0.80 causes the usable data to drop sharply to 49.6%. Furthermore, while the 0.70 threshold also showed perfect precision in this test set, the 0.75 threshold was preferred to provide a conservative safety margin against visually extreme edge cases that might exist in the broader, unlabeled dataset. Regarding the 20-pixel masking range, we also clarified that this specific value was systematically determined as the geometrically optimal width that completely hides the removed and unused stones without occluding any valid stones located at the very edge of the play area. These updates can be found in Section 2.1, Page 4, lines 111-114, and Section 2.2, Pages 6-7, lines 183-199.
-------------------------------------------------------------------------------------------------------------
Comments 4: The positional error verification uses generated images as a substitute for real images for evaluation. It is recommended to supplement quantitative comparisons of visual features between generated and real images and discuss the limitations of transferring evaluation results from generated images to real-world scenarios.
Response 4: We agree with this comment. We have, accordingly, supplemented the manuscript with a quantitative comparison using SSIM and added a discussion on the limitations of transferring the evaluation results to real-world scenarios. To quantitatively prove the visual fidelity of our synthetic images, we created a strictly matched synthetic/real image pair (now shown in the updated Figure 7) and calculated the SSIM. The high score of 0.855 demonstrates that our generation method accurately mimics real-world visual features. Furthermore, we explicitly added a discussion stating that our evaluated error of 0.472 px represents a theoretical lower bound under ideal conditions, because synthetic images lack the compression artifacts and anti-aliasing noise inherent in actual PDFs. While this noise might slightly increase the error in real-world scenarios, we concluded that this minor increase remains well within the bounds of human annotation uncertainty and the 0.5 px quantization limit, making it practically negligible. These discussions have been added to Section 2.5.3, Pages 12, lines 339-346, and Section 3.4, Page 16, lines 402-407.
-------------------------------------------------------------------------------------------------------------
Comments 5: The conclusion section does not explicitly state the limitations of the study, such as the model's ability to handle extreme cases, the constraints of relying on Results Books as a data source, and the potential suboptimality of parameter selections. It is recommended to supplement the limitations discussion and specify future research directions, such as technical challenges in video detection (motion blur, viewpoint changes, etc.) and model optimization directions under real-time requirements.
Response 5: We agree with this comment. We have, accordingly, expanded the Conclusion section to explicitly emphasize these points. We clearly outlined the limitations of the current study, including its reliance on the availability of Results Books, the handling of extreme edge cases, and the potential suboptimality of parameter selections. Furthermore, we detailed the specific technical challenges anticipated in our future work on video detection, such as frame selection, viewpoint changes, and model optimization under strict real-time requirements. This change can be found in Section 4, Pages 17, lines 433-443 and 449-462.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors have addressed all my concerns in the revision, and I believe that it can be accepted with current form.
Author Response
We sincerely thank the reviewer for providing valuable feedback, which has significantly improved the quality of the manuscript.
Reviewer 2 Report
Comments and Suggestions for AuthorsYour responses to the reviewer’s comments are detailed, well-justified, and convincing. The revised manuscript has been significantly improved, and I am satisfied that my previous concerns have been adequately addressed.
Before the manuscript can be considered for acceptance, I recommend a few minor revisions aimed at improving clarity, consistency, and adherence to standard academic writing conventions:
- Capitalization of expanded abbreviations
With the exception of proper nouns or institutional names, it is not necessary in English to capitalize the full expressions corresponding to abbreviations. Therefore, the authors are encouraged to adopt the following forms consistently throughout the manuscript: "non-maximum suppression" (NMS), "mean average precision" (mAP), "intersection over union" (IoU), "structural similarity index measure" (SSIM), and "red, green, blue" (RGB).
- Avoidance of contractions
Contractions such as “it’s” should be avoided in formal academic writing. Please replace them with their full forms (e.g., *“it is”*) throughout the manuscript.
- Textual transitions between section levels
To improve readability and flow, brief transition text should be included between section and subsection titles. For example, there should be at least a short introductory sentence between the title of Section 3 (line 349) and Subsection 3.1 (line 350).
- Clarification needed
Please clarify the meaning of “blue ‘x” in line 327, as it is currently unclear.
Finally, although the authors have provided a detailed response letter indicating the locations of the revisions, it would improve transparency and facilitate the review process if changes in the manuscript were explicitly highlighted (e.g., using track changes or marked text).
Author Response
We sincerely thank the reviewer for pointing out the presentation issues. We have revised the manuscript accordingly, and the modifications are explicitly shown in the attached diff file. Our detailed responses to each comment are given below.
Comments 1: With the exception of proper nouns or institutional names, it is not necessary in English to capitalize the full expressions corresponding to abbreviations. Therefore, the authors are encouraged to adopt the following forms consistently throughout the manuscript: "non-maximum suppression" (NMS), "mean average precision" (mAP), "intersection over union" (IoU), "structural similarity index measure" (SSIM), and "red, green, blue" (RGB).
Response 1: We have corrected the full expressions of these abbreviations (e.g., non-maximum suppression) to lowercase in both the main text and the Abbreviations section.
-------------------------------------------------------------------------------------------------------------
Comments 2: Contractions such as “it’s” should be avoided in formal academic writing. Please replace them with their full forms (e.g., *“it is”*) throughout the manuscript.
Response 2: We have replaced all contractions with their full forms throughout the manuscript, specifically correcting "it's" to "it is" in Section 2.5.2, Page 10, lines 302 and 304.
-------------------------------------------------------------------------------------------------------------
Comments 3: To improve readability and flow, brief transition text should be included between section and subsection titles. For example, there should be at least a short introductory sentence between the title of Section 3 (line 349) and Subsection 3.1 (line 350).
Response 3: We have added brief introductory sentences immediately after the titles of Sections 2 and 3 to improve readability and flow before transitioning into the subsequent subsections.
-------------------------------------------------------------------------------------------------------------
Comments 4: Please clarify the meaning of “blue ‘x” in line 327, as it is currently unclear.
Response 4: We have revised the text to "with or without a blue cross mark, as shown in Figure 3" to clarify its meaning. This change can be found in Section 2.5.3, Page 12, line 330.
