Performance Evaluation of Convolutional Neural Network (CNN) for Skin Cancer Detection on Edge Computing Devices

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors conducted a performance evaluation of CNN for skin cancer detection. The proposed framework was employed as the on-device machine on so-called low-computing IoT devices (Raspberry Pi and Jetson Nano). The experiment was performed on a sample of 10,000 dermoscopic images that span seven classes from the Harward skin Lesion dataset. Performance was assessed using various metrics, including classification accuracy, hardware efficiency, and the scalability and effectiveness of the proposed model. The authors reported that the proposed model reached a 98.25 % accuracy in 0.01 sec on the Raspberry Pi 5 device.

 

This is an interesting research report. In my opinion, the manuscript is well written. The research subject is actual and appropriate for the Journal.

 

The second section, devoted to related works, effectively builds a comprehensive overview of the research domain. It addresses all significant references, and although it presents an overview of existing literature, it is critically written, highlighting both the strengths and weaknesses of the field's current state.

 

The third section (Materials and Methods) presents the research methodology clearly staged. The methodology is articulated in six stages, with detailed explanations in this chapter. The metrics used for performance evaluation are also outlined, along with the hardware utilized during the experiments and the performance testing methods employed.

 

The fourth section (Results and Discussion) presents the results obtained from the proposed framework. The obtained framework is compared with five current state-of-the-art models. Also, a hardware implementation was discussed in a separate sub-section.

 

Before, the publication, I have a few comments fro the authors to address it

 

1. Please consider adding the article structure at the end of the first section (introduction). It will add to the article's readability.
2. Figure 1 is not referenced in the text. Please integrate the figure in the text.
3. L170. Table 3 has been reported before Table 1. I presume it is some mistake. Please check.
4. Please state the limitations of the proposed research. It will strengthen the article.

Author Response

Comment and Respond in PDF file attachments

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The reviewed manuscript presents research findings on applying Convolutional Neural Networks (CNN) to identify skin cancer types using low-computing Internet of Things (IoT) devices. The topic is relevant, and the manuscript is well-structured. The abstract concisely reflects the study’s content. The Introduction and Related Works sections are well-written. The theoretical part clearly presents the research methodology. The obtained results are interesting. In my opinion, the manuscript can be accepted after revision. Below, I provide my comments.

1. The manuscript does not include information regarding the optimization of the CNN structure and the selection of hyperparameters. Please justify this.
2. Figures 7, 8, etc.: Python tools allow the generation of charts, including confusion matrices, in PNG format with a light background and appropriate resolution. The PrintScreen method is not suitable for this purpose. Please improve the image quality.
3. Title of Figure 8: The title "Research Stage" is unclear. What does it represent? In Figure 3, I see seven sample classes, some of which are very small. However, in Figure 8, the class distribution appears almost equal. Please clarify this discrepancy.
4. The manuscript does not report classification results evaluated on a test subset that was not used during the training phase. Please justify this omission.

Comments on the Quality of English Language

The reviewed manuscript presents research findings on applying Convolutional Neural Networks (CNN) to identify skin cancer types using low-computing Internet of Things (IoT) devices. The topic is relevant, and the manuscript is well-structured. The abstract concisely reflects the study’s content. The Introduction and Related Works sections are well-written. The theoretical part clearly presents the research methodology. The obtained results are interesting. In my opinion, the manuscript can be accepted after revision. Below, I provide my comments.

1. The manuscript does not include information regarding the optimization of the CNN structure and the selection of hyperparameters. Please justify this.
2. Figures 7, 8, etc.: Python tools allow the generation of charts, including confusion matrices, in PNG format with a light background and appropriate resolution. The PrintScreen method is not suitable for this purpose. Please improve the image quality.
3. Title of Figure 8: The title "Research Stage" is unclear. What does it represent? In Figure 3, I see seven sample classes, some of which are very small. However, in Figure 8, the class distribution appears almost equal. Please clarify this discrepancy.
4. The manuscript does not report classification results evaluated on a test subset that was not used during the training phase. Please justify this omission.

Author Response

Please see the attachment. Comment and Respond in PDF file attachments

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The study evaluates the usage of CNN models for skin cancer detection on low-power devices like Raspberry Pi and Jetson Nano. The authors claim high accuracy in their solution and highlight the feasibility of on-device diagnosis.

The study is in line with previous studies in the area, that the authors also mention, and proposes to achieve better results. Nevertheless, it is important to pursue the best possible results as in the present work. Some of the objectives, such as using affordable platforms are achieved, but using the solution in areas with limited internet access is somehow more difficult because it would limit telemedicine. Then how would it be used? with local experts? if so, then what is the need for telemedicine? and if without experts how people would use it? This is to mention that the work is done with interesting objectives, but there is a gap between prototyping and real use. In those cases, how would the missclassification cases be handled? or do the authors assume a low TRL for this solution, and that would be ok, since it is not scalable, as also it misses some explainability that would support the evaluation process. Also, there is the need for real-world validation.

In conclusion, the study is interesting and it is another step towards solutions that work with low computational resources and low energy consumption but still far from an high TRL device that could, in fact, be used for remote areas with limited internet access.

Author Response

Please see the attachment. Comment and Respond in PDF file attachments

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

In this paper, the authors compared edge computing devices for running skin cancer detection machine learning models. Overall, the paper addresses an important and interesting topic; however, its organization requires significant improvement.

First, the current sequence—covering software, pipeline, dataset, software framework (again), hardware, training calibration, and finally comparison—is overly complex and difficult to follow. The manuscript would benefit from a more streamlined structure that logically guides the reader through the study.

Moreover, it is crucial for the paper to clearly emphasize its innovation. The use of these datasets to train imaging analysis models for skin cancer analysis is not entirely new, so the authors should explicitly compare previous approaches with what is novel in their software methodology. A similar critique applies to the hardware section: since both Raspberry Pi and Jetson are commercially available, the focus should shift from evaluating these platforms in isolation to demonstrating how integrating the models into simple, cost-effective devices represents a genuine advancement. 

Author Response

Please see the attachment. Comment and Respond in PDF file attachments

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Thanks, I have no further questions