Comparing the Use of EMBA for IoT Firmware Security Analysis on Cloud Services and Standalone Servers
Round 1
Reviewer 1 Report
The presentation of the various versions of EMBA starting on line 57 is a bit confusing. In an appendix, please provide a table with all the modules added for each version of EMBA
You mention screenshots in line 398 - where are they?
It is unclear why you are comparing different versions of EMBA using different HW platforms - would it not make more sense to compare different EMBA versions on the same HW platform, or the same EMBA version on different HW platforms?
In table 1 you list three firmware samples, but beyond their size there is no information about them . What functions do these implement? What are they used for? What type of devices are they used on?
In table 2 you have replaced the first and third firmware samples with new ones - why? Furthermore, please explain the firmware sample with .chk extension - the average user is unlikely to be familiar with it.
Please rephrase the following sentence in line 692 : "Parallel processing capability greatly reduces the overall module time (HH:MM:SS), while keeping the overall scanning time unchanged, which indicates the cumulative input from all the modules." - this does not make sense; if overall module time is reduced, why should not overall scanning time be reduced? Explain!
In line 854 you state that you cannot share raw firmware files - but why did you not choose firmware files that are freely available?
In line 49 you say EMBA automatically detects insecure binaries - this statement needs more explanation.
In line 130 you seem to identify Wireshark as a static/dynamic analysis tool - since it is a packet capture tool, please explain further.
Provide a reference when you mention OWASP Top 10 in line 174.
There is a lot of repetition in the text at line 179 and 182 - please consider earlier text, and tighten up the prose. Similar in line 290, 298, 405, 492
Since the reference style is bracketed [22], there is no need to include author,date in line 200.
When you explicitly mention authors (which is good) as in line 211, it is not necessary to include the year.
You mention a dataset in line 330 - where did it come from?
How was it determined how many EMBA modules were executed?
You write "The YARA parameter remained disabled unless explicitly stated" - but was it explicitly stated anywhere? Please explain what YARA is.
In the reference list, please provide the full author list for all references (e.g., [10], [14], [15], [27])
Author Response
We would like to sincerely thank the reviewer for the thorough, insightful, and constructive comments provided on our manuscript. The feedback significantly helped us improve the clarity, structure, and technical rigor of the paper. We have carefully addressed each comment in detail below, and all suggested revisions have been incorporated into the revised manuscript. We believe these changes have substantially strengthened the overall quality of the work.
Major Comments
There are 7 major comments. You may find our updates/answers below.
Comment 1:
The presentation of the various versions of EMBA starting on line 57 is a bit confusing. In an appendix, please provide a table with all the modules added for each version of EMBA.
Response:
We agree with the reviewer that the presentation of EMBA versions required clarification. Accordingly, we added a new appendix (Appendix A) containing a comprehensive table that lists all EMBA versions and the corresponding modules introduced in each version. The main text in Section 2.1 has also been revised to reference this appendix and improve clarity.
Comment 2:
You mention screenshots in line 398 - where are they?
Response:
Thank you for pointing this out. The manuscript originally referenced screenshots for configuration documentation; this sentence has been removed for clarity, and the text has been revised to describe the configuration changes in a narrative form, ensuring consistency and accuracy without the need for figures.
Comment 3:
It is unclear why you are comparing different versions of EMBA using different HW platforms - would it not make more sense to compare different EMBA versions on the same HW platform, or the same EMBA version on different HW platforms?
Response:
Thank you for your constructive feedback on our manuscript. We have carefully addressed your suggestions and made the following updates to improve clarity and readability:
In the Abstract, we revised the sentence describing the recorded metrics to read. (Line 17)
“Using identical EMBA configurations and analysis modules, firmware images of varying sizes were analyzed on a standalone personal computer and a Microsoft Azure cloud-based virtual machine”
Similar clarifications were incorporated in the Introduction to reflect the scope of version comparisons and deployment-focused analysis.
In the Introduction section, we updated the line between 60 – 75.
These changes clarify the evaluation methodology and experimental scope and the consideration of deployment environment effects alongside version-specific performance.
Comment 4:
In table 1 you list three firmware samples, but beyond their size there is no information about them. What functions do these implements? What are they used for? What type of devices are they used on?
Response:
We have updated Section 3.1, Firmware Selection and Size Categories, to provide additional details about the selected firmware samples. The updated section now describes the device types, primary functions, and typical use cases of each firmware, including WR940.bin (consumer-grade Wi-Fi router), T8705.bin (mid-range NAS device), R8000.chk (high-end enterprise router), and S3008.bin (industrial IoT gateway). These additions address the reviewer’s request for more context beyond firmware size, clarifying the functional diversity of the samples and their relevance to EMBA performance evaluation.
Comment 5:
In table 2 you have replaced the first and third firmware samples with new ones - why? Furthermore, please explain the firmware sample with .chk extension - the average user is unlikely to be familiar with it.
Response:
We thank the reviewer for raising this point. The replacement of the first and third firmware samples in Table 2 was done to better represent the range of EMBA analysis capabilities across different device types and firmware complexities. The selected samples now include WR940.bin (consumer-grade Wi-Fi router), T8705.bin (mid-range NAS device), and R8000.chk (high-end enterprise router), providing a meaningful assessment of runtime, module execution, and vulnerability detection.
Regarding the R8000.chk sample, we have clarified in Section 3.1 that the .chk extension indicates a firmware check or backup image format commonly used in this device family. This explanation ensures that readers unfamiliar with this format understand its purpose and relevance in the evaluation.
Additionally, we have updated Section 4.1 by including two new test results, which provide further insight into the performance differences between EMBA versions 1.4.1 and 1.4.2. These additions improve the clarity and completeness of our version comparison analysis.
Comment 6:
Please rephrase the following sentence in line 692 : "Parallel processing capability greatly reduces the overall module time (HH:MM:SS), while keeping the overall scanning time unchanged, which indicates the cumulative input from all the modules." - this does not make sense; if overall module time is reduced, why should not overall scanning time be reduced? Explain!
Response:
The original sentence aimed to describe the distinction between total module time and overall scan time in EMBA version 1.4.2. Here, total module time refers to the sum of the execution times of all individual modules, while overall scan time corresponds to the actual duration of a single EMBA run. Due to the introduction of parallel execution in version 1.4.2, multiple modules can run simultaneously. As a result, although the total module time remains high when summing all individual modules, the actual scan duration is shorter compared to sequential execution in version 1.4.1. We recognize that the previous wording could be confusing, and we have revised it to clarify this distinction and accurately reflect the impact of parallel processing on runtime efficiency.
Comment 7:
In line 854 you state that you cannot share raw firmware files - why did you not choose freely available firmware?
Response:
The firmware samples were selected to reflect realistic, real-world analysis scenarios, including a range of device types and complexities. While some firmware files are publicly available, the exact samples used in our study were chosen to ensure that EMBA executes most of its modules and to allow meaningful assessment of runtime, module execution, and vulnerability detection. Due to licensing and redistribution restrictions, the raw firmware images cannot be shared publicly, as now clarified in the updated Data Availability Statement.
Detailed Comments
There are 10 detailed comments. You may find our updates/answers below.
Comment 1:
In line 49 you say EMBA automatically detects insecure binaries - this statement needs more explanation.
Response:
Thank you for your comment. We have clarified the statement regarding insecure binaries in the revised manuscript. Specifically, we added the following explanation: “In this context, insecure binaries refer to executable files or compiled firmware components that exhibit security weaknesses, including the use of outdated or vulnerable libraries, improper permissions, hard-coded credentials, or known exploit patterns. EMBA identifies these binaries by analyzing their metadata, embedded libraries, and behavioral characteristics during both static and dynamic analysis phases, allowing the tool to flag potential security risks automatically.” This addition provides more detail on how EMBA detects insecure binaries and the types of vulnerabilities it targets. (Line 52 – 57)
Comment 2:
In line 130 you seem to identify Wireshark as a static/dynamic analysis tool - since it is a packet capture tool, please explain further.
Response:
Thank you for your comment. We have clarified the description of the analysis tools in the revised manuscript. The updated text now reads: “Their study employs static and limited dynamic analysis techniques using tools such as IDA Pro and Ghidra for reverse-engineering malware and identifying vulnerabilities, while Wireshark is used to monitor network traffic generated by malware, rather than code analysis.” This change ensures accurate representation of each tool’s capabilities and distinguishes between code-level analysis and network monitoring.
Comment 3:
Provide a reference when you mention OWASP Top 10 in line 174.
Response:
Thank you for your comment. We have added a reference for the OWASP IoT Top 10, now cited as reference [20] in the revised manuscript (see line 191).
Comment 4:
There is a lot of repetition in the text at line 179 and 182 - please consider earlier text, and tighten up the prose. Similar in line 290, 298, 405, 492
Response:
Lines 182–195 have been revised to tighten the prose and reduce repetition.
The following sentence has been completely removed:
“While the above articles mainly discuss EMBA's capabilities and limitations in firmware security analysis, none specifically address our desired focus on comparing EMBA's performance on Cloud versus standalone personal computer or evaluating the differences between its various versions.”
The following sentence has been completely removed:
“The systems differed in processor capabilities, with PC1 featuring a quad-core processor and PC2 an octa-core processor. Both systems were equipped with 32 GB of RAM, ensuring sufficient memory for analysis”
The preceding text has been updated and improved for clarity and conciseness.
Comment 5:
Since the reference style is bracketed [22], there is no need to include author, date in line 200.
Response:
Date is removed.
Comment 6:
When you explicitly mention authors (which is good) as in line 211, it is not necessary to include the year.
Response:
Date is removed.
Comment 7:
You mention a dataset in line 330 - where did it come from?
Response:
Thank you for the comment. To avoid potential confusion, we have removed reference to the approximately 1,500 firmware samples and clarified that selection was based on size categories and the ability to execute most EMBA modules, keeping the focus on comparative runtime evaluation.
Comment 8:
How was it determined how many EMBA modules were executed?
Response:
The number of executed EMBA modules was determined by reviewing the customized scan profile configuration file. Each module enabled for the experiment was explicitly listed in this configuration, allowing us to verify which modules were active during the analysis. The corresponding section in the text (3.3.1) has been updated to reflect this clarification. (Line 410 - 421)
Comment 9:
You write "The YARA parameter remained disabled unless explicitly stated" - but was it explicitly stated anywhere? Please explain what YARA is.
Response:
The sentence regarding the YARA parameter has been removed, and the text has been updated accordingly.
Comment 10:
In the reference list, please provide the full author list for all references (e.g., [10], [14], [15], [27])
Response:
The reference list has been updated to include the full author lists for all relevant references ensuring complete and accurate attribution for each cited work.
Reviewer 2 Report
This paper presents a comparative evaluation of the EMBA firmware security analysis tool deployed in standalone and Microsoft Azure cloud environments. While the topic is relevant to IoT security, the manuscript suffers from several conceptual, methodological, and presentation weaknesses that prevent it from meeting the standards required for publication.
First, the title does not accurately reflect the specific scope, methodology, or experimental context of the study and should be revised accordingly. Similarly, the abstract lacks clarity regarding the paper’s primary contributions and novelty, making it difficult for readers to understand the significance of the work. A more concise and focused abstract is needed.
The introduction is insufficiently structured. It should clearly and explicitly list the main contributions of the paper in a point-wise manner, explain how these contributions relate to each other, and distinguish them from existing studies. In addition, the organization of the paper should be stated at the end of the introduction to improve readability.
The related work section is notably underdeveloped. It does not adequately cover recent literature, nor does it clearly identify research gaps that this study aims to address. The authors are encouraged to expand this section, include more up-to-date references (particularly from 2025–2026), and provide a comparative table highlighting differences between their approach and existing work.
From a methodological perspective, the experimental setup lacks representativeness. The evaluation relies on only two machines with nearly identical hardware characteristics, differing primarily in CPU core count. This limits the validity and generalizability of the results, especially for heterogeneous or enterprise-scale environments. Moreover, the hardware specifications of both the standalone system and the Azure virtual machine are not described in sufficient detail to ensure a fair comparison.
The evaluation dataset is limited in both size and diversity. Only a small number of firmware images with similar sizes are analyzed, which does not adequately demonstrate how EMBA scales to larger or more complex firmware. The manuscript also lacks a clear justification for the selection of these firmware samples and does not convincingly argue that they are representative of real-world IoT devices.
The performance analysis is incomplete, as it is based on a single or very limited number of runs. No statistical analysis is provided to capture execution-time variability. Including repeated experiments, averages, and confidence intervals would significantly strengthen the evaluation.
The cost analysis is also insufficiently detailed. It lacks a transparent breakdown of cloud resource usage, pricing assumptions, and billing duration, which undermines reproducibility and limits the practical usefulness of the results.
Finally, the discussion section does not adequately translate the findings into actionable insights. Clearer deployment recommendations tailored to different organizational sizes or use cases would improve the practical relevance of the study.
In summary, although the paper addresses an important problem, substantial revisions are required in terms of experimental design, evaluation depth, literature coverage, and clarity of presentation. For these reasons, the manuscript is not suitable for acceptance in its current form.
This paper presents a comparative evaluation of the EMBA firmware security analysis tool deployed in standalone and Microsoft Azure cloud environments. While the topic is relevant to IoT security, the manuscript suffers from several conceptual, methodological, and presentation weaknesses that prevent it from meeting the standards required for publication.
First, the title does not accurately reflect the specific scope, methodology, or experimental context of the study and should be revised accordingly. Similarly, the abstract lacks clarity regarding the paper’s primary contributions and novelty, making it difficult for readers to understand the significance of the work. A more concise and focused abstract is needed.
The introduction is insufficiently structured. It should clearly and explicitly list the main contributions of the paper in a point-wise manner, explain how these contributions relate to each other, and distinguish them from existing studies. In addition, the organization of the paper should be stated at the end of the introduction to improve readability.
The related work section is notably underdeveloped. It does not adequately cover recent literature, nor does it clearly identify research gaps that this study aims to address. The authors are encouraged to expand this section, include more up-to-date references (particularly from 2025–2026), and provide a comparative table highlighting differences between their approach and existing work.
From a methodological perspective, the experimental setup lacks representativeness. The evaluation relies on only two machines with nearly identical hardware characteristics, differing primarily in CPU core count. This limits the validity and generalizability of the results, especially for heterogeneous or enterprise-scale environments. Moreover, the hardware specifications of both the standalone system and the Azure virtual machine are not described in sufficient detail to ensure a fair comparison.
The evaluation dataset is limited in both size and diversity. Only a small number of firmware images with similar sizes are analyzed, which does not adequately demonstrate how EMBA scales to larger or more complex firmware. The manuscript also lacks a clear justification for the selection of these firmware samples and does not convincingly argue that they are representative of real-world IoT devices.
The performance analysis is incomplete, as it is based on a single or very limited number of runs. No statistical analysis is provided to capture execution-time variability. Including repeated experiments, averages, and confidence intervals would significantly strengthen the evaluation.
The cost analysis is also insufficiently detailed. It lacks a transparent breakdown of cloud resource usage, pricing assumptions, and billing duration, which undermines reproducibility and limits the practical usefulness of the results.
Finally, the discussion section does not adequately translate the findings into actionable insights. Clearer deployment recommendations tailored to different organizational sizes or use cases would improve the practical relevance of the study.
In summary, although the paper addresses an important problem, substantial revisions are required in terms of experimental design, evaluation depth, literature coverage, and clarity of presentation. For these reasons, the manuscript is not suitable for acceptance in its current form.
Author Response
We sincerely thank you for the detailed and constructive comments. The suggestions significantly helped us improve the clarity, structure, and technical rigor of the manuscript. All comments have been addressed point by point below.
Comment 1:
This paper presents a comparative evaluation of the EMBA firmware security analysis tool deployed in standalone and Microsoft Azure cloud environments. While the topic is relevant to IoT security, the manuscript suffers from several conceptual, methodological, and presentation weaknesses that prevent it from meeting the standards required for publication.
Response:
We thank the reviewer for the constructive feedback and for recognizing the importance of the topic to IoT security. In response to these concerns, the manuscript has been thoroughly revised to address the conceptual, methodological, and presentation-related issues raised.
Specifically, the conceptual framework has been clarified, the methodology section has been refined to improve transparency and reproducibility, and several sections of the manuscript have been revised to enhance clarity, reduce redundancy, and improve overall presentation quality. We believe that these revisions significantly strengthen the manuscript and bring it in line with the standards required for publication.
Comment 2:
First, the title does not accurately reflect the specific scope, methodology, or experimental context of the study and should be revised accordingly. Similarly, the abstract lacks clarity regarding the paper’s primary contributions and novelty, making it difficult for readers to understand the significance of the work. A more concise and focused abstract is needed.
Response:
Thank you for the reviewer’s feedback. The title has been revised to better reflect the study’s scope and experimental context by emphasizing the comparative performance evaluation of EMBA in cloud based and standalone environments. The abstract has also been rewritten to be more concise and to clearly state the paper’s primary contribution and novelty, highlighting its deployment focused experimental analysis. We believe these revisions address the reviewer’s concerns and improve the clarity and significance of the manuscript.
Comment 3:
The introduction is insufficiently structured. It should clearly and explicitly list the main contributions of the paper in a point-wise manner, explain how these contributions relate to each other, and distinguish them from existing studies. In addition, the organization of the paper should be stated at the end of the introduction to improve readability.
Response:
We thank the reviewer for the constructive feedback. The Introduction has been revised to improve structure and clarity. The main contributions of the paper are now explicitly listed in a point-wise format, highlighting their relationship with one another and their distinction from prior studies. Additionally, the organization of the paper is now clearly stated at the end of the Introduction to enhance readability and guide the reader through the subsequent sections.
Comment 4:
The related work section is notably underdeveloped. It does not adequately cover recent literature, nor does it clearly identify research gaps that this study aims to address. The authors are encouraged to expand this section, include more up-to-date references (particularly from 2025–2026), and provide a comparative table highlighting differences between their approach and existing work.
Response:
We appreciate the reviewer’s comment regarding the related work section. We have expanded the literature review to ensure comprehensive coverage, comprising more than 20 references covering EMBA, IoT firmware analysis techniques, and related security assessment frameworks. To further address the suggestion for up-to-date literature, we have incorporated a recent 2025 study by Wang, which provides a theoretical analysis of IoT firmware vulnerabilities and proposes countermeasure strategies. This addition underscores the ongoing relevance of integrating static, dynamic, and hybrid analysis methods in firmware assessment. With these updates, the literature review adequately covers recent work, highlights existing research gaps, and establishes a strong foundation for understanding the motivation and context of our study.
Comment 5:
From a methodological perspective, the experimental setup lacks representativeness. The evaluation relies on only two machines with nearly identical hardware characteristics, differing primarily in CPU core count. This limits the validity and generalizability of the results, especially for heterogeneous or enterprise-scale environments. Moreover, the hardware specifications of both the standalone system and the Azure virtual machine are not described in sufficient detail to ensure a fair comparison.
Response:
We thank the reviewer for the insightful comments regarding the representativeness of our experimental setup. We would like to clarify that the primary objective of this study was to compare EMBA’s behavior across three platforms, two standalone servers and a cloud-based virtual machine, while evaluating the impact of CPU resources and EMBA versions on analysis outputs. This setup provides sufficient contrast in processing power and architecture to assess performance and module execution effectively.
To address concerns regarding hardware detail transparency, we have included comprehensive specifications for all platforms, including CPU model, base clock, L3 cache, RAM type and size, disk type and capacity, and operating system. These details ensure that the experimental environment is clearly documented and reproducible.
While we acknowledge that the tested configurations do not capture all possible enterprise-scale or heterogeneous environments, they are adequate for evaluating the effects of EMBA versions and CPU scaling on firmware analysis outputs, which is the focus of this study. These limitations are now explicitly discussed in the manuscript to clarify the scope of the reported results.
Comment 6:
The evaluation dataset is limited in both size and diversity. Only a small number of firmware images with similar sizes are analyzed, which does not adequately demonstrate how EMBA scales to larger or more complex firmware. The manuscript also lacks a clear justification for the selection of these firmware samples and does not convincingly argue that they are representative of real-world IoT devices.
Response:
We thank the reviewer for highlighting the importance of dataset size and diversity. In response to this suggestion, the Materials and Methods section, particularly Section 3.1 “Firmware Selection and Size Categories,” has been revised to provide a more detailed justification of the selected firmware samples, including their size categories, device types, and typical use cases.
The selected firmware spans three representative size categories—small (<10 MB), medium (10–30 MB), and large (>30 MB)—and includes examples from consumer-grade devices (e.g., Wi-Fi routers), mid-range NAS systems, and industrial IoT gateways. These choices ensure that EMBA executes most of its modules, allowing a meaningful end-to-end performance evaluation. While the total number of firmware images remains limited due to the computationally intensive nature of EMBA scans, the revised selection provides diverse coverage in terms of size, complexity, and functional context, enhancing the representativeness of the dataset.
This revision reflects the reviewer’s valuable input and clarifies how the selected firmware supports the evaluation of EMBA’s performance across different types of real-world embedded systems.
Comment 7:
The performance analysis is incomplete, as it is based on a single or very limited number of runs. No statistical analysis is provided to capture execution-time variability. Including repeated experiments, averages, and confidence intervals would significantly strengthen the evaluation.
Response:
We thank the reviewer for highlighting the importance of multiple test repetitions and statistical analysis. In the revised manuscript, a total of 39 test executions were conducted across all platforms and EMBA versions. Each firmware sample was run multiple times, and we included average execution times and standard deviations for both the standalone PC and Azure VM environments. This information demonstrates the reproducibility of our results and provides a clear view of execution-time variability, fully addressing the reviewer’s concern regarding the completeness and statistical robustness of the performance evaluation.
Comment 8:
The cost analysis is also insufficiently detailed. It lacks a transparent breakdown of cloud resource usage, pricing assumptions, and billing duration, which undermines reproducibility and limits the practical usefulness of the results.
Response:
We thank the reviewer for this suggestion. Section 5.1 has been revised to provide a detailed cost analysis of EMBA deployment. We now report the VM configuration (Azure Standard D8s_v4, 8 vCPU, 32 GB RAM), attached 1 TB Premium SSD, Central US (San Antonio) region, and total runtime of 265 hours. Costs are transparently broken down as compute: 265 × $0.384 = $101.76, storage: $122.88, totaling approximately $225. Relevant pricing sources are also cited [35–37] to ensure reproducibility.
Comment 9:
Finally, the discussion section does not adequately translate the findings into actionable insights. Clearer deployment recommendations tailored to different organizational sizes or use cases would improve the practical relevance of the study.
Response:
We have added a set of deployment recommendations tailored to different organizational scales at the end of Section 5.1. Specifically, we now provide guidance for small research labs or individual practitioners, medium-sized groups, and large-scale or enterprise environments.
These recommendations highlight that cloud-based deployments (e.g., Azure virtual machines) offer flexibility, low upfront costs, and remote accessibility for smaller or occasional workloads, whereas standalone servers provide stable and reproducible execution for medium-sized groups. For large-scale or high-volume analyses, a hybrid approach combining high-performance standalone servers with supplementary cloud resources balances performance, reproducibility, cost-efficiency, and security.
We believe these additions improve the practical relevance of our discussion and clarify how EMBA can be deployed effectively across different use cases and organizational sizes.
Comment 10:
In summary, although the paper addresses an important problem, substantial revisions are required in terms of experimental design, evaluation depth, literature coverage, and clarity of presentation. For these reasons, the manuscript is not suitable for acceptance in its current form.
Response:
We sincerely thank the reviewer for the comprehensive feedback and constructive suggestions. In response, the manuscript has undergone substantial revisions to fully address the highlighted concerns:
- Experimental design and evaluation depth: All platform specifications, firmware sample selection, EMBA module configurations, and test repetitions (total 39 runs) are now clearly documented, with averages, standard deviations, and runtime variability reported. The cloud-based deployment cost analysis has been updated with transparent hourly and storage cost calculations (Section 5.1, [35–37]).
- Clarity and presentation: Text revisions have removed ambiguity and repetition, clarified version comparisons, and streamlined descriptions of tools, datasets, and workflows. Appendices and tables now provide comprehensive supplementary information (e.g., EMBA modules per version, firmware details).
- Literature coverage: The related work section has been expanded with one more recent study, emphasizing gaps addressed by our work and clearly distinguishing our contributions.
- Practical relevance: The discussion now includes actionable deployment recommendations for small, medium, and large organizations, bridging experimental results to real-world applications.
In addition, once the review process is completed, we plan to further refine the manuscript by utilizing the journal’s Author Services to perform a final professional English language improvement. This step is intended to ensure maximum clarity, consistency, and overall linguistic quality of the final version.
We believe these revisions comprehensively address the reviewer’s concerns, improve reproducibility, and strengthen the manuscript’s contribution to IoT firmware security analysis.
Round 2
Reviewer 1 Report
Authors have addressed my comments appropriately.
Authors have addressed my comments appropriately.
Author Response
We thank the reviewer for careful evaluation and for acknowledging that the comments have been appropriately addressed.
Reviewer 2 Report
The instance used for the simulation and experimental results is not sufficiently representative to substantiate the paper’s contributions.
The instance used for the simulation and experimental results is not sufficiently representative to substantiate the paper’s contributions.
Author Response
Comment:
The instance used for the simulation and experimental results is not sufficiently representative to substantiate the paper’s contributions.
Response:
We thank the reviewer for this comment and the opportunity to further clarify the scope and robustness of the experimental design. The objective of this study is to provide a systematic, controlled, and reproducible evaluation of EMBA across different execution environments and tool versions, rather than to exhaustively model all possible real-world IoT deployment scenarios.
Within this clearly defined scope, the experimental setup ensures meaningful representativeness across multiple dimensions. The evaluation includes four distinct firmware images spanning different size categories, three execution environments (two standalone systems and one cloud-based virtual machine), and three EMBA versions, resulting in a total of 39 experimental runs. To explicitly capture execution-time variability, each test was repeated three times on standalone PC environments and two times on the cloud-based platform, with averaged results reported in the manuscript.
In the revised manuscript, the experimental coverage has been further strengthened. Specifically, in the EMBA Version 1.4.1 and 1.4.2 Comparison section, two additional test runs were conducted on the second standalone platform (PC2), enabling a more robust comparison of version-dependent behavior under identical hardware conditions.
Taken together, this multi-dimensional and repeated-execution evaluation framework provides sufficient empirical evidence to substantiate the paper’s stated contributions while ensuring transparency and reproducibility. In addition, the Future Research section has been updated to explicitly outline natural extensions of this work, including broader firmware diversity, additional deployment environments, and large-scale orchestration strategies, without detracting from the validity of the current study.
