IoT Traffic: Modeling and Measurement Experiments
Round 1
Reviewer 1 Report
The authors present a useful testbed for smart home iot. They derive a iot traffic generator to finally analyse (though entropies values) the observed traffic for classication.
The paper is well written and easy to understand. we can regret that iottgen is not available publicly for reproducible and open research. is that possible ton include for the current article?
I am wondering if there is no measure of similarity/dissimilarity for the presented Behavior Shapes (please discuss about this oint somewhere)
I think section 6 can be reorganised (between synthetic and measured traffic) to appear more clearly than now
last but not least, it should be interesting to improve the ML part (very classical for the moment) in future works specially without trafic entropy (using trafic features by example) to verify the efficiency of the method
please do a last careful proofreading before publication
li 385 paramters
li 389-390 put a sentence btwn
caption of Figure 16: to read
let a blank btwn Figure caption and text e.g page 12
li 513 Biomedical -> biomedical
Author Response
Dear reviewer, thank you very much for your careful and constructive comments We review your comments point by point. The manuscript has been carefully revised based on your comments and suggestions. We hope that it fulfills your expectations. Each comment is explained and answered in detail as following:
Comment 1: The authors present a useful testbed for smart home iot. They derive a iot traffic generator to finally analyse (though entropies values) the observed traffic for classication.
The paper is well written and easy to understand.
Response 1: Thank you for your careful reading of our manuscript.
Comment 2: we can regret that iottgen is not available publicly for reproducible and open research. is that possible ton include for the current article?
Response 2: Thank you for your comment. IoTTGen is still a prototype under development and we are planning to release publicly the source code as well as the Data set soon.
Comment 3: I am wondering if there is no measure of similarity/dissimilarity for the presented Behavior Shapes (please discuss about this point somewhere)
Response 3: Thank you for your comment. There are several options to represent the network entropy. In this work, we represent the entropy value into Behavior Shape graphs in order to visually observe the characteristics of IoT Traffic. For measuring the similarity of shapes, we can use Euclidean distance between shapes. Based on that value, we can evaluate the difference between shapes; thereby, we can detect anomalies. However, in this paper, we want to focus on IoTTGen and its ability. We thank gratefully the reviewer for your suggestion, our future work will investigate deeper to this point.
The detailed revision can be shown in the Conclusion from line 524 to line 526.
Comment 4: I think section 6 can be reorganised (between synthetic and measured traffic) to appear more clearly than now
Response 4: Thank you for your suggestion. Section 6 presents most of the results regarding the Traffic Behavior Shape. From previous Sections, we showed (i) Smart-home IoT Testbed (Section 3); (ii) our new IoTTGen tool for generating IoT Traffic (Section 4) ; (iii) Measurement experiments relying on the testbed (measured) and IoTTGen (synthetic). We also discussed in Section 5 about the anomalous traffic.
Thus, for Section 6, there could have been several ways for presenting our results. For instance, we could have chosen (1) to present first all the experiments with synthetic traffic from IoTTGen, and then the experiments with measured traffic from testbed. Differently, we have chosen (2) to present first all the experiments with regular traffic, and then second, experiments with anomalous traffic. The Rationale is that we aim at comparing the measured traffic from testbed with regards to the synthetic traffic generated by IoTTGen and this organization is more suitable for comparisons.
We beneficiate from your suggestion as we improved the standfirst of the Section 6 and we introduce more clearly the experiments presented throughout this Section.
The detailed revision can be shown in Section 6 from line 343 to line 348.
Comment 5: last but not least, it should be interesting to improve the ML part (very classical for the moment) in future works specially without trafic entropy (using trafic features by example) to verify the efficiency of the method
Response 5: Thank you very much for your comment. As this paper main focus is on IoT traffic modeling and experiments, device identification and anomaly detection through ML algorithms is an application for cybersecurity and is still ongoing work. As you mention, it is definitely part of our future work and especially as it is now possible to generate any kind of IoT platform configuration with our IoTTGen tool; our future work will entail deeper use of AI for detecting anomalies.
The detailed revision can be shown in the Conclusion from line 529 to line 530.
Comment 6: please do a last careful proofreading before publication
Response 6: We sincerely present our deepest apologies for any inconvenience. For this second version, we paid deep attention to improve the quality of our manuscript. We took into account all your comments and we carefully proofread the entire manuscript. We are confident that our manuscript fulfills now all your expectations. Thank you very much for your important feedback.
Finally, thank you for taking time to review this manuscript, and we appreciate your careful comments to improve our manuscript.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This paper contains a network packet generation model and results under various conditions in a smart home. Overall, it is intuitive, but it would be better to emphasize the value as a thesis as follows.
- The patterns in which data is generated by the sensor can also vary – periodic or aperiodic. It is difficult to see that the experimental results reflect all cases. Therefore it would be better to conduct a traffic generation experiments for more diverse cases (more experiments for various experimental conditions).
- The importance of the results through the various figures doesn't seem too great. It feels like it just showed the collected figures nicely. I hope to reinforce this part a little more.
Author Response
Dear reviewer, thank you very much for your careful and constructive comments We review your comments point by point. The manuscript has been carefully revised based on your comments and suggestions. We hope that it fulfills your expectations. Each comment is explained and answered in detail as following:
Comment 1: This paper contains a network packet generation model and results under various conditions in a smart home. Overall, it is intuitive, but it would be better to emphasize the value as a thesis as follows.
Response 1: Thank you for your careful reading of our manuscript. We are paying deep attention to your valuable following comments.
Comment 2: The patterns in which data is generated by the sensor can also vary – periodic or aperiodic. It is difficult to see that the experimental results reflect all cases. Therefore it would be better to conduct traffic generation experiments for more diverse cases (more experiments for various experimental conditions).
Response 2: Thank you very much for your comment. Our network packet generator for IoT, IoTTGen, is presented in Section 4. Its architecture and way of working derived from our observations from the testbed we set up (section 3).
First of all, for clarifications, it is possible to use IoTTGen and generate traffic with any kind of parameters (see Section 4.2.1): synthetic or measured from realistic testbed. In our experiments, as we observed that IoT traffic parameters (packet size, period) do not vary with time (e.g., devices send data with same features – size or period); it is however perfectly possible with IoTTGen to generate traffic whose parameters (packet size, period) are not periodic or vary with payload size (e.g., 1B-1KB, 1ms-100s, etc.). For instance, it is possible to model a camera which generates low traffic when no detection occurred, while generating high traffic when detecting movements (larger packets, higher frequency, etc.).
Furthermore, sensor traffic is also generated based on human behavior which is not predictable. For instance, users can follow daily routine, but unexpected schedule modification can happen and impact the use of the smart home environment (e.g., not using the light or plug if coming home too late, etc.) This unpredictable human behavior can also be modeled within IoTTGen by configuring the specific scenario. For instance, for a one-week trace, one can configure routine on Monday/Tuesday/Wednesday/Friday, while another usage on Thursday and different pattern on the weekend. Thus, the generated traffic can range from fixed parameters to random traffic following scenarios used by the generator (See Section 4.2.2).
Second, we would like to bring to the attention of the reviewer on the Section 5.3 which presents another experiment set up with more than 50 devices. Even though we focus on a smart-home configuration in our experiments (camera, light, plugs), this experiment is to show the ability of IoTTGen to generate traffic for any kind of configuration. Besides, in our previous paper already quoted in [9], we were also generated traffic for Biomedical environment, another testbed use case. In this extended version of our paper, we chose to focus on smart home environment as we also wanted to include a Section on the device identification and IoT traffic anomaly detection (Section7)
The detailed revision can be shown in the Section 4 from line 224 to line 229 and line 237 to line 240.
Comment 3: The importance of the results through the various figures doesn't seem too great. It feels like it just showed the collected figures nicely. I hope to reinforce this part a little more.
Response 3: Thank you very much for your comment and we understood your point of view and we will try to provide you more information. First of all, we update the standfirst of Section 6 in order to introduce more clearly all the experiments in this Section. This Section is also all about comparing generated traffic (IoTTGen) and measured traffic (Testbed), globally (smart-Home) or for each device (camera, plug, etc.), and then including certain kind of anomalies. Furthermore, the traffic characteristics are observed visually through Entropy Behavior Shape graphs and it is essential to show a large collection of data to compare traffic.
Thus, Fig 7 and Fig 8 illustrate the impact of different set of parameters on the traffic. Figure 9 is showing global traffic for the IoT platform, while Figures 10-14 show the traffic for each individual device. Figure 15 compared the synthetic traffic with measured traffic and Figure 16 shows on/off scenario for a specific device. Fig 17-21 are showing the synthetic and testbed traffic including the anomalies. Finally, all figures have their own interest to demonstrate the ability of IoTTGen to capture the essential characteristics of smart-home IoT traffic and its efficiency to generate IoT traffic.
The detailed revision can be shown in the Section 6 from line 343 to line 348.
Finally, thank you for taking time to review this manuscript, and we appreciate your careful comments to improve our manuscript.
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper designed novel set up for an IoT design using the smart home environment and performed extensive measurement experiment campaigns in order to study IoT Traffic. Their IoTTGen model is formed as a packet-level traffic generator to generate traffic from multiple devices to emulate larger-scale scenarios with different devices and under different network conditions. The idea is interesting, and the work is publishable. There are some comments the reviewer has that are (i) The complexity of the proposed IoT design model should neatly be added. (ii) It is important o better reflect the contribution and motivation. How they can address (contribution) need to form better. (iii) The background is limited and some recent techniques can be integrated and addressed like ‘Neural Architecture Search for Robust Networks in 6G-enabled Massive IoT Domain’ and ‘Voice-transfer attacking on industrial voice control systems in 5G-aided IIoT domain’.
Author Response
Dear reviewer, thank you very much for your careful and constructive comments. We review your comments point by point. The manuscript has been carefully revised based on your comments and suggestions. We hope that it fulfills your requirements. Each comment is explained and answered in detail as following:
Comment 1: The paper designed novel set up for an IoT design using the smart home environment and performed extensive measurement experiment campaigns in order to study IoT Traffic. Their IoTTGen model is formed as a packet-level traffic generator to generate traffic from multiple devices to emulate larger-scale scenarios with different devices and under different network conditions. The idea is interesting, and the work is publishable.
Response 1: Thank you very much for your careful reading of our manuscript and your positive feedback. We will take all your comments into account to improve our manuscript.
Comment 2: The complexity of the proposed IoT design model should neatly be added.
Response 2: Thank you for your valuable comment. We now propose a discussion and add a table (Table 4) on the complexity of our model in Section 5. IoTTGen is able to generate a huge amount of traffic in a very limited time, which makes this tool very useful to model IoT network.
The detailed revision can be shown in the Section 5 from line 251 to line 255.
Comment 3: It is important o better reflect the contribution and motivation. How they can address (contribution) need to form better.
Response 3: Thank you very much for your suggestion. Based on your comment and from other reviewers, we update our manuscript which has now a better emphasize on our contribution.
The detailed revision can be shown at the lines 177-182, 343-348, 361-362, 406-408.
Comment 4: The background is limited and some recent techniques can be integrated and addressed like ‘Neural Architecture Search for Robust Networks in 6G-enabled Massive IoT Domain’ and ‘Voice-transfer attacking on industrial voice control systems in 5G-aided IIoT domain’.
Response 4: Thank you very much. ML algorithm is an application of our study to identify devices and detect anomalies. The future work will focus mostly on this identification part based on synthetic and generated traffic. We thank gratefully the reviewer for its useful recommendation; the provided papers were very informative and are now included in our Related work.
The detailed revision can be shown in the Section 2 from lines 74-78 and lines 133-136.
Finally, thank you for taking time to review this manuscript, and we appreciate your careful comments to improve our manuscript.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
The updated file is properly addressed my raised comments. It can publish in this form.
