Performance Evaluation of Communication Systems Used for Internet of Things in Agriculture
Round 1
Reviewer 1 Report
This paper introduced the result of performance evaluation of LoraWAN.
The authors experimented by changing various parameters, and presented the result graphs.
There are some comments as follows:
1. The authors said that they measured the communication system performance in agriculture, but in the opinion of the reviewer, it is difficult to think that the characteristics of agriculture are reflected in the experimental environment.
The authors have to justify why this experiment is a performance measure for an argriculture environment.
2. Following the above opinion, except for agriculture environment,the experiments in this paper have already been performed in many other papers. The authors have to present any other novelties of this paper, if any.
3. The authors should provide detailed simulation parameters. In particular, it should be specified how many times the experiment was performed.
4. In Figure 11, when the number of nodes is 1500, why does the packet delivery ratio increase? In the opinion of the reviewer, it seems to be an error of not repeating the experiment sufficiently.
5. In Figure 11, when the number of nodes increases, the reason why the packet delivery ratio decreases is because of obstacles and interference in the propagation medium. Is there any other effect of the protocol itself?
Author Response
Responses to Reviewer 1
This paper introduced the result of the performance evaluation of LoraWAN.
The authors experimented by changing various parameters and presented the result graphs.
Thank you for the positive comments
There are some comments as follows:
Comment 1: The authors said that they measured the communication system performance in agriculture, but in the opinion of the reviewer, it is difficult to think that the characteristics of agriculture are reflected in the experimental environment.
The agriculture scenario is presented in the form of an environment where there are few obstacles, almost no neighbor networks working on the same frequency bands, and long-distance fields to be covered. In the document, sensor data and application servers related to agriculture are out of the scope of the study.
Comment 2: The authors have to justify why this experiment is a performance measure for an agriculture environment.
Metrics like Packet Delivery Ratio, Energy Consumption and Packet Collisions are collected from the simulations. Those are metrics related to the scalability of the network to be applied in an agricultural environment, thus the performance is measured.
Comment 3: Following the above opinion, except for agriculture environment, the experiments in this paper have already been performed in many other papers. The authors have to present any other novelties of this paper, if any.
We have simulated more nodes than other studies, since in the agricultural scenario presented, the sensor deployment density needed is higher. We also applied the test over the frequency plan for Ecuador, which is different from the used in Europe and US, where most of the studies have been made.
Comment 4: . The authors should provide detailed simulation parameters. In particular, it should be specified how many times the experiment was performed.
For each measurement presented, the simulation window time was 1 day, and was the result of the average of 5 simulations for node quantity below 1000. For node quantities above 1000, the results are the outcome of one simulation due to the long computation time taken.
Comment 5: . In Figure 11, when the number of nodes is 1500, why does the packet delivery ratio increase? In the opinion of the reviewer, it seems to be an error of not repeating the experiment sufficiently.
The data presented is the result of the simulations. As explained in comment 4, for nodes below 1000 nodes the result is the average of 5 simulations, while for nodes above 1000 the result is taken from just one simulation.
Comment 6: . In Figure 11, when the number of nodes increases, the reason why packet delivery ratio decreases are because of obstacles and interference in the propagation medium. Is there any other effect of the protocol itself?
The delivery ratio is degraded also by the collisions of the packets sent by the nodes over the medium itself.
Reviewer 2 Report
The article is devoted to Performance evaluation in communication systems used in the Internet of things in agriculture. Indeed, few studies have focused on the use of communication protocols such as RFID/WiFi/Bluetooth for monitoring variables related to the agricultural sector. This manuscript is about the performance evaluation of communication systems used in the Internet of Things for the agricultural sector. LPWAN technology was used as the communication system, as it allows the deployment of low-power wide area networks specifically for the Internet of Things. Within LPWAN, LoRaWAN was chosen because it meets the project's requirements in terms of scalability, power consumption, frequency, data transfer rate, and cost.
Comments
- Abstract - should be more in line with the goals, objectives and results of the articles, and the volume of general phrases should be minimized
- Introduction - although there are few articles in the field of agriculture, a review of related fields is needed. It would not be superfluous to have a related work section
- Methodology - table 1 and tables 4 left the margins of the document
- 2.7. Simulation Environment - you need more description to be able to repeat the computational experiment, or place it in an open range
Results - More information about the design of the experiment is required
Conclusion - all is well
Author Response
Reviewer 2
The article is devoted to Performance evaluation in communication systems used in the Internet of things in agriculture. Indeed, few studies have focused on the use of communication protocols such as RFID/WiFi/Bluetooth for monitoring variables related to the agricultural sector. This manuscript is about the performance evaluation of communication systems used in the Internet of Things for the agricultural sector. LPWAN technology was used as the communication system, as it allows the deployment of low-power wide area networks specifically for the Internet of Things. Within LPWAN, LoRaWAN was chosen because it meets the project's requirements in terms of scalability, power consumption, frequency, data transfer rate, and cost.
Thank you for the positive comments
Comment 1: - Abstract - should be more in line with the goals, objectives, and results of the articles, and the volume of general phrases should be minimized
The summary has been changed:
The rapid development of the Internet of Things (IoT) technology provides more opportunities for agriculture production. This technology can be used to connect various types of agricultural devices, which can collect and send data to servers for analysis. These tools help farmers to optimize the production of their crops. However, one of the main problems that arise in agricultural areas is the lack of connectivity or its poor quality. For these reasons, this paper presents a performance evaluation of the communication systems used in the IoT for agriculture, such as the packet delivery factor, energy consumption, and packet collisions. To achieve this aim, an analysis of the main LPWAN protocols and their application was carried out, concluding that the best ones are LoRa and LoRaWAN. After that, we analyzed simulation tools and select Omnet++ together with the FLora library as the best option. In the first stage of the simulations, the performance of LoRa and LoRaWAN is evaluated by comparing the average propagation with ideal conditions against moderate losses over the air that emulates the rural environment of the coastal region in Ecuador. In the second phase, metrics like delivery ratio and energy consumption are collected by simulating the communication between an increasing number of nodes with one and two gateways. Results show that two gateways with Adaptive Data Rate active increase the delivery ratio of the network while consuming the same amount of energy per node. Finally, a comparison is made of the proposed simulated scenario in this project with other research works that allowed validating the analytical and simulated results.
Comment 2: - Introduction - although there are few articles in the field of agriculture, a review of related fields is needed. It would not be superfluous to have a related work section
The following references was included in the introduction section:
[1] Y. Fang, et al., “Irregular-mapped protograph LDPC-coded modulation: A bandwidth-efficient solution for 6G-enabled mobile networks,” IEEE Transactions on Intelligent Transportation Systems, vol. PP, no. 99, pp. 1-14, Oct. 2021, doi: 10.1109/TITS.2021.3122994.
[2] Shao, et al., “Survey of turbo, LDPC, and polar decoder ASIC implementations,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2309–2333, 3rd Quart. 2019.
[3] L. Dai, et al., “Protograph LDPC-coded BICM-ID with irregular CSK mapping in visible light communication systems,” IEEE Transactions on Vehicular Technology, 2021, 70(10): 11033-11038.
[4] H. Ji, S. Park, J. Yeo, Y. Kim, J. Lee, and B. Shim, “Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects,” IEEE Wireless Commun., vol. 25, no. 3, pp. 124–130, Jun. 2018. URLLC
The following paragraph was included in the introduction section:
On the other hand, low-density parity-check (LDPC) codes and Polar codes, and Ultra-Reliable and Low-Latency Communications (URLLC) are of particular importance to improve the transmission reliability of wireless networks and hence have been included in the 5G New Radio Standard [24–26].
[24] H. Ji, S. Park, J. Yeo, Y. Kim, J. Lee, and B. Shim, “Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects,” IEEE Wireless Commun., vol. 25, no. 3, pp. 124–130, Jun. 2018. URLLC.
[25] L. Dai, et al., “Protograph LDPC-coded BICM-ID with irregular CSK mapping in visible light communication systems,” IEEE Transactions on Vehicular Technology, 2021, 70(10): 11033-11038.
[26] Shao, et al., “Survey of turbo, LDPC, and polar decoder ASIC implementations,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2309–2333, 3rd Quart. 2019.
[27] Y. Fang, et al., “Irregular-mapped protograph LDPC-coded modulation: A bandwidth-efficient solution for 6G-enabled mobile networks,” IEEE Transactions on Intelligent Transportation Systems, vol. PP, no. 99, pp. 1-14, Oct. 2021, doi: 10.1109/TITS.2021.3122994
Comment 3: - Methodology - table 1 and tables 4 left the margins of the document
The tables were placed in the format and centered with the text
Comment 4: - 2.7. Simulation Environment - you need more description to be able to repeat the computational experiment or place it in an open range
For each measurement presented, the simulation window time was 1 day and was the result of the average of 5 simulations for node quantity below 1000. For node quantities above 1000, the results are the outcome of one simulation due to the long computation time taken.
Comment 5: Results - More information about the design of the experiment is required
For each measurement presented, the simulation window time was 1 day, and was the result of the average of 5 simulations for node quantity below 1000. For node quantities above 1000, the results are the outcome of one simulation due to the long computation time taken.
Comment 6: Conclusion - all is well
Thank you for the positive comments
Reviewer 3 Report
This work achieves the evaluation of the LoRaWAN in internet of things for the agricultural sector via OMNET++ simulator. The ideal conditions and moderate losses scenarios are considered in this paper for the evaluation. Based on the results of the simulations performed for the evaluation of the LoRaWAN network in the proposed scenarios, some useful conclusions are collected in this paper.
- Tables 1 and 4 are not complete. Please modify the format.
- The authors use too long space for introducing the basics of LoRa/LoRaWAN in section 2, which makes it difficult for readers to understand the key points of this work.
- Please provide the download URLs of NS3, PMNET++, and LoRasim simulators so that readers can better follow this work.
- The reviewer does not understand how the authors implement the real agricultural-environment (e.g., humidity parameter, temperature parameter……) in the simulator.
- It is necessary for the authors to briefly and clearly summarize the contributions of this work in section 1.
- What is the relationship between the first stage simulation and the second stage simulation mentioned in the abstract?
- Another important concern is the transmission reliability of the LoRAWAN in IoT. How to guarantee the data transmission reliability in such a framework? Which type of channel coding scheme are considered? The authors ignored this issue. Please provide a short discussion in Introduction Section and the System model. In particular, LDPC codes and Polar codes are of particular importance to improve the transmission reliability of wireless networks, and hence have been included in 5G New Radio Standard [1-4]. The authors may property mention this issue and include the corresponding reference.
[1] Y. Fang, et al., “Irregular-mapped protograph LDPC-coded modulation: A bandwidth-efficient solution for 6G-enabled mobile networks,” IEEE Transactions on Intelligent Transportation Systems, vol. PP, no. 99, pp. 1-14, Oct. 2021, doi: 10.1109/TITS.2021.3122994.
[2] Shao, et al., “Survey of turbo, LDPC, and polar decoder ASIC implementations,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2309–2333, 3rd Quart. 2019.
[3] L. Dai, et al., “Protograph LDPC-coded BICM-ID with irregular CSK mapping in visible light communication systems,” IEEE Transactions on Vehicular Technology, 2021, 70(10): 11033-11038.
[4] H. Ji, S. Park, J. Yeo, Y. Kim, J. Lee, and B. Shim, “Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects,” IEEE Wireless Commun., vol. 25, no. 3, pp. 124–130, Jun. 2018.
- Some typos should be corrected throughout this paper.
Author Response
Reviewer 3
This work achieves the evaluation of the LoRaWAN in the internet of things for the agricultural sector via OMNET++ simulator. The ideal conditions and moderate losses scenarios are considered in this paper for evaluation. Based on the results of the simulations performed for the evaluation of the LoRaWAN network in the proposed scenarios, some useful conclusions are collected in this paper.
Thank you for the positive comments
Comment 1: Tables 1 and 4 are not complete. Please modify the format.
The tables were placed in the format and centered with the text
Comment 2: The authors use too long space for introducing the basics of LoRa/LoRaWAN in section 2, which makes it difficult for readers to understand the key points of this work.
This section intends to present the most relevant characteristics of LPWAN Technologies for those readers related to agronomy who are unaware of the subject.
Comment 3: Please provide the download URLs of NS3, OMNET++, and LoRasim simulators so that readers can better follow this work.
Added three references (with download URLs) on page 10.
[55]. NS3, https://www.nsnam.org (revised january 2022).
[56]. Omnet++, https://omnetpp.org (revised january 2022).
[57]. LoRaSim, https://www.lancaster.ac.uk/scc/sites/lora/lorasim.html (revised January 2022)
Comment 4: The reviewer does not understand how the authors implement the real agricultural-environment (e.g., humidity parameter, temperature parameter……) in the simulator.
The agriculture scenario is presented in the form of an environment where there are few obstacles, almost no neighbor networks working on the same frequency bands, and long-distance fields to be covered. In the document, sensor data and application servers related to agriculture are out of the scope of the study.
Comment 5: It is necessary for the authors to briefly and clearly summarize the contributions of this work in section 1.
Section 1:
Latency is considered one of the parameters that allow measuring the performance of a communications network; however, there are other variables that are indicators of the quality of links for data transmission, such as throughput, latency, speed and propagation delay, network capacity, range coverage, device lifetime, duration of useful life, service quality and cost. If a network does not have good performance, it can cause delays, loss of information, and limitations on sending information.
The importance of implementing this system of networks for innovation in the agricultural sector will allow for greater profitability in production and at the same time reduce fertilizer use and environmental impacts. Farmers will have information and statistics on crop growth, smartphones may be used to remotely control crops, equipment, and decision-making.
The summary has been changed:
The rapid development of the Internet of Things (IoT) technology provides more opportunities for agriculture production. This technology can be used to connect various types of agricultural devices, which can collect and send data to servers for analysis. These tools help farmers to optimize the production of their crops. However, one of the main problems that arise in agricultural areas is the lack of connectivity or its poor quality. For these reasons, this paper presents a performance evaluation of the communication systems used in the IoT for agriculture, such as the packet delivery factor, energy consumption, and packet collisions. To achieve this aim, an analysis of the main LPWAN protocols and their application was carried out, concluding that the best ones are LoRa and LoRaWAN. After that, we analyzed simulation tools and select Omnet++ together with the FLora library as the best option. In the first stage of the simulations, the performance of LoRa and LoRaWAN is evaluated by comparing the average propagation with ideal conditions against moderate losses over the air that emulates the rural environment of the coastal region in Ecuador. In the second phase, metrics like delivery ratio and energy consumption are collected by simulating the communication between an increasing number of nodes with one and two gateways. Results show that two gateways with Adaptive Data Rate active increase the delivery ratio of the network while consuming the same amount of energy per node. Finally, a comparison is made of the proposed simulated scenario in this project with other research works that allowed validating the analytical and simulated results.
Comment 6: What is the relationship between the first stage simulation and the second stage simulation mentioned in the abstract
The stage is the word used to describe that firstly we only make simulations with one gateway to measure the performance of an ideal medium vs a medium with moderate losses. Then, we use two gateways for collecting the remaining metrics; this would be the second stage.
Comment 7: Another important concern is the transmission reliability of the LoRAWAN in IoT. How to guarantee data transmission reliability in such a framework? Which type of channel coding scheme is considered? The authors ignored this issue.
The coding ratio for reliability used by default in the simulations are 4 / 5. However, there are no options in the simulation tool to dynamically change this parameter while the simulations are running.
Comment 8: Please provide a short discussion in Introduction Section and the System model. In particular, LDPC (low-density parity-check) codes and Polar codes are of particular importance to improve the transmission reliability of wireless networks, and hence have been included in 5G New Radio Standard [1-4]. The authors may property mention this issue and include the corresponding reference.
[1] Y. Fang, et al., “Irregular-mapped protograph LDPC-coded modulation: A bandwidth-efficient solution for 6G-enabled mobile networks,” IEEE Transactions on Intelligent Transportation Systems, vol. PP, no. 99, pp. 1-14, Oct. 2021, doi: 10.1109/TITS.2021.3122994.
[2] Shao, et al., “Survey of turbo, LDPC, and polar decoder ASIC implementations,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2309–2333, 3rd Quart. 2019.
[3] L. Dai, et al., “Protograph LDPC-coded BICM-ID with irregular CSK mapping in visible light communication systems,” IEEE Transactions on Vehicular Technology, 2021, 70(10): 11033-11038.
[4] H. Ji, S. Park, J. Yeo, Y. Kim, J. Lee, and B. Shim, “Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects,” IEEE Wireless Commun., vol. 25, no. 3, pp. 124–130, Jun. 2018. URLLC
The following paragraph was included in the introduction section:
On the other hand, low-density parity-check (LDPC) codes and Polar codes, and Ultra-Reliable and Low-Latency Communications (URLLC) are of particular importance to improve the transmission reliability of wireless networks and hence have been included in the 5G New Radio Standard [24–26].
[24] H. Ji, S. Park, J. Yeo, Y. Kim, J. Lee, and B. Shim, “Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects,” IEEE Wireless Commun., vol. 25, no. 3, pp. 124–130, Jun. 2018. URLLC.
[25] L. Dai, et al., “Protograph LDPC-coded BICM-ID with irregular CSK mapping in visible light communication systems,” IEEE Transactions on Vehicular Technology, 2021, 70(10): 11033-11038.
[26] Shao, et al., “Survey of turbo, LDPC, and polar decoder ASIC implementations,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2309–2333, 3rd Quart. 2019.
[27] Y. Fang, et al., “Irregular-mapped protograph LDPC-coded modulation: A bandwidth-efficient solution for 6G-enabled mobile networks,” IEEE Transactions on Intelligent Transportation Systems, vol. PP, no. 99, pp. 1-14, Oct. 2021, doi: 10.1109/TITS.2021.3122994.
Comment 9: Some typos should be corrected throughout this paper.
The errors detected in the document were corrected
Round 2
Reviewer 1 Report
This paper introduced the result of the performance evaluation of LoraWAN.
The authors experimented by changing various parameters and presented the result graphs.
In the opinion of the reviewer, the authors did not respond sufficiently to the comments of the previous review, and could not find any improvement in the paper over the previous version.
In the opinion of the reviewer, it would be better if corrections to previous comments were clearly presented.
Author Response
Suggested corrections are in the attached file
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors carefully treated my comments, I am ready to agree with the arguments of the authors.
Article can be accepted
Author Response
Thank you for the positive comments
Reviewer 3 Report
The authors have very carefully address all my comments in the previous round review. I believe this paper is of high quality and can be accepted as is.
Thanks for the great effort of the authors.
Author Response
Thank you for the positive comments
