Review Reports - Vulnerabilities and Limitations of MQTT Protocol Used between IoT Devices

Round 1

Reviewer 1 Report

The topic treated by the authors is of great interest and relevance: the myriad of networked objects with which we share our data require us to guarantee their privacy, security, confidentiality and integrity. About that, the authors present a method, called Value-to-Keyed-Hash Message Authentication Code mapping that uses signatures to send messages, instead of encryption, by implementing a Keyed-Hash Message Authentication Code generation algorithm.

Although, as I said, the topic is particularly relevant, I believe that the proposed solution is not so original and innovative. Some useful references can be found here:

https://acadpubl.eu/hub/2018-119-12/articles/5/1291.pdf

https://ralphholz.science/publications/AutomatedAnalysisOfSecureInternetOfThingsProtocols.pdf

For this reason, I think that the paper does not present a sufficiently innovative solution to the problem addressed to be accepted.

Author Response

Point 1: The topic treated by the authors is of great interest and relevance: the myriad of networked objects with which we share our data require us to guarantee their privacy, security, confidentiality and integrity. About that, the authors present a method, called Value-to-Keyed-Hash Message Authentication Code mapping that uses signatures to send messages, instead of encryption, by implementing a Keyed-Hash Message Authentication Code generation algorithm.

Although, as I said, the topic is particularly relevant, I believe that the proposed solution is not so original and innovative. Some useful references can be found here:

https://acadpubl.eu/hub/2018-119-12/articles/5/1291.pdf

https://ralphholz.science/publications/AutomatedAnalysisOfSecureInternetOfThingsProtocols.pdf

Response 1: The paper ‘Vulnerabilities and Limitations of MQTT Protocol Used Between IoT Devices’, presents a novel approach of achieving confidentiality and integrity of information shared between IoT devices, called Value-to-HMAC mapping.

Although there are other security solutions, such as SSL/TLS, that are more comprehensive, the paper considers that IoT devices have a very limited amount of resources and cannot use such protocols efficiently. Therefore, the method was designed in order to maximize performance on resource-constrained devices.

The Value-to-HMAC mapping method is not comparable in security features with the SSL/TLS, as it is only able to achieve confidentiality and integrity of information. However, the method of mapping values to hashes could serve as a base for future improvements and additions. As the data set generated by the timing script shows, the Value-to-HMAC mapping method assures both confidentiality and integrity of information while being considerably faster than using an encryption algorithm in conjunction with an integrity checking mechanism.

A different implementation of the HMAC algorithm is covered in [1], where the authors assess the performance of HMAC functions, based on different hashing algorithms, in order to achieve integrity of information. However, the method described in ‘Vulnerabilities and Limitations of MQTT Protocol Used Between IoT Devices’, designated Value-to-HMAC mapping, uses an HMAC digest to achieve both integrity and confidentiality.

As mentioned, the method is able to achieve confidentiality and integrity by using only one mechanism. This fact allows the HMAC mapping method to be significantly faster than even using a symmetric-key encryption algorithm in conjunction with an integrity checking mechanism. The chart in Figure 1 uses the data set created by the script’s timing function and shows a comparison between different HMAC functions (based on different hashing algorithms) and the encryption phase of AES in CBC mode.

As described in the paper, the hash tables are being generated on each node and do not need to be shared. However, initial table generation does add some overhead as demonstrated by the data set. The initial cost of creating the table is also affected by the amount of value mappings it needs to insert into the table. The graph below shows the timings of table generation for 1000, 10.000 and 100.000 values respectively. Figure 2 displays a comparison between HMAC table generation time for each case, against the search function, further consolidating the fact that table generation is only a one-time cost to performance.

Author Response File: Author Response.pdf

Reviewer 2 Report

I am pleased to have the opportunity to review this research manuscript. This study attempted to contribute to the understanding of vulnerabilities and limitations of the MQTT protocol used between IoT devices.

In order to improve this research, I would like to give several suggestions to the authors.

if we talk about IoT Devices, the concern of privacy is an important issue to the industry and users that will use these devices. Consequently, the authors should add in the abstract the study contributions for the industry and they should explain why it is useful to apply the solutions proposed in this research study.

In Conclusions part, the authors must include a final paragraph describing the “practical implications or Social implications" and "originality or value" of this study. What practical/professional and academic consequences will it have for the future? In addition, authors should make references to what is the originality and value of the research work to the industry of IoT devices.

Despite the above comments I recommend the article to be published after revision. The paper is well structured, solid development and a clear methodology was correctly explained.

Author Response

Point 1: If we talk about IoT Devices, the concern of privacy is an important issue to the industry and users that will use these devices. Consequently, the authors should add in the abstract the study contributions for the industry and they should explain why it is useful to apply the solutions proposed in this research study.

The Value-to-HMAC mapping method is not comparable in security features with the SSL/TLS, as it is only able to achieve confidentiality and integrity of information. The method could be used as a base for future improvements and additions.

As mentioned, the method is able to achieve confidentiality and integrity by using only one mechanism. This fact allows the mapping method to be significantly faster than using an encryption algorithm in conjunction with an integrity checking mechanism. The chart below uses the data set created by the script’s timing function and shows a comparison between different HMAC functions (based on different hashing algorithms) and the encryption with AES in CBC mode.

Point 2: In Conclusions part, the authors must include a final paragraph describing the “practical implications or Social implications" and "originality or value" of this study. What practical/professional and academic consequences will it have for the future? In addition, authors should make references to what is the originality and value of the research work to the industry of IoT devices.

Response 2: As previously mentioned, the paper presents a novel approach of achieving confidentiality and integrity of information which relies on one mechanism (creation of an HMAC digest).

This method would mean that information is passed onto the broker as a hash. The Keyed-Hash Message Authentication Code algorithm takes a variable (e.g. temperature value) as an input and a secret key in order to produce the digest. On the receiving device, a mapping table is created, from the range of values and a chosen hashing algorithm and using the shared secret key.

In this regard, only the nodes that posses the secret key can recover the original value, through a simple search within the previously generated mapping table.

This method is ideal of environments where the data produced is limited or within certain interval of values (e.g. temperature sensor). Complete data flow is presented in Figure 2.

This design has a proven experimental advantage of being significantly faster than using an encryption algorithm in conjunction with an integrity checking mechanism; therefore it has a lower performance impact on the IoT device and on the network than when compared to using SSL/TLS suite. The script that produced the data set was run on an Onion Omega 2+ device, in order to simulate a resource-constrained device.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors present a novel method to assure confidentiality and integrity of messages within IOT devices, by means of signatures to send messages. The work presented is very promising and I would like to clarify the following remarks:

At lines 66-69 it is stated that “Mosquitto, is able to provide most of standardized features of the MQTT protocol, such as SSL/TLS and client certificate support. The Mosquitto broker, by default, does not provide security for its messaging scheme and authentication information is sent in plaintext, therefore it requires security mechanisms to protect the transferred information.” If the Mosquitto solution provides SSL/TLS capabilities, why do you need to include new security mechanisms?

Section 2.2 should be extended with more references and, fundamentally, a comparison among them and the proposed solution.

Lines 150-152: "The main objective that the method needs to achieve is the safe distribution of secret keys to all parties. In order to be secure, different keys need to be created and distributed to clients even if they are subscribing to the same topic". How will the secret key be distributed among all the participants? How will the hash tables be distributed among all the participants? How scalable is a system with that amount of keys (i.e., in a network with 100/1000 nodes)?

Author Response

Point 1: At lines 66-69 it is stated that “Mosquitto, is able to provide most of standardized features of the MQTT protocol, such as SSL/TLS and client certificate support. The Mosquitto broker, by default, does not provide security for its messaging scheme and authentication information is sent in plaintext, therefore it requires security mechanisms to protect the transferred information.” If the Mosquitto solution provides SSL/TLS capabilities, why do you need to include new security mechanisms?

Response 1: The paper ‘Vulnerabilities and Limitations of MQTT Protocol Used between IoT Devices’ presents a different approach to achieving confidentiality and integrity of information sent between IoT devices within an industrial environment. In this regard, considerations were made as to what the type of information could be shared between devices and as such, when referring to ‘IoT devices in an industrial environment..’, the paper considers the devices could be sensors, actuators or controllers that share predictable data (e.g. temperature values). The design below was used as a test bed for the script.

The development of a new method of sharing information was based on the fact that IoT devices have a very limited amount of resources and could not efficiently use SSL/TLS protocol, therefore it is aimed at providing a lightweight solution for secure communication. A comparison between chart, between AES-CBC encryption and HMAC based on different hashing algorithms is shown in Figure 2.

As mentioned in the paper, as of version 1.0, the script does not represent a complete and robust solution that is resistant to all forms of attacks and it is not comparable in security with the SSL/TLS suite (e.g. message replay protection mechanisms).

However, the method of mapping values to hashes could serve as a base for future improvements and additions. As the data set generated by the timing script shows, the Value-to-HMAC mapping method assures both confidentiality and integrity of information while being considerably faster than using an encryption algorithm in conjunction with an integrity checking mechanism.

Point 2: Lines 150-152: "The main objective that the method needs to achieve is the safe distribution of secret keys to all parties. In order to be secure, different keys need to be created and distributed to clients even if they are subscribing to the same topic". How will the secret key be distributed among all the participants? How will the hash tables be distributed among all the participants? How scalable is a system with that amount of keys (i.e., in a network with 100/1000 nodes)?

Response 2: In order to address the secure key distribution, a main node (broker) must be able to authenticate each device uniquely and in a way that cannot be forged. As mentioned in the paper, each node must have a digital certificate to prove their identity to the broker, before being allowed to transmit any information.

From this point, the broker should act as an intermediary for the two nodes which will use a secure iteration of the Diffie-Hellman algorithm that is not susceptible to the man-in-the-middle attack.

In this case, the broker is not able to see the information being transmitted or the secret key.

As described in the paper, the hash tables are being generated on each node and do not need to be shared. However, initial table generation does add some overhead as demonstrated by the data set. The initial cost of creating the table is also affected by the amount of value mappings it needs to insert into the table. The graph below shows the timings of table generation for 1000, 10.000 and 100.000 values respectively. The figure below compares the table generation for each case against the search function, further consolidating the fact that table generation is only a one-time cost to performance.

Additionally, because the keys are not stored on the broker, the amount of keys required will not have a significant or prolonged impact on the network’s performance. The bandwidth required for the key distribution process will only be occupied during the initial exchange phase. Another factor that will affect the performance in a large network is how long the keys are valid for.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The replies to the comments are sufficiently reasoned. The paper can be accepted.

Reviewer 2 Report

The authors have made the indicated changes, therefore, the manuscript is ready to be published.

Reviewer 3 Report

All my comments have been addressed.