Challenges and Optimization of Message Queuing Telemetry Transport-Resource Discovery Operation ^†

Abstract

With the rapid development of the Internet of Things (IoT) applications, the ability to automatically discover and retrieve resource information has become increasingly enhanced. Despite being one of the most commonly used IoT communication protocols, Message Queuing Telemetry Transport (MQTT) does not natively support resource discovery. To address this limitation, MQTT-resource discovery (MQTT-RD), a resource discovery mechanism based on MQTT, has been used for resource management. In this study, we tested and evaluated MQTT-RD using the Sniffer system that manages the resource directory and synchronizes data via MQTT. When too many Sniffers are activated, the MQTT-RD system becomes unsustainable. However, the experimental results in this study revealed that frequent updates to the resource directory (RD) and high-frequency heartbeat messages (pingalive) significantly increase network traffic and system load. In this study, we identified performance and stability issues to propose improvement strategies, including refining the topic design, reducing message transmission frequency, and improving the synchronization mechanism. Additionally, the feasibility of incorporating centralized management was explored to enhance system efficiency.

1. Introduction

With the rapid growth of IoT technology [1], efficient resource management in large-scale IoT systems has become critical [2,3]. Message Queuing Telemetry Transport (MQTT) is a widely used lightweight protocol for efficient resource management, but it lacks built-in resource discovery and management functionalities. To address this, we adopted an MQTT-resource directory (MQTT-RD) system to enable resource discovery and access in IoT networks [4].

The MQTT-RD system relies on Sniffer nodes to maintain and update RD, using the MQTT protocol for synchronization. Despite its decentralized architecture ensuring timeliness and completeness, testing and real-world deployments have presented performance issues. Frequent RD updates and high-frequency heartbeat messages (pingalive) significantly increase network traffic and system load. In addition, the redundancy in the point-to-point communication model further amplifies system inefficiencies as the number of Sniffers grows. In this study, we analyzed the MQTT-RD system’s structure to identify bottlenecks and suggest an improvement strategy.

Table 1. Terminology in MQTT-RD.

Term	Description
Sniffer	Maintains and syncs the resource directory; classified as Local or Internet Sniffer.
Broker	Handles communication and data management; includes Local and Cloud (Remote) Broker.
Devices	IoT entities connected to the Broker, such as sensors.
Network	Local network for discovering Sniffers using multicast (e.g., 224.0.1.1:9876).
Sniffer Configuration	configPath/config1.json: Contains Sniffer ID, Broker settings, and network parameters.
Resource Directory	RD/listPath/list_path/yellowpages1.json: Stores and syncs lists of Devices and Sniffers.
/sniffercommunication	Facilitates communication between Sniffers.
/registdevice	Handles device registration and updates the Resource Directory.
/getlist	Publishes or subscribes to the Resource Directory for synchronization.

provides a summary of the terminologies and functionalities in the developed MQTT-RD system in this study, including system components, configuration files, RD, and the definitions and uses of MQTT topics.

2. Literature Review

Eliseu Pereira, Rui Pinto, João Reis, and Gil Gonçalves presented a paper titled “MQTT-RD: A MQTT-based Resource Discovery for Machine-to-Machine Communication” [5]. The MQTT-RD protocol is designed to enhance resource discovery in the Internet of Things (IoT) applications, particularly in the automatic detection and management of devices and services. The main features of the MQTT-RD system include the following.

• Decentralized architecture [6,7]: Resource management is implemented via Sniffers, enabling resource discovery and data exchange across multiple networks. Sniffers maintain information about devices (including other Sniffers and devices) and synchronize resource directories through MQTT Brokers.
• RD: Each sniffer maintains a resource directory that records information about Devices and Sniffers within the network. The data is dynamically updated using the MQTT protocol, ensuring real-time synchronization and system scalability.
• Topic management: Specific MQTT topics are utilized for resource discovery, device registration, and message exchange with the following subcommands.

  -

  /getlist: Retrieves the resource directory, providing a list of available devices in the network.

  -

  /registdevice: Devices register their information with Sniffers, updating the resource directory.

  -

  /sniffercommunication: Facilitates message exchange and data synchronization between Sniffers.

MQTT-RD integrates constrained RESTful environment’s resource directory (CoRE RD) [8,9] and replaces the CoAP link format with a custom JavaScript Object Notation (JSON) file format for resource collection. The resource directory contains detailed information about all devices in the network, providing foundational support for resource sharing and interaction among devices. The zero-configuration mechanism of MQTT-RD combines multicast technology with the MQTT protocol.

• Multicast mechanism: Sniffers and devices use multicast IP to discover Sniffers within the network and retrieve the corresponding Broker’s location information.
• MQTT: By subscribing to specific topics (e.g., /getlist), network resource information is retrieved and synchronized.

The scalability and performance of MQTT-RD were tested, using metrics such as message loss rate, latency, and Broker processing load. The results demonstrated that MQTT-RD was effective in resource discovery in IoT applications. However, performance bottlenecks were identified for Sniffers under high-traffic conditions. The MQTT-RD system is a decentralized IoT system architecture based on the MQTT protocol (Figure 1), comprising components such as a cloud broker, an internet Sniffer, a local sniffer, a local broker, and devices. These components communicate via the MQTT protocol to achieve resource discovery, synchronization, and message exchange.

The Sniffer serves as a core component of the MQTT-RD architecture, responsible for resource detection, management, and data synchronization. It communicates with devices and other Sniffers for resource discovery and IoT network sharing using the MQTT protocol.

• Resource detection and management to maintain information on Sniffers and Devices within the network and ensure the accuracy of the resource directory.
• Resource directory synchronization to synchronize and update the resource directory with other Sniffers, ensuring consistency across multiple Sniffers.

Sniffers are classified based on their communication capabilities with external networks:

• Local Sniffer operates exclusively within the local network, detecting and managing local devices without external network communication capabilities.
• Internet sniffers include the functionalities of a local Sniffer with the added capability of external network communication. They exchange information with other Internet Sniffers via the cloud broker, suitable for applications requiring cross-network resource synchronization.

The broker is a critical component for device communication and resource synchronization, managing MQTT topics and message distribution [10]. The developed system uses the Mosquitto Broker [11] and consists of the following: (1) a local broker that facilitates communication between sniffers and local devices, handling the reception and distribution of local data; and (2) a cloud broker that supports cross-network communication by connecting internet sniffers, enabling resource directory synchronization and sharing. The devices represent the endpoints in IoT systems, typically including sensors, controllers, or other physical devices connected to the network. The devices perform tasks in IoT networks through data publishing or subscribing.

3. MQTT-RD System Operation

The MQTT-RD system comprises modules with different configurations and initialization steps. The hardware and software configurations, as well as experimental settings, are detailed in

Table 2. Experimental setting.

System	Ubuntu 22.04 LTS
Virtual Machine	VMware [12]
Java Environment	OpenJDK Version: 11.0.25 [13]
Tools and Compiler	Apache Ant™ Version: 1.10.12 [14]
Broker (Sniffer)	Moquette MQTT Broker (supports only MQTT 3.1.1 protocol [15], not MQTT 5.0) [16]

.

The virtual machine (VM) configuration is shown in

Table 3. Virtual machine and Sniffer configuration.

VM	IP	Multicast ID	Multicast IP
Ubuntu	192.168.138.10	None	None
SnifferA1	192.168.138.11	n1	224.0.1.11
SnifferA2	192.168.138.12	n1	224.0.1.11
SnifferB1	192.168.138.21	n2	224.0.1.12
SnifferB2	192.168.138.22	n2	224.0.1.12

. Five VMs were set up, including one dedicated broker (cloud broker) and four VMs running Sniffer instances to simulate different network isolation environments.

The Sniffer startup process consists of system initialization, the resource directory updating, subscription operations, and data publishing, as illustrated in Figure 2.

• Initialization stage: The Sniffer uses multicast to discover other Sniffers within the same subnet and retrieves the latest resource directory data via the /getlist topic. For Internet Sniffers, additional information is obtained from the cloud broker.
• Update resource directory stage: The Sniffer integrates the retrieved resource lists and updates the local directory to ensure data synchronization and consistency.
• Subscribe stage: The Sniffer subscribes to topics such as /sniffercommunication and /registdevice, allowing it to receive and process updates from other Sniffers and devices.
• Publish stage: The Sniffer periodically publishes the updated local resource directory via the /getlist topic, making it accessible for other devices and Sniffers to retrieve and synchronize.

Figure 3 illustrates the configuration of the five VMs in the MQTT-RD system, including one cloud broker and four Sniffers, along with the normal message flow relationships between them. The cloud broker serves as the central node, responsible for coordinating and consolidating resource directories from Internet Sniffers. Internet Sniffers synchronize local and remote data, while local Sniffers are responsible for data transmission and device management within their local networks. In Figure 3, green arrows represent the normal connection between Internet Sniffers and the cloud broker to synchronize local resource directories to remote locations via the /getlist topic. Circular green arrows indicate Sniffers publishing their local resource directories to the local broker via /getlist. Purple bidirectional arrows present synchronization between Sniffers via the /sniffercommunication topic.

Sniffers periodically publish their resource directories to the /getlist topic on the local broker. For Internet Sniffers, the resource directories are also synchronized and published to the /getlist topic on the cloud broker, as illustrated in Figure 4. Using the MQTT X tool to monitor the Broker, the contents of the /getlist and /sniffercommunication topics are observed (Figure 5). In the /sniffercommunication topic, the Internet Sniffer continuously receives heartbeat signals (pingalive) from the local Sniffer. Sniffers send heartbeat signals to the /sniffercommunication topic at 1-s intervals, with each signal containing the Sniffer ID.

The resource directory in Figure 5 records the user ID and password of the cloud broker, but these credentials are stored in plaintext within the resource directory. Any user subscribing to the /getlist topic can retrieve these authentication details. This design weakens the security and practical effectiveness of authentication to some extent.

4. Results and Discussion

The reasons for system instability under high-load conditions were explored in the experiment. The analysis began by examining message sizes and the traffic load on the cloud broker. By observing system logs, the message size and traffic for each topic were calculated. The calculation of message sizes involved the MQTT message format and the content. While the calculation process was complex, the capacity of messages was directly obtained by examining the logs of the cloud broker. The relevant data is shown in Figure 6, which displays the message sizes for the /getlist and /sniffercommunication topics, respectively.

In the /getlist topic, messages primarily transmit the resource directory. According to the system log, the message sizes were calculated as follows.

• Internet Sniffer: 0.285 KB
• Internet Sniffer & Local Sniffer: 0.473 KB
• 2 × (Internet Sniffer and local Sniffer): 0.744 KB

To simulate high-load conditions, we assumed that each group consisted of one Internet Sniffer and one local Sniffer, with a total of 20 groups connected to the cloud broker. The message size for the /getlist topic for each Internet and local Sniffer group was 0.473 KB. When the number of groups increased to 2, the message size increased to 0.744 KB, indicating that each additional Sniffer group increased the message size by approximately 0.271 KB. Based on this observation, the total message size for /getlist with 20 Internet and local Sniffer groups was calculated as follows: 0.744 + 0.473 × 19 = 9.731 KB.

Each Sniffer published messages to the /getlist topic, sending one message to the cloud broker every 6 s. The total traffic for 1 min was calculated using the following formula: Traffic (1 min) = 9.731 KB (message size) × 20 (Sniffer) × 10 (times per min).

In the /sniffercommunication, heartbeat messages (pingalive) were primarily transmitted. Based on testing, the size of each pingalive message was 0.0547 KB. Traffic analysis indicated that as the number of Sniffers increased, network load increased significantly. In Internet scenarios, the load was reflected in the receiving-end bandwidth, whereas in local networks, message congestion is more likely to occur.

• Internet: Each Sniffer needed to send one message, but the number of received messages was m, where m is the total number of Sniffers connected to the cloud broker.
• Local network: Each Sniffer encountered network congestion during n − 1 message exchanges, where n represents the total number of local Sniffers.

5. Discussion

After observing the system’s operation, a room for improvement in network communication between Sniffers was revealed. The communication architecture (Figure 7) presents effective synchronization between Sniffers to a certain extent. However, both point-to-point and broadcast communication modes increased network traffic, especially as the number of Sniffer devices grows. Further optimization of the communication methods is required to reduce network burdens. In the MQTT-RD system, performance and resource utilization need to be improved during its operation. First, the /getlist topic exhibited performance issues as each Sniffer continuously published updated resource directory messages every 6 s. This resulted in frequent network traffic fluctuations and high resource consumption. Additionally, multiple Internet Sniffers simultaneously sending similar or redundant messages to the cloud broker led to unnecessary data processing and increased network load. As the content recorded in the resource directory grows, the size of each published file significantly increases network traffic and system load.

As shown in Figure 8, both message size and Traffic (1 min) increased as the number of Sniffer Groups rose. When the number of Sniffer Groups increased from 1 to 20, the message size increased by 13-fold, whereas Traffic (1 min) increased by approximately 263-fold. Moreover, frequent Pingalive operations imposed a significant burden on the system. Excessive operation frequency placed considerable pressure on system performance. As illustrated in Figure 9, the per-minute processing frequency of Pingalive messages increased sharply with the number of Sniffers. When there were 20 Internet Sniffers, a single Internet Sniffer handled up to 1200 communications per minute. This design placed immense stress on network traffic and system resources in multi-device environments, highlighting the need to optimize message publishing frequency and redundancy control mechanisms to enhance overall performance and reduce network load.

To address these challenges, the following optimization directions are proposed:

• Publish/subscribe optimization: Refine topic design (e.g., /getlist/controls) and use retained messages and random delays to reduce traffic and improve performance.
• Centralized management: Upgrade the cloud broker for centralized synchronization and publication to enhance data efficiency.
• PingAlive optimization: Lower message frequency and improve processing to reduce redundancy and traffic.
• Hierarchical design: Classify Sniffers into local and regional layers to optimize data exchange and load distribution.

6. Conclusions and Future Work

We analyzed the operational characteristics of the MQTT-RD system to enhance its performance under high-load conditions. Frequent message publishing and redundant transmissions increase network traffic and resource consumption, with system load growing exponentially as device scale expands. To address these challenges, we proposed refinements to topic design, considered applying random delay techniques, and outlined an optimization of the /getlist data structure. Additionally, we considered introducing a centralized management system as a supplement to the existing decentralized architecture. By integrating MQTT-RD with new features of MQTT 5.0, we aim to enhance its adaptability and efficiency in large-scale IoT deployments. Through continuous improvement and optimization, MQTT-RD can provide a more efficient and stable solution for IoT resource management.