In this section, we conduct a comprehensive analysis using carefully selected evaluation metrics and methodologies to evaluate the performance of the entire system. The evaluation process encompassed diverse experiments, representing various scenarios and use cases, to ensure a robust assessment:
5.1. Energy Consumption Comparison
Choosing the appropriate consensus algorithm to achieve efficient energy consumption in IoT devices is crucial for several reasons such as the energy constraint, the scalability, the resource constraints, and the latency consideration.
As mentioned in
Section 4.1, both Quorum and Ethereum are based on the Ethereum blockchain, but there are several key differences between them. Ethereum is a more general-purpose blockchain platform with a larger developer community and more established ecosystem and currently uses a Proof of Work (PoW) consensus mechanism. In comparison, Quorum is a specialized platform with enhanced privacy features and a modified version of the practical Byzantine fault tolerance (PBFT) consensus mechanism. The difference in the network architectures between these two blockchains can lead to differences in energy consumption.
However, the energy consumption of a blockchain network depends on many factors, including the number of nodes in the network, the computing power of each node, and especially, the specific implementation of the consensus algorithm. It is important to mention here that, in a blockchain network, a node refers to any device or computer that is connected to the network and participates in the validation and propagation of transactions and blocks.
In this paper, the energy consumption measurement of a node implemented in the IoT system (Raspberry Pi) is addressed for two different types of consensus mechanisms, namely the PoW and QBFT, to decide which one is more appropriate for IoT applications. Hence, this comparison test consisted of three phases:
Measuring the Raspberry Pi power while the DApp is implemented with the Ethereum blockchain based on PoW consensus.
Measuring the Raspberry Pi power while the DApp is implemented with the Quorum blockchain based on QBFT consensus.
Comparing the power consumption between both cases.
The energy consumption of a transaction in a blockchain network can vary based on a number of factors such as the complexity of the transaction, the gas price, the network congestion, and the network size. The scale of deployment used for this comparison test was 3 nodes for the Ethereum network, as well as for Quorum, since QBFT consensus requires a threshold of 2/3 of the total validator nodes to reach consensus. So, the minimum scale was used here, but it is important to mention that having more nodes in the Quorum network increases fault tolerance and makes the network more resistant to attacks or failures. For the Ethereum network, we conducted experiments using a mining difficulty set to 0x2000, representing a moderately challenging level for miners in the network.
Figure 6 presents the nodes’ implementation in each case. The distribution of the nodes is as follows:
Two nodes were implemented on a PC, with the processor features of an Intel(R) Core(TM) i5-7200U CPU @ 2.50 GHz 2.70 GHz with 8 GB of RAM;
The third, which is the node dedicated to testing, was implemented on a Raspberry Pi 3 model B+ with a 64-bit OS and 1 GB of RAM.
To measure energy consumption in IoT devices, which is the Raspberry Pi in this DApp, a USB voltage- and current-detection module was used, as presented in
Figure 7. It is a simple tester displaying the voltage present on the USB connector and showing the current consumption of the attached USB devices. The voltage measured by the USB power detector can be in the range of 3–9 V, while the intensity is measured in the range of 0–5 A. In this experiment, the Raspberry Pi is powered by 5.21 V/2.5 A, and its consumption is estimated to be 500 mA on average.
After implementing the three nodes, each scenario begins with the first transaction, which involves deploying the smart contract. Subsequently, the different functions presented in
Table 2 are executed while measuring the power consumption of the Raspberry Pi for each operation. The procedure was repeated 10 times for each transaction to ensure obtaining results as precise as possible, and the deviation between executed transactions was about 3 min.
The power measured in watts is calculated using Equation (
1).
where
W represents power,
A represents current intensity, and
V represents voltage (V = 5.21 v in this case).
Table 2 includes the acquired readings and calculations of Raspberry Pi power consumption (measured in watts) obtained from the experiments.
Figure 8 illustrates also a comparison of both results.
As shown in
Table 2, the improvement of Quorum reached 8%, which is an important value considering that the number of transactions is over a hundred per second for the Quorum blockchain. This experiment validated our expectations that the Quorum blockchain is more energy efficient in IoT devices than Ethereum with a remarkable difference. Quorum consensus eliminates the energy-intensive mining process associated with PoW algorithms, thereby significantly reducing the energy consumption, and consequently, the costs will be reduced. This contribution demonstrates that the energy consumption of blockchain technology varies and depends on the consensus mechanism used and that using blockchain technology with non-PoW consensus, such as QBFT consensus, may substantially mitigate sustainability issues in IoT systems.
However, the use of the Quorum blockchain, particularly PBFT consensus, showed better energy efficiency and performance. In addition, PBFT creates a new node faster than PoW consensus, which may indicate that it is more viable for the IoT environment. The outcomes of this experiment also reveal that both the type and number of parameters in a smart contract function have an impact on energy consumption. Indeed, as is shown in
Table 2, for example, the energy consumption of the transaction addPrec(), which has nine parameters, was approximately 2.71 w, whereas the addDrugMA() function with only four parameters consumed 2.61 w. A difference of 0.1 w becomes significant when considering the entire blockchain network, particularly in scenarios involving approximately 1000 transactions per second (tps). We can also notice that, the more parameters a function has, the more computational power is required to execute it, and thus, the more energy is consumed, as presented in
Figure 9.
Furthermore, the number of parameters can also affect the size of the data that need to be stored on the blockchain, which can in turn affect the energy consumption. When data are stored on a blockchain, they need to be replicated across all nodes in the network, which, in turn, can increase the energy consumption of the system. Consequently, optimizing the implemented smart contract, particularly in terms of the parameters, can improve the energy consumption in the Quorum network more and more.
5.2. Performance Analysis
After studying the energy consumption and obtaining better results with the QBFT consensus algorithm, one other crucial aspect that warrants further investigation is the scalability issue. Specifically, it is imperative to study the impact of scaling up the system and increasing the number of nodes on the DApp performances. Accurately assessing and evaluating the performance of a blockchain network relies heavily on the utilization of specific metrics [
28,
29]. In this paper, our focus lies in evaluating the Quorum blockchain network operating under the QBFT consensus mechanism focusing on the following metrics:
Transaction throughput: This metric measures the number of transactions the blockchain can process per second (tps). Higher throughput indicates better performance and scalability.
Latency: This refers to the time it takes for a transaction to be propagated across the network and confirmed. Lower latency is desirable for time-sensitive applications.
To achieve this, we utilized the Hyperledger Caliper tool [
30]. Specifically, we made use of a customized version of the Caliper tool, designed to be compatible with the Quorum blockchain since Quorum is not yet supported as a System Under Test [
31].
On the other side, we opted for the most commonly used functions in the system to execute the experiments.
Table 3 shows the details of the selected functions.
The addPresc() and updatePresc() functions are the most invoked by the blockchain network in real-word applications. We conducted a comprehensive evaluation of these selected functions by performing a series of experiments within Quorum networks of varying sizes. The experimentation involved three distinct network configurations, specifically 5 nodes, 10 nodes, and 20 nodes, each utilizing the Quorum Byzantine fault tolerance (QBFT) consensus algorithm. To carry out the experiments, we executed the chosen functions under different send rates (25, 50, 75, 100, and 200), ensuring a wide range of data points to assess system performance comprehensively.
All the experiments were conducted on the same machine with 16 CPUs (two sockets with 8 CPUs per socket running one thread each) and 32 GB of RAM. Finally, a Ubuntu 20.04.4 LTS was installed on this machine.
Throughout the experiments, we recorded the throughput results of the selected functions within the 5-node, 10-node, and 20-node networks. These results are shown in
Figure 10. The throughput results of the functions addPresc() and updatePresc() are depicted in
Figure 10a and
Figure 10b, respectively. The curves of the studied functions exhibit a comparable pattern and network behavior with better results for the function updatePresc(). This can be explained by the fact that the function addPresc() has more string parameters, which might require additional processing and a longer computation time.
On the other side, both functions demonstrate that the throughput is higher in blockchain networks with fewer nodes due to the smaller number of voters compared to larger networks. This can be attributed to the reduced complexity and overhead in blockchain networks with fewer nodes. Since the number of voters, which participate in the QBFT consensus process (2/3 of the total voters), is smaller in these networks, the coordination and agreement among nodes are achieved more swiftly. As a result, the throughput, representing the rate of successful transactions processed, tends to be higher in such scenarios. In contrast, larger networks with a greater number of voters can experience relatively more intricate consensus negotiations, potentially leading to a slightly lower throughput due to the increased coordination efforts required among a larger set of nodes.
Finally, notice that, as the send rate increases, the throughput also experiences a corresponding rise. This trend can be linked to the network’s capacity to manage a higher influx of transactions, allowing for faster processing and validation.
However, beyond a certain threshold specific to each network size, a noteworthy transition occurs. The throughput, after initially rising, reaches a ceiling and becomes relatively constant despite further increases in the send rate. This phenomenon can be explained by the inherent limitations of the network’s capacity, resource allocation, and processing capabilities. As the network becomes saturated with a high volume of incoming transactions, it reaches a point where additional transaction submissions do not significantly enhance the overall throughput. Instead, the network operates at its peak processing capacity, and any subsequent increase in the send rate has a limited effect on throughput improvement.
Investigating the latency variations,
Figure 11 illustrates the results of the studied functions within networks of 5 nodes, 10 nodes, and 20 nodes with various send rates. Actually, these results are correlated with the throughput ones. Indeed, as the size of the blockchain network grows larger, the latency tends to increase. This increase can be attributed to the larger number of nodes and the complexity involved in reaching consensus across a larger distributed network. Furthermore, the latency experiences an upward shift as the send rate increases. This happens because more transactions competing for network resources and processing capacity result in increased latency.
Finally, the latency and throughput results of the getPresc() function are reported in
Figure 12. Experiments on read-only transactions showed that the process of reading data from the blockchain incurred minimal latency, meaning the time delay between initiating the request and receiving the data was negligible (ranging from 0.01 s to 0.04 s). This is due to the inherent design of blockchain systems, where data retrieval operations primarily involve accessing already stored information from the local data of the node receiving the request, without performing any complex computational tasks. As a result, the time taken to retrieve data during read operations is typically quite low. Moreover, notice that the latency outcomes remain consistently comparable across the networks with 5 nodes, 10 nodes, and 20 nodes.
Furthermore, the throughput of the getPresc() function within all the tested blockchain networks exhibits a linear relationship with the number of transactions or requests being processed concurrently (send rate). This means that, as the number of simultaneous read requests increases, the system’s capacity to handle these requests scales proportionally. The linear relationship between throughput and concurrent read requests underscores the decentralized nature of blockchain networks and their ability to efficiently serve multiple pharmacists accessing data in parallel.
To summarize, we provide a comparative analysis of our proposed solution for an optimized blockchain-based secure medical prescription management system with existing solutions. A condensed overview of this analysis is provided in
Table 4.